Title: RUAGO: Effective and Practical Retain-Free Unlearning via Adversarial Attack and OOD Generator
Venue: NeurIPS 2025
Accepted: yes
Keywords: Machine Unlearning

Right to be forgetten

User Privacy

Data Forgetting

Training Dataset Privacy
Abstract: With increasing regulations on private data usage in AI systems, machine unlearning has emerged as a critical solution for selectively removing sensitive information from trained models while preserving their overall utility. While many existing unlearning methods rely on the *retain data* to mitigate the performance decline caused by forgetting, such data may not always be available (*retain-free*) in real-world scenarios. 
To address this challenge posed by retain-free unlearning, we introduce **RUAGO**, utilizing adversarial soft labels to mitigate over-unlearning and a generative model pretrained on out-of-distribution (OOD) data to effectively distill the original model’s knowledge. We introduce a progressive sampling strategy to incrementally increase synthetic data complexity, coupled with an inversion-based alignment step that ensures the synthetic data closely matches the original training distribution. Our extensive experiments on multiple benchmark datasets and architectures demonstrate that our approach consistently outperforms existing retain-free methods and achieves comparable or superior performance relative to retain-based approaches, demonstrating its effectiveness and practicality in real-world, data-constrained environments.

RUAGO: Effective and Practical Retain-Free Unlearning via Adversarial Attack and OOD Generator

Abstract

With increasing regulations on private data usage in AI systems, machine unlearning has emerged as a critical solution for selectively removing sensitive information from trained models while preserving their overall utility. While many existing unlearning methods rely on the retain data to mitigate the performance decline caused by forgetting, such data may not always be available ( retain-free ) in realworld scenarios. To address this challenge posed by retain-free unlearning, we introduce RUAGO , utilizing adversarial soft labels to mitigate over-unlearning and a generative model pretrained on out-of-distribution (OOD) data to effectively distill the original model's knowledge. We introduce a progressive sampling strategy to incrementally increase synthetic data complexity, coupled with an inversionbased alignment step that ensures the synthetic data closely matches the original training distribution. Our extensive experiments on multiple benchmark datasets and architectures demonstrate that our approach consistently outperforms existing retain-free methods and achieves comparable or superior performance relative to retain-based approaches, demonstrating its effectiveness and practicality in real-world, data-constrained environments.

1 Introduction

In recent years, the success of machine learning across diverse domains has been driven by the availability of massive datasets. A prominent example is GPT-4, a milestone in advancing machine learning, which was trained on approximately 5 trillion data points [1]. However, these advancements have raised serious privacy concerns due to including sensitive or unauthorized information in training data. Widely deployed models such as ChatGPT have also demonstrated risks of information leakage [2]. In response, regulations [3, 4] such as the EU/US Copyright Law [5] and the General Data Protection Regulation (GDPR) emphasize the Right to be Forgotten , ensuring 'the data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay' [6]. Consequently, model owners, including those of AI systems, must carefully manage personal data and comply with removal requests. However, removing data may alter the original model's behavior and can lead to performance degradation. The most straightforward solution is to retrain the model without the requested data, referred to as the forget data , but this strategy is computationally and financially prohibitive.

To address rising privacy concerns and regulatory demands, Machine Unlearning (MU) [7, 8] has emerged as a framework for removing specific training data. While early work focused on simple

models [9-11], recent efforts have extended MU to deep neural networks [12-16]. In particular, MU can be interpreted as a multi-objective task, where the two main opposing goals are removing the forget data while preserving the utility of the remaining data, which we refer to as the retain data . Since forgetting significantly affects model behavior and performance, achieving both objectives is challenging. A further challenge is to minimize overall training costs. Although recent SOTA methods [13, 17, 18] show strong unlearning performance, they often assume full access to the original or retain data. However, such assumptions are often impractical due to storage limitations, expired permissions, or privacy constraints. To mitigate this issue, recent approaches [12, 14-16, 19, 20] explore the retain-free setting, using only the forget data. Although these retain-free methods have demonstrated strong performance in class-wise unlearning, a scenario involving the removal of entire information from a model, their effectiveness in instance-wise unlearning [12] remains uncertain and unexplored. Instance-wise unlearning, which targets specific samples instead of entire classes, is particularly challenging due to distributional overlap between forget and retain data.

In this work, we present a novel unlearning method for the retain-free scenario, R etain-free U nlearning via A dversarial attack and G enerative model using O OD training, RUAGO as briefly illustrated in Fig. 1. Our method investigates the critical issue of existing instance-wise unlearning methods, particularly over-unlearning , which occurs when excessive removal severely degrades model performance. To mitigate this, we generate adversarial probabilities and use them as soft labels for forget data. These soft labels are positioned near decision boundaries and guide the forgetting process more smoothly than deterministic hard labels, which often induce drastic parameter shifts and information loss in instance-wise scenarios.

However, relying solely on the above strategy may still degrade model generalization performance, especially in the retain-free setting, where access to retain data is restricted. To overcome this challenge, we use a generative model trained on outof-domain (OOD) data, which can avoid training on the retain data and eliminates the privacy risks associated with using a generator trained on the original dataset. In contrast, we believe synthetic

100

93.25

92.98

100

93.99

91.88

80.49

74.63

73.9

99.94

93.38

92.32

0

20

40

60

80

100

Retain Acc

Unlearning Acc

Test Acc

Retrain

Baseline 1

Baseline 2

Ours

Baseline 1

retain-based

Unlearn

model

Unlearn

model

Unlearn

model

Forget data

Retain data

Baseline 2

retain-free

Ours

retain-free w/ G

G trained w/ OOD

血

data generated from OOD distributions can inherently and completely mitigate privacy concerns, while effectively transferring essential knowledge from the original model, thus preserving model utility [21]. To enhance the quality of synthetic supervision, we introduce two additional strategies. First, through a VC theory-based analysis, we demonstrate that progressively increasing the model's complexity during knowledge distillation significantly improves the final model's performance and generalization stability. Therefore, we design a dynamic sample difficulty scheduler, assigning higher initial weight to simpler synthetic samples to support stable early convergence and gradually introducing more complex samples to enhance model robustness. Second, we extract key characteristics from the original model via inversion-based fine-tuning, and progressively align the generator's output distribution with the original data distribution, even without access to retain data. Consequently, after refinement, the OOD-trained generator produces samples similar to the original distribution, enabling effective knowledge transfer and preserving robust performance in retain-free conditions. We validate RUAGO through extensive experiments on four benchmark datasets and three network architectures, showing superior performance over retain-free baselines and competitive results with SOTA retain-based methods.

Our contributions are summarized as follows:

We propose RUAGO, a novel retain-free unlearning framework that leverages adversarial soft labels to prevent over-unlearning and employs synthetic data from an OOD-trained generator to preserve model performance and protect data privacy in retain-free scenarios.

We introduce a dynamic scheduling strategy guided by sample difficulty and grounded in VC theory, gradually increasing model complexity during distillation. We also refine the OODtrained generator via inversion-based alignment to better match the original data distribution and improve knowledge transfer.

Through comprehensive experiments, we show that RUAGO consistently outperforms prior retain-free methods and matches retain-based approaches across diverse datasets and architectures, highlighting its effectiveness and practical utility in realistic scenarios.

2 Related Work

2.1 Machine Unlearning

The primary objective of machine unlearning (MU) is to effectively remove information from the forget data, while maintaining the utility of the retain data [19]. We categorize MU into retain-based and retain-free methods based on whether the retain dataset is required during unlearning.

Retain-based methods utilize the retain data to support the unlearning process. Various methods have been proposed to enhance the efficiency of the retraining process for the retain data [7, 22, 23]. However, due to the substantial computational resources and memory requirements, many studies have focused on updating parameters within pre-trained models. Goel et al. [24] proposed two methods, retraining the last k layers from scratch (EU-k) and fine-tuning the last k layers (CU-k) using the retain data. Bad-T [17] and SCRUB [13] employ a teacher-student framework wherein the teacher induces forgetting through positive and negative knowledge transfer. SalUn [25] identifies and modifies weights with significant influence on the forget data, thereby effectively removing them. Despite demonstrating strong unlearning performance, these approaches require large amounts of the retain data, limiting their applicability in scenarios where access to the retain data is constrained.

Retain-free methods focus on unlearning without the retain data. In such scenarios, effectively forgetting specific data while maintaining overall model performance is difficult and often necessitates additional metadata. Yoon et al. [26] used model inversion to train a conditional GAN (CGAN) along with the forget data to perform unlearning, but it suffers from the complexity of model inversion and underperforms compared to recent methods. Boundary unlearning [19] employs adversarial attacks to guide the forget data toward incorrect labels near decision boundaries. SSD [14] computes the relative importance of parameters between the entire training dataset and the forget data, selectively dampening parameters. While it achieves effective unlearning without training, it requires importance information from the entire training dataset. SCAR [16] uses Mahalanobis distance to delete the influence of the forget data and maintain performance without accessing the retain data by leveraging OOD data. However, its reliance on metadata such as means and covariance matrices limits its applicability in scenarios where this information is unavailable. Moreover, these methods generally exhibit lower unlearning performance than the retain-based methods, and face additional challenges when applied to instance-wise unlearning.

2.2 Curriculum Learning

Curriculum Learning (CL) is a training strategy that mimics human learning by gradually adjusting the difficulty of the training process, progressing from easier to harder samples [27]. This approach facilitates stable learning by allowing models to incrementally build their understanding across various tasks [28-30]. We reinterpret the teacher-student relationship in knowledge distillation through the lens of VC theory, showing that achieving generalization in early stages with limited model capacity leads to more stable subsequent learning. Building on this theoretical insight and inspired by CL, we implement a progressive knowledge transfer strategy by assigning weights to generated images based on their difficulty, thus guiding the model from easier to harder samples. To the best of our knowledge, we are the first to utilize the concept of sample difficulty in the field of MU.

2.3 Data-Free Knowledge Distillation

Data-Free Knowledge Distillation (DFKD) aims to train student model without the use of original training data. The core idea of DFKD is to generate synthetic data that mimics the distribution of the original training data through a model inversion process, considering the teacher model as a

(b)

(a)

𝐺𝜓

OOD Data

ℒ𝑓

መ

ℒ𝑟

ℒ𝑢

𝐺𝜓

ℒ

inv

ℱ𝜃

(c)

Copy

ℱ𝜃

𝑈

SDM

𝒟𝑓

×

Sample

෤

𝑥1

෤

𝑥2

෤

𝑥3

Hard                    Easy

𝑤

Kullback-Leibler divergence

1

2

3

4

5

6

7

8

9

10

𝑝 adv

4

:

1

2

3

4

5

6

7

8

9

10

𝑝 adv

3

:

1

2

3

4

5

6

7

8

9

10

𝑝 adv

2

:

1

2

3

4

5

6

7

8

9

10

𝑝 adv

1

:

𝑥 𝑓

1

𝑥 𝑓

2

𝑥 𝑓

4

𝑥 𝑓

3

Decision

Boundaries

𝒟𝑓

: Forget data

𝐺𝜓

: Generator

: Generated images

𝐹𝜃

: Original model

𝐹 𝜃

𝑈

: Unlearn model

ℒ inv

: Inverse loss

ℒ𝑓

: Forget loss

መ

ℒ𝑟

: Retain loss

ℒ u

: Unlearn loss

APM

discriminator. This process often involves using generative models such as Generative Adversarial Networks (GANs) to generate synthetic data [31-34]. The student model is trained on these synthetic data to match the predictions of the teacher model using distillation process. In this work, we apply the model inversion strategy to the unlearning process, focusing on effectively preventing performance decline from a model without requiring access to the original training data. By leveraging synthetic data generated through a generator, our approach enables precise unlearning, addressing key challenges in real-world scenarios.

3 Preliminaries and Problem Statement

We first introduce notation for a supervised image-classification setting. Let X ⊂ R d be the image space and Y = { 1 , . . . , c } the label set. We denote the full training dataset D = { ( x i , y i ) } N i =1 ⊆ X × Y . Let D t ⊆ X × Y be an independent test set from the same distribution. We assume that F θ : X → Y is a deep learning model with parameter θ trained on D .

Machine unlearning seeks an updated model F U θ that 'forgets' the forget set D f ⊂ D , while preserving performance on the retain set D r = D \ D f and on D t . Formally, F U θ should behave such as F θ ∗ , a model retrained from scratch on D r . In the instance-wise unlearning scenario, where individual examples are removed regardless of their class, rather than removing entire classes as in class-wise unlearning, the resulting class distribution of the forget set D f and the retain set D r remains nearly identical. This heavy overlap between D f and D r makes it difficult to remove information of D f without inadvertently degrading performance on D r . We therefore evaluate F U θ against F θ ∗ using standard accuracy metrics on D r and D t , as well as membership inference attack (MIA) metric.

4 Our Approach

This section introduces RUAGO, our framework for instance-wise unlearning without access to a retain set D r . Instead of relying solely on the forget set D f , which can degrade overall performance and utility, RUAGO focuses on adversarial probabilities to guide the forgetting process and a generator pretrained on out-of-domain (OOD) data to preserve decision-boundary fidelity to the original model. To stabilize distillation from synthetic samples generated by the OOD-pretrained generator, we develop a VC-theoretic analysis of knowledge distillation and implement a dynamic sample-difficulty schedule. Additionally, we refine the generator via a model-inversion step to align its outputs with the original training distribution D . The complete pipeline is illustrated in Fig. 2.

4.1 Adversarial Probability

Adversarial attacks [35, 36] inject imperceptible perturbations into images to maximize model prediction errors, thereby inducing misclassification toward the nearest alternative class in the loss landscape. While traditionally employed to evaluate model robustness or construct defenses against adversaries, these perturbations have recently been applied in machine unlearning [12, 15, 19], where adversarial labels generated by perturbations serve as alternative targets for forget-set samples D f . However, unlike class-wise unlearning methods, instance-wise unlearning does not aim to completely misclassify samples in D f away from their original classes. Since D f constitutes unseen data from the retrained model's perspective, these samples inherently carry higher ambiguity than those in the retain set, D r . Consequently, although the retrained model may exhibit uncertainty toward D f , it can still correctly predict a subset of it. Therefore, strictly assigning completely different class labels to samples in D f risks causing over-unlearning [25]. Specifically, recall that using hard one-hot targets q with cross-entropy loss, defined as L = -∑ q log( p ) , results in gradients ∂L ∂z = p -q . For the targeted class j , this yields a gradient of p j -1 , which can become large in magnitude, triggering drastic parameter updates and potentially destroying useful general features.

To circumvent this issue, we instead propose using the adversarial probability vector p adv as a soft target. These soft labels distribute gradient contributions more evenly, as ∂L ∂z = p -p adv , thereby producing milder parameter updates and preserving the model's inherent uncertainty toward samples in the forget set. This approach effectively treats D f analogously to unseen or ambiguous data, mitigating its influence without excessive parameter drift. Moreover, this design aligns with knowledge distillation principles [37], wherein soft targets convey richer probabilistic information and exhibit lower gradient variance than hard labels. Consequently, our instance-wise unlearning method not only prevents over-unlearning but also better maintains the model's overall utility.

Formally, adversarial examples and probabilities are computed as:

where σ denotes softmax. Finally, the forget loss is defined by:

where L CE is cross-entropy, and N f denotes the number of samples. The overall procedure is depicted in the Adversarial Probability Module (APM) in Fig. 2(a).

4.2 Pre-trained Generator with OOD Data

Due to our assumption that D r is inaccessible, it is essential to practically obtain additional data that can facilitate the extraction of knowledge from F θ . Thus, to replace D r while obtaining high-quality images, we use a generator trained on an OOD dataset as shown in Fig. 2(b). This strategy offers remarkable flexibility: One may employ readily available open-source datasets or pre-train a generator well in advance of any unlearning requirement.

Previous studies [21, 38, 39] have shown that knowledge distillation (KD) can be performed using OOD data to address similar assumptions that are inaccessible to training data. Fang et al. [21] demonstrated effective KD using OOD data with GANs. Motivated by this, RUAGO employs a generator G ψ , trained on an OOD data to replace the D r and preserve model utility.

This strategy provides the advantage of using an open-source pre-trained generator or training the generator with an open-source dataset without D r . Hence, our approach can always respond to rapid unlearning requests. In RUAGO, G ψ does not condition on label space Y , which means it cannot assign labels to the generated images. To address this issue, RUAGO calculates the KL divergence between the logits z o and z u , which are obtained by passing the generated images ˜ x from G ψ through F θ and F U θ , respectively, as shown by the following equation:

where N g is the number of generated images, D KL is the KL divergence function, and τ is a temperature.

4.3 Sample Difficulty-Driven Distillation

To effectively transfer information from F θ to F U θ , we design the loss in Eq. (3). However, since ˜ x is generated from the OOD dataset distribution, Eq. (3) may not fully capture comprehensive information. To enhance the distillation process using samples generated by G ψ , RUAGO employs an easy-to-hard training strategy inspired by curriculum learning (CL) [27, 40-44].

Theorem 1 (Generalization Bound under Sample Difficulty Scheduling) . Let H 1 ⊆ H 2 ⊆ · · · ⊆ H T be a nested sequence of hypothesis spaces with VC-dimensions d t = VCdim( H t ) in each curriculum stage t , and define the true risk by R ( F ) = E ( x,y ) ∼D [ ℓ ( F ( x ) , y )] , and the empirical risk by ˆ R n ( F ) = 1 n ∑ n i =1 ℓ ( F ( x i ) , y i ) . Let the original (teacher) model be F θ ∈ H 1 , and let the unlearned (student) model be F U θ ∈ H T . Consider the optimal retrained model F θ ∗ ∈ H T such that R ( F θ ∗ ) = min F∈H T R ( F ) . Then, drawing n i.i.d. samples,

See Appendix D for the full proof. Theorem 1 provides a theoretical guarantee that our easy-tohard sample schedule preserves model utility while controlling generalization error. This theorem builds on the principle of Structural Risk Minimization, where the model's effective capacity is gradually increased. In the initial phase, training focuses on 'easy' samples, which corresponds to learning within a hypothesis space H t with a small VC-dimension d t . As progressively more challenging samples are introduced, the model's effective capacity grows, but it updates from a stable, near-optimal state achieved in the previous stage. To theoretically ground this approach, we adapt a line of VC-based analysis from the related field of DFKD [45], which allows us to ensure robust generalization even with noisy OOD samples.

In RUAGO, we quantify each sample's difficulty via its loss and adjust it dynamically:

where ℓ k is the loss of the k -th generated sample (see Eq. (3)), and λ > 0 is a temperature hyperparameter controlling the softness of the difficulty weighting. By modulating weight w k from easy to hard, RUAGO achieves faster, more stable convergence. This loss-based difficulty scheduling serves as a practical implementation of the curriculum described in our theoretical analysis. We begin with easy (low-loss) samples to keep the empirical risk small, preventing divergence before gradually incorporating harder samples. The Sample Difficulty Module (SDM) in Fig. 2(a) illustrates this process. To the best of our knowledge, RUAGO is the first method to leverage sample difficulty scheduling during unlearning, directly addressing OOD challenges in knowledge distillation to boost generalization and model utility.

4.4 Inversion-based Generator Alignment

Inspired by model inversion techniques from DFKD [21, 33, 34, 45-48], We insert a brief generator refinement step into our pipeline to ensure that the synthetic samples used in unlearning closely mirror the teacher model's implicit data distribution, thereby promoting a more stable knowledge transfer process. This step optimizes the generator so that its outputs 1) elicit high-confidence predictions from the teacher, 2) expose challenging hard regions of the decision boundary for the student, and 3) match the teacher's internal feature-map statistics. Figure 2(c) represents this procedure.

Concretely, let ˜ x = G ( z ) be a generated sample. We minimize the following inversion loss:

In particular, the cross-entropy term L CE compels the generator to emit samples that the teacher model classifies with high confidence. The negative KL divergence term, by contrast, encourages

the synthesis of challenging hard examples that broaden the output discrepancy between teacher and student. Finally, the statistic-matching term enforces agreement between the batch-wise feature statistics ( µ l , σ l ) and the teacher's stored BatchNorm running statistics ( µ BN l , σ BN l ) . This batch normalization regularizer, primarily used to speed convergence, is only applicable to architectures that include such layers. For models without them, such as Vision Transformers, this term is simply deactivated by setting γ bn = 0 , with our method still demonstrating strong performance. We adopt this inversion formulation (Deep Inversion [33]) as a drop-in module for high-fidelity sample synthesis. Note that after completing the unlearning process, we safely discard the refined generator to eliminate any potential privacy risks.

4.5 Retain-Free Unlearning

We now introduce final unlearning loss of RUAGO, which combines Eq. (2) for forgetting objective and Eq. (4) for retaining objective, formulated as follows:

where γ 1 and γ 2 are hyper-parameters that control the trade-off between the forgetting and retaining components. Algorithm 1 provides a detailed overview of our method's operational procedure. In each epoch, our approach first performs model inversion. Once the model inversion is completed, it generates the adversarial probability p adv and the synthesized image ˜ x . These generated components are then used to compute the unlearning loss L u . The calculated loss is subsequently employed to update the model parameters. This entire process is repeated over multiple epochs. RUAGO enables rapid and effective unlearning without requiring the D r . Our method suits scenarios where privacy concerns, resource limitations, or expired access rights restrict data storage or access.

5 Experimental Results

5.1 Datasets, Models and Unlearning Setups

We conduct our experiments using the CIFAR-10, CIFAR-100 [49], TinyImageNet [50] and VGGFace2 [51] datasets. For each dataset, we employ deep learning architectures including VGG16 [52], ResNet18 [53] and Vision Transformer (ViT) [54]. In addition, we employed the COCO dataset [55] as an out-of-distribution resource for training our generative model. We randomly designate 10% of the entire training dataset as D f , focusing on evaluating instance-wise forgetting. Further experimental details and additional results, including those for CIFAR-100, TinyImageNet, and the deletion of 50% of D , are provided in the Appendix due to space constraints.

5.2 Baselines and Evaluation Metrics

In our experiments, we evaluated our method against the Original and Retrain models, as well as six unlearning baselines, including three retain-based methods, Bad-T [17], SCRUB [13], and SalUn [25], and three retain-free methods, Boundary Shrink (BS) [19], SSD [14], and SCAR [16].

To evaluate the performance, we utilize 6 different metrics: 1) RA: accuracy on D r ; 2) UA: accuracy on D f ; 3) TA: accuracy on D t ; 4) A VG: measures the mean of the absolute differences between each method's RA, UA, and TA values and those of the Retrain model; 5) Membership Inference Attack (MIA) [56]: a canonical privacy metric for evaluation unlearning models [13, 14, 16, 17, 19, 25, 57]. The maximized or minimized MIA score could lead to the Streisand effect, unintentionally providing information to attackers. Thus, an MIA value near to the Retrain model is ideal; 6) Running-Time Efficiency (RTE): measure the time efficiency of each method in seconds.

Algorithm 1 RUAGO

(a) Results on CIFAR-10.

(b) Results on VGGFace2.

5.3 Results

Accuracy Performance. Table 1 presents unlearning accuracy on two datasets for the original model, the retrained model, six baselines, and RUAGO. Table 1(a) presents the accuracy results for CIFAR-10 dataset, indicating that retain-free baselines are largely ineffective at unlearning, regardless of the model type. In the worst case, an AVG value of 42.62 is observed.

The similar RA and UA values across the three methods indicate that a general performance degradation has occurred rather than erasing D f . In contrast, RUAGO demonstrates remarkable performance. The AVG values indicate that RUAGO outperforms the other three methods, with RA, UA, and TA closely aligning with those of the Retrain model. Although, Bad-T, SCRUB, and SalUn are slightly more effective than RUAGO for the ResNet18 model, our method still demonstrates comparable performance. Notably, RUAGO outperforms these methods for VGG16 and ViT, with AVG values of 0.28 and 0.05, respectively, indicating the closest match to the Retrain model.

Table 1(b) presents the results for the VGGFace2 dataset. Retain-free methods, such as Boundary Shrink, SSD, and SCAR, exhibit significant performance degradation, similar to the CIFAR-10 results. In contrast, our approach outperforms all other methods, including retain-based methods. This result indicates that even when G ψ is trained on an OOD dataset (COCO) that is significantly different from D (VGGFace2), it still supports the unlearning process effectively. In conclusion, although unlearning without D r is challenging, RUAGO succeeds admirably, comparable to or outperforming other baselines.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

VGG16

ResNet18

ViT

(b) VGGFace2

Bad-T

SCRUB

SALUN

BS

SSD

SCAR

RUAGO

0

0.2

0.4

0.6

0.8

1

1.2

VGG16

ResNet18

ViT

(a) CIFAR-10

Retrain

MIA Score. To evaluate privacy leakage, we analyze the MIA scores of all methods. The Bad-T method, which shows excellent accuracy performance, records the worst MIA score. This suggests that the Bad-T approach does not guarantee privacy protection. In contrast, the MIA values of SCAR show a minor discrepancy from the Retrain model, but it shows poor accuracy, as mentioned above. As a result, the evaluation of effective unlearning requires a comprehensive set metrics. Our method, which performs accurately, also demonstrates strong privacy protection. As shown in Fig. 3, our approach achieves MIA values comparable to those of the Retrain model in all experiments except for ResNet18 trained on CIFAR-10. Particularly for ViT on both datasets, the MIA scores are almost identical to the Retrain model, with only about a 1% difference. This indicates that RUAGO exhibits excellent unlearning performance in accuracy and protecting privacy.

Running-Time Efficiency. Efficient unlearning requires latency that is lower than that of a full retraining procedure. Table 2 reports each unlearning method's running-time efficiency (RTE), measured in seconds. Notably, this metric omits all preparatory computations, such as the Fisher information matrix estimation in SSD, the mean and covariance computations in SCAR, and the outof-distribution generator training in our approach, and concentrates solely on post-request execution time. Although these offline steps can be time-consuming, they may be precomputed without affecting the immediate unlearning latency. Methods that depend on the retain dataset D r (Bad-T, SCRUB, and SalUn) incur substantial RTE costs, especially when applied to large architectures like ViT or high-resolution datasets such as VGGFace2. In contrast, techniques that do not use D r (BS, SSD, and SCAR) achieve lower RTE values but do not deliver effective unlearning performance, as noted above. Our method, however, strikes an ideal balance: it matches or exceeds the efficiency of retain-free baselines (providing up to a 12× speed-up over full retraining) without sacrificing unlearning efficacy. These results underscore the practical advantage of our framework for real-time unlearning, even in the most challenging scenarios where the retain data is unavailable.

5.4 Ablation Studies

In this section, we present ablation studies demonstrating the importance of various design choices in RUAGO. We summarize the main findings here, with additional results and analyses provided in Appendix G.

Component Analysis. We first analyze the impact of each component using the VGG16 model on CIFAR-10. As shown in Table 3, each component of RUAGO plays a crucial role. First, unlearning with hard labels instead of our soft targets significantly reduces test accuracy (TA) and yields membership inference attack (MIA) scores that diverge from the retrained model, indicating lower utility and potential privacy risks. Second, employing generators trained on different OOD datasets, such as TinyImageNet, consistently ensures robust unlearning, highlighting RUAGO's versatility. Third, using a randomly initialized generator leads to markedly poor performance, underscoring the necessity of a pre-trained OOD generator. Fourth, while RUAGO is effective without the model inversion technique, its inclusion further improves unlearning outcomes. Lastly, omitting the sample difficulty scheduling from the distillation process critically harms model utility. These findings collectively validate our design choices in achieving efficient and robust unlearning.

Hyperparameter Sensitivity Analysis. To address concerns about the hyperparameter space, we conducted a sensitivity analysis on the loss weights ( γ adv , γ bn , γ cls , γ 1 , γ 2 ) for the VGG16 model on CIFAR-10, varying one while keeping others fixed. The detailed results are presented in the Appendix (Tables 9 and 10). Our findings show that the performance of RUAGO is not overly sensitive to its hyperparameters. Key metrics remained stable across a wide range of values, demonstrating the robustness of our method. This leads to a practical and efficient tuning guideline. In our experiments,

a simple one-dimensional sweep for γ 1 after fixing other weights was sufficient to find a near-optimal configuration. This demonstrates that RUAGO can achieve strong performance without extensive, multi-dimensional hyperparameter tuning.

Analysis of Alignment and Re-injection Risk. A critical consideration is whether the generator alignment step could inadvertently re-inject forgotten information. We mitigate this risk through safeguards like the forget loss L f . In the class-wise unlearning scenario, we employ an additional safeguard of output filtering. To empirically validate the safety of this component, we analyzed the effect of alignment strength. As detailed in the Appendix (Table 11), using a high learning rate to induce overly aggressive alignment leads to a general performance collapse rather than selective re-injection, indicating optimization instability.

Furthermore, we investigated the impact of the number of alignment epochs ( E g ) across multiple architectures, as shown in Table 4. The results are highly consistent, as increasing the alignment epochs from 10 to 100 maintains robust unlearning performance (stable UA) while slightly improving utility (RA and TA) across VGG16, ResNet18, and ViT. This compellingly demonstrates that the alignment module operates within a safe regime, stabilizing distillation without re-introducing forgotten data, regardless of the model architecture.

6 Conclusion

We propose RUAGO, a unified retain-free unlearning framework that prevents over-unlearning issues and preserves the model's utility without access to retain data. We aim to tackle the performance drop common in retain-free scenarios, especially in instance-wise unlearning. Our method generates adversarial probabilities to prevent forget samples from being pushed beyond decision boundaries, thereby mitigating over-unlearning. Simultaneously, to compensate for the absence of retain data, we leverage a generator pretrained on OOD data to distill knowledge from the original model using synthetic samples. To address potential instability caused by synthetic samples and to enhance generalization during distillation, we incorporate a sample scheduling strategy informed by VC theory, and apply inversion-based alignment to adjust the generator's outputs toward the original data distribution. These components form a cohesive framework that achieves effective and practical unlearning in retain-free scenarios. Our extensive experimental results show that RUAGO outperforms existing retain-free methods, and matches or exceeds retain-based approaches, confirming its practicality for real-world unlearning scenarios without the retain data.

References

Epoch AI. Data on notable ai models. https://epochai.org/data/notable-ai-models , 2024. URL https://epochai.org/data/notable-ai-models . Accessed: 2024-11-04.

Matt Burgess. Chatgpt has a big privacy problem. https://www.wired.com/story/ italy-ban-chatgpt-privacy-gdpr/ , 2023. Accessed: 2023-04-04.

Eric Goldman. An introduction to the california consumer privacy act (ccpa). Santa Clara Univ. Legal Studies Research Paper , 2020.

Kate Kaye. The ftc's 'profoundly vague' plan to force companies to destroy algorithms could get very messy. https://www.protocol.com/enterprise/ ftc-algorithm-data-model-ai/ , 2022. Accessed:2022-05-17.

Vadym Kublik. Eu/us copyright law and implications on ml training data. https://valohai. com/blog/copyright-laws-and-machine-learning/ , 2019. Accessed:2019-03-01.

Alessandro Mantelero. The eu proposal for a general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review , 29(3):229-235, 2013.

Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141-159. IEEE, 2021.

Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299 , 2022.

Ananth Mahadevan and Michael Mathioudakis. Certifiable machine unlearning for linear models. arXiv preprint arXiv:2106.15093 , 2021.

Jonathan Brophy and Daniel Lowd. Machine unlearning for random forests. In International Conference on Machine Learning , pages 1092-1104. PMLR, 2021.

Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems , 32, 2019.

Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learning to unlearn: Instance-wise unlearning for pre-trained classifiers. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 11186-11194, 2024.

Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. Advances in neural information processing systems , 36, 2024.

Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 12043-12051, 2024.

Hyunjune Kim, Sangyong Lee, and Simon S Woo. Layer attack unlearning: Fast and accurate machine unlearning via layer level attack and knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 21241-21248, 2024.

Jacopo Bonato, Marco Cotogni, and Luigi Sabetta. Is retain set all you need in machine unlearning? restoring performance of unlearned models with out-of-distribution images. In European Conference on Computer Vision , pages 1-19. Springer, 2025.

Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 7210-7217, 2023.

Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Systems , 2023.

Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7766-7775, 2023.

Jack Foster, Kyle Fogarty, Stefan Schoepf, Cengiz Öztireli, and Alexandra Brintrup. Zero-shot machine unlearning at scale via lipschitz regularization. arXiv preprint arXiv:2402.01401 , 2024.

Gongfan Fang, Yifan Bao, Jie Song, Xinchao Wang, Donglin Xie, Chengchao Shen, and Mingli Song. Mosaicking to distill: Knowledge distillation from out-of-domain data. Advances in Neural Information Processing Systems , 34:11920-11932, 2021.

Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 11516-11524, 2021.

Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. In IJCAI , volume 6, page 19, 2022.

Shashwat Goel, Ameya Prabhu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640 , 2022.

Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. arXiv preprint arXiv:2310.12508 , 2023.

Youngsik Yoon, Jinhwan Nam, Hyojeong Yun, Jaeho Lee, Dongwoo Kim, and Jungseul Ok. Few-shot unlearning by model inversion. arXiv preprint arXiv:2205.15567 , 2022.

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning , pages 41-48, 2009.

Sheng Guo, Weilin Huang, Haozhi Zhang, Chenfan Zhuang, Dengke Dong, Matthew R Scott, and Dinglong Huang. Curriculumnet: Weakly supervised learning from large-scale web images. In Proceedings of the European conference on computer vision (ECCV) , pages 135-150, 2018.

Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, and Tom M Mitchell. Competence-based curriculum learning for neural machine translation. arXiv preprint arXiv:1903.09848 , 2019.

Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research , 21(181):1-50, 2020.

Hanting Chen, Yunhe Wang, Chang Xu, Zhaohui Yang, Chuanjian Liu, Boxin Shi, Chunjing Xu, Chao Xu, and Qi Tian. Data-free learning of student networks. In Proceedings of the IEEE/CVF international conference on computer vision , pages 3514-3522, 2019.

Jaemin Yoo, Minyong Cho, Taebum Kim, and U Kang. Knowledge extraction with no observable data. Advances in Neural Information Processing Systems , 32, 2019.

Hongxu Yin, Pavlo Molchanov, Jose M Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K Jha, and Jan Kautz. Dreaming to distill: Data-free knowledge transfer via deepinversion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8715-8724, 2020.

Gongfan Fang, Kanya Mo, Xinchao Wang, Jie Song, Shitao Bei, Haofei Zhang, and Mingli Song. Up to 100x faster data-free knowledge distillation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6597-6604, 2022.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014.

Aleksander Madry. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 , 2015.

Mandar Kulkarni, Kalpesh Patil, and Shirish Karande. Knowledge distillation using unlabeled mismatched images. arXiv preprint arXiv:1703.07131 , 2017.

Gaurav Menghani and Sujith Ravi. Learning from a teacher using unlabeled data. arXiv preprint arXiv:1911.05275 , 2019.

MKumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. Advances in neural information processing systems , 23, 2010.

Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander Hauptmann. Self-paced curriculum learning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 29, 2015.

Chang Xu, Dacheng Tao, and Chao Xu. Multi-view self-paced learning for clustering. In Twenty-Fourth International Joint Conference on Artificial Intelligence , 2015.

Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence , 44(9):4555-4576, 2021.

Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. Curriculum learning: A survey. International Journal of Computer Vision , 130(6):1526-1565, 2022.

Jingru Li, Sheng Zhou, Liangcheng Li, Haishuai Wang, Jiajun Bu, and Zhi Yu. Dynamic data-free knowledge distillation by easy-to-hard learning strategy. Information Sciences , 642: 119202, 2023.

Gongfan Fang, Jie Song, Chengchao Shen, Xinchao Wang, Da Chen, and Mingli Song. Data-free adversarial distillation. arXiv preprint arXiv:1912.11006 , 2019.

Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty. Zero-shot knowledge distillation in deep networks. In International Conference on Machine Learning , pages 4743-4751. PMLR, 2019.

He Liu, Yikai Wang, Huaping Liu, Fuchun Sun, and Anbang Yao. Small scale data-free knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6008-6016, 2024.

Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.

Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N , 7(7):3, 2015.

Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018) , pages 67-74. IEEE, 2018.

Karen Simonyan. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. CoRR , abs/1405.0312, 2014. URL http://dblp.uni-trier. de/db/journals/corr/corr1405.html#LinMBHPRDZ14 .

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) , pages 3-18. IEEE, 2017.

Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security , 18:2345-2354, 2023.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248-255. Ieee, 2009.

Jonathan Krause, Jia Deng, Michael Stark, and Li Fei-Fei. Collecting a large-scale dataset of fine-grained cars. 2013.

Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Gaël Varoquaux. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning , pages 108-122, 2013.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Mehryar Mohri, A Rostamizadeh, and A Talwalkar. Foundations of machine learning, ser. adaptive computation and machine learning, 2012.

Shai Shalev-Shwartz and Shai Ben-David. Understanding machine learning: From theory to algorithms . Cambridge university press, 2014.

Vladimir Vapnik. The nature of statistical learning theory . Springer science & business media, 2013.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems , 33:12104-12114, 2020.

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8110-8119, 2020.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS Paper Checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main contributions of this work are accurately reflected in both the abstract and the introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Appendix I includes a dedicated 'Limitations' section, highlighting our reliance on generators pretrained on richly-featured datasets and noting that applicability beyond supervised image classification remains unverified.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions are explicitly stated in the statement of Theorem 1 in the main text (Section 4), and the full proof of Theorem 1 is provided in Appendix C.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The experimental procedure and pseudo-code are described in the main text (see Algorithm 1), Section 5 details all datasets, model architectures, and hyperparameters, and we have made the full code available for reproducers.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The full code is available at the repository URL listed in the appendix, and all experiments use standard open-source datasets; detailed instructions for data access and environment setup are provided in that repository.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Section 5 and Appendix C details all datasets, model architectures, hyperparameters, and optimizers.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Error bars in Figs. 3 and 4 denote one standard deviation computed over five runs with different random seeds (capturing variability from initialization and data splits), and all main table results are reported as mean ± standard deviation.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Section 5 specifies that all experiments were conducted on a single NVIDIA RTX A5000 GPU (24 GB VRAM) and reports running time efficiency in Table 2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research fully conforms to the NeurIPS Code of Ethics: we use only publicly available datasets, involve no human subjects or sensitive personal data, maintain transparency in methods and reporting, and preserve anonymity where applicable.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [No]

Justification: Our work's primary objective is to enhance privacy through machine unlearning, and we assess that the risk of malicious misuse is negligible. We anticipate solely positive societal outcomes and improved protection of user data and, therefore, have not included a discussion of negative impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work's primary objective is to enhance privacy through machine unlearning, and we assess that the risk of malicious misuse is negligible.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly cite the original publications for all datasets, models and code used in this work, and note that they are publicly available under their respective open licenses.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We have ensured comprehensive documentation of our assets in Section 5 and Appendix C. We make it available alongside the assets via an anonymized URL.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: We used LLMs solely for manuscript writing, editing, and formatting, and they did not contribute to the core methodological development.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

This Appendix provides additional insights and detailed explanations supporting our main findings. We begin with a brief description of the datasets and evaluation metrics. We then describe the hyperparameter configurations in our method. We subsequently present the full proof of Theorem 1 and additional experimental results, including CIFAR-100 and TinyImageNet. Furthermore, we explore the effects of deleting 50% of the training dataset and provide ablation studies for datasets and models not covered in Section 5.4. To demonstrate the versatility of our method, we also conduct supplementary experiments based on a class-wise unlearning scenario. Finally, we outline our limitations and propose avenues for future work. The source code is available here: https: //github.com/Lemma1727/RUAGO

A Datasets

A.1 CIFAR-10 and CIFAR-100

The CIFAR-10 and CIFAR-100 datasets [49] are widely used benchmarks for image classification. Each dataset contains 60,000 color images at a resolution of 32 × 32 pixels. CIFAR-10 has 10 classes, such as airplanes, automobiles, and animals, while CIFAR-100 expands this to 100 fine-grained classes. Both datasets are split into a training set of 50,000 images and a test set of 10,000 images. These datasets are valuable in the computer vision area for evaluating the performance of classification models due to their balanced class distribution and moderate complexity.

A.2 TinyImageNet

TinyImageNet dataset [50] is an image classification dataset commonly used in deep learning research. It is a smaller version of the original ImageNet dataset [58]. The dataset comprises 200 classes, with each class containing 500 training images and 50 validation images. Each image has a resolution of 64 × 64 pixels, making it smaller than the original ImageNet dataset. The dataset includes 100,000 training images, 10,000 validation images, and 10,000 test images. We utilize the train set and validation set for experiments.

A.3 VGGFace2

The VGGFace2 dataset [52] is a large-scale collection intended for face recognition applications. This dataset comprises facial data and is related to privacy preservation tasks. The dataset's high similarity among classes makes it essential for evaluating the effectiveness of unlearning methods in practical applications involving facial data. The dataset comprises various facial images that differ in identity, pose, illumination, background, and expression. The dataset comprises over 3.31 million images sourced from more than 9,000 individuals. To conduct our unlearning task, we randomly selected 100 individuals from a training dataset and resized them into 224 × 224 resolution.

A.4 Stanford Cars

The Stanford Cars [59] dataset is a fine-grained image classification benchmark for car recognition. It consists of 196 classes, categorized by the manufacturer, model, and year of the vehicle. The dataset includes 8,144 training images and 8,041 test images. Many cars from the same manufacturer or similar models and years share very similar designs. Thanks to these characteristics, the Stanford Cars dataset is useful for evaluating how effectively unlearning methods can remove fine-grained visual details.

B Metrics

B.1 Accuracy

In order to assess a classifier's performance, accuracy is frequently utilized. It measures the percentage of samples for which the true classes can be predicted with maximum certainty. Accuracy of a model F tested on a dataset of N samples { ( x 1 , y 1 ) , ..., ( x N , y N ) } is formulated as follows:

where δ ( · , · ) is the Kronecker delta function.

B.2 Membership Inference Attack Score

Membership inference attack (MIA) [56] is a metric used to determine whether a specific data point was used during the model training. We employ MIA to verify the effectiveness of unlearning. To compute the MIA score, we pass the retain set ( D r ) and the test set ( D t ) through the target model to obtain probabilities for each data point:

After that, We assign label 1 to the probabilities from D r and label 0 to those from D t , and train a shallow model using these labeled data points:

In our case, we used the LogisticRegression model from Scikit-learn [60]. Following this, we pass the forget set ( D f ) through the target model to compute probabilities for each data point and use the shallow model to predict labels:

The average of these predicted values is used as the MIA score:

B.3 Running Time Efficiency

To measure the Running Time Efficiency (RTE), we record the time taken for the entire unlearning process, from the execution of the unlearning algorithm to its completion, expressed in seconds. This measurement excludes any preparations, such as data loading, preprocessing, or any metadata computation that may need to be conducted in advance. These steps are not included in the RTE metric to ensure a fair comparison that focuses solely on the efficiency of the unlearning methodology itself.

C Implementation and Hyperparameter Settings

We set the hyperparameters for RUAGO as follows. The weights γ cls , γ adv , and γ bn were selected within the range [0 , 5] . The coefficient γ 1 was varied within [0 . 01 , 1 . 50] , while γ 2 was fixed at 0.01. The learning rate was adjusted within [5 × 10 -6 , 5 × 10 -4 ] , and training was performed for 1 to 50 epochs. We used the SGD optimizer for training the original model and the Adam optimizer [61] for unlearning baselines. For adversarial attacks, we applied the PGD attack [36]. All experiments were conducted using five different random seeds on a single NVIDIA RTX A5000 GPU.

D Proof

Theorem 1 (Generalization Bound under Sample Difficulty Scheduling) . Let H 1 ⊆ H 2 ⊆ · · · ⊆ H T be a nested sequence of hypothesis spaces with VC-dimensions d t = VCdim( H t ) in each curriculum stage t , and define the true risk by R ( F ) = E ( x,y ) ∼D [ ℓ ( F ( x ) , y )] , and the empirical risk by ˆ R n ( F ) = 1 n ∑ n i =1 ℓ ( F ( x i ) , y i ) . Let the original (teacher) model be F θ ∈ H 1 , and let the unlearned (student) model be F U θ ∈ H T . Consider the optimal retrained model F θ ∗ ∈ H T such that R ( F θ ∗ ) = min F∈H T R ( F ) . Then, drawing n i.i.d. samples,

Proof. Fix δ > 0 . For each t = 1 , . . . , T , define

where C > 0 is a universal constant from the VC uniform-convergence bound, d t ≥ 1 is the (finite) VC-dimension of the stage-t hypothesis space, n ≥ 1 is the sample size, and δ ∈ (0 , 1) , respectively. Hence ϵ t > 0 . By the standard VC uniform-convergence theorem [62-64], for any single hypothesis space H t ,

To ensure this bound holds simultaneously for all T stages, we apply a union bound. Thus, with probability at least 1 -δ , we have

Let and define

Therefore

which is a fixed constant since F 1 and F ∗ are fixed; hence, their error gap depends on the generator. Then

Next, telescope the first term:

For each t = 2 , . . . , T , using H t -1 ⊆ H t and the fact that F t minimizes the empirical risk over H t , we have ˆ R n ( F t ) ≤ ˆ R n ( F t -1 ) . Thus,

The first line is an identity. The second line follows from the property that x ≤ | x | . The third line follows because: (1) by the uniform convergence bound, | R ( F t ) -ˆ R n ( F t ) | ≤ ϵ t and | ˆ R n ( F t -1 ) -R ( F t -1 ) | ≤ ϵ t -1 ; and (2) since F t minimizes the empirical risk over H t and F t -1 ∈ H t (due to the nested spaces), we have ˆ R n ( F t ) ≤ ˆ R n ( F t -1 ) , which implies the middle term is non-positive. Summing over t = 2 , . . . , T gives

Noting that each ε t = O (√ d t log n/n ) , we conclude

(a) Results on CIFAR-100.

(b) Results on TinyImageNet.

E Additional Experiments

We additionally conducted experiments using the CIFAR-100 and TinyImageNet datasets. The results for CIFAR-100 are summarized in Table 5(a). Consistent with the findings from previous experimental results in Section 5.3, RUAGO demonstrates the most robust performance among retain-free methods. Retain-free baselines, such as those evaluated on VGG16 and ResNet18 models, demonstrate AVG values exceeding 10.00. In contrast, our proposed method achieves an AVG value of approximately 3.00, indicating a significant improvement over the other approaches. Additionally, RUAGO demonstrates unlearning performance comparable to retain-based methods. Notably, for the ResNet18 model, RUAGO achieves an AVG value of 3.09, representing the highest unlearning performance across all baselines. In terms of the MIA score, our method effectively mitigates privacy leakage. As shown in Fig. 4(a), Bad-T (Bad-T), which demonstrated the highest accuracy performance, exhibits a substantial discrepancy in the MIA score compared to the Retrain model. This discrepancy suggests a potential Streisand effect, which could lead to significant privacy leakage. On the other hand, while the SCAR method achieves an MIA score close to that of the Retrain model, it falls short in accuracy performance, as indicated in Table 5(a). Unlike these methods, RUAGO not only achieves superior unlearning performance in accuracy but also shows an MIA score similar to that of the Retrain model, indicating better privacy protection compared to other methods.

The experimental results on TinyImageNet, as shown in Table 5(b), further confirm that our method significantly outperforms retain-free baselines. These findings indicate that retain-free approaches struggle with datasets containing many classes, such as TinyImageNet, which includes 200 classes. In contrast, RUAGO demonstrates remarkable unlearning performance under these challenging conditions. Furthermore, when compared to retain-based methods, our approach consistently achieves superior results. Specifically, for the ResNet18 model, competing methods fail to deliver satisfactory unlearning outcomes, deviating from the desired unlearning objectives. Conversely, RUAGO achieves an AVG value of 3.1, the best among all baselines. Regarding the MIA score, our approach consistently achieves an MIA score relatively close to that of the Retrain model as illustrated in Fig. 4(b), similar to results observed on other datasets. This alleviates concerns about the Streisand effect. Furthermore, our method demonstrates superior accuracy performance, indicating its capacity to achieve a remarkable equilibrium between model efficacy and privacy safeguarding. These results emphasize the effectiveness of RUAGO in the instance-wise unlearning scenario, even in settings without access to D r .

(b) Results on VGGFace2

F Results with 50% deletion

We designate 50% of the entire training dataset as forget set, thereby removing half of the training data. We conduct these experiments on the CIFAR-10 and VGGFace2 datasets. The results are summarized in Table 6. RUAGO demonstrates consistently robust unlearning performance even in large-scale unlearning. On the CIFAR-10 dataset, results are similar to those presented in Section 5.3, where retain-free methods fail to achieve effective unlearning, leading instead to a significant decline in overall model performance. In particular, the Boundary Shrink method demonstrates a substandard outcome with an AVG value of 18.52. Retain-based methods also exhibit suboptimal unlearning performance; for instance, the SCRUB method records an AVG value as high as 3.94 in certain cases. In contrast, RUAGO achieves an AVG value of 0.75 and 0.08 on ResNet18 and ViT models, respectively, outperforming both retain-free and retain-based methods. Similarly, on the VGGFace2 dataset, various baseline methods fail to achieve complete unlearning. Retain-free methods consistently fail to achieve effective unlearning, and even retain-based methods, such as the Bad-T method, exhibit excessive misclassification of D f when applied to the ResNet18 model. Conversely, RUAGO not only outperforms all retain-free methods but also achieves lower A VG values than retain-based methods on ResNet18 and VGG16 models. These results highlight RUAGO as a practical and scalable solution for unlearning, effectively removes forget data while consistently maintaining model utility.

G Ablation Studies

Table 8 presents the ablation study results on the CIFAR-10, CIFAR-100, TinyImageNet, and VGGFace2 datasets. To assess the utility of p adv , we perform experiments using hard labels, which lead to UA values significantly lower than TA, indicating over-unlearning. This indicates that most samples in D f are misclassified as an outcome counter to the objectives of instance-wise unlearning. If D f , which should behave as unseen data due to model generalization, is entirely misclassified, it risks triggering the Streisand effect.

Furthermore, unlearning experiments with a generator trained on the TinyImageNet dataset demonstrate effective unlearning performance, highlighting the robustness of RUAGO when employing a generator trained on different OOD datasets. Specifically, we exclude experiments with classification models trained on the TinyImageNet dataset, as the training datasets for both the classification model and generator are identical. The unlearning performance is significantly diminished when a generator is trained from scratch through model inversion rather than pre-trained on an OOD dataset. Interestingly, omitting model inversion occasionally results in slightly better A VG values; however, the overall benefit of incorporating model inversion is evident across different datasets. Lastly, the

importance of including sample difficulty is apparent, as omitting this strategy, particularly for the TinyImageNet dataset, leads to complete unlearning failure.

To further demonstrate the effectiveness of our approach, we conduct experiments on a fine-grained dataset where class-level similarities make discrimination particularly challenging. The experimental results on the Stanford Cars dataset [59] are presented in Table 7. The Stanford Cars dataset poses challenges for unlearning due to many similar classes and features. Nevertheless, our method shows consistently strong performance, achieving an AVG of 0.65 on the ViT model, outperforming all baselines.

H Class-wise Unlearning Scenario Experiments

As an extension of our main experiments, we explore class-wise unlearning to assess how well unlearning methods can eliminate information tied to specific semantic categories. Unlike the instancewise unlearning scenario, which aims to delete individual instance samples independently of their classes, class-wise unlearning involves erasing all information about a specific class. Hence, an ideal class-wise unlearning outcome for a classification model should result in entirely incorrect predictions

Table 10: Sensitivity analysis of main loss weights ( γ 1 , γ 2 ) on CIFAR-10 with ResNet18.

for the targeted class. Table 13 presents the performance of various unlearning methods across three models trained on the CIFAR-10 dataset.

Specifically, for TRA and TUA, the reported accuracies correspond to the test retain set and test forget set, respectively, based on the targeted class within the test set D t . In alignment with the Retrain model, RA and TRA should remain as high as possible, while UA and TUA should be close to 0, indicating complete deletion.

All baseline methods generally exhibit robust class-wise unlearning performance; however, certain limitations are observed. For instance, the Bad Teaching method records high RA and TRA values, demonstrating that model utility remains intact, yet the elevated UA and TUA scores reveal incomplete deletion. SCRUB exhibits highly unstable unlearning results across different seeds, particularly for the VGG16 and ViT model. SalUn outperforms the other two baselines that utilize the retain set, showcasing more stable unlearning performance but still falling short of complete deletion.

Turning to retain-free methodologies, Boundary Shrink not only fails to preserve model utility but also does not accomplish complete deletion. In contrast, SSD and SCAR deliver impressive unlearning performance without relying on a retain set. However, SSD fails to handle larger models, as shown by its poor results on ViT. SCAR similarly struggles with large models and incurs substantial training time costs, as evidenced by the ResNet18 scenario, making it inefficient since it requires significant computation for each unlearning request. In contrast, our approach involves training the generator on out-of-distribution (OOD) data, allowing us to prepare for unlearning requests proactively before they are made.

In our proposed method, RUAGO, we filter out the generated outputs ˜ x that are predicted as belonging to the class slated for deletion. While not exhibiting the highest performance metrics, our approach provides adequate and stable unlearning performance. This characteristic stems from the core design of RUAGO: its main signal, the adversarial soft target, is optimized for reshaping decision boundaries locally around individual samples, making it highly effective for instance-wise unlearning. In contrast, class-wise unlearning requires a more global ob-

jective of erasing an entire semantic concept across the input space. To maintain a consistent and unified pipeline, we applied the same mechanism to both scenarios. This design choice led to effective unlearning, especially on VGG-16 and ViT where UA/TUA values were near-zero, though it resulted in residual information in some specific model-setting combinations. Notably, this highlights a crucial trade-off. Other retain-free methods such as SSD and SCAR, which perform well in class-wise deletion, tend to struggle significantly with instance-wise unlearning, larger models, or high computational costs. In contrast, RUAGO maintains commendable and balanced performance across both class-wise and instance-wise unlearning scenarios. This versatility underscores the applicability of RUAGO to diverse unlearning challenges, demonstrating its robustness and efficacy in various operational contexts.

I OODDatasets and Generator Pretraining

To assess distributional differences between classification datasets and generator training sets, we embed each image with a CLIP [65] encoder and compute the Fréchet Distance (FD):

where ( µ X , Σ X ) and ( µ Y , Σ Y ) represent the means and covariances of the features extracted from two datasets, X and Y , respectively. Lower FD indicates more similar distributions. Figure 5 shows these FD scores as a confusion matrix. COCO diverges markedly from all four classification datasets, and TinyImageNet has an exceptionally high FD with VGGFace2. These observations confirm that COCO and TinyImageNet are valid OOD sources under our easy-access assumption.

Table 12 summarizes ablation results using generators pretrained on three datasets: in-domain CIFAR10 and two OOD sources (COCO, TinyImageNet). Although a generator pretrained on the in-domain data (CIFAR-10) yields the best unlearning performance, it presents a critical flaw: to prevent information leakage, it must be discarded after a single unlearning task. This single-use nature makes it impractical for scenarios requiring sequential unlearning requests and introduces substantial overhead. This highlights a fundamental challenge in retain-free unlearning: a mechanism is needed to approximate the distribution of data that should be retained. We posit that an OOD-trained generator serves as an efficient and straightforward surrogate for this purpose. The importance of this pretraining step is demonstrated in Table 8, where using an untrained, randomly initialized generator results in a significant performance drop. Therefore, we focus our main approach on OOD-pretrained models.

We further evaluate an extreme OOD case by pretraining on VGGFace2. Despite its much smaller size, making it less readily accessible and poses challenges for generator training, the VGGFace2 generator only slightly underperforms the COCO model. Still, it outperforms all retain-free baselines

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

VGG16

ResNet18

ViT

(b) TinyImageNet

Bad-T

SCRUB

SALUN

BS

SSD

SCAR

RUAGO

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

VGG16

ResNet18

ViT

(a) CIFAR-100

Retrain

from Table 1, demonstrating robustness even in challenging worst-case scenarios.

We use a StyleGAN2-ADA [66, 67] backbone for these experiments. While this requires a pretraining step, we consider its cost a practical trade-off for effective unlearning under the strict retain-free constraint. Training on COCO took ≈ 4 hours and on TinyImageNet ≈ 3 hours (32×32 resolution on our GPU), with generated samples shown in Figures 6 and 7. To mitigate this dependency in future work, one could explore the use of off-the-shelf generative models, assuming their training data does not introduce other privacy issues. We expect higher-resolution training to boost unlearning performance further.

J Limitations and Future Work

We can point out several limitations of RUAGO. Initially, our experiments utilize a generator trained on datasets with rich feature content. Generally, numerous open-source datasets with rich features are easily accessible, making them easy to obtain. However, there may be extreme scenarios where such datasets are not accessible. In such situations, it remains uncertain whether unlearning would be complete when employing a generator trained on simpler datasets, such as those with single-channel images, to unlearn classification models trained on complex, high-feature datasets. Additionally, our experiments are limited to supervised image classification models. However, we believe that the fundamental design principles of RUAGO, particularly curriculum-based training over OOD samples,

-80

CIFAR-10

0.00

9.14

27.66

83.93

51.32

-70

CIFAR-100

9.14

0.00

22.21

80.98

47.48

-60

50

TinylmageNet

27.66

22.21

0.00

70.84

33.35

40

30

VGGFace2

83.93

80.98

70.84

0.00

65.26

20

COCO

51.32

47.48

33.35

65.26

0.00

-10

0

COCO

CIFAR-10

CIFAR-100

VGGFace2

TinylmageNet

adversarial soft-target regularization, and generator-teacher alignment, are not inherently tied to this specific domain. For instance, in natural language processing (NLP), these principles could be adapted by replacing the image generator with a small language model, defining sample difficulty via token-level cross-entropy, and performing alignment in the hidden feature space rather than the pixel space. Adversarial soft-targets could similarly be constructed through perturbations applied to token logits. Nonetheless, we do not claim that our method can be directly applied to generative models. Adapting RUAGO to architectures like GANs, diffusion models, or large language models would require careful redesign and new, task-specific metrics. In summary, this work establishes our method in a foundational supervised setting to analyze the core mechanics of unlearning, while acknowledging that extending it to other domains is feasible but requires further investigation.

Consequently, future work should explore more versatile methodologies, including generators trained on low-quality images or even randomly initialized generators that do not rely on OOD data. On the theoretical front, our current analysis offers a generalization bound for the curriculum learning aspect, which helps explain how scheduling sample difficulty stabilizes the unlearning process. However, a unified theoretical framework that encompasses all components of RUAGO, including the adversarial soft-target mechanism, has not yet been established and remains an important direction for future work. Another key direction for future research is to realize the roadmap for adapting RUAGO to other domains. This includes empirically verifying its effectiveness in tasks such as object detection and NLP, guided by the principles outlined above. Expanding the applicability of RUAGO is an important avenue for future research.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces **RUAGO**, a retain-free unlearning method that uses soft-label adversarial examples to achieve unlearning while avoiding over-unlearning. To preserve model performance, the paper leverages synthetic data from an OOD-trained generator, which is aligned with the original data distribution through inversion-based alignment.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n*S1 (Soundness).* All claims are well motivated and supported;\n\n*S2 (Novelty).* Original method for performing machine unlearning;\n\n*S3 (Performance).* RUAGO exhibits improved test accuracy compared to other unlearning methods;\n\n*S4 (Clarity).* The paper is clearly written.\n\n**Weaknesses**\n\n*W1.* The work does not provide guarantees regarding the success of the unlearning process. This is the main weakness of the proposed method, as the effectiveness of unlearning depends on the quality of the synthetic data generation. However,  this is a limitation shared by other state-of-the-art methods.\n\nQUESTIONS:\nI would like to point out two trivial questions (**TQ**), one minor question (**mQ**), and one major question (**MQ**) that, if addressed, would lead me to reconsider the scores for the quality and clarity of this work.\n\n**TQ1:** In Theorem 1, $R(\\cdot)$ is not defined; it becomes clear that it refers to the true risk function only in the proof. Readability would be improved if terms were defined as soon as they are introduced. For the same reason, the authors should define $E$ and $E_{g}$ as they appear in Algorithm 1.\n\n**TQ2:** In the proof of Theorem 1, on a first read, it is not immediately clear that $\\epsilon_t > 0$. The authors should emphasize this fact to improve readability.\n\n**mQ1:** The proof of Theorem 1 contains one unclear step that I would like the authors to clarify. In lines 997–998, the authors define $\\epsilon_0$, which is then used in lines 999-1000. However, it's unclear why the authors use $\\leq$ instead of $=$. While this is not a flaw per se - since the majority argument in lines 1001-1002 would lead to the same conclusion - it is not clear why the inequality is used rather than an equality.\n\n**MQ1:** My main concern regarding the proposed method is its efficacy in _truly_ forgetting information. Specifically, I suspect that the realignment process, where the updated model is aligned with the original data distribution by matching the generator's distribution to that of the original model, might inadvertently reintroduce information about the forget set. This concern is supported by several observations: first, the method often outperforms the retain model in UA performance, and sometimes even in TA performance; second, the MIA score is occasionally higher than that of the retain model; third, the method's performance appears sensitive to the choice of synthetic data generator. It would strengthen the work significantly if the authors could discuss this potential weakness in more detail, as incomplete unlearning undermines both the validity of the method and the overall contribution of the paper. Specifically, when $\\gamma_1$ and $\\gamma_2$ are kept frozen, does a strong alignment of the generator to the original data distribution hinder the unlearning process?\n\nLIMITATIONS:\nYes. The authors have addressed some of the limitations of their work in the supplementary material (see Section J - \"Limitations and Future Work\").", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-19"}

comments: SUMMARY:
This paper introduces retain-free machine unlearning framework, RUAGO. It utilizes adversarial probability module to generate soft labels to mitigate over-unlearning and a OOD data generation model to distill the original model's knowledge to unlearned model. RUAGO generally outperforms existing retain-free unlearning methods and achieves comparable performance to retain-based methods, demonstrating its effectiveness.

STRENGTHS AND WEAKNESSES:
### Strength:
- This paper proposes retain-free unlearning framework which is effective for instance-wise unlearning.
- This paper provides theoretical analysis, as well as empirical results for the proposed methods.

### Weakness:
- Proposed methods seem to be applicable on the limited situations, for example, this paper only considers image classification. Moreover, one term in L_inv is only applicable to the batchnorm-based model.
- Proposed methods introduce additional hyperparameters, such as γ_cls, γ_adv, and γ_bn, but there is no analysis about the effect of each hyperparameter.
- Pretrained generative model is necessary. Unlearning performances are highly degrades without it.

QUESTIONS:
- Although it outperforms other methods in instance-wise unlearning, it seems to be not effective in class-wise unlearning for some cases. What makes this difference? Is there any analysis or ablation study for class-wise unlearning?

LIMITATIONS:
- Although the authors theoretically have proven the generalization bound, it is limited to the curriculum learning, not to each method.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes Retain-free Unlearning via Adversarial attack and Generative model using OOD training (RUAGO), a novel retain-free image classifier unlearning method that does not require retain-data to prevent over-unlearning and preserve post-unlearning utility. RUAGO achieves this by: (1) generating adversarial soft targets from the forget set to guide the model’s previously learned decision boundaries, and (2) training a generative model on out-of-distribution (OOD) data to produce synthetic data, which can be used in place of retain-data without privacy concerns. The paper supports the use of synthetic data with a VC-theoretic analysis that informs a dynamic sample difficulty scheduler, and also employs inversion-based fine-tuning to align the generative model’s output distribution with the original data. RUAGO is evaluated against existing unlearning methods on the CIFAR-10/100, TinyImageNet, and VGGFace2 datasets using the VGG16, ResNet18, and ViT architectures, demonstrating strong empirical performance.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- [S1] **Clear motivation and novelty.** Over-unlearning and the assumption of access to original retain-data are clear limitations in existing unlearning methods. A novel technique that resolves both would be a valuable addition to the unlearning toolkit. Incorporating VC theory to design a difficulty-based learning curriculum is also a novel idea in this context and could be of significant interest to the community.\n- [S2] **Strong empirical performance.** Experimental results clearly show that RUAGO performs well in terms of both classification accuracy and privacy protection.\n\n## Weaknesses\n- [W1] **Narrow scope and applicability.** The applicability of RUAGO is currently limited to unlearning image classifiers. There does not appear to be a straightforward way to extend RUAGO to generative models (where privacy concerns are more pressing due to higher pretraining costs and the potential for data memorization) since many of RUAGO's components assume a classification setup.\n- [W2] **Unclear theoretical implications.** Section 4.3 is quite unclear in its current form. For instance in Theorem 1, what does VC-dimension and $R(\\mathcal{F})$ mean? How does Theorem 1 justify quantifying each sample’s difficulty using its loss, as in Equation 4? What assumptions underlie the generalization bound, and are they reasonable within the distillation setup? Several such pieces are missing, making it difficult to assess the theoretical contribution of the paper.\n- [W3] **Experimental complexity.** The method involves multiple objective terms during training and requires extensive tuning of loss weights (e.g., the $\\\\gamma$ weights in Equation 5 and 6). The tested ranges of these hyperparameters also seem quite broad ($\\\\gamma_{cls}, \\\\gamma_{adv}, \\\\gamma_{bn} \\\\in [0, 5]$, $\\\\gamma_1 \\\\in [0.01, 1.50]$ and $\\\\gamma_2 = 0.01$). Could the authors provide guidance on how these values/ranges were chosen, how the results are expected to behave with different hyperparameter choices, and what the actual sensitivity of the method was?\n\nQUESTIONS:\n- [Q1] Could Figure 1 be revised to make it more understandable? Specifically, what does \"G\" mean in retain-free w/ \"G\" and G trained w/ OOD? Could Baseline 1 & 2 also be specified?\n\nLIMITATIONS:\nThe authors adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-08"}