AN INFORMATION THEORETIC EVALUATION METRIC FOR STRONG UNLEARNING

## Abstract

Abstract Machine unlearning (MU) addresses privacy concerns by removing information of ‘forgetting data’ samples from trained models.
Typically, evaluating MU methods involves comparing unlearned models to those retrained from scratch without forgetting data, using metrics such as membership inference attacks (MIA) and accuracy measurements.
These evaluations implicitly assume that if the output logits of the unlearned and retrained models are similar, the unlearned model has successfully forgotten the data.
Here, we challenge if this assumption is valid.
In particular, we conduct a simple experiment of training only the last layer of a given original model using a novel masked-distillation technique while keeping the rest fixed.
Surprisingly, simply altering the last layer yields favorable outcomes in the existing evaluation metrics, while the model does not successfully unlearn the samples or classes.
For better evaluating the MU methods, we propose a metric that quantifies the residual information about forgetting data samples in intermediate features using mutual information, called information difference index or IDI for short.
The IDI provides a comprehensive evaluation of MU methods by efficiently analyzing the internal structure of DNNs.
Our metric is scalable to large datasets and adaptable to various model architectures.
Additionally, we present COLapse-and-Align (COLA), a simple contrastive-based method that effectively unlearns intermediate features.

## 1 Introduction

Machine unlearning (MU) aims to effectively remove the influence of specific data samples on a trained model [].
It emerged to prevent the leakage of private data post-training, aligning with legal protections such as “the right to be forgotten” [].
The most straightforward method for MU that guarantees information removal is exact unlearning,
which involves retraining the model from scratch after excluding the data to be forgotten.
However, this approach is computationally intensive and not scalable [, , ].
As a result, attention has turned to approximate unlearning, which enhances efficiency by relaxing the strict guarantees of influence removal.
Yet, these guarantees often rely on strong assumptions [, , , , ], such as convexity of the problem, which do not generally apply to deep neural networks (DNNs).

From a practical standpoint, MU methods for DNNs are commonly evaluated by comparing the output (i.e., predictions) of the unlearned model
with those of a model that has been retrained from scratch [, , , , , ], known as the gold standard [].
This evaluation methodology typically incorporates techniques such as membership inference attacks (MIA) [, ] and accuracy measurements.
The similarity in the outputs between the models is commonly believed to be a reliable standard for assessing the effectiveness of unlearning.

In this work, we examine whether this belief holds in a typical classification task.
We begin by making a minimal adjustment, altering only the last layer (using a novel masked-distillation technique) of the model
while preserving all information in the intermediate layers.
Surprisingly, we find that this simple change can misleadingly satisfy current evaluation metrics, casting doubt on the efficacy of existing MU methods.
Specifically, it calls into question whether methods that yield favorable outputs might still retain information in the intermediate layers,
potentially undermining the intended purpose of unlearning.

Consequently, inspired by the Information Bottleneck principle [, ], which views DNN training as modulating information flow through layers,
we aim to measure the residual information in the intermediate features using Shannon’s mutual information [].
This analysis focuses on how much information the features retain about input data labels, evaluating the model’s difficulty in effectively extracting valuable features from forgetting data.
Interestingly, we observe that several existing MU methods, despite producing favorable outputs, still retain significant information about the forgetting data within intermediate layers.
These persistent features can reconstruct forgotten information with minimal data, thereby illustrating the risk of incomplete unlearning and highlighting the inadequacy of current evaluation metrics that overlook the model’s internal dynamics.

To address this, we introduce the information difference index (IDI), a novel metric that quantifies the information retained within intermediate features.
The IDI compares the mutual information of intermediate features and the forgetting labels for both the unlearned and retrained models.
By employing consistent model structures for estimating mutual information, IDI is adaptable across various architectures and remains reliable despite randomness in the unlearning process.
Furthermore, IDI effectively evaluates MU methods relative to each other without requiring full training data, making it ideal for large-scale or incomplete datasets common in real-world scenarios.

Additionally, our observations indicate that existing methods, which retain substantial residual feature-level information, primarily modify the linear classifier rather than the feature extractor during the unlearning process.
To effectively address this issue, we introduce COLapse-and-Align (COLA), a method specifically designed to directly eliminate residual information from intermediate features of forgetting data.
COLA employs a two-step framework: the first step uses supervised contrastive loss [] to remove feature-level information,
and the second step fine-tunes the entire model.
Despite its simplicity, COLA outperforms other MU methods in IDI and is comparable to existing metrics on CIFAR-10, CIFAR-100 and ImageNet-1K, and architectures such as ResNet-18, ResNet-50 and ViT.
Furthermore, it does not access forgetting data during unlearning, making it suitable for real-world scenarios where such data may be unavailable.

We summarize our contributions as follows:

- e challenge existing assumptions by demonstrating that altering only the last layer of a model can misleadingly satisfy current unlearning metrics,
highlighting the insufficiency of these evaluation criteria.
- 1.
We introduce the information difference index (IDI), a novel metric that uses mutual information to quantify residual information about forgetting data in intermediate features, ensuring applicability across various architectures.
- 2.
We propose COLapse-and-Align (COLA), a simple contrastive-based method that effectively unlearns intermediate features without necessitating access to the forgetting data.
- 3.
We perform comprehensive experiments across a wide range of datasets, models, methods, and unlearning scenarios.
Our findings consistently demonstrate the critical importance of the IDI metric and the remarkable performance of the COLA method.

## 2 Problem Statement and Preliminaries

### 2.1 Problem Statement

Let $\mathcal{D}=\{(x_{i},y_{i})\}_{i=1}^{N}$ denote the training dataset,
which consists of $N$ samples, each represented by an image-label pair $(x_{i},y_{i})$.
In a supervised learning setup, we partition the dataset into two subsets: the forget set $\mathcal{D}_{f}$,
which comprises instances whose information is intended to be forgotten, and retain set $\mathcal{D}_{r}=\mathcal{D}\setminus\mathcal{D}_{f}$, which includes instances whose information is to be retained.
The initial model, $\mathbf{\theta_{o}}$ (referred to as Original), is trained on the complete dataset $\mathcal{D}$ via empirical risk minimization.
The unlearned model, $\mathbf{\theta_{u}}$, is the result of modifying Original through a specific unlearning algorithm that aims to remove influence of $\mathcal{D}_{f}$.
The objective of machine unlearning (MU) is to ensure that the unlearned model $\mathbf{\theta_{u}}$ approximates the model retrained from scratch exclusively on the $\mathcal{D}_{r}$ (referred to as Retrain), which is the ideal outcome.
Both Original and Retrain employ identical training methodologies, differing only in their respective datasets, $\mathcal{D}$ and $\mathcal{D}_{r}$.

MU is commonly studied in the context of image classification [, ], where the methodology can vary based on the nature of the forget set.
Two primary scenarios are identified: class-wise forgetting, where all samples of a specific class are targeted,
and random data forgetting, which involves indiscriminate sample selection across all classes.
We primarily explores class-wise forgetting, given the complexity and challenges associated with quantifying the influence of random samples in DNNs.

Throughout the paper, within a given model $\mathbf{\theta}$, we define the head as the last few layers responsible for classification; typically one to three linear layers.
The encoder, on the contrary, encompasses the remainder of the network, which usually consists of convolutional layers or transformer encoders.

### 2.2 Preliminaries

#### Machine Unlearning (MU).

Exact unlearning, which involves creating Retrain, guarantees information removal but is computationally burdensome [, , , ].
To address this, approximate unlearning methods have been developed, prioritizing efficiency over theoretical guarantees.
Several studies have employed differential privacy (DP) [] inspired approaches to make models “indistinguishable” in parameters from those of Retrain, rather than identical [, , , , ].
However, the concept of “indistinguishability” for parameters within DNNs has recently faced criticism [, , ].
Consequently, the focus has shifted to empirically assessing unlearning by comparing the outputs (i.e., predictions) between the unlearned model and Retrain.
In our work, we primarily address the limitations of this empirical assessment in MU for DNNs.

#### Approximate MU Methods.

Several key methodologies in MU for DNNs have been developed: Finetuning (FT) [] fine-tunes Original $\mathbf{\theta_{o}}$ with retain set $\mathcal{D}_{r}$, inducing catastrophic forgetting [, ] of $\mathcal{D}_{f}$.
Random labeling (RL) [] involves fine-tuning $\mathbf{\theta_{o}}$ with randomly labeled forget set $\mathcal{D}_{f}$.
Gradient ascent (GA) [] trains $\mathbf{\theta_{o}}$ with reverse gradient steps using $\mathcal{D}_{f}$.
Bad-T [] uses a teacher-student framework that utilizes distillation techniques,
distinguishing between beneficial and detrimental influences through good and bad teachers to refine the learning process.
Catastrophic forgetting-k (CF-k) and exact unlearning-k (EU-k) []
involve either fine-tuning (CF-k) or retraining (EU-k) the last $k$ layers of the model using $\mathcal{D}_{r}$ while freezing the prior layers.
SCRUB [] employs a technique of positive distillation from $\mathbf{\theta_{o}}$ using the $\mathcal{D}_{r}$,
and negative distillation on the $\mathcal{D}_{f}$, which helps in selectively retaining beneficial knowledge while discarding the unwanted influences.
$\mathbf{\ell_{1}}$-sparse [] enhances the model’s ability to forget by strategically inducing weight sparsity in $\mathbf{\theta_{o}}$.
SALUN [] fine-tunes the salient weights of $\mathbf{\theta_{o}}$ using a method that incorporates random labeling.

#### Evaluation Metrics for Unlearning.

Prevalent evaluation metrics that assess the outputs of unlearned models and Retrain include accuracy measurements and membership inference attack (MIA) [, , , , ].
Accuracy is segmented into unlearning accuracy (UA), which quantifies the model’s ability to forget specific data by calculating UA$(\mathbf{\theta_{u}})$ $=1-\text{Acc}_{\mathcal{D}_{f}}(\mathbf{\theta_{u}})$, where $\text{Acc}_{\mathcal{D}}(\mathbf{\theta})$ is the accuracy of model $\mathbf{\theta}$ on dataset $\mathcal{D}$;
remaining accuracy (RA), which assesses the retention of predictive performance in $\mathcal{D}_{r}$;
and testing accuracy (TA), which evaluates generalization to unseen data.
The membership inference attack (MIA) [, ] determines whether a specific data record was part of the model’s training set.
Thus, a lower success rate of MIA on a forget set is typically preferred in unlearned models.
Additionally, other metrics such as activation distance and Jensen-Shannon divergence (JSD) directly compare output logits between the unlearned model and Retrain [].

While many studies primarily focus on the outputs of unlearned models, concerns have been raised about the validity and sufficiency of these evaluations.
Time-based metrics, such as the run-time efficiency (RTE) measures the computational effort involved in the unlearning process,
with shorter times indicating more efficient unlearning [, , , ].
Anamnesis index (AIN) [, ] and relearn time (RT) [, , , ] measure the time (epochs) needed for the unlearned model to regain a specified performance level on the forgetting data.
Although these metrics serve as proxies for unlearning efficacy, precise evaluations can be challenging due to sensitive hyperparameters [].
Becker and Liebig [] introduced a metric assessing information within model parameters, yet their empirical results did not align with theoretical expectations.
Hayes et al. [] provided thorough experiments demonstrating the overestimation of MIA.
Additionally, Goel et al. [] introduced the interclass confusion (IC) test to evaluate the validity of unlearning algorithms through adversarial mislabeling scenarios, although it still relies on output-based metrics.

## 3 Rethinking the Evaluation of Machine Unlearning

### 3.1 Head Distillation: Simple Unlearning Technique Challenges Existing Metrics

Figure: Figure 1: Overview of head distillation (HD). Original distills knowledge into the unlearned model’s head after masking the forgetting class logit, while keeping the encoder frozen.
Refer to caption: /html/2405.17878/assets/x1.png

In this section, we discuss the limitations of current MU evaluation criteria that focus on output logits and overlook intermediate features.
Using a simple unlearning strategy, we show how commonly used criteria can misinterpret the effectiveness of unlearning methods in a single-class forgetting task.

Drawing inspiration from the teacher-student framework [, ],
our strategy, termed head distillation (HD), employs logit distillation from Original $\mathbf{\theta_{o}}$.
Specifically, the unlearned model $\mathbf{\theta_{u}}$ is initialized from $\mathbf{\theta_{o}}$ with the encoder of the network frozen,
leaving only the head trainable.
During the unlearning process, the head is fine-tuned on training dataset $\mathcal{D}$ using KL-divergence loss []
to follow the masked output from $\mathbf{\theta_{o}}$, with the logit for the forgetting class set to 0 while maintaining the logits for the remaining classes,
as shown in Figure [1](#S3.F1).
This approach enables the head to mimic a pseudo-retrained model, as the masked logits closely resemble those of Retrain.
By doing so, HD aims to simulate the desired unlearning effect by aligning the output behavior of the unlearned model’s head with that of Retrain.

We use the CIFAR-10 [] dataset and the ResNet-18 [] model for our experiments.
In ResNet architecture, the head is comprised of a single layer.
For evaluation, we employ metrics such as unlearning accuracy (UA), remaining accuracy (RA), testing accuracy (TA), and membership inference attack (MIA).
Additionally, we measure run-time efficiency (RTE) to assess algorithm efficiency, consistent with recent studies [, , , ].

Figure [2](#S3.F2) shows the results of the experiment.
Despite its simplicity, HD outperforms other baseline methods (FT, Bad-T, $\ell_{1}$-sparse, SALUN) in testing accuracy (TA),
achieving this with the lowest computational cost of just 6.21 seconds while also providing robust performance against membership attacks.
All methods achieve perfect performance (100%) in unlearning accuracy (UA).

The experimental results indicate that HD is exceptionally efficient under current evaluation criteria.
However, the validity of HD as an effective MU algorithm merits scrutiny.
A critical concern with HD is its substantial similarity to Original $\mathbf{\theta_{o}}$, where the only difference is the single-layer head.
Specifically, the encoder in HD remains identical to that of $\mathbf{\theta_{o}}$, thereby preserving all intermediate features associated with $\mathcal{D}_{f}$.
This observation prompts a question:

Figure: Figure 2: Performance of six methods (HD, FT, Bad-T, SCRUB, $\ell_{1}$-sparse, SALUN) in single-class forgetting on (CIFAR-10, ResNet-18). Left: run-time efficiency, lower the better, Middle: testing accuracy, closer to Retrain better, Right: membership inference attack, closer to Retrain better.
Refer to caption: /html/2405.17878/assets/x2.png

Figure: (a) Original
Refer to caption: /html/2405.17878/assets/x3.png

### 3.2 Residual Information of Forgetting Data in Intermediate Features

To answer the above question, we present a qualitative analysis of features from current unlearning methods that show favorable results
under existing evaluation criteria.
Our aim is to determine whether these methods effectively remove feature-level information about the forget set.

Figure [3](#S3.F3) shows the t-SNE visualizations of the encoder outputs for each unlearned network.
These visualizations provide additional insights that extend beyond standard evaluation metrics, revealing nuanced aspects of the unlearning process.
Initially, we notice that the features corresponding to the forgetting class (represented in purple) of Retrain are highly scattered.
These features spread across various clusters, indicating that the model struggles to extract distinctive and coherent information from images of the forgetting class.
Such scattering could be interpreted as an ideal outcome in unlearning,
suggesting that the unlearned model has effectively ‘forgotten’ how to extract semantic representations from the forget set.

In contrast, features from RL [] and SALUN [] closely mirror those of Original, rather than Retrain.
This observation indicates that these models continue to retain a substantial ability to recognize and process features of the forgetting class,
even after the unlearning process.
Thus, it raises a significant concern that the apparent success of the unlearning process in MU methods might primarily be due to modifications in the head, deceiving the current metrics, while deeper and more discriminative information of the forget set is preserved in the intermediate features.

### 3.3 Risks of Residual Information in Intermediate Features

Figure: Figure 4: Test accuracy on forget test data for Original, Retrain, and unlearned models after head retraining (with fixed encoders) using 2% of $\mathcal{D}$ on (CIFAR-10, ResNet-18). HD’s performance matches that of Original.
Refer to caption: /html/2405.17878/assets/x8.png

To highlight the importance of considering intermediate features in MU,
we conduct a follow-up experiment inspired by time-based metrics (e.g.,  [, ]).
This experiment assess how well-preserved intermediate features can easily reconstruct forgotten information with a minimal amount of data.

Precisely, we replace the heads of various unlearned models, including Retrain and Original, with randomly initialized ones.
We then freeze all encoders and train these new heads using $\mathcal{D^{\prime}}$, a small subset (2% of the total) of $\mathcal{D}$
consisting of randomly chosen samples.
After training, we evaluate the predictive accuracy of new models on a forget test data.
Surprisingly, as shown in Figure [4](#S3.F4), while the retrained head of Retrain achieves no more than 41% accuracy on the forget test data,
retrained heads from some unlearning algorithms (RL, Bad-T) exhibit over 95% accuracy.
Note that the test accuracy of HD matches that of Original, as both use the same encoders.

The recovery of high accuracy in some unlearned models highlights the risk of residual influence of the forgetting data in intermediate layers, demonstrating the limitations of the current black-box approach to evaluating MU methods.
This finding emphasizes that true ‘unlearning’ must involve removing information from intermediate features,
rather than merely adjusting the output layer as seen in HD.
Therefore, it is imperative to expand evaluation criteria in MU to thoroughly assess internal information,
ensuring comprehensive unlearning and genuine privacy protection.

## 4 An Information Theoretic Metric Using Intermediate Features

Current MU evaluation criteria focus on outputs and overlook residual information in intermediate layers, creating a gap in accurately assessing unlearning efficacy.
To address this gap, we propose a novel metric that moves beyond output-based evaluations,
providing a comprehensive assessment of unlearning efficacy by examining the internal representations of DNNs.
Specifically, this metric quantifies how much information the intermediate features retain about data labels (indicating data to retain or forget), revealing the model’s ability to differentiate between these categories.

### 4.1 Information Difference Index (IDI)

Figure: Figure 5: Illustration of IDI. Curves show estimated mutual information $I(\mathbf{Z}_{\ell};Y)$ for Original (⚫), unlearned (▲), and Retrain (★). IDI is the ratio $\frac{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\textbf{ID}(\mathbf{\theta_{u}})}}{{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\textbf{ID}(\mathbf{\theta_{o}})}}$, corresponding to the red area divided by the blue area.
Refer to caption: /html/2405.17878/assets/x9.png

To quantify the relationship between high dimensional intermediate features and data labels, we utilize Shannon’s mutual information (MI), a robust measure that effectively captures variable dependencies across various dimensional complexities.

For an input $\mathbf{X}$, let $\mathbf{Z}^{(\mathbf{u})}_{\ell}$ and $\mathbf{Z}^{(\mathbf{r})}_{\ell}$ denote the features from the $\ell$-th layer of $L$-layer unlearned encoder and Retrain encoder, respectively. Let $Y$ be a binary label where $Y=1$ indicates that the corresponding input $\mathbf{X}$ belongs to the forget set $\mathcal{D}_{f}$, and $Y=0$ otherwise.
We measure the MI, denoted as $I(\mathbf{Z}^{(\mathbf{u})}_{\ell};Y)$, across each layer from 1 to $L$, to determine whether intermediate features retain information about $\mathcal{D}_{f}$.

We define the information difference (ID) of $\mathbf{\theta_{u}}$ as the MI difference across intermediate layers between the unlearned model and Retrain, calculated as:

$$ $\mathbf{ID}(\mathbf{\theta_{u}})=\sum_{\ell=1}^{L}\bigl{(}I(\mathbf{Z}^{(\mathbf{u})}_{\ell};Y)-I(\mathbf{Z}^{(\mathbf{r})}_{\ell};Y)\bigr{)}.$ (1) $$

ID of $\mathbf{\theta_{u}}$ shows the extent of information retention through ensuing layers of the unlearned encoder.

To provide a normalized measure, we introduce the information difference index (IDI):

$$ $\mathbf{IDI}(\mathbf{\theta_{u}})=\frac{\mathbf{ID}(\mathbf{\theta_{u}})}{\mathbf{ID}(\mathbf{\theta_{0}})}=\frac{\sum_{\ell=1}^{L}\bigl{(}I(\mathbf{Z}^{(\mathbf{u})}_{\ell};Y)-I(\mathbf{Z}^{(\mathbf{r})}_{\ell};Y)\bigr{)}}{\sum_{\ell=1}^{L}\bigl{(}I(\mathbf{Z}^{(\mathbf{o})}_{\ell};Y)-I(\mathbf{Z}^{(\mathbf{r})}_{\ell};Y)\bigr{)}},$ (2) $$

where $\mathbf{Z}^{(\mathbf{o})}_{\ell}$ is the output of the $\ell$-th layer of Original encoder.
Figure [5](#S4.F5) illustrates IDI, which is conceptually the ratio of the areas between MI curves.

By comparing the MI curves of Original and Retrain,
our metric quantifies the degree to which the unlearned model has reduced this informational gap.
An IDI of 0 denotes that the unlearned model has completely removed all information related to the forget set,
achieving indistinguishability from Retrain.
In contrast, an IDI of 1 indicates that the encoder retains all the information present in Original.
Interestingly, a negative IDI value, termed over-unlearning, occurs when a model removes more information than Retrain,
potentially exceeding the intended unlearning objectives.

As shown in Figure [5](#S4.F5), significant differences in the MI curves appear in the later layers, which predominantly extract task-specific features [].
Therefore, computing IDI for just these layers is generally sufficient for efficiency.
Additionally, while we have defined IDI for class-wise forgetting,
it can also be appropriately defined for random data forgetting.
Please refer to Appendix [A.1](#A1.SS1).

### 4.2 Estimating Mutual Information for IDI

To estimate MI for computing IDI, we use the InfoNCE loss [].
InfoNCE is widely used in MI estimation across various contexts
(e.g.,  [, ]) due to its robustness and effectiveness.
It uses noise-contrastive estimation to provide reliable approximations of MI, making it suitable for high-dimensional data typically encountered in DNNs.

For this purpose, we use the separable InfoNCE lower bound. Given a batch of size $K$, defined as $\{(U^{(k)},V^{(k)}):1\leq k\leq K\}$ sampled from the joint distribution $P_{U,V}$, where $U$ and $V$ are random variables, the InfoNCE loss is formulated as follows:

$$ $\mathcal{L}_{\text{NCE}}(\nu,\eta)=\frac{1}{K}\sum_{k=1}^{K}\log\frac{\exp(f_{\nu}(U^{(k)})^{\intercal}g_{\eta}(V^{(k)}))}{\frac{1}{K}\sum_{k^{\prime}=1}^{K}\exp(f_{\nu}(U^{(k)})^{\intercal}g_{\eta}(V^{(k^{\prime})}))},$ (3) $$

where $f_{\mu}$ and $g_{\eta}$ are critic functions optimized to maximize the InfoNCE loss.
Although the InfoNCE loss is indeed a lower bound of MI, maximizing it effectively estimates $I(U;V)$ [].

Figure: Figure 6: Illustration of estimating MI using InfoNCE. The critic function $f_{\nu_{\ell}}$ represents a trainable network to capture features from $\mathbf{Z}_{\ell}$, while the critic function $g_{\eta_{\ell}}$ handles the binary input $Y$.
Refer to caption: /html/2405.17878/assets/x10.png

To tailor the InfoNCE loss to estimate $I(\mathbf{Z}_{\ell};Y)$, the choice of network architecture for $f_{\nu_{\ell}}$ and $g_{\eta_{\ell}}$ is crucial.
The critic function $f_{\nu_{\ell}}$ should be able to extract features from $\mathbf{Z}_{\ell}$, which may vary by layer $\ell$.
For early layers (small $\ell$), $f_{\nu_{\ell}}$ must be sufficiently deep to capture relevant features, whereas for higher layers (large $\ell$),
a shallower function might suffice.
Thus, the natural selection for $f_{\nu_{\ell}}$ is the remainder of the original network, spanning from the $\ell+1$-th to the $L$-th layer.
To ensure dimensional compatibility between $f$ and $g$, we add an additional linear layer with $d$ outputs,
producing $f_{\nu_{\ell}}(\mathbf{Z}_{\ell})\in\mathbb{R}^{d}$.

More precisely, our setup involves freezing the parameters of the network up to the $\ell$-th layer and training from the $\ell+1$-th layer onwards,
with an added projection layer at the end for dimensional consistency.
Figure [6](#S4.F6) provides a visual illustration that details the choice of $f_{\nu_{\ell}}$.
The function $g_{\eta_{\ell}}$ is simpler because it deals with the binary input $Y$.
It utilizes a $d$-dimensional trainable vector, either $g_{\eta_{\ell}}(0)$ or $g_{\eta_{\ell}}(1)$, corresponding to the binary label.

The critic function $f_{\nu_{\ell}}$ is effectively chosen to classify $Y$ using the intermediate feature $\mathbf{Z}_{\ell}$, providing key insight.
This process is central to our investigation,
as it tests the unlearned model’s ability to identify whether an input belongs to the forget set based solely on the feature of the $\ell$-th layer.

### 4.3 Re-evaluating Unlearning Schemes Using IDI

With our new unlearning metric,
we re-evaluate current unlearning methods to verify whether this metric can capture the amount of information retained in intermediate features.
We consider three datasets: CIFAR-10, CIFAR-100 [], and ImageNet-1K [],
and three model architectures: ResNet-18, ResNet-50 [], and ViT [].
For simplicity, we approximate IDI using the intermediate features of ResNet blocks instead of every layer. See Appendix [B](#A2) for implementation details.

We begin by plotting the estimated MI between the intermediate layers and the binary label indicating whether the data belong to the forget set, as shown in Figure [7](#S4.F7) for ResNet-18 and ResNet-50 on CIFAR-10.
As expected, MI decreases across layers, aligning with the Information Bottleneck principle [].
This figure highlights the distinctions between the unlearned models that standard metrics fail to capture.
Furthermore, Figure [7](#S4.F7) provides the corresponding IDI values,
effectively quantifying the amount of residual information about the forget set in the intermediate features.

Figure: Figure 7: MI curves and their corresponding IDI for the Original, Retrain, and unlearned models from six methods (FT, RL, GA, $\ell_{1}$-sparse, SCRUB, SALUN) across ResNet-18 (left), ResNet-50 (right) blocks on CIFAR-10 in single-class forgetting task.
Refer to caption: /html/2405.17878/assets/x11.png

In particular, SCRUB and $\ell_{1}$-sparse, which approximate the MI levels of Retrain, are more likely to meet the MU objective at the feature level for both model architectures.
Conversely, SALUN and RL show curves that are close to that of Original, indicating the opposite.
Note that HD has the same MI curve as Original, since all layers within the encoder remain unchanged.
Additionally, the IDI values for ResNet-18, as shown in Figure [7](#S4.F7) (left), are consistent with the recovery experiment results in Figure [4](#S3.F4) (high IDI correlates with high accuracy).
These values also align with the t-SNE plots in Figure [3](#S3.F3), supporting the reliability of IDI.
Further analysis is available in Appendix [D](#A4).

We observe similar patterns in the CIFAR-100 and ImageNet-1K datasets, as well as in the ViT architecture.
Furthermore, we extend our experiment to multi-class forgetting tasks involving multiple forgetting classes (e.g., 5 classes on CIFAR-100),
which reveal more pronounced MI differences among the MU methods.
For more details, see Appendix [C](#A3).

#### Consistency and Scalablility of IDI.

Figure: Figure 8: IDI of six methods with various binary label ratios, in the single-class forgetting task on (CIFAR-100, ResNet-18). The ratio $a:b$ indicates the relative amounts of forgetting and retaining samples, respectively.

Consistency is crucial for unlearning metrics, an area where some metrics often fall short
[, , ].
Also, a major issue with using model parameters for MU evaluation in DNNs is their inconsistency.
With the same algorithm, parameters can vary significantly due to stochastic training factors (e.g., random seeds),
making comparisons between the unlearned model and Retrain ambiguous [, ].
However, IDI produces consistent results, as shown by the low standard deviation across independent trials (see Table [1](#S5.T1)).

Furthermore, IDI provides consistent outcomes without requiring the entire dataset $\mathcal{D}$.
As shown in Figure [8](#S4.F8), the relative rankings of MU methods remain stable across various ratios of binary data in the class-wise forgetting task on CIFAR-100.
This demonstrates the effectiveness of IDI for evaluating methods on large-scale datasets or incomplete datasets where only a subset of $\mathcal{D}$ is available.

## 5 Proposed Baseline: COLlapse and Align (COLA)

Building on our findings about the limitations of existing MU methods and the efficacy of the IDI metric,
we propose a two-step unlearning scheme called COLlapse and Align (COLA).
This framework ensures comprehensive unlearning by effectively removing information from intermediate layers, thereby mitigating the risks identified in previous sections.
Precisely, COLA removes feature-level information in the collapse phase,
and optimizes the entire model in the alignment phase.

A straightforward unlearning strategy used by methods such as FT and CF-k is catastrophic forgetting,
which involves preserving essential knowledge while allowing unnecessary information to naturally diminish.
However, our analysis in Section [4](#S4) reveals that these methods still retain substantial feature-level information about the forget set $\mathcal{D}_{f}$
after the unlearning process.
This suggests that models are more inclined to adapt the head rather than modify the encoder,
potentially limiting the effectiveness of catastrophic forgetting as an unlearning strategy.

Hence, during the collapse phase, we focus on directly inducing catastrophic forgetting within the encoder to remove information in intermediate features.
To this end, we apply supervised contrastive loss [] to the encoder outputs from the retain set $\mathcal{D}_{r}$,
which aims to preserve the model’s ability to distinguish features within $\mathcal{D}_{r}$ by promoting tight intra-class clustering.
As these clusters shrink, they indirectly force the features of the forget set $\mathcal{D}_{f}$ to disperse,
leading to their collapse into the clusters of $\mathcal{D}_{r}$.
Subsequently, in the align phase, after collapsing the features, we apply standard cross-entropy loss to $\mathcal{D}_{r}$ to align the entire model for desirable outputs.
For an intuitive illustration of COLA and a detailed mathematical formulation of the loss term, please refer to Appendix [B.5](#A2.SS5).

**Table 1: Performance summary of MU methods (including the proposed COLA and 11 other baselines) for a single-class forgetting task on (CIFAR-10, ResNet-18) and (CIFAR-100, ResNet-50). Results are shown as $a\pm b$, with $a$ being the mean and $b$ the standard deviation from 5 independent trials. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
|  | CIFAR-10 | CIFAR-100 |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | IDI | RTE (min) | UA | RA | TA | MIA | IDI | RTE (min) |
| Retrain | $100.0$ | $100.0$ | $95.64$ | $10.64$ | $0.0$ | $154.56$ | $100.0$ | $99.97$ | $79.42$ | $3.40$ | $0.0$ | $312.24$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.12_{\pm 0.09}$ | $0.17_{\pm 0.05}$ | $0.671_{\pm 0.008}$ | $6.44_{\pm 0.07}$ | $99.33_{\pm 0.09}$ | $99.93_{\pm 0.03}$ | $77.71_{\pm 0.18}$ | $0.40_{\pm 0.16}$ | $0.618_{\pm 0.018}$ | $16.34_{\pm 0.48}$ |
| RL [] | $99.93_{\pm 0.01}$ | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{95.66}_{\pm 0.05}$ | $0.0_{\pm 0.0}$ | $0.830_{\pm 0.005}$ | $3.09_{\pm 0.03}$ | $\textbf{100.0}_{\pm 0.0}$ | $99.95_{\pm 0.02}$ | $\underline{79.56}_{\pm 0.04}$ | $0.0_{\pm 0.0}$ | $0.649_{\pm 0.013}$ | $8.38_{\pm 0.14}$ |
| GA [] | $\textbf{100.0}_{\pm 0.0}$ | $99.06_{\pm 0.25}$ | $93.10_{\pm 0.50}$ | $25.37_{\pm 3.24}$ | $0.334_{\pm 0.014}$ | $4.00_{\pm 0.08}$ | $99.60_{\pm 0.43}$ | $98.00_{\pm 0.72}$ | $72.73_{\pm 1.16}$ | $13.33_{\pm 4.43}$ | $0.526_{\pm 0.009}$ | $9.50_{\pm 0.54}$ |
| Bad-T [] | $99.90_{\pm 0.14}$ | $99.99_{\pm 0.0}$ | $94.99_{\pm 0.12}$ | $68.17_{\pm 42.80}$ | $1.014_{\pm 0.004}$ | $4.64_{\pm 0.05}$ | $\textbf{100.0}_{\pm 0.0}$ | $99.90_{\pm 0.10}$ | $77.53_{\pm 1.21}$ | $94.80_{\pm 2.75}$ | $0.990_{\pm 0.033}$ | $12.69_{\pm 1.54}$ |
| EU-5 [] | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.25_{\pm 0.02}$ | $0.06_{\pm 0.03}$ | $0.528_{\pm 0.005}$ | $\textbf{1.54}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{99.97}_{\pm 0.01}$ | $78.31_{\pm 0.21}$ | $\textbf{1.20}_{\pm 0.99}$ | $0.520_{\pm 0.023}$ | $\textbf{6.81}_{\pm 0.01}$ |
| CF-5 [] | $98.13_{\pm 1.39}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.54_{\pm 0.09}$ | $0.0_{\pm 0.0}$ | $0.675_{\pm 0.027}$ | $\underline{1.57}_{\pm 0.03}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{99.97}_{\pm 0.01}$ | $78.98_{\pm 0.16}$ | $0.27_{\pm 0.09}$ | $0.575_{\pm 0.016}$ | $\underline{6.82}_{\pm 0.01}$ |
| EU-10 [] | $\textbf{100.0}_{\pm 0.0}$ | $99.50_{\pm 0.02}$ | $93.61_{\pm 0.08}$ | $15.24_{\pm 1.08}$ | $-0.349_{\pm 0.019}$ | $2.42_{\pm 0.11}$ | $\textbf{100.0}_{\pm 0.0}$ | $98.52_{\pm 0.14}$ | $75.66_{\pm 0.03}$ | $15.00_{\pm 1.45}$ | $\underline{0.050}_{\pm 0.004}$ | $7.81_{\pm 0.01}$ |
| CF-10 [] | $\textbf{100.0}_{\pm 0.0}$ | $99.98_{\pm 0.0}$ | $94.95_{\pm 0.05}$ | $\textbf{11.61}_{\pm 0.91}$ | $-0.060_{\pm 0.017}$ | $2.31_{\pm 0.03}$ | $\textbf{100.0}_{\pm 0.0}$ | $99.95_{\pm 0.01}$ | $78.47_{\pm 0.10}$ | $\underline{5.87}_{\pm 0.09}$ | $0.302_{\pm 0.035}$ | $7.82_{\pm 0.02}$ |
| SCRUB [] | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.37_{\pm 0.04}$ | $19.73_{\pm 1.92}$ | $\underline{-0.056}_{\pm 0.008}$ | $3.49_{\pm 0.02}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{99.97}_{\pm 0.0}$ | $79.61_{\pm 0.09}$ | $0.20_{\pm 0.16}$ | $0.620_{\pm 0.035}$ | $4.59_{\pm 0.13}$ |
| SALUN [] | $99.99_{\pm 0.01}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.42_{\pm 0.12}$ | $0.01_{\pm 0.01}$ | $0.936_{\pm 0.012}$ | $3.54_{\pm 0.11}$ | $99.73_{\pm 0.38}$ | $\textbf{99.98}_{\pm 0.0}$ | $\textbf{79.51}_{\pm 0.15}$ | $0.0_{\pm 0.0}$ | $0.679_{\pm 0.010}$ | $12.83_{\pm 0.87}$ |
| $\mathbf{\ell_{1}}$-sparse [] | $\textbf{100.0}_{\pm 0.0}$ | $99.93_{\pm 0.02}$ | $94.90_{\pm 0.10}$ | $1.56_{\pm 0.09}$ | $0.293_{\pm 0.012}$ | $2.96_{\pm 0.03}$ | $96.20_{\pm 0.16}$ | $99.42_{\pm 0.06}$ | $76.16_{\pm 0.31}$ | $2.60_{\pm 0.33}$ | $0.325_{\pm 0.018}$ | $15.78_{\pm 0.05}$ |
| COLA (Ours) | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{95.55}_{\pm 0.06}$ | $\underline{12.64}_{\pm 0.92}$ | $\textbf{0.010}_{\pm 0.006}$ | $4.91_{\pm 0.04}$ | $\textbf{100.0}_{\pm 0.0}$ | $99.90_{\pm 0.01}$ | $78.59_{\pm 0.28}$ | $10.27_{\pm 0.90}$ | $\textbf{0.016}_{\pm 0.031}$ | $16.25_{\pm 0.10}$ |

**Table 2: Performance summary of methods for a multi-class forgetting task (forget 5 classes) on (ImageNet-1K, ResNet-50). A better performance of an MU method corresponds to a smaller performance gap with Retrain, with the top method in bold and the second best underlined.**
|  | ImageNet-1K |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | IDI |
| Retrain | $100.0$ | $88.80$ | $75.88$ | $9.41$ | $0.0$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{88.52}_{\pm 0.0}$ | $\underline{76.16}_{\pm 0.01}$ | $8.24_{\pm 1.23}$ | $0.102_{\pm 0.026}$ |
| RL [] | $99.96_{\pm 0.04}$ | $86.48_{\pm 0.06}$ | $75.25_{\pm 0.02}$ | $0.06_{\pm 0.02}$ | $1.296_{\pm 0.084}$ |
| GA [] | $\textbf{100.0}_{\pm 0.0}$ | $80.77_{\pm 0.22}$ | $71.49_{\pm 0.10}$ | $4.20_{\pm 0.46}$ | $0.328_{\pm 0.023}$ |
| Bad-T [] | $97.93_{\pm 0.01}$ | $84.01_{\pm 0.03}$ | $73.39_{\pm 0.05}$ | $71.42_{\pm 13.62}$ | $1.683_{\pm 0.077}$ |
| EU-5 [] | $\textbf{100.0}_{\pm 0.0}$ | $79.62_{\pm 0.0}$ | $71.22_{\pm 0.13}$ | $13.33_{\pm 1.53}$ | $0.183_{\pm 0.028}$ |
| CF-5 [] | $\textbf{100.0}_{\pm 0.0}$ | $84.28_{\pm 0.05}$ | $74.15_{\pm 0.08}$ | $10.38_{\pm 4.67}$ | $0.696_{\pm 0.143}$ |
| EU-10 [] | $\textbf{100.0}_{\pm 0.0}$ | $71.84_{\pm 0.03}$ | $65.78_{\pm 0.02}$ | $16.65_{\pm 1.91}$ | $\underline{-0.051}_{\pm 0.021}$ |
| CF-10 [] | $\textbf{100.0}_{\pm 0.0}$ | $81.12_{\pm 0.07}$ | $72.32_{\pm 0.09}$ | $14.60_{\pm 6.19}$ | $0.608_{\pm 0.135}$ |
| SCRUB [] | $99.36_{\pm 0.02}$ | $\underline{88.41}_{\pm 0.03}$ | $76.48_{\pm 0.02}$ | $7.49_{\pm 0.78}$ | $0.517_{\pm 0.107}$ |
| SALUN [] | $89.67_{\pm 0.27}$ | $86.25_{\pm 0.15}$ | $75.54_{\pm 0.10}$ | $0.50_{\pm 0.09}$ | $0.343_{\pm 0.017}$ |
| $\mathbf{\ell_{1}}$-sparse [] | $97.57_{\pm 0.61}$ | $85.33_{\pm 0.07}$ | $74.77_{\pm 0.03}$ | $\underline{8.84}_{\pm 1.39}$ | $0.239_{\pm 0.031}$ |
| COLA (Ours) | $\textbf{100.0}_{\pm 0.0}$ | $87.93_{\pm 0.05}$ | $\textbf{76.15}_{\pm 0.04}$ | $\textbf{9.95}_{\pm 1.21}$ | $\textbf{0.040}_{\pm 0.042}$ |

Despite its simplicity and without accessing the forget set during the unlearning process, COLA achieves state-of-the-art performance on the IDI, ensuring effective removal of feature-level information.
It also shows superior performance on existing metrics for the class-wise forgetting task on CIFAR-10, CIFAR-100, and ImageNet-1K, as detailed in Table [1](#S5.T1) and [2](#S5.T2).

While Zhang et al. [] apply contrastive loss during the unlearning process,
they focus on explicitly dispersing features from $\mathcal{D}_{f}$.
Additionally, Kurmanji et al. [] employ a two-step SCRUB method which requires multiple iterations and sensitive hyperparameters.
In contrast, COLA uses a simple loss term and avoids iterative steps, facilitating straightforward and effortless implementation.

Although COLA induces feature collapse in $\mathcal{D}_{f}$ through catastrophic forgetting, COLA+ explicitly accelerates this by using the full dataset $\mathcal{D}$.
Specifically, COLA+ improves the supervised contrastive loss used in the collapse phase by incorporating data from both the retain set $\mathcal{D}_{r}$ and the forget set $\mathcal{D}_{f}$, where the second-highest predicted logit is assigned as the label to forgetting samples, promoting their dispersion.
This causes the features of the forget set to collapse directly into the retain set clusters.
The subsequent align phase then optimizes the entire model using standard cross-entropy loss solely on $\mathcal{D}_{r}$, consistent with the original COLA.
In particular, while COLA is tailored for class-wise forgetting scenarios, COLA+ excels in random data forgetting tasks, as validated by our experimental results in Appendix [C.3](#A3.SS3), demonstrating its empirical benefit.

## 6 Conclusion

We highlight the shortcomings of the current evaluation criteria for machine unlearning through comprehensive empirical studies.
The typical approach of assessing unlearned models based on their output can be misleading as it overlooks the information retained within the intermediate layers of DNNs.
To address this, we propose information difference index (IDI) metric through an information theoretic perspective and the contrastive-based COLA framework for feature-level unlearning.
Through various experiments, we demonstrate the validity and robustness of the new metric,
as well as the effectiveness of the proposed framework in achieving comprehensive unlearning.

## Appendix A Random Data Forgetting

Another scenario in machine unlearning (MU) is random data forgetting, which involves forgetting a randomly selected subset of data across multiple classes. This differs from the class-wise forgetting task, which aims to forget entire data from single or multiple classes.

### A.1 Information Difference Index for
Random Data Forgetting

To calculate the information difference index (IDI) for class-wise forgetting, we employ a binary label $Y$ to determine whether a sample belongs to the retain or forget set.
However, this approach is inadequate for random data forgetting, where samples span multiple classes and a minor fraction of each class is targeted for forgetting.
As a result, no single class is completely removed.
To address this, we transform the binary label $Y$ into a multiclass label $Y_{C}$, which reflects the ground-truth class label of each sample.
Consequently, we define the IDI for random data forgetting as follows:

$$ $\mathbf{IDI}_{random}(\mathbf{\theta_{u}})=\frac{\mathbf{ID}_{random}(\mathbf{\theta_{u}})}{\mathbf{ID}_{random}(\mathbf{\theta_{0}})},$ (4) $$

where $\mathbf{ID}_{random}(\mathbf{\theta_{u}})=\sum_{\ell=1}^{L}\bigl{(}I(\mathbf{Z}^{(\mathbf{u})}_{\ell};Y_{C})-I(\mathbf{Z}^{(\mathbf{r})}_{\ell};Y_{C})\bigr{)}$.
Unlike the ID computed in a class-wise forgetting, $\mathbf{ID}_{random}(\cdot)$ utilizes only the forget set $\mathcal{D}_{f}$.
Intuitively, we expect the mutual information $I(\mathbf{Z}^{(\mathbf{o})};Y_{C})$ to be higher than $I(\mathbf{Z}^{(\mathbf{r})};Y_{C})$ because Original explicitly learned the relationship between the forget samples and their ground truth labels, while Retrain did not.
Although the labels have transitioned from binary to multiple classes, the function $f_{\nu_{\ell}}$ remains unchanged.
For $g_{\eta_{\ell}}$, it now employs $C$ numbers of vectors, where $C$ represents the total number of data classes.

### A.2 COLA+

The core idea behind COLA is to induce catastrophic forgetting within the model’s encoder in the collapse phase, making the influence of the forget set vanish implicitly.
This approach is effective for class-wise forgetting tasks, where the forget set includes distinct classes.
However, it may be less effective for random data forgetting, where the forget set and retain set samples generally share the same classes and are not easily distinguishable.
To address this, we aim to explicitly remove the information of the forget set through pseudo-labeling.
This variant, called COLA+, assigns the second-highest predicted label to the forget set samples before unlearning with supervised contrastive loss [].
This pseudo-labeling effectively collapsing the forget set features into the retain set clusters, while reduces the confusion of the knowledge of the retain set.
The experiment results of COLA+ in the random data forgetting task are presented in Appendix [C.3](#A3.SS3).

## Appendix B Experiment Details

### B.1 Datasets and Models

We conduct image classification experiments utilizing well-established datasets and models.
The datasets include CIFAR-10, CIFAR-100 [], and ImageNet-1K []; and the models are ResNet-18, ResNet-50 [], and Vision Transformer (ViT) [].
CIFAR-10 and CIFAR-100 each comprise 50,000 training images distributed across 10 and 100 classes, respectively, each with an original resolution of 32 x 32 pixels.
In our experiments, we resize the images in ImageNet-1K, which consists of 1,281,167 training images across 1,000 classes, to 224 x 224 pixels.
Similarly, for the ViT experiments, we resize CIFAR images to 224 x 224 pixels to accommodate the architecture’s requirements.
Throughout the training process, including pretraining and unlearning phases, we employ basic data augmentation techniques such as random cropping and random horizontal flipping.

### B.2 Pretraining Settings

To perform unlearning, we require two models: Original, trained on the entire dataset $\mathcal{D}$, and Retrain, trained on the retain set $\mathcal{D}_{r}$. Original initializes the unlearning models, while Retrain evaluates them. Table [3](#A2.T3) summarizes the training configurations for each dataset and model combination. We train ResNet models from scratch and initialize ViT models with ImageNet-21K pretrained weights. For training on ImageNet-1K, we follow the configurations provided by Pytorch(^1^11https://github.com/pytorch/examples/tree/main/imagenet).

**Table 3: Training configuration for Original and Retrain.**
| Settings | CIFAR-10 / CIFAR-100 | ImageNet-1K |  |  |
| --- | --- | --- | --- | --- |
| Resnet-18 / Resnet-50 | ViT | ResNet-50 | ViT |  |
| Epochs | 300 | 3 | 90 | 30 |
| Batch Size | 128 | 256 | 512 |  |
| LR | 0.1 | 0.00002 | 0.1 | 0.02 |
| Optimizer | SGD |  |  |  |
| Momentum | 0.9 |  |  |  |
| L2 regularization | 0.0005 | 0 |  |  |
| Scheduler | CosineAnnealing |  |  |  |

### B.3 Unlearning Settings

We aim to follow the hyperparameters provided by the original papers.
However, many hyperparameters are missing since most existing works do not experiment with large-scale datasets and models.
Additionally, some values from the original papers result in poor performance, likely due to different experiment settings, as most previous work performed unlearning without any data augmentation, unlike our experiments.
Therefore, we conduct thorough hyperparameter searches for each baseline.
The detailed hyperparameters of each baseline, including our method COLA and COLA+, are shown in Table [4](#A6.T4) and Table [5](#A6.T5).
We use the same optimizer and batch size from the original papers and focus on finding the best epoch number and learning rate in terms of unlearning accuracy (UA) and testing accuracy (TA).
Note that we implement gradient ascent (GA) from SCRUB [] (referred to as ’NegGrad+’) due to its strong performance.

### B.4 IDI Settings

To derive IDI from features, it is necessary to train the critic functions $f_{\nu_{\ell}}$ and $g_{\eta_{\ell}}$, as referenced in Section [4](#S4).
For the training of $g_{\eta_{\ell}}$, a learning rate of $5\cdot 10^{-4}$ is applied in all architectures and datasets.
Meanwhile, for $f_{\nu_{\ell}}$, the learning rates are set at $2\cdot 10^{-5}$ for CIFAR10 ResNet-18, $2\cdot 10^{-6}$ for ViT ImageNet-1K, and $1\cdot 10^{-5}$ for the remaining architectures of the data set.

To get IDI, we analyzed the outputs from the layers of different models.
Specifically, we evaluated the last two bottleneck outputs for ResNet18 and the final three for ResNet50.
For Vision Transformer (ViT), we examined the outputs of the final three transformer encoder blocks.
Note that these selections of layers is based on the observation that the information differences of outputs from the initial layers of both original and retrained models are similar.

### B.5 COLA and COLA+ Pseudo Code

Figure: Figure 9: Illustration of the collapse phase of COLA. Features (post-encoder, pre-head) from forget set $\mathcal{D}_{f}$ are represented in red, while features from retain set $\mathcal{D}_{r}$ are represented in green and blue. The figure shows a class-wise forgetting task. Best viewed in color.
Refer to caption: /html/2405.17878/assets/x14.png

Algorithm [1](#alg1) shows the pseudo code of our two-step framework COLA. Only using the retain set $\mathcal{D}_{r}$, in the collapse phase (see Figure [9](#A2.F9)), We first train the encoder of the model using supervised contrastive loss [] as follows:

$$ $\text{SupConLoss}(b,\mathbf{\theta_{enc}})=\frac{1}{|b|}\sum_{i}\frac{1}{|P(i)|}\sum_{p\in P(i)}-\log\frac{\exp(\mathbf{z}_{i}\cdot\mathbf{z}_{p}/\tau)}{\sum_{a\in A(i)}\exp(\mathbf{z}_{i}\cdot\mathbf{z}_{a}/\tau)},$ (5) $$

where $P(i)$ is the set of indices of positive samples sharing the same label as sample $i$, $A(i)$ is the set of all indices excluding sample $i$, $\tau$ is a temperature, and ${z}_{i}=F(x_{i};\mathbf{\theta_{enc}})$, the output feature of the model encoder.
Then we train the whole network using cross-entropy loss in the align phase.
COLA+ additionally utilizes forget set $\mathcal{D}_{f}$ in the collapse phase, where the label of forget samples is changed to the class label closest to the original label, determined by the logit output of the head of Original.
Its pseudo code is presented in Algorithm [2](#alg2).

Figure: Algorithm 1 Pseudo Code of COLA

Figure: Algorithm 2 Pseudo Code of COLA+

### B.6 Other Metrics Settings

#### UA, RA, TA

We compute accuracy as follows:

$$ $\mathbf{\text{Acc}_{\mathcal{D}}}(\mathbf{\theta})=\frac{1}{|\mathcal{D}|}\sum_{(x,y)\in\mathcal{D}}1[argmax(f(x;\mathbf{\theta}))=y].$ (6) $$

Then, unlearning accuracy (UA), which quantifies the model’s ability to forget specific data, is defined as $UA(\mathbf{\theta_{u}})=1-\text{Acc}_{\mathcal{D}_{f}}(\mathbf{\theta_{u}})$.
Remaining accuracy (RA), assessing retention of information in $\mathcal{D}_{r}$, is defined as $RA(\mathbf{\theta_{u}})=\text{Acc}_{\mathcal{D}_{r}}(\mathbf{\theta_{u}})$, and testing accuracy (TA), evaluating generalization to unseen data, is defined as $TA(\mathbf{\theta_{u}})=\text{Acc}_{\mathcal{D}_{test}}(\mathbf{\theta_{u}})$.

#### MIA

Membership inference attack (MIA) [, ] evaluates whether a specific data record was part of a model’s training set by inspecting the model’s output.
A low success rate in predicting the membership of samples in the forget set indicates that the influence of this data has been effectively removed from the model.
MIA is measured through the following steps. First, we collect balanced outputs from the model for data from the retain set $\mathcal{D}_{r}$ and the test set.
Next, we train the MIA predictor, which is a simple SVM model.
The trained MIA predictor is then used to infer the membership of the model outputs of samples from the forget set $\mathcal{D}_{f}$.
There are two widely adopted variants of MIA in the context of model unlearning.
C-MIA is confidence-based MIA, where the predicted probability of the true class is used as the output of the model [, ].
E-MIA is entropy-based MIA, where the MIA predictor infers membership by inspecting the entropy of the model outputs, calculated as $H(x)=-\sum_{i}\mathbf{p}_{i}(x)\cdot\log{\mathbf{p}_{i}(x)}$  [, ]. We mainly report E-MIA due to its wider variations across the baselines compared to C-MIA. Note that our head distillation (HD) achieves similar results in both E-MIA and C-MIA.

#### JSD

Jensen-Shannon divergence (JSD) is presented in Bad-T []. It measures the distance between the output distributions of the unlearned model and Retrain.
JSD is measured as follows:

$$ $\mathbf{\text{JSD}_{\mathcal{D}}}(\mathbf{\theta_{u}},\mathbf{\theta_{r}})=0.5\cdot KL(f(x;\theta_{u})\mid\mid m)+0.5\cdot KL(f(x;\theta_{r})\mid\mid m),$ (7) $$

where $KL(\cdot)$ is Kullback-Leibler divergence, $x$ is data from $\mathcal{D}$, and m = $\frac{f(x;\mathbf{\theta_{u}})+f(x;\mathbf{\theta_{r}})}{2}$.
A smaller distance means better unlearning as the unlearned model better mimics Retrain.

#### RTE

Runtime efficiency (RTE) measures the time that an algorithm spends to complete the unlearning, where smaller RTE indicates more efficient unlearning [, , ]. Since it measures the experiment wall-clock time, it has high variance depending on the experiment environment.

### B.7 System specification

For fair comparison, all experiments are executed in Python 3.10, on an Ubuntu 18.04 machine with 72 CPU cores, 4 Nvidia RTX A6000 GPUs and 512GB memory.

## Appendix C Additional Unlearning Results

In this section, we provide the full experiment results on various machine unlearning settings, extending the results in Section [4](#S4) and Section [5](#S5),

### C.1 Single-Class Forgetting Results

#### CIFAR-10 with Various Architectures

Table [6](#A6.T6) shows the full experiment results of the single-class forgetting experiments from Table [1](#S5.T1) on CIFAR-10 using different models.
This extended table includes Jensen-Shannon divergence (JSD) and the unlearning results with ResNet-50 and ViT.
Although many baselines show promising results on output based evaluation metrics, they generally exhibit poor feature-level unlearning.
On the contrary, COLA not only outperforms existing baselines on the current metrics but also significantly excels in IDI, demonstrating its effectiveness in removing the influence of the forget set within the encoder of the model.

#### CIFAR-100 with Various Architectures

As demonstrated in Table [7](#A6.T7), we also compare COLA with other baselines on CIFAR-100.
The results consistently highlight the difficulty of comparing and validating the efficacy of each unlearning method using existing output-based metrics.
With the help of IDI, it is clear that COLA shows robustness in model unlearning on datasets with a large number of classes across various model architectures.
Although SCRUB has achieved IDI near 0 for the CIFAR-10 ResNet-18 experiment, it shows significant variations in feature-level unlearning across different datasets and architectures.

### C.2 Multi-Class Forgetting Results

#### Multi-Class Forgetting on CIFAR-10 and CIFAR-100

Table [8](#A6.T8) presents the results of multi-class forgetting experiments on CIFAR-10 and CIFAR-100 using ResNet-18, which involves erasing the information of more than one class in the training set.
We remove two classes from CIFAR-10 and five and twenty classes from CIFAR-100.
Notably, all baselines exhibit higher IDI values as the number of forgetting class increases, demonstrating that the tendency to modify the head of the model strengthens with the difficulty of the unlearning tasks.
In contrast, COLA shows remarkable effectiveness, achieving metric values closely aligned with Retrain.
Specifically, COLA consistently achieves the lowest IDI values among the evaluated methods, indicating the necessity of the collapse phase for effective feature-level unlearning no matter the number of class to forget.

#### Multi-Class Forgetting on ImageNet-1K

We conduct 5-class unlearning on ImageNet-1K using ResNet-50 and ViT.
Table [9](#A6.T9) provides the complete results of Table [2](#S5.T2), including all evaluation metrics and outcomes on the ViT architecture.
However, it is important to note that IDI alone should not be used to assess unlearned models, as a low IDI might indicate a loss of overall information, including that from the retain set, which should be maintained at the same level as Original.
This issue is evident in the RA, TA, and IDI of EU-10 and CF-10 in Table [9](#A6.T9).
In contrast, COLA consistently achieves IDI near 0 while maintaining accuracy measurements comparable to Retrain, demonstrating the scalability of our framework to the large-scale datasets.

### C.3 Random Data Forgetting Results

Table [10](#A6.T10) presents the results of the random data forgetting task conducted on ResNet-18.
For CIFAR-10 and CIFAR-100 datasets, we randomly selected 500 and 50 forget samples per class, respectively.
This table demonstrates the adaptability of various methods in different unlearning tasks.
Particularly, COLA fails to remove the influence of the forget set in the encoder, as indicated by the UA and IDI metrics. In contrast, COLA+, which incorporates pseudo-labeling, successfully eliminates the influence of the forgetting data while maintaining accuracy similar to Retrain.

## Appendix D Additional Analysis

### D.1 Mutual Information Curves

Figure [13](#A6.F13) illustrates the estimated mutual information $I(\mathbf{Z}_{\ell};Y)$ of the features from the $\ell$-th layer $\mathbf{Z}_{\ell}$ and the binary label $Y$, computed by the InfoNCE loss.
We compute mutual information (MI) for all layers from the ResNet encoder and last five layers from the ViT encoder based single-class forgetting retain and forget sets. The upper bound of MI is given by the entropy $H(Y)\geq I(\mathbf{Z}_{\ell};Y)=H(Y)-H(Y\mid\mathbf{Z}_{\ell})$.
The estimated MI values fall within the range of the upper bound and the lower bound (which is 0), validating the use of InfoNCE for MI estimation.
Notably, all MI curves consistently show a larger difference between Original and Retrain in the later layers of the encoder across various datasets and architectures, while differences are minimal in the earlier layers.
These observations underscore the validity of computing the information difference (ID) for the last few layers to quantify unlearning.

### D.2 Mutual Information and Accuracy

We extend the experiment to measure the accuracy of the intermediate features of the model’s encoder.
Similar to measuring MI using the InfoNCE loss, we freeze the layers up to the $\ell$-th layer of the encoder and train the remaining encoder layers and an additional head using cross-entropy loss.
The additional head perform binary classification to determine whether the input belongs to the retain or forget set.

Figure [14](#A6.F14) shows the train accuracy curves on the CIFAR-10 single-class forgetting dataset with ResNet-18.
For Original encoder, the trained model readily classifies the retain and forget sets.
However, for Retrain encoder, the model fails to classifies all samples at the last two layers, with the accuracy dropping more in the later layer.
These curves correspond to the those from Figure [13](#A6.F13), indicating that the estimated MI accurately reflects the model’s knowledge of the retain and forget sets.
In addition, the small accuracy gap between Original and Retrain provides the necessity of MI for accurate residual information quantification.

### D.3 IDI and t-SNE Relationship

Figure [10](#A6.F10) presents t-SNE plots illustrating the intermediate features and corresponding IDI measurements of MU baselines on the single-class unlearning on CIFAR-10 with ResNet-18.
In the plots, the purple points represent the forgetting class.
A high IDI corresponds to better clustering and similarity among features of the forgetting class, as seen in (l) SALUN and (f) Bad-T, which show inadequate unlearning performance.
These examinations show the high relationship between IDI and the residual information of forget set.
Additionally, the IDI metric reveals instances of over-unlearning, where the forgetting class becomes excessively dispersed, as demonstrated in (i) EU-10.
Among the evaluated methods, (n) COLA has the closest IDI to Retrain, suggesting its high efficacy in achieving the desired removal of the forget set influence in the intermediate layers of the model.
This trend is also visible in ResNet-50 (see Figure [11](#A6.F11)) and ViT (see Figure [12](#A6.F12)).

Furthermore, we confirm that IDI for the random data forgetting correctly captures the encoder‘s information, similar to IDI for the class-wise forgetting.
In Figure [15](#A6.F15), the t-SNE plots of forget sample features for two baselines with the same unlearning accuracy (UA) – Bad-T and $\mathbf{\ell_{1}}$-sparse – and their IDI values in the random data forgetting task is visualized.
Comparing them, IDI successfully reflects the residual information in the features, as the features of Bad-T form more compact clusters than those of $\mathbf{\ell_{1}}$-sparse, indicating more influence of the forget set remains in Bad-T.
IDI the for random data forgetting captures the hidden information that cannot be noticed from existing metrics, which may suggest that both methods unlearn similarly due to their same forget accuracy.

## Appendix E Broader Impact

Our work on improving machine unlearning focuses on foundational research aimed at enhancing privacy and data removal. However, there is a potential risk that our methodology could be misused to evade data retention policies or obscure accountability. Despite this possibility, it is unlikely that our work will introduce new harmful practices beyond what existing unlearning methods already permit, as we are not introducing new capabilities. Therefore, while there might be concerns related to privacy, security, and fairness, our work does not pose a greater risk compared to other foundational research in machine unlearning.

## Appendix F Limitations

Our methodology accomplishes its main objective, but there are a few limitations we point out.
Although our IDI successfully investigates hidden information in intermediate features,
its computation requires multiple training runs, which can be computationally intensive.
For instance, The computation of IDI for ResNet-50 on the CIFAR-100 dataset takes approximately 40-50 minutes.
However, one can mitigate this by computing mutual information for only the last few layers,
as the early stages of the encoder are largely similar for both the Retrain and Original models.
Thus, this approach requires fine-tuning only the later layers, reducing the overall computational burden.
Additionally, by adjusting the forget-to-retain ratio, it is possible to improve efficiency and possibly decrease the processing time to merely 3-4 minutes.

**Table 4: Hyperparameters of baselines for class-wise forgetting. Retain Batch Size is the batch size of retain set $\mathcal{D}_{r}$ and Forget Batch Size is the batch size of forget set $\mathcal{D}_{f}$. Baselines without Forget Batch Size imply that they do not use forget set $\mathcal{D}_{f}$. Bad-T uses the entire dataset $\mathcal{D}$, so there is no separation of retain and forget of Batch Size. SCRUB has separate epochs for retain set and forget set, which is visualized as Retain Epochs (Forget Epochs). For COLA, A + B Epochs indicates collapse epochs A and align epochs B.**
| Class-wise Forgetting |  |  |  |
| --- | --- | --- | --- |
| Settings | CIFAR-10 | CIFAR-100 | ImageNet-1K |
| ResNet-18 / ResNet-50 / ViT | ResNet-50 / ViT |  |  |
| FT | 25 Epochs, Adam | 3/4 Epochs, Adam |  |
| LR $10^{-5}$/$10^{-5}$/$10^{-4}$ | LR $10^{-5}$ |  |  |
| Retain Batch Size 64 | Retain Batch Size 128 |  |  |
| RL | 7 / 7 / 10 Epochs, SGD | 7 / 7 / 10 Epochs, SGD | 3 Epochs, SGD |
| LR $10^{-5}$ / $2\cdot 10^{-5}$ / $10^{-3}$ | LR $2\cdot 10^{-5}$ / $10^{-4}$ / $10^{-4}$ | LR $10^{-3}$ / $10^{-4}$ |  |
| Retain Batch Size 64 | Retain Batch Size 64 | Retain Batch Size 128 |  |
| Forget Batch Size 16 | Forget Batch Size 16 | Forget Batch Size 16 |  |
| GA | 10 Epochs, SGD | 10 Epochs, SGD | 3 Epochs, SGD |
| LR $2\cdot 10^{-3}$ / $2\cdot 10^{-3}$ / $5\cdot 10^{-3}$ | LR $9\cdot 10^{-4}$ / $9\cdot 10^{-4}$ / $5\cdot 10^{-3}$ | LR $2\cdot 10^{-3}$ / $10^{-3}$ |  |
| Retain Batch Size 64 | Retain Batch Size 64 | Retain Batch Size 128 |  |
| Forget Batch Size 16 | Forget Batch Size 16 | Forget Batch Size 16 |  |
| Bad-T | 10 Epochs, Adam | 3 Epochs, Adam |  |
| LR $10^{-5}$ | LR $10^{-5}$ |  |  |
| Batch Size 256 | Batch Size 256 |  |  |
| EU-5 /<br>EU-10 | 14 Epochs, SGD | 5 Epochs, SGD |  |
| LR $10^{-2}$ | LR $5\cdot 10^{-3}$ |  |  |
| Retain Batch Size 64 | Retain Batch Size 128 |  |  |
| CF-5 /<br>CF-10 | 14 / 14 / 18 Epochs, SGD | 2 Epochs, SGD |  |
| LR $10^{-2}$ / $10^{-2}$ / $3\cdot 10^{-2}$ | LR $5\cdot 10^{-3}$ |  |  |
| Retain Batch Size 64 | Retain Batch Size 128 |  |  |
| SCRUB | 3(2) Epochs, SGD | 3(2) Epochs, SGD | 2(2) Epochs, SGD |
| LR $5\cdot 10^{-4}$ / $5\cdot 10^{-4}$ / $10^{-4}$ | LR $5\cdot 10^{-4}$ | LR $5\cdot 10^{-4}$ / $10^{-4}$ |  |
| Retain Batch Size 64 | Retain Batch Size 128 | Retain Batch Size 128 |  |
| Forget Batch Size 256 / 256 / 64 | Forget Batch Size 8 | Forget Batch Size 256 |  |
| SALUN | 10 Epochs, SGD | 15 Epochs, SGD | 5/2 Epochs, SGD |
| LR $5\cdot 10^{-4}$ / $10^{-3}$ / $10^{-3}$ | LR $10^{-3}$ | LR $10^{-3}$ |  |
| Retain Batch Size 64 | Retain Batch Size 64 | Retain Batch Size 128 |  |
| Forget Batch Size 16 | Forget Batch Size 16 | Forget Batch Size 16 |  |
| $\mathbf{\ell_{1}}$-sparse | 10 Epochs, SGD | 10 Epochs, SGD | 5 Epochs, SGD |
| LR $2\cdot 10^{-4}$ / $2\cdot 10^{-4}$ / $9\cdot 10^{-4}$ | LR $2\cdot 10^{-4}$ / $2\cdot 10^{-4}$ / $5\cdot 10^{-4}$ | LR $9\cdot 10^{-4}$ |  |
| Retain Batch Size 64 | Retain Batch Size 64 | Retain Batch Size 128 |  |
| COLA | 10+10 Epochs, Adam | 10+10 Epochs, Adam | 1+2 Epochs, Adam |
| Contrast LR $2\cdot 10^{-4}$ / $2\cdot 10^{-4}$ / $1.5\cdot 10^{-4}$ | Contrast LR $5\cdot 10^{-4}$ / $5\cdot 10^{-4}$ / $5\cdot 10^{-4}$ | Contrast LR $2\cdot 10^{-5}$ / $5\cdot 10^{-5}$ |  |
| Finetune LR $5\cdot 10^{-6}$ / $10^{-5}$ / $5\cdot 10^{-5}$ | Finetune LR $5\cdot 10^{-6}$ / $10^{-5}$ / $5\cdot 10^{-5}$ | Finetune LR $1\cdot 10^{-5}$ / $5\cdot 10^{-5}$ |  |
| Retain Batch Size 64 | Retain Batch Size 256 | Retain Batch Size 256 |  |

**Table 5: Hyperparameters of baselines for random data forgetting. Retain Batch Size is the batch size of retain set $\mathcal{D}_{r}$ and Forget Batch Size is the batch size of forget set $\mathcal{D}_{f}$. Baselines without Forget Batch Size imply that they do not use forget set $\mathcal{D}_{f}$. Bad-T uses the entire dataset $\mathcal{D}$, so there is no separation of retain and forget of Batch Size. SCRUB uses separate epochs for retain set and forget set, which is visualized as Retain Epochs (Forget Epochs). For COLA+, A + B Epochs indicates collapse epochs A and align epochs B.**
| Random Data Forgetting |  |  |
| --- | --- | --- |
| Settings | CIFAR-10 | CIFAR-100 |
| ResNet-18 |  |  |
| FT | 25 Epochs, Adam |  |
| LR $10^{-4}$ | LR $2\cdot 10^{-4}$ |  |
| Retain Batch Size 64 |  |  |
| RL | 7 Epochs, SGD |  |
| LR $10^{-3}$ | LR $5\cdot 10^{-4}$ |  |
| Retain Batch Size 64 |  |  |
| Forget Batch Size 16 |  |  |
| GA | 10 Epochs, SGD | 10 Epochs, SGD |
| LR $2.5\cdot 10^{-3}$ | LR $1\cdot 10^{-3}$ |  |
| Retain Batch Size 64 |  |  |
| Forget Batch Size 16 |  |  |
| Bad-T | 10 Epochs, Adam |  |
| LR $1\cdot 10^{-5}$ |  |  |
| Batch Size 256 |  |  |
| EU-5 /<br>EU-10 | 14 Epochs, SGD |  |
| LR $10^{-1}$ | LR $5\cdot 10^{-2}$ |  |
| Retain Batch Size 64 |  |  |
| CF-5 /<br>CF-10 | 14 Epochs, SGD |  |
| LR $10^{-1}$ | LR $5\cdot 10^{-2}$ |  |
| Retain Batch Size 64 |  |  |
| SCRUB | 5(5) Epochs, SGD |  |
| LR $2.5\cdot 10^{-5}$ | LR $5.4\cdot 10^{-4}$ |  |
| Retain Batch Size 16 |  |  |
| Forget Batch Size 64 |  |  |
| SALUN | 10 Epochs, SGD | 15 Epochs, SGD |
| LR $8.3\cdot 10^{-4}$ | LR $5\cdot 10^{-4}$ |  |
| Retain Batch Size 64 |  |  |
| Forget Batch Size 16 |  |  |
| $\mathbf{\ell_{1}}$-sparse | 10 Epochs, SGD |  |
| LR $4\cdot 10^{-4}$ | LR $3\cdot 10^{-4}$ |  |
| Retain Batch Size 64 | Retain Batch Size 64 |  |
| COLA /<br>COLA+ | 10+10 Epochs, Adam | 10+10 Epochs, Adam |
| Contrast LR $2\cdot 10^{-4}$ | Contrast LR $2.5\cdot 10^{-4}$ |  |
| Finetune LR $1\cdot 10^{-4}$ | Finetune LR $2\cdot 10^{-5}$ |  |
| Retain Batch Size 64/32 | Retain Batch Size 256/64 |  |
| Forget Batch Size -/64 | Forget Batch Size-/192 |  |

**Table 6: Single-class forgetting result on CIFAR-10 dataset across different model architectures. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
| CIFAR-10 - ResNet-18 |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $100.0$ | $95.46$ | $91.50$ | $3.21$ | $1.000$ | $170.32$ |
| Retrain | $100.0$ | $100.0$ | $95.64$ | $10.64$ | $0.0$ | $0.0$ | $154.56$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.12_{\pm 0.09}$ | $0.17_{\pm 0.05}$ | $0.57_{\pm 0.03}$ | $0.671_{\pm 0.008}$ | $6.44_{\pm 0.07}$ |
| RL | $99.93_{\pm 0.01}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.12_{\pm 0.09}$ | $0.0_{\pm 0.0}$ | $0.79_{\pm 0.01}$ | $0.830_{\pm 0.005}$ | $3.09_{\pm 0.03}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $99.06_{\pm 0.25}$ | $93.10_{\pm 0.50}$ | $25.37_{\pm 3.24}$ | $0.59_{\pm 0.05}$ | $0.334_{\pm 0.014}$ | $4.00_{\pm 0.08}$ |
| Bad-T | $99.90_{\pm 0.14}$ | $99.99_{\pm 0.0}$ | $94.99_{\pm 0.12}$ | $68.17_{\pm 42.80}$ | $3.69_{\pm 0.85}$ | $1.014_{\pm 0.004}$ | $4.64_{\pm 0.05}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.25_{\pm 0.02}$ | $0.06_{\pm 0.03}$ | $0.53_{\pm 0.02}$ | $0.528_{\pm 0.005}$ | $\textbf{1.54}_{\pm 0.00}$ |
| CF-5 | $98.13_{\pm 1.39}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{95.54}_{\pm 0.09}$ | $0.0_{\pm 0.0}$ | $0.56_{\pm 0.04}$ | $0.675_{\pm 0.027}$ | $\underline{1.57}_{\pm 0.03}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.50_{\pm 0.02}$ | $93.61_{\pm 0.08}$ | $15.24_{\pm 1.08}$ | $\textbf{0.40}_{\pm 0.01}$ | $-0.349_{\pm 0.019}$ | $2.42_{\pm 0.11}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.98_{\pm 0.0}$ | $94.95_{\pm 0.05}$ | $\textbf{11.61}_{\pm 0.91}$ | $\underline{0.41}_{\pm 0.01}$ | $-0.060_{\pm 0.017}$ | $2.31_{\pm 0.03}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.37_{\pm 0.04}$ | $19.73_{\pm 1.92}$ | $0.47_{\pm 0.01}$ | $\underline{-0.056}_{\pm 0.008}$ | $3.49_{\pm 0.02}$ |
| SALUN | $99.99_{\pm 0.01}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.42_{\pm 0.12}$ | $0.01_{\pm 0.01}$ | $0.73_{\pm 0.04}$ | $0.936_{\pm 0.012}$ | $3.54_{\pm 0.11}$ |
| $\mathbf{\ell_{1}}$-sparse | $\textbf{100.0}_{\pm 0.0}$ | $99.93_{\pm 0.02}$ | $94.90_{\pm 0.10}$ | $1.56_{\pm 0.09}$ | $0.47_{\pm 0.03}$ | $0.293_{\pm 0.012}$ | $2.96_{\pm 0.03}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.00}$ | $\textbf{95.55}_{\pm 0.06}$ | $\underline{12.64}_{\pm 0.92}$ | $0.44_{\pm 0.04}$ | $\textbf{0.010}_{\pm 0.006}$ | $4.91_{\pm 0.04}$ |
| CIFAR-10 - ResNet-50 |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $100.0$ | $95.42$ | $95.58$ | $4.11$ | $1.000$ | $341.86$ |
| Retrain | $100.0$ | $100.0$ | $95.49$ | $14.92$ | $0.0$ | $0.0$ | $312.24$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $99.99_{\pm 0.0}$ | $95.28_{\pm 0.11}$ | $2.17_{\pm 1.28}$ | $0.73_{\pm 0.02}$ | $0.607_{\pm 0.009}$ | $14.50_{\pm 0.34}$ |
| RL | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.56_{\pm 0.03}$ | $0.0_{\pm 0.0}$ | $0.99_{\pm 0.02}$ | $0.804_{\pm 0.006}$ | $6.26_{\pm 0.04}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $98.06_{\pm 0.34}$ | $92.07_{\pm 0.63}$ | $20.56_{\pm 3.87}$ | $0.66_{\pm 0.06}$ | $0.334_{\pm 0.023}$ | $8.69_{\pm 0.03}$ |
| Bad-T | $\textbf{100.0}_{\pm 0.0}$ | $99.94_{\pm 0.04}$ | $94.74_{\pm 0.24}$ | $49.95_{\pm 40.74}$ | $3.02_{\pm 0.64}$ | $1.153_{\pm 0.026}$ | $10.19_{\pm 0.32}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.59_{\pm 0.08}$ | $0.0_{\pm 0.0}$ | $0.78_{\pm 0.08}$ | $1.047_{\pm 0.005}$ | $\underline{4.86}_{\pm 0.43}$ |
| CF-5 | $17.84_{\pm 0.93}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.64_{\pm 0.11}$ | $0.0_{\pm 0.0}$ | $1.43_{\pm 0.04}$ | $0.906_{\pm 0.002}$ | $\textbf{4.84}_{\pm 0.10}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{95.51}_{\pm 0.12}$ | $0.17_{\pm 0.05}$ | $0.65_{\pm 0.02}$ | $0.757_{\pm 0.011}$ | $6.92_{\pm 0.02}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{95.49}_{\pm 0.13}$ | $0.07_{\pm 0.03}$ | $0.67_{\pm 0.08}$ | $0.579_{\pm 0.009}$ | $7.09_{\pm 0.02}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.23_{\pm 0.20}$ | $\underline{18.19}_{\pm 0.10}$ | $0.59_{\pm 0.01}$ | $\underline{0.067}_{\pm 0.020}$ | $8.69_{\pm 0.03}$ |
| SALUN | $\textbf{100.0}_{\pm 0.0}$ | $99.67_{\pm 0.17}$ | $93.90_{\pm 0.48}$ | $1.58_{\pm 0.98}$ | $0.67_{\pm 0.03}$ | $0.832_{\pm 0.027}$ | $11.00_{\pm 0.06}$ |
| $\mathbf{\ell_{1}}$-sparse | $\textbf{100.0}_{\pm 0.0}$ | $99.88_{\pm 0.06}$ | $94.49_{\pm 0.29}$ | $4.06_{\pm 0.91}$ | $\textbf{0.47}_{\pm 0.01}$ | $0.184_{\pm 0.023}$ | $12.33_{\pm 0.04}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.99_{\pm 0.0}$ | $95.45_{\pm 0.05}$ | $\textbf{13.69}_{\pm 0.84}$ | $\underline{0.52}_{\pm 0.02}$ | $\textbf{0.019}_{\pm 0.025}$ | $11.98_{\pm 0.03}$ |
| CIFAR-10 - ViT |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.36$ | $99.55$ | $98.40$ | $89.12$ | $3.96$ | $1.000$ | $100.68$ |
| Retrain | $100.0$ | $99.40$ | $97.96$ | $4.96$ | $0.0$ | $0.0$ | $90.96$ |
| FT | $98.10_{\pm 0.24}$ | $99.85_{\pm 0.06}$ | $\textbf{97.58}_{\pm 0.36}$ | $21.14_{\pm 0.92}$ | $0.71_{\pm 0.13}$ | $-0.871_{\pm 0.141}$ | $130.13_{\pm 0.63}$ |
| RL | $97.88_{\pm 2.12}$ | $99.88_{\pm 0.01}$ | $99.01_{\pm 0.02}$ | $0.0_{\pm 0.0}$ | $0.74_{\pm 0.04}$ | $1.052_{\pm 0.011}$ | $65.45_{\pm 0.12}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $99.80_{\pm 0.03}$ | $98.49_{\pm 0.12}$ | $\textbf{4.82}_{\pm 0.98}$ | $0.39_{\pm 0.05}$ | $0.498_{\pm 0.025}$ | $68.32_{\pm 0.80}$ |
| Bad-T | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.55}_{\pm 0.03}$ | $\underline{98.40}_{\pm 0.20}$ | $0.0_{\pm 0.0}$ | $0.84_{\pm 0.06}$ | $0.997_{\pm 0.018}$ | $100.90_{\pm 1.02}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.76_{\pm 0.01}$ | $98.80_{\pm 0.01}$ | $0.30_{\pm 0.01}$ | $0.28_{\pm 0.03}$ | $0.901_{\pm 0.006}$ | $\underline{29.89}_{\pm 0.09}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.76_{\pm 0.0}$ | $98.86_{\pm 0.02}$ | $0.35_{\pm 0.03}$ | $0.26_{\pm 0.01}$ | $0.941_{\pm 0.001}$ | $34.12_{\pm 0.09}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.72_{\pm 0.02}$ | $98.63_{\pm 0.04}$ | $0.64_{\pm 0.02}$ | $\underline{0.23}_{\pm 0.03}$ | $\underline{0.268}_{\pm 0.016}$ | $32.74_{\pm 0.19}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.77_{\pm 0.01}$ | $98.75_{\pm 0.02}$ | $0.64_{\pm 0.04}$ | $\textbf{0.21}_{\pm 0.02}$ | $0.377_{\pm 0.039}$ | $36.79_{\pm 0.15}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $\underline{99.66}_{\pm 0.0}$ | $98.57_{\pm 0.01}$ | $94.74_{\pm 0.26}$ | $3.87_{\pm 0.07}$ | $0.907_{\pm 0.027}$ | $\textbf{22.99}_{\pm 0.24}$ |
| SALUN | $\textbf{100.0}_{\pm 0.0}$ | $99.78_{\pm 0.02}$ | $98.89_{\pm 0.02}$ | $0.01_{\pm 0.01}$ | $0.39_{\pm 0.05}$ | $1.066_{\pm 0.041}$ | $61.37_{\pm 0.10}$ |
| $\mathbf{\ell_{1}}$-sparse | $\textbf{100.0}_{\pm 0.0}$ | $97.48_{\pm 0.27}$ | $95.78_{\pm 0.16}$ | $\underline{3.89}_{\pm 0.79}$ | $0.41_{\pm 0.03}$ | $-0.573_{\pm 0.290}$ | $51.44_{\pm 0.04}$ |
| COLA | $99.44_{\pm 0.02}$ | $100.0_{\pm 0.0}$ | $98.82_{\pm 0.06}$ | $11.90_{\pm 1.36}$ | $0.63_{\pm 0.11}$ | $\textbf{0.032}_{\pm 0.010}$ | $116.01_{\pm 0.96}$ |

**Table 7: Single-class forgetting result on CIFAR-100 dataset across different model architectures. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
| CIFAR-100 - ResNet-18 |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $99.98$ | $78.18$ | $92.80$ | $2.91$ | $1.000$ | $175.08$ |
| Retrain | $100.0$ | $99.96$ | $79.48$ | $2.00$ | $0.0$ | $0.0$ | $171.27$ |
| FT | $\textbf{100.00}_{\pm 0.0}$ | $\textbf{99.97}_{\pm 0.0}$ | $77.49_{\pm 0.14}$ | $0.07_{\pm 0.09}$ | $0.37_{\pm 0.01}$ | $0.610_{\pm 0.022}$ | $9.5_{\pm 0.03}$ |
| RL | $93.80_{\pm 0.75}$ | $99.98_{\pm 0.0}$ | $\underline{77.94}_{\pm 0.10}$ | $0.0_{\pm 0.0}$ | $0.52_{\pm 0.01}$ | $0.467_{\pm 0.010}$ | $3.52_{\pm 0.0}$ |
| GA | $99.93_{\pm 0.09}$ | $96.87_{\pm 0.52}$ | $69.87_{\pm 0.78}$ | $21.40_{\pm 2.04}$ | $1.18_{\pm 0.02}$ | $0.392_{\pm 0.021}$ | $5.32_{\pm 0.01}$ |
| Bad-T | $\textbf{100.0}_{\pm 0.0}$ | $99.98_{\pm 0.0}$ | $77.66_{\pm 0.26}$ | $40.87_{\pm 36.87}$ | $2.53_{\pm 0.44}$ | $1.079_{\pm 0.024}$ | $5.78_{\pm 0.02}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.78_{\pm 0.01}$ | $75.01_{\pm 0.04}$ | $9.33_{\pm 0.75}$ | $0.66_{\pm 0.01}$ | $\underline{0.064}_{\pm 0.037}$ | $\textbf{2.14}_{\pm 0.0}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.97}_{\pm 0.0}$ | $77.30_{\pm 0.28}$ | $\textbf{2.87}_{\pm 0.66}$ | $0.40_{\pm 0.03}$ | $0.388_{\pm 0.010}$ | $\textbf{2.14}_{\pm 0.01}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $91.94_{\pm 0.08}$ | $72.84_{\pm 0.04}$ | $12.67_{\pm 0.47}$ | $0.53_{\pm 0.02}$ | $-0.221_{\pm 0.009}$ | $4.39_{\pm 0.02}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.89_{\pm 0.02}$ | $76.49_{\pm 0.02}$ | $7.07_{\pm 0.84}$ | $0.49_{\pm 0.01}$ | $0.175_{\pm 0.040}$ | $4.29_{\pm 0.04}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $99.98_{\pm 0.0}$ | $\textbf{78.17}_{\pm 0.04}$ | $0.07_{\pm 0.09}$ | $\underline{0.31}_{\pm 0.01}$ | $0.339_{\pm 0.069}$ | $2.27_{\pm 0.02}$ |
| SALUN | $95.73_{\pm 0.85}$ | $99.22_{\pm 0.13}$ | $74.20_{\pm 0.52}$ | $\underline{0.09}_{\pm 0.02}$ | $0.65_{\pm 0.01}$ | $0.529_{\pm 0.022}$ | $4.63_{\pm 0.06}$ |
| $\mathbf{\ell_{1}}$-sparse | $96.93_{\pm 0.19}$ | $98.90_{\pm 0.12}$ | $74.69_{\pm 0.06}$ | $6.60_{\pm 0.43}$ | $0.34_{\pm 0.01}$ | $0.334_{\pm 0.026}$ | $4.55_{\pm 0.01}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.80_{\pm 0.00}$ | $76.48_{\pm 0.11}$ | $9.60_{\pm 1.31}$ | $\textbf{0.26}_{\pm 0.01}$ | $\textbf{-0.037}_{\pm 0.006}$ | $7.51_{\pm 0.02}$ |
| CIFAR-100 - ResNet-50 |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $99.98$ | $79.84$ | $91.60$ | $3.43$ | $1.000$ | $345.54$ |
| Retrain | $100.0$ | $99.97$ | $79.42$ | $3.40$ | $0.0$ | $0.0$ | $338.58$ |
| FT | $99.33_{\pm 0.09}$ | $99.93_{\pm 0.03}$ | $77.71_{\pm 0.18}$ | $0.40_{\pm 0.16}$ | $0.57_{\pm 0.02}$ | $0.618_{\pm 0.018}$ | $16.34_{\pm 0.47}$ |
| RL | $\textbf{100.0}_{\pm 0.0}$ | $99.95_{\pm 0.02}$ | $\underline{79.56}_{\pm 0.04}$ | $0.0_{\pm 0.0}$ | $0.80_{\pm 0.0}$ | $0.649_{\pm 0.013}$ | $8.38_{\pm 0.14}$ |
| GA | $99.60_{\pm 0.43}$ | $98.00_{\pm 0.72}$ | $72.73_{\pm 1.16}$ | $13.33_{\pm 4.43}$ | $0.99_{\pm 0.04}$ | $0.526_{\pm 0.009}$ | $9.50_{\pm 0.54}$ |
| Bad-T | $\textbf{100.0}_{\pm 0.0}$ | $99.90_{\pm 0.10}$ | $77.53_{\pm 1.21}$ | $94.80_{\pm 2.75}$ | $3.98_{\pm 0.25}$ | $0.990_{\pm 0.033}$ | $12.69_{\pm 1.54}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.97}_{\pm 0.01}$ | $78.31_{\pm 0.21}$ | $\underline{1.20}_{\pm 0.99}$ | $0.61_{\pm 0.04}$ | $0.520_{\pm 0.023}$ | $\underline{6.81}_{\pm 0.01}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.97}_{\pm 0.01}$ | $78.98_{\pm 0.16}$ | $0.27_{\pm 0.09}$ | $0.50_{\pm 0.02}$ | $0.575_{\pm 0.016}$ | $6.82_{\pm 0.01}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $98.52_{\pm 0.14}$ | $75.66_{\pm 0.03}$ | $15.00_{\pm 1.45}$ | $0.69_{\pm 0.01}$ | $\underline{0.050}_{\pm 0.004}$ | $7.81_{\pm 0.01}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.95_{\pm 0.01}$ | $78.47_{\pm 0.10}$ | $5.87_{\pm 0.09}$ | $0.50_{\pm 0.02}$ | $0.302_{\pm 0.035}$ | $7.82_{\pm 0.02}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.97}_{\pm 0.0}$ | $79.61_{\pm 0.09}$ | $0.20_{\pm 0.16}$ | $\underline{0.43}_{\pm 0.02}$ | $0.620_{\pm 0.034}$ | $\textbf{4.59}_{\pm 0.13}$ |
| SALUN | $99.73_{\pm 0.38}$ | $99.98_{\pm 0.0}$ | $\textbf{79.51}_{\pm 0.15}$ | $0.0_{\pm 0.0}$ | $0.80_{\pm 0.01}$ | $0.679_{\pm 0.010}$ | $12.83_{\pm 0.87}$ |
| $\mathbf{\ell_{1}}$-sparse | $96.20_{\pm 0.16}$ | $99.42_{\pm 0.06}$ | $76.16_{\pm 0.31}$ | $\textbf{2.60}_{\pm 0.33}$ | $\underline{0.43}_{\pm 0.01}$ | $0.325_{\pm 0.018}$ | $15.78_{\pm 0.05}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.90_{\pm 0.01}$ | $78.59_{\pm 0.28}$ | $10.27_{\pm 0.90}$ | $\textbf{0.42}_{\pm 0.02}$ | $\textbf{0.016}_{\pm 0.031}$ | $16.25_{\pm 0.10}$ |
| CIFAR-100 - ViT |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $7.00$ | $95.85$ | $90.78$ | $69.20$ | $2.71$ | $1.000$ | $102.45$ |
| Retrain | $100.0$ | $95.79$ | $90.58$ | $10.00$ | $0.0$ | $0.0$ | $94.29$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $99.79_{\pm 0.04}$ | $88.69_{\pm 0.11}$ | $14.80_{\pm 2.40}$ | $0.57_{\pm 0.03}$ | $-0.934_{\pm 0.113}$ | $140.61_{\pm 0.25}$ |
| RL | $99.73_{\pm 0.19}$ | $97.05_{\pm 0.02}$ | $92.26_{\pm 0.03}$ | $0.20_{\pm 0.0}$ | $0.84_{\pm 0.01}$ | $1.446_{\pm 0.027}$ | $71.87_{\pm 0.21}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $98.19_{\pm 0.20}$ | $\textbf{90.59}_{\pm 0.21}$ | $17.60_{\pm 4.78}$ | $0.31_{\pm 0.01}$ | $0.587_{\pm 0.109}$ | $75.22_{\pm 0.61}$ |
| Bad-T | $96.20_{\pm 0.09}$ | $\textbf{95.88}_{\pm 0.12}$ | $\underline{90.77}_{\pm 0.10}$ | $0.0_{\pm 0.0}$ | $1.08_{\pm 0.21}$ | $2.004_{\pm 0.023}$ | $97.87_{\pm 0.0}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $97.59_{\pm 0.04}$ | $92.04_{\pm 0.02}$ | $\underline{7.10}_{\pm 0.70}$ | $\underline{0.27}_{\pm 0.01}$ | $1.565_{\pm 0.007}$ | $\underline{31.43}_{\pm 0.01}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $97.83_{\pm 0.02}$ | $92.00_{\pm 0.04}$ | $7.03_{\pm 0.42}$ | $0.267_{\pm 0.01}$ | $1.483_{\pm 0.049}$ | $36.75_{\pm 0.01}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $97.87_{\pm 0.01}$ | $91.45_{\pm 0.07}$ | $13.30_{\pm 1.97}$ | $0.36_{\pm 0.02}$ | $0.849_{\pm 0.123}$ | $34.23_{\pm 0.02}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $97.87_{\pm 0.01}$ | $91.61_{\pm 0.05}$ | $15.80_{\pm 0.80}$ | $0.32_{\pm 0.02}$ | $0.734_{\pm 0.112}$ | $39.12_{\pm 0.0}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.00}$ | $96.95_{\pm 0.03}$ | $92.12_{\pm 0.06}$ | $17.00_{\pm 1.21}$ | $\underline{0.27}_{\pm 0.02}$ | $\underline{0.037}_{\pm 0.036}$ | $\textbf{17.84}_{\pm 0.13}$ |
| SALUN | $99.47_{\pm 0.75}$ | $96.95_{\pm 0.03}$ | $92.41_{\pm 0.07}$ | $0.90_{\pm 0.12}$ | $0.79_{\pm 0.03}$ | $1.752_{\pm 0.075}$ | $200.55_{\pm 0.55}$ |
| $\mathbf{\ell_{1}}$-sparse | $\textbf{100.0}_{\pm 0.0}$ | $\underline{96.37}_{\pm 0.06}$ | $90.82_{\pm 0.07}$ | $3.80_{\pm 1.62}$ | $\textbf{0.23}_{\pm 0.01}$ | $1.144_{\pm 0.002}$ | $56.93_{\pm 0.32}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.76_{\pm 0.02}$ | $90.23_{\pm 0.04}$ | $\textbf{12.00}_{\pm 2.20}$ | $0.54_{\pm 0.01}$ | $\textbf{-0.022}_{\pm 0.016}$ | $112.58_{\pm 0.82}$ |

**Table 8: Multi-class forgetting on CIFAR-10 and CIFAR-100 datasets on ResNet-18 model. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
| CIFAR-10 - 2-class forgetting |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $100.0$ | $95.76$ | $91.10$ | $3.55$ | $1.000$ | $170.32$ |
| Retrain | $100.0$ | $100.0$ | $96.38$ | $29.58$ | $0.0$ | $0.0$ | $135.23$ |
| FT | $99.98_{\pm 0.01}$ | $\textbf{100.0}_{\pm 0.0}$ | $\underline{96.36}_{\pm 0.09}$ | $0.96_{\pm 0.53}$ | $0.58_{\pm 0.08}$ | $0.750_{\pm 0.009}$ | $5.92_{\pm 0.09}$ |
| RL | $99.70_{\pm 0.02}$ | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{96.39}_{\pm 0.01}$ | $0.0_{\pm 0.0}$ | $1.07_{\pm 0.01}$ | $0.863_{\pm 0.001}$ | $2.79_{\pm 0.02}$ |
| GA | $99.07_{\pm 0.38}$ | $99.43_{\pm 0.13}$ | $94.83_{\pm 0.22}$ | $\underline{26.71}_{\pm 3.68}$ | $0.42_{\pm 0.02}$ | $0.612_{\pm 0.001}$ | $3.72_{\pm 0.13}$ |
| Bad-T | $99.96_{\pm 0.05}$ | $\textbf{100.0}_{\pm 0.0}$ | $95.33_{\pm 0.09}$ | $67.47_{\pm 34.59}$ | $3.98_{\pm 1.08}$ | $1.010_{\pm 0.005}$ | $4.40_{\pm 0.20}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{100.0}_{\pm 0.0}$ | $96.48_{\pm 0.06}$ | $0.06_{\pm 0.03}$ | $0.57_{\pm 0.05}$ | $0.624_{\pm 0.001}$ | $\textbf{1.39}_{\pm 0.02}$ |
| CF-5 | $80.06_{\pm 8.26}$ | $\textbf{100.0}_{\pm 0.0}$ | $96.70_{\pm 0.04}$ | $0.0_{\pm 0.0}$ | $0.80_{\pm 0.02}$ | $0.781_{\pm 0.006}$ | $\underline{1.41}_{\pm 0.05}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.67_{\pm 0.02}$ | $94.94_{\pm 0.17}$ | $25.92_{\pm 0.79}$ | $\underline{0.35}_{\pm 0.01}$ | $\textbf{-0.011}_{\pm 0.011}$ | $2.20_{\pm 0.17}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.67_{\pm 0.02}$ | $94.94_{\pm 0.17}$ | $21.20_{\pm 1.43}$ | $\underline{0.35}_{\pm 0.01}$ | $0.221_{\pm 0.007}$ | $2.19_{\pm 0.14}$ |
| SCRUB | $99.98_{\pm 0.0}$ | $99.99_{\pm 0.0}$ | $96.31_{\pm 0.08}$ | $46.74_{\pm 5.31}$ | $1.47_{\pm 0.10}$ | $0.374_{\pm 0.005}$ | $3.27_{\pm 0.01}$ |
| SALUN | $95.86_{\pm 4.18}$ | $99.99_{\pm 0.01}$ | $96.27_{\pm 0.11}$ | $0.04_{\pm 0.01}$ | $0.89_{\pm 0.05}$ | $0.951_{\pm 0.019}$ | $3.17_{\pm 0.02}$ |
| $\mathbf{\ell_{1}}$-sparse | $99.91_{\pm 0.05}$ | $99.98_{\pm 0.0}$ | $96.47_{\pm 0.09}$ | $1.57_{\pm 0.11}$ | $0.50_{\pm 0.02}$ | $0.560_{\pm 0.004}$ | $2.62_{\pm 0.06}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.92_{\pm 0.0}$ | $96.41_{\pm 0.15}$ | $\textbf{31.40}_{\pm 2.98}$ | $\textbf{0.26}_{\pm 0.01}$ | $\textbf{0.011}_{\pm 0.029}$ | $4.59_{\pm 0.02}$ |
| CIFAR-100 - 5-class forgetting |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $99.98$ | $77.95$ | $95.00$ | $3.18$ | $1.000$ | $175.08$ |
| Retrain | $100.0$ | $99.98$ | $78.45$ | $7.12$ | $0.0$ | $0.0$ | $165.92$ |
| FT | $\textbf{100.00}_{\pm 0.0}$ | $99.93_{\pm 0.06}$ | $77.43_{\pm 0.20}$ | $0.20_{\pm 0.06}$ | $0.38_{\pm 0.01}$ | $0.596_{\pm 0.009}$ | $9.21_{\pm 0.06}$ |
| RL | $98.61_{\pm 0.22}$ | $\textbf{99.98}_{\pm 0.0}$ | $\textbf{77.78}_{\pm 0.19}$ | $0.0_{\pm 0.0}$ | $0.71_{\pm 0.01}$ | $0.613_{\pm 0.008}$ | $3.39_{\pm 0.09}$ |
| GA | $79.99_{\pm 4.75}$ | $95.18_{\pm 0.40}$ | $68.68_{\pm 0.52}$ | $32.25_{\pm 2.02}$ | $1.36_{\pm 0.06}$ | $0.236_{\pm 0.010}$ | $4.99_{\pm 0.04}$ |
| Bad-T | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{99.98}_{\pm 0.0}$ | $75.93_{\pm 0.57}$ | $44.6_{\pm 31.96}$ | $2.86_{\pm 0.25}$ | $1.021_{\pm 0.031}$ | $5.51_{\pm 0.11}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.75_{\pm 0.02}$ | $75.14_{\pm 0.12}$ | $12.40_{\pm 0.26}$ | $0.54_{\pm 0.01}$ | $\underline{0.054}_{\pm 0.010}$ | $\textbf{2.01}_{\pm 0.0}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.97_{\pm 0.0}$ | $77.36_{\pm 0.06}$ | $\underline{3.37}_{\pm 0.52}$ | $\textbf{0.36}_{\pm 0.02}$ | $0.319_{\pm 0.011}$ | $\underline{2.10}_{\pm 0.0}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $91.76_{\pm 0.12}$ | $73.24_{\pm 0.11}$ | $21.96_{\pm 0.49}$ | $0.48_{\pm 0.01}$ | $-0.155_{\pm 0.008}$ | $4.25_{\pm 0.0}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.88_{\pm 0.01}$ | $76.59_{\pm 0.24}$ | $\textbf{10.69}_{\pm 1.29}$ | $0.40_{\pm 0.01}$ | $0.087_{\pm 0.019}$ | $4.29_{\pm 0.01}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.0}$ | $99.97_{\pm 0.0}$ | $\underline{77.64}_{\pm 0.11}$ | $0.95_{\pm 0.35}$ | $0.56_{\pm 0.03}$ | $0.289_{\pm 0.015}$ | $2.27_{\pm 0.03}$ |
| SALUN | $\textbf{100.0}_{\pm 0.0}$ | $99.96_{\pm 0.01}$ | $77.18_{\pm 0.14}$ | $0.13_{\pm 0.09}$ | $0.55_{\pm 0.01}$ | $0.597_{\pm 0.029}$ | $4.46_{\pm 0.04}$ |
| $\mathbf{\ell_{1}}$-sparse | $98.63_{\pm 0.37}$ | $97.5_{\pm 0.14}$ | $73.46_{\pm 0.25}$ | $12.35_{\pm 0.82}$ | $0.38_{\pm 0.01}$ | $0.196_{\pm 0.011}$ | $4.19_{\pm 0.01}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.82_{\pm 0.0}$ | $77.47_{\pm 0.26}$ | $11.16_{\pm 0.54}$ | $\textbf{0.29}_{\pm 0.01}$ | $\textbf{0.044}_{\pm 0.010}$ | $7.31_{\pm 0.02}$ |
| CIFAR-100 - 20-class forgetting |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $99.97$ | $78.03$ | $95.04$ | $3.15$ | $1.000$ | $175.08$ |
| Retrain | $100.0$ | $99.98$ | $80.01$ | $7.55$ | $0.0$ | $0.0$ | $139.93$ |
| FT | $99.81_{\pm 0.04}$ | $\underline{99.97}_{\pm 0.00}$ | $\textbf{79.11}_{\pm 0.35}$ | $0.22_{\pm 0.05}$ | $0.37_{\pm 0.01}$ | $\underline{0.474}_{\pm 0.007}$ | $7.43_{\pm 0.07}$ |
| RL | $95.77_{\pm 0.09}$ | $\textbf{99.98}_{\pm 0.01}$ | $78.42_{\pm 0.05}$ | $0.0_{\pm 0.0}$ | $0.63_{\pm 0.01}$ | $1.207_{\pm 0.004}$ | $2.94_{\pm 0.01}$ |
| GA | $67.06_{\pm 2.58}$ | $96.65_{\pm 0.47}$ | $70.80_{\pm 0.65}$ | $30.16_{\pm 1.42}$ | $1.46_{\pm 0.11}$ | $1.027_{\pm 0.006}$ | $4.11_{\pm 0.02}$ |
| Bad-T | $95.27_{\pm 0.65}$ | $\underline{99.99}_{\pm 0.0}$ | $69.60_{\pm 0.34}$ | $31.11_{\pm 38.11}$ | $2.89_{\pm 0.26}$ | $1.406_{\pm 0.010}$ | $5.17_{\pm 0.11}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.82_{\pm 0.02}$ | $76.89_{\pm 0.03}$ | $14.50_{\pm 0.54}$ | $0.52_{\pm 0.01}$ | $0.807_{\pm 0.003}$ | $\underline{1.83}_{\pm 0.04}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $99.96_{\pm 0.02}$ | $\underline{78.82}_{\pm 0.06}$ | $2.68_{\pm 0.21}$ | $\underline{0.33}_{\pm 0.01}$ | $1.060_{\pm 0.008}$ | $\textbf{1.80}_{\pm 0.03}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $93.25_{\pm 0.32}$ | $74.79_{\pm 0.39}$ | $25.63_{\pm 0.38}$ | $0.47_{\pm 0.01}$ | $0.617_{\pm 0.005}$ | $3.61_{\pm 0.51}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $99.91_{\pm 0.01}$ | $78.39_{\pm 0.24}$ | $13.57_{\pm 0.32}$ | $0.39_{\pm 0.01}$ | $0.889_{\pm 0.005}$ | $3.68_{\pm 0.16}$ |
| SCRUB | $95.03_{\pm 0.75}$ | $99.90_{\pm 0.00}$ | $77.61_{\pm 0.07}$ | $0.93_{\pm 0.13}$ | $0.38_{\pm 0.01}$ | $0.997_{\pm 0.007}$ | $2.14_{\pm 0.02}$ |
| SALUN | $90.69_{\pm 0.76}$ | $98.97_{\pm 0.14}$ | $74.72_{\pm 0.54}$ | $0.17_{\pm 0.03}$ | $0.60_{\pm 0.01}$ | $1.113_{\pm 0.008}$ | $3.85_{\pm 0.0}$ |
| $\mathbf{\ell_{1}}$-sparse | $83.49_{\pm 0.46}$ | $99.52_{\pm 0.03}$ | $76.79_{\pm 0.20}$ | $\textbf{6.36}_{\pm 0.59}$ | $0.38_{\pm 0.01}$ | $1.035_{\pm 0.007}$ | $3.08_{\pm 0.07}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $99.92_{\pm 0.0}$ | $78.59_{\pm 0.32}$ | $\underline{11.52}_{\pm 0.39}$ | $\textbf{0.24}_{\pm 0.01}$ | $\textbf{0.007}_{\pm 0.010}$ | $6.97_{\pm 0.01}$ |

**Table 9: 5-class forgetting results on ImageNet-1K dataset across different model architectures. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
| ImageNet-1K - ResNet-50 |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $11.72$ | $87.45$ | $76.11$ | $61.69$ | $3.73$ | $1.000$ | $2680.15$ |
| Retrain | $100.0$ | $88.80$ | $75.88$ | $9.41$ | $0.0$ | $0.0$ | $2661.90$ |
| FT | $\textbf{100.0}_{\pm 0.0}$ | $\textbf{88.52}_{\pm 0.0}$ | $\underline{76.16}_{\pm 0.01}$ | $8.24_{\pm 1.23}$ | $\underline{0.24}_{\pm 0.01}$ | $0.102_{\pm 0.026}$ | $140.04_{\pm 1.42}$ |
| RL | $99.96_{\pm 0.04}$ | $86.48_{\pm 0.06}$ | $75.25_{\pm 0.02}$ | $0.06_{\pm 0.02}$ | $1.59_{\pm 0.02}$ | $1.296_{\pm 0.084}$ | $200.78_{\pm 2.36}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $80.77_{\pm 0.22}$ | $71.49_{\pm 0.10}$ | $4.20_{\pm 0.46}$ | $0.42_{\pm 0.03}$ | $0.328_{\pm 0.023}$ | $212.14_{\pm 2.61}$ |
| Bad-T | $97.93_{\pm 0.01}$ | $84.01_{\pm 0.03}$ | $73.39_{\pm 0.05}$ | $71.42_{\pm 13.62}$ | $3.56_{\pm 0.38}$ | $1.683_{\pm 0.077}$ | $205.27_{\pm 1.66}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $79.62_{\pm 0.0}$ | $71.22_{\pm 0.13}$ | $13.33_{\pm 1.53}$ | $0.26_{\pm 0.01}$ | $0.183_{\pm 0.028}$ | $193.38_{\pm 0.78}$ |
| CF-5 | $\textbf{100.0}_{\pm 0.0}$ | $84.28_{\pm 0.05}$ | $74.15_{\pm 0.08}$ | $10.38_{\pm 4.67}$ | $\textbf{0.22}_{\pm 0.01}$ | $0.696_{\pm 0.143}$ | $\textbf{82.53}_{\pm 0.46}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $71.84_{\pm 0.03}$ | $65.78_{\pm 0.02}$ | $16.65_{\pm 1.91}$ | $0.35_{\pm 0.04}$ | $\underline{-0.051}_{\pm 0.021}$ | $193.79_{\pm 0.47}$ |
| CF-10 | $\textbf{100.0}_{\pm 0.0}$ | $81.12_{\pm 0.07}$ | $72.32_{\pm 0.09}$ | $14.60_{\pm 6.19}$ | $0.26_{\pm 0.01}$ | $0.608_{\pm 0.135}$ | $\underline{83.31}_{\pm 0.31}$ |
| SCRUB | $99.36_{\pm 0.02}$ | $\underline{88.41}_{\pm 0.03}$ | $76.48_{\pm 0.02}$ | $7.49_{\pm 0.78}$ | $0.25_{\pm 0.02}$ | $0.517_{\pm 0.107}$ | $428.04_{\pm 3.61}$ |
| SALUN | $89.67_{\pm 0.27}$ | $86.25_{\pm 0.15}$ | $75.54_{\pm 0.10}$ | $0.50_{\pm 0.09}$ | $0.88_{\pm 0.01}$ | $0.343_{\pm 0.017}$ | $793.82_{\pm 3.32}$ |
| $\mathbf{\ell_{1}}$-sparse | $97.57_{\pm 0.61}$ | $85.33_{\pm 0.07}$ | $74.77_{\pm 0.03}$ | $\underline{8.84}_{\pm 1.39}$ | $0.32_{\pm 0.02}$ | $0.239_{\pm 0.031}$ | $226.74_{\pm 1.35}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $87.93_{\pm 0.05}$ | $\textbf{76.15}_{\pm 0.04}$ | $\textbf{9.95}_{\pm 1.21}$ | $\underline{0.24}_{\pm 0.01}$ | $\textbf{0.040}_{\pm 0.042}$ | $171.44_{\pm 0.75}$ |
| ImageNet-1K - ViT |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $2.48$ | $98.18$ | $80.59$ | $71.00$ | $4.45$ | $1.000$ | $1943.69$ |
| Retrain | $100.0$ | $98.33$ | $80.42$ | $8.09$ | $0.0$ | $0.0$ | $1920.77$ |
| FT | $96.39_{\pm 0.01}$ | $98.85_{\pm 0.03}$ | $80.93_{\pm 0.06}$ | $3.88_{\pm 0.33}$ | $0.65_{\pm 0.02}$ | $0.937_{\pm 0.009}$ | $281.73_{\pm 2.30}$ |
| RL | $98.27_{\pm 0.02}$ | $99.01_{\pm 0.04}$ | $81.77_{\pm 0.08}$ | $0.0_{\pm 0.0}$ | $2.17_{\pm 0.12}$ | $2.321_{\pm 0.045}$ | $151.32_{\pm 4.12}$ |
| GA | $\textbf{100.0}_{\pm 0.0}$ | $97.04_{\pm 0.01}$ | $\underline{80.17}_{\pm 0.04}$ | $\underline{8.26}_{\pm 2.14}$ | $0.52_{\pm 0.23}$ | $0.674_{\pm 0.021}$ | $193.73_{\pm 2.23}$ |
| Bad-T | $98.19_{\pm 0.03}$ | $\underline{97.91}_{\pm 0.08}$ | $\textbf{80.63}_{\pm 0.04}$ | $0.0_{\pm 0.0}$ | $2.67_{\pm 0.05}$ | $2.743_{\pm 0.018}$ | $715.34_{\pm 6.28}$ |
| EU-5 | $\textbf{100.0}_{\pm 0.0}$ | $93.82_{\pm 0.02}$ | $80.00_{\pm 0.01}$ | $4.74_{\pm 1.33}$ | $0.63_{\pm 0.02}$ | $0.519_{\pm 0.008}$ | $300.55_{\pm 0.76}$ |
| CF-5 | $98.75_{\pm 0.0}$ | $96.57_{\pm 0.01}$ | $80.09_{\pm 0.04}$ | $4.49_{\pm 0.34}$ | $0.64_{\pm 0.01}$ | $0.731_{\pm 0.024}$ | $\textbf{122.39}_{\pm 0.53}$ |
| EU-10 | $\textbf{100.0}_{\pm 0.0}$ | $87.33_{\pm 0.10}$ | $76.26_{\pm 0.13}$ | $\textbf{8.09}_{\pm 0.20}$ | $\textbf{0.36}_{\pm 0.02}$ | $-2.662_{\pm 0.231}$ | $345.37_{\pm 0.70}$ |
| CF-10 | $99.95_{\pm 0.01}$ | $93.86_{\pm 0.02}$ | $78.69_{\pm 0.01}$ | $7.68_{\pm 1.11}$ | $0.72_{\pm 0.03}$ | $\textbf{0.009}_{\pm 0.021}$ | $\underline{140.11}_{\pm 0.49}$ |
| SCRUB | $\textbf{100.0}_{\pm 0.00}$ | $98.84_{\pm 0.02}$ | $81.62_{\pm 0.01}$ | $3.19_{\pm 0.91}$ | $1.062_{\pm 0.03}$ | $-0.846_{\pm 0.032}$ | $404.02_{\pm 2.96}$ |
| SALUN | $94.44_{\pm 0.37}$ | $\textbf{98.12}_{\pm 0.22}$ | $80.73_{\pm 0.06}$ | $0.02_{\pm 0.01}$ | $1.81_{\pm 0.10}$ | $1.928_{\pm 0.280}$ | $325.70_{\pm 3.57}$ |
| $\mathbf{\ell_{1}}$-sparse | $93.55_{\pm 0.62}$ | $94.69_{\pm 0.37}$ | $78.84_{\pm 0.10}$ | $2.98_{\pm 0.33}$ | $\underline{0.49}_{\pm 0.01}$ | $0.831_{\pm 0.022}$ | $717.42_{\pm 3.21}$ |
| COLA | $\textbf{100.0}_{\pm 0.0}$ | $96.35_{\pm 0.04}$ | $79.25_{\pm 0.24}$ | $5.92_{\pm 1.36}$ | $0.65_{\pm 0.01}$ | $\underline{0.131}_{\pm 0.022}$ | $502.32_{\pm 2.36}$ |

**Table 10: Random data forgetting on CIFAR-10 and CIFAR-100 datasets on ResNet-18 model. A better performance of an MU method corresponds to a smaller performance gap with Retrain (except RTE), with the top method in bold and the second best underlined.**
| CIFAR-10 - ResNet-18 |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $100.0$ | $95.54$ | $92.90$ | $0.09$ | $1.000$ | $170.32$ |
| Retrain | $3.94$ | $100.0$ | $95.26$ | $75.12$ | $0.0$ | $0.0$ | $152.87$ |
| FT | $5.03_{\pm 0.40}$ | $98.95_{\pm 0.21}$ | $92.94_{\pm 0.26}$ | $83.52_{\pm 0.58}$ | $0.07_{\pm 0.11}$ | $\underline{-0.069}_{\pm 0.013}$ | $8.11_{\pm 0.03}$ |
| RL | $4.77_{\pm 0.27}$ | $\underline{99.92}_{\pm 0.0}$ | $\textbf{93.54}_{\pm 0.04}$ | $22.47_{\pm 1.19}$ | $0.38_{\pm 0.02}$ | $0.084_{\pm 0.030}$ | $2.75_{\pm 0.01}$ |
| GA | $2.86_{\pm 0.76}$ | $98.37_{\pm 0.71}$ | $91.90_{\pm 0.70}$ | $85.49_{\pm 2.17}$ | $0.09_{\pm 0.01}$ | $0.924_{\pm 0.028}$ | $4.31_{\pm 0.03}$ |
| Bad-T | $5.47_{\pm 1.05}$ | $99.87_{\pm 0.05}$ | $91.51_{\pm 0.61}$ | $39.53_{\pm 3.43}$ | $0.27_{\pm 0.03}$ | $0.939_{\pm 0.053}$ | $4.78_{\pm 0.09}$ |
| EU-10 | $3.16_{\pm 0.19}$ | $98.68_{\pm 0.08}$ | $93.07_{\pm 0.12}$ | $\underline{83.40}_{\pm 0.21}$ | $\underline{0.06}_{\pm 0.01}$ | $-0.110_{\pm 0.126}$ | $\underline{2.13}_{\pm 0.05}$ |
| CF-10 | $2.71_{\pm 0.24}$ | $99.11_{\pm 0.06}$ | $\underline{93.47}_{\pm 0.15}$ | $84.33_{\pm 0.05}$ | $\textbf{0.05}_{\pm 0.01}$ | $0.219_{\pm 0.029}$ | $\textbf{2.10}_{\pm 0.06}$ |
| SCRUB | $\underline{4.31}_{\pm 1.50}$ | $96.21_{\pm 1.70}$ | $88.83_{\pm 1.86}$ | $37.88_{\pm 7.65}$ | $0.56_{\pm 0.09}$ | $0.322_{\pm 0.161}$ | $3.37_{\pm 0.05}$ |
| SALUN | $2.74_{\pm 0.30}$ | $97.77_{\pm 0.04}$ | $91.68_{\pm 0.44}$ | $83.52_{\pm 2.20}$ | $0.098_{\pm 0.03}$ | $0.861_{\pm 0.123}$ | $5.69_{\pm 0.04}$ |
| $\mathbf{\ell_{1}}$-sparse | $5.47_{\pm 0.22}$ | $96.66_{\pm 0.07}$ | $91.31_{\pm 0.25}$ | $\textbf{77.12}_{\pm 0.21}$ | $0.09_{\pm 0.01}$ | $-0.157_{\pm 0.026}$ | $3.03_{\pm 0.04}$ |
| COLA | $1.22_{\pm 0.07}$ | $\textbf{99.99}_{\pm 0.01}$ | $94.71_{\pm 0.05}$ | $88.95_{\pm 0.31}$ | $0.07_{\pm 0.01}$ | $0.849_{\pm 0.076}$ | $6.95_{\pm 0.03}$ |
| COLA+ | $\textbf{3.90}_{\pm 0.08}$ | $99.24_{\pm 0.17}$ | $93.23_{\pm 0.09}$ | $83.48_{\pm 0.10}$ | $\underline{0.06}_{\pm 0.01}$ | $\textbf{0.024}_{\pm 0.101}$ | $7.80_{\pm 0.02}$ |
| CIFAR-100 - ResNet-18 |  |  |  |  |  |  |  |
| Methods | UA | RA | TA | MIA | JSD | IDI | RTE (min) |
| Original | $0.0$ | $99.98$ | $78.09$ | $95.82$ | $0.56$ | $1.000$ | $175.08$ |
| Retrain | $23.10$ | $99.98$ | $77.78$ | $39.72$ | $0.0$ | $0.0$ | $0.0$ |
| FT | $17.44_{\pm 1.12}$ | $98.46_{\pm 0.24}$ | $70.99_{\pm 0.45}$ | $67.35_{\pm 0.53}$ | $0.46_{\pm 0.02}$ | $0.311_{\pm 0.034}$ | $8.40_{\pm 0.13}$ |
| RL | $24.67_{\pm 0.42}$ | $\textbf{99.66}_{\pm 0.0}$ | $73.10_{\pm 0.49}$ | $2.13_{\pm 0.17}$ | $0.84_{\pm 0.02}$ | $-0.246_{\pm 0.056}$ | $2.95_{\pm 0.03}$ |
| GA | $11.73_{\pm 1.43}$ | $95.21_{\pm 0.78}$ | $68.38_{\pm 1.03}$ | $74.97_{\pm 1.10}$ | $0.65_{\pm 0.01}$ | $0.704_{\pm 0.039}$ | $4.66_{\pm 0.03}$ |
| Bad-T | $64.35_{\pm 7.44}$ | $99.07_{\pm 0.56}$ | $53.05_{\pm 2.53}$ | $\textbf{11.85}_{\pm 5.93}$ | $1.51_{\pm 0.16}$ | $1.003_{\pm 0.006}$ | $5.04_{\pm 0.05}$ |
| EU-10 | $24.15_{\pm 0.09}$ | $90.15_{\pm 0.08}$ | $72.25_{\pm 0.36}$ | $\underline{59.47}_{\pm 0.39}$ | $0.27_{\pm 0.01}$ | $0.404_{\pm 0.085}$ | $\underline{2.31}_{\pm 0.02}$ |
| CF-10 | $20.40_{\pm 0.20}$ | $95.06_{\pm 0.24}$ | $\underline{74.44}_{\pm 0.23}$ | $62.18_{\pm 0.27}$ | $\underline{0.25}_{\pm 0.01}$ | $0.464_{\pm 0.061}$ | $\textbf{2.30}_{\pm 0.02}$ |
| SCRUB | $3.47_{\pm 2.85}$ | $97.77_{\pm 2.31}$ | $71.89_{\pm 2.87}$ | $71.49_{\pm 4.15}$ | $0.37_{\pm 0.02}$ | $0.528_{\pm 0.132}$ | $3.59_{\pm 0.05}$ |
| SALUN | $32.77_{\pm 1.20}$ | $\underline{99.87}_{\pm 0.02}$ | $71.97_{\pm 0.37}$ | $3.32_{\pm 0.28}$ | $0.81_{\pm 0.02}$ | $\underline{-0.226}_{\pm 0.078}$ | $5.99_{\pm 0.09}$ |
| $\mathbf{\ell_{1}}$-sparse | $\textbf{22.83}_{\pm 0.15}$ | $88.94_{\pm 0.41}$ | $69.54_{\pm 0.73}$ | $62.36_{\pm 0.37}$ | $0.26_{\pm 0.01}$ | $0.634_{\pm 0.072}$ | $3.37_{\pm 0.03}$ |
| COLA | $15.44_{\pm 0.23}$ | $96.29_{\pm 0.01}$ | $\textbf{75.61}_{\pm 0.11}$ | $64.62_{\pm 0.42}$ | $0.18_{\pm 0.01}$ | $0.724_{\pm 0.072}$ | $6.95_{\pm 0.03}$ |
| COLA+ | $\underline{23.50}_{\pm 0.16}$ | $93.78_{\pm 0.07}$ | $73.15_{\pm 0.59}$ | $59.58_{\pm 0.24}$ | $\textbf{0.24}_{\pm 0.01}$ | $\textbf{0.078}_{\pm 0.013}$ | $10.2_{\pm 0.16}$ |

Figure: (a) Original
Refer to caption: /html/2405.17878/assets/x15.png

Figure: (a) Original
Refer to caption: /html/2405.17878/assets/x29.png

Figure: (a) Original
Refer to caption: /html/2405.17878/assets/x43.png

Figure: (a) CIFAR-10 - ResNet-18
Refer to caption: /html/2405.17878/assets/x57.png

Figure: Figure 14: Binary train accuracy on CIFAR-10 single-class forgetting remain and forget sets using ResNet-18. It shows a similar graph with binary MI.
Refer to caption: /html/2405.17878/assets/x63.png

Figure: (a) Bad-T
Refer to caption: /html/2405.17878/assets/x64.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: In addition to addressing W1 and W2, the authors are also expected to answer the following questions:

Q1. In Figure 7, COLA appears to obtain a negative IDI value, which is even lower than Retrain. Does this imply that COLA is "over-unlearned”, according to the statement in Line 410?

Q2. Intuitively, for the random data forgetting task, if the retained dataset contains many samples similar to a forgotten one, residual information will likely persist within the intermediate layers. Given that the value of the Information Difference Index (IDI) heavily relies on the amount of this residual information, the authors should provide empirical evidence or theoretical analysis to demonstrate the effectiveness (or potential limitations) of IDI in random data forgetting tasks, particularly when there is a high similarity between forgotten and retained samples. This would help clarify the robustness and generalizability of the proposed metric.
soundness: 3
strengths: S1. The assertion that "black-box metrics may not be sufficient for assessing strong unlearning" is well-motivated and compelling.

S2. The paper includes extensive empirical evaluations that support the claims made by the authors.

S3. The technical quality of the paper is good, and the ideas presented may be of significant interest to the machine learning community.
summary: This paper addresses a crucial and well-motivated issue in recent machine unlearning research: the inadequacy of common black-box metrics for assessing strong unlearning. To tackle this problem, the authors propose a new white-box metric called the Information Difference Index (IDI), which quantifies the residual information of the forget set in the intermediate layers of the network, providing a more comprehensive evaluation of unlearning efficacy.
weaknesses: W1. This study seems to focus predominantly on the single-class forgetting scenario in the main text, relegating results related to other cases (e.g., random data forgetting, multi-class forgetting) to the appendices. Given that the paper aims to provide a general approach and is not limited to single-class forgetting, the authors are strongly encouraged to include essential findings from all three tasks in the main text. For instance, empirical evaluations demonstrating whether the proposed method consistently outperforms other baselines across all scenarios should be highlighted.

W2. The head distillation (HD) approach introduced in Section 3 seems to be applicable only to the single-class forgetting task. The authors should either demonstrate HD's applicability to multi-class and random data forgetting tasks, or explicitly state its limitations if it cannot be generalized. 

If the generalization is possible, the authors should address whether the main observations regarding residual information and the limitations of black-box metrics are applicable in these other contexts. Specifically:
+ Would residual information still be retained within the intermediate layers of unlearned models?
+ Would black-box metrics continue to fail in capturing this residual information?

W3. I agree that a primary limitation of the proposed COLA approach is its computational burden due to the mutual information (MI) estimation process for each network layer, although this point is also noted by the authors in the paper.

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 2
questions: In Figure 5, why does the mutual information decrease with the layers? Won't the model get more information as the features are passed through the network?
soundness: 3
strengths: Common unlearning metrics are known to be incomplete and not always reliable. This paper proposes a new evaluation methods for unlearning by estimation the mutual information using InfoNCE between intermediate layer features and output labels. This is a method that takes the internal behavior of the model into account, which measures unlearning at a deep level.

The paper also proposed a new unlearning methods from a similar motive. The CoLA method is able to achieve stronger unlearning compared to other unlearning methods.
summary: The paper address the issue that current black-box unlearning evaluation methods does not reflect the true unlearning quality -- many methods that pass such evaluation methods actually contains substantial information about the forget set, which make the unlearned model prone to relearning attacks. The paper proposes a new white-box evaluation metric based on mutual information. The estimated InfoNCE loss is used to calculate Information Difference Index (IDI) to ensure effective unlearning.

Based on this intuition the paper further proposes a new unlearning method by collapsing forgot set representation on the encoder layers then re-aligning the model. The paper demonstrated that such unlearning methods achieve low IDI compared to other unlearning methods.
weaknesses: One motivation for the new evaluation metric is from the concern that current metric only measures shallow unlearning which can be prone to relearning attacks, However, the paper does not present the strength of this new metric in a systematic manner. Figure 4 briefly mentioned retrain attacks but does not have the SCRUB method, it also comes before the new metric which makes it a bit confusing to read.

The evaluation method uses up to $\ell$-th layer and trains the rest to estimate the InfoNCE loss, which seems resource-intensive and scales poorly with the increase size of neural networks. It also requires a retrained model for IDI statistic, which seems infeasible for evaluating unlearning in practice.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: 1. Could you explain why the sum of absolute values is not used when calculating Equations (1) and (2)?

2. Is there any possibility of encountering a near-zero denominator during the calculation in Equation (2)?

3. Could you provide a comparison of the effectiveness of computing mutual information for just the last few layers versus all layers?
soundness: 2
strengths: 1. The topic of designing evaluation metrics for machine unlearning is both significant and challenging. This paper introduces a new metric based on the mutual information between the features and labels to be forgotten, effectively capturing the residual information in the unlearned model.

2. A new unlearning method inspired by the proposed evaluation metric is introduced, which is greatly appreciated.

3. The presentation is clear and easy to follow.

4. Several experiments are conducted to demonstrate the efficacy of the proposed evaluation method.
summary: This paper first introduces a naive machine unlearning method that modifies only the last layer of the model, achieving strong performance based on existing black-box evaluation metrics. Building on this, the author introduces a new white-box metric based on the mutual information between hidden features and labels to be forgotten. Furthermore, the paper proposes a new unlearning method, Collapse and Align (COLA), which effectively removes residual information at the feature level.
weaknesses: 1. If I understand correctly, for each unlearned model with $L$ layers, mutual information must be estimated $3L$ times, each requiring data sampling and the training of two networks. Coupled with the time needed for retraining, the overall complexity is substantial.

2. Given the complexity, the necessity of such calculations needs further scrutiny. A simpler white-box baseline, using an MLP to train a mapping from concatenated hidden features to forget_labels and obtaining a similar metric based on equation 2, might reduce complexity. It would be interesting to see if your proposed method outperforms this baseline.

3. One benefit claimed by the author is that the proposed metric exhibits a low standard deviation. However, this seems to hold true only when the number of forgettings is low, as demonstrated in Table 1.

4. The experiments comparing IDI with white-box MIA should be expanded and included in the main manuscript rather than the appendix to allow for a fair comparison.

5. Regarding Membership Inference Attacks (MIAs), only score-based MIA is employed, while Likelihood Ratio Attack (LiRA) [1] is not.

6. There lack of literature on auditing machine unlearning.


[1] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: - What is your point of using HD to support your argument the black-box is insufficient? Why do you think the change in most of the parameters is a good sign of unlearning? And why do treat HD as a good method to illustrate your opinion. From my perspective, there are some probabilities that the information is controlled by the key parameters, where is those in last layer in your example. 
- From the practical perspective, I wonder whether the meaning of this work will be decreased as long as the evaluation is mostly based on the output.
- What is the meaning of $g_{n_l}(0)$ and $g_{n_l}(1)$? What is the output? Could you please give an example?
- In Table 1, there are both positive and negative IDIs. Is there any difference between the positive and negative value? 
- It seems that the proposed method needs some training. Could you please present the testing time of the proposed method?
- In the sentence of Line 358,
soundness: 3
strengths: - The topic itself is really interesting and necessary for MU.
- The experiments and plots are detailed and promising.
- The baseline method selection is relatively complete and the description of the method is complete.
summary: This paper investigate the shortcomings of current white-box unlearning evaluation method. The author use an example in class-wise unlearn to illustrate the residual information in the model. Then, they analyze the InfoNCE loss and propose the new white-box evaluation metrics called IDI. To better evaluate the model, the author proposes COLA. The extensive experiment results show the new evaluation metrics can evaluate the residual information in the model better.
weaknesses: - The references need revision. For example, Model sparsity can simplify machine unlearning is the paper from NeurIPS 2023 and the author list is incorrect. 
- What MIA is used? May I ask the reason for choosing this MIA over the others, like the MIA in [2]? I guess the MIA used here is based on [1].  
- In addition to the proposed evaluation metrics, it would be more promising if the author can report 
- The existing metrics like UA, TA, which is not perfect. There are still some metrics, like ZRF [3], AUS [4], etc. Since the author wants to introduce a new and better evaluation metrics, a complete overview for the existing metrics should be necessary.
- The introduction of the proposed evaluation metrics is a little bit hard to follow because of the notation mass. For example, there are several weights in this paper, like $\theta$, $\nu$, $\eta$. Maybe a notation table can make everything clear.
- The performance in Table 1 is useful for the reader to connect your proposed method with the existing metrics. However, I wonder why there is no such a table for random forgetting.
- More baselines like Boundary Unlearning [5] should be added.


[1] Song, Liwei, and Prateek Mittal. "Systematic evaluation of privacy risks of machine learning models." 30th USENIX Security Symposium (USENIX Security 21). 2021.

[2] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 2022.

[3] Poppi, Samuele, et al. "Multi-Class Unlearning for Image Classification via Weight Filtering." IEEE Intelligent Systems (2024).

[4] Cotogni, Marco, et al. "Duck: Distance-based unlearning via centroid kinematics." arXiv preprint arXiv:2312.02052 (2023).

[5] Chen, Min, et al. "Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Meta-review: This work proposes a new metric to evaluate performance of machine unlearning algorithms. The proposed approach is model dependent, namely needs to access internal states of the underlying machine learning algorithms. A rich set of experiments are included to demonstrate that the new metric can characterize the level of unlearning.

Reviewers agree that this work considers an important problem and appreciate the extensive empirical study.

However, reviewers also have concerns that the computational cost is high and needs to access the retrained model, and thus it is unclear how feasible it is to deploy such method to large-scale problems in practice.

Decision: Reject