CoUn: Empowering Machine Unlearning via Contrastive Learning
Authors: Yasser H. Khalil
Mehdi Setayesh
Hongliang Li
Venue: NeurIPS 2025 poster ()
Keywords: Machine Unlearning
Contrastive Learning
Supervised Learning
Representation Learning
Semantic Similarity
Primary area: general_machine_learning
TL;DR: This paper presents CoUn, a contrastive learning (CL)-based machine unlearning (MU) framework using only retain data. Further, our proposed CL module can be integrated with existing baselines to empower their performance.

Abstract: Machine unlearning (MU) aims to remove the influence of specific ''forget'' data from a trained model while preserving its knowledge of the remaining ''retain'' data. Existing MU methods based on label manipulation or model weight perturbations often achieve limited unlearning effectiveness. To address this, we introduce CoUn, a novel MU framework inspired by the observation that a model retrained from scratch using only retain data classifies forget data based on their semantic similarity to the retain data. CoUn emulates this behavior by adjusting learned data representations through contrastive learning (CL) and supervised learning, applied exclusively to retain data. Specifically, CoUn (1) leverages semantic similarity between data samples to indirectly adjust forget representations using CL, and (2) maintains retain representations within their respective clusters through supervised learning. Extensive experiments across various datasets and model architectures show that CoUn consistently outperforms state-of-the-art MU baselines in unlearning effectiveness. Additionally, integrating our CL module into existing baselines empowers their unlearning effectiveness.

CoUn: Empowering Machine Unlearning via Contrastive Learning
Yasser H. Khalil
Mehdi Setayesh
Hongliang Li
Huawei Noah's Ark Lab, Montreal, Canada
(yasser khalil1, mehdi setayesh1, hongliang. 1ii))hhuawei.com com
Abstract
Machine unlearning (MU) aims to remove the influence of specific "forget" data from trained model while preserving its knowledge the remaining "retain" data. Existing MU methods based on label manipulation model weight perturbations often achieve limited unlearning effectiveness. address this, we introduce CoUn, novel MU framework inspired by the observation that model retrained from scratch using only retain data classifies forget data based on their semantic sim- ilarity to the retain data. CoUn emulates this behavior by adjusting learned data representations through contrastive learning (CL) and supervised learning, applied exclusively retain data. Specifically, CoUn (1) leverages semantic similarity between data samples to indirectly adjust forget representations using CL. and (2) maintains retain representations within their respective clusters through supervised learning. Extensive experiments across various datasets and model architectures show that CoUn consistently outperforms state-of-the-an MU baselines unlearn- ing effectiveness. Additionally, integrating our CL module into existing baselines empowers their unlearning effectiveness.
Introduction
The widespread adoption of machine learning (ML) has raised concerns regarding data privacy and regulatory compliance, such as the General Data Protection Regulation (GDPR) 21. Machine unlearning (MU)
4 5 addresses these concerns by removing the influence of specific training data forget data) from trained model, termed the Original model, while preserving the knowledge of the remaining data (i.e., retain data). Retraining the model from scratch on retain data is considered exact unlearning (6, termed the gold-standard Retrain model. While exact unlearning is effective, it is computationally inefficient. Alternatively, approximate unlearning aims to efficiently achieve an unlearned model that performs approximately the same as the Retrain model 8 [0]
To develop an effective approximate unlearning algorithm, we first analyze how the Retrain model classifies forget data. Figure []] illustrates the representation space of two Retrain mod- els: one trained without "truck" class samples (i.e., class-wise forgetting) and another trained without 10% randomly selected samples (i.e., random forgetting). Our analysis reveals that the Retrain model classifies forgel samples into clusters of retain samples that exhibit the high- est semantic similarity them. For instance, in class-wise forgetting, forget "truck" sam- ples are mostly misclassified as semantically similar clusters, like "automobile" (69.32%). "air- plane' (13.47%), and 'ship' (12.60%) with no correct classifications due to the absence of "truck' cluster. In random forgetting, 97.42% forget 'truck' samples are correctly classified 'truck' due to their semantic similarity with 'truck' samples in the retain data, while most of the others are misclassified as "automobile" (1.23%), 'airplane' (0.38%), and 'ship' (0.4%).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Thus, to closely match the performance of ex- act unlearning, approximate unlearning should adjust the learned data representations of the Original model such that: 1 retain samples are correctly classified, and @ forget samples are pushed into clusters of other retain samples that exhibit the highest semantic similarity to them. In random forgetting, these clusters may be the same or different from the original clusters of the forget samples; however, in class-wist forgetting, they must be different from the orig- inal clusters of the forget samples.
Jacosh chattnaa
Jocia
Deer
Dreg
Figure I: Representation space of the Retrain model trained with ResNet-18 18 and CIFAR- 10, excluding 'truck' class samples (leff) and excluding 10% randomly selected samples (right). Small dots represent retain samples from different clusters, while larger dots indicate forget sam- ples classified into clusters of retain samples that exhibit the highest semantic similarity them.
Existing MU methods g TO 11 12 attempt to close the performance gap between approx- imate and exact unlearning, focusing on three criteria: good forget quality (evaluating misclassification of forget data and success in membership [T)) @ high utility (maintaining high accuracy retain data and generaliz- ing well to test data), and ① efficiency. Prior methods [14] 9 15 16] often rely on label manipulation, assigning forget samples to semantically inconsistent random clusters to increase misclassification. However, this strategy can degrade the performance on retain data, thereby reducing utility. Moreover, these methods require access to forget data, which may not always be federated unlearning settings where clients may leave the network after requesting unlearning 17 g Recent methods focus on model weight perturbations [8] 10]. and eliminate the need for forget data. Yet, excessive perturbations can compromise utility by erasing important knowledge, while weak perturbations degrade forget quality. Thus, inadequate perturbations limit unlearning effectiveness.
inference attacks [9]
pattubblllppttullrrr.y in
In light of the above, we propose CoUn (Contrastive learning for empowering Unlearning), a novel MU framework that leverages contrastive learning (CL) and supervised learning, applied exclusively to retain data. CoUn exploits the fact that the learned representations of the Original model has already captured the semantic similarities among samples. Consequently, adjusting the retain representations during unlearning indirectly influences the forget representations. In CoUn, the CL module (Figure 2) pulls together representations of augmented views of the same retain samples (positives) while pushing apart representations of different retain samples (negatives) [88 19 negatives, which samples that belong to the same cluster positives but treated as negatives, can cause clusters to overlap. This phenomenon is known as cluster collision [21 22 23 As result, forget representations are indirectly pushed toward clusters of other retrain samples that exhibit the highest semantic similarity to them, thereby improving forget quality. Additionally, CoUn preserves high utility by mitigating cluster collision among retain representations through supervised learning. This adjustment of forget representations toward semantically similar retain representations, while maintaining the separation of retain clusters, enables effective unlearning. Our key contributions are:
20 21.. However, false
We introduce CoUn, a novel MU framework that achieves effective unlearning by adjusting learned data representations based semantic similarity using CL and supervised learning on retain data.
We provide empirical insights into how CoUn effectively unlearns by pushing forget representations into clusters of semantically similar retain representations. We also present theoretical analysis showing that CoUn induces a higher misclassification rate on forget data while maintaining low misclassification rate on retain data, thereby contributing to better forget quality and utility.
We validate CoUn across different datasets, model architectures, and forgetting scenarios, showing a reduced performance gap with exact unlearning compared to MU basclines. We also demonstrate that integrating our CL module into existing baselines empowers their unlearning effectiveness.
2 Related Work
Label Manipulation.
Existing MU methods attempt to unlearn by manipulating labels of forget data, thereby misleading the model into learning incorrect labels. NegGrad [24] 25 achieves this by applying gradient ascent on the forget data. NegGrad+ [14] combines fine-tuning (minimizing loss with respect to retain data) and gradient ascent (maximizing loss with respect to forget data).
Both NegGrad and NegGrad+ exemplify label manipulation strategies by modifying the model's
2

---

output distribution of the forget data through targeted gradient- gradient-based adjustments. Amnesiac [ and SalUn 9 randomly relabel the forget data and then apply joint optimization on both retain and forget data. BadTT [66] uses knowledge distillation from two teacher models (Original and Random models) into the unlearned model, while SCRUB [14 extends Bad'T' by incorporating NegGrad+. Both BadTT and SCRUB manipulate the model's 's output distributions of forget data by using random teacher, effectively altering the perceived labels of the forget data. However, such methods that assign forget samples to semantically inconsistent or random classes to increase misclassification rates often degrade performance on retain data, resulting in lower utility. Moreover, as shown in Figure [I exact unlearning does not aim misclassify all forget samples, some can be correctly classified.
Model Weight Perturbation. Recent methods perturb the model's weights to achieve unlearning. ssspars
10 introduces an l regularization term in the objective function, inspired by model pruning [26 27 In addition to random labeling, SalUn [ adjusts model weight parameters using gradient-based saliency masks. SSD [28] uses the Fisher information matrix to identify and dampen weight parameters critical to forget data. NoT [8] negates layer-wise weights to support unlearning. However, inadequate perturbations reduce unlcarning effectiveness. Excessive perturbations degrade utility by erasing essential knowledge, while insufficient perturbations fail to properly eliminate the influence of forget data, thereby compromising forget quality.
Contrastive Learning. The goal of CL is to maximize similarity between augmented views of the same sample while minimizing similarity between different samples [I8] 19 CL's ability to adjust representations aligns with the objective of MU, which seeks to disrupt the representations of forget data. This work focuses on CL. with the InfoNCE loss, as used in SimCLR
20
29
30
31 32
[20]. One method that applies CL for unlearning pushes forget samples away from retain samples of the same class and pulls them closer to retain samples of different classes [33] However, this method harms the model's utility by pushing forget samples away from their original clusters, resulting in high misclassification rates. CoUn differs from 33 by utilizing semantic similarity and cluster collision for effective unlearning, instead of forcing forget samples to be misclassified. Similar to exact unlearning, CoUn yields higher misclassification rate for forget data while maintaining low misclassification rate on retain data. Further discussion is provided in Appendix A
3 Methodology
Section 3.2 then introduces our proposed MU framework, CoUn. Section [33 follows with empirical insights demonstrating how CL and super- vised learning adjusts the representation space to facilitate effective unlearning. Finally, Section 3.4 provides theoretical insights.
We begin by formulating the MU problem in Section 3.1
3.1 Machine Unlearning: Background
Given a dataset D, partitioned into forget data D. and retain data D, = D \ Du., the goal of MU is to transform an Original model 0., trained on D, into an unlearned model o,, that effectively removes the influence of D... We define the forget data ratio as: [Dal/ID] x 100.
In exact unlearning, 0,, is obtained by training a randomly initialized model only on D,. yielding the gold-standard Retrain model. While this ensures precise unlearning, it incurs significant computational cost. In contrast, approximate unlearning methods aim to obtain O,, more efficiently, often at the expense reduced effectiivnesss, resulting a performance gap relative to the Retrain model. In approximate unlearning, O,, is initialized with the parameters of 0,.
3.2
CoUn Overview
CoUn is a CL-bbasd MU framework designed to adjust the learned representation space of both retain and forget data for effective unlearning. The framework of CoUn incorporates two key components: contrastive learning and @ supervised learning. The overall architecture of CoUn is illustrated in Figure 2 In CoUn, CL is applied on retain data via a CL module to adjust their representations by pulling the representations of two augmented views of the same sample (positives) closer together, while pushing representations of different samples (negatives) further apart [8 19 20
3

---

Let (I,Y) denote batch of images and their corresponding labels sampled from D,. We have (I,Y) = ((in: Vo
Supervised Learning
Ces
) n==1. where N is the batch size and (1, yn) E Dr. We con- sider that Y E R RNNK represents a batch of one-hot encoded target labels, with K denot- ing the number of classes. Two augmented views, = ((I) and X' = (I( are gener- ated using transformations t, t' ~ T, respec- tively, where T is transformation distribution combining multiple image augmentations (e.g. random cropping, random horizontal flipping, and color normalization). These augmented views are encoded by the feature extractor fo., producing representations Z = fo. (X) and com-
8
X
h6.
(CL Module
Contrastive
Learning
Ccu
Figure CoUn framework. Two augmented views are generated from a batch of retain image samples I. These views are processed by the feature extractor fe.. yielding retain representations (Z, Z'). A CL module adjusts the representations, while supervised learning applied the classifier head he. enforees their cluster separation.
Z' -Je.. (X') E RND, where D is the representation dimension. The representations pared using cosine similarity, a commonly employed measure in CL to assess the similarity between normalized vectors. The cosine similarity between two normalized vectors and is computed as w v w²u. v. Let Zn and z denote the n-th row of the representation matrices Z and Z', respectively. For a given representation nn corresponding image in. the CL loss for the positive pair of representations (Z,, xx,) is calculated as:
exp(zn 2/./T)
(zz.)= log Σド= exp(=n 2n 2²/-T)
where r is a temperature constant. Each representation 2/ J z in the batch is considered negative for Zn. This CL loss encourages positive pairs of representations to be closer in the representation space while pushing them away from negative representations. Following SimCLR [20 we employ a symmetric formulation for the overall CI. loss, defined as:
CZ,
1 N
Σ (1(z)) (((nn)(((1)) ((()))
Since CoUn samples a random batch, it does not explicitly select negatives. Consequently, false negatives, which are negative image samples sharing the same class label as the positive image sample, may be present. Specifically, for each image sample i, images i, in the batch (with j * n) are false negatives if y, yn, Including false negatives in CL introduces the cluster collision problem 22 23,, where samples from the same cluster are pushed apart, potentially bringing them closer to semantically similar samples from other clusters, thus causing cluster overlap. However, false negatives that are more semantically similar with other samples within their own clusters will remain in their original clusters. In other words, not all false negatives will leave their original clusters.
Moreover, since the Original model is trained on both retain and forget data, which share semantic information, this information is captured in the model's weights and reflected in its learned rep- resentations. Arora et al. 34 provide theoretical and empirical evidence showing that learned representations connect similarity in the training data to the semantic information that is implicitly present in downstream tasks. Their work also establishes provable guarantees on the performance of such representations in downstream settings. This supports the observation that the Retrain model-trained solely on retain data aatta-llsssifies forget samples based on semantic similarity, and this explains why in random forgetting it can correctly classify some of the forget data. From the Retrain model's ' perspective, retain data serve as the training set, and forget data act as the downstream task. Consequently, any adjustment to retain representations will indirectly influence forget representations.
Building on this principle, when the learned retain representations are adjusted using CL, forget representations are indirectly pushed toward clusters of other retain samples that exhibit the highest semantic similarity to them. In random forgetting scenario, these retain samples may belong to clusters that are either the same as or different from the original clusters of the forget samples. In contrast, class-wise forgetting scenario, they necessarily belong to different clusters. As a result, the CL module in CoUn can induce cluster collisions among both retain and forget representations. While this improves forget quality, it also degrades model's 's utility. To address this, CoUn mitigates the impact of cluster collision retain representations by applying supervised learning to the retain

---

data. This ensures that retain representations are preserved within their respective clusters, thereby maintaining high model utility.
(Z), where the classifier head ho. takes representations Z as input to produce predictions Y. Cross-entropy (CE) loss is used in CoUn for supervised learning to maintain cluster separation for retain samples, and is defined as:
Let Y N represent the model predictions. We have Y (e,,(2)
N
K
1
Lce (Y, 8)
vo log(Un,k).
(3)
n= k=1
Therefore, the final loss function combines CE and CL losses, weighted by scaling factor A:
L
Lce + XLCL.
(4)
Appendix
B presents the pseudo-code and the PyTorch implementation of CoUn.
3.3 Empirical Analysis
Chewnaa Tirgeting: Truce
Jandon Eorpeitg 10%
Figure visualizes the representation space of four O_ models produced by the fine-tune (FT) and CoUn unlearning methods under class- wise and random forgetting scenarios. All four On models are initialized from 0,, which is trained the entire CIFAR-10 10 training data using ResNet-18. FT reduces the influence of Dr, through catastrophic forgetting by fine- tuning O, on D,. Although the CL module in CoUn applied only retain data, the visu- alization shows that the forget representations are affected -heetedd-hhyeeeeeeeeee are pushed toward clusters of other retain samples that exhibit the highest semantic similarity them. Additionally, we can see that for random forgetting these clus- ters may either be the same as or different from the original clusters of the forget samples; and for class-wise forgetting, they are necessarily different. Meanwhile, retain representations re- main well-clustered due to supervised learning. Notably, under class-wise forgetting setting, FT performs comparably to Retrain (Figure []) due to the relatively low entanglement between D, and Du in this setting, making it a simpler un- learning task than random forgetting (II].
Fugee
Papataa
Eurpetzag
Figure 3: Representation space of FT and CoUn un- learned models (rows). Columns correspond to two for- getting scenarios: class-wise ('truck') and random (10% forget ratio). The Original model trained on CIFAR-10 -10 using ResNet-18. Small dots represent retain samples from different clusters, while larger dots indicate forget samples classified into the corresponding clusters. To achieve effective unlearning. CoUn adjusts representations based on semantic similarity, pushing forget representa- tions into clusters of other retain samples with the highest semantic similarity to them, and preserving retain repre- sentations within their clusters.
Table further confirms that CoUn more effectively classifies forget samples based on semantic similarity than FT, achieving performance closer to that of the Retrain model in both class-wise and random forgetting scenarios. In particu- lar, CoUn produces forget sample predic- tions that more closely align with those of the Retrain model. Further experi- ments is provided in Appendix C
Table 1: Predictions of forget 'truck' samples based most semantically similar classes. Experiments are conducted using CIFAR- 10 and ResNet-18. The difference (A) and the (hest) av- crage difference hetween each method and Retrain are reported.
Forgetting Sccnario
Method
Truck
Automobile
Predictions (%) (1)
Airplane
Ship
Ditt
Avg.
Class ('inekk)
Retrain
0.00 (0.00)
69.32 (0.00)
13.47 (0.00)
12.80 (0.00
0.00
13,96
Randum (109)
FT
Retrain
97.42 (1000)
96.1 (0 (0.70)
0.75 (0:A5)
1.23 (0000)
0.36 (0.00)
0.38 (0.000
(.4000000
(.32)
0.32
0.00
97.84 (0-42)
(0.24)
0.32 (0.06)
(0,02)
0.19
3.4 Theoretical Analysis
In this section, we provide a theoretical analysis showing that CoUn yields a higher misclassification rate on forget data compared to retain data. Achieving higher misclassification rate for forget data while maintaining a low misclassification rate for retain data contributes to both good forget quality and high model utility. Recall that K denotes the number of classes in the dataset. In particular, each
5

---

data sample belongs to one of classes C1, C2 ..... CK. For given transformation distribution T, let (ă,,, i2) min Et(in) ( || I2 denote the augmented distance between two images 21 and 82, where i, t' T. We consider that a (a, -uugmentaaio is being used. That is, for each class Ck, there exists a subset C% S Ck. such that both P E Cgl 2 o P(i E C&. and supi,, (81, i2) - ô hold, where
(o, 1]. Let μ4; denote the center of class Ct. We have μa = Biec, ( [fo. (z)]. Let Rie] denote the probability that, when a sample is drawn from the dataset, the distance between the encoder representations of its two augmented views exceeds e, with a small value of R(c] indicating good alignment in the representation space. We have
Rid
i
C
CR
sup 2=) E(E
llfo, fo., (x.()))) >
(5)
The following theorem shows the generalization ability of CL by providing upper bound for the misclassification rate: Theorem 1. For an I--Lipschitz feature extractor fo., given a (o, aaagmeentttiooon used in CL. if (o, 8, 1 2m 6,
M1 p. < min mn:l11 - pm'
(6)
2
holds for any pair of (1, with I * k. then the misclassification rate of classifier head he,, is Err(ho.)) s (1-o) + R(c), where (o, better alignment (i.e., smaller R(e)) and sharper concentration of augmented samples (i.e., larger for a given 8) result in a lower value of pma (a. 8, c).
2(1 o)) miny P(iECu) + P o + 2c). Note that
Proof. Refer to Theorem in 35
In an unlearning problem, the training data is divided into retain and forget data. In CoUn, the feature extractor fe.. and the classifier head he. are trained solely based on the samples in the retain data. From Theorem [T] can be inferred that a, 8, E, L are not dependent on the samples in the retain and forget data. Furthermore, in random forgetting scenario, f4r. k E {l,..., K} would be relatively similar for the forget and retain data. Thus, the only parameter that may have different
values for retain and forget data is R(e]. Let R and [e] characterize R[e] for retain and forget data, respectively. We have the following lemma:
Lemma 1. Considering a feature extractor Se, which is trained by both CL and supervised learning only on the retain data, we have [.[e] < Rule).
Proof. See Appendix D
From Theorem I we can see that the misclassification rate of classifier head he in CoUn is obtained as follows: Ehr( < (1 σ) + R, [c]. From Lemma we can see that R,[e] < [,, Thus, the upper-boune bound on the misclassification rate of classifier he. cannot be met for the forget data. This implies that CoUn provides higher misclassification rate on the forget data compared to the retain data. The misclassification rate on retain data remains low because supervised learning is applied to fe. using only retain data, Our experiments also confirm this.
Experiments
4.1 Experimental Setup
Datasets and Model Architectures. We evaluate CoUn on three datasets: CIFAR-10/100 (36) and TinyImageNet [37], using three model architectures: ResNet-18 (38), VGG-16 [39], and ViT [40]
Additional details regarding the model architectures are provided in Appendix E.I
Baselines. We compare CoUn with the following baselines: ① Retrain: Training from scratch on
retain data D,; FT: Fine-tuning the Original model A. on D, NegGrad+ (NeurIPS, 2023)
L -ssparse PPS 2023) @ SalUn (ICLR, 2024): and @ NoT (CVPR, 2025)

---

Table 2: Performance comparison of CoUn to the baseline methods with 10% random data removal. The
gap (A) and the (best) average gap between each method and the Retrain model are reported.
Dataset & Model
Method
Accuracy (%)
Efficacy (%)
Avg. Gap J
Comp. Cost
Retain (A P)
Unlearn (A ()
Test (A 1)
MIA (A J)
(PFLOP)) )
Retrain
100.00 (0,00)
4. (0.00)
94.67+
11,02- wa (0,00)
0.00
27.37
FT
w9 (0.01)
3,764 (1.05)
94,70= (0.03)
(.53))
0.65
6.32
NegGrad+
99.95= ouc (0.05)
(0.0))
94.32+
0.58
6.02
CIFAR-10
ResNet-18
8 Es-sparee
99 (0.03)
5.40
93.81+ (0.86)
10.97= (0.05)
0.38
6.92
SalUn
99.10= e2s (0.90.)
(.3.5)
a3.84:
20x
0.59
8,66
NoT
99.99 (0.01)
4.19
94.65 (0.02)
10.45. (0.567)
0.30
7.52
CoUn
99.99= (0.01))
4.12+
94.57a
0.25
8.02
Retrain
55.9% one (0.00)
24.26_
75.561 (0.00)
48.44
0.00
27.37
FT
09.97= (0.01)
=0.39=
76.75 (1.19
44.06= (4.388
3.36
7.22
NegGrad+
99.96: om (0.02)
30.09 (5.83)
75.46 (0.10.
47.72-
1.67
7.62
ResNet-18
UFAR-100
Ey-sparee
99.95= (0.03)
23.94= (0.322
oz
=2.11=
1.65
7.22
SalUn
98.55=
20.35- L2i (3.91)
72.02+ an (3.54))
52.37- uz (2.93)
2.95
5.69
NoT
99.97 (0.01)
17.99_ (6.227
76.27
44.28 umt (4.16)
2.79
CoUn
09.97
22.01 na (2.25)
72.88+ oz (2.668)
47.82 wx (0.62)
1.39
9.63
Retrain
99 (0.00)
56 16 (0.00)
63.82 (0.00
63.73 (0.00)
0,00
218.98
FT
99.98 wx (0.00)
32.76= 40)
6z.65
56.93 (6.80)
2.76
60.16
TinyInngeNet
NegGrad+
99.98= oxe (0.000
1z.01=xz(1185
64.68+ (0.60
2.15
80.21
ResNet-18
spssee
we
36.96- (0.800
62.62+ (1.20)
56.74 (6.999
2.25
60.16
SalUn
98.52
34,03- (2.13)
61.21a
67.72 (3)
2.55
51.08
NoT
99.98, ore (0.00))
35.64_ om (0.52)
63.662
56.08 ner (7.65)
2,08
80:21
CoUn
99.95:
(0.03)
35.10 (1.06)
63 (0.55)
57.57 (6.16.
1,95
80.21
Retrain
99.75 (0.01)
33 13.33_nsi (0.00)
67.07
40.69 (0.00)
0.00
15:38
FT
99.26 (0.49)
26.02 wm.02-xxx1.2 (1.21)
68.42+
(1.35)
35.51= wz (8.18)
3,56
3.42
CIFAR-100
VGG-16
2 Ey-spase
NegGrad+
99.27= (0.48)
94.92, 04..833)
35.44 wuz (2.21)
(6.27)
oz.54:am(1.5
un
40.67 (0.02)
35.31=ww(55.3
2.15
3.27
3.42
SalUn
92.65
33.00 uu (0.23)
oz.04:
42.55 14 (2.16)
3.13
2.79
NoT
30.11 um (3.12)
66.75.
(0..32)
36.47_ us (4.22)
2.81
4.28
CoUn:
99.82 om (0.07)
32.37 nus 86)
63.80 (3.2277)
39.64 oze (1.05)
1.31
Retrain
99.97
38.75_ (0.00)
ol (0.00) 0..00
61.75 we (0.00)
0.00
86.83
FT
wx.78=.u(0011))
10.83= (27.90)
61.12 (0.77)
31.50= 14.00=>xx33.33 (30.25)
14.78
5.79
CIFAR-100 100
VIT
& Ey-ppase
NegGrad+
99.88 (0.09)
99.32:
31.71=
45.26 (6.53)
46.49 63.33)
on.33)
55.00 (6.755)
3,98
6.09
11.58
14.47
SalUn
o9.18
2x.01=
oz.78:
69.24 (7..49))
4.03
5,10
NoT
99.89 (0.08)
20.29_1sx:
43.55_ (1.02
(188.20)
9.20
8.68
CoUn:
99.91 o0. (0.06)
36.81 ua (1.92)
as.49
53.92. 8az (7.83)
3,80
19:29
Evaluation Metrics. Following 8 10,, we evaluate the unlearning effectiveness and efficiency of CoUn using the following empirical metries: O Retain Accuracy (RA): Accuracy of the unlearned model On, on retain data D,. @ Unlearn Accuracy (UA): Measured as 1 1-FA, where Forget Accuracy (FA) is the accuracy of O., on forget data D... @ Test Accuracy (TA): J: Generalization performance of 0_ on test data. Q Membership Inference Attack (MIA): The efficacy of unlearning, evaluated using a confidence- based MIA predictor [9] 10 [3) applied to 0,, on D,, The MIA success rate reflects how effectively forget data is excluded from training. @ Computation Cost: Efficiency measured by the number of floating-point operations (FI.OPs) required to generate 0.. The model's utility is assessed using RA and TA, while its forget quality is evaluated using both UA and MIA metrics. A higher value in any individual metric (e.g., RA, UA, TA, or MIA) does not necessarily indicate better performance. An effective unlearning method minimizes the performance gap with the gold-standare Retrain model. Therefore, the performance (i.e., unlearning effectiveness) of an MU method is measured by the average gap:
Aug. Gap = 1/4(IRA - RA + UUA - UA + -A-ATAI IMIA - MIA"I), (7)
where denotes metrics for the Retrain model.
Implementation Details. For training the Original and Retrain models: we follow prior work 8 8. [0] by using an initial learning rate of 0.1, which is reduced by a factor of 10 at 50% and 75% of the total 182 training epochs. The batch size is set to 256. For unlearning: all MU methods are applied to 0,, for 50 epochs, using cosine learning rate scheduler with a minimum learning rate of 10-4 All reported results are averaged over 10 trials. Additional details can be found in
Appendix E.2
7

---

Table 3: Performance comparison of CoUn to the baseline methods with 50% random data removal. The
gap (A) and the (best) average gap between each method and the Retrain model are reported.
Dataset & Model
Method
Accuracy (%)
Efficacy (%)
Avg. Gap
Comp. Cost
Retain (A D
Unlearn (a !)
Test (A D
MIA (A J)
(PFL.OP)) J
Retrain
100.00= (0.00)
7.29
92.28
17.332
0.00
15,24
FT"
6.32 (0.97)
91.91 (0.37)
12.64g (4.69)
1,66
2.51
CIFAR-10 R8
NegGrad=
f spapaae
99.77..1000233)
100.00..x.00.00
9.02
(.062)
91.81gen
90.661 cl16 62)
9.05gam(1.97)
16.052 28)
0.90
1.22
4.19
5.58
SalUn
98. 3
(..55
92.37 (0.09)
16.40
7.82
NoT
99.98..m
(0.02)
5.95
92.84, cn (0.556
13.90
1.34
3.35
CoUn
(1.05)
(.111)x)(1110)
92.36 (0.08)
16.94
0.40
3.35
Retrain
99.98 (0.00)
(0000)
58.35 (0.00)
0.00
15.24
FT
99.98+ (0.00)
17.36=uuv(43
(4.16
50.60amm (7.75)
6.89
4.19
CIFAR-100
NegGrade
99
(0.00)
(3.57)
52.32aum
3.67
5.36
R8
{ f--sparse
(0.04)
32.26
(0.88)
67. 67.66 (0.75)
51.54am2/(6.8)
2.11
4.19
SalUn
95.61 (4.37)
25.45 (5.98.88
60.35
17.14
4,91
1.95
NoT
un (4,988
(7.97
43.82a (14,53)
5.32
2.01
CoUn
99 (0.00)
45 (2.0022)
65.60 (2.81)
15.99
1.30
5.58
Reirain
309 20.99..000..00
43.01 nas (0.00.
c40000
71.22 (0.00)
0.00
121.93
FT
un
66.28e (3.31
66
3.62
33.30
TiyImmggNeet
NegGral+
2s.99
47.62
(4..61)
cl.55
a6.43
2.74
33.50
ResNet-18
{ f--spare
99.99;
38.83 (4.118))
65.82
(5:40)
3.14
33.50
SalUnn
(3.59..3566.00)
44.74 (1.73)
45.53gcc((1177
a (0.81)
5.17
25.01
NoT'
39.99:
(8.97
66.23e
2.01
33.50
CaUn
99.98 (0.01)
(m.220
(5..5)
1.59
44.66
Retrain
99.65+ (00000
42.85 (0.00)
(0.00)
50.19 (0.00)
0.00
8.67
97.71
29,8220<(130)
(6.12
3998 (10.21)
7.80
1.43
NegGrad+
(4.14
oz.42.
c8.52
(6.61)
3.04
3.18
CIFAR-100 100
VGG-16
Pu-sparee
98.25 (x1.40)
62.76:11s5500
42.
5.79
1.91
Sallin
(1767
37.60
(5.2255
c7.30g
53.84; (3.65))
4.24
2.45
NoT
(5.23
34.64 (8.221)
(e.68
39.84 (100,5)
6.96
2.38
CoUn
(0.23)
4 (0.448
c5.15)
50.00 (0.19)
0.85
3.18
Retrain
99.98+ (0.00)
(0.00))
52.40s (0.00)
69.
(0.44s
0.00
48.35
10.91 ww.
cm.79
(41.6))))
21.05
1.6
NegGrad+
(0.60=
50.82:cae(1588
55.07
4.86
6.45
CIFAR-10U
VIT
fi-sparse
(1.18:39
47 47.30:
53.32s
44.22a==
13:95
8.06
SalUnn
(8.03=12(1.05)
43.64=2zz
39.46:
ux.49=
5.84
12.03
NoT
97.862
(1.8)
(3.11:)1((.11))
(20.6)
10.55
3.22
Coll
(0.21
(2.95=sso(.12))
(2.45=2((2.55
(10.4s
3.91
10,74
4.2 Results
Comparison with Baselines. We begin by presenting results for the 10% random forget data ratio. As shown in Table CoUn consistently outperforms state-of-the-art baselines, with computational costs that are either comparable or slightly higher. Key observations from Table 2 include the following: FT exhibits the highest average gap, indicating weakest performance. SalUn shows limited performance compared to other baselines, as the random labeling of forget data reduces the model's utility. While NegGrad+, fs-spas and NoT achieve the most competitive results, CoUn demonstrates superior performance across different datasets and model architectures. For example, using ResNet- 18. CoUn outperforms the best baseline by 16.7% on CIFAR- 10, 15.8% on CIFAR-100, and 6.3% on
Similarly, with VGG-16 and ViT on 110,R-100, CoUn achieves performance improvements of 53.4% and 4.5%, respectively. Even under a 50% random forget data ratio, CoUn still outperforms the baselines (Table 3). Comparisons with additional baselines and class-wise forgetting are provided in Appendix and Appendix F.2 respectively.
Integration of CoUn's CL Module into Baselines. To evaluate whether our proposed CL module empowers baseline methods, we conduct experiments integrating CoUn's 's CL module with competitive baselines. Figure 4 presents the average gap comparisons, with percentage improvements, across various datasets and model architectures for 10% and 50% forget data ratios. Detailed results can be found in Appendix [ Our results demonstrate substantial performance boost achieved by incorporating our CL module into the baselines. For example, using CIFAR-10 with ResNet- 18, integrating our CL module with NegGrad+, & fp-spar and NoT results in percentage improvements of 44.8% 42.1%, and 43.3% %3 respectively. under a 10% forget data ratio; while for a 50% forget ratio, improvements of 50.0%, 87.7%, and 73. 1 1% are achieved. Although the CL module introduces a slight increase in computational cost due to the additional model inference required for obtaining representations of the second sample view, Figure 5 shows that the performance enhancements remain significant even when the computation budgets are matched.
8

---

Sequential Unlearning. Figure presents results for scenarios where 10% random data is sequentially removed every 10 epochs, up to epochs. CoUn consistently outper- forms baseline methods across all five stages, with varying forget ratios. Additionally, inte- grating CoUn's CL module into baseline meth- ods further empowers their unlearning effec- tiveness.
CIPAR-12, BesNeG-18
Ferget Racto
Peepet
Withowert
1.00
6:05
33.0%
P7
Neytrrult
Nr1
Neykoadd
Nel
Netted
Matted
Firgee Badi
Teepue flatto
4.3 Ablation Study
Mithbert
Oapar
8.03
All ablation experiments are conducted using
2,00
CIFAR- 100, ResNet- 18, and forget ratio of
2.00
50%.
1.02
7%
NeyGrul
Neptiaad
Effect of Scaling Factor. The scaling factor X, defined in Equation (4) controls the relative contribution of the CE and CL losses. Fig-
Thytmagguler Beahee 18
3.52
Willowr
Foryue
With CL.
ure demonstrates the substantial impact of on CoUn's performance. Improper tuning lead to suboptimal results, emphasizing the importance of careful hyperparameter se- lection. For example, a high À reduces the influence of supervised learning in the objec- tive, causing retain representations to be less constrained within their respective clusters and more susceptible to cluster collisions. This, in turn, degrades model utility and compromises unlearning effectiveness.
2.50
2.00
116-44
2.50
2.00
0.30
Negtirud Mettwl
NeT
Neptzad- Mitted
NeT
Ferget Bodio 10%
Foeyot Ratio:
Bareo Cop
6.00
Wilh dL
Millawi CL
2,00
37.44
11
370
Effect of CL Temperature. Similar Sim- CLR 200 we investigate the influence of the CL temperature т, defined in Equation ( on unlearning effectiveness. Figure demon- strates that decreasing 7 generally improves the effectiveness of CL, and reduces the aver- age gap with the Retrain model. Nevertheless, excessively low values can negatively impact performance. As shown in Figure optimal performance is observed at r = 0.1, with per- formance deteriorating as deviates from this value. In line with previously reported find- ings from SimCLR, the results confirm that best performance emerges at low whereas performance gradually declines as T increases beyond the optimal value.
.2.1%%
Negtrade
Negorad-
NeT
CETAR 100,
31.2%
24.0%
5514%
Negdrate 6
NoT
PT
Ney
NoT
Modhvs
Metted
Figure 4: Percentage improvement from integrating CoUn's 's CL module into baseline methods. Incorporating our CL module consistently improves baseline unlearning performance compared the original MU methods (with- out CL). The performance improvements further increase with a 50% forget ratio.
Effect of CL Transformation Distribution. To isolate the impact of CL transformation distribu- tions TeL. we the supervised learning transformation TeE CHN, consistent with the transfor- mation used in the Retrain model. The details of the augmentation operations C, H, J. G. and N are provided Appendix E.3 Figure[Jdemonstrart that employing stronger transformations for CL (e.g.. CHJGN), compared to those used for supervised learning, can degrade performance. This degradation arises primarily from the formation of tighter clustering of forget representations, along with slower convergence caused by the additional complexity introduced by stronger transformations On the other hand, overly simple transformations (e.g., CN) may not sufficiently adjust the forget representations, which in turn result in suboptimal performance. Our experiments show that the best performance achieved when
Tes = TeL, i.e.. when T = CHN. This configuration also enables shared of representations from a single augmented image between CL and supervised learning, thereby

---

Porget Indio: 10%
Porget Ratio: 50%
22.5
Forget Ratio: 110-20-330-40--0
Negtäraé+ sparse NOT
NegläradeCL
IT+CL (Coin)
FT
NeyGrad-
NegGrad+CL
FT-CL (CoUn)
Avunqu Cup (4)
sparssecC
sparrs
spprrsseCL
NoT+GL
NoT
NoT+CL
12.5
10.0
5.0
10
5.0
10
20
30
40 50
Computation (PFLON))
Computtatio (FFLOPP)
Epoch
Figure 5: Performance comparison of MU methods CIFAR- 100 with ResNet-18, where 10% (lef)) and 50% (right) of training data are randomly selected as forget data. The best performance of cach method is reported. CoUn outperforms all basclines, and integrating its CL module empowers bascline performance. Although CL increases computational cost, the performance im- provement persists even same computational budget.
Figure 6: Sequential data removal. Ex- periments with CIFAR-I00, ResNet-18 18, up to 50% random forget data (10% of data is removed every 10 epochs), CoUn consis- tently outperforms baselines, and fur- ther empower baselines' performance when CoUn't 's CL module is integrated into them.
Calin
Average Cop (3
5
20
4-
1.5
2
CoUn
Col
0.0 0.2
11.0
0.00
0.8
1.0
CHN
CHIN
CHIN
CHION
%L
Figure 7: Effeet of scaling con- Figure 8: Effect of CL tempera- Figure 9: Effeet of CL transfor-
stant Properly tuning in Equa- ture T. Properly tuning r Equa- mation Ta Strong TeL. degrades tion (4) is essential for optimizing tion [1] is essential for optimizing performance, while simple TeL fails to sufficiently push representations.
CoUn's performance.
CoUn's performance.
reducing computational cost. The impact of strong sus simple CL transformations on forget representations further illustrated in Appendix F.4
3.0
Aparage Cla (
2.0
Effect of Batch Size Batch size impacts the perfor- mance of CoUn. Figure [io presents results for varying batch sizes, with the batch size for the Retrain model fixed at 256. Our findings indicate that the best per- formance for CoUn is achieved with batch size of 256, which matches the batch size used for the Retrain model.
1.0
0.0
12.8
256
1024
Figure 10: Efffect of batch size n. Different for CoUn, results in varying performance. Re- train batch size is set to 256.
5 Conclusion
We presented CoUn, a novel CL-based MU framework that enables effective unlearning by adjusting learned data representations based on semantic similarity. CoUn applies CL module retain data to adjust their representations and leverages the cluster collision issue to promote cluster overlap. Due to semantic similarity between retain and forget samples, forget representations are indirectly influenced in the same manner; thereby enhancing forget quality. To preserve utility, CoUn applies supervised learning retain data mitigate cluster collision for retain representations. Our results showed that CoUn consistently outperforms state-of-the-an MU baselines, and that integrating its CL module into existing baselines empowers their unlearning effectiveness.
References
[1] Alessandro Mantelero. The EU proposal for a general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review, 29(3):229-25 2013.
10

---

[2] Alessandro Achille, Michael Kearns, Carson Klingenberg, and Stefano Soatto. Al model disgorgement: Methods and choices. Proceedings of the National Academy of Sciences, 121(11))6230733442,, 2024.
[3] Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, and Anmin Fu, Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 2025.
[4] Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unlearning: A comprehensive survey and taxonomy. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[5] Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia. Machine unlearning: Solutions and challenges. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024.
[6] Jimmy Z Di, Jack Douglas, Jayadev Acharya, Gautam Kamath, and Ayush Sekhari. Hidden poison: Machine unlearning enables camouflaged poisoning attacks. In NeurIPS ML Safety Workshop, 2022.
[7] Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. IJCAL, volume page 19, 2022.
[8] Yasser H Khalil, Leo Brunswic, Soufiane Lamghari, Xu Li, Mahdi Beitollahi, and Xi Chen. Not: Federated unlearning via weight negation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 25759-25769, 2025.
[9] Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. SalUn: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In The Tweifth International Conference on Learning Representations, 2024.
[10] Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[11] Kairan Zhao, Meghdad Kurmanji. Acorge-Octavian Bärbulescu, Eleni Triantafillou, and Peter Triantafillou. What makes unlearning hard and what do about it. Advances in Neural Information Processing Systems, 37: 37:12293-12333, 2024.
[12] Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7766-7775, 2023.
[13] Liwei Song, Reza Shokri, and Prateek Mittal. Privacy risks of securing machine learning models against adversarial examples. Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, pages 241 241-257, 2019.
[14] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards bounded machine unlearning. Advances in neural information processing systems, 36: 36:1957- 1987, 2023.
[15] Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1516-11524, 2021.
[16] Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teach- ing induce forgetting? unlearning in deep networks using an incompetent teacher. Proceedings of the AAAI Conference on Artificial Intelligence, 3710)7221-7217, Jun. 2023.
[17] Ziyao Liu, Yu Jiang, Jiyuan Shen, Minyi Peng, Kwok- Yan Lam, Xingliang Yuan, and Xiaoning Liu. A survey on federated unlearning: Challenges, methods, and future directions. ACM Computing Surveys, 57(1):1-38, 2024.
[18] Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, and Dacheng Tao. A survey on superved learning: Algorithms, applications, and future trends. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.

---

[19] Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. A survey on contrastive self-supervised learning. Technologies, 9(1):2, 2020.
[20] Ting Chen, Simon Korbllith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[21] Julien Denize, Jaonary Rabarisoa, Astrid Orcesi, Romain Hérault, and Stéphane Canu. Similarity contrastive estimation for self-supervised soft contrastive learning. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 2706-27011111 2023.
[22] Chen Wei, Huiyu Wang. Wei Shen, and Alan Yuille. CO2: Consistent contrast for unsupervised visual representation learning. In International Conference on Learning Representations, 2021.
[23] Ching- Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, and Stefanie Jegelka. Debiased contrastive learning. Advances in neural information processing systems, 33:8765- 8775, 2020.
[24] Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling SGD: Understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P) P), pages 303-319, IEEE, 2022.
[25] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9304 9312, 2020.
[26] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019.
[27] Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, et al. Sanity checks for lottery tickets: Does your winning ticket really win the jackpot? Advances in Neural Information Processing Systems, 34:12749-12760, 2021.
[28] Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12043-12051, 2024.
[29] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in neural information processing systems, 0116827-6839 2020.
[30] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, arXiv preprint arXiv:2003.04297, 2020.
[31] Jure Zhontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In International conference on machine learning, pages 22310-12320. PMLR, 2021.
[32] Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: Marianceiivariance--vvaiii regular- ization for self-supervised learning. In International Conference Learning Representations, 2022.
[33] Qiuchen Zhang, Carl Yang, Jian Lou, Li Xiong, et al. Contrastive unlearning: A contrastive approach to machine unlearning. arXiv preprint arXiv:2401.10458. 2024.
[34] Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khande- parkar. A theoretical analysis of contrastive unsupervised representation learning. In Interna- tional Conference on Machine Learning, pages
PMLR, 2019.
[35] Weiran Huang, Mingyang Yi, Xuyang Zhao, and Zihao Jiang. Towards the generalization of contrastive self-supervised learning. In The Eleventh International Conference on Learning Representations, 2023.
12

---

[36] Alex Krizhevsky, Vinod Nair, Geoffrey Hinton, et al. The CIFAR-10 dataset. online: http://www.
cs. toronto. edu/krizeiffar html. 55(5):2, 2014.
[37] Ya Le and Xuan Yang. Tiny ImageNet visual recognition challenge. CS 23/N, 7(7):3, 2015.
[38] Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
[39] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
[40] Seung Hoon Lee, Seunghyun Lee, and Byung Cheol Song. Vision transformer for small-size datasets. arXiv preprint arXiv:2112. 3492, 2021.
[41] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:18661-18673, 2020.
[42] Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural computation, 9(1):1 1997.
[43] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Proc. Advances in Neural Inf. Process. Syst. (NeurIPS), Vancouver, Canada, Dec. 2019.
[44] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. ImageNet large scale visual recognition challenge. International journal of computer vision, 1:5:211-2555 2015.
13

---

We provide more details and results about our work in the appendices. Here are the contents:
Appendix
More discussion on related work.
Appendix
Pscudo-Code and PyTorch implementation of CoUn.
Appendix
Additional empirical analysis.
Appendix
Proof of Lemma
Appendix
More details about experimental and implementation settings.
Appendix
Additional experiment results.
Appendix
Broader impacts of our proposed method.
Appendix
Limitations of our proposed method.
A Related Work: Further Details
Contrastive Learning. Zhang et [33 applies supervised CL 41 to push forget samples away from retain samples of the same cluster and pull them closer to retain samples of different clusters. Essentially, [33 pushes representations away from positive samples and toward negative ones. This approach requires that forget and retain samples from the same cluster to be included in each batch. Moreover, under this definition there are no positive samples in class-wise unlearning, thus [3] modifies the objective to only pull forget samples toward retain samples from different clusters. However, this approach aims to push forget samples outside their clusters, potentially harming model utility. As shown in Figure] [ the goal of unlearning not misclassify forget samples some be correctly classified to maintain model performance. Additionally, 33 requires access to forget data.
In contrast, CoUn follows the way how Retrain model classifies forget data, which is based on 30 31 32] to achieve the same goal. Self-supervised CL uses augmentations to generate positive views, instead of using samples from different clusters the positive views. The use of augmented samples as positives and the remaining as negatives allows three advantages: access to class labels is not required during CL, e do not need to guarantee that samples from different clusters need to exist in the batch, and @ it does not force samples out their original clusters. Lastly, CoUn does not require access to forget data.
semantic similarity. CoUn utilizes selersue CL [18 19 20
29
B
CoUn Algorithm
B.1
Pseudo-Code
Algorithm details our proposed unlearning method, which leverages CL and supervised learning.
Algorithm 1 CoUn Algorithm
Input: Original model 0,, transformation distribution T, and retain data D,
Hyper-parameter: Learning rate 17, temperature and scaling factor À Output: Unlearned model 0,.
1: 0.. + -,
2: for epoch e 1, ..... E do
3: for each batch Y) E D, do
4:
Sample transformations t T
5:
X. X' = t(T),
6:
Z, Z' (X), fx.
7:
Y ho.(
8:
La is obtained from Equation using 2, Z' using Y. Y
9:
LCE is obtained from Equation
10:
0, f O,, Vo. (LeF + XCe..
11:
end for
12:
end for
13: return 0,,
14

---

B.2 PyTorch Code
This section provides the PyTorch implementation of CoUn.
import torch
from torch import from torch. nn import functional
features None hook_ft fn = lambda module, output: globals O ..- wet.ttsmee......... (aamm___"('aaae ('features', output)
def coun (model, layer, optimizer, retain,loader, transform, lambda_scale, temp): A>pppy contrastive learning for unlearning using only retain data
10
Args:
11
model: The original model that needs to be unlearned
layer: The penultimate layer for extracting embeddings
optimizer: The optimizer train the model
14
retain_loader: The dataloader containing retain data
15
transform: The transformation to be applied to input images lambda_scale, A scaling constant for the CL loss temp: The temperature to applied in the CL loss returns uunlearned model' ".
16
18
19
20
layer layer.register reggstterfoovaaeeee hook (hook_fn) Attach hook at penultimate layer
for images, targets in retain_loader: # Load retain data batch_size int (images shape
22
23
24
# Create two views of images
imagesl, images2 transform (images), transform (images)
26
27
Get model outputs and extract embeddings for images1 outputs model (images1) featuresi features view (batch_size -1)
28
31
# Extract embeddings for images2 model (images2) features.viexxbatthh.
32
33
features2 features. view (iatch -1)
# Compute supervised learning loss for images1 supervised_loss mn. CrossEntropyLoss (outputs, targets)
36
37
# Compute contrastive learning using the two views of embeddings
39
target torch (bathh unsqueeze (0)
40
intra_mask (torch forch.eq(target. target. float ())
41
cos_sim_ij = w_cosin__similarae (featuresi [: None :],
features2 (None, :,,Nooe,,;::.. :;: dim=-1)
43
cos_sim_ij torch div (aav(ccossssii temp)
44
log-prob_ij cos_sim_il - torch. log (torch (i)
sum (1, keepdim=True))
mean intra_mask. sum (1)
(intra_mask log.prob_ij) sum (1) /
46
47
cos _os_sim_ji = F. featuresi [aatures[[oone... [None ;;;], dim=-1)
(features2 None ;]
com_aim_ji torch div sim_ temp)
49
1og-prob_j1 cos_sim_ji torch log (torch exp (oppccosssn_i)))
sum (1, xeepdim-True 50
mean laan__lo-proooppp
(intra_mask log_prob_j1). sum (1) /
intra_mast sum
15

---

constraative_lor loss (mean__og_prob_ppp mean ()
tean_log-probbpos. mean())
53
54
loss supervised_ loss + lambda_ s con.tlsstsntttiiee..uue
optimizer zero_grad ()
37
loss backward optimizer step ()
58
return model
C Further Details on Empirical Analysis
Representation Space of Original Model. Figure illustrates the representation space of the Original model trained on CIFAR-I0 using ResNet-18. Since it trained on both retain and forget data, the model achieves perfect accuracy both.
Statistical Comparisons. We provide statistical comparison between forget representations and retain clusters, we grouped the forget samples by their class labels and, for each group, com- puted the average Euclidean distance (L2) from its samples to all retain class centroids. This yielded a per-class dis- tance profile showing how far forget rep- resentations lie from each retain cluster. To enable comparison across different models, we then applied normalization on each group's 's averaged distances. Table summarizes the results for CIFAR-IO. ResNet-18, and the statistics for 'truck' forget samples 10% random forgetting (same setup as Table TT. The findings show that forget representations in CoUn are consistently closer to semantically similar retain clusters, and more importantly. CoUn achieves distance statistics that are closer to those the Retrain model compared to other baselines. The smaller the distance means higher semantic similarity. From Table we can see that 'truck' samples have the highest semantic similarity with 'automobile'.
Table L2 distances of forget 'truck' to retain centroids. The most semantically similar clusters to 'truck' samples are presented. Experiments conducted using CIFAR- 10 and ResNet- 18. The difference (A) and the (hest) average difference between each method and Retrain reported.
Forgetting Scenario
Method
L2- (A
Avg.
Automobile
Airplane
Ship
DifT. J
Original
Class ("truck")
Retrain
0.90 (0.00)
0.96 (0.00)
0.95 (0.00)
0.00
(0.04)
(0.04)
Colla
0,87 (0.03)
0.96 (0.00)
0.93 (0.02)
0.017
Prediction-Leve Results. Tables and additional prediction-level results for a single forget class across different baselines and datasets. The Original model has 100% accuracy on the forget 'truck' samples since these samples are part of its training data. Furthermore, baselines based on label manipulation or weight perturbation produce predictions somewhat similar to Retrain, but their misclassification CoUn more effectively redirects forget samples toward semantically similar clusters, thereby yielding prediction distributions that are closer to those of the Retrain model.
less concentrated on semantically related classes. By comparison,
D Proof of Lemma 1
We have
[l (x)) (x')) < L|[ x x [|,
sup (()) et' f'
|l (x) fo. (x')ll L
sup x'er''
x
x
(9)
t
where inequality 8 is obtained due to the L-Lipschitz assumption for feature extractor fo. and inequality ( is obtained by taking supremum from both sides of inequality (8)
Training fo,, on the retain data tends to converge to flat minima, i.e., regions in parameter space where the loss landscape is broad and has low curvature 42). Consequently, flat minima correspond
16

---

Original
Airplane
Bird
Deer
Frog
Ship
Automobile
Cat
Dog
Horse
Truck
Figure 11: Representation space of the Original model. The Original model is trained on the entire CIFAR-10 training data (i.e., union of retain and forget data) using ResNet-18. There are no misclassifications for either retain forget samples since the model was trained them. A single visualization of the Original's model representation space shown for both class-wise and random scenarios, this model serves the checkpoint for both scenarios. Dots denote training samples, where each heing correctly clustered into the corresponding class.
to functions whose outputs vary only gently under small perturbations around the training points (i.e., they have a low local Lipschitz constant at those points). However, there is no guarantee that, at these flat minima, the loss landscape remains flat with respect to the forget data. Therefore, in general, after convergence we have
Lr
Lu L,
(10)
where I.p and Lu denote the Lipschitz constants evaluated on the retain and forget data, respectively. Based on Equation ( and inequality we have
R P ie
Cx
nD,
sup
| x' x'1>> e/L,
(11)
k=1
==t(i), (() 1,7'T
and
K Uc k=1
R = P ie
Ch
nD
sup == a' (t) t,8'
æ x x1|1> c/bu
(12)
In particular, Equation computes the probability that, for images in the retain data, suP==1()), (()) [x x' exceeds e/L,, while Equation 12 computes the probability that, for ('''T images in the forget data, suP===((), x' (t) || - æ'll exceeds e/Lu. Considering inequality (TO). we 1, TT have Ln, L/L.. Since random forgetting the samples are drawn I.I.., we have R S (e) This follows because the threshold e/T.,. used to compute R,,[e] is lower than the threshold e/ T.r used compute Ry[e].
E
Further Implementation Details
E.1
Model Architectures
For VGG-16 16 39 we use a 1024-dimensional encoder head. For the ViT model, we adopt patch size of 4 x 4, an embedding dimension of 512, MLP hidden dimension of 1024, 12 attention heads of size 64, and depth of 6 transformer layers. Both dropout and embedding dropout are set to O.
17

---

Table 5: Predictions of forget 'truck' samples based on most semantically similar classes, The experiments conducted using CIFAR- 10 and ResNet-18. The difference (A) and the (best) average difference between each method and Retrain are reported.
Forgetting Scenario
Predictions (%) (A
Avg. Diff. |
Method
Truck
Automobile
Airplane
Ship
Original Retrain
100.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00
0.00 (0.00)
69.32 (0.00)
13.47 (0.00)
12.60 (0.00)
0.00
FT
0.00 (0.00)
70.29 (0.97)
12.38 (1.09)
13.12
0.65
Class
NegGrad+
0.00 (0.00)
43.56 (25.76)
16.03 (2.56)
17.58 (4.98)
8.32
('truck')
-p-spee
0,00 (0.00)
65.70 (3.62)
9.84 (3.63)
18.25 (5.65)
3.23
SalUn
0.00 (0.00)
66.05 (3.27)
16.67 (3.20)
16.71 (4.11)
2.65
NoT
0.00 (0.00)
68.12 (1.20)
12.43 (1.04)
15.03 (2.43)
1.17
CoUn
0.00 (0.00)
69.60 (0.28)
13.96 (0.49)
13.13 (0.53)
0.33
Original
100.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00
Retrain
97.42 (0.00)
1.23 (0.00)
0.38 (0.00)
0.40 (0.00)
0.00
FT
98.12 (0.70)
0.75 (0.48)
0.32 (0.08)
0.32
Random (10%)
NegGrad+
97.86 (0.44)
0.97 (0.26)
0.42 (0.04)
0.30 (0.10)
0.21
sparre
98.06 (0.64)
0.85 (0.38)
0.30 (0.08)
0.38 (0.02)
0.28
SalUn
96.76 (0.66)
0.88 (0.35)
0.34 (0.04)
0.34 (0.06)
0.28
NoT
97.98 (0.56)
0.89 (0.34)
0.40 (0.02)
0,32 (0.08)
0.25
CoUn
97.84 (0.42)
0.99 (0.24)
0.32 (0.06)
0.42 (0.02)
0.19
Table Predictions forget 'man' samples based on most semantically similar classes. The experiment conducted using CIFAR-100 100 and VGG-16. The difference (A) and the (best) average difference between each method and Retrain are reported.
Forgetting Scenario
Predictions (%) (A 4)
Avg. DifT. |
Method
Man
Woman
Boy
Baby
Original Retrain
100.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00
49.53 (0.00)
15.18 (0.00)
10.06 (0.00)
3.42 (0.00)
0.00
FT
60.53 (11.00)
9.11 (6.07)
9.68 (0.38)
3.61 (0.19)
4.41
Random (10%)
NegGrad+
41.94 (7.59)
13.47 (1.71)
12.71 (2.65)
3.32
f-spars
57.69 (8.16)
13.47 (1.71)
10.06 (0.00)
2.85 (0.57)
2.61
SalUn
(5.1635.63)
13.52 (1.66)
9.68 (0.38)
2.63 (0.79)
2.12
NoT
51.23 (1.70)
14.61 (0.57)
9.68 (0.38)
3.61 (0.19)
0.71
CoUn
50.15 (0.62)
14.67 (0.51)
10.09 (0.03)
3.28 (0.14)
0.33
E.2 Training and Unlearning Configurations
Original Model 0, Training. Following 10], we train original models for all different datasets and model architectures for a total of 182 epochs. The batch size is set to 256. An SGD optimizer used with an initial learning rate of 0.1 and a multi-step learning rate scheduler that reduces the learning rate by factor of 10 50% and 75% of the training epochs. Momentum is 0.9, and weight decay is set to 5 x 10-1 The transformation distribution used for data augmentation is linear warmup phase for the first 75 epochs.
described in Appendix H.3
For VGG-16,
Unlearned Model 6, Training. For unlearning, all methods are run for 50 epochs. We used the SGD optimizer with learning rate tuned within the range of [0.01. 0.1] for each MU method. A cosine annealing learning rate scheduler is used with a minimum learning rate set to 1 x 10-4 Momentum is set to 0.9, and weight decay is set to 10-1 The transformation distribution used for data augmentation described in Appendix Additional details for each MU method are listed follows:
BadT: The temperature set to 1.
SSD: The weight selection is tuned in the interval [0.1, 100), while the dampening constant is tuned in the interval [0.1, 5].
18

---

NegGrad+: The 3 hyperparameter is tuned in the interval [0.95, 0.9999].
SCRUB: The temperature is set to I, The number of maximization steps is tuned in the interval [1, 10J. Both 7 and a are tuned in the interval [0.1, 3).
CU: The contrastive scaling constant tuned within the interval [0.1, 2] and the temperature is tuned within the interval of (0, 0.2]. The constant is tuned within the interval [1, 5J.
l -s-spase:: The & regularization parameter is tuned in the interval [10-4, 10-"]. f regularization applied for epochs, except for sequential forgetting experiments, where is applied for 2 epochs and then reapplied every 10 epochs for total of 50 epochs.
SalUn: The mask threshold is tuned within the interval of [0.1, 1, 1.0].
NoT: For all model architectures, the first CNN layer (index: 0) negated. For VIT 40)
the positional representations and the second patch representation layer are negated (indices:
0 and 4).
CoUn: The scaling constant À is tuned within the interval [0.1, 6], and the CL. temperature is tuned within the interval of (0, 0.3].
All experiments are implemented using the PyTorch platform 43 and run using NVIDIA Tesla V100
GPUs.
E.3 Data Augmentation
In our experiments, the following operations are applied sequentially to augment images:
Random cropping (C): Output image size of 32 x 32 for CIFAR-10/100 and 64 x 64 for TinylmageNet, with padding of 4 each image border.
Random horizontal flip (H): Applied with a probability 0.5.
Color normalization (N): Applied using mean values (0.4914, 0.4822, 0.4465) and standard deviations (0.2023, 0.1994, 0.2010).
However, in some ablation experiments, the following operations added between horizontal flip and color normalization to augment images:
Random color jitter (J): Applied with probability of 0.8. Brightness, contrast, saturation, and hue are set to 0.8, 0.8, 0.8, and 0.2, respectively.
Random grayscale (G): Applied with probability of 0.2.
F
Further Results
F1 Random Forgetting: Forget Ratio 10% (Additional Baselines)
comparisons of CoUn with additional baseline methods: BadT (AAAL, 2023) SSD (AAAL, 2024), SCRUB (NeurIPS, 2023) and CU (33 using random data forgetting with a 10% forget ratio. CoUn consistently achieves superior performance compared to all basclines. Since BadT, SSD, SCRUB, and CU do not perform better than other baselines, we did not include them under different settings. Further, BadT and SCRUB demand higher computational resources due their reliance on two teacher models (Original and Random) to guide the unlearned model.
F.2 Class-wise Forgetting
Table 8 presents the results for class-wise forgetting. All baselines, including FT, exhibit minimal performance gaps with the Retrain model, suggesting that class-wise forgetting is relatively easy and can be effectively addressed using just FT (i.e., through catastrophic forgetting). This observation aligns with the findings from [II], which shows that lower entanglement between retain and forget data simplifies the unlearning task, making class-wise forgetting easier than random forgetting. This trend is further illustrated by the similarity between the representations of and CoUn Figure] and those of the Retrain model in Figure Nevertheless, CoUn achieves competitive results across all baselines. In this paper, the majority our experiments focus on the more challenging unlearning scenarios (i.c., random forgetting).
19

---

Table 7: Performance comparison of CoUn additional baseline methods with 10% random data removal. The gap (A) and the (best) average gap between each method and the Retrain model are reported.
Dutuset
Accurncy (%)
Effiewcy (%)
Avg.
Comp. Cost
& Model
Method
Refain (A ())
Unlearn (A ()
Test (A ()
MIA (A
Gap
(PFLOPS)
Retrain
100,00+ sw (0.00)
4.8 (0.00)
94.67=a2(0000)
11.02 (0.00)
0.00
27.37
FT
99.992 oon (1101)
3.76g43:
94.70g00(0.05
9.51gou(13]
0.63
6.32
BadT
99,94; (0.06)
0.07 (4.74)
44 (0.62)
10.10
1.58
9.89
SSD
100.00= swe (0.00)
0.02+cue(4.99)
us.80=
0.62=ax
3.83
0.06
CIFAR-10
NegGrud+
99 99.95= (0.005
4.82a:22
04.
9.09
0.58
6.02
ResNet- 18
SCRUB
99,97 (0.03)
3.931 (0.88)
94.61
9.53Lax(1449
0.62
8,93
CU
99.32= (0.68)
5,48+c1x
(0.48)
11.63ezzz
0.61
3,36
fi-spurse
-0.97= om (0.03)
93.81= 5.40=:
(0.86)
10.97
0.38
6.92
SalUn
99.10= (0,90)
4.31+ (0.50)
93.84=az
(1.15)
0.59
8.66
NoT
99.99 om (0.01)
4.19 (0.62)
94.65=02(0002)
10 (0.57)
0.30
7.52
CoUn
99.99 oon (0.01)
4.12 (0.69)
(0.10))
10.81 (0.21)
0.25
8.02
Table 8: Performance comparison of CoUn to the baseline methods with class- 'truck" samples removal. The gap (A) and the (best) average gap between each method and the Retrain model reported.
Dataset & Model
Method
Accuracy (%)
Effieacy (%)
Avg. Gap J
Comp. Cost
Retain (A J)
Unlearn (A) ()
Test (A) ))
MIA (2 ()
(PFLOPS)
Retrain
100.000.aee (0.00)
100.00, om (0.00)
95. 95.14 (0.00)
100.00, now (0.00))
0.00
27.37
FT
99.972 (0.D3)
100:00 (0.000
100.00: (0.100)
0.14
6.02
NegGrad+
99.98 oz (0.02)
100.00 om (0.00)
a (0.04)
100.00 ww. (0.000)
0.02
9.03
CIFAR-10 R8
Epaspas
100.00g
100.00 (0.00)
95. a5.
100.00, now (0.000
SalUn
100.00 (0.00)
100.00 om (0.00)
95. a
100.00s noe (0.00)
0.01
8.66
NoT
100.00 (0.00)
100.00= om (0.00)
an
100.00= now (0.00)
0.00
6.02
CoUn
100.00: 60c (0.00)
100.00: (0.00)
93.18:s2000004
100.00 (0.00)
0.01
9.03
F33 Integration of CoUn's CL Module with Baselines: Detailed Results
Tables 9 and 10 provide detailed results for integrating CoUn's CL module with existing baselines for 10% and 50% random data forgetting, respectively. Results show that adding our CL module significantly empowers the performance of existing MU methods, as measured by the average gap.
F4 Effect of CL Transformation on Representation Space
Huang et al. 35 provides a theoretical framework for understanding the generalization ability of CL, emphasizing the role of data augmentation and representation properties. They show that stronger transformations]]] in CL yield more compact and well-separated clusters in the representation space. These insights particularly relevant our observations on the effect of transformation strength in the context of MU.
In MU, the goal of CL is not to enhance clustering, but rather to weaken the clustering of forget data. Looser clustering allows their representations to be pushed toward clusters of other retain representations that are semantically similar to the forget representations, even if those clusters are differrent from the original ones the forget data, Therefore, strong augmentations reduce the likelihood of cluster overlap, even in the presence of false negative samples.
As illustrated Figure 9. weaker transformations lead to more effective unlearning. This is because they result in less tightly clustered representations, allowing forget data to overlap with retain data from other classes, particularity in the absence of supervised signals for forget data. Figure [2 further shows that stronger CL transformations (c.g., Ta. = CHJGN) produce tighter clusters of forget data compared to weaker ones (e-g., Ta, = CHN), consistent with the findings of [35). However, from an MU perspective, such tight clustering impedes unlearning. Hence, using simpler CL transformations generally enhances unlearning effectiveness. This inverse relationship between representation compactness and unlearning efficacy is reinforced by the results in Figure 9
'Transformations drawn from distribution combining multiple data augmentations.
20

---

CoUn (Trr CHN To. CHN)
CoUn (Tc: CHN & CHIGN)
Airplane
Bird
Deer
Frog
Ship
Automobile
Dog
Horse
Truck
Figure 12: Effect of CL transformation on forget data representations. t-SNE visualizations of forget data
representations extracted from the penultimate layer of 0w on CIFAR- 10 with ResNet-18 under a 50% forget data ratio. Left: CoUn with simple CL transformation (TeL = CHN). Right: CoUn with strong CL transformation (Ta CHJGN). The transformation for supervised learning is fixed at Tcu CHN. The CHJGN transformation applics a sequence of operations: crop, horizontal flip, color jitter, grayscale, and color normalization. Stronger transformations lead to tighter clustering of forget data representations, reducing unlearning effectiveness. Additionally, tighter clustering leads to less cluster overlap compared to weaker clustering (from simpler Tc,..
Table 9: Performance comparison when CoUn's 's contrastive learning (CL) module is integrated into basclines under 10% % random data forgetting, The gap (A) and the (best) average gap between each method and the Retrain model are reported.
Dataset & Model
Method
CL
Accuracy (%)
Efficacy (%)
Avg. Gap
Comp. Cost (PFLOIS) I
Retuin (A ()
Unlearn (1 J)
Test (A. J)
MIA (A J)
Retruin
100.00, (0.000
4.81
94.67 (0.00)
11.02 (0.00)
0.00
27.37
FT
90.99 one (0.0))
a.
94.70 (0.03)
0.65
6.32
/
99.99 um (0.01)
4.12a (0.69)
94,57= t0. 10)
10.81
0.25
8.02
CIFAR-I0 ResNet-18
NegGrad+
/
99.98 ox (0.02)
99.95 (0.05)
5.21:-12
4.82
94.55 (0.12)
94.32 (0.35)
(1.9))
10.30
(0.72)
0.58
0.32
6.02
7.52
X
99.97_ (0.03)
a.40g
93.81 (0.86)
10.97 (0.05)
0.38
6.92
spuree
4. ow..00
94.48 (0.
10.80 (0.22)
0.22
7.62
NoT
99.99 ome (0:01)
4.19 (0.62)
94.65
(0.02)
10:45 (0.57)
0.30
7.52
100.00+ (0.00)
4 4,20 (0.61)
94.65 (0.02)
11 18.8840m (0.06)
0.17
7.52
Retrain
99.98 om (0:00)
24 34.76-01000000
75.50
48 44 (1100)
0100
27.37
X
99.97 ose (0:01)
cm.39_cm(1..7
(1.19
44.061 nos (4.33)
3.36
7.22
FT
/
99 09 (0.01)
22.01.ea( (2.25)
72.88
(2.668
47 47.82a (0.62)
1.39
9.63
CIFAR-100 ResNct-18
NegGrad+
99.96_ on (0.02)
99.98 os (0.00)
22.48 ese (1.7))
M0.09 (5.83)
74.52 (1.00)
15.46
(0.10)
47.72
(0.72)
0.74
1.67
7.62
12.53
99.95_ (0:03)
23.94_ (0100.2)
74.95
(0.61)
1.65
7.22
99.96 ox (0.02)
24.57
exn (0.31)
44.46..2(1..0
44.62 ou (3.82)
1.31
9.63
NoT
99.97 (0.01)
17.9 ow (6.27
16.27
(0.711
44,28
(4.16
2.79
7.22
99.97 one (0.01)
21.
(2.733)
73.61
46. n4t (1.60)
1.57
9.63
Reirain
99.98_ on (0.00)
16
(0.0000
12822e000.000
63.73am (0.00)
0.00
218.98
FT
99.98 ose (0.00)
32.76 ond (3.40
64.65 6:0 (0.33)
56.93 nos (0)
2.76
60.16
99.95- on (0:03)
(..00)
63.27
57.57 16)
1.95
TinyImageNet R8
NegGrad+
99.88 ose (0.00)
38.01
(1.35
64.68
(0.86)
57.84 (5.889
211
80.21
99.96 (0.02)
36.68- (0.52)
63.73 (0.09)
57.87: ooe (5 86)
1.62
100.27
4 spppre
99.96 on (0:02)
99.97 ase (0.01)
36.33 en (0.17)
36.06 (0
62.62
63.27
e22 (0.55)
56.961 (3
56 (6.99)
2.25
1.88
60.16
80.21
NoT
99.88 (0000)
35.64 (0.52)
63.602 (0. (6)
26 (3633
2.08
80.21
99. 94_ or (0:04)
36. 19 (0.03)
63.06
(0.76)
56
56.00nn (7.03)
1.97
80.21
Retrain
99 75 on (0.00)
33.23 6023360000000 (0.00)
67.07
(0.00)
40.692 (0.00)
0.00
15:58
99.26 ous (0.49)
26.02 c25 (1.211
68.42
38.31
3,36
3:42
FT
/
99.82 on (0.07)
02.77.cm00886
63.80
39.64amm
(1.05)
1.31
5.71
CIFAR- 100) VGG--6
NegGrad+
X
94.92 (4.83)
08.43=
34.65= (1.42)
35.44_ (2.21)
65.98
65.54gem(1.55
40.67
39.12:
(0.02)
2.15
1.35
5.71
3.42
E spsree
x
9m.27= (0.48)
26.96 (6.277
68.0) (0.994)
35.31anst
3.27
3.42
98.94_ (0.81)
o9.70
22. (0.95))
2.17
5.71
X
46 58)
(3.11
66.7521
3647a11
2.81
4,28
NoT
98.72_ oo
35.25 (2.02)
61.90g
(w..17
40.51 (O. 18)
2.10
4.57
Retrain
w0.77 (0.00)
38.752 (0.00)
61.89
(0.00)
61
78278 (11.0)
0.001
86.83
FT
/
09.912
om (9.19
10.83_ (27
16.112
61.12..
56.49
(5.40)
53 53.92 83)
31.50
(30.55)
3.80
5.79
19.29
CIFAR-100
NegGrad+
99.88__ (0.09)
45.26_ (6.53)
59.38
(2.56)
(6.
3,98
11.58
VIT
/
99.96 eu (0.01)
38.71 (000022)
59.07 (2.82)
52.381na((.37)
3,05
24.12
x
99.32_ (0.65)
31.71 (7.02)
63.3gem((144
46 46.49 (15.266
6.09
14.47
99.63 oa (0.34)
39.07 (0.34.
(3.78)
30.26
3.97
19.29
NoT
/
x 99.99 one (008)
99.93 (0.04)
20,29 1s (1844)
6m.90.om
(1.20)
on.50 (3.04)
43.53 128
2.29
9.20
8.68
19.29
21

---

Table 10: Performance comparison when CoUn's contrastive learning (CL) module is integrated into baselines under 50% random data forgetting. The gap (A) and the (best) average gap between each method and the Retrain model are reported.
Dataset Model
Method
CL.
Accuracy (%)
Emecacy (%)
Avg. Gap |
Comp, Cost
Retain (A 1)
Unlearn (A) L)
Test (3 J)
MIA (A J)
(PFLOP)) )
Retrain
7.29 100.00=...00.00)
92. (0.000
17 (0.00)
0.00
15.24
FT
X
99. 99.38g ox (0.62)
6.32g (0.97)
91.91gen (0.37)
12.64gos (4.69)
1.66
2,51
99.97- om (0.03)
6.19
92.36.
16.94; (0.399
0.40
3.35
CIFAR-I0 ResNet-18
NegGrad+
X
100.00+ (0.00)
(0..00)
4.65
(0.75)
(0.16)
93.81gen (0.86)
93.45.e>x
9.05gem(1.97)
11.45aiss
0.45
0,90
5.58
5.58
4p-aass
99.77
100.000..0000000
(0:23)
6.91
(0.388
90.66
92.30 i (0.02)
(26
17.14 (0. (0.19)
0.15
1.22
4.19
4.47
NoT
99.98 oz (0:02)
5.953
92.84 (0.55)
13.90 (3.43)
1.34
3.35
99.99 oue (0:01)
(z.45)
91.64
17.00 use (0.33)
0.36
5.58
Retrain
99.98 oue (0.00)
(o..0
68.41 (0.00)
58.
38.35 (0.00)
0.00
15.24
FT
99.98_ on (0.00)
99.98 (0.00)
31.43- 13s 10.02)
c7.36g
(2.81) 65.60
74.16
(5.75)
55.99, un (2.36)
6.89
1.30
4.19
5.58
CIFAR-100 ResNet- 18
NegGrad
99.98- (0.00)
99.8 on (0.00)
31.55= (0.10.1)
26.32=ca1
68.34 (0.07)
52.32 (6:3)
56
34.34: (2.01)
0.55
3.67
6.70
3.36
4p-spase
9994 one (0.04)
22.26-c210033
67.66 e (0.75)
34 (6.81)
2.11
4.19
99.98 (0.00)
29.93
(1.48)
68. 68.35 (0.06)
54.84 (3.511 (3.51)
1.26
5,58
NoT
V
99.98 ur (0.00)
98.64 (4)
26.43-cos (4.988
26.01
69.01=.* (0.600
67.97 (0.44)
43.82
54. 30 (4.05)
(14.33)
5.32
2.51
2.01
5.58
Retrain
99.99 om (0.00)
(3.00=c2100.00)
57.28
21 (0.00)
0.00
121.93
FT
99.99 (0:00)
99.98 ex (0.01)
o6.78
43.21-1
(0.39...(3331))
55.75
66
66.59
28 (4.94)
(4.63)
3.62
1.59
33.50
44.66
TinylmageNet ResNet-18
NegGrad+
99.99_ (0.00)
47.622
58.85
66.43amm (4. 79)
2.74
33.50
98.9..... ou (0.01)
(2.7))
w...6)
66.77
(4.45)
1.96
55.83
99.99 om (0.00)
38.33 (4.18)
60.25
63.82
wan (5:40)
3.14
33.30
99.9 (0.05)
12.11 (0.900
$7.02s
65.924 (5 (5.30)
1.63
66
NoT
/
X
99.99 ose (0.00)
99.99-mm (0.00)
42.1
(0.900
58.27
57.15 (0.13)
em (0.99)
66.23a (4.99)
65 65.91
2.01
1.58
44 66
33.50
Retrain
99.65 oas (0.00)
42.85 (0.00)
57,70 (0.00)
50. 19 (0.00)
0.00
8.67
X
97. 94)
29.82_es((3300)
63.72g
(
39 n0 (10,21)
7.80
FT
/
99.88 ore (0.23)
42.37 0m (0.488
$5.19 ess (2.51)
50.001 ous (0.19)
0.85
3.18
CIFAR-100 VGG-16
NegGrad+
/
05.34g
96.70 one (2.95)
43.4220
om..2.)
58.522
59.14g
46.61 (3.58)
33.3 (148)
2.01
3.18
4s-paasc
98.25 (1.01.00
54,24
(8.6111
62.76 (5.008
42.
(8.00)
3.79
1.91
99.34- (0.3)
42.78-uox00.0
55.06=a
46.41
(3.78)
1.70
3.18
NoT
94.23 Tee (3.42)
54,04 (8:221
61.58 40 (3.88)
39.34: ( (10.35)
6.96
2.38
99.55 cas (0.10)
42.78- (0.07)
$5.17 (2.53)
48 48.88 (1.31)))
1.00
3.18
Retrain
99.98 cae (0.00)
48.07 eas (0.00)
s2.40 (0.00)
69.54: (0.00)
0.00
48.35
FT
94.71
(0:27)
45.95
10.91=am(37.10
49.45 (2.95)
36.79 (4.399
28 18: (41.6
59.24:
(100.330
21.05
3,91
L.61
10.74
CIFAR-100 100
NegGrade
X
99.30= eas (0.68
(2
55.07 30.82
4,86
6.45
VIT
14.14-200 (0.84)
oM
50.95.em(1.47)
57.96ame (11.57)
4.04
5,37
Ex-sparse
/
95.83 use (4.15)
71.18 can (2).88
49.36.em(1.99)
cot.77
50.87
33.32
44..2
53.662
(15.88)
(23.32)
13,95
5,71
8.06
10.74
NoT
97.86_
(1..1==((1626)
(5.51
48.85
(20.69)
10.58
3.22
/ 99.93 wu (0.05)
50.21=cm(2.14)
(2.90
65.802 (3.74)
2.21
10.74
G Broader Impacts
Research in machine unlearning holds significant societal value by empowering users to request the removal of their data from models and enhancing model safety and fairness through the climination of harmful or outdated information. This work is exploratory in nature- propose an approximate unlearning framework that uses contrastive learning to push forget representations into clusters of other retain samples that are semantically similar to the forget samples. Due to cluster collision, these retain samples may belong to clusters different from the original clusters of the forget samples. Given the nature of our approach, we do not foresee any direct negative societal impacts stemming from this work.
H Limitations and Future Work
While CoUn marks a step forward in leveraging CL for unlearning, its evaluation is currently limited to vision-basee classification tasks on relatively small datasets (CIFAR- 10/100 [36] and TinylmageNet
Future work could explore its scalability to larger datasets such as ImageNet (44) and its applicability to other domains, including natural language processing. Additionally, although CoUn uses InfoNCE as the contrastive loss, investigating alternative sulf-super objectives, such as
37
cross-correlation-baee losses [3] 32) on unlearning performance presents an interesting direction
22

---

for future research. Moreover, like prior methods 8 9 10,, CoUn relies heavily on access to retain data for effective unlearning. Investigating performance of approximate unlearning methods under partial access to retain data would be an important direction for future research. Finally, while CL introduces additional computational overhead due to augmented data and extra forward passes, CoUn consistently outperforms baselines even when computational budgets are matched. Future work could explore strategies to reduce this cost without sacrificing performance.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: In the abstract and introduction, we first introduce the relevant concepts of machine unlearning and summarize existing methods, analyzing their shortcomings. Notably, existing methods manipulate labels or perturb model weights for unlearning. Most methods also require access to forget data. We focus on addressing the machine unlearning problem by studying how a retrained model behaves with respect forget data. Our proposed method CoUn, which employs contrastive learning and supervised learning, aims to close the performance gap with retraining from scratch. Additionally, we analyze CoUn effectiveness across various scenarios including theoretical and experimental evaluation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Appendix Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer:
[Yes]
Justification: Please Section 3.4 and Appendix D
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer Appendix E.2 for information on reproducing the main experi- mental results of the paper including the baselines.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results.
25

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We uploaded CoUn code in the supplementary material and will be made publicly available. All data used in this study are publicly available. The baseline methods compared in the paper open source and results be reproduced directly after following the implementation details provided in Appendix E.2
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (httpss:/niis.cce)
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co httpssscccpbblccuuuueeeee hebbl////guddsssddeeuuuimiii (ooodeSumisssioniiii ssiliii for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Section 4.1 and Appendix H.2 for more experimental setting/details, along with the submitted code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification; We report standard deviations of the results obtained after 10 independent runs (see Tables 2
and T0p.
Guidelines:
The answer NA means that the paper does not include experiments.
26

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer:[ Yes]
Justification: Please see Appendix Also, the compute cost for all experiments are provided in Tables 2
and 10
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that NeurlPS Code of Ethics ttctps://neurr: ct.ppuulic/thiiiccculeellllllll
didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification:
We have read the NeurlPS Code of Ethics and confirm that the paper complies.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please Appendix G
27

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cited the original paper that produced datasets see Section
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
28

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new assets in our paper
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
29

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt:p//eearrrppcc//oooeeeeeeee/ /ct////ooeeeeeccc//22////.. for what should or should not be described.
30


Reviews:
[{"review_id": "yoY6jOwmqN", "reviewer": "Reviewer_z7vM", "summary": "This paper tackles the problem of machine unlearning by incorporating contrastive learning. Specifically, the proposed method applies contrastive learning to adjust forget representations and supervised learning to maintain retain representations within their clusters sharing similar semantics. Experimental results show the effectiveness of the proposed method across small image datasets with CNNs and a ViT.", "questions": "Please address concerns in Weaknesses above.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n+ Machine unlearning is an important topic for deployment of machine learning models, and this work aims to make a step forward in this field.\n\n+ The idea is simple and supported with some theoretical results.\n\n\n**Weaknesses**\n\n- I agree with the importance of machine unlearning, but skeptical whether the current evaluation strategy is proper to assess machine unlearning methods. it is possible that the information about \"training data to forget\" still remain in the new model but its accuracy matches with the gold-standard retrain model by another reason, e.g., losing other information. In other words, accuracy matching does not imply that two models exhibit exactly the same knowledge, and it might not satisfy GDPR; rather, prediction matching would be more suitable.\n\n- Basically, I don't think classification is a proper task to assess forgetting of specific data; rather, reconstruction for specific data would be more suitable. For example, it is possible that forgetting some data allows the new model to have more capacity to better keep the information of other training data, resulting in better classification accuracy. In this case, unlearning would increase classification accuracy.\n\n- Contrastive learning has been applied to many fields in machine learning, so the usage of contrastive learning without much adaptation as done in this paper has a limited novelty.\n\n- Theorem 1 is originated from Huang et al. [34], which considers pure contrastive learning settings; however, the proposed method learns through both contrastive learning and supervised learning, so Theorem 1 might not hold.\n\n- No forgetting measure in experiments, which has commonly been employed in prior works.\n\n- Performance gain seems to be consistent but not so significant, raising concerns on the trade-off between the additional cost introduced by contrastive learning versus the gain.\n\n- Experiments are done in small image datasets, so its scalability and generalizability to other datasets/domains are not clear.\n\n- Experimental results might not be reproducible as the description on experimental settings is not sufficient and the provided code is only for a portion of experiments. For example, I couldn't find the configuration for ViT experiments, and \"ViT\" is not enough to figure out and replicate its specification.\n\n- A similar work by [Lee et al.] is not cited/discussed.\n\n[Lee et al.] Contrastive Unlearning: A Contrastive Approach to Machine Unlearning. arXiv 2024.\n\n- typo: NIPS -> NeurIPS", "was_revised": true, "final_justification": "The authors successfully addressed my concerns and promised to release their code for reproducibility. Hence, I raise my score accordingly.", "rebuttal": "We thank the reviewer for their constructive feedback. We are pleased the reviewer finds our paper **simple**, **supported by theoretical analysis**, **demonstrating effectiveness through experimental results**, and acknowledges that **our work advances an important topic**. Below, we address their specific comments and questions.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W1:** We thank the reviewer for the opportunity to clarify this point. \n1. Indeed, evaluating machine unlearning methods requires metrics **beyond accuracy alone**, as accuracy matching alone does not guarantee genuine forgetting or GDPR compliance. We would like to emphasize that our evaluation follows standard practices established in recent machine unlearning literature (e.g., [8] @ CVPR 2025, [9] @ ICLR 2024, [11] @ NeurIPS 2024, [10] @ NeurIPS 2023). Consistent with these studies, we report **Membership Inference Attack (MIA) alongside retain accuracy, unlearn accuracy, and test accuracy**. The MIA success rate specifically measures how effectively forget data samples are identified as non-training in the unlearned model. Note: unlearn accuracy = 1 - forget accuracy. Importantly, our dataset is explicitly partitioned into retain and forget subsets. Since, we evaluate both retain and forget accuracies then accuracy matching can not happen due to unintended loss of retain data (i.e., \"losing other information\"), as this would lead to a drop in retain accuracy. However, our results show that retain accuracy is maintained, while forget accuracy drops, indicating that the loss of information is localized to the forget data. This pattern is further confirmed by the MIA scores, which consistently show reduced influence of the forget data post-unlearning. \n\n2. We also concur that **prediction-level comparison** offers a finer-grained assessment of model behavior. Accordingly, please refer to **Table 1 in our paper**, where we incorporated prediction-level comparison metrics to evaluate alignment with the gold-standard Retrain and FT baseline. Additionally, we provide extended results in the Table below showing prediction-level comparisons for a single forget class across additional baselines and datasets. These results consistently demonstrate that CoUn achieves the lowest average prediction divergence compared to the gold-standard Retrain model. This supports our claim that CoUn effectively mimics the clustering behavior of the Retrain model with respect to forget data based on semantic similarity better than the baselines.\n\n**Table: Predictions of forget 'truck' samples based on most semantically similar classes. The difference ($\\Delta$) and the (best) average difference between each method and Retrain are reported.**\n|Forgetting Scenario|Method|Truck|Automobile|Airplane|Ship|Avg. Diff. ↓|\n|-|-|-|-|-|-|-|\n|Random (10%), CIFAR-10, ResNet-18|Original|100.00|0.00|0.00|0.00|-|\n||Retrain|97.42 (0.00)|1.23 (0.00)|0.38 (0.00)|0.40 (0.00)|0.00|\n||FT|98.12 (0.70)|0.75 (0.48)|0.36 (0.02)|0.32 (0.08)|0.32|\n||NegGrad+|97.86 (0.44)|0.97 (0.26)|0.42 (0.04)|0.30 (0.10)|0.21|\n||$\\ell_1$-sparse|98.06 (0.64)|0.85 (0.38)|0.30 (0.08)| 0.38 (0.02)|0.28|\n||SaUn|96.76 (0.66)|0.88 (0.35)| 0.34 (0.04)|0.34 (0.06)| 0.28|\n||NoT|97.88 (0.56)|0.89 (0.34)| 0.40 (0.02)| 0.32 (0.08)| 0.25|\n||**CoUn (Ours)**|97.84 (0.42)| 0.99 (0.24)| 0.32 (0.06)|0.42 (0.02)|**0.19**|\n||\n|Random (10%), CIFAR-100, VGG-16|Original| 100.00|0.00|0.00|0.00|-|\n||Retrain|49.53 (0.00)|15.18 (0.00)|10.06 (0.00)|3.42 (0.00)|0.00|\n||FT|60.53 (11.00)|9.11 (6.07)|9.68 (0.38) |3.61 (0.19)|4.41|\n||NegGrad+|41.94 (7.59)|13.47 (1.71)|12.71 (2.65)|4.74 (1.32)|3.32|\n||$\\ell_1$-sparse|57.69 (8.16)|13.47 (1.71)|10.06 (0.00)|2.85 (0.57)|2.61|\n||SaUn|55.16 (5.63)|13.52 (1.66)| 9.68 (0.38)|2.63 (0.79)|2.12|\n||NoT|51.23 (1.70)|14.61 (0.57)|9.68 (0.38)|3.61 (0.19)|0.71|\n||**CoUn (Ours)**|50.15 (0.62)|14.67 (0.51)|10.09 (0.03)|3.28 (0.14)|**0.33**|\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W2:** We thank the reviewer for suggesting this potential area for exploration. While we agree that reconstruction-based evaluations may provide additional insights into data-specific forgetting, classification remains widely adopted in the top-tier machine unlearning literature [8,9,10,11]. \n\nImportantly, proper unlearning should not result in improved generalization (i.e., higher classification test accuracy). Although not explicitly reported in the main results, we confirm that the test accuracy of the Original model is higher than that of the Retrain model, as expected. This drop reflects the reduced training set used by the Retrain model (i.e., retain data only), which naturally leads to a decrease in accuracy. Moreover, even if removing some data allows a model to better utilize its capacity and improve performance on the remaining data, this benefit would apply equally to both the Retrain and the unlearned models—since both are trained without access to the forget data. Also, in unlearning evaluations, we focus on the relative difference between the unlearned model and the Retrain model, rather than absolute improvements in classification accuracy. \n\nIntroducing reconstruction-based metrics targeting specific samples could offer a complementary view of forgetting and is an interesting direction. Due to time constraint, we will explore and discuss this direction in future work.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W3:** Even though contrastive learning (CL) is widely adopted, its application to machine unlearning remains novel. Our contribution lies in **strategically integrating CL with supervised learning**, **utilizing only retain data** to guide the unlearned model toward the behavior of the Retrain model **through the lens of semantic similarity**. Furthermore, **our modular CL design allows easy incorporation into existing and future unlearning methods**, enhancing alignment fidelity broadly. \n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W4:** We thank the reviewer for raising this point. It is true that Theorem 1 is derived under the assumption that the feature extractor is trained using contrastive learning, and the classifier head is fine-tuned on a downstream task. However, when the feature extractor is trained using a combination of contrastive and supervised learning, the misclassification rate bound provided in Theorem 1 can still serve as a valid upper bound. Therefore, the inequality stated in Line 258 continues to hold, and our justification in Lines 257–262 remains applicable.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W5:** We evaluate forgetting using two widely adopted metrics: unlearn accuracy: indicating model performance on forget data, and MIA: reflecting how effectively forget data is excluded from training. These metrics align with what recent top-tier unlearning literature [8,9,10,11] have used in their empirical evaluation. Detailed explanations of these metrics appear in Lines 48–52 and the Evaluation Metrics section.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W6:** We would like to refer the reviewer to Figure 5 in our paper, demonstrating that under equal computational budgets, our CoUn method consistently and significantly outperforms baseline methods. The Table below provides exact improvements from Figure 5 (50% forget ratio) under matched computational costs, emphasizing CoUn's superior performance without additional computational expense.\n\n**Table: Improvement in average gap of CoUn and baselines relative to FT, under matched computational cost.**\n|Method|Avg. Gap ($\\downarrow$)|Improvement ($\\uparrow$)|Comp. cost (PFLOPs) ($\\downarrow$)|\n|-|-|-|-|\n|FT|6.89|-|4.19|\n|NegGrad+|3.70|+46.30%|5.58|\n|$\\ell_1$-sparse |2.11|+69.38%|4.19|\n|NoT|5.66|+17.85%|4.19|\n|**CoUn (Ours)**|**1.87**|**+72.86%**|**4.13**|\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W7:** We acknowledged this limitation explicitly in Appendix G (Limitations) of our paper. We stated that \"our evaluation is limited to relatively small datasets (CIFAR-10/100, TinyImageNet). Future work could explore larger datasets like ImageNet.\" Additionally, [8] @ CVPR 2025 mentioned in their Limitations section that they evaluated on small datasets. \n\nMoreover, our current experimental setup on CIFAR-10/100 and TinyImageNet aligns with [11] @ NeurIPS 2024, which used the same datasets (CIFAR-10/100 and TinyImageNet) for their experiments. Lastly, these datasets align to some degree with [8,10], which also evaluated on CIFAR-10/100. Thus, our dataset selection ensures that our evaluation setup is consistent with established benchmarks, enabling meaningful comparisons with state-of-the-art methods.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W8:** We thank the reviewer for identifying this. We would like to note that the complete ViT model configuration is provided in our supplementary code (utils/model.py, Line 15):  \"patch_size=(4, 4), num_classes=num_classes, dim=512, mlp_dim=1024, dim_head=64, depth=6, heads=12, dropout=0.1, emb_dropout=0.1\". \n\nWe acknowledge this omission in the paper and we will update it accordingly. Full training details appear in Lines 283-288 and in Appendix C. ViT-specific configurations for baselines, such as NoT, are mentioned explicitly (see Line 621). The ViT architecture file is included in our supplementary code, and we also cited the ViT paper (see Line 266).\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W9:** We would like to clarify that the work by [Lee et al.] is indeed cited in our paper as Reference [32]. We discussed, compared, and contrasted their approach in the Related Work section, with further detail provided in Appendix A. Moreover, we present in Table 4 a direct experimental comparison between CU [32] and our method CoUn, demonstrating that CoUn significantly outperforms CU.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W10:** Thank you for highlighting this typo. We will correct it."}, {"review_id": "92D3YzgRsu", "reviewer": "Reviewer_CMM6", "summary": "This paper proposes to perform both contrastive learning and supervised learning on retain data to improve the unlearning effects.", "questions": "See the weaknesses.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The proposed method is simple yet effective. It incorporates contrastive loss in the finetuning loss and presents clear performance improvement compared to standard fine-tuning.\n\n2. Experiments are comprehensive.\n\n3. The proposed method surpasses baseline methods in multiple unlearning scenarios.\n\n4. Notations, figures, tables, and theorems are very clear. The paper is well-organized.\n\t\n\n\nWeaknesses:\n\n1. Sec 3.2 emphasizes that conducting CL on retain data indirectly pushes forget representations toward high-similarity retain clusters, and this can improve forget quality. Here, more empirical results are needed to support this motivation and assumption. Specifically, for a model A trained on the whole dataset and a model B retrained on only retain data, what are the differences in the similarity between forget representations and the \"high-similarity\" retain clusters? For comparison, there should be two more charts of the Model A before unlearning in Figure 1. Moreover, a rigid table of statistical values is more desired than Figures 1 and 3 for a valid presentation. \n\n2. Furthermore, it's unclear why pushing forget representations toward other retain samples can improve forget quality, which is defined as forget accuracy or MIA efficacy. Since approximate unlearning sets retraining as a standard, improving forget quality should refer to a closer gap in forget accuracy or MIA efficacy. Regarding this, will the action of pushing forget representations increase or decrease these two metrics? Is this action imitating Retraining? Otherwise,  how do you determine whether there will be an improvement?\n\n3. The theoretical analysis says that CoUn yields a higher misclassification rate on forget data compared to retain data. This property is not surprising since the CoUn performs both contrastive learning and supervised learning on retain data, and the forget data are treated as test data. However, this analysis does not support the reason why CoUn provides better empirical unlearning performance, since from Table 2 all other baseline methods have higher retain accuracy and lower forget accuracy. The authentic significance of such theoretical analysis is quite questionable.\n\n4. Although CoUn consistently shows the best Avg. Gap in the figures and tables, different settings yield different second-best methods. For example, recent methods like Salun and NoT sometimes outperform NegGrad, while in other cases they do not. Yet, each new method claims to outperform previous ones when introduced. This inconsistency in performance raises concerns about the fairness and accuracy of reproducing baseline methods.\n\n5. In some cases CoUn's superiority in Avg. The gap is marginal, but it costs more computation resources.", "was_revised": true, "final_justification": "After the authors' rebuttal, I raised my rating from 3 to 4.", "rebuttal": "We thank the reviewer for their valuable feedback. We are pleased they find our proposed method **simple yet effective**, presenting a **clear performance improvement**, and that the **experiments are comprehensive**. We also appreciate the reviewer's acknowledgment of our **clear notations, figures, tables, and theorems**, and that the **paper is well-organized**. Below, we address their specific comments and questions.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W1:** We thank the reviewer for their suggestion. Following the reviewer’s suggestion, we have generated the t-SNE visualization for the Original model trained on the complete dataset (retain + forget data). This Original model serves as a starting checkpoint for both class-wise and random data forgetting scenarios; thus, its the same for both scenarios. We will add the t-SNE visualization of the Original model in the paper. Visually, this plot resembles Figures 1 or 3 (left) but with 10 clusters and no misclassifications. From another perspective, it also resembles the random forgetting scenario plots (right), again without misclassifications.\n\nIn response to the reviewer’s suggestion to provide a more rigid statistical comparison between forget representations and retain clusters, we grouped the forget samples by their true class labels and, for each group, computed the average Euclidean distance (L2) from its samples to all retain class centroids. This yielded a per-class distance profile showing how far forget representations lie from each retain cluster. To enable comparison across different models, we then applied normalization on each group’s averaged distances. The Table below summarizes the results for CIFAR-10, ResNet-18, and the statistics for 'truck' forget samples in a 10% random forgetting (same setup as Table 1 in our paper). The findings show that forget representations in CoUn are consistently closer to semantically similar retain clusters, and more importantly, CoUn achieves distance statistics that are closer to those of the Retrain model compared to other baselines. The smaller the distance means higher semantic similarity. From the Table below, we can see that 'truck' samples have the highest semantic similarity with 'automobile'. These results will be reflected in the paper.\n\n**Table: L2 distances of forget 'truck' to retain centroids. The most semantically similar clusters to 'truck' samples are presented. Experiments conducted using CIFAR-10 and ResNet-18. The difference ($\\Delta$) and the (best) average difference between each method and Retrain are reported.**\n|Forgetting Scenario|Method|Automobile|Airplane|Ship|Avg. Diff. ↓|\n|-|-|-|-|-|-|\n|Random (10%)|Original|0.93|0.97|0.96|-|\n||Retrain|0.90 (0.00)|0.96 (0.00)| 0.95 (0.00)|0.000|\n||FT|0.86 (0.04)|0.94 (0.02)|0.91 (0.04)|0.033|\n||**CoUn (Ours)**|0.87 (0.03)|0.96 (0.00)|0.93 (0.02)|**0.017**|\n\nFurthermore, we kindly refer the reviewer to Table 1 in our paper, which presents part of the confusion matrix for forget samples under both forgetting scenarios, comparing predictions by the Retrain model (oracle) and our CoUn method. We can see that the predictions are based on semantic similarity. Smaller differences between their prediction distributions indicate better forgetting quality by CoUn. For example, Table 1 shows the top-4 predictions for forget samples labeled 'truck'. While Table 1 does not include the Original model or other baselines, we have conducted those evaluations and will include them in the paper. Below we show the top-4 predictions for the forget 'truck' samples. The Original model classifies all 'truck' forget samples correctly (100%) as expected, as it was trained on them. In contrast, Retrain and other methods (FT, NegGrad+, $\\ell_1$-sparse, SalUn, NoT, CoUn), under random forgetting, predict most forget samples as 'truck', 'automobile', 'airplane', and 'ship', based on semantic similarity with retain clusters. CoUn consistently shows the smallest deviation from Retrain's predictions, highlighting its superior imitation of the Retrain model. To clarity, if a 'truck' forget sample is predicted as another label, then it means its representation was pushed toward a semantically similar retain cluster that is different than the forget's sample cluster; yet, if it is still predicted as 'truck', then it means the forget representation was pushed to a retain representation within the same cluster.\n\n**Table: Predictions of forget 'truck' samples based on most semantically similar classes. The difference ($\\Delta$) and the (best) average difference between each method and Retrain are reported.**\n|Forgetting Scenario|Method|Truck|Automobile|Airplane|Ship|Avg. Diff. ↓|\n|-|-|-|-|-|-|-|\n|Random (10%), CIFAR-10, ResNet-18|Original|100.00|0.00|0.00|0.00|-|\n||Retrain|97.42 (0.00)|1.23 (0.00)|0.38 (0.00)|0.40 (0.00)|0.00|\n||FT|98.12 (0.70)|0.75 (0.48)|0.36 (0.02)|0.32 (0.08)|0.32|\n||NegGrad+|97.86 (0.44)|0.97 (0.26)|0.42 (0.04)|0.30 (0.10)|0.21|\n||$\\ell_1$-sparse|98.06 (0.64)|0.85 (0.38)|0.30 (0.08)| 0.38 (0.02)|0.28|\n||SaUn|96.76 (0.66)|0.88 (0.35)| 0.34 (0.04)|0.34 (0.06)| 0.28|\n||NoT|97.88 (0.56)|0.89 (0.34)| 0.40 (0.02)| 0.32 (0.08)| 0.25|\n||**CoUn (Ours)**|97.84 (0.42)| 0.99 (0.24)| 0.32 (0.06)|0.42 (0.02)|**0.19**|\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W2:** We thank the reviewer for raising this point. Forget quality is typically assessed using forget accuracy: the drop in accuracy on forget data, and MIA (Membership Inference Attack) efficacy: how indistinguishable forget samples become from non-training samples. \n\nIn Figure 1 and Table 1, we showed that the Retrain model classifies forget samples into clusters of retain samples that exhibit the highest semantic similarity to them. Now, since in CoUn we push forget representations towards other retain representations that exhibit highest semantic similarity to them, this results in an unlearned model that imitates the behavior of Retrain model in classifying forget samples.\n\nWhen forget samples are pushed into semantically similar retain clusters, this reduces their distinctiveness, making them behave more like non-training sample (improving MIA). At the same time, pushing forget samples away from their original clusters increases misclassification rates, thereby decreasing forget accuracy (or equivalently, increasing unlearn accuracy, defined as 1 - forget accuracy).\n\nTherefore, improving forget quality involves increasing MIA efficacy and decreasing forget accuracy. Our CoUn method achieves this by imitating the Retrain model's semantic-based clustering of forget data, outperforming other methods without compromising utility.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W3:** We thank the reviewer for the opportunity to clarify this point.\n\n1. Our theoretical analysis aims to formally characterize this intuitive behavior and offer a provable foundation for reasoning about misclassification on forget data. Specifically, our analysis quantifies how feature representations learned from retain data fail to generalize to forget data. This helps ensure that forgetting is not only effective empirically but also theoretically justified.\n\n2. We believe that theoretical analysis of the misclassification rate contributes to the overall assessment of unlearning performance. In particular, as mentioned in Lines 231–234, achieving a higher misclassification rate on forget data while maintaining a low misclassification rate on retain data supports both good forget quality and high model utility. A low misclassification rate on retain data indicates high retain accuracy, while a higher misclassification rate on forget data corresponds to lower forget accuracy and thus improved unlearn accuracy.\n\n3. In fact, Table 2 **does not** indicate that \"all other baseline methods have higher retain accuracy and lower forget accuracy\" compared to CoUn. On the contrary, CoUn consistently achieves retain accuracy and unlearn accuracy (i.e., 1 - forget accuracy) closer to the Retrain model, which represents the ideal target performance.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W4:** We acknowledge this variability, which aligns with results reported in [11] (NeurIPS 2024). For example, rankings among NegGrad+, $\\ell_1$-sparse, and SalUn frequently shift across evaluations.  In [11, Table 12] using CIFAR-10 and ResNet-18, the ranking based on average gap of methods differs across evaluations as follows:\n\n* [11, Table 12a]: NegGrad+ < $\\ell_1$-sparse < SalUn. \n* [11, Table 12b]: $\\ell_1$-sparse < NegGrad+ < SalUn. \n* [11, Table 12c]: SalUn < NegGrad+ < $\\ell_1$-sparse.\n\nSimilarly, [11, Table 10] shows:\n* [11, Table 10a]: NegGrad+ < SalUn < $\\ell_1$-sparse. \n* [11, Table 10b]: NegGrad+ < SalUn < $\\ell_1$-sparse.\n* [11, Table 10c]: $\\ell_1$-sparse < SalUn < NegGrad+.\n\nWe carefully tuned the hyperparameters for all baseline methods, including CoUn, to ensure they yield the best performance. Further, we compared all methods under equal computational budgets to ensure experimental fairness (see Figure 5).\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to W5:** We kindly refer the reviewer to Figure 5 in our paper, where we demonstrated that CoUn consistently outperforms baseline methods under equal computational budgets. For clarity, we also include the Table below, which presents the exact percentage improvements from Figure 5 (at 50% forget ratio) under matched computational cost. This highlights that the contrastive learning component substantially improves performance without additional computational cost.\n\n**Table: Percentage improvement in Avg. Gap of CoUn and baselines relative to FT, under matched computational cost.**\n|Method| Avg. Gap($\\downarrow$) | Improvement ($\\uparrow$) | Comp. cost (PFLOPs) ($\\downarrow$)|\n|-|-|-|-|\n|FT|6.89|-| 4.19|\n|NegGrad+ |3.70|+46.30%|5.58|\n|$\\ell_1$-sparse|2.11|+69.38%|4.19|\n|NoT|5.66| +17.85%|4.19|\n|**CoUn (Ours)**|**1.87**|**+72.86%**|**4.13**|"}, {"review_id": "cVVELhiESC", "reviewer": "Reviewer_1SXk", "summary": "This paper proposed applying the contrastive learning method assisted with supervised learning method exclusively on retain data to conduct machine unlearning tasks. The experimental results demonstrated that the proposed approach achieved superior performance equivalence w.r.t. the gold-standard retrain model.", "questions": "1. It would be good to conduct experiments to understand how likely CL would suffer from excessive or insufficient perturbation, for example, with longer or less training epochs.\n2. How sensitive is the $\\lambda$ choice? Are the reported results in Table 2 using the same $\\lambda$? If not, how much performance loss would be introduced if using the same $\\lambda$ (from a careless hyperparameter tuning).\n3. Does the reported computation cost consider the hyperparameter tuning costs for both baseline methods and proposed methods?\n4. What would happen if the supervised term were completely neglected?\n5. In Line 362, it claims that the transformation should stay the same for CL and supervised learning when $CHN$ is used. Is this also the case for other transformation distribution choices?", "limitations": "There is no obvious limitation or negative societal impact.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength\n1. Although contrastive learning is a known technique, it is relatively new to apply this technique in machine unlearning tasks, which relieves the constraints for forget data access, and achieve good model utility. In addition, the reviewer empirically agrees that the forget data representation is not necessarily pushed away from its original cluster, especially in a random erasing setting. The proposed approach may inspire future research in this area for better privacy and security in ML space, given the richness in contrastive learning.\n2. The empirical experimental study provides a clear view of the performance equivalence compared with retrain model with multiple state-of-arts machine unlearning methods.\n3. The theoretical analysis presents a lower misclassification upper bound for retain data, implying higher misclassification rate over forget data using contrastive learning.\n\nWeakness\n1. The proposed approach is a vanilla contrastive learning method, which can also be considered as model weight perturbation categories in general. There is no specific mechanism to prevent excessive or insufficient perturbation.\n2. More ablation studies can be conducted to understand the behavior of the proposed approach and hyperparameter choices. Please check the question section for more details.", "was_revised": true, "final_justification": "The reviewer thinks the contrastive learning approach applied in machine unlearning is well justified in the paper, and certain theoretical results are provided. The proposed method seems to be very straightforward, and easy to follow, which may inspire future research in the area. The experimentation side may need more ablation studies to understand how each component contributes to the overall performance. Although the rebuttal response addresses partial concerns, full results should be provided in the later revisions. Therefore, the reviewer thinks the paper should be rated as \"4: borderline accept\".", "rebuttal": "We thank the reviewer for their constructive comments and for highlighting that our idea is **relatively new**, **eliminates the constraint for forget data access**, **achieves good model utility**, and **inspires future research** in privacy and security within ML. Below, we address their specific comments and questions.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q1 & W1:** We thank the reviewer for their suggestion. Our CoUn's objective function, CE + λ·CL, is designed such that the cross-entropy (CE) term preserves retain representations within their clusters, while the contrastive learning (CL) component pushes forget representations toward semantically similar retain clusters. Extending training allows the model to further refine these adjustments, reducing the average gap and improving unlearning performance. In contrast, shorter training may result in insufficient refinement, leading to suboptimal unlearning. In both cases the clustering of the retain representations will be preserved due to CE.\n\nIn response to the reviewer’s suggestion, we provide experimental results in the Table below, which show that reducing training time increases the average gap, while longer training improves it.\n\nAdditionally, we would like to clarify that the terms \"excessive\" and \"insufficient\" perturbation were used in reference to related works that directly modify model weights, which can lead to excessive or insufficient **weight perturbation** if not carefully controlled. For example, NoT [8] applies layer-wise weight negation. In contrast, CoUn does not directly perturb weights; instead, the CL component operates on the data representations. In our revision, we will explicitly clarify this distinction by referring to perturbation specifically in the context of **weight perturbation**.  Lastly, in Figures 7 and 8 of the paper, we demonstrate that tuning the hyperparameters will enhance the effectiveness of our proposed CoUn, thus controlling the indirect weight perturbation via learning the new forget representation adjustments toward semantically similar retain samples.\n\n**Table: Average gap for CIFAR-10 and CIFAR-100 using ResNet-18 at different training epochs.**\n| Epoch | CIFAR-10 | CIFAR-100 |\n| -------- | -------- | -------- |\n| 25    | 0.54     | 3.56     |\n| 50    | 0.25     | 1.39     |\n| 75    | 0.21     | 1.30     |\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q2 & W2:** We thank the reviewer for pointing this out. In Table 2 of our paper, we provide results with tuned $\\lambda$ for each experiment. Similarly, for all other baselines we tuned their hyperparameters to get the best performance. This is consistent with widely adopted practice in the literature where they report the outcome of an experiment after fine-tuning its hyperparameters.\n\nFollowing the reviewer’s suggestion, we present results using a fixed λ=0.5 in the Table below. Although performance decreases with a fixed λ, careful tuning is standard practice. Recommended tuning ranges are provided in Appendix C.\n\n**Table: Performance comparison of CoUn when $\\lambda$ is tuned per experiment vs $\\lambda$ is fixed for all experiments.**\n| Epoch | Tuned $\\lambda$| Fixed $\\lambda$ |\n| - | - | - |\n| CIFAR-10, ResNet-18 | 0.25 | 1.48|\n| CIFAR-100, ResNet-18 | 1.39| 1.70 |\n| TimyImageNet, ResNet-18 | 1.95 | 2.16 |\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q3 & W2:** In our reported computational cost, we followed the baseline works that have been published in top-tier conferences [8,9,10,11], focusing solely on training costs under final tuned hyperparameters. Consistent with standard practice in machine unlearning literature, we did not include hyperparameter tuning costs for either CoUn or baseline methods, ensuring fair comparisons. Our goal was to provide a fair comparison of the training cost under the best-known or officially recommended hyperparameter settings, as commonly done in the baselines.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q4 & W2:** Thank you for raising this point. When training without supervision, retain samples fail to preserve cluster structure, leading to cluster overlap and notable performance degradation. Furthermore, if the classifier layer is not trained alongside the feature extractor, then relying solely on contrastive learning (CL) being applied to the feature extractor, the classifier will become misaligned with the learned features. This dramatically reduces overall performance. For instance, an experiment (CIFAR-100, ResNet-18, 10% forget ratio) without supervision resulted in an average gap of 70.56, compared to 1.39 when using CE+λ·CL.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q5 & W2:** We thank the reviewer for this observation. In the unlearning literature [8,9,10], the transformation used for training the Original and Retrain models is $CHN$, and the same transformation is also applied to the baselines. Following this convention, we used $CHN$ for training the Original, Retrain, and baseline models. Since our proposed approach incorporates contrastive learning (CL), we explored the possibility of adjusting the transformation distribution specifically for the CL component. The results presented in Figure 9 are based on the experimental setup where both the Original and Retrain models were trained using the $CHN$ transformation. Therefore, the statement in Line 362 holds only under that specific setup. In our paper revision, we will explicitly clarify that this finding is specific to that experimental configuration."}, {"review_id": "WpEUiKme7m", "reviewer": "Reviewer_6v6H", "summary": "This paper introduces a novel approximate unlearning method leveraging the advantage of contrastive learning to manipulate the similarity of retain and forget data. The paper challenged the traditional contrastive learning can indirectly push the forget data to the clusters of retain data and proposed additional supervised learning loss function to cluster the forget data. Therefore, the contrastive learning is pushing away the entire cluster of forget from the retain. This paper also implements extensive experiments and intuitive visualization to prove the effectiveness of proposed method", "questions": "1.\tThe theoretical analysis is hard to follow. Notations are missing, (e.g. R[\\epsilon] ) I guess this is the probability that the encoder representation to generate two views differences larger than $\\epsilon$, which is a typical Lipchitz equation definition. But there is no exact notation for that. \n2.\tWhat is $\\mu_{l}^{T}$?  and What is $\\rho^{max}$. Althought the author mention that the proof of Theorem 1 can be found at paper [34], I would still suggest to place it under appendix and show clearly how (e.g. R[\\epsilon]) interact with Theorem 1. \n3.\tIn line 648, I am not sure how author ensures the equation (10)  $L_r  \\leq L_u \\leq L$ holds in general. More clarifications needed.\nExperiments in Table 2 can apparently be extended to more data drops (e.g. 20%, 30%, 40%, 50%...). 10% data forgetting may not be representative enough. It will be interesting to see comparison on class-wise forgetting with comprehensive baselines. Table 1 only implement on CIFAR-10 with FT and Retrain baseline.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\nThe motivation of the work is clear. The results in Table 1 and Figure 4 look promising demonstrating effectiveness of the work.  Introduction, Technical Method and Experiment are easy to follow. The method is simple, well-motivated and efficient enough. More importantly, the proposed method can be integrated with other baseline for better unlearning performance.\n\nWeaknesses\nTheoretical contribution is weak as the main part of the theory came from other existing paper and the author did not state and explain the theory clearly. \n1.\tThe theoretical analysis is hard to follow. Notations are missing, (e.g. R[\\epsilon] ) I guess this is the probability that the encoder representation to generate two views differences larger than $\\epsilon$, which is a typical Lipchitz equation definition. But there is no exact notation for that. \n2.\tWhat is $\\mu_{l}^{T}$?  and What is $\\rho^{max}$. Althought the author mention that the proof of Theorem 1 can be found at paper [34], I would still suggest to place it under appendix and show clearly how (e.g. R[\\epsilon]) interact with Theorem 1. \n3.\tIn line 648, I am not sure how author ensures the equation (10)  $L_r  \\leq L_u \\leq L$ holds in general. More clarifications needed.\nExperiments in Table 2 can apparently be extended to more data drops (e.g. 20%, 30%, 40%, 50%...). 10% data forgetting may not be representative enough. It will be interesting to see comparison on class-wise forgetting with comprehensive baselines. Table 1 only implement on CIFAR-10 with FT and Retrain baseline.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive comments and for noting that our work is **novel**, **challenging**, **simple**, **well-motivated**, implements **extensive experiments**, is **efficient**, and provides **intuitive visualization demonstrating effectiveness**. We appreciate the acknowledgment that the **introduction, technical method and experiment are easy to follow** and that our approach **can be integrated with other baselines for enhanced unlearning performance**. Below, we address their specific questions.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q1:** We appreciate the reviewer’s observation. While our intention was to define all notations, including $R[\\epsilon]$, we acknowledge that defining them solely through mathematical expressions may not have been sufficiently explicit. To improve clarity and facilitate understanding of the theoretical analysis, we will revise the manuscript to clearly and formally define all variables at their first occurrence. Specifically, as shown in Eq. (5), $R[\\epsilon]$ denotes the probability that, when a sample is drawn from the dataset, the distance between the encoder representations of its two augmented views exceeds $\\epsilon$. Thus, a small value of $R[\\epsilon]$ indicates good alignment in the representation space.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q2:** We thank the reviewer for pointing this out. \n\n1. $\\mu_l$ denotes the center of class $C_l$. It is defined in Line 240, and $\\mu_l^T$ refers to its transpose.\n\n2. $\\rho^{\\text{max}}$ is defined in Line 245 as a function of $(\\sigma, \\delta)$, which characterize the data augmentation, as well as the threshold $\\epsilon$. Intuitively, better alignment (i.e., smaller $R[\\epsilon]$) and a sharper concentration of augmented samples (i.e., larger $\\sigma$ for a given $\\delta$) result in a lower value of $\\rho^{\\text{max}}$.\n\n3. Following the reviewer’s suggestion, we will include the full proof of Theorem 1 in the appendix and explicitly clarify how quantities such as $R[\\epsilon]$, $\\rho^{\\text{max}}$, and class means interact in the derivation. This will enhance the clarity and self-containment of the theoretical analysis.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q3.1:** In Line 243, we state that the feature extractor $f_{\\theta_u}$ is an $L$-Lipschitz function. That is, for any input sample (whether from the retain or forget data), the inequality in Eq. (8) holds. This implies that $f_{\\theta_u}$ is globally Lipschitz with constant $L$, and therefore the Lipschitz constant over the forget data (i.e., $L_u$) must satisfy $L_u \\leq L$.\n\nMoreover, as noted in Lines 644-648, training $f_{\\theta_u}$ exclusively on the retained data encourages the function to vary more smoothly on those data. This indicates that $f_{\\theta_u}$ exhibits greater stability and lower sensitivity on the retained data compared to the forgotten data. This leads to the inequality $L_r \\leq L_u$. \n\nTaken together, these observations support the bound $L_r \\leq L_u \\leq L$.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q3.2:** Regarding the reviewers suggestion about having experiments with \"more data drops\", we would like to refer the reviewer to Table 3 in our paper, where we repeated all experiments in Table 2 but with a 50% forget data ratio. Additionally, Figures 4 and 5 also present results for both 10% (left) and 50% (right) forget ratios. Moreover, Figure 6 demonstrates a sequential data removal, where 10% of the data is removed initially, followed by an additional 10% every 10 epochs, reaching up to 50% (i.e., 10\\% $\\rightarrow$ 20\\% $\\rightarrow$ 30\\% $\\rightarrow$ 40\\% $\\rightarrow$ 50\\%).\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q3.3:** Regarding experiments related to **class-wise forgetting**, we would like to refer the reviewer to Section E.2, where we discussed the class-wise forgetting. In Table 5, we compare our method, CoUn, with state-of-the-art baselines under class-wise forgetting. Results demonstrate CoUn's competitive performance under class-wise forgetting.\n\n$\n\\textcolor{white}{f^{*}}\n$\n\n**Authors' response to Q3.4:** We thank the reviewer for pointing the following out: \"Table 1 only implement on CIFAR-10 with FT and Retrain baseline\".  \n\nFollowing the reviewer’s suggestion  and to further validate the robustness of our findings, we conducted additional experiments to include the missing baselines and we also conducted new experiments with another model architecture and dataset. These comprehensive results are presented in the Table below. These extended results will be reflected in the paper. As expected, CoUn yields predictions more aligned with the Retrain model compared to other baselines, further demonstrating that it better mimics the Retrain model in terms of classifying the forget samples based on semantic similarity.\n\n**Table: Predictions of forget 'truck' samples based on most semantically similar classes. The difference ($\\Delta$) and the (best) average difference between each method and Retrain are reported.**\n|Forgetting Scenario|Method|Truck|Automobile|Airplane|Ship|Avg. Diff. ↓|\n|-|-|-|-|-|-|-|\n|Random (10%), CIFAR-10, ResNet-18|Original|100.00|0.00|0.00|0.00|-|\n||Retrain|97.42 (0.00)|1.23 (0.00)|0.38 (0.00)|0.40 (0.00)|0.00|\n||FT|98.12 (0.70)|0.75 (0.48)|0.36 (0.02)|0.32 (0.08)|0.32|\n||NegGrad+|97.86 (0.44)|0.97 (0.26)|0.42 (0.04)|0.30 (0.10)|0.21|\n||$\\ell_1$-sparse|98.06 (0.64)|0.85 (0.38)|0.30 (0.08)| 0.38 (0.02)|0.28|\n||SaUn|96.76 (0.66)|0.88 (0.35)| 0.34 (0.04)|0.34 (0.06)| 0.28|\n||NoT|97.88 (0.56)|0.89 (0.34)| 0.40 (0.02)| 0.32 (0.08)| 0.25|\n||**CoUn (Ours)**|97.84 (0.42)| 0.99 (0.24)| 0.32 (0.06)|0.42 (0.02)|**0.19**|\n||\n|Random (10%), CIFAR-100, VGG-16|Original| 100.00|0.00|0.00|0.00|-|\n||Retrain|49.53 (0.00)|15.18 (0.00)|10.06 (0.00)|3.42 (0.00)|0.00|\n||FT|60.53 (11.00)|9.11 (6.07)|9.68 (0.38) |3.61 (0.19)|4.41|\n||NegGrad+|41.94 (7.59)|13.47 (1.71)|12.71 (2.65)|4.74 (1.32)|3.32|\n||$\\ell_1$-sparse|57.69 (8.16)|13.47 (1.71)|10.06 (0.00)|2.85 (0.57)|2.61|\n||SaUn|55.16 (5.63)|13.52 (1.66)| 9.68 (0.38)|2.63 (0.79)|2.12|\n||NoT|51.23 (1.70)|14.61 (0.57)|9.68 (0.38)|3.61 (0.19)|0.71|\n||**CoUn (Ours)**|50.15 (0.62)|14.67 (0.51)|10.09 (0.03)|3.28 (0.14)|**0.33**|"}]

Author rebuttal:

Meta-review: **Summary:** This paper introduces CoUn, a machine unlearning framework that leverages contrastive learning in combination with supervised learning on retained data. The central idea is to indirectly adjust forget representations based on semantic similarity to retain samples, thereby emulating the behavior of retraining from scratch on retain-only data. The authors provide both theoretical insights and extensive experiments across multiple datasets and architectures, demonstrating that CoUn achieves performance close to the retrain baseline while surpassing existing unlearning methods. The approach is modular and can also be integrated with other baselines to enhance their unlearning effectiveness.


**Decision:** Initially, the reviewers raised several concerns, including the limited novelty of using contrastive learning, clarity of the theoretical analysis, scalability of experiments beyond small datasets, and reproducibility of results. During the rebuttal, the authors responded with substantial clarifications, additional empirical studies, and commitments to release detailed code. Most reviewers raised their scores after the rebuttal, acknowledging that the paper makes a meaningful and technically sound contribution. Given the importance of machine unlearning, the clarity of the proposed framework, and the convincing experimental evidence, I recommend acceptance.
Decision: Accept (poster)