Label Smoothing Improves Machine Unlearning
Authors: Zonglin Di
Zhaowei Zhu
Jinghan Jia
Jiancheng Liu
Zafar Takhirov
Bo Jiang
Yuanshun Yao
Sijia Liu
Yang Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Machine unlearning
Label smoothing
Differential privacy
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: The objective of machine unlearning (MU) is to eliminate previously learned data from a model. However, it can be challenging to strike a balance between computation cost and performance when using existing MU techniques. Taking inspiration from the influence of label smoothing on model confidence and differential privacy, we propose a simple gradient-based MU approach that uses an inverse process of label smoothing. This work introduces UGradSL, a simple, plug-and-play MU approach that uses smoothed labels. We provide theoretical analyses demonstrating why properly introducing label smoothing improves MU performance. We conducted extensive experiments on several datasets of various sizes and different modalities, demonstrating the effectiveness and robustness of our proposed method. UGradSL also shows close connection to improve the local differential privacy. The consistent improvement in MU performance is only at a marginal cost of additional computations. For instance, UGradSL improves over the gradient ascent MU baseline constantly on different unlearning tasks without sacrificing unlearning efficiency. A self-adaptive UGradSL is also given for simple parameter selection.

Published as a conference paper at ICLR 2026
LABEL SMOOTHING IMPROVES MACHINE UNLEARN- ING
Zonglin Di', Zhaowei Zhu?, Jinghan Jia"* Jiancheng Liu Zafar Takhirov Bo Jiang", Yuanshun Yao". Sijia Liu', Yang Liu
University California, Santa Cruz, Docta.ai, "Michigan State University, *OpenAI, 'Independent Contributor, Meta Superlntelligence Lab {zdi, yangliu/@ucse edu, 2 22zw@docta. ai, j[jiajingh, 1iusi ji5 ensu edu, $1 "Ijcceoeenai com, ((z. tahirov, bjiang51 8) @mga com, Pkevinyaowwrkkemall com
ABSTRACT
The objective of machine unlearning (MU) is to eliminate previously learned data from a model. However, it can be challenging to strike a balance between com- putation cost and performance when using existing MU techniques. Taking in- spiration from the influence of label smoothing on model confidence and differ- ential privacy, we propose simple gradient-based MU approach that uses inverse process label smoothing. This work introduces UGradSL, simple, sang-and-laay MU approach that uses smoothed labels. We provide theoreti- cal analyses demonstrating why properly introducing label smoothing improves MU performance. We conducted extensive experiments several datasets of various sizes and different modalities, demonstrating the effectiveness and ro- bustness of proposed method. UGradSI. is also closely connected to im- proving the local differential privacy. The consistent improvement in MU per- formance only at a marginal cost of additional computations. For instance, UGradSL consistently outperforms the gradient- -sscent MU baseline on differ- ent unlearning tasks without sacrificing unlearning efficiency. A self-adaptive UGradSL is also given for simple parameter selection. The code is available at https: /ttps://giin thub. com
www/UCSC UCCCC REAL/Labe, Smootmoo perootiiin----e ng- Unl earn
INTRODUCTION
Building reliable MI. model has become important topic in this community. Machine unlearning (MU) is a task requiring removing the learned data points from the model. The concept and the technology of MU enable researchers to delete sensitive improper data in the training set to improve fairness, robustness, and privacy and get a better ML model for real-world use (Chen et al., [021b, et al., (02T). Retraining from scratch (Retrain) is straightforward method when we want to remove the data from the model; yet it incurs prohibitive computation costs for large models due to computing resource constraints. Therefore, an efficient and effective MU method desired.
The most straightforward MU approach should be retraining-basee based method (Bourtoule et al.) 2021), meaning that we retrain the model from scratch without using the data to be forgotten. The method can guarantee privacy protection but the computational cost is intensive. Most existing works (Koh & Liang 2017: Golatkar et al. [020 Warnecke et al. 2021 Graves et al. 202T] Thudi al. 2021 Izzo et al. [02 Becker & Liebig 2022 Via al., (023) focus approximate MU to achieve a balance between unlearning efficacy and computational complexity, making them more
suitable for real-world applications, meaning that they make the model unlearn the forgetting dataset
without retraining the model.
We desire an approach that enjoys both high performance and high efficiency. Since MU can be viewed as the inverse process of ML, we are motivated to think it would be a natural and efficient
'Equal Contribution.
"ooresponning author: Yang Liu <yangl aeuuccc.edu

---

Published as a conference paper at ICLR 2026
Perfernsanee IaageNit Dutaset
Perfersance Readon CIFAR-
Nocrap Cop
Medboda
Netlods
(a) Performance classwise forgetting on ImageNet.
Performance of random forgetting on CIFAR-100
The performance comparison of our proposed methods and baseline methods using aver- age gap and runtime (RTE), where lower values indicate better performance. Bars represent average gap while red dotted lines show RTE. Since retraining does not have gap by definition, only RTE reported for this baseline and the bar empty. For classwise forgetting on ImageNet, UGradSI. achieves the lowest average gap (2.23%) with acceptable RTE increase. For random forgetting CIFAR-100 100, UGradSL+ attains the best average gap (3.75%), while UGradSL demonstrates an op- timal gap-runtime trade-off.
Figure 1:
way to develop unlearning process that imitates the reverse of gradient descent. Indeed, gradient ascent (GA) (Thudi et al. (021) is one of the MU methods but unfortunately. it does not fully achieve the potential of this idea. One of the primary reasons is that once the model completes training, the gradient of well-memorized data that was learned during the process diminishes (close to loss) and therefore the effect of GA is rather limited.
Our approach is inspired by the celebrated idea of label smoothing (Szegedy et al. 2016). In the forward problem (gradient descent), the smoothed label proves to be able to improve the model's generalization power. In our setting, we treat the smoothed label term as the regularization in the loss function, making the unlearning more controllable. Specifically, we show that GA with a "neg- ative" label smoothing process (which effectively results in standard label smoothing term in descending fashion) can quickly improve the model's deniability in the forgetting dataset, making the model behave close to the retrained model, which is exactly the goal of MU, We name our approach UGradSL, Unlearning using Gradient-based Smoothed Labels.
Our approach is plug-and-play method that can improve the gradient-based MU performance con- sistently and does not hurt the performance of the remaining dataset and the testing dataset a gradient-mixed way. At the same time, provide theoretical analysis of the benefits of our approach for the MU task. The core contributions of this paper are summarized as follows:
We propose a lightweight tool to improve MU by joining the label smoothing and gradient ascent.
We theoretically analyze the role of gradient ascent in MU and how negative label smoothing is able to boost MU performance.
Extensive experiments in six datasets in different modalities and several unlearning paradigms regarding different MU metrics show the robustness and generalization of our method.
We investigate the relationship between label smoothing and label differential privacy (LDP), showing that label smoothing can aid LDP.
RELATED WORK
Machine Unlearning (MU) was developed to address information leakage concerns related to pri- vate data after the completion of model training (Cao Yang. 2015) Bourtoule al. 2021]] Nguyen et al. 2022), gained prominence with the advent of privacy-focused legislation (Hoofnagle et al.) 2019 Pardau 2018). One direct unlearning method involves retraining the model from scratch after removing the forgetting data from the original training set. It is computationally inefficient, prompting researchers to focus on developing approximate but much faster unlearning techniques
(Becker & Liebig 2022, Golatkar al. 2020 Warnecke et al.) 2021 Graves et al. 202 Thudi
cet al., 2021; Izzo et al.] 2021] Jia et al.] 2023). Beyond unlearning methods, other research efforts
aim create probabilistic unlearning concepts (Ginart al. 2019: Guo et al. 2019) Neel et al.
2

---

Published as a conference paper at ICLR 2026
Ullah al. 2021: Sekhari et al., 2021 and facilitate unlearning with provable guar- antees, particularly in the context of differential privacy (DP) (Dwork et al. 2006 Di et al. 2014 Hall et al. 2012). However, it typically necessitates stringent model and algorithmic assumptions, potentially compromising effectiveness against practical adversaries, such as membership inference attacks (Graves al., 202T] Thudi al. 2021). Additionally, the interest in MU has expanded to encompass various learning tasks and paradigms (Wang et al. 2022b; Liu et al., 2022bb Chen et al.) 2022 Chien al. 2022, Marchant al., 2022 Di et al. (022). These applications demonstrate the
2021
growing importance of MU techniques in safeguarding privacy. The rest of the related work about
influence function label smoothing and differential privacy are given in Appendix B
LABEL SMOOTHING ENABLES FAST AND EFFECTIVE UNLEARNING
This section sets up the analysis and shows that properly performing label smoothing enables fast and effective unlearning. The key ingredients of our approach are gradient ascent (GA) and label smoothing (LS). We start with understanding how GA helps with unlearning and then move on to show the power of LS, At the end of the section, we formally present our algorithm.
3.1 PRELIMINARY
Machine Unlearning Consider a K-class classification problem on the training data distribution Dr, (X' Y), where X' and y are the feature and label space, respectively. Due to some privacy regulations, there exists a forgetting data distribution Dj that the model needs to unlearn. We denote by Our the original model trained on Drr and A,, the model without the influence of D The goal of machine unlearning (MU) is how to generate O, from Oyr.
Label Smoothing In K-class classification task, let y; denote the one-hot encoded vector form of E V. Similar to (Wei et al. (202)), we unify positive label smoothing (PLS) and negative label smoothing (NLS) into generalized label smoothing (GLS). The random variable of smoothed
label y;
GLS,a with smooth rate e (-o0, 1] is y,
GLS,a (1 (1- a) yi + K - -.
I-K x ]. where (1 1xxxx (x) is the y;-th element in the encoded label vector. When a < O, I
GLS becomes NL..
3.2 GRADIENT ASCENT CAN HELP GRADIENT-BASED MACHINE UNLEARNING
We discuss three sets of model parameters in the MU problem: 1) 0 the optimal parameters trained from D.y Der 2) 0 the optimal parameters trained from D, ~ D,, such that D, DrA and 3) fj. the optimal parameters unlearned using gradient ascent (GA) on Dj ~ Dy. Note 0; can be viewed as the exact MU model. The definitions of 0r and 0r follow the standard empirical risk minimization as
0* = arg min 1 Σ (ho.
n seD
and by using the influence function, 07 is
07 arg Rer (0) + Σ {(ha,z)))
zieD,
where Re, (0)= ExtreD,. (he "") and (0) =EseD, ((ho. are the empirical risk on Dr and D,, respectively. We use notations ((hg.2) 2) to specify the loss an example = (x, y) in the dataset. he is function h parameterized by 0. is the weight of Dy compared with Day. The optimal parameter can be found when the gradient is 0:
Voler (07)+ + Σ Vollhor,
(2)
ztED,
Expanding Eq. 2 at 0 0r using the Taylor series, we have
0- 0
Σ
vol(hor +e Σ
val(hor
E Σ vol(her, (3)
Ltte
stED,
sfeD;
3

---

Published as a conference paper at ICLR 2026
Here, ignore the Lagrange Remainder. Similarly, expand VeRo at 0 0r; and derive 0r; f as
- Oir 20
Σ Dtr
Vol(hor:z")
Σ vol(hez, D'rrDer
(4)
We ignore the average operation in the original definition of the influence function for computation convenience because the size of Drr or D is fixed. For GA, let E in Eq. and we have
0 - = -- - 0r -0; - 20, A0,-
0
(5)
where (-40,) represents the learning gap from 0 0ür while vector 20, represents how much the model unlearns (backtracked progress) between 0 and o The details 20, and A are given Eq. 17 in Appendix. Ideally, when 20, and 20, f are exactly the same vectors, GA can lead the model to the optimal retrained model since we have 0; 0j. However, this condition is hard to satisfy in practice. Thus, GA cannot always help MU. We summarize it in Theorem 1 The proof and the error analysis are given Appendix C.I and C.2
Theorem 1. Given the approximation in Eq. ( GA achieves exact MU if and only if
Σ xfED
vol(hoz; 20) = HO,
Σ vol(hor,
sfED,
(6,,,, 07.) [Extren,,, Vol(ho; =n)] [ Exren. vol(ho; z)] that GA cannot help MU, ie.. 10 071 10; -0,l--
Otherwise, there exist 07, Ofr such
3.3 LABEL SMOOTHING IMPROVES MU
Practically, cannot guarantee that GA always helps MU shown in Theorem | To alleviate the possible undesired effect of GA, we propose to use label smoothing as plug-in module. Consider the cross-entropy loss an example. For GLS, the loss is calculated as
1- K a)) EE(ho, (x,1)) E(ho, GLS. (1+
Σ (2, y/E''
(6)
K
where E(ho. (I, v)) := ((h. 2) and E(ho, (x.8/)) ) to denote the loss of an example when its label is replaced with v/ Intuitively, Term Enerro ((ho, v)) ) in Eq. (6 leads to state where the model
makes wrong predictions on data in the forgetting dataset with equally low confidence (Wei et al., 2021 Lukasik et al.) 2020).
With smoothed label given in Eq. 66. we show that there exists a vector A0, such that Eq. 5 can be written
0 0 f.L = A0, 20,- 1-K K a. (Λ0, -00))
K
We leave the detailed form of AQ, to Eq. (3).. But intuitively, 2d, captures the gradient influence of the smoothed non-target label on the weight. We show the effect of NLS (a in Theorem below and its proof is given in Appendix C.3
Theorem 2. Given the approximation in Eq. (5) and 1A0,-A0) A0, A0,-A00))>>) s O, there exists an < O such that NLS improves GA in unlearning, i.e., 07 < ||0;- -0il. where 0 j is the optimal parameters unlearned using GA and NLS,, and (.,-) the inner product of two vectors.
Now we explain the above theorem intuitively. Vector 20, A0, the resultant of Newton's direction of learning and unlearning. Vector 20, 20, is the resultant of Newton's direction of learning non-target labels and unlearning the target label. When the condition (AA,- A0,, AQ.. 20,) 0 holds, 20,- A0, captures the effeett of the smoothing term in the unlcarning process. we assume that the exact MU model able to fully unlearn an example, vector A0, contributes a direction that pushes the model closer to the exact MU state by leading the model to give the wrong prediction. The illustration of (A0, - A0, AQ,- - A0,) is shown in Figure in the Appendix.
The effect of the smoothed term in gradient ascent (GA) with NLS is equivalent to performing gradient descent optimization with traditionally defined (positive) LS. The gradient of the smoothed term is exactly the same a// K Eyery Ve(ho, (x,n')) in both cases.
4

---

Published as a conference paper at ICLR 2026
3.4 LABEL SMOOTHING HELPS LOCAL DIFFERENTIAL PRIVACY
When a < O, the smoothing term will incur positive effect in the gradient ascent (GA) step. Label smoothing can also viewed through the lens privacy protection. This interpretation stems from the fact that label smoothing reduces the likelihood of specific label, thereby allowing it to better blend in with other candidate labels. Particularly, we consider a local differential privacy (LDP) guarantee for labels as follows.
Definition 1 (Label-LDP), A privacy protection mechanism M satisfies e-Label-LDP, if for any =(M())=y
labels v.y. yored y, P(M(F) se.
The operational meaning of M is to guarantee any two labels and y in the label space, after privatization, have similar likelihood to become any yaed in the label space. That is, the prediction on the forgetting dataset should be similar no matter what the ground-trule label is. The similarity is measured by the privacy budget e E [0, +o0). Smaller implies stronger indistinuuishabiiity between y and y/. and bence, stricter privacy.
Recall Rey (0) E,, E(ha, ="ha,""") Denote by RNIS(O: (0; (x) = Ess. E(ha, 215:0),a a < the empirical risk of forgetting data with NLS. After MU with label smoothing on D, by GA, the resulting model can be seen as minimizing the risk 71 Rer (0) -2 RNIS(O; a). which a weighted combination of the risk from two phases: machine learning on Der with weight γ > and 2) machine unlearning on D with weight > O. By analyzing the risk, we have the following theorem show NLS in MU induces --abel-LDPP for the forgetting data.
Theorem 3. Suppose "i - (1 + 1-K a) > O. MU using GA+NLS achieves e-Label-L.DP on Dj K
where
K 11-3)= + (1 722
k a<
c
log
Intuitively, when a is more negative, the privacy of the labels in the forgetting dataset is better. When a (1 7 7/2)), we have 0, indicating the best label-L.DP result, which is the goal of MU. The theorem also warns that cannot be arbitrarily negative.
UGRADSL: A PLUUG-AND-PLAY AND GRADIENT-MIXED MU METHOD
Df
Request: Forget the eat class
Cat
he
X
LL
(VE(a. D
Ve (5
Dy
Gradtont
Gradieat
Doseat
Asceat
X
The framework of UGradSL. When there is an unlearning request, we can split the Der into Dy and D,.. We first apply label smoothing on 2' (x,y) e Dy to get 2 2.S.a. = (x, a where the smooth rate be pre-defined or self-adaptive. In the backpropagation process, apply gradient descent on the data z D, and gradient ascent on the smoothed forgetting data Dy, which the mix- gradient way.
Figure 2:
Given the effect of label smoothing on MU and LDP, propose our method here. Compared with retraining, Fine- Tune (FT) and GA are much more efficient as illustrated in Section with comparable better MU performance. FT and GA focus on different perspectives of MU. FT is to transfer the knowledge of the model from D,, to D, using gradient descent (GD) while GA is to remove the knowledge of Dy from the model.
5

---

Published as a conference paper at ICLR 2026
Due to the flexibility of label smoothing, method is suitable for the gradient-based based methods including FT and GA, making our method a plug-and ala- algorithm. UGradSL is based on GA while UGradSL+ is FT. Compared with UGradSL, UGradSL+ will lead to more compre- hensive result but with a larger computation cost.
How to choose the smooth rate is worth discussing. Normally, the 0; for every data point 2f D, can be set identically. To gain better performance, we improve UGradSL and UGradSL+ by taking every data point into consideration and assigning ag individually and adaptively based on the distance d( d(ho, (z"), ho, (=f)) E [0. [] for each (f) pair. The intuition is that if an
instance 2! resides in a dense neighborhood of Dr. its inherent deniability is higher and therefore the requirement for "forgetting" is lower and should be reflected through a smaller ag, The algorithm is presented in Algorithm| 1 and the framework is illustrated in Figure 2 We leave the details of the implementation, complexity analysis and the additional classification results in Appendix D
Assuming the amount of retained data is significantly larger than the amount of data to be forgotten (D, >Dy)), D2 will be iterated several times when D, is fully iterated once. We calculate the loss using a gradient-mixed method as:
-1 T(he, By... NIS.aa Br.P) = = E ((1b,=))(1-p))
E
E(hg 2/
(.N.S.a.)
(8)
z"ef,
E.N.S.OEB B
where [0, 1] is used to balance GD and GA and the minus sign between two elements on the RHS stands for the GA. is the vector for the smoothing rate of every data point z!_ he is updated according to L in Eq. (8). UGradSL is similar to UGradSL+ and the dataset used is given in bracket in Algorithm|| 1 The difference between UGradSL and UGradSL+ is the convergence standard. UGradSL is based on the convergence of Ds while UGradSL+ is based on D,.. It should be noted that the Hessian matrix in Theorem I is only used in the theoretical proof. In the practical calculation, there is no need to calculate the Hessian matrix. Thus, our method does not incur substantially more computation but improves the MU performance empirical evidence in Section Compared with applying the label smoothing evenly, the improved version takes the similarity of the data points between D, and Dj into consideration and provides self-adaptive smoothed labels for individual 2 as well as protects label-LDP.
a large scale. We present
5 EXPERIMENTS AND RESULTS
5.1 EXPERIMENT SETUP
Dataset and Model Selection We validate our method using various datasets in different scales and modalities, including CIFAR-10, 10, CIFAR-100 (Krizhessky et al. 2009). SVHN (Netzer et al. 2011
CelebA (Liu al. 2015), Tiny-ImageNN ImageNet (Deng et al. 2009) and 20 Newsgroups (Lang) 1995) datasets. For the vision and language dataset, we use Res- (He et al. 2016) and Bert (Devlin al. 2018) as the backbone model, respectively. Due to the page limit, the details of
the training parameter and the additional results of different models including VGG-16 (Simonyan
& Zisserman 2014) and vision transformer (ViT) (Dosovisskiy et al. 2020 are given in the Ap-
pendix E.5
Baseline Methods We compare the proposed methods with series of baselines, including retrain, fine-tuning (FT) Warnecke et al. 2021 Golatkar et al.) 2020). gradient ascent (GA) (Graves et al.
2021 Thudi et al.) 2021), unlearning based on the influence function (IU) (Izzo al. 2021 Koh
& Liang] 2017), boundary unlearning (BU) (Chen et al. 2023), 4-sparse (Jia et al. (023). ran-
dom label (RL) (Hayase et al.] 2020), SCRUB (Kurmanji et al. (2033), SalUN (Fan et al. 2023).
EU-k, CF-k (Goel et al. (022). GLI (Choi al.) 2024) and PABI (Koloskova et al. 2025 The implementation details of these basclines are given in Appendix E.I
Evaluation Metries The evaluation metrics we use follow (Jia et al. 2023), where we jointly con- sider unlearning accuracy (UA), membership inference attack (MIA). remaining accuracy (RA), testing accuracy (TA), and run-time efficiency (RTE). UA is the ratio of incorrect prediction on D,, showing the MU performance. TA is the accuracy used to evaluate the performance on the whole testing set Dre, except for the class-wise forgetting because the task is to forget the specific class.

---

Published as a conference paper at ICLR 2026
Algorithm 1 UGradSL+: A plug-and-play, efficient, gradient-based MU method using LS. UGradSL can specified by imposing the dataset replacement in brackets. If a is not given, the algorithm turns to the self-adaptive version.
Require: An almost-converged model hō., trained with Dor. The retained dataset D,. The for- getting dataset D-- Unlearning epochs E. GA ratio p. Distance threshold B. The optional smoothing ratio a.
Ensure: The unlearned model her
1: Set the current epoch index as fe +
2: while f- E do
3: while Dr(D,) is not fully iterated do
4: Sample batch B, in D,
5:
Sample a batch By from D, where |Byl = [B,] o not given then
6:
The improved and self-adaptive version
7:
Extract the feature 2f and 2/ using hey- Calculate the distance d(he, (zf), ho, (z))) for each 2f) pair where zf B,
8:
and o/} By.
9:
For each z/.- count the number o/ of 2 whose d(ho, (=f) ho, (ef)) < Calculate the smooth rate a; = c By for each zf By
10:
11:
end
Update the model using Br, B5. and a, according to Eq. (8)
13:
end while
te +
15: end while
RA is the accuracy on D,.. To evaluate the effectiveness of "forgetting", we resort to the MIA met- rics described in (Jia al. 2023 Fan et al.) 2003), i.c. accuracy of an attack model against target model On. such that the score is reported as true negative rate (TNR) on the forget set. Formally, this is global MIA score (Yeom et al. 2018), which we rewrite MIAScore 1 Pr ( where y Dy are the forget samples and D. is the model under test. Overall, we use Avg. Gap to quantify the mean performance gap between each unlearning method and the retrained model across all individual metrics above. A lower value indicates better performance.
Unlearning Paradigm We mainly consider three unlearning paradígms, including class-wise florget ting. random forgetting, and group forgetting. Class-wise forgetting is to unlearn the whole specific class where we remove one class in D, and the corresponding class in Dic completely. Random forgetting across all classes is to unlearn data points belonging to all classes. As special case of random forgetting, group forgettin means that the model trained to unlearn the group sub-class of the corresponding super-classes, A more detailed description is given in Appendix B.2
5.2
EXPERIMENT RESULTS
5.2.1 CLASS-WISE FORGETTING
We select the class randomly and run class-wise forgetting on five datasets. We report the results of CIFAR- 100 ImageNet and CIFAR-10 in Table I and respectively. The results of News- Group and SVHN are given in Appendix E.3 As the performance of GA and FT, respectively without an increment in RTE or drop in TA and RA, leading to comprehensive satisfaction in the main metrics, despite randomness in D,, showing the robustness and flexibility of our methods in MU regardless of the size of the dataset and the data modality. Moreover, in terms of Avg. Gap, the proposed method shows its similarity the retrained model.
see, UGradSL and UGradSL+ can boost
5.2.2 RANDOM FORGETTING
We select data randomly from every class as D,, making sure all the classes are selected and the size of Dy is 10% of the Drr. We report the results of CIFAR-100 and TinyImageNet in Table

---

Published as a conference paper at ICLR 2026
Table 1: Results of class-wise forgetting in CIFAR-100 and ImageNet. The best comprehensive metrics are bold.
Imogee
Mechod
Ui
MiAggom
RA
TA
Aug.
RIEG mitn)
UA
MiAAgoom
RA
TA
Ang. Cip
RIE
Retuass
1000000..00
100 10000 oo
50 30.00us01
21.3
26.95
102,00aoee
100,00
y1.0
02.57atun
26.15
GA
99. 99.30
00/07un.0
10.33
63.50,00
16/00.a
64.77
0.01
10
2.77.155
83,20
50 18.99
71 71.30
41.25
1.24
11.5 33.56
40.8545557
56.25.1.00
06.2541.18
31.30
1:51
BE
70.81,2:0
40.50
13.03
0.55
$8.02
0.
58.18
08.72
33.14
0.24
BSS
man
75
71 mm
30.05.100
13.0
36 um
56.55a.
0.57
RL
19,
100
40.07.300
100,00ao
6205..10
02.58uut
1.17
BU.
1010000..0
68 68.70.000
41.50
40.84
4.50
103:00aoe
32.9
87. 87.39ucu5
42.75
0.52
CR-
01.68,1.00
01.354111
28.35 73
3.01
56.81am
26.33
1.25
SCHU8
29.06...
28.50.577
78.000..0
56.08
18.00,om
75.00.0.1
100000
100/00.emm
6U
55.57,211
09.55
PABC
100.050 10000..0
98, 38.94,100
73.41a0m
0.83
20.09
15.15.10
20.3
11:93
0107
10
76.00 21
63.8%
2.23
6.01
100/06 m0
en
44.10
72.32.6
0.64
3.37
100000ove
100000ee
16
08.8
4.39
Table 2: Results of random forgetting in CIFAR-100 and ImageNe. The best comprehensive metries are bold.
CIFAR-100
Tiny ImmggnNet
Method
UA
MlAg
RA
TA
Asg. Gap (1)
RIE (, matt
||
UIX
MEA
RA
TA
Jug. Gap
RIE (1.mi)
Rernin
M
25:01
M1,80una
50:Mus
235.00
FF
2.55ase
10.50ach
18.85
1.95
20 25
37 00
62.51uca:
00:00usa
11:01
18.51
GAA
2.58ano
5.66
97.45.3.22
7G 77.05a8.0
20.64
0.29
25 1aus
S1.51,1.0
30:40 30:16us:1
16.39
5,55
IU
18.71as..
18.00.c15
$4.65.5.3
52.20 62:20
18.05
1.20
60.01.040
85.60aa.5
10.36arn
25. 25.44
35.04
7.50
BE
1:45,433
TV: u0
21.33
0.34
M1.85un
56. ss.16.sss
17:03
3,53
2:20a121
10.75wum
08.22.1.3
20:23
0.34
10.77...
25-68ua15
81.25au.1t
17.09
R1.
50.38...
7.41
1.30
10
77.58mm
00-18.m
77.06
EU-2
3,5
05.44.3:5
50.02 20.024243
22.51
1.95
53
22 22.19,175
St.41a0.77
05.08
14.00
20,02
Ch-E
B07aai
98.06 a.17ams
07 12
21.37
0.86
23.20,
61:3Puum
56. 55.15.
17.25
13.18
SCKUB
4.01.1.35
23397
21.40
1,05
20:11a1:5
22.25,255
50:92a6.7
00:11aam
16.42
25,79
SaIUN
a3
80.30aca
00.83.141
3.33
10.39..
62.32,1047
77.00mm
00:50umm
2,6
9.58.
2..
PABI
72
18:30
10.71am0
95.35
08 L2
6.95
0.55
10 10.73am
37 38
07 M0.0
50.38.re
1980
9.47
DGrod5L
21,09,.
00.47112
43387
78.00
3.75
3.52
39.44 29.49.1.1
81.38
32.02
3.57
25.95
Compared with class-wise forgetting, it is harder to improve the MU performance and still keep the RA and TA close to the retrained model. Benefit from the mix mri--gradeet design, the proposed method can make a good balance between forgetting Dy and retaining the knowledge in D,.. The rest of the experiments are given in Appendix E.4
5.2.3 GROUP FORGETTING
Although group forgetting can be seen as part of random forgetting, we want to highlight its use case here due to its practical impacts egg., facial attributes classification. The identities can be regarded as the subgroup in the attributes.
Table 3: Results of Group Forgetting on CIFAR-20 and CelebA. For CIFAR-20, the model is trained classify 20 super-classes, with D, representing one of five subclasses within single super-class. In the CelebA dataset, the model performs binary classification to determine whether a person is smiling, with Dj selected based on specific identities. The best comprehensive metrics are bold.
CIAR D1
MIAguore
Aug Gap (3)
RIES mm)
MiAgcore
Aug. (ip
(J. min)
6:74
9.77 aa
11.38.n.
FT
2.70uss
99.57.acx
70 4m
10.21
2.47
5.M0uuxe
90.9
95. 93.18uss:
1.79
25,04
GAA
50.900 30.30433
$1.66.1.7
14.07.490
42.58
0.31
6:00umts
0.76
02.50 92.52
1.75
1,30
IU
2.
10.01
1.30
5.50pm
1005mmm
92400
1,97
219.77
59.07.1.39
01.75am7
78.90.242
0.33
11.00aase
45.44..99
88.37acas
88.00um
8:50am
f.13
38.57
0.35
R1.
16303.34
18.00...
90.00
59
76.57
0.31
8.3mm
18.55...
91.62
1.76
40.00
Fii-
8004.55
16.35
07.07_3.5
00:07
7.0
0.87
7:30pa10
18.7..0
20.35.000
91.04
1.01
1.98
CF.6
0,80,.
17.86
11.20
1.21
5.86n00
17.36.13
02
02.72umss
2,45
1.50
1:8.am:
09.00.111
0.30
8.7aur
18.5
22
10.50uue
98.00.0.0
24.25
2388
6.50nas
25.07.433
PABI
10.83.00
88.39,
1996
45.10.197
10.55mm
99.57am
D.oaSS
32.87.m
58.95...
97.20_...
7,03
0.13
6:38..
573.2.5
00.Haam
92.50.a3r
1.61
2.17
LGod
78.44.1.3
07.60,3
76
77777
32.30
4.13
6.88nss
835
10.306.1
92.9
0.81
16.78
8

---

Published as a conference paper at ICLR 2026
CIFAR-10 and CIFAR-100 share the same image dataset while CIFAR-100 is labeled with 100 fine-grained classes and 20 coarse (super) classes (Krizhevsky et al. 2009 Chundawat et al. 2 We train model to classify 20 super classes using CIFAR- 100 training set. The setting of the group forgetting within one coarse class is to remove one fine- lie-graieed class from one super class in CIFAR-10W 100 datasets. For example, there
five fine-grained fishes in the Fish coarse class and
want to remove one fine-grained fish from the model. Different from class-wise forgetting, we do not modify the testing set. We report the group forgetting in Table 3
CelebA We select CelebA dataset as another real-world case and show the results in Table 3 We train a binary classification model classify whether the person is smile or not. There are 8192 identities in the training set and we select 1% of the identities (82 identities) as Dy. Both smiling and non-smiling images are D.. This experiment has significant practical meaning, since the biometric, such as identity and fingerprint, needs more privacy protection (Minaee et al.) (023). Compared with baseline methods, our method can forget the identity information better without forgetting too much remaining information in the dataset. This paradigm provides practical usage of MU and our methods provide faster and more reliable way to improve the MU performance.
way
All Metrics
100
30
Mantc VaM
00
MIA
Monic
00
Difference
0.50
0.22
0.64
11.55
0.99
0.50
0152
0134
0.00
0.55
0.50 0.82
0.94
0.50
0.85
0.90
0.92
0:34
0.96
0.35
Figure 3: Ablation study on the GA ratio p for random forgetting (10%) on CIFAR-10. 10. Our methods (left: UGradSI., right: UGradSL++) remain relatively stable across a wide range of p under multiple evaluation metrics.
All Metries
All Metries 12
100
80
Monri Valae
60
MIA
Morie
RA
40
TA
RA
Diffcrence
20-
43.
400
42.5
10.2 40.
10.9
23.5
12.7
10.5
40.5
10.4
133
4..2
-0.1
Figure 4: Ablation study on the smoothing rate for random forgetting (10%) CIFAR-10. Our methods (left: UGradSI, right: UGradSL+) remain relatively stable across a wide range of a under multiple evaluation metries.
5.3 ABLATION STUDIES
To evaluate the robustness of our method, we conduct ablation studies on the forgetting set size, the GA ratio p. and the smoothing rate a. For the forgetting set size, we do the experiments on CIFAR- 10 and CIFAR-IOW 100. The results in Tables 17 and show that our method is consistently robust to the size of Dy and always outperforms the other baselines. For the trade-off parameter p. Table T9 indicates that pure GA is unstable, and its performance becomes more stable as more GA steps practice, we therefore choose p > 0.9 to stabilize performance
are combined with our method. while still leveraging the effect of GA for unlearning. We further study random forgetting (10%) on CIFAR-10 10, fixing a = -0.4 for both UGradSL and UGradSL+. and sweep p in [0.8, 1.0] with step size of 0.01. The resulting UA, MIA, RA, TA, and Avg. Gap curves in Figure S show that our methods are relatively stable with respect to p. For the smoothing rate a, we fix 0.9 and
vary from -0.9 to -0.1 with step size of 0.1. The results in Figure 4 demonstrate that both

---

Published as a conference paper at ICLR 2026
UGradSL and UGradSL+ remain stable. Overall, these ablations show that our methods robust to the choice of hyperparameters.
Table 4: Results of class wise forgetting and random forgetting on CIFAR-10 with additional (add.)
MIA. The best comprehensive metrics are bold. Avg. Gap is calculated with additional MIA.
Clasa-rite
Rundon
Mils
Reaoh
100 x
3462
17.40
24.66
37
00:11
10-51 -mo0
19
16. 14
(0:0)-
20155
3.60
1,00y1
511
1:58
GA
13.10
13-0
08.441144
24.05
8
1,0kai0
10-45a-0
44.5
0.37
88
16.10 10
19:330m
16:33.
1010)
151
0.30
600
1010
15 m
6.14
3.13
130.00u
00..3...0
17
06:12220
043
150
2.65
1.58
BL--
03100
19
16:00
145
3131.131
10.12
100.30
10310..m
(8-1acas
SCRLD
100 139.0...
100.50..
5.75
S+N
13.76-194
16:20/03
19.05.955
4.18
1.2
0335.1
28.24
244
UiraBi
30.75
(6/
0.2
1350
2.07
0.45
19100 x
19.3
06.23p00
191.00u
8.43
140
10.7
$1
10.0
2353
300
5.4 DISCUSSION
Inftuence Function in Deep Learning Influence functions were originally proposed for the convex function. As given in Section 3.2 we apply the influence function to the converged model, which can be regarded as a local convex model. A plot of loss landscape of the retrained model 6, on CIFAR-10 10 dataset is given in Figure in Appendix.
MIA as a Proxy for "Forgetfulness". Given a model 0.. we can evaluate the degree of its general- ization by running membership inference attack on the model. In the context of the current work, generalization is equivalent to the degree of "forgetfulness" that the forgetting algorithm achieves. Given the distribution of model response observations A, = A(O., .D)) and Ate A(O., Dre), where A is an adversary and A = Ay U Are is the observation visible to A, one can get the degree of generalization by analyzing the observations. In the context of MU, the most straightforward way is to get the accuracy of A on the seen and unseen samples (Die and D, respectively). This could be done by computing the (TP + TN) y1y [Dtel), where the true positive (TP) predictions corre- spond to "seen" samples, and true negative (TN) predictions are "unseen" samples. We conducted the experiments on CI both for class-wise wise and random forgetting. The results are given in Table 4 where Avg. Gap is calculated with additional MIA. We assume that the distribution of D and Dre should be the same. For class-wise forgetting, the additional MIA is almost because D, is separate single class and the distribution of D, and Die without the corresponding class totally different. For random forgetting, the additional MIA is almost 0.5 because D, is randomly selected from Dtr and the distribution of D, and D... should be the same. The plots of loss distri- bution for random and class-wise forgetting are given in Figure 8 in the Appendix. In Table g the proposed methods still outperform the other baseline methods, showing the robustness to the other MIA auditing methods and the generalization capability in privacy preservation.
Difference between UGradSL and UGradSL+ Although two methods are similar in the math- ematical formulation, there exists fundamental difference in their design and behavior. Compared with UGradSL, UGradSL+ can be more stable and less sensitive due to its origin from FT. As shown in the experiment results in the tables, UGradSL+ can always perform as top-tier methods. However, the RTE of UGradSL++ would be higher. We present more analysis in Appendix E.
The study of the Streisand effect (Jansen & Martin 2015) and gradient analysis are given in Ap- pendix E.6 and H. 10 respectively.
6 CONCLUSIONS
We have proposed UGradSL, pluggandd-play efficient, gradient- based MU method using smoothed labels. Theoretical proofs and extensive numerical experiments have demonstrated the effectiveness of the proposed method.
10

---

Published as a conference paper at ICLR 2026
ETHIC STATEMENT
This work does not involve potential malicious or unintended uses, faimess considerations, privacy considerations, security considerations, crowd sourcing, research with human subjects.
REPRODUCIBILITY STATEMENT
We provide details to reproduce our results Appendix D Daadd Algorithm and will release the code upon acceptance.
E
We also provide pseudo-code in
ACKNOWLEDGMENTS
Z. Di and Y. Liu are partially supported by the National Science Foundation (NSF) under grants IIS-2143955 and IIS-2040800. The work of J. Jia and S. Liu is supported in part by the Open Philanthropy Research Award.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308-318, 2016.
Samyadeep Basu, Xuchen You, and Soheil Feizi. On second-order group influence functions for black-box predictions. International Conference on Machine Learning, pp. 715-724. PMLR, 2020.
Alexander Becker and Thomas Liebig. Evaluating machine unlearning via epistemic uncertainty. arXiv preprint 1XXIv:2208.183666 2022.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141-159. IEEE, 2021.
Brunet, Colleen Alkalay-Houlihan Ashton Anderson, and Richard Zemel. Under- standing the origins bias in word embeddings. International conference on machine learn- ing. pp. 801-811. PMLR, 2019.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. 2015 IEEE Symposium Security and Privacy, pp. 463-480. IEEE, 2015.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee. Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pp. 2633-2650) 2021.
Min Chen, Zhikun Zhang, Tianhao Wang. Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearming jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC confer- ence on computer and communications security, pp. 996-911, 2021a.
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearning jeopardizes privacy. Proceedings of the 2021 ACM SIGSAC Con- ference on Computer and Communications Security, pp. 896-911, 2011b.
Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang Graph unlearning. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Com- munications Security, pp. 499-513, 2022.
Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning. arXiv preprint arXiv:2303.115:7 /5500, 2023.

---

Published as a conference paper at ICLR 2026
Eli Chien, Chao Pan, and Olgica Milenkovic. Certified graph unlearning. arXiv:2206.09140. 2022.
arXiv preprint
Dasol Choi, Soora Choi, Eunsun Lee, Jinwoo Seo, and Dongbin Na. Towards efficient machine un- learning with data augmentation: Guided loss-increasing (gli) to prevent the catastrophic model utility drop. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 93-102, 2024.
Christopher A Choquette- Choqu,tte-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. Label-only membership inference attacks. In International conference on machine learning, pp. 1964-1974. PMLR, 2021.
Jan Chorowski and Navdcep Jaitly. Towards better decoding and language model integration in sequence sequence models. arXiv preprint arXiv:1612.022693 2016.
Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using incompetent teacher. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 7210-7217, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Jacob Devlin, Ming- Meig Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018.
Jimmy Z Di, Jack Douglas, Jayadev Acharya, Gautam Kamath, and Ayush Sekhari. Hidden poison: Machine unlearning enables camouflaged poisoning attacks. In NeurIPS ML Safery Workshop, 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint UrXiv:2010.11922 2020.
Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, llya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Annual international conference on the theory and applications of cryptographic techniques, pp. 486-503. Springer, 2006.
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, and Sijia Liu. Salun: Em- powering machine unlearning via gradient- basded weight saliency in both image classification and generation. arXiv preprint arXiv:2310. 12508, 2023.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confi- dence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC confer- computer and communications security, pp.
Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp. 619-633, 2018.
Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in machine learning. Advances in neural information processing systems, 32, 2019.
Shashwat Goel, Ameya Prabhu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06644) 2022.
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9304-9311 2020.
12

---

Published as a conference paper at ICLR 2026
Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11516-11524, 2021.
Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv:1911.0300 2019.
Rob Hall, Alessandro Rinaldo, and Larry Wasserman. Differential privacy for functions and func- tional data. arXiv preprint /203.2 2012.
Tomohiro Hayase, Suguru Yasutomi, and Takashi Katoh. Selective forgetting of deep networks at finer level than samples. arXiv preprins arXiv:2012. /849, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Sorami Hisamoto, Matt Post, and Kevin Duh. Membership inference attacks on sequence-to- sequence models: Is my data in your machine translation system? Transactions the Association for Computational Linguistics, 8:49-63, 63, 2020.
Nils Homer, Szaboles Szelinger, Margot Redman, David Duggan, Waibhav Tembe, Jill Muehling. John Pearson, Dietrich A Stephan, Stanley F Nelson, and David W Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLoS genetics, 10001677, 2008.
Chris Jay Hoofnagle, Bart van der Sloot, and Frederik Zuiderveen Borgesius. The european union general data protection regulation: what it is and what it means. Information Communications Technology Law, 28(1):65-99) 69,9, 2019.
Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. Approximate data dele- tion from machine learning models. In International Conference on Artificial Intelligence and Statistics, pp. 2008-2016 PMLR, 2021.
Saachi Jain, Hadi Salman, Alaa Khaddaj, Eric Wong, Sung Min Park, and Aleksander Madry. A data-based perspective on transfer learning, arXiv preprint arXiv:2207.05733, 2022.
Sue Curry Jansen and Brian Martin. The streisand effect and censorship backfire. International Journal of Communication, 9:16, 2015.
Zhanglong Ji, Zachary C Lipton, and Charles Elkan. Differential privacy and machine learning: survey and review. arXiv preprint arXiv:1442.P... 2014.
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsification can simplify machine unlearning. arXiv preprint arXiv:2304.04934 2023.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pp. 1885-1894. PMLR, 2017.
Pang Wei W Koh, Kai-Siang Ang. Hubert Teo, and Percy Liang. On the accuracy of influence functions for measuring group effects. Advances in neural information processing systems, 32, 2019.
Anastasia Koloskova, Youssef Allouah, Animesh Jha, Rachid Guerraoui, and Sanmi Koyejo. Certi- fied unlearning for neural networks. arXiv preprint arXiv:2506.0604 2025.
Shuming Kong, Yanyan Shen, and Linpeng Huang. Resolving training biases via influence-based data relabeling, In International Conference on Learning Representations, 2021.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Meghdad Kurmanji, Peter Triantafillou, and Eleni Triantafillou. Towards unbounded machine un- learning. arXiv preprint arXiv:2302.09880 2023.
13

---

Published as a conference paper at ICLR 2026
Ken Lang. Newsweeder: Learning to filter netnews. In Machine learning proceedings 1995, pp- 331-339. Elsevier, 1995.
Klas Leino and Matt Fredrikson. Stolen memories: Leveraging model memorization for calibrated [White-Box) membership inference. In 29th USENIX security symposium (USENIX Security 20), pp. 1605-1622, 2020.
Zheng Li and Yang Zhang. Membership leakage in label-only exposures. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, pp. 880-895, 2021.
Yang Liu, Mingyuan Fan, Cen Chen, Ximeng Liu, Zhuo Ma, Li Wang, and Jianfeng Ma. Backdoor defense with machine unlearning. arXiv preprint Sativ:220110953.. 2022a.
Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang. and Bo Li. The right be forgotten in federated learning: An efficient realization with rapid retraining. arXiv preprint arXiv:2203.0753 2022b.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Michal Lukasik, Srinadh Bhojanapalli, Aditya Menon, and Sanjiv Kumar. Does label smoothing mitigate label noise? In International Conference on Machine Learning, pp. 6448-6458. PMLR, 2020.
Neil G Marchant, Benjamin IP Rubinstein, and Scott Alfeld. Hard to forget: Poisoning attacks on certified machine unlearning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 7691-7700, 2022.
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. arXiv preprint arXiv: 10.0666,, 2017.
Shervin Minaee, Amirali Abdolrashidi, Hang Su, Mohammed Bennamoun, and David Zhang. Bio- metrics recognition using deep learning: survey. Artificial Intelligence Review, pp. 1 1-49, 2023.
Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help? Ad- vances in neural information processing systems, 32, 2019.
Milad Nasr, Reza Shokri, and Amir Houmansadr. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In 2019 IEEE symposium on security and privacy (SP), pp. 739-753. IEEF, 2019.
Seth Neel, Aaron Roth, and Saeed Sharifi Maleiii-Ma..j.rdd... for machine unlearning. In Algorithmic Learning Theory, pp. 931-962. PMLR, 2021.
:eeele-- Gradient based methods
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoe Viet Hung Nguyen. survey of machine unlearning. arXiv preprint 2rXiv:2209.02299, 2022.
Stuart I. Pardau. The california consumer privacy act: Towards european-style privacy regime in the united states. J. Tech. L. Pol'y, 23:68, 2018
Gabriel Pereyra, George Tucker, Jan Chorowski, Fukasz. Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. arXiv preprint 170 2017.
Md Atiqur Rahman, Tanzila Rahman, Robert Laganière, Noman Mohammed, and Yang Wang. Membership inference attack against differentially private deep learning model. Trans. Data Priv. 11(1):61-79, 2018
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Hervé Jégou. White- box vs box: Bayes optimal strategies for membership inference. In International Confer- on Machine Learning, pp. 5558-5567. PMLR, 2019.
14

---

Published as a conference paper at ICLR 2026
Prasanna Sattigeri, Soumya Ghosh, Inkit Padhi, Pierr Dognin, and Kush R. Varshney. Fair in- finitesimal jackknife: Mitigating the influence biased training data points without refitting, In Advances in Neural Information Processing Systems, 2022.
Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Pro- cessing Systems, 34:18075-18088 2021.
Reza Shokri and Vitaly Shmatikov.
deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pp. 1310-1321,
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. 2017 IEEE symposium on security and privacy (SP), pp. 38. IEEE, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint vrXiv:1409.15555 2014.
Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX security symposium (USENIX security 21), pp. 2615-2632. 2021.
Thomas Steinke, Milad Nasr, and Matthew Jagielski. Privacy auditing with one (1) training run. Advances in Neural Information Processing Systems, 36:49268 49288,,99..000 2023.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
Anvith Thudi, Gabriel Deza, Varun Chandrasckaran, and Nicolas Papernot Unrolling sgd: Under- standing factors influencing machine unlearning, arXiv preprint arXiv:2109.13330 2021.
Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. Machine unlearning via algorithmic stability. In Conference on Learning Theory, pp. 4126-4142. PMLR, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
Jialu Wang, Xin Eric Wang, and Yang Liu. Understanding instance-level impact of fairness con- straints. In International Conference on Machine Learning, pp. 231 13-14-33.0. PMLR, 2022a.
Junxiao Wang, Song Guo, Xin Xie, and Heng Qi. Federated unlearning via dlassddiscimminative pruning. In Proceedings of the ACM Web Conference 2022, pp. 622-632, 2022b.
Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. arXiv preprint arXiv:2108.1 1157,, 2021.
Jiaheng Wei, Hangyu Liu, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Yang Liu. To smooth or not? when label smoothing meets noisy labels. Learning, ((1)ee1, 2021.
Haotian Ye, Chuanlong Xie, Yue Liu, and Zhenguo Li. Out-of -uitt--diistiiutionn generalization analysis via influence function. arXiv preprint arXiv:2 1xXix:2201.0880 101.0221..111.. 2021.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learn- ing: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pp. 268 -22. IEEE, 2018.
15


Reviews:
[{"review_id": "MhWTIkbMtR", "reviewer": "Reviewer_be9Q", "summary": "This paper proposes UGradSL, which is a gradient-based unlearning methods that integrate label smoothing. The approach performs gradient ascent on forget data and gradient descent on retain data, using smoothed labels to balance forgetting and retaining.\nTheoretical analyses explain when NLS improves gradient ascent and link the method to label-local differential privacy (Label-LDP) guarantees. Experiments on several datasets demonstrate the effectiveness of the proposed.", "questions": "1. About forget-set size: have the authors evaluated the method under different forget-set sizes or compositions? Since forget-set size can significantly affect both unlearning and retention, understanding how the method scales with it would help assess its general applicability.\n2. In Eq (8), p controls the balance between gradient ascent (forgetting) and gradient descent (retaining). Could the authors provide experimental results for different p values? This would clarify how sensitive the method is to this balance.\n3. How was the smoothing rate chosen? It would be valuable to include sensitivity results or justification for the chosen range.\n4. How the density is computed in UGradSL+ and how it interacts with the theoretical framework?\n5. The paper suggests that UGradSL can be easily integrated into other unlearning frameworks. Have the authors tested this claim empirically?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper has clear problem framing and the motivation for introducing label smoothing to stablilize gradient ascent is sound.\n2. The theoretical analysis part is clear and provides useful insights.\n3. The proposed UGradSL is simple yet effective.\n4. The experiments cover class-level, random, and group unlearning and sufficient datasets and baselines.", "weaknesses": "1. The forget-set size: The forget-set size appears fixed for each experiment. Varying forget-set sizes is important for assessing the effectiveness of the method. This would reveal whether the proposed method scales well when more data need to be forgotten.\n2. Lack of ablation study/sensitivity analysis: The paper lacks a discussion on the contribution of each term in the mixed gradient objective. For example, results for different p should be provided, since it is an important factor used to balance GD and GA.\n3. Smoothing rate sensitivity: the effect of the label smoothing rate should be analyzed, and a discussion on how much the observed benefits arise from smoothing is worth adding in.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JTwgOXPemv", "reviewer": "Reviewer_dpJc", "summary": "This paper concentrates on the low-computation and high-efficiency machine unlearning in the field of computer vision. The author proposed a novel method UGradSL and UGradSL+ (finetune based) combine the GA (gradient ascent) with the NLS (negative label smoothing) and apply mix-gradient strategy which perform GD (gradient descent) on the retain data and GA with NLS on the forget data. Moreover, the stronger variant self-adaptive UGradSL supports the automatic selection of the smooth rate. The paper provides the theoretical analysis and abundant experiments to prove the effectiveness and robustness of method. In addition, the study also explored the label smoothing and local differential privacy (LDP).", "questions": "* Could the authors do more comparative experiment with new methods, such as [1],[2] and etc.\n* The authors should detailly check the symbols in the article and conduct standardized descriptions and definitions.\n* The authors should analyze and explain the generalization gap between the RA and TA, where the increase of RA and decrease of TA reflect the overfitting and memory to the retain data rather than learning the generalization features.\n* Since the efficiency is the core advantage of the proposed method, the authors should compare the memory cost during the unlearning process (such as peak GPU memory usage and etc.), which may help other researchers make decision with limited resources.\n* Could the authors do another ablation experiment to the distance computation and automatic parameter selection in Algorithm 1, which helps the observation of whether the improvement of self-adaptive UGradSL comes at the cost of disproportionate calculation time.\n\nReference:\n[1] Certified Unlearning for Neural Networks\n[2] Towards Efficient Machine Unlearning with Data Augmentation: Guided Loss-Increasing (GLI) to Prevent the Catastrophic Model Utility Drop", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ This paper is clear, logical and easy to understand. The tables and figure are clear and detailed, with good instructions.\n+ The proposed method is a simple and plug-and-play tool, which can directly integrate into the existing gradient-based unlearning methods (such as GA and finetune) and improve their performance.\n+ This paper provides the mathematical and theory proof to explain the limitation of existing GA methods and the effectiveness of NLS in specific circumstances. Moreover, the authors innovatively establish the connection between label smoothing and LDP.", "weaknesses": "- Although the baseline methods are representative, the experiments lack comparison with the latest schemes between 2024 and 2025.\n- Some symbols and formulas lack precise definitions and explanations or exist clerical, such as “distance d()” in the Algorithm 1 where different distance calculation methods can lead to the difference between computational overhead and performance.\n- The performance of the method is sensitive to the hyperparameters settings (such as p and α in Eq.8), which may result in the risk of overfitting to the retain classes.\n- Since the time cost is only a part of the computational overhead, the analysis of computational resource overhead is incomplete. And the automatic parameter selection in self-adaptive version also may lead to the additional time cost and computational complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XfLvbGto5Q", "reviewer": "Reviewer_Xe6w", "summary": "This paper integrates label smoothing into the unlearning loss to reduce time cost and mitigate drops in remaining and test accuracies. It proves that gradient ascent can achieve exact unlearning, and generalized label smoothing tightens errors. Building on this, it introduces UGradSL and UGradSL+, which iterate over retained and forgetting datasets, and show gains on class, random, and sub-class unlearning versus baselines.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Label smoothing combined with a gradient-based method is interesting for handling unlearning\n\n2. Both theoretical and experimental evidence support the effectiveness of the proposed method.", "weaknesses": "1. The presentation of the technical sections is not clear, for example, why there exists an $\\approx$ symbol in the condition of Theorem 1 and why $\\epsilon$ in the conclusion of $\\epsilon$-Label-LDP relies on the weights $\\gamma_1$ and $\\gamma_2$.\n\n2. The proposed method requires calculating the distance with samples in the minibatch, which will lead to a large computation cost.\n\n3. The proposed UGradSL does not work better in performance, while UGradSL+ requires a longer time than other baselines.\n\n4. This paper does not contain essential abolition studies for the GA ratio and the optional smoothing ratio", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RvhEcVngQb", "reviewer": "Reviewer_Zyp9", "summary": "The paper proposes a gradient-based machine unlearning (MU) method, the UGradSL, that integrates label smoothing (LS) into the unlearning in the ways of fine-tuning and gradient accent.  \nThe key idea is to perform gradient ascent with generalized label smoothing (GLS) on the forgetting set $D_f$, and gradient descent on the retaining set $D_r$, forming a mixed-gradient optimization.  \nThe authors show a theoretical condition under which NLS improves gradient ascent.\nExperiments across multiple datasets and model architectures (ResNet18, ViT, BERT) show improved unlearning–retaining trade-offs with small computational overhead.", "questions": "Besides the weaknesses:\n1. In Algorithm 1, the distance computation $d(z^r_i, z^f_i)$ is not clearly defined. What distance metric? In which space (feature/parameter)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Easily pluggable into existing GA/FT unlearning pipelines without retraining.\n2. Broad Empirical Evaluation. Tested on six datasets and multiple unlearning types (class, random, group), consistently outperforming baselines.\n3. Provided theoretical intuitions. Explained why NLS can guide ascent toward the equivalent retrained solution.", "weaknesses": "1. The condition of the inner product of $\\Delta\\theta$'s in Theorem 2 is critical but only verified empirically on one dataset (CelebA, Figure 4 in the Appendix). This lacks theoretical justification and broader empirical validation.\n\n2. Theorem 3's connection to LDP is interesting but the practical meaning is unclear - it provides label-level privacy for the forgetting set, but doesn't directly translate to model-level unlearning guarantees.\n\n3. Hyperparameter sensitivity needs ablation studies. A demonstration about the smoothing rate $\\alpha$'s influence on performance is expected. While an adaptive version is proposed, the distance threshold $\\beta$ introduces another parameter.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)