Remaining-data-free Machine Unlearning by Suppressing Sample Contribution
Authors: Xinwen Cheng
Zhehao Huang
Wenxing Zhou
Zhengbao He
Ruikai Yang
Yingwen Wu
Xiaolin Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Machine Unlearning
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: We develop an effective and efficient machine unlearning method that could unlearn without utilizing the remaining data to compensate model utility.

Abstract: Machine unlearning (MU) aims to remove the influence of specific training samples from a well-trained model, a task of growing importance due to the ``right to be forgotten.” The unlearned model should approach the retrained model, where forgetting data do not contribute to the training process. Therefore, unlearning should withdraw their contribution from the pre-trained model. However, quantifying and disentangling sample's contribution to overall learning process is highly challenging, leading most existing MU approaches to adopt other heuristic strategies such as random labeling or knowledge distillation. These operations inevitably degrade model utility, requiring additional maintenance with remaining data. To advance MU towards better utility and efficiency for practical deployment, we seek to approximate sample contribution with only the pre-trained model. We theoretically and empirically reveal that sample's contribution during training manifests in the learned model's increased sensitivity to it. In light of this, we propose MU-Mis (Machine Unlearning by Minimizing input sensitivity), which directly suppresses the contribution of  forgetting data. This  straightforward suppression enables MU-Mis to successfully unlearn without degrading model utility on the remaining data, thereby eliminating the need for access to the remaining data. To the best of our knowledge, this is the first time that a remaining-data-free method can perform on par with top performing remaining-data-dependent methods.

Published as a conference paper at ICLR 2026
REMAINING-DATA--RRE MACHINE UNLEARNING BY SUPPRESSING SAMPLE CONTRIBUTION
Xinwen Cheng' Zhehao Huang' Wenxing Zhou' Zhengbao He' Ruikai Yang' Yingwen Wu' Xiaolin Huang*t:2
Institute of Image Processing and Pattern Recognition,
School of Automation and Intelligent Sensing. Shanghai Jiao Tong University
Shanghai Key Laboratory of Flexible Medical Robotics
{xi xxinwencheng, kinght_h, apricvivi, 1stefanie @sjtu. edu. yangg yang,,
(ruikai.
yingwenwu, xiaolinhuang dajjt..ee edu.cn cn
ABSTRACT
Machine unlearning (MU) aims to remove the influence of specific training samples from a well-trained model, a task of growing importance due to the "right to be forgotten." The unlearned model should approach the retrained model, where for- getting data do not contribute to the training process. Therefore, unlearning should withdraw their contribution from the pre-trained model. However, quantifying and disentangling sample's contribution to overall learning process is highly chal- lenging, leading most existing MU approaches to adopt other heuristic strategies such as random labeling or knowledge distillation. These operations inevitably degrade model utility, requiring additional maintenance with remaining data. To advance MU towards better utility and efficiency for practical deployment, we seek to approximate sample contribution with only the pre-trained model. We theoretically and empirically reveal that sample's contribution during training man- ifests in the learned model's increased sensitivity to it. light of this, we propose MU-Mis (Machine Unlearning by Minimizing input sensitivity), which directly suppresses the contribution of forgetting data. This straightforward suppression enables MU-Mis successfully unlearn without degrading model utility the remaining data, thereby eliminating the need for access to the remaining data. To the best of knowledge, this is the first time that a remaining-data can perform on par with top performing remaining- code is released in https: /ttp://gtttt gigghu......... com/ pwwppp/pppppppannooooooooooooee. 903 903/MU-Mis.
method methods. The
INTRODUCTION
Deep neural networks (DNNs) are revealed to store information of training data (Feldman, 2020; Feldman & Zhang, 2020; Tian et al., 2025) and such information could be reproduced by privacy attacks (Shokri et al., 2017: Zhu et al., 2019), raising data privacy concerns. The "right to be forgotten" privacy, which entails ensuring that the DNN performs as if the data were never involved in the training.
(Regulation, 2018) is introduced to safeguard
While retraining from scratch would ideally achieve this, it is often infeasible due the high cost of training DNNs. This has motivated the study of "Machine Unlearning" (MU) (Cao & Yang, 2015), which fine-tunes the pre-trained model to approach the retrained model closely as possible. The essential distinction in pre-trained and retrained model lies in the contribution of forgetting data, whose role shift from "contributors" that affect parameter updates the pre-trained model to "bystanders" that exert no influence in the retrained model. Therefore, unlearning should aim to withdraw their contribution the learning process.
However, identifying such contribution is highly challenging. Learning is dynamic process that gradually remembers and assimilates data, while unlearning, which is the reverse process that gradually removes data information, is achieved by backtracking the training trajectory to withdraw
"Corresponiigg author

---

Published as a conference paper at ICLR 2026
historical gradients in early study (Graves et al., 2021; Thudi et al., 2022). Nevertheless, such tracking not only contradicts the efficiency demands of unlearning but also yields limited effectiveness due to the incrementality of training (Wang et al., 2024b).
Consequently, most existing MU methods circumvent the difficulty of estimating sample contribution through other heuristics. A common strategy is to introduce confusion, e.g., random relabeling (Golatkar et al., 2020; Graves et al., 2021; Fan et al., 2024b) or knowledge distillation from useless teacher (Chundawat et al., 2023; Kurmanji al., 2023). However, these approaches suffer from several limitations: (i) such confusion causes catastrophe unlearning (Wang et al., 2024b) or over- forgetting (He et al., 2025). i.e., severe degradation of model utility on the remaining data; (ii) the degradation in turn necessitates costly maintenance using the remaining data, thereby substantially undermining MU efficiency; (iii) the remaining data are not always accessible in practice. These limitations collectively underscore the importance of moving beyond heuristic confusion strategies and developing more principled unlearning mechanisms to advance MU toward higher utility and efficiency. Therefore, although quantifying sample contribution is inherently challenging, in this paper, we make efforts to ground unlearning in a precise characterization of sample's 's contribution.
Instead of accumulating the historical contributed gradient update during training, we identify the clue of contribution directly from the derivative of the training algorithm w.r.t a training sample. The learning process is mapping by the training algorithm A from the training set D == {(x,, y.)} to a learned function f: denoted as f = A(D). Therefore, the training sample 2; contributes to the output: 0.A/ & 0 while a sample out of the training set does not. A simple yet enlightening example lies in the support vector machine (Cortes & Vapnik, 1995; Christmann & Steinwart, 2008), where only the training data can act support vectors that impact the decision boundary. Thus, withdrawing the sample contribution can be achieved by suppressing 0A/O 0A/0x;;.
The main challenge is that A corresponds to dynamic training process without a closed-form expression. To address this, we theoretically illustrate that 0,/Oxx; could be approximated by the learned model's sensitivity to its input æ, i.e. (f(z) with f = A(D) in Section 3.2. To derive a guideline aligned with the behavior of retrained model, we
principled and optimization-friendee delve deeper into the input sensitivity across different logits. Our empirical investigations under the machine learning (Section 3.3) and machine unlearning (Section 3.4) scenarios reveal that sample's contribution manifests as disproportionately higher input sensitivity of the target logit relative to irrelevant logits. In light of this finding, propose MU-Mis (Machine Unlearning by Minimizing Input Sensitivity), which suppresses sample contribution by reducing the sensitivity disparity between the target and non-target logits to the forgetting data.
We evaluate MU-Mis on 3 standard unlearning tasks across 6 datasets, benchmarking against 6 com- samaiiing-dataddeeeeeeee data-dependent unlearning methods and 4 existing remaining-daii data-free baselines. The results demonstrate that MU-Mis achieves effective unlearning while preserving model utility on the remaining data without utilizing them, performing on par with SoTA manaining-datt-deeeneee approaches and outperforming all remaining-datee tage of notable computational efficiency. Moreover, due to its principled forgetting mechanism. MU-Mis exhibits stable and effective behavior in sequential unlearning, whereas existing methods are disclosed to exhibit several deficiencies. Collectively, these results underscore the practicality and reliability of MU-Mis for real-world deployment.
petitive remaining
methods significantly, with the added advan-
Our key contributions can be summarized as follows:
We theoretically and empirically reveal that sample's s contribution is reflected in the amplified sensitivity gap between the target logit and irrelevant logits, enabling the identification of sample contribution with only the pre-trained model.
Based on the above analysis and findings, we propose MU-Mis, which suppresses the sample's contribution by minimizing the sensitivity magnitude gap for the forgetting data.
Comprehensive experiments demonstrate the effectiveness and efficiency of MU-Mis. To our best knowledge, it is the first time that remaining -fta--freeee method can perform on par with top performing manaiingg-dat-deeeeeee methods.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Machine unlearning (MU) (Xu et al., 2024; Bourtoule al., 2021) aims to remove the influence of specific training data from pre-trained model to mitigate privacy risks. MU is commonly divided into exact and approximate settings (Shaik et al., 2023). Exact MU approaches the parameters of the retrained model and provides statistical guarantees of privacy risk(Guo et al., 2020; Suriyakumar & Wilson, 2022; Neel et al., 2021; Giordano et al., 2019), whereas approximate MU targets the oupur distribution of the retrained model. In this paper, we concentrate approximate unlearning, it is more practical in large-scale models and situations with limited time and resources.
MU via gradient- based updates. One straightforward way to retrieve sample contribution is keep and utilize the historical information (e.g., checkpoints and gradients) during training. Ammesiac (Graves al., 2021) withdraw gradient updates of related batches, and DeltaGrad (Wu et al., 2020) utilize intermediate checkpoints and quasi-newton method for rapid retraining, Memory concerns arising from storing historical information motivate estimating sample contribution from learned model via influence function (Guo et al., 2020). However, inverse Hessian is computationally prohibitive for DNNs, prompting various approximation methods tailored for unlearning (Peste et al., 2021; Mehta et al., 2022; Meng et al., 2022). However, influence function is revealed to be fragile in DNNs (Basu et al., 2020; Bae et al., 2022; Hammoudeh Lowd, 2024) due to its reliance on convexity and optimality assumptions, accounting for their failure to preserve utility on remaining data (Wu et al., 2022). Regrettably, all the existing DNN data-influence estimators (Pruthi et al., 2020; Chen et al., 2021) require retracing training trajectories, therefore cannot be readily optimized and applied to MU. In this paper, we shift sample contribution from parameter space to function space, i..,, Aw to 0A/Oz, and demonstrate that sample's contribution would reflect in pre-trained model's sensitivity to itself, opening up a new perspective for measuring sample contribution in DNNs.
MU via loss guided re-optimization. Above gradient-based unlearning methods suffer from practical limitations for DNNs. Generally, practical MU unlearn by fine-tuning the model to optimize a proposed loss. The loss typically follows two design ideas: (i) make model's behavior on the forgetting data similar to that on unseen data through knowledge distillation (Chundawat et al., 2023; Lin et al., 2023; Kurmanji et al., 2023) or label confusion (Graves al., 2021; Fan et al., 2024b), and (ii) suppress parameters that are responsible for the forgetting set (Liu et al., 2024; Foster et al., 2024a; Fan al., 2024b). However, for lack of identifying "what to unlearn,, these methods unlearn either in an "impair-then-repe"
regime (Tarun et al., 2023) or introduce tailored mechanisms (Hoang et al., 2024; Fan et al., 2024b) to mitigate collateral damage. In contrast, we pursue a more principled forgetting operation by explicitly identifying sample contributions, eliminating ad-hoc compensation.
Remaining-data freee MU. Developing camaining-data-lle methods aligns more closely with the essence and practical demands of MU, given the limited accessibility of retained data and MU efficiency in practice. JiT (Foster et al., 2024a) smooths model outputs around forgetting data by minimizing local Lipschitz value, while SCAR (Bonato al., 2024) distills from the tre-trained model and utilizes Out-of-distributioo (OOD) data to preserve utility. However, both methods have clear performance gap SoTA dadeeeeeed methods, and SCAR still relies on additional OOD data. We achieve a more nuanced removal by identifying sample contribution.
MACHINE LEARNING, MACHINE UNLEARNING AND INPUT SENSITIVITY
3.1 PROBLEM FORMULATION
Machine Learning (ML) is to learn mapping from the input space X' to the output space )/ denoted as the function f(-) X + y: As we mainly focus on classification models, the output of f is C-dimensional in C-category classification model. Learning is performed by a training algorithm A, which generally takes in a training dataset D and returns the learned function f. ie., the outcome of A varies with different training datasets. To investigate sample-wise influence on the learning process, we consider A in a broader sense and distinguish different training processes by the training dataset D [xx, ) That is to say, we have a family of the training algorithm Ap and each one is a multivariate function that takes all the samples {I, E X'} as input, regardless of whether they in the training dataset D. Therefore, the output of Ap does not vary with each input variable, but only varies with the change of the training data x; E D. and makes no response to the change of samples out of the training set.
3

---

Published as a conference paper at ICLR 2026
Machine Unlearning (MU) is to remove the influence of forgetting data Dj C. D from the pre- trained model Wp. while preserving model utility on the remaining data D, = D. The learned function f is parameterized by parameters w E R² with input variable æ, i.e. instantiated as f(;; w). A good approximate unlearning mechanism should efficiently and effectively transform up into sanitized model Wu, such that the output distribution of wh closely matches retrained model Wr-
Remark on notation. To facilitate the understanding of the objectives in our analysis, only bold the input variables of the training algorithm Ad and learned function f() which are respectively 24 and in the following analysis.
3.2 THEORETICAL ANALYSIS CONNECTING SAMPLE CONTRIBUTION AND INPUT SENSITIVITY
As previously discussed, machine unlearning is to withdraw sample's 's contribution to the learning process, and an efficient unlearning method should explore the contribution directly from the pre- trained model. To detach per-sample contribution with the pre- pre-trained model, we propose to identify the clue of contribution from the derivative of training mapping Ap to training sample at, i.e. PAAp/ar: Recall that Ap is determined by the training dataset D = ((rz yt)) and outputs the learned function (a; wp). Then 0Ap/ is to compute (f;;; /80g. However, there is no explicit expression for this derivative. Therefore, in this part, reflect on the learning dynamics to seek a surrogate with the pre-trained model. Figure provides an overview of the key objectives investigated in our following analysis.
Learned function
Sample Contribution
wx;
Input Sensitivity
0A/Ox;
o 10x
Train sample x;
GD update Awk
Pre-trained Model Wpp
Model input X
Figure I: A brief overview of the theoretical connection between sample's contribution and a pre-trained model's input sensitivity. The dashed lines illustrate how the influence of training sample propagates through gradient updates to the pre-trained model. When a sample participates in training. the gradient contributes induces an update of the model in function space, which inherently increases the Iearned function's 's sensitivity to that sample's 's input.
Gradient Descent (GD). After T iterations training updates in the parameter space, we have pre- trained model parameter up = Wy + Ei= Aw;, where Wp is randomly initialized model parameters and AwA- = Wk+1 - w) is the 1.th parameter update. Specifically, when training loss L and gradient descent with step size 17 are used, we have
OC(x;) Ow
0f w)
OC(x;) 0s
Awx
dw
w==
Function space update induced by GD. Viewing machine learning from the function space with first-order Taylor expansion on parameters, correspondingly we have f = fo + ΣI= Afe, where fo = f(x, wa) initial function and Afx is induced by parameter update Awk. The evolution in function induced by parameter update is:
A Afe(;; w) ~
Of(;; w
T
AWkk-
m
0f(;; w) Ow
T
af w)
al(x))
Ow
womork
Ow
0f
Learned function. To better explain the idea, we make simplifications: (i) Note that af((co))
Owr w= is
the mapping from model input æ to the induced backpropagation gradient with parameters wa. We abbreviate this mapping as g (a:) X + Rd and its derivative to input æ as g where d total number of model parameters. (ii) In classification problem with cross-entropy loss as L, we have ac(ms) of e. p(æ:), where e, is one-hot vector with only th element equals 1, and p is the probability vector of x;, The final learned function f
T
f(x; wp) = f(x; wo) + Σ Afr(e, w) = f(x; wa)
Ik (xx) p(xx))
1=
k= l
(2)
4

---

Published as a conference paper at ICLR 2026
Notice that term (1) related the forward inference process while term (2) is related to the machine learning process. Derivative of f w.r.t æ indicates how the prediction input æ at inference time, while derivative w.r.f æ; indicates how the learned function varies when the training sample 2; varies. The former implies the learned model's sensitivity to its input, and the latter is the training sample's influence on learning. Next, we take the derivative of f w.r.t and z; respectively to view their relationship. Note that is a probability vector determined by d;- Due to softmax activation, we consider p(æ;; hardly changes around aú and omit its derivative term w.r.t Vg. The difference in mapping 9a when æi changes is also omitted.
varies with its
(z; mp))
>
T
gl (24))) (a)) p(æi)).
8xc
k=
(w.w)
(I)
0f wp)
(z wa)
Σ.
()(()gg((s)((e p(x:))
dæ
Input sensitivity of learned function reflects sample contribution. C& (z, ax)) determines the prediction change on æ when æg changes, and S2 stands for the part of model's sensitivity to
contributed by training sample Ty. Note that @f(mm:)))) is similar to the definition of memorization, Umg
which is framed as self-influence (Feldman, 2020; Feldman & Zhang, 2020). To be more specific, memorization of sample is defined as the prediction difference in itself when training with or without it. Similarly, the self-influence here is the prediction difference on 2; when it slightly changes, i.e.
af(m)w))) Thus we consider @f(erw))) as the reflection of sample az 's contribution. From the de;
dis; formulation, we have Sy Sg(Di, mz4) (2; æt). For a specific training sample E 'D the learned model's s sensitivity to it can be further decomposed as
(5;;; wp)
0f wo)
T
m
Sa x.)
dx
ax
(z Wo)
Sx(2, +
Σ (2)
Oæ
BED
af( wo)
Sh. (5, 5)
Σ k= BED//
Σ ())
(2)
Ox
Contribution
Residial
The randomly initialized function fo is generally quite insensitive to input change. Thus, the first term of the above residual term is very small. The second term is related to the correlation between the gradient on i and the sensitivity of the gradient on é. We use a simple MLP model to illustrate the insight of S& (â. B) << S& (., &) with & * & in Appendix D. Therefore, the residual term is relatively smaller than the contribution term. In summary, the contribution of a training sample to the training process would be approximately reflected in the pre-trained model's output sensitivity the sample.
Empirical validation. We validate the con- tribution to learning by comparing V P
Frquency
Frequesey
0.050
of the training data before and after training in
0.02
0.055
Figure In a randomly initialized model, there is little response to input changes, only about 10-4. After training, there significant or- der of magnitude growth to 10 indicating increased attention of the trained model to the training data's 's variations. This implies that the training data contribute to model performance, and such efforts include promoting the model's sensitivity to them during training.
1.4
L6
2
Figure 2: Input sensitivity V/leee of training data before and after training, Left: In randomly initialized model up- Right: In well-trained model Wp. After training, the model exhibits significantly increased sensitivity the training data, reflecting their contribution during training.
3.3 INPUT SENSITIVITY OF THE TARGET AND IRRELEVANT CLASS LOGIT
During training, model predictions samples are driven toward their correct labels, sample contributions might differ across logits. To further refine our view sample contribution, we examine individual logits of ((x) E RCC in the following part.
5

---

Published as a conference paper at ICLR 2026
Let fc denotes the logit output of the target class and fer denotes the logit output of irrel- evant classes. Figure 3 compares distributions between / and o-i Eefe V ler (denoted as VV for brevity in the fol- lowing) of training data before and after train- ing. In the randomly initialized model, these two quantities are of comparative magni- tude, but //Veeell becomes much larger than V/ F after training. This observa- tion implies that samples contribute to ampli- fying [/Vleellr to surpass Ve || during training, generating discernible difference in whether sample has been learned. A complementary explanation of this finding comes from the generative view of discriminative models: the softmax-based discriminative classifier is revealed to be implicitly a density model which learns data distribution (Grathwohl et al.: Srinivas & Fleuret, 2021). From this viewpoint, the logits f(x) of standard classifiers are un normmalll log-densities, and corresponding input-grm prput-gradient V (x) In other words, we have V, log po (at rationale for the observed discrepancy.
0.06
Frequncy
0.06
Fresuenccy
0.04
0.04
0.02
0.02
0.00
0.00
2.0
2.5
3.0
1e.5
0.2
1.0
Norm
Norm
Figure 3: Input sensitivity |VxfellF and V || before and after training. Left: ran- domly initialized model Wo. Right: well-trained model wp. After training, the gap between target and irrelevant class sensitivities enlarges, providing a clearer signal of the sample's contribution.
(x) are log- grgdriidsstts of class class-conditional density model.
i) (i)i( in the classification model, providing a
3.4 INPUT SENSITIVITY OF SAMPLES PRESENT AND ABSENT IN TRAINING
For effective unlearning, the optimization objective should accurately steer the pre-trained model toward the retrained model. To validate that the theoretically grounded sensitivity gap provides a reliable measure of sample contribution to guide unlearning, we empirically examine the input sensitivity of forgetting data under MU scenarios (introduced in Section 5.1 and Appendix F.I).
For each forgetting sample, we compute the difference A between the retrained and the pre-trained model's sensitivity to it, where the sensitivity including V F, fe F and ae v le.. Aiming for a liight unlearning algorithm, we prefer an optimization direction rather than modeling a distribution or specifying a target value for each sample. Hence, we focus on the sign of A and count the ratio of rise and fall of A to examine the overall trend in Figure 4.
From left right in Figure 4, the sample-wise difference between the retrained and pre-trained model on V leallr//VV.... a/ | and V/ - ver l|. For each quantity, there is a consistent trend across different unlearning settings. Generally, fo of the retrained model exhibits lower sensitivity and for exhibits higher sensitivity to the forgetting data than the pre-trained model. And their sensitivity magnitude gap is consistently smaller the retrained model across different settings. Therefore, the sensitivity magnitude gap faithfully reflects the behavior of the retrained model and thus serves as a reliable objective to guide unlearning.
Rullclass-Cinn Cifarin
4=0
5=00
A
fulleiases coneniaiooooonniiooo
6>0
3>0
5>0
fullctoss- Thyy innggeentt
subeles-Ciifrtt Cif--claall-ekee
subeliss-Cifar Cifar2
suhclaos-CCin Cifa220-amm
andomsubses- Innglmagenet
Ratio
Ranio
Ranio
Figure 4: Ratio of input sensitivity difference A rise and fall of the forgetting data under different unlearning settings. From left to right, A is the sample-wise difference between the retrained and pre-trained model I/V ve || and ||Va/ellr |/ V p. Sample's contribution to input sensitivity includes promoting VVa.fa felfll and suppressing || V Fl thereby enlarging the magnitude gap V - V
PROPOSED METHOD
4.1 MU-MIS: MACHINE UNLFARNING BY MINIMIZING INPUT SENSITIVITY

---

Published as a conference paper at ICLR 2026
In the above section, theoretically and em- pirically derived an optimizable and lightweight approximation of sample contributions from the perspective of input sensitivity, showing that they manifest as disproportionately higher sensitivity of the target logit relative to irrelevant logits.
Algorithm 1 MU-Mis: Machine Unlearning by Minimizing Input Sensitivity
Input: Forgetting data Dy; Pre-trained model weights Wpi Learning rate 17; Stopping threshold ratio 8. # Initialization
In light of this finding, we propose to withdraw the
1: wo Wp. E
sample's s contribution by reducing such enhance- ment on the sensitivity magnitude gap. Minimiz- ing this loss guides the pre-trained model to roll back | xf and pick up V II.. Mathe- matically, our proposed unlearning loss is:
# Iterative optimization
2: repeat
3:
for each forgetting sample x E D do
4:
Randomly select an irrelevant class d' tc
Σ [) = Ni xyEDs
([V w)ll
end for
(3)
6: Compute loss L according to equation 3
(afe(2z),)) where 2 is number of the forgetting data, c rep- resents the target class of sample x and e' * c denotes an irrelevant class. For each forgetting sample, a new is randomly selected every time the loss is computed.
Update wt+1 < wi 7
8: Update <- min(e, |/aff,,/VVvv
until JeV (x. (r),::))rr > 6 and
|I Vxfoo w,)lFF > 8
V Vxfeer wollFF
Output: Updated model weights wt
Stopping Guideline. To ensure practical deployment of MU-Mis, we design a stopping rule for terminating optimization once the withdrawal is completed. Empirical analysis in Appendix E reveals a consistent trend of metrics during our optimization: as the MU-Mis loss decreases, forgetting accuracy (FA) drops steadily, while the accuracies retained (RA) and test data (TA) initially decline slightly and then grow with the recovery of irrelevant-clas logit sensitivity. Crucially, RA approaches the retrained model when this sensitivity returns its initial level. Therefore, we introduce threshold ratio ó to govern the termination of unlearning. This criterion ensures that optimization halts when irrelevant-cha class sensitivity is sufficiently restored. The overall algorithm is outlined in Algorithm 1.
EXPERIMENTS
5.1 EXPERIMENT SETUPS
Tasks, Datasets and Models. We evaluate unlearning across settings: full-class (CIFAR-100 (Krizhevsky et al., 2009), PinsFaceRecoonittion (Burak, 2020), and Tiny ImageNet (Le & Yang. 2015)), sub-class (CIFAR-20 (Krizhevsky et al., 2009)), and random-subset (CIFAR-IO (Krizhevsky et al., 2009) and SVHN (Netzer et al., 201 I)). ResNet- 18 (He et al., 2016) is adopted the default backbone, and we additionally evaluate under ViT (Dosovitskiy et al., 2021) to highlight the efficiency of remaining-daar methods by executing multiple full-class and sub-class unlearning requests iteratively.
methods. Beyond unlearning utility, we assess the resilience of unlearning
Evaluation Metrics. MU methods should be assessed from three aspects (Xu et al., 2024): utility, privacy, and efficiency. For utility, we compute forgetting data accuracy (FA), remaining data accuracy (RA), and test data accuracy (TA) of the unlearned model. The average gap (Avg. Gap) between the retrained model and the unlearned model across above 3 accuracy-related metrics computed illustrate the utility disparity. We compute the train (FGTA) and valid (FGVA) accuracy on the forgotten classes in sequential unlearning. Regarding the privacy guarantee, we use 2 complementary membership inference attack (MIA) methods, MIA--ntropy (Chundawat et al., 2023) and MIA-SCRUB (Kurmanji et al., 2023) to probe the remaining information of the forgetting data. For efficiency, we provide the run time efficiency (RTE) in seconds to indicate timeliness.
Bad Teacher(BT) (Chundawat et al., 2023), (in)- (Warnecke et al., 2023), SCRUB (Kurmanji et al., 2023), SSD (Foster et al., 2024b), DUCK (Cotogni et al., 2023), SalUn (Fan et al., 2024b), MUNBa (Wu & Harandi, 2025) and LoTus (Spartalis et al., 2025), as well as 4 remaining-das data-faa-e methods: RL (Golatkar et al., N020).NG (Thudi et al., 2022), JiT (Foster et al., 2024a) SCAR (Bonato et al., 2024). Notably, unlike SCAR, our method requires no auxiliary OOD data. Further details on sequential unlearning settings, metrics, and baselines are provided in Appendix F.I.
Baselines. We compare against 8 remaining-data methods:
7

---

Published as a conference paper at ICLR 2026
5.2 UNLEARNING UTILITY
Table 1: Performance overview for full class unlearning task evaluated on CIFAR-100 and Tiny ImageNet using ResNet-18. This table includes performances of our proposed MU-Mis, 6 remaining- data-dependent and remaining-data result format is given by a=b with mean and standard deviation b over 5 independent trials. The metric average gap (Avg. Gap) is calculated by the average of the performance gaps measured in accuracy- metrics, including FA, RA and TA. RTE is reported in seconds. Values in terms of accuracy-related metrics deviating by more than 5% from the retrain model are highlighted in red.
methods, which are delineated by a horizontal line. The
Method
CIFAR-100
Tiny ImageNet
RA
FA
Asg. Gapl.
MIA
RTE
RA
FA
Aug. Gapl.
MIA
RTE
Pretrain
76.41
79.69
75.47
26.84
95.80
10880
65.85
62,00
65.50
21.03
93.59
13600
Retrain
76.52
75. 75.76.a
0.00
2.87
7432
63.30umm
0.00gass
64.90
0.00
4.80lamn
10367
BT
76.87
0.00
76.02.um
0.14
0.00gue
32
64.90gun
0.00:1m0
61.53amm
0.28
0.00
240
FT
76.07
0.2%uuu
75. M.uua
0.11
0.25wore
250
64. Huuas
0.00,200
63.874232
0.74
4,4fluss
262
SCRIT
76 11.0
0.00.am
76 02-000
0.18
5.57
124
65.06uani
0.00-2.0
64 694404
0.17
14. 14,60a
860
SSD
76.27ume
75. 75.49.uai 0.00,am
0.17
0:00gue
63.58uue
0.00,2a)
65. 68.19
0.17
0.00,um
DUCK
75.82an1
0,21lave
75. R
0.51
0:00aare
64.97
0.00.,30
64.61.2
0.25
2.6llaue
76.63,00
1.20uue
85
0.00auue
Laio
0.00,2a0
88
4.40uue
MUNBa
74,09
0.00aam
1,60
9,30aa
64.22.a11
0.00.220
0.72
7.
LaTus
76.45yue
5:00gum
75 75.87
1.72
8:00yue
140
68.02
0.00ga0
64:6
0.20
0.00guu
182
NG
69, 69.76mm
0.00,a00
09.23.am
4.43
0:00uee
59.02 02.u
59 59.26,2a 0.00
3.79
1.80une
RL.
65.95an12
4:22aa.4
66 52.11
8.65
8:00arr
12
53. 53,41.00
0.002200
53 04,221
7.94
10
SCAR
71.33
70 70.68-114
5.29
13.25mm
367
59, 98
0.00:::0
59 62
3.55
0.67a111
1052
JII
65 14.14
3.00,a36
64.87aa1
8.32
4,44w00
15
53. 53.82uue
0.00,200
53 16
7.26
5.29
MUI-Mis
76.42amm
0.D0Laa0
75 75.64.097
0,07
0.00
30
64. 95ann
0.00.:00
64 64.85a
0.15
0.20uane
83
Table 2: Performance overview for sub-class unlearning task evaluated on 'Rocket' and 'Sea" (where the retrain model exhibits different degrees of generalization ability on the unlearned sub-class) of CIFAR-20 using ResNet-16 18. The content format follows Table 1.
Method
Rocket
Sea
RA
FA
TA
Avg. Gapl
MIA
RTF
RA
FA
TA
Asg. Gap!
MIA
RTF
Pretrain
85:26
50.75
65,21
26.53
92.89
6910
$5.09
97.66
85.21
5.94
91.81
6910
Retrai
$4.85woo
2.69
84.07
0,00
12.05war
4298
84.604;
80193,
$4.61-119
0.00
51.61.3
4298
BT
85, 24400
2.80waxs
84. 36we
0.26
0.00uamr
27
B1 00-00-33
K2 63-30
1.38
15.00-am
47
FT
82.70aus
4.20,138
81. 11
1.92
1.40,104
128
29
82
88:00.141
$2.43.100
3.83
58.06.19
SCRUB
84.75
5.804150
83 13.
1.15
13,28aud
113
84
96.66.
88.17..35
2.58
57.07.171
SSD
64.23
2.80ums
38.00
3:00-1
50.09
81 a3
32. 27.8
58
26.45_330
77,11
$4.97,129
SalUn
$4.82an0s
2.99
84.00ces
0.13
0.00
1042
N2,85
$1.00_.30
13,40.230
MUNBa
51.43
7:00pum
30130
3.67
7.20
362
8016440 so
60666,11
3.65
60.00...3)
564
LoTus
35.93
39.00gous
36.044330
44.42
18. 18.60
105
23. 15
81:00
73.39
7.59
61.20
16
NG
62 Mhusss
5:67a0
62.48.559
15:52
72. 22.70aa 30
4
M0.95 09
25.00.a330
80,84,a0
4.45
RL
60,8 60.8941ss
6.52amm
60.50 50
17.11
3 3,70as:1
5
80.48+4 et
77:00
80 34.0
4.11
48 70-20111
3
SCAR
76.6%
43.812444
26.26
19.09
28 28.09a1a
442
76.30 76.30
77.40.271
76.12
6.77
51.84..
434
RT
59. 95.15
4.00wore
58 88.00ass
17.49
20 29.03w330
4
51
A8
7:3021.0
51 $1.04,ami
46.81
52. $2.20la31
4
MU-MS
84,28a01
2.91.u
33
0.49
21
34.35
81.00.2
84.33.uu
0.20
1.25
10
MU-Mis outperforms existing remaining-data-ffee methods significantly and remains highly competitive with SoTA to MU performances on full-class and sub-class unlearning respectively. More experiment results are referred to Appendix G.1. In terms of unlearnig utility, MU-Mis achieves the smallest Avg. sub-class-Sed and sub-class-Lang Lamp
methods. Table I and Table 2 correspond
Gap in class-CIFAR- unlearning, outperforming all the baseline methods. From the highlighted values in red in the tables, we could see that existing remaining-dane -aatnng--a---reeeeeee methods suffer from poor utility preservation. From Table A8, could see that there is clear gap between MU-Mis and RUM in when removing mixture of different memorization level samples. But surprisingly, MU-Mis indicates a lowest KL divergence to the retrained model in the forgetting data, indicating more principled removal than SalUn and RUM. Overall, MU-Mis surpasses strong remaining-data-deeen data-dependent methods in full-class and sub-class unlearning, falls short of the RUM in the particularly challenging aandom-subset setting. But importantly, MU-Mis outperforms all existing remaining-data. margin across all scenarios. In terms of privacy, MIA-Entropy indicates the residual membership of the forgetting data and MIA-SCRUB indicates non membership of the forgetting data in the unlearned remain consistently low and MIA-SCRUB remain consistently
methods by a substantial
model. We can see that MIA-Entronn to the retrained model in Table A cross 3 tasks, collectively demonstrating a successful privacy protection of MU-Mis. In addition to resolving the issue of constrained access to the remaining data, our remaining-dan data- free method also offers a notable advantage in MU efficiency. In unlearning full class Tiny ImageNet, MU-Mis is up to 30x faster than SalUn, with only 0.09 higher Avg. Gap.
8

---

Published as a conference paper at ICLR 2026
Efficiency advantage is pronounced on larger scale models. Table shows the performance when unlearning full class of Tiny ImageNet under ViT (Dosovitskiy et al., 2021). MU-Mis outperforms other remaining- data-free methods significantly and performs comparably with the most competitive method SalUn in terms of model utility and privacy. The efficiency advantage of MU-Mis becomes
Table Performance overview for full class un- learning task evaluated Tiny ImageNet using VIT. RTE is reported in minute.
Aug. Eap
84.21
30.44
55.40
Retrin
86.35
0.00
85.92.0.a
0.00
5.50
Sohon
83.94,0
0.00
83.49,04
1.88
0.00
12.0
0ft
07.43amm
0f
82.13
0.00
92.17.005
2,69
0.0
markedly pronounced: the unlearning time is reduced from more than 1 hour to 3 minutes. We CIFAA20-sea unlearning under ViT and show the results Table A10, where MU-Mis exhibits the best Avg. Gap and is 20 faster than SalUn.
also evaluate
5.3
UNLEARNING RESILIENCE: SEQUENTIAL MACHINE UNLEARNING
Orchid
Poppy
Rose
Sunflower
Tullp
100
100
100
100
50
FVA
50
FTA
residue
FVA
residue
FT
BT
Salun
SSD
Mt MU-MHs
BT
Salun
SSD
MU Mi--MiG
BT
Salun
SSD
MU MU-Mis
BT
Salun
SSD
MU Mis
RA
RA
-10
10
breakdown
breakdown
FT
BT
Satun
SSD
MU-Mi5
FT
BT
Salun
SSD
MU Mis
BT
Salun
SSD
MU-MIs
BT
Salun
S50
MU MU-MIs
100
100
100
100
FGTA
FOVA
FGTA
FowW
recovery
recovery
recovery
recovery
FT
BT
Salun
55D
MU MU--is
FT
BT
Salun
SSD
MU-Mis
BT
Salun
550
BT
Salun
55D
MU
MU-
MU
Figure 5: Disparities in accuracy-related metries between the unlearned model and the retrained
model for full class and sub-class sequential unlearning, Left: Iteratively unlearns 5 distinct full classes of CIFAR-100. Right: Iteratively unlearns 5 sub-classes of the same super-class 'Flower'.
Sequential unlearning requires principled unlearning mechanisms. In practice, unlcarning requests may arrive sequentially, requiring multiple executions of the unlearning method. Wang et al. (2024a) point out that sequential unlearning greatly challenges the memorization management ability of unlearning methods due to underlying associations among unlearned classes. The sequentially unlearned model might break down due to disordered forgetting operation, exposing its accumulated effects model knowledge. Therefore, to highlight the importance of principled forgetting, we perform sequential unlearning. We examine the impact of subsequent requests on previous unlearning efforts and present the disparities between the unlearned model and the retrain model each iteration in accuracy-related MU metrics in Figure 5. For detailed experiment settings, refer to Appendix F.I.
Dificiencies in existing MU methods. From Figure we can see that there are 3 kinds of deficiencies in existing SoTA MU methods:
(i) Performance Recovery. The performance on the forgotten classes stages a recovery in BT and Salun unlearned model, indicated by the above zero FGTA and FGVA. This suggests that retargeting model's outputs of the forgetting data does not completely remove associated knowledge, posing a substantial risk since the concealed information might still be exploited by privacy attackers.
(ii) Knowledge Residue. High disparity of FTA and FVA in sub-class task indicates that FT method, which relies "catastrophic forgetting" (Kirkpatrick et al., 2017) to unlearn, fails to unlearn effectively in sub-class task due to the resemblance between the forgetting and remaining data.
(iā) Utility Breakdown. In sub-class task, SSD exhibits marked decline in utility after the last unlearning request, demonstrated by the final RA of 76.33%. In contrast, RA in the retrained model
We failed identify effective byper-parameters for JiT and SCAR for this experiment.

---

Published as a conference paper at ICLR 2026
and MU-Mis unleamedd model are respectively 84.83% and 84.59%. Such plummet implies
potential risk of model utility breakdowns when the magnitude of parameters is continuously scaled.
Resilient performance of MU-Mis to sequential unlearning requests. To facilitate an intuitive assessment in terms of utility and resilience, we compute the utility Avg. Gap and resilience Avg. Gap for each iteration in Figure 6. The utility Avg. Gap is averaged over FTA, FVA, RA and TA, and the resilience Avg. Gap is averaged over FGTA and FGVA. From Figure 6, it is evident that MU- Mis and SSD are significantly better than BT, FT, and Salun, demonstrating a notably small disparity to the retrained model regarding both the utility and resilience Avg. Gap across the full class and sub-class tasks. Importantly, MU-Mis achieves these results without relying on the remaining data, which are required by SSD.
Beaver
400
375.51
Unkr Ang Gan
13:72
10
2.0
3.60
0.0
0.0
1.04
Orchis
Poppy
Rose
Sunflower
Tulp
200
87
20.54
100
Figure 6: Overview of utility Avg. Gap and resilience Avg. Gap during full class (upper) and sub-class (bottom) sequential unlearning.
Minimal KL divergence of MU-Mis from retrain model during sequential unlearn. Beyond model predictions, we further pro- vide the empirical KL divergence (intro- duced in Appendix F.2) between the un- learned model and the retrained model's output distributions during sequential re- quests in Figure 7. It is evident that MU- Mis exhibits the lowest KI. divergence from the retrained model throughout both the full-class and sub-class sequential un- learning processes.
BT FT
S50
-
Salun
Mis
KL Divergence
Divergence
0.6
0.6
0.4
0.4
Apple
Fish
Baby
Bear
Beaver
Orchid
Poppy
Rose
Sunftower
Tulpp
Iterations
Iterations
Figure 7: KL divergence between outputs of un- learned model and retrained model during full class (left) and sub-class (right) sequential unlearning.
Summary. In general, MU-Mis stands out with its comprehensive capabilities terms of unlearning utility, unlearning resilience well as output indistinguishability. while current SoTA MU meth- ods are disclosed to exhibit limitations and deficiencies in certain aspects. Their inappropriate or inadequate unlearning approaches undermine their reliability and applicability in practical scenarios.
5.4 SUPPLEMENTARY EXPERIMENTS AND ANALYSES
We provide the following experiments and analyses for completeness in Appendix: (i) ablation study of MU-Mis in Appendix G.3; (ii) a hyper-parameter sensitivity analysis showing stability of MU-Mis in Appendix G.4; (iii) visualizations of attention map confirming effectiveness of MU-Mis in Appendix G.5; (iv) an empirical analysis attributing the effectiveness of MU-Mis to the orthogonality of input sensitivity gradients among samples in Appendix H; (vi) A comprehensive analysis covering the theoretical link between sensitivity gaps and loss curvature, empirical signatures of sensitivity across memorization and influence levels, and the broader role of unlearning in shaping memorization, generalization, and sample contribution in Appendix I.
6 CONCLUSION
There are 3 main challenges in machine unlearning: the stochasticity of training, incrementality of training, and catastrophe of unlearning (Wang et al., 2024b). We address incrementality by quantifying sample contribution through the lens of input sensitivity. Building on this, our proposed MU-Mis achieves effective and efficient unlearning without compromising model utility, alleviating catastrophic unlearning. Experiments validate the superiority of this principled forgetting mechanism. Overall, MU-Mis is well-grounded, lightweight and remaining-data-fiee competitive alternative to existing unlearning methods. Furthermore, we highlight in Appendix I that there is profound connection between input sensitivity view and machine unlearning, which we believe is an interesting direction to further improve remaining data- data-free unlearning in the most challenging random subset scenario.
offering practical and
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors thank the anonymous reviewers for their insightful comments and constructive sugges- tions, which have helped improve the quality of this manuscript. The research leading to these results has received funding from the National Natural Science Foundation of China (No. 62376155) and the Al for Science Program, Shanghai Municipal Commission of Economy and Informatization (No. DDD5.G7A-RGG7N-BBBBX--2222
BIBLIOGRAPHY
Juhan Bac, Nathan Ng. Alston Lo, Marzych Ghassemi, and Roger B Grosse. If influence functions are the answer, then what is the question? Advances in Neural Information Processing Systems (NeurIPS), 35:17953-17967 2022. 3
Samyadeep Basu, Phil Pope, and Soheil Feizi. Influence functions in deep learning are fragile. In International Conference on Learning Representations (ICLR), 2020. 3
Jacopo Bonato, Marco Cotogni, and Luigi Sabetta. Is retain set all you need in machine unlearn- ing? restoring performance of unlearned models with outt---diitributionn images. In European Conference on Computer Vision (ECCV), pp. 1-19. Springer, 2024. 3,7, 17, 18
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Phooquette-Chod Hengrui Jia, Adelin Travers. Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. 2021 IEFE Symposium on Security and Privacy (SP), pp. 141-159. IEEE, 2021. 3
Burak. Pinterest face recognition dataset. www. www.laggle
pirs-facerecognitioo 2020. 7, 16
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In IEEE Symposium on Security and Privacy, pp. 463-480. IEEE, 2015.
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. Advances Neural Information Processing Systems (NeurIPS), 35:13263- 13276, 2022. 16
Yuanyuan Chen, Boyang Li, Han Yu, Pengcheng Wu, and Chunyan Miao. Hydra: Hypergradient data relevance analysis for interpreting deep neural networks. In AAAI Conference on Artificial Intelligence (AAAI), volume 35. pp. 7081-7089, 2021. 3
Andreas Christmann and Ingo Steinwart. Support Vector Machines. Springer, 2008
Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using incompetent teacher. In AAAI Conference on Artificial Intelligence (AAAI), pp. 7210-7217, 2023. 2, 3, 17, 26
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine Learning (MLJ, 20:273-297, 1995.
Marco Cotogni, Jacopo Bonato, Luigi Sabetta, Francesco Pelosin, and Alessandro Nicolosi. Duck: Distance-based unlearning via centroid kinematics. arXiv preprint arXiv:2312.02052, 2023, 7, 17
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations (ICLR), 2021. 7,9
Harris Drucker and Yann Le Cun. Improving generalization performance using double backpropaga- tion. IEEE Transactions on Neural Networks, 3(6)99119977, 1992. 26
Chongyu Fan, Jiancheng Liu, Alfred Hero, and Sijia Liu. Challenging forgets: Unveiling the worst- case forget sets in machine unlearning. In European Conference on Computer Vision, pp. 278-297. Springer, 2024a. 28
11

---

Published as a conference paper at ICLR 2026
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong. Dennis Wei, and Sijia Liu. Salun: Em- powering machine unlearning via gradient- based weight saliency in both image classification and generation. In International Conference on Learning Representations (ICLR), 2024b. 2, 3, 7, 17, 26
Vitaly Feldman. Does learning require memorization? a short tale about long tail. In the Annual ACM SIGACT Symposium on Theory of Computing, pp. 954-959, 2020. 1, 5, 27
Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems (NeurIPS), 33: 2881-2891, 2020.
5,77
Jack Foster, Kyle Fogarty, Stefan Schoepf, Cengiz Öztireli, and Alexandra Brintrup. Zero-shot machine unlearning at scale via lipschitz regularization. arXiv preprint arXiv:2402.11011, 2024a. 3, 7, 17, 18, 26
Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In AAAI Conference on Artificial Intelligence (AAAI), pp. 12043-12051, 2024b. 7, 17
Isha Garg, Deepak Ravikumar, and Kaushik Roy. Memorization through the lens of curvature of loss function around samples. In International Conference on Machine Learning, (ICML), 2024. 27
Ryan Giordano, William Stephenson, Runjing Liu, Michael Jordan, and Tamara Broderick. A swiss army infinitesimal jackknife. In International Conference on Artificial Intelligence and Statistics, pp. 1139-1147. PMLR, 2019. 3
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9304 2312, 2020. 2, 7, 17, 18
Will Grathwohl, Kuan-Chieh Wang, Joern- Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference on Learning Representations (ICLR). 6
Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In AAAI Conference on Artificial Intelligence (AAAI), pp. 11516-11524, 2021. 2, 3, 26
Chuan Guo, Tom Goldstein, Awni Y. Hannun, and Laurens van der Maaten. Certified data removal from machine learning models. In International Conference on Machine Learning (ICML), pp. 3832-3842, 2020. 3
Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: survey. Machine Learning (ML), 13(6):2351-24033 2024. 3
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. 7,
Zhengbao He, Tao Li, Xinwen Cheng, Zhehao Huang, and Xiaolin Huang. Towards natural machine unlearning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)). 2025. 2
Tuan Hoang, Santu Rana, Sunil Gupta, and Svetha Venkatesh. Learn to unlearn for deep neural networks: Minimizing unlearning interference with gradient projection. In IEEF/CVF Winter Conference on Applications of Computer Vision, pp. 4819-4828, 2024. 3
Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with jithbaco regularization. arXiv preprint
2019. 24
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho. Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526, 2017.
12

---

Published as a conference paper at ICLR 2026
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
7. 16
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning, Advances in Neural Information Processing Systems (NeurIPS), 36, 2023, 2, 3, 7, 17, 23, 26
Ya and Xuan Yang. Tiny imagenet visual recognition challenge. CS 23IN, 7(7):3, 2015. 7, 16
Shen Lin, Xiaoyu Zhang, Chenyang Chen, Xiaofeng Chen, and Willy Susilo. Erm-ktp: Knowledge- level machine unlearning via knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20147-201555 2023. 3, 26
Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, PRANAY SHARMA, Sijia Liu, et al. Model sparsity can simplify machine unlearning, Advances in Neural Information Processing
Systems (NeurIPS), 36, 2024. 3, 17, 26
Ronak Mehta, Sourav Pal, Vikas Singh, and Sathya N Ravi. Deep unlearning via randomized conditionally independent hessians. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10422-10431, 2022. 3
Xianjia Meng, Yong Yang, Ximeng Liu, and Nan Jiang. Active forgetting via influence estimation for neural networks. International Journal of Intelligent Systems, 37(1 (7(10))9080-1107 2022. 3
Fan Mo, Anastasia Borovykh, Mohammad Malekzadeh, Soteris Demetriou, Deniz Gündüz, and Hamed Haddadi. Quantifying and localizing usable information leakage from neural network gradients. arXiv preprint arXiv:2 105. 13929, 2021. 27
Seth Neel, Aaron Roth, and Saced Sharifi-Man Mariiii-Maaaajrrdi. Descent-to-delctes Gradient-based methods for machine unlearning. In Algorithmic Learning Theory, pp. 931-9622 PMLR, 2021. 3, 26
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al.
Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 7. Granada, Spain, 2011. 7, 16
Vardan Papyan. Traces of class/cross-class structure pervade deep learning spectra. Journal of Machine Learning (JMLR), 210))10197-100260, 2020. 26
Vardan Papyan, XY Han, and David L Donoho. Prevalence neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences (PNAS), 117 (40):24652-24666 2020. 26
Alexandra Peste, Dan Alistarh. and Christoph H Lampert. SSSE: Efficiently erasing samples from trained machine learning models. In NeurlPS 2021 Workshop Privacy in Machine Learning, 2021.
Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems (NeurIPS), 13:10000-19930, 2020. 3
Deepak Ravikumar, Efstathia Soufleri, Abolfazl Hashemi, and Kaushik Roy. Unveiling privacy, memorization, and input curvature links. In International Conference on Machine Learning, (ICML), 2024. 27
General Data Protection Regulation. General data protection regulation (GDPR). Intersoft Consulting, Accessed in October, 24(1), 2018.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based lo- calization. In IEEE International Conference on Computer Vision (ICCV), pp. 618-626, 2017. 25
Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unlcarning: A survey and taxonomy. arXiv preprint 005660030, 1(2),
2023. 3
13

---

Published as a conference paper at ICLR 2026
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pP- 3-18. IEEE, 2017.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad:
removing noise by adding noise. arXiv preprint (rXiv:7706003255 2017. 25
Christoforos N Spartalis, Theodoros Semertzidis, Efstratios Gavves, and Petros Daras. Lotus: Large- scale machine unlearning with taste of uncertainty. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10046-10055, 2025. 7, 17
Suraj Srinivas and François Fleuret. Rethinking the role of gradient-basee attribution methods for model interpretability. In 9th International Conference on Learning Representations (ICLR), 2021.
Vinith Suriyakumar and Ashia C Wilson. Algorithms that approximate data removal: New results and limitations. Advances in Neural Information Processing Systems (NeurIPS), pp. 18892-18903, 2022. 3
Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Systems, 2023. 3
Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Under- standing factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pp. 303-319. IEEE, 2022. 2, 7, 17, 18, 26
Hanling Tian, Yuhang Liu, Mingzhen He, Zhengbao He, Zhehao Huang, Ruikai Yang, and Xiaolin Huang. Simulating training dynamics to reconstruct training data from deep neural networks. In The Thirteenth International Conference on Learning Representations (ICLR). 2025.
Cheng-Long Wang, Qi Li, Zihang Xiang, and Di Wang. Has approximate machine unlearning been evaluated properly? from auditing to side effects. arXiv preprint arXiv:2403.1288 2024a. 9, 16
Weiqi Wang, Zhiyi Tian, and Shui Yu. Machine unlearning: A comprehensive survey. arXiv preprint arXiv:2405.07406, 2024b. 2, 10
Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. In 30th Annual Network and Distributed System Security Symposium, NDSS, 2023. 7, 17
Ga Wu, Masoud Hashemi, and Christopher Srinivasa. Puma: Performance unchanged model augmentation for training data removal. In AAAI Conference on Artificial Intelligence (AAAI), volume 36, pp.
2022. 3
Jing Wu and Mehrtash Harandi. Munba: Machine unlearning via nash bargaining. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4754-4765, 2025. 7, 17
Yinjun Wu, Edgar Dobriban, and Susan Davidson. Deltagrad: Rapid retraining of machine learning models. In International Conference Machine Learning (ICML), pp- 10355-10366 PMLR, 2020. 3, 26
Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia. Machine unlearning: Solutions and challenges. IEFE Transactions on Emerging Topics in Computational Intelligence, 8(3):2150-2168, 2024. 3, 16
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint 0tt:::2000.026110. 2020. 27
Kairan Zhao and Peter Triantafillou. Scalability of memorization-basee machine unlearning. arXiv preprint arXiv:2410. 16516, 2024. 27
Kairan Zhao, Meghdad Kurmanji, George-Oetar Octavian Bärbulescu, Eleni Triantafillou, and Peter Tri- antafillou. What makes unlearning hard and what to do about it. Advances in Neural Information Processing Systems, 37:12293- 2024. 17, 21, 22, 27,
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019. 1, 27
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A ETHICS STATEMENT
This work studies machine unlearning (MU), motivated by the "right to be forgotten." with the goal of enhancing user privacy and data protection. All experiments are conducted on publicly available datasets and standard benchmark models; no sensitive or personally identifiable information is used. While unlearning techniques could in principle be misused to manipulate model behavior, our focus strengthening trust and accountability in machine learning systems. We believe this work contributes positively to the development of privacy-preserving and ethically responsible Al.
B REPRODUCIBILITY STATEMENT
We include anonymized supplementary materials containing the complete algorithm implementations for executing all experiments. We provide detailed experimental settings, hyperparameters, datasets, and evaluation metrics in our Appendix to ensure reproducibility.
C THE USE OF LLMS
Large language models (LLMs) were employed solely as auxiliary writing tools. Their usage was strictly limited surface-level assistance, including grammar correction, stylistic polishing, clarity improvement, and formatting consistency. LLMs were not involved in formulating research ideas, designing methods, conducting analyses, interpreting results, or drawing conclusions. At no stage were LLMs used to generate original content, experimental designs, theoretical claims. All text segments refined with LLM assistance were subsequently reviewed, validated, and, where necessary, rewritten by the authors to ensure technical accuracy and precision of expression. The authors bear full responsibility for the final presentation and content of this paper. This disclosure is made in accordance with conference guidelines L.M usage ensure transparency and research integrity.
D A ToY EXAMPLE COMPLEMENTING SAMPLE CONTRIBUTION DERIVATION
We use a simple MLP model illustrate the insight of (, 7) << S,(,, with & + 5. Assume the nth layer output of model is x' ((0'x'-1), where 0' refers to 1th layer parameter and á refers to activation function. Then,
0
Ofv
1-17
(A1)
a0
-01
f
0x--12
gk
0(1z-11) Ox 00
0.
3 6100--1//
20"
(A2)
a 00 x/
dr
Thereby, the inner-dot (t)g) x - (00--12122 ReLL activation is used, where ('(x) if r > ( else ø (z) = 0, (2( 7) gf ()) will be quite small. The conclusion here is that the residual term is relatively smaller than the contribution term. Therefore, the contribution of a training sample to the training process would be approximately reflected in the pre-trained model's output sensitivity to the sample.
Additionally, we investigate sensitivity signatures across different samples (i.e., different memoriza- tion levels and influence scores) in Appendix 1.2. We find that highly memorized samples exhibit smaller sensitivity gap than low and middle memorized sample, and more influencial samples exhibit higher sensitivity gap. This empirical evidence further confirms that our proposed sensitivity gap successfully reflect sample contribution and the residual term is not that crucial some extent.
15

---

Published as a conference paper at ICLR 2026
E STOPPING GUIDANCE
The optimization should cease once the withdrawal is completed, requiring a stopping guideline for practical use. We monitor both the optimization objective and unlearning metries during mini- mizing the MU-Mis loss (3) in Figure AI, using the example of unlearning with ResNet- 18 on Cullclas-CCIAAL CIFARI00- purple dashed line represents the accuracy of the retrained model. A consistent trend on accuracy change during unlearning is observed across different learning rates: as the optimization of MU-Mis loss progresses, the accuracy of the forgetting data (FA) gradually decreases, the accuracy of the remaining (RA) and test data (TA) first decrease slightly and then grow up with the recovery of fe up]lpp.
In Figure AI, different colors represent different learning rates and the
Notably, when f/ W)lFF re- covers close to the level of its ini- tial value VV (z, wp)lle. RA ap- proaches the retrained model across different learning rates. There exists a clear relationship between the unlearn- ing progress and model performance, allowing for effective unlearning by stopping the optimization timely. To this end, we introduce a stopping threshold ratio 8 to regulate the time of stopping. We record the minimal value of irrelevant class logit sensi- tivity as e and terminate unlearning process when || Ve ()),, and (D,.vll > 6. (D
2+00
Tw-C
FA
RA
TA
60
70
Epoch ||V.reli
Epoch
Epoch
|VW.elli
10000
1000
Epoch
Epoch
Epoch
Figure A1: Accuracy and optimization objective during Milcclas-CIFAAR 100-Rke0- unlearning with different learn- ing rates on ResNet-18. FA decreases gradually, RA and TA first drop slightly and then rise with the recovery of fe w) w)lle. The endpoint of each curve corresponds to the time when a/V || exceeds 90% of its initial value.
F EXPERIMENT DETAILS
F.1 EXPERIMENT SETTING
Tasks, Datasets and Models. We in-
vestigate 3 kinds of unlearning tasks in supervised image classification scenarios, including forgetting a full class, a sub-class under a super-class, and random subset. We evaluate full class unlearning on 100 (Krizhevsky et al., 2009), PinsFaceRecognition (Burak, 2020), and Tiny ImageNet (Le & Yang, 2015), sub-class unlearning on three sub-classes of CIFAR-20 (Krizhevsky et al., 2009), random subset unlearning CIFAR-10 (Krizhevsky et al., 2009) and SVHN (Netzer et al., 201 I). We perform sequential unlearning by iteratively unlearning a full and a sub-class to evaluate the algorithm's 's robustness to privacy onion effect (Carlini al., 2022). For iterative full class MU, iteratively unlearn 5 distinct classes (label 0-4, corresponding to "AApll",, "Fish", "Baby", "Bear" and "Beaver") of CIFAR-100 in line with Wang al. (Wang al., 2024a). For sub-class setting, iteratively unlearn 5 sub-classes ("orchid", "poppy", "rose". "sunflower", "'ulipp") under the same superclass "flower" of CIFAR-20. We use R8 (He et al., 2016) for all the above experiments. To further indicate the significant efficiency advantage of our remaining-datee -ftt---re method, we perform full class unlearning on Tiny-ImageNet and sub-class unlearning on CIFAR-20 with ViT.
Evaluation Metrics. MU methods should be assessed from three aspects: utility, privacy, and efficiency(Xu al., 2024). Beyond that, in practice, where the unlearning requests are made constamtly, the unlcarning resilience should be assessed, i.c. subsequent unlearning should not spoil previous unlearning efforts. For utility, we compute forgetting data accuracy (FA), remaining data accuracy (RA), and test data accuracy (TA) of the unlearned model. FA and RA are computed on the valid set in class-wise unlearning and on the train set in random subset unlearning. We compute the average gap (Avg. Gap) between the retrained model and the unlearned model on accuracy-relates metrics, including FA, RA and TA to illustrate the utility disparity. In terms of resilience, we evaluate on sequential unlearning tasks. We compute the train (FGTA) and valid (FGVA) accuracy the forgotten classes and quantify the unlearning resilience with the average of their disparity to the retrained model (Resilience Avg. Gap). To further examine the indistingushaabiityy between the
16

---

Published as a conference paper at ICLR 2026
retrained and unlearned model, we compute the KL divergence between their output distributions over the entire dataset. The retrained model is an oracle of approximate MU, therefore, above disparity metrics should be small as possible. Regarding the privacy guarantee, we membership inference attack (MIA) (Chundawat et al., 2023) to probe the remaining information of the forgetting data. The MIA success rate indicates how many samples in D are predicted as membership samples of the unlearned model. From a privacy perspective, lower MIA value implies less information leakage in the unlearned model and is preferred (Liu et al., 2024). For efficiency, we provide the run time efficiency (RTE) in seconds to indicate timeliness.
Baselines. We compare our method along with baselines which utilize the remaining data, as well as 4 remaining-daiie -oaeg-datff-eee methods. The 6 baselines include Bad Teacher (BT) (Chundawat et al., 2023).Finetune (FT) (Warnecke et al., 2023), SCalable Remembering and Unlearning unBound Seneecive-Snpaati--aaaaaa
(SCRUB) (Kurmanji et al., 2023), Distance-based Unlearning via Centroid Kinematics (DUCK) (Cotogni al., 2023), Saliency- unlearning (SalUn) (Fan et al., 2024b), Large-Scale Machine Unlearming with a Taste of Uncertainty (LoTUS) (Spartalis et al., 2025), Machine Unlearning via Nash Bargaining (MUNBa) (Wu & Harandi, 2025). We also add strong (RUM) (Zhao et al., 2024) for the most challenging random subset setting. The 4 remaining- daaann---t---reee free methods include Random Labeling (RL) (Golatkar et al., 2020), Negative Gradient (NG) (Thudi et al., 2022), Just in Time unlearning (JiT) (Foster et al., 2024a) and Selective-distillation for Class and Architecture-agnoste unleaRning (SCAR) (Bonato et al., 2024).
(SSD) (Foster et al., 2024b),
momaining-ddta--ae (ata-dependent method Refined- Untemnnninn Meta- Meta-algorithm
The detailed method of each baseline is as the following:
BT (Chundawat et al., 2023): Bad Teacher transfers knowledge from useful and useless teachers for the remaining data and the forgetting data. The code source is https: :/ github.com i f- -oops/op///oooo selective weweerivvessaaapiiii.. dampening.
FT (Warnecke et al., 2023): Finetune optimizes the pre-trained model with the remaining data, unlearning relying on "catastrohpic forgetting". The code source is https:// github. if 1 loops selectivee synaptic- dampening.
SCRUB (Kurmanji al., 2023): SCRUB aims to push outputs of the student model (the unlearned model) away from the teacher model (the pre-trained model) to distill knowledge. This is achieved by first performing several max-steps (distill the knowledge) and then perform several min-steps (regain performance on the remaining data with cross-entropy loss). The code source is https: / thub. com/ meggdadk SCCUB.
SSD (Foster al., 2024b): SSD uses the Fisher information matrix to assess parameter importance and suppress parameters that are important to the forgetting data while less im- portant to the remaining data. The code source is https: //gtttb...// com/ i f- loops/ / selective iepertiptesyyytttbeeeo io- dampening:
DUCK (Cotogni al., 2023): DUCK employs metric learning guide the removal of samples matching the nearest incorrect centroid in the embedding space. The code source is coom/ocraM11 (ocraM17/DUCC....
https:/ htt///gttuuu
SalUn (Fan et al., 2024b): SalUn computes weight saliency map to enable the most im- portant weights for the forgetting data. The code source is https: / //g/tttttttttt /github. com/
STTML-Group//nl/.
Saliency.
LoTUS (Spartalis et al., 2025):LoTUS performs large-scale machine unlearning by es- timating and propagating uncertainty to guide parameter update suppression, enabling scalable forgetting without relying on remaining data. The source code is https: //github. cwm/ www.sghoshhh/oooooooooo LLTTS.
MUNBa (Wu & Harandi, 2025): MUNBa formulates machine unlearning as a Nash bar- gaining problem and jointly optimizes forgetting and retention objectives balance util- ity preservation and effective removal. The source code is https: /tt://gttttt http///ggtttttttte//e//etteehh com/ OPTMI.GGroup MMNBa.
RUM (Zhao et al., 2024): RUM analyzes fundamental factors that impact unlearning entanglement between forgotten and retained data, and memorization levels), and proposes meta-algorithm that partitions the forget set into homogeneous subsets and applies pur-stbset unlearning to improve performance. The source code is https: ///g thub. com/ ka Tranzhho/RUM,
difficulty (e.g., embedding
17

---

Published as a conference paper at ICLR 2026
NG (Thudi al., 2022): Negative gradient computes several steps of gradient ascent with the forgetting data. The source code is https: / /ttg//gtttb/bbb com/ jbonat /scar.
RL (Golatkar et al., 2020): Random label relabels the forgetting data with randomly assigned class and fine-tune the model with computed cross-entropy loss. The source code is httpp::/ggitiiiii thub....ttuuu.ut com/ jbonatol/scan
SCAR (Bonato et al., 2024): SCAR utilizes anttof-distribution (OOD) data as a surrogate for the remaining data and distills the knowledge of the original model into the unlearned model to preserve model utility. The source code https: /github......ggg com/ jbonat ol/ scar.
JiT (Foster et al., 2024a): JiT smooths the model output around the forgetting data by minimizing the local Lipschitz constant. The source code is https: /tgpss///ghhu.....// com/ JwE40/Informate wwwaattiioo-thoeiiieeeeeo Theoretic-Unlearm.nn.
F.2 KL DIVERGENCE
The KL divergence between two distributions is:
(xL(pp((r))llpp - f pz (w
(A3)
We calculate empirical KL divergence with the entire dataset (including both the train and valid set). We first collect the predicted class probabilities from both the unlearned and retrained models each sample, then compute the output KL divergence as follows:
C
1 N
Pe(srg Wp)
DKL
ΣΣ
Pe( wx) log
(A4)
Pe( w.)
where N total number of dataset, denotes the total number of classes, w, is the unlearned model parameter and w, is the retrained model parameter, Pe(z;, w) represents the e-th posterior probability of i-th sample model w.
F.3 TRAINING DETAILS
For ResNet-18, training SGD with momentum of 0.9, weight decay of 5 10-4, and batch size of 128 with learning rate initialized at 0.1. The learning rate decays at 60,120, 160 by 0.1 with total of 200 epochs.
For ViT. we initialize with model pre-trained on ImageNet provided by torchvision. Then we randomly initialize the last fully connected layers and train with SGD with a momentum of 0.9, weight decay of 5 10-4, batch size 64, constant learning rate 77 0.1 for 10 epochs. All the experiments are conducted on single RTX 4090.
F.4 PARAMETERS
For MU-Mis, we optimize the pretrained model under model.eval() mode with vanilla SGD without momentum for ResNet-18 and Adam for ViT. For only the forgetting data are used in MU-Mis, we must freeze the batch norm layers avoid spoiling the remaining data. We use batch size of 256 for MU-Mis across all the experiments with ResNet-18 and batch size of 32 with ViT. In random subset unlearning, the target class è' is fixed to facilitate unlearning. We report the learning rate n and stopping threshold 8 used in different settings in Table Al for reproducibility.
For each baseline, We perform grid search to find the best hyper-parameter in each setting. The hyper-pan prper-parammter sweep range for each method is presented in Table A4. The hyper- parameters used for all the methods in sequential full class and sub-class tasks are shown in Table A2 and Table A3. For all the methods, we fix batch size as 256 for ResNet-18 88 and 64 for ViT unless otherwise stated in hyper-pane parameter ranges. For BT, we use constant learning rate. For fine-tune based methods, e.g. FT, as well as SCRUB and SalUn, gamma = 0.99, weight decay = x 10 4 for SCRUB. We fix weight decay = 5 x 10" for DUCK and SCAR.
cosine scheduler. We fix temperature= 1, alpha 0.5.
18

---

Published as a conference paper at ICLR 2026
Reweighted Variant of MU-Mis Loss. In practice, the TC (Target Class) and OC (Other Classes) terms in Eq. (3) can be re-weighted to enhance the optimization process of the MU-Mis loss. The ablation study in Table A12 implies that the TC term promotes the unlearning and the OC term contributes to the recovery of performance on the remaining data. As shown in Figure AI. TC term is much larger than OC and dominants the MU-Mis loss at the beginning of optimization. To facilitate the unlearning and subsequent recovery, could amplify the significance of the TC term when its magnitude bigger than OC term, and conversely prioritize the OC term when the TC term's influence diminishes in comparison. The parameter serves as an indicator of this dynamic re- weighting switch, while quantifies the strength of such adjustment. Mathematically, the reweighted MU-Mis loss has the following formulation:
C(w, D1)
lac. [Vxfe(u,x)] ad (w, xEDJ
Σ
(A5)
N
where
ae = (x 16-11: +))
ae' (1 - - tK,
T T= I(IIVV/ee x)11 > (x,,(xx
Thereby, we have
ag = K. 02 = Lif V 2 (x)
ac = 1, = Vx
(x)
The original MU-Mis loss Eq. (3) corresponds to x = 1. We apply this strategy in sub-class sequential and for iterations
MU task and Tiny-Imagen ImageNet dataset. We set x 80 for iterations 4 ~ 5 in sequential sub-class, and 80 for TineeImageNet.
Table Al: Hyperparameters of MU-Mis hllearning rate y and stopping threshold ratio 8).
Setting
7
fullclas-CIIAR--IO0
7 10-5
1.45
fullkass-iisFaaceeeccgitton
4 10
0.68
fullclass-Tiny-Imagehh
4 10-6
1.1
Jubclas-CCIFAR-20--oos
3 10-3
3.00
2 10-5
0.93
subclas-CIIFAR-20-AA
5 10-3
1.80
fullchas-Tiny-mmaggee-Viii
5 10
1.03
Table A2: Hyperparameters for full class sequential unlearning in Fig.
Methods
Hyperparameters
Retrain
epoch 200, Ir 0.1, milestones [60, 120, 160).
BT
epoch 10, Ir 5, 10-b, temperature scalar {3, 1, 1,5).
FT SSD
epuch 10, 10 for all iterations.
dampening constant X 1, selection weight
(95, 70, 70, 80).
SalUn
epach = 10, Ir 10 -3 threshold = 0.6 for iterations.
MU-Mis
epoch 50, Ir 12, 0,5,, 0.8) x 10
Table A3: Hyperparameters for subclass sequential unlearning in Fig.
Methods
Hyperparameters
Retrain
epoch 200, Ir 0.1, milestones [60, 120, 160).
BT
epach = (5, 10,5, 10, 5). lr == (0.5. 0.5, 1, 1,15) x 10- temperature scalar = (5, 5,35,5.3.1
FT
epuch = 20, Ir = 10 for all iterations.
SSD
dampening constant == 0.1, 1, selection weight a = (71, 100, 90, 87, 85). opoch = Ir Ir= 10-3, threshold = 0.6 for all itcrations.
SalUn
MU-Mis
epoch 30, (,1,,3))) 10 stopping threshold (1.4, 0.95, 10, 10).
19

---

Published as a conference paper at ICLR 2026
Table A4: Hlyper-parameters range overview for different methods in all the experiments.
Methods
Hyperparameters
epoch (1,3,5, 10).
BT
Ir (10 1,- 10-3,
10 1000-,,1111101110
10- 2,00 9,5 10-9 10-")
temperature scalar (1,3,5).
epoch E [5. 10, 15, 20
Ire (00
dampening constant À E 0.9,,
SSD
selection weight a E {1,5, 10, 25, 30, 40, 50, 60, 90, 100).
epoch = 10, (10- ,5 10-2, x 10-9,10 1,
10 10 10-1,10-1,5 x
SCRUB
10-5,10 10-100-... 5)
max step (2,3,5,8).
opoch E (10.20)
SalUn
(10- 1
10-6, 10-5),
threshold (02,000,,0000000,,0000,,,,,
I (5 10-2, 10-3 10-3,10~1),
DUCK
A1, 6 (0.5,1,12,11,,23355))
NG
Ire (100
10,,
RL
10 I (10-1,10-2
dampening constant
Ire (=000-000-))
lipschitz weight [0,1
Ire -0
10-3,
10-9)
batch size {256, 512, 1024).
SCAR
temperature 31) λ1,d2 3,,
(10-5,5 10 10-9,10 10-2.10-13. epochs 82,
10 10
10 2,
LoTUS
15,
(10- 10-2, 10-1).
10 100 9,- 10 10-
10 10
MUNBa
epochs (5, 10, 20,
25,30,0040 40).
20

---

Published as a conference paper at ICLR 2026
G ADDITIONAL EXPERIMENT RESULTS
G.1 UNLEARNING UTILITY
G.1.1 FULL CLASS MU ON PINFFACERECOGNTTION
Results of full class unlearning on PinsFacceeccogntiion dataset are presented in Table A5. MU-Mis exhibits the smallest Avg. Gap the retrained model, alongside low MIA susceptibility. SCRUB and Salun exhibit notably high MIA score, demonstrating high risk of privacy leakage.
Table A5: Performance overview for full class unlearning (including MU-Mis and 6 baselines) evaluated on PinsFaceRecognition with ResNet-18. 18. The content format follows Table 1.
Method
RA
FA
TA
Avg. Gap 4
MIA
RTE
Pretrain
93.49
100
93.59
34.54
100.00
13144
Retrain
93.06.021
0.00sano
92.25a0.32
0.00
0.00.a.00
11400
BT
92.692n01
0.00ga00
91.48
0.26
0.00ga.00
36
FT
93. 23,9
8.78g1.34
92,94 2..9420.0
3.47
0.00...00
146
SCRUB
92.8220.02
0.00gano
91.61a0.0
0.29
19 19.63
112
SSD
93.412000
0.00gauro
92. 172000
0.14
0.00
DUCK
22.17
0.00
91.06a0.11
0.69
0.00.0.00
SalUn
93.28
0.62ga00
02.10s
0.34
54.2221106
154
MunBa
91 01.4421.12
1.63ganz
90.27
1.74
0.00ga.00
522
LoTus
92.87.0ne
0.00,am0
91.75.0.00
0.23
0.00...00
374
MU-Mis
92.98
0.00gano
12.13
0.07
0.00.0.00
24
G.1.2 SUB CLASS MU ON CIFAR-20-LAMP
We evaluate sub-class unlearning on CIFAR-20-L.an as presented in Table A6. The FA of the retrained model is 11,31, indicating certain generalization capability on the unlearned class. MU-Mis exhibits the smallest Avg.Gar to the retrained model.
Table A6: Performance overview for sub-class unlearning (including proposed MU-Mis and baselines) evaluated on lamp of CIFAR-20 using ResNet- 18. The content format follows Table 1.
Method
RA
FA
TA
Avg. Gap
MIA
RTE
Pretrain
85,31
74.22
85,21
21.30
92.82
6910
Retrain
85.122022
11.31
84.40go.0
0.00
7.06ga1
4298
BT
85.52.004
10.00
84. 84.844004
0.72
0.00gano
29
FT
82.47.0.15
14.00,219
81,90ga18
1.97
2.80ga36
128
SCRUB
82.17.n.8
19.00.474
81 11.60.am0
4.48
26.20..4.8
113
SSD
84.562000
15.004a00
83.84200
1.60
0.60
18
DUCK
83.25
31.024274
82.692034
7.75
27.68
68
SalUn
84.44.005
13.70.1.6
83.74.am4
1.24
1.68.a17
1007
MunBa
81.17.0.5
17.0040
80.69a0
4.45
5.00
676
LoTus
27.44,022
12.00go08
27.40g0.0
38.45
33.60_0.18
14
MU-Mis
84.36.n.51
11.70.1.9
83.6640.0
0.63
0.00+amm
10
G.1.3 RANDOM SUBSET UNLEARNING
We evaluate random subset unlearning on CIFAR-IO and SVHN presented in Table A6. We could see that it is much more challenging for remaining-daar data- free methods to preserve model utility in this setting, for forgetting and remaining data are highly entangled this scenario. Across CIFAR- 10 and SVHN, MU-Mis exhibits consistently low MIA value, demonstrating a good privacy preservation.
Comparison with RUM in random subset setting. RUM(Zhao et al., 2024) provides careful analysis of what makes unlearning hard and applies tailored unlearning strategies to each group of homogeneous subsets (i.c., by memorization level), achieving excellent performance in especially challenging random-subset setting. Therefore, we follow the experiment of RUM (i.e., Table in its original paper, where random subset consist of 3000 samples of high, middle, low memorization levels in CIFAR-10.) We compare performances of RUM, SalUn and MU-Mis in the following Table A8. We could see that MU-Mis still outperforms GA, but there is a clear performance gap between MU-Mis and RUM, indicating room for further improvement.
21

---

Published as a conference paper at ICLR 2026
Table A7: Performance overview for random subset (10%) unlearning task evaluated on forgetting 10% CIFAR-10 and SVHN using ResNet-18. The content format follows Table 1.
Methol
CIFAR-10
SVHN
Aug. Cap.
Ang. Cap.
MIA
Pretram
100,00
99.50
$4.63
1.84
92.64
14322
100,00
100000
1665
85.11
16904
Korrain
10000
0/00
84.77
84.77ar11
12H00
100,00c.
95.12
50.M0ba
0.00
81.24ar11
13500
BT
99.64 99.64.ort
90.00us
93.0 93.01ato
1.18
7. 7,
50
95, 23
96.71are
55.01amm
1.42
22 22.32ax00
215
FT
10000
94 14.25
98. 17a829
0.51
08 Ahuse
395
100.0cam
96
96.18
56.37am00
0.56
50 10775
905
SCRUB
100,00
05. 95.79au
03. 03.45am1
0.87
77 36
207
95. 15.40
94.70
54.94a.
1.08
23 23.15
104
SSD
09.99 2uo
59 03.58am
94 94.60am
1.86
92 22.5
18
95.67
98.63a.0
56.59aa11
1.52
84 84.25aa77
55
DUCK
2.57
90 MMa
96. 10-r
1:51
116
SalEin
100 Duom
19
16 97amo
99.77 77-m7
0L47
ManBa
10000
94.57am0
19a0s
0.54
58.20,11
99.73,00
90.4fu
96.34a00
0.65
06:70,a28
583
LoTts
90.75
50.55amm
3.17
16.19
50,05
5.36
18.80sa1
35
96, 66
96.15
90.52am3
5.13
58 38.21a33
95.08
98.5ur
96,56am
1.55
54.55
25
96.
58 18.29
4.27
53 $3.11,
90.53u
73
JT
95 43aro
05.46a
59 6905
3.54
16 o
255
98. 30ai
96 M..
65 14
1.88
62. 62.78uu
335
SCAR
08.63au
98 38.6
92. 92.45ax1
261
48 48.36a
197
96, Mause
96.46u
92.39 92.39ax
2.86
55 55.20k2
158
MU MU-MIs
97.78,00
97 97.45
91. 91.50a0
2800
3305
116
V5, 28.00
95 95.50gm
54.22 58.22a0e
2:36
26 2a.0
116
The complementary roles of RUM, SalUn, and MU-Mis. Nonetheless, is worth noting that RUM, SalUn, and our MU-MIS focus on different aspects: RUM answers how samples (low > middle - high memorization) should be scheduled and treated during unlearning. SalUn identifies which parameters (gradient- based saliency map) should be updated, and MU-Mis figures out what information (sample contribution) should be removed. The three aspects are all important and numerically has their own advantages, e.g.. RUM is perfect in random subset unlearning and MU-Mis work well without using remaining data. Therefore, RUM, SalUn and MU-Mis are not competing in the same design space, and are naturally complementary y and can be combined. And are very happy to see performance of RUM and especially the combination of RUM and MU-Mis. Combining RUM could improve performance of MU-Mis some extent, but it still under-perfone RUM by noticeable margin on model utility.
MU-Mis exhibit lowest KL-divergence on forget set to the retrain model than SalUn and RUM. Furthermore, we examine the KL divergence between MU-Mis and retrained model in Table A8. Surprisingly, find that MU-Mis achieves the smallest KL-divergence the retrain model in forget set across different methods. We attribute this to their fundamentally different unlearning mechanisms: RUM (inherited from fine-tuning and SalUn) relies on random relabeling or catastrophic forgetting. As discussed in our paper, random relabeling might not provide a principled way to align the output distribution of the retrain model on forgetting data, leading to sub-optimal output distribution. Also, compared with RL, MU-Mis does improve existing RDF method in random subset unlearning. The overall KL divergence of RL is 4.3, while that of MU-Mis is 0.65, which is an substantial improvement, indicating that MU-Mis produces output distribution much closer than other RDF methods. Consequently, although MU-Mis still underperforms remaining-data-deee deta-dependent methods such as RUM+SalUn in terms of model utility, we think its lower KL divergence on the forgetting data is a meaningful step that advances unlearning towards a more faithful/reassnable unlearning.
Challenges of remainig-data-free methods in random subset removal. As is investigated in Figure A4 in Appendix H. we could see that intra-class samples assemble highly similar gradients. Moreover, many existing gemaining-data-deeene this setting. Therefore,
approaches are still suffering from utility degradation in important to acknowledge the daunting challenges such a vision of remaining -ishiing-dtt---ee data-free method in random subset setting faces. Although not perfect, but MU-Mis has method in this challenging scenario. Therefore, we remain hopeful
advanced ammiiing-ddata-fe about the vision of perfectly RDF method that preserves model utility under random subset settings and we believe a better location of sample contribution might offer a promising path.
Table A8: Unlearning performance and KL divergence metrics of SalUn, RUM and MU-Mis on CIFAR-10 with ResNet- 18 following Zhao et al. (2024).
Method
FA
XA
TA
Avg. Cap.
ToW
MIA
MIA.GAP
Tel MIA
KL Forget
Kest
KetRetiin
KL_All
Preczin
100000
100.00
$5.10
12.24
0.64
0.03
0.45
0.44
2.7131
0.3040
0.0015
0.1826
Retrin
63.93ams
10000
$4.45a.
0.00
1.00
0,47auss
0.80
1.0cmm
0.0000
0.0000
0.0000
0.0000
SalUS
73.03amm
90.9
51 $1,68
4.17
0.8
0399mm
0.51
8.64amm
0.9941
01300]
0.0545
0.1175
KUM
66 40ss
100005s0r
088
0.9
0.9
0.52
0.dbe
1.482
0.277
0.0957
0.8675
15.40
62:92amm
43
26.62
0.94ma
0.49
0.2
0.34e
50851
4.5557
4
4300
68.00s1s
70.48um
57.37
19.85
0,50.mm
0:24ess
1,239]
0.7509
RUM+MU
60.70aa
77.45eeu
61.18 a
16.35
0,58
0.4uae:
0.56
2.7765
1.0000
0.6421
0.7005
22

---

Published as a conference paper at ICLR 2026
G.1.4 FULL CLASS MU COMPARED WITH JIT WITH VGG16
JiT unlearns by regularizing lipshitz constant around the forgetting data, which might fail DNNs with batch norm layers. Following the architecture used in the original paper, we com- pare with it on CIFAR-100 ro0-kee unlearning Table A9, The results show that JiT exhibits greater efficacy when implemented with VGG- 16 as opposed to ResNet-18 8. However, it still ex- hibits noticeable performance disparity when compared to MU-Mis.
Table A9: Performance comparison between MU- Mis and JiT of full class unlearning on CIFAR-100 using VGG-16.
Merrics
RA
FA
TA
Avg. Clap)
MIA
RTE
Pretrain
64.72
66.75
64.76
23.64
88.20
Remrain
64 64,96,
0.00,ue
63. 54
0.00
1577
JT
60.98.a
3.73
60:45.38:
3.21
8.02
Mis
64.60 o0s
0.00
63 23.90
0.32
3.28pum
G.1.5 SUB-CLASS MU ON CIFAR-20- WITH VIT
We conducted experiments with ViT on Mapclass-CIFAR2200 to further demonstrate our effective- ness across different tasks. MU-Mis exhibits the best RA, FA, and TA, and provides advantages in unlearning time.
Table A10: Performance overview for sub-class unlearning task evaluated on Cifar20-Sea using VIT. RTE is reported in minute.
Methods
RA
FA
TA
Avg. Gap
MIA
RTB (min)
Pretrain
93.68
91.32
93.63
0.93
69.80
Retrain
93.90
88 88.98
93.84us14
0.00
29.00amms
SalUn
94.15
89.29.0os
94. 13,ae
0.27
62. 10
10
MU-Mis
93,70
88 24
43.674aua
0.17
00.67amm
0,5
G.2 MEMBERSHIP INFERENCE ATTACK
We examine our privacy leakage with a comparably good mnleaming--adapted LiRA proposed by SCRUB Kurmanji et al. (2023). We report SCRUB-LiRA examined results across 3 unlearning settings in Table A11. We could see that in MU-Mis unlearned model, SCRUB-LiRA is quite close random guessing, indicating that the forgetting data in MU-Mis unlearned model is similar to non-members.
LIRA-SCRUR and MIA-Entropy complementary. The goal of the adversary in LiRA-based MIA is to distinguish "forgotten samples" from unseen (non-member) samples. Therefore, their criteria of successful unlearning is that in the unlearned model, the adversary could not well distinguish forgotten samples from unseen samples, i.e., MIA collapse to random guessing (50%) for random subset unlearning. Lise- MIA examines how indistinguishable the forgetting data to the non-members, thereby the closer of SCRUB-ME MMA value to 50% in random subset setting is the better. While MIA our paper examines how much forgotten samples are predicted as members, thereby the lower is the better. That is say, LiRA-based MIA examines the extent of non- memmessbessh of forgetting data, while our MIA-Entropy examines the extent of membership, i.e., the residual membership signals of forgetting data. Therefore, these 2 MIAs but complementary, which all support our conclusions.
contradictory competing,
Table A11: SCRUB-LiRA scores across different unlearning settings.
Setting
Model
FA (%)
RA (%)
TA (%)
MIA-Entropy
MIA-SCRUB
Pretrain
86.00
76.37
76.56
95.40
73.40.1..00
Vulliass-CIFARIOO
Retrain
0.00
76.86
76.07
6.60
96,40:1 1.60
MU Mis
0.00
76.37
75.71
0.00
90.40.1.000
Pretrain
97.00
85.10
85.14
64.40=2.40
Subelass-CFFAR23..
Retrain
80.00
84.85
85.00
53,00
47. 47.40-6.40
MU-Mis
80.00
84.26
84.15
0.60
$3.00=4.60
Pretrain
99.96
100.00
94.68
92.64
$6.64_1..0
Random-CIRe CIEARIO-10%
Retrain
49.68=0.30
MU-Mis
97.43
97.42
91.50
83.34
52.66_0.400
23

---

Published as a conference paper at ICLR 2026
G.3 ABLATION STUDY
G.3.1 ABLATION STUDY ON EACH TERM OF MU-MIS
We study the role of each term in our loss through ablation, illustrating with fullclass-CIFAR 100-RReo-- on ResNet-18. We denote the first term
Table A12: Ablation study on each term of our loss in full class (Rocket) unlearning. TC (Target Class) refers to the first term and OC (Other Class) refers to the second term.
fe, 2) in our loss Eq. (3) as TC (Target Class) and the second term Ve (w, x)) |/ as OC (Other Class). In Table A12, we showcase the un- learning performance of decreasing TC, increasing OC, and decreasing TC OC (MU-Mis) respectively. We also investigate another variant of MU-Mis: regressing sensitivity norm of TC(target class) term to OC(other class) term. During unlearning, we observe that as 2/Vlfell decreases, FA decreases gradually. We find that although the gradient of = is detached, its norm exhibits slight decrease well when minimizing. As the sensitivity norm of fa approaches fcr i.e., the loss approaches O, FA gradually stops to decrease and converges at 17.35%. Therefore, this variant could unlearn successfully but failed to preserve model utility by regressing sensitivity magnitude of fo to fc'. As demonstrated by Figure AI, when minimizing [V lel and retain accuracy (RA) will drop slightly well, then RA will recover to its original level as the picking up of V || in our loss. Therefore, the magnitude of =/V [e is essential for preserving model utiliiy on remaining data (which is also supported by the ablation study in our Table A10). Thus, when regressing VV to Va F. the norm of ||V fe is not explicitly preserved, and drops during minimizing. Therefore, this regressing variant is a useful diagnose and it confirms that the joint-term optimization in MU-Mis is necessary.
Methods
Avg. Gap J
MIA
RTB
Protrzin
76.41
79.69
76,47
26.84
95.50
Retrain
76.52
0.00uue
75.76
0.00
2.87,240
TC
65.4
0.00am
64 64.73401
7.38
1:40.a0)
44
OC
70.13
74.62 74.62.156
a0.2
28.85
0.00.2m0
25
70 70.34
17. g7.35
69 40.45amm
9.98
1.60
OC
76.42amu
75.64am0
0.00,a00
30
From the ablation study, we know that minimizing the first term obliviates information of the forgetting data, but greatly hurts performance on the remaining data. Solely increasing OC brings slight accuracy drop on remaining data, but hardly unlearns the forgetting data. While our MU-Mis loss could unlearn effectively meanwhile preserve model utility on the remaining data.
G.3.2 PERFORMANCE UNDER SENSITIVITY REGULARIZATION TECHNIQUE
We employed Jacobian regularization Hoffman et al. (2019) to our pre-training and re-training. Jacobian regularization penalizes model's input-output sensitivity around training sample x, i.c.. 0f())
||(()]]| = || Bir || to encourage a smoother decision boundary for better robustness and general- ization. We evaluated the unlearning of an full class the CIFAR- 10 dataset by training ResNet18 for 30 epochs with Ir 0.1, bs = 256, regularization coefficient =0.01. The unlearning performance is shown in Table A14, where the first 3 rows performances on vanilla model (with data augmenta- tion but without Jacobian regularization). In Jacobian regularized model, it seems to be more difficult to fully forget a class by MU-Mis, but it still preserves model utility well without remaining data.
G.3.3 ROBUSTNESS OF PERFORMANCE TO SENSITIVITY METRIC.
We investigate performances of MU- Mis with different sensitivity norm choices in Table A13. We can see that the Le norm yields similar perfor- mance to the Frobenius norm (origi- nal MU-Mis) and retrain, suggesting that the method robust as long as the norm aggregates all components of the sensitivity vector. In contrast, norms that emphasize only extreme coordinates (e.g., infinity norm) tend to over over-lenalize the worst-case direction, which empirically hurts model utility. We observed that the model tends to break down even under very small learning rate (1c-8) for L norm, which we attribute this to its non non-smooth optimization, which induces abrupt and sparse updates in model outputs.
Table A13: Performances of MU-Mis under different sensi- tivity norm choices.
Methods
RA
FA
TA
Avg. Gap J
MIA
76.41
79.69
26.84
Retrain
76.52g0.2
0.00
75.76
0.00
2.87
1.2
25.53
1.95
74 21.89
1.27
0.000
Inf
50.34
1.95
50. g0.11
17.93
0,710
Frobenius
76.42_m0.r
0.000moe
75.64_n.0
0.07
0.000
24

---

Published as a conference paper at ICLR 2026
Table A14: Unlearning performance w/o Jacobian regularization on CIFAR-
Regularization
Jacob Regularization
Method
FA
RA
TA
Avg. Gup.
MIA
FA
RA
TA
Avg. Gap.
MIA
Pretrain
84,03
32.54
94.98
83.33
83.11
32.58
Retmin
0.00
86.1 1a01)
77.56
0.00
9.56gn05
0.00g003
85,77
77.35
0.00
0.1
1.40g335
84.13
75
0.98,a
5.67
82.37
75
0.21
G.4 HYPER-PARAMETER SENSITIVITY
We show that there is clear relationship between the norm ratio and model performance during unlearning in Figure A1. The stopping threshold ratio controls the magnitude ratio of final irrelevant class sensitivity norm V fer (ar, w)llF to initial one Jer (z, "lp).... In this part, we investigate the sensitivity unlearning performance to 8 in Figure A2, taking fullclass-CIFAR 100-Rocket with ResNet-18 18 green dashed line indicates the performance of the retrained model. Notably, under a specific learning rate, the Avg. Gap exhibits minimal variation with changes in 8. Generally, MU-Mis demonstrates resilience against variations hyper-parameten ó. In summary. the hyper-parameter tuning for MU-Mis is effortless, resilient and well-guided. This advantage in hyper-tuning indirectly enhances the efficiency of unlearning and facilitates straightforward application of MU-Mis, making it valuable for practical applications of MU methods.
example. Different colors represent different 6, ranging from 70% to 100%. The
Aag. 6:0
Figure A2: MU-Mis unlearning performance with different stopping threshold ratio under different learning rates on Ballelas-CIFARRIII 100-oke00- with ResNet-18 18. Performance varies little with the threshold ratio when the learning rate fixed.
G.5 EFFECTIVENESS VISUALIZATION ATTENTION MAP
To further investigate and understand the behavior of our unlearned model, we showcase the attention heatmaps (Selvaraju et al., 2017) of models before and after applying MU-Mis on PinsFaceRecogni tion dataset in Figure A3. For the forgetting data, the original attention concentrates on the faces. After applying MU-Mis, the attention the faces either disappears or significantly weakens, and is shifted towards the background. For the remain- ing data, MU-Mis fully maintains previous atten- tion. Notably, an alternative interpretation of input sensitivity is the measurement of how changes in the image influence model prediction (Smilkov et al., 2017). Our proposed method reduces the target class logit sensitivity to the forgetting data while recovering irrelevant classes", thereby en- abling the unlearned model to disregard the seman- tic information in the forgetting data meanwhile preserve prediction sensitivity and performance on the remaining data.
Origin
Pretrain
Retrain
MU-Mis
Fboee Ioepet
Retbin Eorget 3
Figure A3: Visualization of attention maps for the full class unlearning task PinsFac- eRecognition. MU-Mis distracts attention from forgetting data regions while preserving atten- tion on remaining data.
25

---

Published as a conference paper at ICLR 2026
H UNDERLYING REASONS FOR MODEL UTILITY PRESERVATION
We suggest the favorable preservation on model utility might be attributed to two factors:
Conceptual motivations: The forgetting operation in MU-Mis differs fundamentally from previous methods. While previous methods involved relabelling the forgetting data (Liu et al., 2024; Foster et al., 2024a; Fan et al., 2024b) or knowledge distillation from a useless teacher (Chundawat et al., 2023; Lin et al., 2023; Kurmanji et al., 2023), which unlearn by introducing incorrect information to spoil original knowledge, MU-Mis unlearns by solely withdrawing the contribution of forgetting data. In Figure A1, we show that RA and TA stay at a high level throughout the withdrawal process.
Empirical investigation: Current gradient-based unlearning methods (Liu et al., 2024; Foster et al., 2024a; Fan et al., 2024b; Wu et al., 2020; Graves et al., 2021: Neel et al., 2021; Thudi et al., 2022) all employ cross-entropy loss gradient (v,( to unlearn. However, the intra-class gradients in a well-trained model are quite similar (Papyan, 2020; Papyan et al., 2020). While the gradient of input sensitivity norm w.r.t parameters is double bropapropng (Drucker & Le Cun, 1992), enhancing sample-specificity by first back- propagating to the input samples before reaching the parameters. Our pairwise analysis of cosine similarity among intra-class and inter-class samples, detailed in the following, reveals distinctive orthogonality in input sensitivity gradients, spontaneously reducing the interference between the forgetting and remaining data.
We calculate the pairwise cosine similarity within a class and between classes of the derivatives of four metrics w.r.t. parameters in a well-trained model in Figure A4 to demonstrate an inherent orthogonality input sensitivity view. They are V. V Wwfe. V V and V. Ec & ll (denoted as w & lI in the following for brevity). The first two metrics are commonly used in current unlearning methods, and the last two are utilized in MU-Mis. Derivatives of all four metrics are approximately orthogonal between samples from different classes. However, intra-class similarities differ across four metrics. We can see that both V and we bear resemblance within a class, with cosine similarity centering around 0.3 and reaching up to 0.6. While V V centers around 0.1 with intra-class similarity scarcely exceeding 0.3. Directly taking the derivative of output w.r.f parameters preserves within-class similarity of the output, but such similarity is reduced when output first takes the derivative to the input and then back to parameters.
Interestingly, V Ee V || seems to be pairwise similar. We speculate that there are two reasons why this portion of our loss function does not significantly harm the performance of the remaining data. Firstly, as shown in Figure 3. Ex' || p is significantly smaller than V/V F on well- trained models. Therefore, in the early stages of optimizing the relative magnitudes of input sensitivities, the contribution of V w Ee #c ll ae || to the optimization direction can be neglected. Secondly, Ee' V || for the forgetting data actually corresponds to the target class of the remaining data. The inter-class similarity of V Eetfe V>feelle l|r implies that when sensitivity of other irrelevant classes increases, the input sensitivity of the remaining data of the corresponding classes increases as well, thereby preserving their sample contributions.
Closs
Pieuaacy
10
3.2 Sinilurtyy
0.8
0.2
0.2
0.6
0.5
20
Inter Cane Cans
Cluw
Clum
0.8
-0.2
Figure A4: Inter-class and intra-class cosine similarity of four different metrics w.r.t parameters. From left to right: V.L. Vwre. Vefllv and V V.felll lr.. Directly taking the derivative of output w.r.t parameters bears a resemblance across samples, but such similarity is reduced when output first takes derivative to input and then back to parameters.
26

---

Published as a conference paper at ICLR 2026
1 DISCUSSION
Beyond the advantages of input sensitivity for measuring sample contribution, we raise the attention that the input sensitivity perspective possesses profound connection MU. MU emerges from model's memorization of the training data and seeks to safeguard the privacy of training data. Recent studies by Garg et al. (Garg et al., 2024) and Ravikumar al. (Ravikumar et al., 2024) reveal the intrinsic relationship between memorization, privacy and sample's 's input curvature. Also, Mo et al. (Mo et al., 2021) demonstrates that the input sensitivity of model gradient the underlying cause of information leakage exposed by Model Inversion Attack (MIA) (Zhu et al., 2019; Zhao et al., 2020). Collectively, these works further underscore the inherent advantages of adopting an input sensitivity perspective for machine unlearning.
1.1 CORRELATIONS BETWEEN SENSTIVITY GAP AND LOSS CURVATURE IN FORMULA
The loss curvature for memorization proxy used in Zhao et al. (2024); Zhao & Triantafillou (2024) is Curvature(x)) x tr (z)). For input sensitivity of loss, have
Val(x)= = (p eo) V vxf(x) = (1 Po) V
V
d'fc
(A6)
C = [c - poc - (Vzfe(z) -Vefelo)-Vafeeee vefe(()).
d=1
d=11
By comparing the formula of loss curvature and sensitivity gap used in MU-Mis, we could see that:
(r(V2c(a)) Σ Petr polr(v? (v² (Ja(z) - (1))) v.s. (ell)))-(())))ss////V/CCCC -
(A7)
While both terms reflects sensitivity gap between target class and irrelevant classes, they refers to different order of gradient. Specifically, loss curvature primarily aims at second-order sensitivity. while sensitivity gap in MU-Mis refers to the first-order sensitivity. Mathematically, it seems that there is no straightforward equality or conserved bound that could directly links this two metrics.
1.2 INPUT SENSITIVITY SIGNATURES ACROSS DIFFERENT SAMPLES
Although there is few clue of mathematical relationship for analysis, we further investigate their correlations by examining the sensitivity signatures across different samples empirically, i.e, samples of different memorization/inftune levels.
We partition training samples of CIFAR-10 dataset according to their sample-wise memorization score (provided by Feldman (2020); Feldman & Zhang (2020) through training many models on different held-out subsets to measure each sample's 's self-influence middle, high memorization levels (following Zhao et al. (2024)). We partition training samples of CIFAR- 100 dataset according to their influence score (provided by Feldman (2020); Feldman Zhang (2020) through training many models on different held-out subsets to quantify each sample's cross-influence on test data) into the same levels. The scores are available at https: / thub. theldout www.innneeeeeseeeeee iew.eeeteeeiimiiiioio..iio
its own prediction) into low,
Interstingly, the distributions of sensitivity gap of different sample groups are shown in Table A15. For memorization level, highly memorized samples exhibit smaller sensitivity gap than low and middle memorized sample. For influence score, more influencial samples exhibit higher sensitivity gap. Importantly, we think this empirical findings provide meaningful support for use of sensitivity gap as a proxy for sample contribution.
We evaluate MU-Mis when respectively unlearning samples with low, medium, and high memoriza- tion levels in Table A16. We could see that removing those highly-memorizer samples causes a more substantial utility drop in the remaining data, indicating that the performance of MU-Mis is not uniform across samples of different memorization levels. But maybe this understandable, to some extent, expected, unlearning highly entangled and influential samples is intrinsically difficult for
27

---

MIA Published as a conference paper at ICLR 2026
Table A15: Sensitivity gaps across memorization and influence levels on CIFAR-IO and CIFAR-100.
CIFAR-10 (Memorization Level)
CIFAR-100 (Influence Level)
Level
Mean
Std
10th
50th
90th
Mean
Std
10th
50th
90th
Low
10.7632
4,2386
6.0814
10.0681
16.4851
36.1209
14.7241
19.3663
34.3506
56.0993
Mid
10.5156
5.5244
3,9959
10.0496
17.6092
39.3604
16.7700
21.3345
36.4983
61.0855
High
6.7922
6,0817
-0.7257
6.6459
14.5839
42.5803
17.1963
23.9295
39.5377
65.7531
any unlearning method Fan et al. (2024a): Zhao al. (2024). We acknowledge this is an important limitation and promising direction for future work, where more fine-grained unlearning mechanisms could be developed further improve remaining-data-
unlearning.
Table A16: Performance of MU-Mis when unlearning low, middle, high levels of samples.
Memorization
Method
FA
RA
TA
Avg. Gap
Original
100.00
100.00
85.10
0.45
0.013
Low
Retrain
99.83
100.00
83,93
0.00
0.049
MU-Mis
100.00
99.96
83.32
0.27
0.041
Original
100.00
100.00
85.10
9.02
0.019
Mid
Retrain
74.40
100.00
83.63
0.00
0.539
MU-Mis
93.63
87.09
69.29
15.49
0.194
Original 100.00
100.00
85.10
26.83
0.055
High
Retrain
21.63
100.00
82,99
0.00
0,811
MU-Mis 46.10
66.79
$7.03
27.88
0.607
1.3 ESSENTIAL GOAL OF MU IN TERMS OF MEMORIZATION, GENERALIZATION AND SAMPLE
CONTRIBUTION
There is an essential question underlying machine unlearning (MU), which might also connect core idea of MU-Mis and RUM
(Q1) What is the correlation between sample contribution and memorization?
(Q2) unlearning equivalent to alleviating sample's memorization level?
Terminology and Conceptual Distinctions. To clarify the discussion, we first distinguish three closely related but conceptually different notions:
1. Memorization is defined as the change in its own prediction when sample leaves the training set, i.e., self-influence.
2. Sample influence is the change on prediction of other data (test data), thereby more lies in cross-influence.
3. Sample contribution is the contribution of a training sample to all the model predictions, thereby comprising both the memorization (self-influence) and sample influence (cross- influence).
RQ1. Memorization level is not proportional to sample contribution. On the one hand, high memorization can coincide with large contribution, e.g., long-tail but but genuinely informative examples, the model may need to "memorize" them to support generalization. On the other hand, high memorization does not guarantee substantial contribution: model might over-fit noise, duplicates, or outliers, thereby exhibiting strong memory for those examples even though they contribute little or may harm the remaining data's performance. Conversely, training example may exert substantial influence on the model's s remaining predictions (high contribution) without having been deeply memorized (low memorization).
RQ2. Minimizing sample contribution more essential for unlearning than reducing mem- orization. Generally, in random subset case, to unlearn an instance, we would aim to alleviate model's 's memorization of it to prevent privacy leakage exploited by MIA. However, as discussed above, memorization (self-influenee) and influence ccross-influence consist sample contribution together. Therefore, in a broader sense, de- memorizaaiin not that enough for complete removal, while withdrawing sample contribution is more fundamental.
28

---

Published as a conference paper at ICLR 2026
In light of the effectiveness of MU-Mis in full/sub-class unlearning, where removal is beyond merely reducing memorization, we think MU-Mis is not limited to directly alleviate memorization by optimizing certain curvature-baue basturee--beddddeeee measures. Rather, prefer to view MU-Mis practical and efficient proxy that could implicitly reduce memorization by suppressing a sample's contribution. Additionally, given the less satisfactory results of MU-Mis in the random-subsel unlearning setting, we suspect that direct optimization of loss curvature (i.e., explicitly minimizing loss curvature around speicific sample) might be sufficient prevent privacy leakage and yield better model-utility trade-offs in this scenario.
J LIMITATION
Although MU-Mis could achieve comparable performance to SoTA remaining-daiie lactinngddt-depppaeee dependent meth- ods. there clear room for further improvement the most challenging unlearning scenario, random subset unlearning. We fully acknowledge that our investigation is quite preliminary, but we believe that the input sensitivity might be a valuable and beneficial perspective for developing remaining- date-ddtt--ffre fata-free unlearning, which is collectively demonstrated by MU-Mis and RUM, leaving good starting point for future study.
K BROADER IMPACT
From a practical perspective, meaaiinng-dtt-fiin unlearning is particularly valuable real-world scenarios where access to the retained training data is restricted or infeasible. In many industrial systems, data may be distributed across silos, subject to contractual deletion, encrypted under privacy regulations, too large to reload for repeated maintenance. In such settings, existing unlearning methods that require remaining data for utility recovery impose substantial computational and storage costs, undermining efficiency and scalability. By directly suppressing sample contribution within the pre-trained model, our approach enables effective unlearning without retraining or additional data access, significantly reducing operational overhead and enabling deployment in resource-constrained or cunpliance-sensitive environments.
Beyond privacy benefits, the key impact of our remaining-data-fee design qualitative improve- ment in efficiency. By removing the need to relearn on retained data, our method reduces time overhead compared with existing unlearning approaches, which could make unlearning substantially faster than retraining in practice and enabling responsive deletion in real-worke systems.
29


Reviews:
[{"review_id": "KSuLS8xcNm", "reviewer": "Reviewer_oXDL", "summary": "This paper proposes a remaining-data-free unlearning method that treats a sample’s training \"contribution\" as being reflected by the model’s input sensitivity to that sample. It empirically observes that training enlarges the gap between the target-logit input gradient and irrelevant-logit gradients for seen data, and shows that a retrained-without-forget set model shrinks this gap. The method then unlearns by minimizing this sensitivity-gap on forget samples with a simple stopping rule tied to the recovery of irrelevant-class sensitivity. Experiments across several datasets/architectures claim utility comparable to strong remaining-data-dependent baselines, plus efficiency and resilience under sequential unlearning.", "questions": "1. Do different samples (e.g. highly memorized vs. typical ones) exhibit different sensitivity signatures? Or if the method is uniformly effective at unlearning different samples?\n\n2. Could you provide more details of the MIA (e.g. attack family, calibration, shadow data choices, etc.) you used in the paper? The current MIA details are brief and privacy claims are sensitive to these choices.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes a remaining-data-free unlearning loss that minimizes the gap between target-logit and irrelevant-logit input sensitivities, along with a concrete stopping rule. This is simple and easy to implement, and the method itself uses only the forget set.\n\n2. The method is evaluated on multiple image-classification settings (full-class, sub-class, random-subset; ResNet-18, ViT), and the proposed method \"MU-Mis\" matches or beats SOTA that does use remaining data in several cases (e.g., CIFAR-100) while being efficient, especially on ViT.", "weaknesses": "1. The theoretical analysis in Section 3.2, which aims to formally connect a sample's \"contribution\" to its \"input sensitivity\", is weak. The derivation relies on several convenient \"simplifications\" and \"omissions\". The central claim that the \"Residual Term\" is negligible is not rigorously proven and is instead justified by referencing a simple MLP example in the appendix. The theory feels more like a post-hoc rationalization for a empirical finding rather than a solid derivation that leads to the method.\n\n2. The claim \"first remaining-data-free method to outperform SOTA remaining-data-dependent\" of the paper reads a bit too strong. In several tables the advantages are small or mixed (e.g., CIFAR-100/Tiny-ImageNet where SSD/BT are very close, and the performance in random-subset task is a bit weak). The novelty/significance claim could be scoped more carefully.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JSn1o3mKY8", "reviewer": "Reviewer_wSLf", "summary": "This paper introduces MU-Mis, a novel remaining-data-free machine unlearning (MU) method based on the insight that a sample’s contribution to training is reflected in the model’s input sensitivity to that sample. The authors derive a theoretical connection between sample contribution $\\frac{\\partial A}{\\partial x_i}$ and the model’s input Jacobian $\\frac{\\partial f}{\\partial x}, and empirically validate that learned models exhibit higher input sensitivity to their training data. MU-Mis performs unlearning by minimizing the sensitivity gap between target and non-target logits on the forgetting samples, thus “withdrawing” their contribution without touching the remaining data. Extensive experiments show that MU-Mis achieves utility and privacy on par with or exceeding remaining-data-dependent methods, while being more efficient.", "questions": "**Q1**. How robust is the method to the choice of sensitivity metric (e.g., using $L_1$ vs Frobenius norm of $\\nabla_xf$)?\n\n**Q2**. I wonder if MU-Mis could inadvertently alter representations of semantically similar retained samples? \n\n**Q3** Do you have any results showing larger forget-set sizes? E.g. > 30%?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**S1**. The ability to unlearn without having access to the remaining data (fully or partially) is important and challenging, which this paper seems to be tackling. \n\n**S1**.  The derivation linking training sample contribution to the model’s input sensitivity while not very rigorous, but it makes it very intuitive.\n\n**S2**. The proposed method is computationally light. It requires only gradient computations on forgetting samples and uses simple stopping criteria.", "weaknesses": "**W1**. MU-Mis operates effectively for sample- or class-level forgetting but doesn’t directly extend to feature or attribute unlearning, or probably fine-grained cases where samples partially overlap. In that sense, the model is not domain general. \n\n**W2**.  The method randomly selects a single irrelevant class $c'$ for the loss calculation in each step. An ablation study or justification for this random sampling (versus a deterministic average) can be clarifying.\n\n**W3**. The stopping guideline relies on a threshold ratio $\\delta$, which is a new hyperparameter. Table A1 shows that the optimal $\\delta$ varies significantly across different tasks (ranging from 0.68 to 3.00). This overstates the claim of \"effortless\" hyper-tuning. \n\n**W4**. Why RTE for pretrain and retrain is omitted overall? They can serve as good reference points. \n\n**W5**. The proposed method is built on the input sensitivity of the model. In practice, training deep models usually involves some techniques to reduce such sensitivities. For example, adversarial training, augmentation, or different types of regularizations. It’s not clear how the model would perform in the presence of these practical techniques.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rYKQpApC95", "reviewer": "Reviewer_pcTz", "summary": "The paper addresses the issue of improving unlearning methods using a new per-example signal and developing a method that does not require access to remain data (RD), only to the forget set (FS).\nSpecifically, it proposes MU-Mis, and revolves around a specific per-example signal, namely that of the sensitivity difference between the example’s target-class logit and non-target logits. MU-Mis for each FS example shrinks this gap.\nThe method optimizes a fairly simple objective over only FS, ie no RD access). \nThe paper presents extensive experiments with CIFAR-100, Tiny-ImageNet, and CIFAR-20 with ResNet-18 and ViT models, using a comprehensive set of metrics. The central claim is that MU-Mis performs on par or better than MU methods that access RD, while being faster.", "questions": "Please address the above weaknesses.\nEspecially the ones wrt RUM-Salun and full RUM, and wrt the MIA-related issues.\n\nOverall, the idea is clever and not requiring RD access is highly desirable. \nSo, I remain open to improving my score if, for example, it is shown that the proposed method continues to outperform SOTA RD-based MU methods and if results for better MIA scores show excellent performance.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Simple, nice idea revolving around the notion of per-example sensitivity of FS examples, with a plan to efficiently compute it.\n\n2. No RD access required. \n\n3. Comprehensive evaluation and results/discussion.\n\n4. Competitive performance vs 'older' baselines; often comparable to SalUn, with large speedups in some settings.", "weaknesses": "1. Incorrect/unsupported claims with respect to achieving same or even better performance than MU methods accessing RD.\n\nThis claim is made using rather older methods. MU-Mis is shown to be comparable to Salun (which is typically either the best or 2nd best performer to MU-Mis.\n\nThe authors appear to be unaware of the paper by Zhao et al (NeurIPS 24 - https://arxiv.org/abs/2406.01257) which actually is shown to significantly outperform Salun (on datasets and models also tested in this paper). Specifically, Zhao et al show that their method, coined RUM, can encapsulate Salun (and several other MU methods) and significantly improve their performance. Hence,  RUM-Salun should be expected to also significantly outperform MU-Mis, no?. The authors should address this concern and test whether this expectation is valid and perhaps revisit their claims if so, At any rate, that method appears to be a much stronger baseline than Salun and should be included here.\n\nInterestingly, RUM appears to utiize a signal that is very similar to the sensitivity one used in this paper, namely the per-example memorization of FS examples (which the authors acknowledge to be akin to their sensitivity signal in this paper). So the link between the two approaches is very interesting and should be discussed. \n\nHowever, Zhao et al go further, utilizing different MU methods for examples with different memorization levels. And this approach (let's call it full RUM) appears to significantly outperform even RUM-Salun... So, again, the claims should be revisited in light of these methods. And full RUM should be included as a baseline/competing algorithm.\n\nAnd RUM appears to be able to use known memorization proxies, which are very efficient to compute - see the companion RUM paper (https://arxiv.org/html/2410.16516v2). \n\n2. The MIA method used in the paper may be rather simplistic, and related discussions on MIA results do not really help MU evaluation.\n\nThe paper by Kurmanji et al (cited in the paper) actually put forth a LiRA-like strong MIA for MU. So (i) this should be acknowledged and justified and (ii) this or other similar strong-grade MIA attacks should be used. Or maybe you should use the MIA also used in the RUM papers for consistent comparisons?\n\n3. On a related issue, the claim that \"lower MIA score is better\" seems counter-intuitive.\n\nAs the authors clearly state, performance should be viewed in relation to Retrain. So MIA scores of an MU method should be 'normalized' w.r.t. those of Retrain, no? The notion of MIA-gap (e.g., MIA_Retrain vs MIA_MU-Mis) in the RUM and other papers has been used for this reason. This should better be used here as well (with an appropriate MIA).\n\n4. The paper may/will benefit by incorporating more advanced baselines, in addition to RUM-Salun and full RUM. A highly interesting set of MU methods were proposed for the NeurIPS MU competition - the paper https://arxiv.org/abs/2406.09073 presents a rich set of such methods which notably outperformed well known published algorithms. Other MU methods have emerged more recenlty as well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dACspHFpCp", "reviewer": "Reviewer_6Bdx", "summary": "Overall, this paper focuses on machine unlearning without access to retain set. This paper essentially relies on an observation that, as each sample is used during training, the learned model increases in sensitivity towards it. Specifically, during training of a sample, the difference of gradients between the relevant and irrelevant classes increases. This observation is used to design a loss to suppress the sample contributions, thus achieving unlearning.", "questions": "Please refer to Weaknesses for most questions.\n\nHow would the model be applied in cases without classification tasks? E.g., forgetting a single name or data entry.\n\nThe method aims to minimize the sensitivity difference between relevant and irrelevant classes. Just curious, what would happen if the gradients for the irrelevant classes are fixed, i.e., without backpropagating gradients, and only the gradient for the relevant class is used? I guess, in some sense, it would tell us if the loss is mainly working on the relevant class or the irrelevant classes.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper addresses an important area of machine unlearning which does not use any data from the retain set.\n\nThe paper seems sound in the theoretical areas.\n\nExperimental results in Table 1 and Table 2 look convincing.", "weaknesses": "I think the intuition of the method is not well expressed. Although the math and equations are there, the intuition is not clear, in the sense that, it should be able to be expressed in a few lines at most. However, the paper does not express these simple concepts well.\n\nIn terms of the proposed method, I believe the idea is quite simple. Basically, the sensitivity of each forget sample is computed, with respect to the relevant class and also irrelevant classes. Then, the loss is designed to suppress this difference, thus making the model learn to forget, similar to how a unlearned model would be. However, these concepts have not been explained clearly in the paper.\n\n\nImportantly, the full related work section has been placed in the Supplementary. I do not feel good about this, since I do not think it is fair for most other works that use space to discuss it in the main paper. In some sense, since reviewers are not obligated to read the supplementary, it can be said to some extent that this paper does not have a proper related work section. All in all, since it is a main and important component of the paper, there should be substantial discussion on related work in the main paper. This is a huge issue for me.\n\nI also feel uncomfortable that the implementation details section is fully in the supplementary.\n\nSection 3.2 which explains the input sensitivity is too long in my opinion. Also, the derivations and math look very similar to influence functions (some example papers shown below). In this long section 3.2, I am unable to grasp the novelty and contributions of these theoretical analyses, how they are different from previous insights, and why they are important.\n\nEfficient Machine Unlearning via Influence Approximation. arXiv 2025\n\nReliable Active Learning via Influence Functions. TMLR 2023\n\nInfluence Selection for Active Learning. ICCV 2021\n\n\nThere is a citation error for two references:\n\nTowards Unbounded Machine Unlearning is a NeurIPS 2023 paper, not 2024.\n\nIs retain set all you need in machine unlearning? Restoring performance of unlearned models with out-of-distribution images is an ECCV 2024 paper, not 2025\n\nThe authors can consider reporting results and/or comparing with more recent works such as:\n\nLearning to Unlearn for Robust Machine Unlearning. ECCV 2024\n\nAdversarial Machine Unlearning. ICLR 2025\n\nDecoupled Distillation to Erase: A General Unlearning Method for Any Class-centric Tasks. CVPR 2025\n\nLoTUS: Large-Scale Machine Unlearning with a Taste of Uncertainty. CVPR 2025\n\nMUNBa: Machine Unlearning via Nash Bargaining. ICCV 2025\n\n\n\n\nMinor errors that do not affect the score: Line 105: knowledge -> knowledge", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)