Decoupling the Class Label and the Target Concept in Machine Unlearning
Authors: Jianing Zhu
Bo Han
Jiangchao Yao
Jianliang Xu
Gang Niu
Masashi Sugiyama
Venue: ICLR 2026 Poster (2026)
Keywords: Machine Unlearning
Label Domain Mismatch
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: Machine unlearning as an emerging research topic for data regulations, aims to adjust a trained model to approximate a retrained one that excludes a portion of training data. Previous studies showed that class-wise unlearning is effective in forgetting the knowledge of a training class, either through gradient ascent on the forgetting data or fine-tuning with the remaining data. However, while these methods are useful, they are insufficient as the class label and the target concept are often considered to coincide. In this work, we expand the scope by considering the label domain mismatch and investigate three problems beyond the conventional all matched forgetting, e.g., target mismatch, model mismatch, and data mismatch forgetting. We systematically analyze the new challenges in restrictively forgetting the target concept and also reveal crucial forgetting dynamics in the representation level to realize these tasks. Based on that, we propose a general framework, namely, TARget-aware Forgetting (TARF). It enables the additional tasks to actively forget the target concept while maintaining the rest part, by simultaneously conducting annealed gradient ascent on the forgetting data and selected gradient descent on the hard-to-affect remaining data. Various experiments under our new settings are conducted to demonstrate the effectiveness of our TARF. Our code is publicly available at https://github.com/tmlr-group/TARF.

Published as a conference paper at ICLR 2026
DECOUPLING THE CLASS LABEL AND THE TARGET CONCEPT IN MACHINE UNLEARNING
Jianing Zhu Bo Han't Jiangchao Y
Jianliang Xu'
Gang 2 Masashi Sugiyama
'Hong Kong Baptist University RRKENN Center for Advanced Intelligence Project
OCIC, Shanghai Jiao Tong University "The University of Tokyo
(as jnzhu, xuji, bhanml @ ecompp hkbu. edu. .k Sunarker jtu. edu.on gang. niu. nau ml gma i com sugi @k u-tokyo arcokoo....jpp.... jc.
ABSTRACT
Machine unlearning emerging research topic for data regulations, aims adjust trained model to approximate a retrained one that excludes a portion of training data. Previous studies showed that class-wise unlearning is effective in forgetting the knowledge of a training class, either through gradient ascent on the forgetting data or fine-tuning with the remaining data. However, while these methods are useful, they are insufficient as the class label and the target concept are often considered to coincide. this work, we expand the scope by considering the label domain mismatch and investigate three problems beyond the conventional all matched forgetting, c.g., target mismatch, model mismatch, and data mismatch forgetting. We systematically analyze the new challenges in restrictively forgetting the target concept and also reveal crucial forgetting dynamics in the representation level to realize these tasks. Based that, we propose general framework, namely, TARgel-aware Forgetting (TARF). It enables the additional tasks to actively forget the target concept while maintaining the rest part, by simultaneously conducting annealed gradient ascent on the forgetting data and selected gradient descent on the hard-to-affeer remaining data. Empirically, various experiments under our newly introduced settings are conducted to demonstrate the effectiveness of our TARF. Our code is publicly available https: / thub. com/ tml -rroop//TARRR
INTRODUCTION
In response to data regulations (Hoofnagle et al., 2019), machine unlearning (Cao & Yang, 2015; Shaik et al., 2023; Xu et al., 2023) has emerged to eliminate the influence of training data from trained model (Xu et al., 2023). The intuitive goal is to forget the specific data as if the model had never used during training (Bourtoule et al., 2021). To achieve that, a direct way (Shaik et al., 2023) is to retrain the model from scratch by excluding the data to be unlearned, termed exact unlearning. Considering the intensive computational cost, much attention has been paid to approximate unlearning (Izzo et al., 2021: Chen al., 2023; Fan et al., 2023), which adjusts the trained model for approximating the behaviors of the retrained one. Focusing target granularity as semantic clusters, recent studies (Kurmanji al., 2023; Jia et al., 2023; Chen al., 2023; Fan et al., 2023) showed class-wise unlearning is effective in forgetting the knowledge of a training class, either through reverse optimization (Thudi et al., 2022a: Izzo al., 2021) on the class data fine-tuning on the remaining data (Golatkar et al., 2020) to realize catastrophic forgetting (Kirkpatrick et al., 2017).
Despite the promising achievements, the previously studied scenario (Warnecke et al., 2023; Golatkar et al., 2020; Chen et al., 2023; Jia et al., 2023; Fan et al., 2023) mainly assumed the target concept to coincide with the class label, overlooking that the practical unlearning request (Bommasani al., 2021; Hashimoto et al., 2018; Kovashka et al., 2016) may violate the taxonomy of the pre-training tasks. Raised by the model users, the reported cases to be unlearned can involve different concerns from original tasks, spanning from privacy, fairness, copyright, or the hazardous capabilities (Li et al.,
Work done during an internship at RIKEN Center for Advanced Intelligence Project. Correspondence Bo Han (bhanml @ @wopp.ckbu.edeeee... hww...du.hkk.

---

Published as a conference paper at ICLR 2026
Classes
boy
girl
man
woman
boby
Label Domain
Superclass
people
flowers
Forgetting Data
Model Output
Target Concept
Furgetting Data
Mudel Oulput
Target Concept
Eg:
"Boy"
"girt"
classes (100)
""huy"
"gir""
dessss (1)
'posple"
Exi
Ey:
Ep
a) matched (conv entional): Lp
Lx
Er
b) Target mismatrh: Eo
Lx Ey
For Forgetting Data
Model Output
Targe Concept
Furgetting Data
Model Output
Target Concept
"gidl
superdaas (20)
"sid"
Lp:
"buy"
"girl"
supertiass (20)
"poople"
Ex::
Ey:
c) Model mismatch: Lp Ep Eye
b) Data mismatch: Cy Cy by
Taking the CIEAR 100 (Krizhevsky, 2009) datuset with its classes and superclass (two differentt label domains for modeling different taxonomy from pre-training tasks) as example, instantiate four tasks given the same forgetting data with the class labels of "boy" and "girt"; all matched jorgetting (conventional scenario): unlearn "boy" and "gizl" with the model trained on the classes: b) target mismarch forgetting: unlearn "peaple" with model trained classes: e) model mismateh forgetring: unlearn "bby"" and "girl" with the model trained on the superclass; d) data mismarch forgetting: unleam "people" with the model trained on the superclass. More discussion provided in Appendix D.
Figure I: Illustrations of decoupling the class label and the target concept.
2024), which can not always be the conventional matched scenario where all the identified correspond to one pre-training class. In contrast, those cases may be only semantic subset within a class, for which the model developer needs to unlearn the small set considering reserving model utility on the other parts. In addition, sometimes the user would identify limited cases of the target concept. With a conservative attitude for protecting the reputation of serving (Bommasani et al., 2021; Liu et al., 2024; Li et al., 2024) (e.g., IP conflicts), the developer tends to unlearn a larger semantic cluster.
In this work, we decouple the target concept with the class label, to model the unlearning scenarios for research explorations. To be specific, consider the different label domains of the forgetting data LD. the model output LM, and the target concept Lr in unlearning. We introduce two relations between two label domains, i.e., Li matches L2 (Li L2) and Li the subelass domain of L2 (L1 < Lgy², then modeling scenarios corresponding to the target concept being larger or smaller than the class unit. As the reported forgetting data are included in the target concept, e.g., LD 3 Lr, we have all matched LD = LT = LM; target mismatch LD = LM < Lr; model mismatch Lr < Las; and data mismatch LD Lr- = Las settings (task instances refer to Figure 1).
LD =
Given the aforementioned tasks, we identify new challenges with the mismatched label domains (refer to Figure 2). Unlike the accurate unlearning approximation in the conventional all matched task (Golatkar et al., 2020; Jia et al., 2023; Chen et al., 2023), the representative unlearning meth- ods (Warnecke et al., 2023; Thudi et al., 2022a) exhibit different performance gap with the retrained reference in the other tasks. Specifically, the under-entangled feature representation (when LM < LT) or the under-representative forgetting data (when LD < LT) results in insufficient forgetting, while the entangled feature representation (when Lr 3 LM) prevents the decomposition of target concept with the retaining part. The former requires target identification in the remaining dataset, while the latter requires explicit target separation over the entangled feature representation.
Based on the above analysis, we propose a novel framework, namely, TARget-aware Forgetting (TARF), for unlearning. In general, we consider two parts (refer to Eq. 3), i.e., annealed forgetting and target-aware retaining, which collaboratively enable the target identification and separation for these forgetting tasks. Specifically, the algorithmic framework (refer to Figure 4) incorporates an annealed gradient ascent and target-aware gradient descent in a dynamical manner. First, it actively unlearns the identified forgetting data, and constructs the contrast information to filter out the remaining data which is hard to be affected. Then, simultaneously learning the selected retaining data with gradient descent deconstructs the entangled feature representation. Ultimately, the learning objective can progressively approach standard retraining using the aligned retaining data (refer to Figure 5). We present comprehensive experiments on different setups of benchmarks and also real-world applications to verify the effectivvness. Our main contributions can be summarized as: Conceptually, we introduce new settings that decouple the class label and the target concept, which
L1 x L2: For any label E L1, there exists label y' E La that an instance labeled with y can also be laheled with y/,, but not all instances labeled with y' can be laheled with y.
2

---

Published as a conference paper at ICLR 2026
investigate the label domain mismatch in class-wise unlearning (in Section 3.1); Empirically, systematically reveal the challenges of restrictive unlearning with the mismatched label domains, and demonstrate that the representation gravity in forgetting dynamics is critical for achieving the forgetting target in the new tasks (in Section 3.2); Technically, we propose a general framework, namely, TARF, realize the target identification and separation in unlearning. It consists of annealed forgetting and target-aware retaining which collaboratively approximate retraining on the retaining data (in Section 3.3); Experimentally, we conduct extensive explorations validate the effectiveness of our framework and perform various ablations to characterize algorithm properties (in Section 4),
2 PRELIMINARIES
Problem setup. Following the literature Shaik et al. (2023); Xu et al. (2023), we mainly consider the multi-class classification the original training task for class-wiss unlearning. Let X' C R denote the input space and y = fl..... C} denote the label space, where C' is the number of classes, the training dataset D (1 yt) y;) generally consists of two subsets in machine unlearning, e.g., the forgetting dataset Dr and the retaining dataset DD = DDDr, Dr. Building upon the model so- : X - y trained on with the loss function f, the general goal of this problem is to find an unlearned model Omm which approximates the behaviors of the model 0" that retrained on D, from scratch,
8 = arg min
Σ (a.1)~
s.t. 0" = arg min R(0,0",2,00
Σ {(fol)),)) (a.
D
Dr
0
9
Irsetric
where R indicates a general risk measure for model behavior consistency (Golatkar et al., 2020; Shaik et al., 2023), which can be instantiated by an averaged gap with various evaluation metrics (Jia et al., 2023; Fan et al., 2023) (e.g., unlearning accuracy (UA), retaining accuracy (RA), and others related to privacy) in experiments to pursue the unlearning efficacy and the model utility (Xu et al., 2023). The specific metrics are introduced in Section 4.1, more discussion on related work is in Appendix A.
Dataset partition in mismatched setting, As the target concept is decoupled from the class label, adopt D, to indicate the dataset of the target concept, Dr to indicate the given forgetting dataset, and summarize the notations in Table 1. We can find that the previous assumptions of D;- Dj and D. = D' Dr only hold in all matched setting. In model mismatch forgetting, the former is still held while we notice that there exists affected retaining data in Dar having the same class label with that in Dr; in target mismatch forgetting and data mismatch forgetting, Dj S 'D and the remaining dataset D un = DrDr include both true retaining dataset D. S Dun and the false retaining dataset Drr- = 2, Drr where the data belong to the target concept but included in the remaining dataset. Considering specific task feasibility, we assume that the number of classes in Dan belonging to the target concept is known in target mismatch forgetting, and the retrained model for every task is trained using D. = D D.D More details about unlcarning request construction are provided in Appendix D.4.
Table 1: tion corresponding four major forgetting tasks.
Notation summary and training set data parti-
Notation Explamtiin
Scenatio
Partition
given forgotingg
matched
2
B
D,
D. dataset target concapt
Target mismatch
2y c B
Pa u2 D. D.
Pe, :DV,,, remaining data
Madel mismonch
Dy. tnueffalse retaining dataser
Data mismaaich
Dr 22
D. D2
Different focus from prior methods. Assuming that Dj- D. and D, D(Pr the common approximation unlearning methods either focus on retaining or forgetting objectives. The former, represented by Fine-tuning (FT) (Warnecke et al., 2023), fine-tunes the model 6 D, to induce catastrophic forgetting over Dr. Later advances assign random labels (Golatkar et al., 2020) on D; enforce forgetting or adopt Li Lı-morm (Jia et al., 2023) infuse weight sparsity in approximation. The latter, represented by gradient ascent (GA), reverse gradient updates on D;. And another line of works (Izzo al., 2021) utilizes the influence function (Koh & Liang, 2017) to erase the influence.
TARF: TARget-aware Forgetting
3.1 EXPLORING MISMATCHED TAXONOMY IN UNLEARNING
Given its technical nature of mitigating the data influence from a trained model, unleamiin is given a broader significance in the context of trustworthiness (Bommasani al., 2021), where the requests
3

---

Published as a conference paper at ICLR 2026
dfrered Arui
Forgettin Daft
Fargetting Accurscy
200
Target Paautc
Data Miseatch
Accuracy GaD
Medel Mismatch
Afficted Botaring
False Botairing Data
Model Mismatch F orgetting
Targetilaaa Mismatch Forgetting
Method
Mothod
We conduct various unlearning methods for the lour tasks. In conventionll all-matched forgetting, all the methods can perform similarly to find that model mismalch forgetting and affected retaining data, leaving less accuracy gop between them. In targer or data mismarch forgetting, the class labels cannot fully represent Full results can refer Figure
Retrained. In contrust,
affected by the trained model, coupling the hehaviors on the forgetting
the target concepts, leaving false retaining data (belongs
target concept) not completely
Figure 2: The challenges of restrictive unlearning with the mismatched label domains.
can be varied beyond the withdrawal from data owner (Shah et al., 2023), and may be applied in mitigating bias (Yao et al., 2023) to improve fairness, erasing harmful content (Li et al., 2024) to ensure safety usage, or removing inappropriate content (Gandikota et al., 2023) for social good. Recently, a series of studies (Golatkar et al., 2020; Warnecke et al., 2023; Jía et al., 2023; Fan et al., 2023; Chen et al., 2023) have several proposals on forgetting training class of the models, and demonstrated it can be successfully achieved by partially scrubbing the class data or fine-tuning on the retaining data realize catastrophic forgetting (French, 1999; Goodfellow et al., 2015). However, a general scenario considered in previous works is that the target concept is aligned with the taxonomy of the pre- pre-training tasks, which may not always hold in practical scenarios with the previous meanings (du e to the space, we leave more discussion in Appendix D.5). This naturally motivates the question,
What if the class labels and targel concept do not coincide in unlearning?
In Figure 2, we conduct the unlearning on the four forgetting tasks as instantiated in Figure 1 (full results refer Figure 8). As result, those unlearning methods, e.g., the representative FT, GA, and the recent L, -s-ssae (Jia et al., 2023) and BS (Chen et al., 2023) show different performance gaps compared with the retrained models except in the conventional all matched setting. It can be found that the affected retaining data (which is under the same superclass as the model trained on) are entangled with the forgetting part when Lr < L., as demonstrated by the less accuracy gap between forgetting and affected retaining data than that of Retrained in the left-middle panel of Figure 2; and the false retaining data (which belong to the target concept but are not identified) are under-represented by the given forgetting data when Ln < LT, as evident by the non-zero accuracy on target concept in the right panel of Figure 2.
3.2 SYSTEMATIC EXPLORATION ON FORGETTING DYNAMICS
The mismatch of label domains affects the construction of model representation in unlearning, which requires us to explore further to understand the underlying mechanism of the performance gaps. We delve into the relationship between the representation and forgetting dynamics, for which we first derive the formal analytical results (a full proof can refer to Appendix C) as follows, and then provide empirical verification in Figure 3 with corresponding interpretations different kind of mismatch.
Assumption 3.1 (Representation similarity). Let $1 and 82 be two disjoint yet semantically related subsets of a dataset D trained on a model fo. x1 E 81 and Ø2 E 82 refer to samples drawn from them. Given the representation of input x at intermediate layer be h(a)), the gradient differences at representation level can be controlled by assuming la (.) is Lipschitz smooth with constant Ct, then we have [ (2,) Vl, (#2) S Cellb(z1) h((22)l = Cedy (x1, for local region.
Theorem 3.2 (Gravity effects on forgetting dynamic). Let 6 be the well-trained model parameters for unlearning, and we perform unlearning on $1 via gradient ascent update, i.e., Gt+1 6² V La, (0t)
for epoch t, then we can the following dynamics given ( = (I., (fc+1) - I.,, (0 (0--111
ALSL (0++1 <
(03)+ + (Jor (z1)) (Z1 22) V (09) )| 0 (2)
ah(z) where (Jo:) ) is the largest eigervalue of the Jacobian matrix Jo = 30 Note that when 1 - 0.
the RHS mainly relies on the term measuring representation similarity as (T.s, (0') I. (0')) O.
4

---

Published as a conference paper at ICLR 2026
20.0
20
40
Forgetingg
Loss
12.
Forgelling
Loos
Remaining
20.0
7.5
-0
2.5
0.0
20
Pre-trained
Epoch
Pre-trained
Epoch
We present LSNE visualization the leurned lieatures show the averaged loss value of forgetting data, concepliclass-alie aasss--alggg gigeed data, and the remaining data during GA on the two representations In addition, the cluster-wise instance distance accuracy dynamies can refer to Figure 9, Note that we only show the classes in ISNE as the large number of remaining classes (e.g.. 95 in yellow). also provide the full results of the unlearned representations in the Appendix E.2.
pre-trained model trained by (left) superclass and (right)) classes. also
Figure 3: Forgetting dynamics on entangled/under-entaagg
feature representations of trained model.
Remark 3.1. (Intuitive implication) The Theorem 3.2 connects the unlearning behaviors with the representation-leve relationship in forgetting dynamies, specifically on Eq. (2) where the leading term shows the magnitude of loss change can be proportional to their representation distance. Intuitively, if two portions of data occupy nearby/far-apst apart regions in the latent space, pushing the model to forget the one will imadvertentlyloooely affect the other, reflecting gravity-style co-movement. This idea forms the basis of our later understanding of challenges in mismatch scenarios and our useful cues.
Target data mismatch. In both tasks, we have LD Lr, which means that the forgetting data is a subset of the target concept, i.c., Dr C Dj. As indicated in right of Figure 3, partially relying on the forgetting remaining data not fully represent the target concept due to under-entangled representation, and leaves non-zero accuracy on false retaining data as shown in Figure 2.
Remark 3.2. (Insufficient representation) Given LD Ly that indicates Dr C Di, we can have D as and DA, as 82 corresponds to Theorem 3.2, in which the sample """,,"") ~ 82 and sample (2r, v) ~ 81 exhibit weak gravity effects on the forgetting dynamics due the under-entangled or biased representation with large latent distance d, (x"', x). i.e., the (0s2,,2(06++) - A L 2 82.82 (0') in Eq. indicating loss update gap of two subsets can be also relatively large. that aligns with Figure 3. The forgetting set covers only part of target concept can't 't govern the whole concept forgetting.
Model mismatch forgetting, In this task, we have LD = Cr while - Regarding the model trained by the superclass, it can be found in the left of Figure 3 that the features of forgetting data and affected retaining data are closely entangled, showing that the unlearning of the forgetting data can unavoidably affect the representation of the other part. In contrast, it is also notable in the left-middle of Figure that the accuracy gap between forgetting data and affected retaining data expected to be large in the retrained reference. We provide the following interpretation based on Theorem 3.2.
Remark 3.3. (Decomposition lacking) Given LT < Las that indicates the broader representation region for D₂ = D, UDr within the same class (here Dae C Dr refer the set of affected retaining data as illustrated in left-most of Figure 2), the entangled representation results in small latent distance dx("", ,, .r) for the samples of (xx yy)) is as small as evident in left of Figure 3, requiring bidirectional operation to disentangle it, as the representation is overly entangled that forgetting updates on target concept may spill over onto others.
D₂ and sample (a. y) Dr, so ALm,, (0*+1)- AL,, .a2 (0')
Forgetting dynamics with representation distance. Despite the issues revealed by the observations under label domain mismatch, the forgetting performance varying obviously different representa- tions also provides clues on addressing them. Notably, we can find that GA achieves better forgetting efficacy on the data mismatch forgetting as the feature representation of the forgetting data and false retaining data is entangled. Through the effect of actively forgetting the given data on the other parts of data, we can also utilize the representation gravity defined as follow identify false retaining data,
Definition 3.3 (Representation gravity). Given the empirically supported gravity effects in Theo- rem 3.2, we can have Lcon (z, y. 8) to reflect the similarity d(x",an x) in the model 0t with a small t, e.g. Lcon 4.n. (z, y, 8) ((fo(z). v) {(fo (I), v)], or we can calculate class-wise accuracy change.
is empirically demonstrated in Figure 3 (also the latent representation distance and class accuracy trends in Figure 9), the corresponding changes in accuracy and loss values show that generally the smaller the distance in representation level. the similar forgetting dynamics the model would have on prediction. Regarding the issues of insufficient representation and decomposition missing, we can
5

---

Published as a conference paper at ICLR 2026
utilize the gravity effects identify the unidentified forgetting data in the remaining set, and reveal the needs of deconstructing entangled representation by simultaneously considering two parts.
3.3 ALGORITHM FRAMEWORK OF TARF
Based on the intuition, we introduce the whole framework of TARget-aware Forgetting (TARF), to enable the mismatched class-wise unlearning tasks. Given the identified forgetting data, we illustrate the overall process Figure 4, and introduce its dynamic learning objective as follows:
LIARF = k((t)
1
Σ ((f()).0))
Σ e(f(z).)) - 7(r, D..
Drl
Dunl
(3)
(a-y)~-i
Annealed Forgetting (1)
Target-aware Retaining (r(r)
where k(t) serves as an annealing strategy to control the strength of the forgetting part. Along with training, we expect the overall objective to approximate the retraining ones LTARF + Lietrai through,
(rkk) 1_ 0, (T) b-
Leetrain:
given the initially provided forgetting data Dr and the remaining set Dun- Specifically, we design the two dynamic hyperparameters k(t) and r(., y, t) as follows,
k(()) = max k: (T - tt) .0] tE |0,
-1
Lcon (z, y, 0c,) > or < t1, J (2,, yr 02,) < 33 and 2 4
(O,T) 10.
(5)
T
where T indicates the total training time (e.g., epochs), and the value of k(t) decreases with the training process, 33 can be estimated by the information about the specific unlearning request and the rank of loss/accuracy change (e.g., setting the threshold /3 as the lowest value of top- 10% data with in a descending order, to select the most influenced part) at (1, to and 4 respectively control the end time of active forgetting and the begin time of retaining part. The whole process can refer to Figure 4 for an intuitive understanding how the previous objective controlled by i and T organically consists of three phases to tackling the revealed challenges in mismatched unlcarning, and we also provide functionality explanation about those factors in Appendix and guidance based on empirical results.
Phase I: Target Identification. Be- fore ti since T(x, y, t) = 0, Eq. 3 can be formalized as, LITARF-PaaC-T k(t)- (-- [D. (x,,))>,, {(((x), v)), in which the retaining part is waiting for the dynamic information revealed by this phase. As shown in Figure 3, the false retaining data in Drr can be identi- fied due to the similar forgetting dynam- ics with the forgetting data. We utilize the class label information in main tasks as is also available for unlearn- ing, We can obtain the accuracy drop of each class and estimate 3 (refer to Appendix E for details). In Figure 5(a), we show the selected classes in accuracy drop and identification efficacy. Specif- ically, the left shows that classes belonging to the target concept (blue) experience significantly larger accuracy drop than the remaining classes (yellow), which serves as an indicator for target identification; the right presents the performance with different amounts of given forgetting classes.
Arsdded Perpeting
Colean
Targoooor Bebeising
Clinaity
Terpe Mersttakee
Turge
The overull frumework consists of two ubjective parts, e.g., annealed forgetting target- aware retaining, which he regarded as three pluses to enable all the class-wise unlearning tasks through the view of the unlearning process. (a) Phase I utilizes the gradient ascent to construct dynamic information for all class data: (b) Phase II simultancously considers gradient ascent forgetring data and gradient deseent on remaining data that is hard to affect to separate target coucept, (c) Phase conducts gradient descent on the selected to approximate retraining.
Figure 4: Overview of the proposed framework TARF.
Phase II: Target Separation. After phase I, the retaining part is engaged with the forgetting part with the identified data Dr, and the remaining retaining data D, By simultaneously considering the forgetting and retaining part as Eq. 3. LEARE-PhaSe-I encourages the model to deconstruct the target concept and reconstruct the feature representation of the retaining part, which can effectively decouple the entangled feature in the model mismatch forgetting. In the first panel of Figure 5(b), compare the accuracy gap RA and UA, which indicates the success (refer the dashed line of

---

Published as a conference paper at ICLR 2026
0.
A Cop Maniin
Accumnc
0.4
Accuracy
(QA) (24)
(TARF) RAITARF)
Accuracy
Renaiee
(GA)
TASTARI)
0.2
after GA
Retia Retrained
GA
14 55 Indox
Numbar Classes
Epoch
Epochs Tuning
(a) Target identification
(b) Target separation
We show (a) accuracy changes in target identification, and unlearning performance in data mismatch forgetting; (b) accuracy gap of retaining and forgetting part of the same class, as well as the need of reconstruction. Figure 5: Target identification and target separation for unlearning under mismatch.
Retrained reference) of disentanglement. applies gradient ascent and decent to deconstruct the entangled representation, achieving the expected accuracy gap (e.g., isolating the target concept with affected retaining data as shown in Figure 2).
validates the rationality of our method, which jointly
Phase III: Retraining Approximation. After la. we focus on retaining in the current phase, which approximates the retraining objective follows, TTTAR--POSS-II [ (a.y)~Da (((()), y)-
((z y. t), where we use at t to indicate the identified hard-to-effeee retaining data, and continually reconstruct the representations. Since the general goal of unlearning considered in our work is similar to retraining, this phase can prevent excessive forgetting. In the second panel of Figure 5(b), we compare the performance using different lengths this phase to show retrain approximation. Note that in Phase-lI, our TARF may induce over-deconstruccton (larger Acc Gap than that of the dashed line for Retrained reference), so it demonstrates the necessity of our Phase-Ill III focusing purely on retraining to approximate the Retrained reference by using different epochs of this stage.
Remark 3.3. Note that the ttree-phase are interpreted from a unified framework rather than an ad-hoc pipeline. Each phase builds the previous insights: Phase I identifies potential forgetting targets, Phase II separates entangled representation, and Phase III approximates retraining on those generalizable knowledge. The whole process enables flexible framework for all mismatch scenarios.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets and models. In our experiments, we mainly explore unlearning for conventional image classification tasks. To build an easy-to-ados testbed for our new settings, we adopt the benchmarked datasets, e.g.,
(Krizhevsky, 2009) with their superclass information (refer to Tables 13, 14 and 15) the main experiments. Note the coarse-to-fine label structure of CIFAR- is obtained by grouping based semantic proximity (Dhakad et al., 2024) to enable the controllable experiments. We train two models based on the original classes and its superclass respectively, and instantiate four tasks (as illustrated in Figure 1). More details are summarized in Appendix D.4. Following (Jia et al., 2023; Fan et al., 2023), we use R (He et al., 2016) as the main architecture obtain original models with standard learning, and then set it be the basis for unlearning. And we also adopt Tiny Imtgeeet and ImageNet (Krizhevsky et al., 2012) for large-scale experiments, and adopt ImageNette (Howard, 2019) and TOFU (Maini al., 2024) for case studies.
Evaluation metrics. The general goal of unlearning considered in this work is to approximate the Retrained model. To give comprehensive evaluation. we adopt 5 specific evaluation metrics in classification tasks following previous works (Jia et al., 2023; Fan et al., 2023). We utilize Unlcarning Accuracy (UA) to evaluate the accuracy of the unlearning targeted subset; Retaining Accuracy (RA) to evaluate the accuracy of the retaining subset; Testing Accuracy (TA) evaluate the generalization ability of the model; Membership Inference Attack (MIA) to evaluate the efficacy of unlearning by the confidence-based predictor. Note that any single indicator does not represent optimally in the approximation of Retrained reference. All the above will be compared with that of the Retrained model and summarized in "Gap" value (averaged gap with Retrained, i.e., 1 Σ RRo... Ro |) to indicate the overall performance (the lower the better), and we also adopt TIME to present the computational time. Detailed evaluations of different scenarios provided in Appendix B.2.
7

---

Published as a conference paper at ICLR 2026
Table 3: Main Results (%). All are trained on the same backbone and initialization (except for the reference Retrained from scratch). Bold numbers are superior results, and we also indicate the second-best results of "Gap" for readability, & indicates smaller values are better (Complete results with mean and std values in Appendix F.7). Overall TARF performs well across various tasks.
Type/D D
Dansset
CIFAR-10
CIFAR-100
Method /Metrics
CA
R.A
TA
MBA
Cap1
TIME
UA
MIA
Gapl
TIMEL
Retraised (Ref.)
0.000
09.51
94.00 03.00
100:30
43.3
0.00
97.85
76.03
1001.00
43.2
FT (Wareecke et al.. 2023)
1.07
100.00
4.43
0.67
96.32
72.34
100.00
1.47
5.00
RL. (Toneva et al. 2018)
4.13
07.65
91.23
100.00
2.56
4.83
1.00
96.09
73.00
100.00
1.70
4.96
(ishida al., 2020)
0.49
95.24
58.11
99.75
0.25
08.56
99.89
0.06
mutched
BS (CThen 2021)
25.04
88.67
0.45
100,00
26.67
Lx-sparse
2023)
0.00
94.20
100.00
4.39
100,00
4.39
SalUnn (Fan 2023))
0.00
91.32
85. 87
100.00
5,65
0.00
75.34
62.14
100.00
9.10
5.75
SCRUB (Kurmanji al., 2023)
0.00
99.94
00
100.00
1.03
2.88
0.00
99,95
76.75
100,00
6.71
5.23
TARF (ouns)
0.00
98.25
91.95
100,00
1.01
4.21
0.00
96,90
72.53
100,00
1.11
4.68
Retrained (Red.)
K7.76
99.58
95.99 91
20.57
43.8
88.23 22
96.58
78.50
25.78
43.8
FT (Wamneke et nl. 2025)
94.67
99.55
93.55 16.56
1.56
5.33
4.29
92.67
95.02
79.34
16.33
4.58
4.36
RL (Toneva et al. 2018)
53.60
97.88
972 39
96.66
28.34 84
4.82
S0.11
95.83
79.83
99.00
21. 35.35
4.93
GA (Ishida 2020)
5.76
86.99
82.20
94..98
4.5 68
0.25
6.78
94.83
76 96
97.78
39 68
0.06
Model mismatch
IU [1oso et al., 20213
23 69
87.34
82.5 57
89. 87
59 74
0.44
54.67
96.85
70
86.44
70 14
0.49
BS (Chon et ol., 2023)
10.2 70
50.77
40.35 39
95:
62.05
0.79
11.11
95.00
73.78
95.22
57.
0.30
Ly spasse (Jia et al., 2023)
93.11
94.76
91.065 16.
14.44
5.15
4.24
90.22
96.78
75.81
18.85
3.25
5.00
Sallin (Fan et al., 2023)
8.91
93.95
84.10 38
99.32
43.09
6.04
66.33
78.83
70.78
77.00
25
5.97
SCRUB (Karmaay il.,, 2023)
95.
99.81
15.38
3.61
3.05
91.44
99.74
21.11
2.45
4.12
(ours)
91:11
97.49
92.44
17.82
2.90
4.31
56.67
97.05
50.07
26.00
1,21
Retrained (Ref)
0.00
09.38
100:00
52.1
0.00
97.85
100.00
FT (Warnekk ol. 2023)
50.49
65
25.78
4.38
73.53
RL (Toneva 2020)
97.56
4.79
40.82
20.50
21.38
70:33
Target mismateh
IU tozo 2021)
44.51
88.017
58.75
0.44
29:58
63.09
41.95
0.50
BS (Chen al., 2025)
53.62
88.65
75
76.33
0.82
40.44
68.06
85.10
La-aparso (lia et al. 2023)
49.47
93.61
85.83
$1.
27.36 16
4.38
56.00 09
94.63
73.00
48.04
76,
4.78
Saitin (Fan et ol., 5023)
46.63
91.08
86.31
60:94
25 19
5,50
59 64
75.52
62.37
65. 96
27.
5.81
SCRUB (Kunmanja et sl., 2023)
49.98
99.94
92. 10
50.18
25,53
2.89
39.64
99.99
75.32
44.89
29.
3.52
TARF' (ours)
0.06
97.57
90.81
100.90
1.23
4.23
0.31
97.35
73.68
100.00
0.21
4.85
Retraaned (Ref.)
0.00
99.54
95.56
100.30
52.1
0.00
95.50
50.15
100,00
53.2
FT (Warnecke et al. 2023)
96.79
98. 89
93.26
6,48
48.41
4.32
$2.62
95.65
79.77
37.24
37.15
4.93
RI. (Toneva et al., 2018)
76.47
97.68
91.99
49.81
$3.04
4.76
80.78
96.82
70:90
70.76
50.49
4.97
GA (Ishida al., 20000)
5.69
96.41
90.778
93.03
5.99
0.25
6.00
97.65
79:23
98.04
2.43
0.85
Data mismatch
IU flzso et al.. 2021)
22.80
95.50
89.54
88.57
[1008
0.44
31.51
98.95
75. 78.20
88.09
11.46
0.48
BS (Chen al., 2023)
16.70
61.21
49.70
92:24
22.37
0.83
15.38
98.50
72.78
96.2 23
6.76
0.96
I, spaspas et al., 20253
98.31
91.00
9.52
4.78
59.33
79102
22.49
42
5,00
at al., 2023)
51.77
93.87
63.52
24.75
5-72
72.93
78.87
71.04
54.13
5.72
SCRUB (Kummangs si., 2023)
97.13
99.89
10:99
46.76
2.94
95.50
99.79
79.68
15.11
45.54
3,68
TARF (ours)
0.00
4.20
78.98
100.00
4.78
4.2 PERFORMANCE EVALUATION
In this part, we present the main comparison results with those considered as baselines in the four unlearning tasks. We also report results under multiple runs in Appendix F.7 with std values.
In conventional benchmarks, all the retrained models (termed Retrained) are trained with the fully aligned retaining data. In Table 3, we can find the previous unlearning methods achieved satisfactory performance in conventional all matched forgetting, but did not perform well on the other three newly considered tasks with the label domain mismatch. Note that UA of Re- trained (Ref.) in the model mismatch scenario is not equal O since it is evaluated with su- perclass label. Specifically, since the previous methods partially rely on forgetting data or remaining data, it results in ineffective or excessive forgetting due to the insufficient representation or decomposition missing. For example, FT can retain a similar RA with the Retrained but be less effective in forgetting, while GA reaches the lowest UA across different tasks but sacrifices too much performance on the retaining dataset. In contrast, TARF can generally perform better (or comparable with the best method). We also present Table 2 to show fine-grained evaluation on unlearning target within superclass in model mismatch.
Table 2: Fine-graine evaluation on superclass.
Midel Mibrracch
Mahud
D0-2"
LA--R
RA
1A
MIA
Gap
F7
92.09
97.15
08.53
$3.55
0.36
5.95
CIRAR-I0 (UA.F. UA-II: maack
RL
48.69
0.00
53.59
07.85
$0.39
96.60
94.95
20.88
BS
50.77
Lopane
91.40
04.52
04.70
91.83
14.44
5.47
SCKL'S
91.09
09:21
09.31
54.31
15:38
4.31
TARE (maz))
55.34
17.82
Natramed (llet))
77.35
05.25
06.55
18:50
25:75
FT
90.31
04.35
10.34
16.31
1.51
(LA-ST bey gint:
CIPAR-10
BS
GA
17.00
5.64
15.55
06.5
85 00
76.96
73.78
91.74
01.33
43.0
7358
Legaro
$6.05
96 76
78.81
19:54
4.55
TA RT
34.50
04.65
09
$0.51
36.00
1.36
For verification on large-scale datasets, we evaluate the method on Tiny-ImageNee and ImageNet- Ik with lager models. Due the space, we show the results ImageNet- 1k in main text, and leave other results in Appendix F.5, and forgetting multiple classes in Appendix F.9. It shows that our TARF can achieve satisfactory performance with respect to the overall gap with Retrained references.
8

---

Published as a conference paper at ICLR 2026
Table 4: Results (%). Comparison with the unlearning baselines on ImageNet- Ik. All matched forgetting: unlearn 1 class; Target mismatch forgetting: unlearn three classes belonging to "fish".
Type/D D
Dusaset
All matched
Target mismutch
Method/ / Metric
UA
RA
TA
MIA
Gapl
TIMEL
LA
RAA
TA
MLA
Gspl
TIME4
Retrained (Re(.
0:00
79.77
77.64
100.00
7075.48
0.00
80:09
77:54
100.00
7727 54
FT (Ramocke at al. 2023)
0.00
70.15
71.95
100.00
3.82
608.
0.79
70.26
72.07
100.00
4.07
600 63
RL (Toseva et 2018)
81.38
70.22
11.79
19:45
44.29
909.64
79.69
69.58
71.77
23.03
43.14
972.02
(Eihada 2000)
8.00
67.36
100.00
5.95
8.76
0.00
37.74
47.17
et al., 2023)
8.00
31.15
100.00
22.48
9.03
0.06
21.57
99.97
27.13
sparse
SCRUB (Kuminaji
29.77
74.92
665,42
681.55
TARF (ouns)
0.00
70.53
72.25
100,00
3,66
600.11
71.79
100.00
IeI
Dutaset
Model matched
Data mismatch
Method / Metrics
UA
RA
TA
MIA
Gapb
TIMEL
CA
RA
TA
MIA
Gapl
TIME]
Retrained (Ref.)
79.
50.00
70.29
25. .9.69
6301 .2
0.00
80.36
70.38
100.00
6493 16
FT (Wamecke RL. (Tumeva et 2018)
20231
83.31
70.33
64.005
19.00
6.68
095.
0U06
$9.59
63.76
100100
4.24
693.18
87.62
69.43
63.26
15.23
9.13
9F9 84
88.21
70:13
63.81
12:21
48.15
956. 13
GA (Bihala al. 20200
8.00
66.82
58.91
100.00
44.56
17.44
0.06
15.35
14.34
0.00
55.26
17:58
BS (Chen et al., 2023)
0.00
45.81
40.84
100.00
54.28
19.69
0.00
13.00
12,10
100000
31.41
23.70
L.-iparie (Jix 2023)
82.00
67:94
19:15
1091 22
0.06
66.75
61 61.03
180000
(Kumannji et al., 2023y
86.08
74.82
14.89
6.34
663.61
14.18
1.27
(ours)
80.62
70.27
19:45
601.25
0.00
100.00
4.17
7UFU Rogiggo
Alheadde
Tago Sirmaa
NPO
(IA)
FLENO
Slanall
Sclting Soggun
Bipsscedn aesssceddaan. Mamanch
Dau Manaac
Vieve
Pe
Patkes
E1300
NPO
6:1155
3.2455
Mali
ae (0) GA Parkiw RT QA Pvb
Uh (GA)
TARI DENY
40113
3.2110
0.643
Dan
TKBR (GA)
6.809
(a) SD
(b) CL
(c) TARF
TARe (NP0)
01-07
Figure 6: Application data mismatch concept Table 5: Application on information removal on removal of image generation with stable diffusion. LLM with TOFU dataset for real-world applica- Full results with more Tables are in Appendix E.3. tion. More discussions are in Appendix F.8.
For case study on real-world application, we apply our TARF in the scenario of concept removal with stable-diffusion (Ho al., 2020) and personal information removal with LLama3.2 (Grattafiori et al., 2024). Considering the practical data mismatch forgetting on where users report some undesirable examples to represent the unwanted concept, we show the efficacy of unlearning the "springer" and "tench" in Figure 6. We also constructing the four similar mismatch scenarios using TOFU dataset. Due to the limited space, we leave more details in Appendixes E.3 and F.8.
4.3 ABLATIONS AND FURTHER EXPLORATION
In this part, we provide further exploration of the three class plass-wise unlearning tasks and conduct various ablation studies to characterize TARF, More results and discussions are provided in Appendix F.
Weighted control in annealed gradient ascent. To analyze the annealed gradient ascent, we present the results on the left of Figure to show the effects of initialized strength & on the all matched setting (results on other settings can refer to Figure 17) using the CIFAR-100 dataset. The results show that a proper i (e.g., about 0.05) can achieve satisfactory performance. However, the larger k results in lower retaining performance and higher Gap value as the strength increases feature deconstruction. For the hyperparameters, we discuss the computational stability from functionality understanding and also synthesize practical guideline Appendix E. with the empirical results of ablation study.
Constant or dynamic gradient ascent for forgetting. In the middle-left of Figure 7, we study whether we need the rearning-rate-redueed k for the forgetting part. Specifically, we compare with using constant k and caariing-aate-nccraasse & on two model mismatch forgetting tasks. The results demonstrate that annealed gradient ascent can achieve more similar performance with the Retrained on forgetting data. The gradient ascent is considered simultaneously with gradient descent for restricting the forgetting region, while we adopt the annealed one since the unlearning target is to approximate the retrained model instead of continually maximizing the loss of forgetting data.

---

Published as a conference paper at ICLR 2026
200
100
20
80
Accuracy
60
RM
00
Alroord Booked (o
10
Gap
Coudect Code 21
20
Uneer Perserins
Lineer Disroraity
Rished12
Betriine Seferance
NibeRaddue- 50
0.0
02
OTAR-LS
CPAN-100
Initialized Strength&
Gradient
Different Model Structures
Operation for Selected
Figure 7: Ablation studies: Left: performance using different initialized & on all matched forgetting; middle-left: effects of constant or different dynamic gradient ascent controlled by k(t); middle- right: comparison of forgetting with different model structures; right: comparison of using different operations on the selected forgetting data. More experimental details can refer to Appendix F.
Unlearning on models trained by different structures. In the middle-right of Figure 7, we investi- gate unlearning with different structures, e.g., ResNet-18 (He al., 2016). VGG-16bn (Simonyan & Zisserman, 2015), and WideResNet-5h (Zagoruyko & Komodakis, 2016). The results on the model mismatch forgetting demonstrate that TARF can achieve the lower performance gap than FT, evalu- ated with the retrained reference. With the increasing model capacity the original training tasks, we can also find the model with smaller capacity makes it harder to decompose the entangled feature representation for achieving the unlearning target, which increases the representation complexity.
Different operations on the selected forgetting data. In the right of Figure 7, we present the ablation the specific gradient operation on the identified false retaining data Dr. We compare using the gradient ascent (-k(t)) and cleaning (0) with the Retrained reference in target mismatch forgetting. Except for the similar forgetting efficacy achieved by the three trials, major differences exist in the performance evaluated by RA. The results show that gradient cleaning may be a better choice for Dr, not deconstruct the features too much and affect the retaining accuracy.
Broader explorations of unlearning with TARF. Beyond the performance comparison and ablation on major benchmarks, we also conduct broader exploration our TARF to give a balance view on the unlearning capabilities. Specifically, we also investigate the performance robustness under varied false-retaining set size for quantile-choice in Appendix E; discuss and check the computational cost of TARF in target identification stage in Appendix E.2; verify the robustness of TARF under the reakkly-uuperviedd scenario or more challenging multiple concept unlearning scenarios in Appendix
CONCLUSION
In this work, we decouple the class label and target concept in class-wise unlearning. By introducing the label domain mismatch among forgetting data, model output, and target concept, we uncover three additional tasks beyond the conventional all matched forgetring, e.g., target mismatch, model mismatch, and data mismatch forgetting. We identify the insufficient representation and decomposi- tion lacking of restrictively forgetting the target concept, and reveal the crucial forgetting dynamics in the representation level for the feasibility of these unlearning requests. Based on that, we propose the TARF that assigns an annealed gradient ascent on the identified forgetting data and the normal gradient descent on the selected retaining data. By collaboratively considering the forgetting/retaiing target, TARF is more accurate in unlearning while maintaining the rest. We hope our work can provide new insights and draw more attention toward the practical scenarios of machine unlearning.
Open challenge and future work discussion. Representation gravity, that relies on the forgetting dynamics, is central to the ability of TARF to identify latent target concepts. In challenging regimes where concepts are inherently ambiguous, weakly clustered, or attribute-entangled (e.g.. certain long-tailed or multi-attribute scenarios), the underlying representation structure itself becomes less separable. It affects all existing unlearning methods as the ambiguity originates from the nature of the data. In our exploration, we also observe few preliminary cases where the gravity signal becomes weaker and the ranking slightly noisier. We therefore view these situations as inherent difficulties when the target concept not well-defined in the representation space. At the same time, these cases also suggest promising avenues for future research, such as incorporating external knowledge (e.g.. text embeddings, semantic priors, or multi-modal cues) to assist more challenging unlearning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
JNZ and BH were supported by RGC General Research Fund No. 12200725, RIKEN Collaborative Research Fund, and HKBU CSD Departmental Incentive Scheme. JCY is supported by National Natural Science Foundation of China (No. 62306178) and STCSM (No. 22DZ2229005), JLX was supported by RGC Grant C1043-24GE MS was supported by JST ASPIRE Grant Number JPMJAP2405.
ETHICS STATEMENT
This work adheres fully to the Code of Ethics. Our study does not involve human animal subjects, and all datasets and models used are publicly available as detailed experimental section and appendix. We have taken care ensure that methodology does not propagate sensitive, private, or personally identifiable information. The research is intended purely for advancing scientific understanding on machine unlearning and poses no foreseeable risks of misuse or harm. We confirm compliance with legal, fairness, transparency, and research integrity standards.
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility of our results. A detailed version of reproducibility statement can found in Appendix, where we summarize critical aspects facilitate verification. In addition, and instructions for reproducible results. Detailed descriptions of models, datasets, and experimental setups provided in the Section 4.1 and Appendix F.1. And we also introduce the construction of unlearning task instantiation in detail in Appendix D.4 for a specific reference.
also provide an anonymous repository containing code, training scripts,
REFERENCES
Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey. Machine Learning, 2020.
Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation, 2003.
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportuni- ties and risks of foundation models. In arXiv, 2021.
Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 2018.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In IEEE Symposium on Security and Privacy (S&P), 2021.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In IEEE Symposium on Security and Privacy (S&P), 2015.
Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning. In arXiv, 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. ICMI., 2020a.
Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong. Kewei Chen, and Zhangyang Wang. Self-pu: Self boosted and calibrated positive-unlabelee training. ICML, 2020b.
Narendra Singh Dhakad, Yuvnish Malhotra, Santosh Kumar Vishvakarma, and Kaushik Roy. Sha-cnn: Scalable hierarchical aware convolutional neural network for edge ai. In arXiv, 2024.

---

Published as a conference paper at ICLR 2026
Zonglin Di, Sixie Yu, Yevgeniy Vorobeychik, and Yang Liu. Adversarial machine unlearning. In ICLR, 2025.
Jun Du and Zhihua Cai. Modelling class noise with symmetric and asymmetric distributions. In AAAI, 2015.
Marthinus du Plessis, Gang Niu, and Masashi Sugiyama. Convex formulation for learning from positive and unlabeled data. In ICML, 2015.
Marthinus C du Plessis, Gang Niu, and Masashi Sugiyama. Analysis of learning from positive and unlabeled data. In NeurIPS, 2014.
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, and Sijia Liu. Salun: Em- powering machine unlearning via gradient-based weight saliency in both image classification and generation. In arXiv, 2023.
Chongyu Fan, Jiancheng Liu, Alfred Hero, and Sijia Liu. Challenging forgets: Unveiling the worst-case forget sets in machine unlearning. In arXiv, 2024.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, (4):128-135) 115,8,,,, 1999.
Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kauffan and David Bau. from diffusion models. arXiv preprint arXiv:2303.07345, 2023.
Erasing concepts
Yinggiang Ge, Shuchang Liu, Zuohui Fu, Juntao Tan, Zelong Li, Shuyuan Xu, Yungi Li, Yikun Xian, and Yongfeng Zhang. A survey on trustworthy recommender systems. ACM Transactions on Recommender Systems, 2022.
Antonio Ginart, Melody Guan, Gregory Valiant, and James Zou. Making ai forget you: Data deletion in machine learning. In NeurlPS, 2019.
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In CVPR, 2020.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning. MIT Press, 2016.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, et al. The llama herd of models. In arXiv, 2024.
Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In AAAI, 2021.
Sheng Guo, Weilin Huang, Haozhi Zhang, Chenfan Zhuang, Dengke Dong, Matthew R Scott, and Dinglong Huang. Curriculumnet: Weakly supervised learning from large-scale web images. In ECCV, 2018.
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In NeurlPS, 2018
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In ICML, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
12

---

Published as a conference paper at ICLR 2026
Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius. The european union general data protection regulation: what it is and what it means. Information Communications Technology Law, 2019.
Jeremy Howard. Imagenette: A smaller subset of 10 easily classified classes from imagenet, March 2019. URL https: htt/ggttuuuu com/ fastai/ /imaggmeateee..
Yu-Guan Hsich, Gang Niu, and Masashi Sugiyama. Classification from positive, unlabeled and biased negative data. In ICML, 2019.
Zhehao Huang, Xinwen Cheng, JingHao Zheng, Haoran Wang, Zhengbao He, Tao Li, and Xiaolin Huang. Unified gradient-based machine unlearning with remain geometry enhancement. In NeurlPS, 2024.
Takashi Ishida, Ikko Yamane, Tomoya Sakai, Gang Niu, and Masashi Sugiyama. Do we need zero training loss after achieving zero training error? In ICML, 2020.
Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. Approximate data deletion from machine learning models. In AISTATS, 2021.
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. In NeurIPS, 2023.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 2017.
Ryuichi Kiryo, Gang Niu, Marthinus du Plessis, and Masashi Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In NeurlPS, 2017.
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In ICML, 2017.
Adriana Kovashka, Olga Russakovsky, Li Fei-Fei, and Kristen Grauman. Crowdsourcing in computer vision. In Found. Trends Comput. Graph. Vis., 2016.
Alex Krizhevsky. Learning multiple layers of features from tiny images. In arXiv, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolu- tional neural networks. In NeurIPS, 2012.
Meghdad Kurmanji, Peter Triantafilouu, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. NeurIPS, 2023.
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski. Shashwat Goel, Gabriel Mukobi, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. In ICML, 2024.
Qing Li, Jiahui Geng, Derui Zhu, Fengyu Cai, Chenyang Lyu, and Fakhri Karray. Sauce: Selective concept unlearning in vision-language models with sparse autoencoders. In CVPR, 2025.
Bing Liu, Wee Sun Lee, Philip S Yu, and Xiaoli Li. Partially supervised classification of text documents. In ICML, 2002.
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 2015.
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Machine unlearning in generative ai: A survey. In arXiv, 2024.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. ICLR, 2018.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and Zico Kolter. Tofu: A task of fictitious unlearning for lms.. In arXiv, 2024.
13

---

Published as a conference paper at ICLR 2026
Aditya Menon, Brendan Van Rooyen, Cheng Soon Ong, and Bob Williamson. Learning from corrupted binary labels via class class-probability estimation. In JCML, 2015.
Aashiq Muhamed, Jacopo Bonato, Mona Diab, and Virginia Smith. Saes can improve unlearning: Dynamic sparse autoencoder guardrails for precision unlearning in llms. In arXiv, 2025.
Seth Neel, Aaron Roth, and Saeed Sharif--Malvjeedii Descent-to-delete: Gradient-based methods
for machine unlearning. In Algorithmic Learning Theory, 2021.
Stuart L Pardau. The california consumer privacy act: Towards european-style privacy regime in the united states. J. Tech. L. Pol'y, 2018.
Jeffrey Rosen. The right to forgotten. Stan, I. Rev. Online, 2011.
Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to forget: Algorithms for machine unlearning. NeurIPS, 2021.
Vedant Shah, Frederik Träuble, Ashish Malik, Hugo Larochelle, Michael Mozer, Sanjeev Arora, Yoshua Bengio, and Anirudh Goyal. Unlearning via sparse representations. In arXiv, 2023.
Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unlearning: A survey and taxonomy. In arXiv, 2023.
Shaofei Shen, Chenhao Zhang, Yawen Zhao, Alina Bialkowski, Weitong Tony Chen, and Miao Xu. Label-agnostic forgetting: A supervision-fire unlearning in deep models. In arXiv, 2024.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Under- standing factors influencing machine unlearning. In IEEE European Symposium on Security and Privacy (EuroS&P), 2022a.
Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Papernot. On the necessity of auditable algorithmic definitions for machine unlearning. In USENIX Security Symposium, 2022b.
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey Gordon. An empirical study of example forgetting during deep neural network learning. In ICLR, 2018
Enayat Ullah, Tung Mai, Anup Rao, Ryan A Rossi, and Raman Arora. Machine unlearning via algorithmic stability. In COLT, 2021.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008
Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. NDSS, 2023.
Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S Yu. Machine unlearning: A survey, ACM Computing Surveys, 2023.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. In arXiv,, 2023.
Youngsik Yoon, Jinhwan Nam, Hyojeong Yun, Jaeho Lee, Dongwoo Kim, and Jungseul Ok. Few-shot unlearning by model inversion. In arXiv, 2022.
Hwanjo Yu, Jiawei Han, and KC-C Chang. Pebl: Web page classification without negative examples. IEEE Transactions Knowledge and Data Engineering, 2004.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Eric Zhang, Kai Wang, Xinggian Xu, Zhangyang Wang, and Humphrey Shi. morget-me-not: Learning to forget in text-to-image diffusion models. In arXiv, 2023.
Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. In COLM, 2024.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
Discussion about Related Work
16
A.1 Machine Unlearning
16
A.2 Positive-unlabeled Learning
17
Details about Considered Baselines and Metrics
17
B.1 Unlearning Methods
17
B.2 Evaluation Metrics Regarding Different Scenarios
20
B.3 Full Results of Figure and Figure
21
Proof of Theorem
22
Full Discussion about Label Domain Mismatch
24
D.1 A two-layer Label Structure of Mismatch
25
D.2 A three-layer Label Structure of Mismatch
................................
25
D.3 Further Exploration on the Other Different Scenarios
27
D.4 Specific Information of the Instantiated Tasks
28
D.5 Discussion on the Practicality of Label Domain Mismatch
29
D.6 Discussion on the Scenario Commonalities and Framework Principles
30
Algorithm Implementation and Explanation
31
E.1
Discussion the functionality of hyperparameters and tuning principle
32
E,2
Discussion on the Algorithm Computation Cost
32
E.3 Case study for Unlearning Generation Concept
34
E,4
Discussion on TARF with Limited Class Information
34
E.5
Discussion on the assumption of super-/sub-class information
35
Additional Experimental Results
38
F.I
Extra Experimental Setups
38
F2
Discussion about Target Identification in unlearning
38
E3
Discussion about TARF All Matched scenario
38
F44
Discussion about TARF on Weakly-supervised Scenario Forgetting in the Large-scale Dataset
39
E.5
41
F6
Forgetting with Different Model Structures
F77
Full Results with Different Forgetting Tasks
41
F8
Application on Mismatched Forgetting with LLM
42
F9
Forgetting Multiple Class by TARF
42
Broader Impact
43
15

---

Published as a conference paper at ICLR 2026
The whole Appendix is structured in the following manner. In Appendix A, provide detailed discussion with related works of machine unlearning and other aspects. In Appendix B, we review the representative baseline methods in machine unlearning, which considered in our experimental comparisons. And we also detailed the evaluation for different unlearning scenarios with full results on challenge and representation analysis. In Appendix D, introduce the complete scenarios considering the mismatch issues in machine unlearning, going beyond the four basic scenarios presented in the main text. Appendix E, formally present the algorithm implementation of our proposed TARF with its variant, and further explanation of the rationality of TARF in unlearning. In Appendix F, we provide additional experimental results to characterize forgetting dynamics and the properties of TARF. In Appendix G, we discuss the potential broader impact and limitations of our work.
LLM USAGE STATEMENT
In this paper, no large language models (LLMs) were used in the conception, execution, or analysis of this work. All research ideas, experiments, and text were developed and written solely by the authors.
REPRODUCIBILITY STATEMENT
Below we summarize some critical aspects to facilitate reproducible results:
Datasets. The datasets we used are all publicly accessible, which is introduced in Section 4.1. For our newly introduced unlearning scenarios, we provide the specific dataset construction in our code, implemented as described in Section 4.1 and Appendix D.4.
Assumption. Following the previous work (Warnecke al., 2023; Jia et al., 2023; Fan et al., 2023), We set our experiments to a tuning scenario where well-trained model is available, and all the training samples are available but limited samples are labeled as "to be unlearned".
Open source. The code repository will be available in an anonymous repository for the reviewing purposes. We provide series of unlearning methods considered in work and also the pre-trained model for unlearning.
Environment. All experiments are conducted with multiple runs on NVIDIA Tesla V V100- SXM2-32GB GPUs with Python 3.8 and PyTorch 1.8. More detailed requirements can also refer to the environment descriptions in our aforementioned source codes.
A DISCUSSION ABOUT RELATED WORK
In this section, discuss the related literature machine unlearning, and provide more detailed comparisons of some work with their approaches and motivations.
A.1 MACHINE UNLEARNING
Machine unlearning targets to adjust a trained model to scrub the data influence (Koh & Liang, 2017: Shaik al., 2023; Xu al., 2023). It is initially proposed to protect data privacy (Cao & Yang, 2015; Bourtoule et al., 2021; Ginart et al., 2019), and a series of studies explore probabilistic methods through the differential privacy (Ginart et al., 2019; Guo al., 2018: Neel et al., 2021; Ullah et al., 2021; Sekhari et al., 2021). Although having the provable guarantee on the unlcarning errors, the strong algorithmic assumptions hinders the practical effectiveness (Jia et al., 2023). Current research (Golatkar et al., 2020; Thudi et al., 2022b;a; Fan et al., 2023; Chen et al., 2023; Yao et al., 2023; Gandikota et al., 2023; Zhang et al., 2023) focus more on developing more effective and efficient unlearning methods to approximate the Retrained model, with the given trained model. As for the assumption on data generation, prior works (Golatkar et al., 2020; Warnecke et al., 2023; Jia et al., 2023; Chen et al., 2023) mainly consider all matched forgetting targets, with similar features on the original training tasks. As for the assumption on label generation, most prior works (Bourtoule al., 2021; Graves al., 2021; Thudi et al., 2022a; Jia et al., 2023; Fan et al., 2023; 2024) assume
16

---

Published as a conference paper at ICLR 2026
the accessibility on the fully identified forgetting dataset, and the complementary is the remaining dataset. One recent related work (Yoon et al., 2022) considers unlearning with only a few forgetting samples but requires another generative model to generate approximated data. Our work considers more practical scenario in which we can conduct mismatched forgetting and use limited identified forgetting data with the remaining set. Another work (Shen et al., 2024) proposes label-agnostic forgetting to enable supervision-free free unlearning in deep models, while effective, but the method shows the incompatibility of the assumption since it assumes access fully identified forgetting set and retaining set during optimization which is not aligned with our formulation. With the increasing attention to safety and regulation foundation models, machine unlearning has received growing interest, machine unlearning also draw more research interests and some recent studies (Li et al., 2025; Muhamed et al., 2025) also explore the post-adjustment for foundation model oriented unlearning or concept erasure, and the focuses is structural specified and also follow the conventional assumption having the well-aligned forgetting target. While such advances have pushed the field forward, our work focuses fundamentally different challenge: label-domain mismatch, where the target concept to be forgotten does not coincide with the model's original class taxonomy. We aim to provide rigorous treatment of this overlooked yet practically important setting, including both theoretical insights and a general unlearning framework capable of addressing these mismatches.
A.2 POSITIVE-UNLABEIA LEARNING
Positive-unlabeled learning (du Plessis et al., 2014; Menon et al., 2015) tries to learn a binary classifier from a few labeled positive samples with the rest unlabeled ones. A series of PU algorithms (Liu et al., 2002; Du & Cai, 2015; du Plessis et al., 2015) are developed to train an accurate binary classifier, and can be roughly divided into two categories (Bekker & Davis, 2020). The first branch is cost-sensitive learning. which is related to importance weighting (Liu & Tao, 2015). Given the estimated class prior, these methods (du Plessis et al., 2015; Kiryo et al., 2017; Chen et al., 2020b) can develop an unbiased or consistent risk estimator for PU learning. Another branch of PU learning adopts two heuristic steps to perform binary classification. Such methods (Liu et al., 2002: Yu al., 2004) first identify reliable negative and positive examples from the unlabeled data, and then conduct semi-supervised learning. The model trained using cost-sensitive learning can also be recognizer for positive or negative samples (Hsich et al., 2019). Different from PU learning focusing on binary classification tasks, our work tries enable more practical scenarios in class-wise unlearning (Shaik al., 2023) where the class labels and target concepts are decoupled, and we consider the label domain mismatch.
B DETAILS ABOUT CONSIDERED BASELINES AND METRICS
In this section, we provide details about the considered representative baselines for machine unlearning methods, as well as their general intuitions with specific objectives. For the specifie hyperparameters adopted in different methods, we keep the same setting with previous related works (Jia et al., 2023; Fan al., 2023), and the specific values are listed in detail in our source codes. In addition. we introduce the evaluation metries in detail, corresponding to the implementations in different unlearning scenarios.
B.1 UNLFARNING METHODS
Finetune (FT). Utilizing the catastrophic forgetting (Kirkpatrick et al., 2017) in the model (c.g.. existed in the continual learning), FT (Warnecke et al., 2023) fine-tunes the given trained model partially D, with few training epochs to obtain the 8t with the following objective function,
L
1
Σ ((f(z).)).
(6)
[Dr
(Z,9)~D
Gradient Ascent (GA), Different from the normal gradient descent, GA reverses the gradient signal D, to conduct maximization with ascended gradients, resulting in the increasing loss of the forgetting data to obtain the 8.n The objective is given as follows,
La
1
Σ ((f()),))
|rd
(I.D)DD
17

---

Published as a conference paper at ICLR 2026
With reverse optimization to maximize the loss the specific data, the model can approximate 0" by directly forgetting the learned knowledge represented by the forgetting data.
Random Label (RL). Similar GA, RL (Golatkar et al., 2020) assign the random labels Y the forgetting data in D; and fine-tune the given model with it to obtain the unlearned model 0
LRL.
Σ (((()),)) (x.
Dr
Instead of using the original training label on the forgetting data in D;, RL can destroy the learned feature by using the random label y" on Dr, which violate the minimized loss value.
Influence Unlearning (TU). IU adopts the influence function (Koh & Liang, 2017) to estimate the change if the training point removed from the training loss. designed for random data unlearning (Shaik et al., 2023) with the provable guarantee on the unlearning effects. In general, IU estimates the change model parameters of O and adds the weight perturbation to the given model to obtain the unlearned one. However, it usually requires additional model information and training assumptions for the theoretical guarantee and may suffer hyperparameter tuning with inaccurate hessian estimation (Jia et al., 2023; Fan et al., 2023).
Boundary Shrink (BS). BS (Chen et al., 2023) is recently proposed for class-wise unlearning, especially the all matched forgetting. It focuses on the decision spaces (Goodfellow al., 2016) of the given trained model. The critical idea is to shift the original decision boundary to imitate the decision behavior of the model retrained from scratch. Motivated by adversarial attacks (Madry et al., 2018), it proposes a neighbor searching method to identify the nearest but incorrect class labels Vnear for DDr to guide the model to unlearn the existing class and shift the decision boundary. Using the adversarial attack to find the nearest incorrect label, the objective of BS can be formulated as follows,
LBS
Σ
e(f(x)) Yuear).
Dr
(x.y))>i
where Yoear is obtained by first perturbing the forgetting data and getting the newly predicted result as, x + sign(VV(((r)).))) Unear softmax(f(x'))
(10)
Li-sparse. Developed based on the conventional FT, Li-sparse (Jia et al., 2023) investigate the
model sparsity on machine unlearning. figures out that model sparsification can benefit the unlearning performance on different perspectives via first pruning and then conducting unlearning. By carrying out pruning and unlearning simultaneously. L1 Li-sparse proposes the sparsity- unlearning utilizing the L₁ norm-based penalty. The objective is as follows with hyperparameter 7,
Lap-spase
Σ
E(f(z). v) + >|10"1|.
(11)
DD
(a,N)~D,
and the general sparsity-aware penalty can also be added to different unlearning methods. In this work, we mainly compare the L1 FT as the previous work (Jia et al., 2023; Fan et al., 2023) considered.
SalUn. With the concern on unlearning stability and cross-domain applicability, SalUn (Fan et al., 2023) introduces the concept of weight saliency in machine unlearning. This innovation directs the attention of unlearning into specific model weights for specific data that need to be unlearned. In general, it first generates the gradient-based weight saliency map inspired by model sparsification (Jia et al., 2023) with gradient- value thresholding, where the specific generation method is defined as,
m. = o(V Dy = 0.1 2 7), 0, m. O (60 + 0.) + (1 m) O 0..
(12)
in which (g 7) is element-wise indicator function that yields value of 1 for the i-th element and 0 otherwise, is an element-wise absolute value operation, and y > is a hard threshold. and then conducts saliency-based unlearning using the generated saliency map. Specifically, SalUn
18

---

Published as a conference paper at ICLR 2026
adopts RL (Golatkar et al., 2020) to fine-tune the forgetting data in Dj on the salience map, and the extended objective is given as follows,
SaaUUn = [Prl
E (f.(f( (2,0)~Di
Σ
(ff(z).).
(13)
[Drl
(2,3)~D,
More detailed operations can refer to (Fan et al., 2023), and we keep the same hyperparameter used in (Fan al., 2023) to conduct the Mlass-wise unlearning tasks.
SCRUB. SCRUB is a newly proposed unlearning algorithm based on a novel casting of the problem into a teacher-student framework (Kurmanji al., 2023). It is designed to meet the desiderata of unlearning: efficiently forgetting without hurting the model utility. As the general target of SCRUB in forgetting is application-dependenn
is proposed with recipe that works across applications: SCRUB is first to strive for maximal forget error, which is desirable in some scenarios like removing bias or restricted contents but not in others like user privacy protection. To address the latter case, SCRUB is integrated with a rewinding procedure that can reduce the forget set error appropriately when required.
Given the original model 6 as the teacher model, the goal of SCRUB is formatting training student model 0 that selectively obeys the teacher. The overall objective can be divided into two folds, the first is to remember D, under the teacher model's guide while the second is to forget Dr by disobeying the teacher model's 's guide. To measure the degree to which the student model obeys the teacher model, SCRUB utilizes the following distance measure,
(14) where DKL is the KL-divergence and the overall measures of the distance between the student model's and teacher model's prediction distribution. With the aforementioned distance, the objective of SCRUB is as follows,
d(z; 0*) DKL (PL(p(((z) (O)))
LSCRUB = min ge N,
Σ d(zr;0")+ E.ED.
7 N (s..B.)ED,
Σ ((f(x,;")),))
Σ d(r (15) 2,ED,
NJ
where the first two parts can be regarded as a variant of distillation from a teacher model on D, and the third part is encouraging the student model to disobey the teacher model to forget the target data.
Table 6: Comparison with additional recent class-wise unlearning methods on CIFAR-100
All Matched
UA
RA
MIA
Gapl
Target Mismatch
UA
RA
TA
MIA
Gap!.
Retrained (Ref.)
0.00
97.85
76.03
100.00
Retrained (Ref.)
0.00
97.85
73.72
100.00
FT
0.67
96.32
72.34
100.00
1.47
FT
58.18
96.32
72.53
46.76
28.54
LAU
4.11
80.44
61.64
95.78
10.03
LAU
46.71
88.65
68.19
66.49
23.74
SFR-on
0.00
99.21
74.26
100.00
0.78
SFR
59.21
99.13
74.28
48.32
28.18
SG
0.00
95.21
71.23
100.00
L886
SG
$8.21
96.26
72.18
46.24
28.78
SCRUB
0.00
99.98
76.75
100.00
0.71
SCRUB
59.64
99.99
75.32
44.89
29.90
TARF
0.00
96.90
72.53
100.00
1.11
TARF
0.31
97.35
73.68
100.00
0.21
Model Mismatch
CA
RA
TA
MIA
Gapl
Data Mismatch
UA
RA
TA
MIA
Gapl
Retrained (Ref.)
88.22
98.58
78.50
25.78
Retrained (Ref.)
0.00
98.50
80.15
100.00
FT
92.67
95.02
79.34
16.33
4.58
FT
82.62
95.66
79.77
37.24
37.15
LAU
80.00
96.74
79.86
45.78
7,86
LAU
85.73
96.96
80.00
40.40
36.76
SFR-on
92.12
99.21
79.21
20.65
2.59
SFR-on
92.68
99.21
79.23
18.21
44,03
SG
89.27
93.52
73.45
19.31
4.41
SG
87.52
93.25
73.21
23.08
44.16
SCRUB
91.44
99.74
79.23
21.11
2.45
SCRUB
95.50
99.79
79.68
45.54
TARF
86.67
97.05
80.07
26.00
1.21
TARF
0.00
95.01
78.98
100.00
1.17
In addition, we also incorporate three more recent unlcarning methods into our comparison, c.g.. LAU (Shen al., 2024), SFR-on (Huang al., 2024), and SG (Di et al., 2025), in Table 6. These methods propose some advancements for dlasswwise unlearning in different aspects while not consid- cring the mismatched challenges. Both have been evaluated under the same mismatched unlearning scenarios introduced in our paper, using identical training budgets and evaluation protocols to ensure a fair comparison. The additional results also validate the effectiveness and generality of our TARF framework across all tasks, as the unified framework design enables target identification and sepa- ration. TARF maintains robust unlearning performance, due to its flexible capabilities of handling various mismatched unlearning scenarios.
19

---

Published as a conference paper at ICLR 2026
B.2 EVALUATION METRICS REGARDING DIFFERENT SCENARIOS
In this part, we summarize the following list and tables of the evaluation metrics (adopted from the previous work (Jia et al., 2023; Fan et al., 2023)) and the used labels in different unlearning scenarios,
Unlearning Accuracy (UA): the accuracy of the unlearned model 0g on the dataset of target concept D.. UA = [Di] _(..)) 1 (z) = y]
Retaining Accuracy (RA): the accuracy of the unlearned model 0g on retaining dataset Dr-
RA [2]] [(E:))ED. (z) = yl
Testing Accuracy (TA): the accuracy of the unlearned model 0g on test dataset Diest excluding the data belonging to the target concept. TA = |Dai ((ry)) D. 1 (z) = yl
Model Inversion Attack (MIA): the MIA success rate by confidence- hased MIA predictor of the model Ou on the dataset of target concept D. We follow (Jia et al., 2023) to implement non-training sample by
it to find how many samples in Dj can be correctly predicted the MIA predictor against Og First, we sample a balanced dataset from the retaining dataset D, and the test dataset excluding the forgetting data train the MIA predictor, then it is used to count the rate of true negative predictions for forgetting data of the target concept.
* Gap: 1 (IUAO- UAgun| |RAg - RAguun + (TAg- - TAgun + MMAA- - MIAg
Remark, We follow previous work (Jia et al., 2023) to adopt the TIME which report the wall-clock training time required to perform unleamning from the original model initialization. The termination of each unlearning methods defined by the recommended hyperparameter for training epochs (e.g.. early stopping 10 epochs for FT-style method as performance plateaus or fixed maximum epoch cap like 5 for GA-style method) on specific unlearning tasks.
Generally, in the evaluation phase, adopt the same labels used in pre-training to measure the unlearned model. Note that in the model mismatch forgetting, as the model is trained with superclass labels, the UA is also calculated using the superclass label. Hence, the UA of the Retrained reference is not equal to O as indicated in Table 3, and we compare the methods mainly on the averaged performance "Gap" (calculated based on the previous four metrics) to the Retrained reference.
Table 7: The label used in evaluation metrics on different forgetting scenarios.
Used Label
All matched
Target mismatch
Model mismatch
Data mismatch
UA
Class Label
Class Label
Superclass Label
Superclass Label
RA
Class Label
Class Label
Superclass Label
Superclass Label
TA
Class Label
Class Label
Superclass Label
Superclass Label
MIA
Class Label
Class Label
Superclass Label
Superclass Label
In Table 7, summarize the specific label used in different unlearning scenarios. To provide an intuitive example that corresponds to the instantiated unlearning tasks like Figure 1, we present Table 8 to give overall information about the data and labels considered in each metric.
Table 8: The evaluation data (label number) of different forgetting scenarios with CIFAR-100.
(classes number)
All matched
Target mismatch "boy". "gir"", "man". "woman" "baby" (5) of "bay" and "girl" but with superclass label
Model mismatch
mismatch
part of "people" (1), which is data
UA (D,)
"boy", 'girl' (2)
"people" (1)
other part of "poople" (1) with
RA (D,)
Other classes (98)
Other classes (95)
rest superclasses (19)
Other superclasses (19)
other part of "peoplo" with
TA (Dee)
Other classes (98)
Other classes (95)
the rest superclasses (19)
Other superclasses (19)
"boy", "girl", "man". "woman". "baby" (5) of "boy" and "giz" but with superclass label
part of "pcople" (1), which data
MIA (2))
"boy". "girl" (2)
"people" (1)
20

---

Published as a conference paper at ICLR 2026
B.3 FULL RESULTS OF FIGURE 2 AND FIGURE 3
All Matched Fargetting
Model Nismatch Forgetting
Target Mismatch Fongelting
Data Nismatch Fargetting
Indlig Eo6 Kodse eg toxa
Texley knnelie les
Accuracy
Accursy
Accunicy
Method
Method
Method
Method
Figure 8: Unlearning results across four tasks using different representative methods.
Rasrsentttton
0.30
Aug, Representation
0.25
120
0.23
50
0.22
Accurscy
2.32
Forgelting
Accuracy
Chuss-alhhes Data
2.12
Remahinn
N So.10
Forpetiing Data Sggne Daca
6.03
22
20.05
Betnaiing Data
0.00
0,00
Class Indexes
Epach
Class Indexes
Epoch
Figure 9: Forgetting dynamics on entangled (left) and under-entangled (right) feature representations.
21

---

Published as a conference paper at ICLR 2026
C PROOF OF THEOREM 3.2
Here we provide the complete proof of Theorem 3.2. The proof sketch is from Taylor expansion (Bot- tou et al., 2018) L,, (0*+1)/ 2,,, (0*+1), subtracting two formulas, and bounding the differences term. It intuitively reveals the data representation can affect the forgetting dynamic on the loss differences. As our primary goal is explore the new settings, we believe worth future work to establish more systematical analysis beyond the current gravity effects on the forgetting dynamics.
First, we can have the following assumption about the representation similarity of d+.(,,), which is also adopted and assumed in the various previous works (Belkin & Niyogi, 2003; Chen et al., 2020a).
Assumption C.1 (Representation Similarity). Let 81 and $2 be two disjoint subsets of a dataset D trained on model fo- Given the representation of an input at intermediate layer be h(x), the gradient differences at representation level can be controlled by assuming (n.()) is Lipschitz smooth with constant C₁, then have (x1) Vl, (222)l I Cellh(a,)) - -hx22)ll Ced
Theorem C.2 (gravity effects on unlearning update). Let 6 be the well-trained model parameters for unlearning, and we perform unlearning on 81 via a gradient ascent update, i.e., 0 0¹ V,, (0t)
for epoch t, then we can the following dynamics given ALs (0 (Lo,, (0(+1) - L (0 1).
ALer. (0(+1) S (L., Lez (0*)) + 1pmmxx (Jor (dd
VL, (0)) |1 + O17²))
(16)
where Amax (Jge) ) is the largest eigenvalue of the Jacobin matrix Jo- 8h(a), Note that when 1 - 0, 50
the RHS mainly relies on the term measuring representation similarity as (Le, (0') - L.g (0t)) - 0.
Proof. Let 8 and 82 be two disjoint subsets with associated empirical losses Laj (0) and Lag (0) for
model parameters 0. Suppose the representations (1)) of inputs xı 81 and T2 82 are similar:
Egg~St
[1))(1;)) - st xx222) S e.
Assume the loss (-) is Lipschitz smooth with constant CN, and Jor (z) (h) is the Jacobian of 09
the representation. Suppose update g++1 Ht A0t is applied (e.g., for unlearning). Then by Taylor expansion we have.
1 La, (0++)) -L0 L0,(0') V (0, 2 -0 -00) "y, 0¹)
then subtracting expansions for 51 and 82,
Leg (8++1) - Log (8 = (Ls, (0") Lsg (8') + (VLei (8') - (L)) Ag
+ 1T (Hs, Aar)
using the chain rule,
V VI (0) Exzas [Jo (e
then,
V L.,, (0') - V VLnn (8') = En,,, (xx) [Jo Vvl((z)) - (2)))
we can split this as,
Eny Jo ( Vx(((=)))))))))))))))))))))))))) - -10
then, by triangle inequality and operator norms,
V Tus Vlsgg S. (Jas (21)) Eayor [n((x)) - +((),
22

---

Published as a conference paper at ICLR 2026
assuming (x(+) is Lipschitz smooth with constant Ca.
[[1((11) Vex(22)11 - Cellh(x1) h(x2)1l Cedr (21,
so
VL,, Vo S Amax (z1)) (21,
and,
(VLs, (f') Le () APt Amax (I) (21, x22) L (8*)1l
combining everything and given A (0++1) = (b, - Log (8 )),
22 (f(+)) S (Ie,, (0*) [o)(( mmmaa (Jor (Ed,,a1) -
The loss difference between 81 and $2 after an unlearning update on 8 is bounded above by a term proportional to their representation similarity. This shows that representation similarity implies loss entanglement, where unlearning one set will influence the loss on another if they share similar representations.
23

---

Published as a conference paper at ICLR 2026
Classes
Forgetting Dala
Label Laver
11
*hay
"pirt"
Label Layer
Model Outgals
insadts
people
flowers
Target Concepts
Superdles
The left panel shows an example of two-layer label domains; The middle panel is the Venn diagram to show the hierarchical relation; The right panel illustrates the potentials of three critical class-wise unlcarning aspects.
Figure 10: Label domain mismatch with the two-layer illustration.
Table 9: Mismatching in the label domain of three critical aspects with two-layer label structure.
No.
Forgetting data
Model output
Target concept
Comment
1
Class label
Class label
Class label
All matched
2
Class label
Class label
Superclass
Target mismatch
3
Class label
Superclass
Class label
Model mismatch
4
Class label
Superclass
Superclass
Data mismatch
5
Superclass
Class label
Class label
Impractical since LD Cr
6
Superclass
Class label
Superclass
Similar to all matched Impractical since Lp Lr
7
Superclass
Superclass
Class label
8
Superclass
Superclass
Superclass
All matched
D FULL DISCUSSION ABOUT LABEL DOMAIN MISMATCH
In this section, we discuss the full scenarios of label domain mismatch class-wise unlearning (War- necke et al., 2023; Golatkar et al., 2020; Chen et al., 2023; Jia et al., 2023; Fan et al., 2023). Specifically, we will start by why focusing on class-wise unlearning, and then discuss the motivation for investigating its label domain mismatch, with the newly introduced setting being friendly for em- pirical analysis and further research. In addition, we provide detailed information on our instantiated four tasks using the benchmarked datasets (Krizhevsky, 2009). Finally, discuss the commonalities of mismatch forgetting scenarios and the general principle of unified framework design.
To begin with, machine unlearning (Cao & Yang, 2015; Thudi al., 2022b; Xu et al., 2023; Shaik et al., 2023) is originally proposed in response to "the right to be forgotten" to protect the data privacy, and recently deep machine unlearning a timely research topic associated with foundation models which use massive of data to train (Kurmanji et al., 2023; Bommasani et al., 2021). The ensuing data regulation concerns have also expanded the original privacy-protecting goal to more general needs and scenarios (Yao et al., 2023; Maini et al., 2024; Gandikota et al., 2023). As stated in (Shah et al., 2023; Kurmanji al., 2023: Jia al., 2023). unlearning a subset of the training set has received increasing attention (like removing sensitive information, and inappropriate content). However, the previous scenarios mainly consider the coinciding class labels with the target concept to unlearned. Although achieving promising results in forgetting, it is still not enough in practice.
Considering the problem setups of unlearning, we have three critical aspects, e.g., the well-trained machine learning model 0, and the reported data D; to be unlearned, well the target concept. In previous studies, the three aspects are mainly considered to be under the same label taxonomy. In other words, the unlearning tasks aligned with the pre-training task, where the latter trains a multi-class classification model, and the former aims to unlearn a training class. However, in practice, the unlearning request may violate the taxonomy of the pre-training tasks, while the
24

---

Published as a conference paper at ICLR 2026
specific target concepts always exhibit unified property for specific forgetting data. It naturally motivates us to consider different label domains of the three aspects of unlearning. As listed in Table 10, the label domain of data Lp. the label domain of model output LM, and the label domain of target concept Lr. To begin with, we introduce the relations between two label domains, i.., L1 matches L2 (L1 L2), L1 is the subclass domain of L2 (C1 < l2)² and L1 is the superclass domain of L2 (C1+62) and we have a practical assumption on the relation between label domains of forgetting data and target concept, i.e., LD 3 Lr, indicating that the reported forgetting data should be included in the target concept (as intuitively illustrated in the middle panel of Figure 10). Considering LD Lr, we can have two possibilities on L.. e.g., Las Cr and Lis A Lr. where the former is regarded as all matched when Ln = Lm LT and the latter the model mismatch. To be more specific, we consider model mismatch forgetting as LD = fT < Las, since LM < fT will have no additional effects on the unlearning when LD = Cr and we can regard as similar to the all matched case. Considering LD < Lr, we can have the target mismatch forgetting when LD = Lm and data mismatch forgetting when Lm Cr.
We summarize the mainly considered mismatch cases in Table 10, which can serve as a general reference for further research unlearning tasks. In the following, we further explain the procedure of task instantiating and dis- cuss the other potential scenarios with the typical two-layer label structure considered in the main text and an additional three-layer label structure.
Label Domain C
Relation of Data Co. Model Cu, and Target Cy:
All matched
Lo
=
E
Ey
constructing the
Target mismatch
Lu Cp
Cr
Model mismatch
Lo Py
Car
Data mismatch
Ap br
Eas
Table 10: considering label domain relations of three critical aspects in class-wis unlearning.
D.1 A TWO-LAYER LABEL STRUCTURE OF MISMATCII
In Figure 10, we first show the illustration of a two-layer label structure and the three aspects of unlearning, i.e., forgetting data, model outputs, and target concept. Without losing generality, we utilize the class labels and superclass information (refer to the official information in CIFAR- 100 (Krizhevsky, 2009)) for consideration. Then we have two-layer label structure representing different knowledge regions.
Given two potential label domains in each aspect, we can totally get the 8 scenarios list in Table 9. The first 4 scenarios are mainly considered and detailedly introduced in the main text. For the rest 4 scenarios (i.e., No. 5-8), we consider some (i.e., No. 5 and No. 7) to be impractical as the label domain of forgetting data is larger than the target concept, which means that the unlearning requests identify more forgetting data than the true target concept. It should be more reasonable that only limited forgetting data are identified by server users or internal examiner (Kovashka et al., 2016) in real-world applications. Therefore, we mainly consider the forgetting data Dj belongs a part of or equals to the overall data D, of the target concept. As for No. 6 and No. 8 cases, the former is similar to the conventional all matched forgetting since the forgetting data has the same label domains with the target concept while the model output has a fine ime-grained label domain (e.g., class label) that will not affect the unlearning, and the latter is exactly same as the all matched forgetting.
D.2 A THREE-LAYER LABEL STRUCTURE OF MISMATCH
Since in more extreme cases, some unlearning requests would exhibit only several instances of forgetting an abstract concept not aligned with the pre-training tasks. We then consider an extra label layer (c.g., the sub-set level inside class) to construct three-layer structure beyond the previous one. In Figure 11, we illustrate it with some samples and a Venn diagram.
Considering each aspect can have three potential label domains, we can totally get 27 scenarios in Table 11. In general, we have three rough categories for analysis. First, due to the aforementioned constraint that the target concept should include the forgetting data, we consider several cases (e.g..
2 Li x L2: For any label y E L1, there exist label y' E Le that instance being labeled with y can also heing labeled with y/, but not instance being labeled with y/ can be labeled with y.
3 "1 > Lz: For any label y L2, there exist label y' E Li that instance being labeled with y can also heing laheled with y/, but not all instance being labeled with y' can be laheled with y.
25

---

Published as a conference paper at ICLR 2026
Subbept of "boy"
Lagar
Foryetting Data
Labed Layer
Classos
II
Model Outputs
Label Layer
"bay"
"girl""
"baby"
"mann
"woman"
Target Concepts
insects
people
flowers
Supardass
The panel shows an example of three-layer label domains extended from ordinary setting considered our main text, where the sub-set is sampled from the "boy" class; The middle panel is the Venn diagram to show the hierarchical relation; The right panel illustrates the potentials of three critical class-wise unlcarning aspects.
Figure 11:
Label domain mismatch with the three-layer illustration.
Table 11: Mismatching in the label domain of three critical aspects with three-layer label structure.
No.
Forgetting data
Model output
Target concept
Comment
Sub-set
Sub-set
Sub-set
All matched
2
Sub-set
Sub-set
Class label
Target mismatch
3
Sub-set
Class label
Sub-set
Model mismatch
4
Sub-set
Class label
Class lahel
Data mismatch
5
Sub-set
Sub-set
Superclass
Different
6
Sub-set
Superclass
Sub-set
Different
7
Sub-set
Superclass
Superclass
Different
Sub-set
Class label
Superclass
Different
Sub-set
Superclass
Class label
Different
10
Class label
Class label
Class label
All matched
11
Class label
Class label
Superclass
Target mismatch
12
Class label
Superclass
Class label
Model mismatch
13
Class label
Superclass
Superclass
Data mismatch
14
Class label
Sub-set
Sub-set
Impractical since Lp Lr
15
Class label
Sub-set
Class lahel
Similar to all matched
16
Class label
Sub-set
Superclass
Different
17
Class label
Class label
Sub-set
Impractical since Lp LI Impractical since LD LI
Class label
Superclass
Sub-set
19
Superclass
Superclass
Superclass Class label
All matched
20
Superclass
Class label
Impractical since Lp > Lr Similar to all matched Impractical since Lp Cr Impracticul since Ln Lr Impractical since Lp Li Similar all matched Impractical since Lp > LT Impractical since LD > LT
21
Superclass
Superclass Class label
22
Superclass
Superclass
Class lahel
23
Superclass
Sub-set
Sub-set
24
Superclass
Sub-set
Class label
25
Superclass
Sub-set
Superclass
Superclass
Class label
Sub-set
Superclass
Superclass
Sub-set
No. 14, 17, 18, 20, 22-24, and 26-27) to impractical. Second, the three-layer structure also includes group of scenarios that also existed in the two-layer structure, No. 1-4 and 19 the same the four scenarios (i.c., all matched, target mismatch, model mismatch, and data mismatch). Third, for the rest scenarios, regarded them to be novel cases than those considered in the main text.
To be more specific, there are two groups of cases in the third part. For No. 5, 6, and 7, since they also can be represented using a two-layer structure, the forgetting dynamics target, model. and data mismatch forgetting. By contrast, in No. 8, 9, and 16, all three label domains exist the three aspects of dass-wise unlearning, which is worthy of further discussion.
similar to that in
26

---

Published as a conference paper at ICLR 2026
Class Representation
Class Representation
Class Representation
mismatch
mismatch
mismatch
No.
No.
No.
Class Representation
Class Representation
Class Representation
mismatch
mismatch
mismatch
No.
No. 9
Figure 12: Illustration of 6 scenarios different from the four major tasks according to Table 11.
D.3 FURTHER EXPLORATION ON THE OTHER 6 DIFFERENT SCENARIOS
In this part, further discuss the 6 different scenarios discovered by constructing the three-layer label structure. We illustrated these forgetting tasks in Figure 12 and discuss them as follows,
No. 5&16 In the two scenarios, the model output has the most fine-grane fine-grained label domain (e.g., sub-set as illustrated in Figure 11) for representation. At the same time, the target concept is broader than both model output and identified forgetting data. Different from the aforementioned target mismatch, the mismatch degree of this task is larger (e.g., superclass level) than the previous one (e.g.. class level). other words, the model output further loses the entanglement of feature representation of the samples belonging to target concept (compared with the original setups of target mismatch). To simulate the case, we employ the same model pre-trained by class in target mismatch, but enlarge the target concept (consists of 7 classes with similar semantic features, instead of the original 5) and change the forgetting data (2 class as the given forgetting data in No.5 and classes in No.
No. 8&7. Similar to the previous No. 5, the target concept in these tasks is also broader than the label domains the identified forgetting data. However, in these two scenarios, the model output varied which controls the entanglement of target samples. To construct these two forgetting tasks, respectively adopt the models pre-trained by class labels and superclass, and use the same forgetting data (with class) to investigate the performance change using TARF and other baselines.
No. 6&9. In the last two scenarios, the forgetting tasks are more similar to the previous model mismatch forgetting. However, the distinguishable difference is that the label domain of model outputs can be much different from the identified forgetting data. the No. task, the target concept aligned with the identified forgetting data, while since the remaining data is than the original
27

---

Published as a conference paper at ICLR 2026
Table 12: Results (%) of unlearning with different model structures. All methods are trained on the same backbone, i.e., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers are superior results. indicates smaller are better.
CHEAR-100
Metric
UA
RA
TA
MIA
Gapl
Metric
UA
RA
TA
MIA
Gapl
Retrained
0.00
100.00
0.00
97.85
73.72
100.00
FT (Wameoke et al., 2023)
67.52
96.43
72.96
41.14
32.72
53.11
94.64
71.23
52.70
26.53
RL (Tonova al.,
72.58
41.17
No.
53.90
53.56
25.48
GA (Ishida et ol., 2020)
38.03
97.00
70.98
76.92
16.75
32.24
95.73
69.99
77.62
15.12
TARF
0.00
96.58
72.03
100.00
0.74
0.00
96.98
72.87
100.00
0.43
Retrained
0.00
97.85
73.72
100.00
0.00
98,50
80.15
100.00
(Wamecke al., 2023)
74.09
97.19
36.71
95.16
94.98
46.77
RL (Toneva et al., 2018)
No. 8
76.04
96.76
72.88
36.00
35.49
No. 7
91.51
96.98
80.11
47.24
36.46
GA (bshida 2020)
72.94
77.96
15.91
80.27
93.82
5.59
TARF (ours)
0.00
96.22
72.43
100.00
0.73
0.00
96.54
79.23
100.00
0.65
Retrained
88.22
98.52
84.42
22.22
88.22
98.58
78.50
25.78
FT (Wamecke et al,, 2023)
94.33
95.00
78.77
13.67
5.96
91.78
95.02
78.90
18.44
3.72
RL (Toneva ot al., 2018)
No.
84,22
96.96
80.18
65.77
13.34
No.
96.97
70.22
80.24
94.67
26.94
GA (Ishida et al., 2020)
18.44
96.06
78,20
92.67
37.23
19.11
95.27
77.56
91. 91.56
34.79
TARP (ours)
92.21
82.32
19.17
89.12
97.23
79.21
24.32
model mismatch forgetting, the task separation could be harder than the previous. In the No. 9 task, we can find that is a complex scenario where the target concept is broader than the forgetting data but included in the same superclass. In both tasks, we use class data as the forgetting data.
To further understand the properties of unlearning these tasks, we conducted additional experiments and summarized the results in Table 12. We can find the empirical results well demonstrate the conceptual conjectures in the previous discussion, and the representative baselines exhibit varied performance gap with the Retrained reference. Among them, our TARF can consistently achieve the better performance regarding to the Gap.
D.4 SPECIFIC INFORMATION OF THE INSTANTIATED TASKS
For the four major scenarios (i.e., conventional all matched forgetting, target mismatch forgetting, model mismatch forgetting, and data mismatch forgetting) considered in our work, we provide the dataset construction and partition details in this section. Note that we focus on class-wise unlearning in this work, which is different from random data forgetting that uniformly samples the forgetting target of all classes in the training dataset.
Forgetting target. In previous works (Warnecke et al., 2023: Chen et al., 2023), the target concept to be forgotten is mainly considered as all matched where D, D1 yr has the same label domains (exactly same labels) with the pre-training task and forgetting data Dj D(y = }r In contrast, we assume that the target concept can be decoupled from the class label in practical unlearning requests. As illustrated in Figure 1, we further instantiate with three forgetting tasks given D3- = D(v = yr) with the superclass labels y" of y (classes): i) model mismatch forgetting, e.g., D. D(y 3} and yi S s/ where of E Y" given the model trained on : ii) target mismatch forgetting, e.g.. D. = Dly = ) given the model trained on V: iii) data mismatch forgetting, e.g., D, = D(y v/ given the model trained on y"
To ease the research investigation and empirical verification, we adopt the commonly used (Kurmanji al., 2023; Jia al., 2023; Fan et al., 2023: 2024) benchmark CIFAR-10 and CIFAR-100 for constructing the pre-training task for unlearning. Specifically, the official class labels are kept as classes for ordinary setup, and we provide the superclass information referring to the pre-defined lists (Krizhevsky, 2009) of CIFAR-I00 Since there is no official superclass information for CIFAR- 10 dataset, we manually grouped the classes of CIFAR-10 10 according to their semantic feature similarity and finalized 5 superclass clusters consisting of 2 classes in each. The full structured label layers information is summarized in Tables 14 and 15. For all the unlearning scenarios where the label domain of model output is the superclass, we will first use the superclass information to train the 20-class and 5-class classification models respectively. For the specific data partition in unlearning requests, randomly sampled two classes in CIFAR- 100 and one class in CIFAR-10 forgetting data and kept the setup across the four forgetting tasks as well as other experiments. For other additional experimental setups, will state them at the near positions.
28

---

Published as a conference paper at ICLR 2026
Table 13: Basic setup about unlearning scenarios. More illustrations can be found in Appendix D.4.
Datasel
Forgelling Data
Setup
All matched
Model mismaath
Target mismnatch
Data mismatch
CIFAR-I0
"automobile"
Training Class
10
Target Concept
"automobile"
"automobile"
"vehicle"
"vehicle"
CIFAR-100 100
"boy", "girl"
Training Class
100
20
100
20
Target Concept
"boy". "girl"
"boy", "girl"
"pcople"
"people"
Table 14:
Full list of the 20-class classification on CIFAR-100 with its official superclass la- bels (Krizhevsky, 2009).
Superclass (20)
Classes (5 for each superclass)
aquatic mammals
beaver, dolphin, otter, seal, whale aquarium fish, flatfish, ray, shark, trout orchids, poppies. roses, sunflowers, tulips bottles, bowls, cans, cups, plates apples, mushrooms, oranges, pears, sweet peppers clock, computer keyboard, lamp, telephone, television bed, chair, couch, table, wardrobe bee, beetle, butterfly, caterpillar, cockroach bear, leopard, lion, tiger, wolf bridge, castle, house, road, skyscraper cloud, forest, mountain, plain, sea camel, cattle, chimpanzee, elephant, kangaroo fox, porcupine, possum, raccoon, skunk crab, lobster, snail, spider, worm baby, boy, girl, man, woman crocodile, dinosaur, lizard, snake, turtle hamster, mouse, rabbit, shrew, squirrel maple, oak, palm, pine, willow bicycle, bus, motorcycle, pickup truck, train lawn-mower, rocket, streetcar, tank, tractor
fish
flowers
food containers
fruit and vegetables
household electrical devices household furniture insects
large carnivores
large man-made outdoor things large natural outdoor scenes large omnivores and herbivores
medium-sized mammals
non-insect invertebrates people
reptiles
small mammals
trees vehicles 1
vehicles
Table 15: Full list of the 5-class classifica- tion on CIFAR-10 with its manually set super- class (Krizhevsky, 2009).
Table 16: Specific training set data partition cor- responding to four major forgetting tasks.
Forgetting Tasks
Identified
Unidentified
Superclass (5) Classes (2 for each superclass)
All matched
Dr = D
Da, = D7
airplanc, bird sutomobile, truck
Target mismatch
D C
Dun Dr U
Model mismatch
Dj D.
D., = D,
cat, dog deer. frog horse, ship
Data mismatch
D,
Din Dr, U D.
5
D.5 DISCUSSION ON THE PRACTICALITY OF LABEL DOMAIN MISMATCH
Machine unlearning originally proposed in response to data regulations (Cao & Yang, 2015;
Hoofnagle et al., 2019), which are primarily motivated by desire to protect data owners' right to withdraw from the learning process. However, regarding its technical nature of mitigating the data influence from trained model (Xu al., 2023), unlearning is actually given broader significance in the context of trustworthy AI [4.5], like studies for mitigating bias and unfairness [6], addressing safety issues [7], erasing the NSFW generation [8,9]. It is worth noting that these trustworthy requirements may generally exhibit different concerns from the original training tasks. Motivated by the research problem raised in Section 3.1, our work focuses critical problem from the
29

---

Published as a conference paper at ICLR 2026
Classes
boy
gid
baby
Label Domain
Superclass
people
flowers
matched
(Convestional)
mismatch
mismatch
mismatch
P = LN Ey
b by < Lxt
bp Lps Ly
Pp < Lat by
All Matched Forgetting
Model Mismatch Porgetting
Target Mismatch F Forgelting
Data Mismalch F Forgetting
Taking the CIAR- 100 (Krizhevsky, 2009) dataset, instantiate four unlearning tasks given the same forgetting data with the class labels "boy" and "girl": a) all matched forgetring (comvnntional scenario): unleamn "boy" and "girl with the model trained on the clusses; sarget mismatch for geriing unlcam "people" with the model trained the classes; model mismatch forgetting: unlearn "boy" and "girl" with the model trained on the superclass; d) data mismatch forgetting: unleam "people" with the model triind on the superclass.
Figure 13: Illustrations of class representation with the four unlearning scenarios.
assumption view, i.e., the unlearning request may have a different taxonomy from the original tasks, for which model the three mismatched scenarios for systematical exploration.
Here, we discuss some practical use cases for the three newly proposed settings. For example, 1) the label domain mismatch may exist in some recommendation tasks (Ge et al., 2022) or other generative tasks like image generation (Gandikota et al., 2023) with diversified user feedback (for which we have presented a case study on concept-forge congett-foggetting on Stable Diffusion in Appendix E). 2) When the users raise unlearning requests for some representative disliked item with a message of "don't 't recommend this kind of thing", is similar to target mismatch forgetting. debugging the pre-trained model with some spurious correlation or safety concerns (Bommasani et al., 2021; Gandikota al., 2023; et al., 2024), is similar model or data mismatch forgetting as we only have the forgetting cases (e.g., some figures including NSFW content or adversarial features) that may not aligned with the taxonomy the pre-training task. We hope our exploration can provide insights for further consideration of specific practical applications.
addition, 3) when
D.6 DISCUSSION ON THE SCENARIO COMMONALITIES AND FRAMEWORK PRINCIPLES
The three mismatch scenarios, i.c., target mismatch forgetting, data mismatch forgetting, and model mismatch forgetting, share the common challenge of representation mismatch between the pre-trained model, the identified forgetting data, and the target concept to be forgotten. It breaks the assumption in all-matched scenarios that the three are matched (Shah et al., 2023; Xu et al., 2023; Fan et al., 2023; Jia et al., 2023) and can result in extra-/ine vitraa/ineffectivee forgetting in unlearning tasks, as demonstrated in our Figure 2. Specifically, in the target/data mismatch forgetting, the target concept can be wider than the identified forgetting data; while in the model mismatch forgetting, it can be smaller than the coarse coarse-grained model representation.
To build unified framework like our TARF, it requires considering the aforementioned two issues, ie.. insufficient representation in target/data mismatch, and decomposition lacking in the model mismatch. The former requires flexible controller for forgetting strength while and latter requires a simultaneous consideration on forgetting and retaining. Thus, based on the general equation in Eq. (3), we set two sub-objectives (annealed forgetting and target-aware retaining) to decompose the learned representation and control the forgetting strength by the instance- wise weighting mechanism which selects the targeted- -wwree forgetting data. Then, TARF becomes unified framework for the three mismatched scenarios. Note that although TARF is illustrated with three phases to better explain its functionality, while it is not three independent parts but unified in general objective.
30

---

Published as a conference paper at ICLR 2026
E ALGORITHM IMPLEMENTATION AND EXPLANATION
In this section, we present the pseudo-code of our proposed TARP and its variant, as well as additional discussions to enhance the understanding of our methods. Here we summarize the detailed procedure of algorithm implementation in Algorithm and Algorithm 2. In detail, Algorithm 1 identifies the potential target using the class labels, while Algorithm 2 can use the instance level information.
As introduced in Section 3.3, the objective of our TARF is defined as follows,
LTARF = =k(t)=
(- (-P Dri
Σ E(f()).))
+
1
Σ ((()).))
Pun
(m.D)~
(a.,)) Dua
(17)
Anneuled Forgetting Li(k)
Target--ware Retaining Ln(r)
where k(t) and 7 ((, y, f) are two training-time-reladd tele-relatedd hyperparameters to deal with the mismatch issues raised in our new settings, Specifically, we set a bearning-rate-reduee k(t) as,
k(t) = k -T-ttto)//
t [0,
(18)
where T indicates the total training time (e.g., epochs), and the value of k(t) decreases with the training process. On the other hand, we have the following indicator to measure the model prediction consistency with the training datal (I, y. 0) = (1, y) fo- (z, v)1, with which we set T(x, v, as follows.
So
coom (2r, y, 02)) > 3 or t < 4
"Unonnf. Retain,
T (5, y, ()
[1
Loon (z, y, 0.,) < 3 and t tz t
*Conf. Retain,
(19)
where t is a time stamp to control the start of pursuing the retaining part. The overall two dynamic hyperparameters can divide the whole unlearning process into three phases as illustrated in Figure 4.
Remark on B. The intuition of setting G is identifying those false remaining data in our Phase-l: Target Identification based on the grav- ity effects of forgetting dynamics. According to the dynamic information revealed by Phase-I (e.g.. before f1), B is set to thresholding those data most influential by unlearning the given forgetting data, so is a computable value given the pre-assumed forgetting range. For the task feasibility, we will generally assume the amount of false remaining data or classes is known at our target/data mismatch forgetting, following a similar setup in learning from label noise (Han et al., 2018) where the noise rate can be estimated and utilized as prior information. In the implementation. the 33 value is estimated with the ranked accuracy difference of each class, once time step & (e.g.. the end of Phase-I illustrated in Figure 4) and remain fixed afterward throughout training, as we already identify those potential false remaining data. Specifically, we estimate 3 by ranking samples in the remaining set. For example, setting 3 as the lowest Teon of top-5% data/class with the most loss/accuracy. In Table 17, we further check the performance robustness of quantile- based choice under varied false-retain size CIFAR-100. When the false- retain set is very small, these data remain reliably captured by the top quantiles due to their strong semantic and representation alignment with the forgetting set, and the unlearning performance is good. In contrast, when the false-retain set is significantly enlarged, the ranking could include a number of noisy samples as their semantic similar- ity to the forgetting set is inherently weak. Although we can not achieve accurate forgetting (e.g.. achieve 0% in UA), we can still perform better than plain gradient ascent on the given forgetting data. This situation also corresponds to the challenging settings where representation -rrvvity cues become ambiguous when the false-retain samples are disproportionately large and given forgetting data can not be representative anymore. Nevertheless, across all feasible mismatch scenarios introduced in the paper, the quantile-based ranking choice remains structurally robust, and quantile-based choice of B can be valid if the forgetting dynamics can well capture the semantic representativeness.
Table 17: Sensitive check of quantile-based choice on varied false-retain size using CIFAR-100.
Forgetting Support
Size
UA
RA
TA
MIA
Gap
Retrained (Ref.)
450
0.00
97.76
74,28
100.90
GA (large)
450
6.35
92.32
70.12
94,53
5,36
TARF (lage)
450
0.00
96.42
72.13
100,00
0.87
Retrained (Ref.)
2250
0.00
98.03
73.42
100.90
GA (Small)
2250
35.07
91. 81
66.39
75.91
18.10
TARE (Small)
2250
23.37
85.53
70.68
77.82
15.30
Annealed Forgetting. For the forgetting target, adopt the gradient ascent the given forgetting data to unlearn it. However, to approximate the retrained model, the intuition is not to pursue the maximization of the risk on this part of the data but destroy the learned feature the given model.
31

---

Published as a conference paper at ICLR 2026
So we introduce learning-rate- rate-reduced k(t) realize the annealed gradient ascent where to 1 is adopted for target or data mismatch forgetting, and the value of k(t) decreases with the training process. Resulting in destroyed features, gradient ascent on this part of data also constructs the dynamic information for differentiating the data of different consistency on its loss values, making the risks of the false retaining data higher than the rest, and helping to filter retaining data.
Target-aware Retaining. For the retaining part, we need to selectively learn the data from the remaining set, since the complementary dataset may be biased with unidentified forgetting data. Compared with other remaining data, the false retaining data is easy to be affected by similar feature representation as indicated in Figure 3. Thus, we can have F( y, t) where we can divide the remaining set into unconfidenttconfident parts to note the estimated retaining data like Figure 5. 11 = 2 is adopted at target and data mismatch tasks, and 3 can be estimated by the prior information about the specific unlearning request and the rank of loss values. By simultancously conducting gradient ascent on forgetting data and selective gradient descent on confident retaining data, we can better restrict the forgetting region and deconstruct the entangled feature representation (refer the middle of Figure 5 where we reveal the feature decomposition in deeper layers of model structure using ResNet). Finally, with the partial objective of retaining, can approximate the retrained reference (refer to the right of Figure 5).
E.1 DISCUSSION ON THE FUNCTIONALITY OF HYPERPARAMETER AND TUNING PRINCIPLE
Discussion on the computational stability. Conceptually, the hyperparameters introduced in TARF are structurally constrained by their functionalityy although they introduced extra tuning flexibility to enable the capability of handling different mismatched scenarios. We can understand from an induction view of our unlearning objective, where k and r respectively control the strength of forgetting and the scope of retaining. Generalized from the Phase-ll of target separation in Figure 4, I and lo enable the Phase-I for target identification on target/data mismatch and the Phase-Ill for retaining approximation. Note that as discussed, ,3 is an automatic ranking threshold in realizing the index т. Thus, we only need to decide the proper value of k, t1 and to. which is guided by specific unlearning scenarios. Given that intuition. we have several tuning principles: 1) the initial forgetting strength A can be tuned from a smaller value to avoid extra feature distortion; 2) 4 is generally set to be Epoch as dynamic information to can be also tuned by extending the Phase-IlI to fix the potential feature distortion induced by forgetting. Empirically, the above intuition and tuning principles benefit the computational stability of our framework, as demonstrated in our ablations which consistently shows that TARF is stable across a wide but reasonable range of choices. As shown in Figures and 17, sweeping the values of initial k. é1, and to leads to highly similar outcomes unless: 1) & is set to an unrealistically large value that aggressively destroys the representation; 2) ( is set to an extremely large value obscure large quota of retaining; 3) to is set to near O without considering fixing representation destroy. This is governed by the forgetting dynamics revealed in our theoretical analysis, where the early steps dominate the separation of target representations, while the later can induce unrelated feature distortion.
be captured by ranking mechanism; 3)
Practical guideline for hyperparameters. Based on the aforementioned conceptual discussion, we can synthesize the guideline as: 1) l should be initialized to small value and increased cautiously; a modest early ascent is sufficient to reveal representation gravity, while overly large values cause unnecessary distortion; 2) ( can generally be set very early (typically Epoch 1), since the gravity- based ranking relies on the first few steps of forgetting. 3) in can be tuned by extending Phase III when additional retaining approximation is needed, ensuring that any feature distortion is fixed.
E.2 DISCUSSION ON THE ALGORITIM COMPUTATION COST
We would acknowledge that TARF may require more time in unlearning compared with some methods like GA, which only uses the forgetting data (which sometimes can be extremely limited than other retaining data) for unlearning, while those methods may suffer from excessive forgetting and results in inaccurate unlearning across different scenarios. Regarding the metric "TIME", it originally means to avoid some methods that consume too much time compared with that of Retrained (Ref.). From this perspective, these current methods and TARF actually fall in the acceptable time range, and the efficiency gap between existing explorations in that range is indeed not a bottleneck based on Table 1.
32

---

Published as a conference paper at ICLR 2026
Algorithm 1 TARF
Training dataset = ( Yt, Ri) where s; indicates the identified forgetting dat, otherwise the data is recognized to be unlabeled for unlearning, learning rate 7) number of epochs T, batch size m, number of batches M, data x E X, label y E V. original trained model 0, loss function f, initialized indicator value 7 with the threshold B, time indicator to and t related to Eq. 5.
Input:
Output: model 8";
1: for mini-batch = ....., . do
2: Sample mini-batch {xx, (().)::)) y:))
)m from D
3: {L(z,, }:) < 9. P.fowward ( )m))>)))
4: Collect the initial training accuracy each class based on {((rs, v:))
5: T ++0 0
6: end for
T do 8: Update k(t) according to Eq. 5,
7: for epoch ...
9: if epoch = then
compute /3 in Eq. 5 according to the rank of class accuracy difference, and update т. 11: end if
10:
12:
for mini-batch = I..., .M do
13:
Sample mini-batch {3 Vi, s1));" from D
14:
Assign different weights for identified target samples and the rest retaining data,
15:
LTARF k(t) (-
[DNT Eiawya {((()).)) + 1 [Pinl E(en)) {(f(z), y) 7(2 y, t),
16:
0+0 V (D.T)
17:
end for
end for
From the methodology perspective, the three separately presented phases are integrated in unified framework, instead of adding extra phases before and after Phase II. Compared to other approximate unlearning methods, the unique operation is target identification by com- paring the output information of the unlearned model with the original model for weight assignment, which has similar or less computation than other advanced designs that consider sparse regularization (Jia et al., 2023) or compute the gradient mask for the original model (Fan et al., 2023). Empirically, we check the computation overhead of identification in Table 18. Specifically, we report the computation time (min) of identification (TIME- In) including the forwarding pass and ranking operation, compared with that (TIME-UD Un) of the whole unlcarning process. The resulting overhead is relatively limited compared with the unlcarning cost across the datasets. While extremely large-scale settings could benefit from optional structural priors or sampling- based accelerations, our implementation already shows favorable scalability.
Table 18: Computational overhead of target identi- fication compared with unlearning procedure.
Method
TIME-I
TIME
UA
RA
TA
MIA
Gap
CIEAR-10 CIFAR-100
4.23
1.25
4.85
0.21
Tiny ImageNet
32.81
1.37
ImageNett
11.52
628.87
0.00
69.93
71.79
100.00
3.97
Practical optimization strategies. Considering the future application scenario like foundation foundation-model. model- scale target identification, the single forward pass could also be costive for the entire remaining dataset for monitoring the dynamic information change. Here we further discuss potential optimization strategies. First, sampling- based approaches like uniform or stratified sampling are a viable strategy in large-scale applications, especially when the dataset contains structural information such as category tags or class labels. TARF depends only on relative changes in the gravity shift, rather than exact global estimates, so monitoring the shift using a small, blaaseell validation subset (e-g., a few thousand examples) is feasible. The primary trade-off is representational fidelity: if the sampled subset fails to capture the intra-class structural variation, the estimated gravity shift may introduce mild variance. Second, a lightweight surrogate model may be used to approximate gravity monitoring. Since TARF relies on representation gravity for target identification, we may also compute the gravity shift using compressed encoder if available (e.g., distilled representation model low-rank projection of the backbone). This reduces monitoring costs by optimizing the forwarding pass, with the trade-off that the surrogate model may introduce slight bias in the absolute magnitude of the
33

---

Published as a conference paper at ICLR 2026
Algorithm 2 TARF-I: generalized version on instance-wise identification
Input:
Training dataset = {z yi, 8: {=) where s; = =1 indicates the identified forgetting dat, otherwise the data is recognized to be unlabeled for unlearning, learning rate 7), number of epochs T, batch size m, number of batches M, data x E X. label y E V original trained model 8, loss function f, initialized indicator value with the threshold B, time indicator to and t related to Eq. 5.
Output: model 8";
1: for mini-batch = ....., . do
2: Sample mini-batch {( y:)) __, from D
3: {L(z,, yr) Hi) < 9. forward (Iz 34) (())
4: Collect the initial loss values in each training samples based on {f(r; v:)) T +-
5: end for
6: for epoch = ..... T do
7: Update k(t) according to Eq.
8: if epoch t1 then
9: compute 33 in Eq. 5 according to the rank of difference in instance loss values, and update
10: end if
for mini-batch I..., .M do
12:
Sample mini-batch { yi, s:)] from D
Assign different weights for identified target samples and the rest retaining data,
14:
LTARF k(t)-
pDi E(RNWD, (f(())))) + P... E(Ea)~D, {(f(x, g(x,8,t),
15:
O 0 - VolLAR (D, (DfT)
end for
17:
end for
gravity shift. We may also need consider the time cost of constructing such surrogate model based on the existing literature. could also be a promising future direction for efficient target identification.
E.3 CASE STUDY FOR UNLEARNING GENERATION CONCEPT
To demonstrate the compatibility, we also extend the idea of this work and investigate the performance of TARF on the specific text-to-image generation task with stable diffusion (Gandikota et al., 2023; Fan et al., 2023), and presented the generated images by the original model and unlearned model in Tables 19 and 20.
In detail, we aim unlearn the image generation of a concept with its specific prompt like "a photo of tench". To simulate the practical unlearning request (e.g.. the user raises the request of unlearning specific concept with some identified generation examples, and the developer needs to adjust the model to forget the concept), we construct the given dataset consisting of limited forgetting data and the unidentified remaining data for unlearning, which corresponds to the data mismatch forgetting task. Then we compare the image generation with certain label (CL) mismatching (Fan et al., 2023), and that with our TARF. Note that here we recognize ESD (Gandikota et al., 2023) performance upper bound and do not compare it, since is the same for all matched settings with fully identified forgetting data (as directly encourages the model unlearn the concept from text semanties). For this exploration of TARF, adopt the instance- wise identification during the forgetting process as described in Algorithm 2, to unlearn the target concept with the given limited forgetting data and pursue retaining the selected remaining data with lower loss values.
the original stable diffusion, the unlearned model
The results in Tables 19 and 20 demonstrate that our TARF can achieve better forgetting results given the limited identified forgetting data, with proper target identification in the remaining set, while CL using only identified forgetting data can not unlearn the concept well as the generated examples still maintain some semantic features belongs to the target concept (like "tench" or "English springer").
E.4 DISCUSSION ON TARF WITH LIMITED CLASS INFORMATION
The phase of TARF is for target identification in the target mismatch forgetting where the target concept is wider than the given forgetting data (e.g., forgetting "people" given "boy" and "girl"). The
34

---

Published as a conference paper at ICLR 2026
class information may affect the accurate identification of the target concept but the rationality of our framework. In our experimental setup, the class information is available TARF as the class labels used in pre-training, while the information of the target concept is given by the number of extra classes instead of the superclass label. Regarding the unavailable or implicit class information, first, if the class (i.e., the subclasses w.r.t. target concept) is not available, TARF may also utilize the model prediction to obtain the pseudo labels to conduct the task; Second. if the extra forgetting target beyond the identified data well represent the target concept (e.g., the false retaining data should be easier affected than the other retaining data). We acknowledge that both scenarios would lead larger performance gap with Retrained reference, it is a generally more challenging scenario affecting the task achievability to all of the approximate unlearning methods. We believe it worth future effort to explore.
restricted classes, may require that given forgetting data can
E.5 DISCUSSION ON THE ASSUMPTION OF SUPER-/SUB-CLAS INFORMATION
Though provide the full hierarchical label structure in benchmarks like CIFAR- 1OR-11/IIAARRIII 100 to enable controllable experiments for research purposes. does assume some available structure or proxy signal to distinguish between the forgetting target and the retained knowledge. In practice, this can be: 1) class labels from user requests (c.g.. "please unlearn boy/girl but not man/woman")) "); 2) semantic similarity (e.g., via pretrained embeddings or clustering representation space); 3) model behaviors (e.g., gradients or output confidence shifts, used in our target-aware selection mechanism). Thus, our method is compatible with approximate or user-defined taxonomies long as the information can reflect the representation similarity, and does not strictly require canonical super/sub-class structural information.
35

---

Published as a conference paper at ICLR 2026
Table 19: Image generation results of unlearned Stable Diffusion in the Data mismatch forgetting, compared with the original stable diffusion, certain label (CL) unlearning (Fan et al., 2023), and our TARF. The specific prompt used in the image generation is "a photo of tench".
Original Stable Diflusion
Unlearned
Unlearned by TARF
by CL (Fan et. 2023)
36

---

Published as a conference paper at ICLR 2026
Table 20: Image generation results of unlearned Stable Diffusion in the Data mismatch forgetting, compared with the original stable diffusion, certain label (CL) unlearning (Fan et al., 2023), and our TARF. The specific prompt used in the image generation is "a photo of English springer".
Original Stable Diflusion
Unlearned
Unlearned by TARF
by CL (Fan al. 2023)
37

---

Published as a conference paper at ICLR 2026
F ADDITIONAL EXPERIMENTAL RESULTS
In this section, we provide additional experimental results of our work.
In Appendix F.I, we summarize the additional experimental setups.
In Appendix F.2, we discuss the crucial target identification in unlearning.
In Appendix F.3, we discuss and compare TARF with the advanced method in matched scenario.
In Appendix F.4, we discuss potential ways to extend unlearning to the scenario without class labels.
In Appendix F.5, we verify unlearning on large-scale datasets trained with large models.
In Appendix F.6, we present unlearning with different model structures.
In Appendix F.7, we present the full results under multiple runs with the four forgetting tasks.
In Appendix F.8, we present the real-world unlearning application with LLM.
In Appendix F.9, discuss forgetting multiple class using TARF.
F.1 EXTRA EXPERIMENTAL SETUPS
We introduce additional experimental details the specific unlearning tasks. In our TARF, In general, set 1 for all the target identification parts, and adopt 0.04, to 2 in model mismatch forgetting, and k 0.02, to = 2 for all matched, target mismatch and data mismatch forgetting in the unlearning request on CIFAR-10 10 classification task; for the CIFAR-100 classification task, adopt & = 0.5, to 2 in model mismatch forgetting, and k = 0.05, to 2 for all matched, target mismatch and data mismatch forgetting. For the other hyperparameters, we follow the previous works (Jia et al., 2023; Kurmanji et al., 2023; Fan et al., 2023) to set the specific values. All the forgetting trails use 10 epochs for the total unlearning process except for GA (use 5 epochs) and IU (use the specific fixed step for optimization). The specific parameters and the pre-trained models (unlearn base) are provided in our source codes.
F.2 DISCUSSION ABOUT TARGET IDENTIFICATION UNLEARNING
In this part, we further discuss the important factors for the achievability of the unlearning tasks. To be more specific, for the target or data mismatch forgetting, the scenario assumes that the identified forgetting data is part of the whole samples belonging the target concept, which means there are other forgetting data included in the remaining set that need to be found. Thus, target identification is important for effective unlearning. As demonstrated in Section 3.2, the representation gravity can be a useful clue in forgetting dynamics to identify the other false retaining data. An implicit assumption is that those false retaining data have similar semantic features to the initially provided forgetting data, which has smaller representation distance than the retaining part of data as illustrated in Figure 14. Empirically, the model can have similar prediction changes on those false retaining data with the initial forgetting data. However, not all of the superclasses officially defined for the CIFAR- 100 dataset are suitable for constructing the unlearning request, as some superclasses are not semantically separable like "aquatic mammals" and "fish". : It can be found in Figure 16, where we check the 10p-10 classes with the most accuracy changes after gradient ascent for each superclass in the CIFAR-100 dataset, some false retaining data (class-level indicated by blue arrows) are not casily identified given the two initially provided forgetting data classes (indicated by red arrows). One interesting future problem can be how to handle the spurious correlation given the insufficient representative samples.
F.3 DISCUSSION ABOUT TARF ON ALL MATCHED SCENARIO
Regarding the all matched scenario, there is need for the target identification part to identify extra forgetting data in the all-matched scenario as the target concept matches the forgetting data, then TARF degenerates into general framework using the given forgetting data forget, and the rest to retain. The performance of TARF is comparable to the existing best counterpart like SCRUB regarding the "Gap."" in Table 3. It can be found that the overall performance of the unlearned models
38

---

Published as a conference paper at ICLR 2026
4og Bet of Representtt
Forgeling Data Cancaptia iined Data Remarting Data
0.22
0.13
Class Indexes
(a) Inter-classes distance in the model trained by classes
Sun. Diee of Begresenatto
0.25
0.22
0.
0.02
Class Indexes
(b) Inter-superelass distance in the model trained by superclass
The distance is calculated at the feature representation extracted from the penultimate layer of the model for each class, which measures the averaged Euclidean distance to the cluster center (averaged by the forgetting data).
Figure 14: Inter-class distance and Inter-superclass distance for the unlearning assumption.
40
20
20
-20
Datuiiing Data
Desaring Date
Saterring
Datsrring
Our igried happaiing Forgering Data
Lonnsp hargading Forgetting
Chaoselty Haturing
Classel sgral Hetainin g
-40
Torgetiing
Pre-trod Pre-treined
Retrained
Pre traitrd
Retrained
60
40
20
20
-20
-20
-20
Raturiing capt- agnoo hörpetiin
Beaaiigg Date
Raturring Data
Forgeting Date
Concep- Concep-akgned forgeting
Forgeting Date
-40
Class red Retae
Fo Rargetting Dats
Class ginedd Ratairing
Rorgettin Data
22
-20
20
40
40
20
20
20
We present the ISNE visualization (Van der Maaten Hinton, 2008) of learned features, using two representative unlearning methods, i.. finetune (FT) (Warnecke et al., 2023) and gradient ascent (GA) (Thudi et al., 2022a) with the pre-trained and retrained ones.
Figure 15: The entanglel/under-entanglee feature representations visualized by tSNE.
has already closely approximated the Retrained reference. Furthermore, since TARF is a general framework, we can also adopt the KI. divergence loss with the original model as designed in SCRUB to further improve the performance, for which we present the comparison in Table 21.
F.4 DISCUSSION ABOUT TARF ON WEURLV-SUPERVISED SCENARIO
Our current work mainly focus expanding the scope of conventional class wise unlearning. Regarding the existing approximate unlearning studies (Shah et al., 2023; Kurmanji et al., 2023; Chen al., 2023; Jia al., 2023), considering the all matched forgetting scenario with full supervision,
39

---

Published as a conference paper at ICLR 2026
(a) aquatic mammals
(b) fish
(c) flowers
(d) food containers
15
Dlase Folce
Cliss
(e) fruit and vegetables
electrical devices
(g) household furniture
(h) insects
(i) large carnivores
(j) large man-made out-(k) large natural outdoor (l) large omnivores and door things
scenes
herbivores
Dlans moke
Caa hdus
Caa Indec
Oba rdee
(m) medium-sized mam-(n) non-insect inverte- mals
(o) people
(p) reptiles
brates
32
Frbcx
Indox
(q) small mammals
(r) trees
vehicles
(1) vehicles 2
Target identification results with different unlearning requests and the minimum identified forgetting data on the CIFAR-100 dataset. Note that some target concepts not successfully identified by the identified data.
Figure 16: Task Identification using the CIFAR-100 dataset for target mismatch forgetting.
push towards more practical settings via decoupling the class labels and the target concept. For machine unlearning under weak supervision, there are limited studies (Shah et al., 2023) to our best knowledge, and we believe it is worth an in-depth exploration in future work.
Given that if model is trained with semi-supervised or other weak supervision, we can obtain the pseudo labels by the model prediction for its unlearning phase. Instead of using the predicted label, we can also utilize the distillation objective encourage the unlearned model's output to be far away from (or close to) the original ones. With the guide of model prediction, the data belonging to the same superclass with the forgetting data be figured out constrain the unlearning target. In Table 22, we present the results of our methods when only the given forgetting data are labeled, demonstrating framework can be extended achieve satisfactory performance.
40

---

Published as a conference paper at ICLR 2026
Table 21: Performance comparison in the all matched scenario when TARF with CE loss/KL divergence (refer to Eq. (14) with the original model for the retaining part.
Byp/
Datrset
CIFAR-10
CIFAR-100
Mathod/ Metrics
UIA
RA
TA
MIA
Clap)
TIMR
UIA
R.
TA
MIA
Clip.
TIMF.
Retmined (Ref.) PT [Wareeka et al., 2023)
0.00
09.51
94.59
100100
43.3
0.00
97.85
76.03
100.00
43.2
1,07
98.62
92.36
100100
1.07
4.43
0.67
96.32
72.34
100.00
1.47
3,02
SCRUB (Kurmanjj et 2023)
0,00
99.94
91.00
100100
1.03
2.88
0.00
99.95
76.75
100.00
0.71
3.25
Semi-superiisd
Scenurtos
(with (with KI.)
0.00
0.00
93.33
10000
100
4.33
4.21
0.00
0.00
72.53
74.98
4,80
4.68
Table 22: A case study (%) on the unlearning on CIFAR-100 100 under the weakly-suuprrvised scenario (e.g., using the pseudo-label generated by model prediction to handle unlabeled retaining data).
Type/
Duisset
Model mismatch
Duta mismatcch
Method/ Metries
UAA
RA
TA
MIA
Gapl.
TIMEL
LA
RA
TA
MIA
Gupl.
TIMEI
50
35.7%
98.
FT
92.67
4.58
79.77
17.15
4.93
RL.
80.11
95.83
79.83
99.00
59.78
96.82
79.90
30.49
Semi--mperrised Scenarios
2.43
BS
18.11
98
6.76
(full
86.67
95
78
TARF (unlaboted
90.32
96 58
Table 23: Results (%). Comparison with the baselines on TinylmageNet trained on ResNet101. (More results on large-scale dataset like ImageNet can refer to Appendix F.5)
Type/2 D
Dataset
matched
Model mismatch
Gapl.
TIME
Gapl
TIMEI
Retrained (Ret.)
0.00
74.32
63.13
100.00
217.0
34,80
71.25
64:29
65.30
256,
FT (Wimocke et 2023)
3.80
77.66
62.98
97.30
2.50
50.41
59. 30
77.
62:92
35.00
15.19
37.44
RL (Toneva et 2018)
73.20
69.87
60:19
18.40
40.47
225.13
84.10
68.53
60.63
8.00
28.61
226.7
GA (Ishida et al., 2020)
5.20
63.26
57.09
87.50
8,8.
0.34
6.30
63.17
58.04
90.70
16.66
0.34
BS (Chen al. 2023)
0.30
43.96
40.23
97.70
13.97
1.2
0.10
33.94
31.82
99.10
34.17
0.62
(Ja 2023)
3,70
76.63
62.55
97.50
2.28
40.79
59.40
76.30
62.80
14.81
37.05
SCRUB (Kunmanji
100.00
66.09
37.70
73.89
(ours)
75.47
100.00
74.
Datasel
Targel matched
Dala mismatch
ImageNet
Method / Metrics
UA
R.A
TA
MIA
Gapl.
TIME
UA
RA
TA
MLA
Gapl
TIMEI
Retsined (Ref.)
0.00
72.83
65.12
100.00
213.05
0.00
71:37
65.76
100.00
252.62
FT (Wamecke 2023)
29.67
75.94
62.97
69.30
16.41
30.41
64.33
75.45
62.96
30.60
35.15
37.41
(Toneva 2020)
68.93
69.97
22.00
225.13
84.27
68.64
60.59
46.08
226.79
57.26
7.33
(Chen al., 2023)
1.00
0.00
Ly-apune (Ji 2023)
28.93
75.18
62.55
69.
16.06
40.79
63.90
74
62.80
31.30
34.75
37.05
SCKUB (Kurmanji et al., 2023)
25.67
75.51
63.85
73.80
13.90
66.09
44.07
74.02
64.25
46.93
25:33
58.53
TARF (ours)
5.07
75.78
62.72
97.53
3.22
32.81
0.00
74,85
62.59
100.00
1,66
37.92
F.5 FORGETTING IN THE LARGE- DATASET
In this part, we present more experiments conducted on large-scale dataset like ImageNet- 1k in Table 24, and also unlearning multiple classes the large-scal scale datasets in Table 25.
F.6 FORGETTING WITH DIFFERENT MODEL STRUCTURES
In this part, we further check the unlearning performance of our TARF on different pre- mr-itraeee model structures compared with several baselines. We choose CIFAR-100 as the pre-training classification task and conduct all matched forgetting and model mismatch forgetting. The results are summarized in Table 26. The results validate that our TARF can robustly achieve better unlearning performance across different model structures.
F.7 FULL RESULTS WITH DIFFERENT FORGETTING TASKS
In this section, we provide the full results of Table 3, which is conducted by setting different random seeds (for multiple runs) with the original trails and reported as the mean and std values for each evaluation metric. Tables 27 to 30 presents the performance of unlearning on CIFAR-10 10, and Tables 31 to 34 presents the performance of unlearning CIFAR-100. The performance comparison of our
41

---

Published as a conference paper at ICLR 2026
Table 24: Results (%). Comparison with the unlearning baselines on ImageNet-Ik: All matched forgetting: unlearn 1 class; Target mismatch forgetting: unlearn three classes belonging to "fish".
Type/
Diraser
All matched
Target mismatch
Methodl / Matris
UA
RA
TA
MIA
Goypl
TIMEL
C:A
RA
TA
MIA
(Tspl
TTMF]
Retrainel
0:00
79.77
77.64
100.00
7075.48
0.00
80.09
77:54
100:00
7727 54
FT (Wamocke at al. 2023)
8.00
70.15
71.95
100.00
3.82
608.
0.79
70.26
72.07
180.00
4.02
608.62
(Toevv et al., 2018)
81.38
70.22
11.79
19:45
44.29
969.64
79.69
69.98
71.77
23.03
43.14
972.02
GA (libada et 2020)
0.00
66.25
67.35
100.00
5.95
8.75
0.00
31.21
37.74
0.00
47.17
17:38
(Chen et al., 2023)
8.00
11.15
36.33
100.00
22.48
9.03
0.00
21.57
27.56
09.97
27.13
23.75
E.-apare Oli et
8.00
67.98
70.70
100330
603.21
100100
5,00
SCRUB (Kumangi ot al. 2023)
29.77
74.92
75.66
$1.77
13:71
065.42
75,60
82.77
11.71
TARF (owrs)
8.00
70.53
72.25
100000
3.66
500:11
0.00
69:98
71.79
100100
3.97
628:87
Net-
Ditasct
Model matched
Data misumathh
Metrics
UA
MIA
Gapl
TIMEL
Gapl
TIMEL
Retrained
50.00
70.29
6501.27
(Wamecke 2018)
70.38
87.62
63.25
15.23
9.13
849.84
$8.21
63.81
48.15
956.11
(Balada 2020)
8.00
66.62
100.00
44.56
17.44
14:34
2023)
8.00
40.84
100.30
54.28
19.69
0.00
100100
el 2023)
62.00
67.94
62.58
19:15
7.29
1091.20
29
0.00
66.37
61.03
100.00
5.84
1071.4
SCRUB (Kumonnji et al., 20239
86
74.82
68.04
14.69
6.34
663.6
14. 18
74. 84
07.92
93.10
7.27
689.82
TARF (ours)
80.62
70.27
64.04
19.45
5,92
601.28
0.00
70.00
63.97
100.00
4.17
602.62
Table 25: Results (%). Comparison with the unlearning baselines on TinylmageNet-200 and ImageNet- Ik with more (10+) forgetting classes all matched forgetting scenarios.
Scenarios/ /D
Unleam request
Forget 10 classes in Inngeeet
forget classes Tiny-ImageNet
Method / Metiis
UA
RA
TA
MIA
Cap.
TIME,
UA
RA
TA
MIA
Capl
TIMEL
Retrained
71.00
100000
251.49
0.00
65.26
100.00
181.13
FT
70.63
1.35
27.10
2.79
60.36
97.38
3.71
GA
17.76
61.74
56.12
76.90
13.57
1.37
28.95
59.72
57.54
57.06
19.37
3.49
TARF (ours)
0.00
69.63
39.69
100.00
0.49
28.5
0.00
70.24
60.16
100.00
1.89
39.6
All matched Forgetting
Unleam request
forget 50 classes in Tiny- ImageeNet
Borget 10 classes in ImageNNt
Method Metrics
RA
Gap.
TIMEI
UA
TA
Gapl
TIMEL
Retrained
0.00
66:26
57.88
100.00
161.37
0.00
51.94
56.
100.00
917.66
FT
5.19
75.77
61.29
85.75
8.09
44.62
0.00
55.16
59.53
100.00
1.50
316.14
GA
22.92
44. 12
48.03
62.26
23.16
7.70
5.73
47.35
52.42
87.21
6,85
2.18
(ours)
71.68
100.00
46.97
0.00
55.83
100.00
0.54
TARF with other baseline across the four forgetting tasks (i.e., all matched, target mismatch, model mismatch, and data mismatch) demonstrated the general effectiveness of our algorithm framework.
F.8 APPLICATION ON MISMATCHED FORGETTING WITH LLM
In Table 5, we adapt our introduced four mismatch setting under the context of Large Language Models (LLMs), we conduct experiments on the TOFU (Maini et al., 2024) dataset for real-world application on removing learned authors information which are private and IP-related. We modify the original TOFU forget set to scenarios similarly previous construction on conventional benchmark: specifically, the all matched setting we will have all the forgetting data while the data mismatch setting we use 80% identified forgetting data. Since there is explicit concept in the context of TOFU target, we assume the amount of forget data implicitly represent the representation mismatch with original LLM (e.g., using forget01 to represent easy to cluster and forget to represent hard to cluster set for different unlearn difficulties). The model adopt is I.Lama-3.2 and the evaluation metries used is QA probability on forgetting and retaining set following (Maini et al., 2024). Given the complex LLM representation, previous representative methods like GA (Maini al., 2024) and NPO (Zhang et al., 2024) can easily make the model collapse to achieve low Prob on both forget and retain set. While our TARF can perform robustly to achieve forgetting with better retaining performance resistance, we also find there are some trade-off between forget and retain part, which indicates complex entangled representation on it.
F.9 FORGETTING MULTIPLE CLASS BY TARF
Our proposed TARF framework is applicable to forgetting multiple concepts superclasses simul- tancously, as we didn't 't restrict concept numbers in our algorithm design. Conceptually, the core challenges of mismathced unlearning still lie on insufficient representaiton decomposition lacking
42

---

Published as a conference paper at ICLR 2026
Table 26: Results (%) of unlearning with different model structure. All methods are trained on the same backbone, i.e., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers are superior results. indicates smaller are better.
Task
All matched
Model mismatch
CIFAR-100
Metric
UA
RA
TA
MIA
Gapl
RA
TA
MIA
Gapl
Retrained
82.12
FT (Wamecke et al, 2023)
0.00
90,92
66.86
100.00
3.15
95.22
95.17
71.71
7.56
6.89
VGG-19
(Toneva et al., 2018)
0.00
90.29
66.16
100.00
3.48
96.22
95.26
77.71
98.56
23.71
GA (Ishida al., 2020)
0.00
79.27
56,03
100.00
8.77
0.00
93.09
74.30
100.00
45.13
TARF (ours)
0.00
91,96
67.94
100.00
2.62
82.67
93.71
76.24
24.22
4.87
Retrained
0.00
97.85
76.03
100.00
88.22
98.58
78.50
25.78
(Wamecke al., 2023)
71,97
ResNet-18
R1. (Toneva et al., 2018)
0.11
95.90
71.57
100.00
1.63
94.11
96.70
80. 17
89
20. 14
(Ishida al., 2020)
1.89
95.26
69.14
99.89
2.87
9.33
95.13
77.22
96.89
38.68
TARF (ours)
0.00
96.90
71.51
100.00
1.37
86.00
96.54
74.20
22.78
2.89
Retrained
0.00
97.71
76.95
100.00
88.11
98.37
83.61
23.56
FT (Wameeke et al., 2023)
0.67
96.61
71.29
100.00
1.86
97.44
95.70
78.70
7.33
8.29
WideResNet
(Toneva al., 2018)
0.00
78.26
GA (Ishida et al., 2020)
0.44
91.49
66.29
100.00
2.26
4,33
91.76
75.18
99.11
43.71
TARF (ours)
0.00
96.51
71.77
100.00
1,60
88.00
95.50
79.06
22.67
2.11
Table 27: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers superior results. indicates smaller are better.
TA
Gap
CIFAR-10
Method
std
std
sid
std
Retrained
0.00
99.51
94.69
100,00
FT (Wameeke et al., 2023)
4.66
3.59
98.58
0.04
92.42
0.06
100.00
0.00
1.96
0.89
RL (Toneva al., 2018)
2.23
1.90
98.30
0.65
91.97
0.74
100.00
0.00
1.54
0.82
GA (Ishida el al., 2020)
0.34
0.16
95.48
0.24
88.52
0.35
99.88
0.10
2.67
0.21
All matched
IU (Izza et al., 2021)
0.11
0.05
72.50
15.65
68.28
14,10
99.98
0.02
13.39
7.41
BS (Chen et al., 2023)
24.72
0.32
88.91
0.97
81.84 84
0.94
89 23
0.56
14.74
0.70
I. sparse (Jia al., 2023)
0.00
0.00
94.18
0.03
90.01
0.24
100.00
0.00
2.50
0.05
SalUn (Fan et al., 2023)
0.48
0.46
88.66
2.67
84.48
2.40
100.00
0.00
5.39
1.38
SCRUB (Kurmanji al., 2023)
1.23
99.92
91.25
0.56
100.00
0.00
1.28
0.23
TARF (ours)
0.00
0.00
98.22
0.02
92.09
0.14
100.00
0.00
0.97
0.03
as revealed in our Section 3.2. For the model mismatch setting, it doesn't introduce extra algorithmic diuculty with multiple target concepts, as our objective simultaneously considers gradient ascent and descent to deconstruct the entangled representation. For the target/data mismatch setting, we can also identify those multiple target concepts by respectively utilizing our Phase-I: Target Identification with the base model, given the forgetting data as a representative support set for each concept.
To present the multiplessuerrllas forgetting, we also included new experiments in Table 36, where we conduct unlearning in the target mis- match and model mismatch settings using the CIFAR-I00 dataset for two target concepts for- getting, c.g., "pcople" and "aquatic mammals". As result, TARF demonstrates consistent per- formance effectiveness across all scenarios with- out harming the retention on retaining data, indi- cating its scalability to multi-concept forgetting.
Table 36: Forgetting Multiple Superclass by TARF using CIFAR-100.
Method
UA
RA
TA
MIA
Gap
Retrained (Ref.)
0.00
97.23
71.85
100.00
FT RL GA L Li-sparse
45.68
94.65
71.16
58.87
22.52
49.91
96.12
72.14
55.96
23.84
18.07
92.62
67.75
91.29
8.88
43.40
87,68
68.90
60.31
23,90
SCRUB
50.55
99.61
78.91
29. 12
24.77
TARF
1.08
94.78
69.91
100.00
1.37
G BROADER IMPACT
In this work, we explore the label domain mismatch in class-wise unlearning, which aims to enhance the flexibility of data regulation with the increasing concern about the trustworthiness of machine
43

---

Published as a conference paper at ICLR 2026
Table 28: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.e,, the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers are superior results. indicates smaller are better.
Metric
UA
RA
TA
MIA
Gap!
CIFAR-10
Method
mcan
std
mcan
std
mcan
std
meun
std
mean
std
Retrained
87.76
99.58
95.91
20.57
(Wamecke
2023)
0.11
93.77
0..1
0.86
5.06
0.27
RL (Toneva et al., 2018)
48.25
5.43
98.01
0.12
93.03
0.21
98.10
0.64
30.37
1.53
GA (Ishida et al., 2020)
6,49
0.73
86,91
0.08
82.03
0.18
94.39
0.59
45.41
0.27
Model mismatch
(Izzo al., 2021)
15.84
7.86
85.89
1.45
81.08
1.49
93.58
3.71
43.36
3.62
BS (Chen et al,, 2023)
14.05
3.76
53,28
2.51
51.25
1.86
94,90
1.06
59.75
2.29
La sp-spa (Jia et al., 2023)
92.25
0.87
95.01
0.25
91.67
0.04
17.40
2.86
4.14
1.00
SalUn (Fan
2023)
16.31
7,40
92.91
1,05
86.50
2.12
99.24
0.09
41.55
2.14
SCRUB (Kurmanji et al., 2023)
93,21
1.17
99.83
0.13
93.29
0.81
14.24
0.87
3.65
0.18
TARF (ours)
89,91
1.20
97.73
0.24
92.66
0.17
20.36
2.54
2.45
0.46
Table 29: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.e,, the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers superior results. indicates smaller are better.
Metric
RA
TA
MIA
Gapl
CIFAR-10
Method
std
std
std
std
std
Retrained
0.00
99.38
93.85
100.00
FT (Warnocke et al., 2023)
52.23
1.80
98
0.05
91.74
0.09
50.59
0.15
26.18
0.40
(Toneva al., 2018)
50.63
0.62
98.21
0.65
91.51
0.61
53.88
2.36
0.12
GA (Ishida et al., 2020)
41.64
0.82
97.05
0.04
89.68
0.17
63.23
1.10
21.23
0.43
Target mismatch
al.,
45.32
17.82
65.67
2.76
55.98
2.76
BS (Chen al., 2023)
53.78
0.16
89.67
1.02
79.34
3.95
66.31
10.02
25.36
3.28
Li-sparse (Jia al., 2023)
49.55
0.08
93.57
0.05
89.06
0.23
51.33
0.09
27.21
0.05
SalUn (Fan et al., 2023)
47.85
1.22
87.84
3.25
83.38
2.94
58.10
2.85
27.40
1.10
SCRUB (Kurmanji et al., 2023)
48.53
1.02
99.43
0.21
91.66
0.28
51.27
0.73
24.92
0.51
TARF (ours)
0.05
0.02
97.65
0.08
91.28
0.47
100.00
0.00
1.09
0.14
learning. Pushing forward the practical usage of machine unlearning, our research provides broader consideration of real-world unlearning scenarios and offers significant positive social impacts. It can enhance data privacy protection by allowing individuals to effectively remove their data, ensuring some sensitive data not used for analysis. addition, unlearning can remove bias or discrimination by correcting flawed datasets, promoting the development of fairness or other ethical considerations. This feature also enables enterprises to adhere to data protection standards such as GDPR (Rosen, 201 and CCPA (Pardau, 2018), therefore promoting confidence among users. Our newly introduced unlearning setting, which decouples the class label and the target concept, is more general and discusses the achievability of various unlearning requests, which may often be different from the taxonomy of pre-training tasks.
Although take step forward in more practical class unlearning by considering the label domain mismatch scenarios, is not the end of this direction and there are still many problems to be addressed. Following the previous works (Warnecke et al., 2023: Golatkar al., 2020; Jia et al., 2023; Chen et al., 2023), our work mainly focuses on the class-wise unicarning with the classification model for the exploration, future efforts can also be paid in the unlearning problem of the emerging and powerful generative models. On the technical level, although those compared unlearning methods and our framework can achieve the forgetting target, all requires extra computational cost, and how to make it more efficient can be further studied.
44

---

Published as a conference paper at ICLR 2026
Table 30: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, ie.. the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers are superior results. indicates smaller are better.
Metric
UA
RA
TA
MIA
Gap!
CIFAR-10
Method
std
std
std
std
Retrained
0.00
99.53
95.56
100.00
FT (Wamecke et al., 2023)
96.85
0.06
98.62
0.13
93.47
0.21
6.93
0.45
48.23
0.18
(Toneva
2018)
73.62
52.04
31.55
GA (Ishida et al., 2020)
9.82
1.13
96.14
0.28
90.46
0.33
90.46
0.95
6.56
0.67
Data mismatch
(Izzo al., 2021)
15.19
7.66
94.80
0.70
89.08
0.46
92.83
4.26
8.39
2.69
BS (Chen et al., 2023)
16.72
0.02
61.01
0.21
53.81
4.05
93.47
1.24
25.88
1.27
Li Lp-sparse (Jia et al., 2023)
95.42
0.35
94.57
0.26
91.07
0.01
10.82
1.30
48.51
0.47
SaltIn (Fan al., 20233
55.52
60.23
27.12
2.37
SCRUB (Kummanji al., 2023)
97.06
0.52
99.16
0.23
94.72
0.56
9.98
0.43
46,98
0.21
TARF (ours)
0.00
0.00
98.35
0.18
93.42
0.34
100.00
0.00
0.83
0.13
Table 31: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.e., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages, Bold numbers are superior results. indicates smaller are better.
Metric
UA
RA
TA
MIA
Gapl
CIFAR-100
Method
mean
stdl
mean
std
mean
std
mein
std
mean
std
Retrained
0.00
97.85
76.03
100.00
(Wamecke 2023)
0.67
72.16
0.19
RL (Toneva al., 2018)
0.56
0.45
96.00
0.10
71.79
0.22
100.00
0.00
1.66
0.03
(Ishida al., 2020)
1.61
0.28
95.00
0.26
68.85
0.29
99.89
0.00
2.93
0.07
All matched
IU (Izzo al., 2021)
202))
0.00
0.00
39.80
2.19
31.09
1.51
100.00
0.00
25.75
0.93
BS (Chen al., 2023)
4.83
0.05
90.17
0.06
64.30
0.64
99:45
0.12
6,20
0.22
L, L--spa
2023)
0.00
0.00
71.35
1.27
0.46
SalUn (Fan et al., 2023)
0.00
0.00
77.00
1.66
63.06
0.92
100.00
0.00
8.46
0.64
SCRUB (Kurmanji et al., 2023)
0.00
0.00
99.72
0.26
76.69
0.06
100.00
0.00
0.64
0.08
TARF (ours)
0.00
72.40
Table 32: Main Results (%). Comparison with the unlearning baselines. All methods trained the same backbone, i.e., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages. Bold numbers superior results. indicates smaller are better.
CIFAR-100
Metrie
CA
RA
MIA
Gap!
Method
mean
mean
std
mean
mean
Retrained
88.22
98.58
78.50
25.78
FT (Wamecke et al., 2023)
95.45
2.78
95.91
0.89
79.74
0.40
11.56
4.78
6.34
1.77
RL (Tomeva et al., 2018)
87.11
7.00
96.27
0.44
80.00
0.17
97.95
1.06
20.75
0.61
GA (Ishida et al., 2020)
8.06
1.28
94.98
0.15
77.09
0.13
97.34
0.45
39.18
0.50
Model mismatch
IU (Izzo al., 2021)
39.95
5.28
97.22
0.39
79.71
0.63
83.28
3.17
27.08
2.05
2023)
0.03
74.96
fa-sparse (Jia et al., 2023)
91.11
5.00
94.28
0.18
77.61
0.39
15.56
4.45
5.84
1.69
SalUn (Fan al., 2023)
74.78
79.98
1.14
71.55
0.77
65.61
11.39
19.71
5.44
SCRUB (Kurmanji al., 2023)
92.45
2,80
99.44
0.78
78.75
1.75
20.13
4.56
4.14
1.15
TARF (ours)
84,78
1.90
97.19
0.14
80.02
0.15
28.89
2.89
2.37
1.15
45

---

Published as a conference paper at ICLR 2026
Table 33: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.e, the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages, Bold numbers superior results. indicates smaller are better.
CIFAR-100
Metric
UA
RA
MIA
Gap!
Method
std
std
sid
Retrained
0.00
97.85
73.72
100.00
FT (Wamecke ct al., 2023)
58.58
0.40
96.42
0.10
72.31
0.22
45.94
0.83
28.87
0.34
(Toneva
2018)
96.00
0.10
72.04
0.16
50.67
27.66
GA (Ishida et al., 2020)
22.07
0.69
96.87
0.24
70.52
0.30
90.45
0.23
8.95
0.10
Target mismatch
(Izzo al,, 2021)
30.80
2.25
31.00
1.42
63.83
42,03
0.91
BS (Chen
2023)
40.91
0.47
98.36
0.04
70.04
1.38
85,00
0.16
15.03
0.18
Li-sparse (Jia et al., 2023)
55.31
2.90
94.23
0.44
72.15
1.27
48.47
3.54
30.26
1.18
SalUn (Fan al., 2023)
43.29
77.15
1.63
63.30
0.93
64.63
1.34
27.45
0.10
SCRUB (Kurmanji al., 2023)
59.56
0.09
99.74
0.26
76.14
0.82
45.45
0.56
29.60
0.02
TARF (ours)
0.29
0.03
97.06
0.29
73.27
0.41
100.00
0.00
0,38
0.17
Table 34: Main Results (%). Comparison with the unlearning baselines. All methods are trained on the same backbone, i.e., the basis of unlearning initialization is the same (except for retraining from scratch). Values are percentages, Bold numbers are superior results. indicates smaller are better.
CIFAR-100
Metric
CA
RA
TA
MIA
Gapl
Method
mean
std
mean
stdl
mcca
std
meaan
std
meam
std
Retrained
0.00
98.50
10
100.00
(Wameeke 2023)
90.79
5.18
79.80
RL (Toneva ol., 2018)
93.60
3.82
96.32
0.39
79.92
0.02
65.20
5.56
32.73
2.24
GA (Ishida et 2020)
6.98
0.98
97.78
0.14
79.34
0.11
97.53
0.51
2.75
0.31
Data mismatch
IL (Izzo al., 2021)
37.22
5.71
99.17
0.21
80.01
1.81
85.41
2.41
13.54
2.08
(Chen al., 2023)
15.71
0.33
98.47
0.04
76.02
3.74
96.05
0.18
5.86
0.18
Li-sparse
2023)
89.02
4.67
0.20
4.36
2.20
SalUn (Fan el al., 2023)
79.00
6.07
79.92
1.05
71.55
0.51
44.18
9.96
39.42
3.62
SCRUB (Kurmanji al., 2023)
93.28
2.10
99.25
79.18
0.48
3.55
TARF (ours)
0.00
0.00
79.55
0.57
100.00
0.00
0.05
Table 35: Results (%). Comparison with the unlearning baselines varied difficulties target identification: to explore the effects of differentiating false retaining data with affected actual retaining data. Specifically, we change the given forgetting data classes to unlearn the target (the superclass "aquatic mammals" including sotter, www.w, wew.eerdollhhiin""" it is intuitive that less and biased given forgetting data ("beaver,dolphin") increase the difficulty of representing the whole target concept (" l"aquatic mammals") ",, and "lobster" can be mis-identified as potential forgetting data TARF. With well-represented given forgetting data (left), TARF can perform better; otherwise (right) the benefits upon best baseline decreased.
(Left) rell-repreeented for the concept
(Left Given siterrseal,wbaee
(Right) Given "heuver,dolphin" forget "aquatic mammals"
Type
(Right) binsed and mis-identily "kobster"
forget "aquatic mammals"
CIFAR-100
Method/ / Metrics
UA
RA
TA
MIA
Gapl
UA
RA
TA
MIA
Gapl
(Warnecke
72.26
(Toneva al.,
19.30
mismatch
(vuried)
GA (Ishida et al., 2020)
La-sparse tJin al., 2023)
SCRUB (Kurmanji et al., 2023)
28.77
39.46
12.76
80.72
88.59
99.22
65.80
65.27
75.81
72.5%
63.24
91.91
20.27
19.95
9.61
39.06
59.11
35.07
99.16
91.81
83.22
61.81
66. 39
77.14
66.93
46.09
75.91
29.54
23.13
18.10
TARF (ours)
0.00
97.05
100.00
1.18
23.37
85.53
70.68
77.82
15.20
46

---

Published as a conference paper at ICLR 2026
All Matched
Target Mismatch
Model Mismatch
Data Mismatch
110
200
200
100
Accurec
Accuracy
Accuracy
Accuracy
RA Ref
RA
0
0+
033
0.01
0.6
23
65
335
0.01
0.05
2-7
2.5
3.2
0.01
0.05
6.3
0.5
525
6.07
0.05
0.7
6.5
Intiiaizee Strength
Strongth
Initiaiized Strongth
Strungth
100
Matched
Target Mismatch
200
Model Mismatch
Data Mismatch
60
20
Accuraky
Accurscy
60
Accurscy
Accuracy
40
E1 valuee
value
Ey valuue Mismatch
value
Target
Accurasy
600
Accurasy
Accuracy
60
20
20-
C2 value
value
Cg value
value
Ablation studies of 4, on four settings (left to right: all matched, target mismatch, model mismatch, and data mismatch) using CIFAR-IOM dataset: top-line: performance using different initialized & for controlling the forgetting strength, in which & 2 0.5 may induce decreased retaining accuracy; middle-line: effect of i controlling the length of Phase-I for target identification, generally ( t= 1 is sufficient for differentiate target data like Figure 3 while larger value reduce the retaining epochs; bottom-line line: effect of to controlling the length of Phase-IlI prevents the excessive forgetting and generally to works well. The above flexibly control the forgetting and retaining part.
Figure 17:
47


Reviews:
[{"review_id": "avlmQv8n4r", "reviewer": "Reviewer_7K9D", "summary": "This work addresses the unlearning by exploring label domain mismatch and introduces novel forgetting challenges such as target, model, and data mismatches. The proposed TARF framework addresses these challenges, showcasing promising results in various experiments, thus contributing significantly to advancing the unlearning paradigm.", "questions": "How sensitive is TARF to the scheduling of k(t) and τ(x,y,t)? Can the authors provide empirical or theoretical justification for the annealing rate and threshold choices?", "limitations": "", "rating": 10, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper presents a novel method that decouples the class label from the target concept, addressing practical scenarios where the forgetting target may not align with pre-training taxonomy. \n2. authors provide theoretical insights that introducing the representation gravity. The solid analysis for understanding unlearning dynamics at the representation level is very important.\n3. extensive experiments show consisent results with the proposed claims.", "weaknesses": "TARF introduces multiple dynamic hyperparameters (e.g., k(t), τ(x,y,t), β, t₀, t₁), but their tuning procedure is only heuristically described. Ablation results help, but further discussion on computational stability and robustness would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tBQJIehsje", "reviewer": "Reviewer_gpUj", "summary": "This paper addresses a critical and often overlooked limitation in machine unlearning: the implicit assumption that the \"target concept\" to be forgotten perfectly aligns with a pre-defined \"class label\". The authors argue that in practical scenarios, a mismatch frequently occurs between the labels of the data slated for forgetting, the model's output taxonomy, and the actual target concept. To address this, the paper formalizes this \"label domain mismatch\" by introducing three novel and practical unlearning scenarios: target mismatch, model mismatch, and data mismatch. These scenarios significantly expand the scope of unlearning research beyond the conventional \"all matched\" setting. The authors systematically analyze the challenges arising from these scenarios, revealing crucial representation-level forgetting dynamics, which they term the \"gravity effect\". Based on these insights, they propose a general framework, TARF (TARget-aware Forgetting), designed to actively forget the true target concept while preserving the utility of the remaining data. TARF achieves this through a unified objective that simultaneously performs annealed gradient ascent on the identified forgetting data and a selective, target-aware gradient descent on a subset of the remaining data that is at risk of being unintentionally affected. Extensive experiments on benchmarks (CIFAR-10/100, ImageNet) and modern applications (LLMs, diffusion models) demonstrate that TARF significantly outperforms existing methods in these challenging new scenarios, showcasing its effectiveness and general applicability.", "questions": "1.The data-driven approach for setting β by ranking dynamic responses is elegant. My question centers on its robustness in more challenging scenarios: How sensitive are the final unlearning outcomes to the choice of the quantile used to determine β? Does this estimation method remain stable in cases where the \"false retain\" set is either very small or disproportionately large compared to the initially provided forget set?\n2.The appendix provides a good discussion on computational cost, noting its comparability to other methods. Focusing specifically on foundation-model-scale applications, where even a single forward pass on the entire remaining dataset (Dun) for monitoring can be expensive, could the authors elaborate on practical optimization strategies? For example, would a sampling-based approach to estimate the \"gravity effects\" on a subset of Dun be a viable strategy, and what might be the trade-offs?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.Pioneering Problem Formulation: The paper introduces a novel, highly practical, and significant problem by decoupling the class label from the target concept in machine unlearning. This new perspective, formalized into three distinct mismatch scenarios, fundamentally expands the scope of the field and bridges a critical gap between academic research and real-world requirements for privacy, copyright, and AI safety.\n2.Principled and General Framework (TARF): The proposed TARF framework is an elegant and well-motivated solution. Rather than being an ad-hoc fix, its design is grounded in a solid theoretical and empirical analysis of representation-level forgetting dynamics. Its ability to handle all four scenarios (matched and mismatched) within a single, unified objective highlights its robust and general design.\n3.Comprehensive and Rigorous Empirical Validation: The paper provides extensive and convincing experimental results. TARF's superiority is demonstrated across multiple datasets of varying scales, against a wide array of strong baselines, and in all proposed scenarios. The inclusion of case studies on modern generative models (Stable Diffusion and LLMs) powerfully underscores the method's relevance and practical utility in contemporary AI systems.", "weaknesses": "1.While the appendix contains a detailed ablation study for key hyperparameters (Figure 17), the discussion is currently presented as an empirical result. To improve practical adoption, the paper would be strengthened by synthesizing these findings into explicit guidelines or heuristics for practitioners. For example, the authors could provide a recommended strategy for setting k based on dataset characteristics or model capacity.\n2.The core mechanism of TARF cleverly leverages \"representation gravity\" for target identification. The authors commendably acknowledge the limitations of this assumption in Appendix G.2, especially for semantically indistinct concepts. Given its foundational importance, this discussion is critical. Acknowledging this limitation in the main paper (e.g., in the conclusion or future work section) would offer a more balanced perspective on the method's applicability. This also opens up an interesting avenue for future research, such as integrating external knowledge to bolster target identification when representation similarity is ambiguous.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pYEu6iy5oA", "reviewer": "Reviewer_ES8Q", "summary": "This manuscript addresses the machine unlearning problem and proposes a target-aware dynamic forgetting framework (TARF). The framework innovatively formulates the forgetting task as a complex optimization scenario involving a mismatch among the “forgetting data”, “model outputs”, and “target concepts”. It leverages “representation gravity” effect to dynamically identify forgetting targets that are not fully labeled, and employs a collaborative optimization mechanism combining “annealed gradient ascent” with a “target-aware retention” strategy to balance the completeness of forgetting with the preservation of the model’s general performance. Experimental results convincingly demonstrate the effectiveness and superiority of this approach across various mismatch scenarios. Nonetheless, there remain several aspects of the paper that could be improved.", "questions": "1. In Section 3.3, the manuscript employs two time functions to control the loss function at each training round. Since the complexity of different datasets can influence the loss dynamics, how are the hyperparameters involved in the time functions(such as β and annealing rate)? How are the temporal checkpoints t_0andt_1  determined among Target Mismatch, Target Mismatch, and Retraining Approximation? Although the ablation studies in the appendix demonstrate the robustness of the method within certain ranges, it is recommended that the authors provide more practical guidance or heuristics on how to select these parameters for new tasks.\n\n2. The manuscript presents four different types of mismatch scenarios. In Phase 1, the method performs Target Identification to locate samples among the remaining data that are similar to the forgetting data. However, in my opinion, this approach may not be applicable when the target scope is broader than that of the forgetting samples (Target Mismatch scenario described in the paper). It is suggested to further clarify how the method addresses this issue.\n\n3. The manuscript primarily focuses on scenarios involving the forgetting of a single or closely related target concept. It would be valuable for the authors to discuss the scalability of the proposed framework when multiple target concepts need to be forgotten simultaneously, particularly when these concepts are entirely unrelated or partially overlapping.\n\n4. How much additional computing overhead in target recognition introduced in phase 1? Will this overhead increase significantly with the number of categories and the proportion of false data?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The manuscript introduces new settings that decouple the class label and the target concept, which investigate the label domain mismatch in class-wise unlearning. \n2. The manuscript systematically reveals the challenges of restrictive unlearning with the mismatched label domains, and demonstrates that the \"representation gravity\" in forgetting dynamics is critical for achieving the forgetting target in the new tasks.", "weaknesses": "1. Some of the figure legends in the manuscript obscure content. For example, Fig 5 and Fig 7. Additionally, certain legends are not fully explained, which affects readability. For example, the line legend in Fig 5(b). The readability of the tables is poor. It is recommended to also highlight the second-best values to improve readability in Table 2.\n2. Compared with some simpler forgetting methods, the Target Identification stage in TARF requires additional computation to observe the “gravity effect.” The authors do not provide a clearer analysis and comparison of the extra computational overhead introduced by the TARF framework in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vo4hpO3wy6", "reviewer": "Reviewer_MLTw", "summary": "This paper focuses on a specific case for machine unlearning, where the target concept to be forgotten does not neatly fit into the classes that the model was trained on. Under such a data mismatch, the authors first provide some theoretical analysis of the forgetting. Then, the authors propose a method to handle this scenario, consisting of three steps. Essentially, the forgetting and retaining objectives are used in a dynamically scaled manner.", "questions": "In Phase 1 Target Identification, how is the dynamic information learnt? How are these information stored?\n\nIn Phase 2, how is the dynamic information used to retain the knowledge? I do not observe any weighting or dynamic knowledge directly affecting the loss here.\n\nWhat does “feature deconstruction” mean? How is this achieved in the method?\n\nI do not understand the results in Fig 5a and 5b. I do not think these experiments and results have been explained clearly anywhere. What is the experiment and what is being visualized here?\n\n\nPlease see more questions in the Weaknesses Section.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This paper tackles a novel setting of mismatched classes between training time and unlearning time. This may be practical in cases when large models are trained on classification tasks.\n\nExperiments are run on both vision and language settings, which is good.", "weaknesses": "A big issue is that the method and theory is not well explained.\nFor instance, the intuition in Section 3.2 is not well explained, and it is mostly theorems and equations. This is important, because the proposed method relies on these analyses of the forgetting, but the intuition is unclear. For instance, the “gravity effects”, “decomposition lacking”, “representation gravity” terms should be explained intuitively as they seem to be important concepts. \n\nFurthermore, the link between Section 3.2 and 3.3 is not clear. The equations look very different, and how the previous concepts apply here is totally not clear. Thus, the proposed method looks very arbitrary.\n\nThere isn’t a related work section that discusses the various related works, and comprehensively covers the similarities and differences to those works.\n\nThe words in the figures are of very small size. There is also a lot of text in these figures, and they act like a long caption. Even though I think  adding text in such a manner is technically within the rules, I do not feel comfortable about this.\n\n\nMany of the compared methods are quite dated, such as  SCRUB, L1-Sparse, SalUn, and  BS which are from 2023. These are already the most recent methods compared. The authors should compare with more recent methods.\n\n\nMinor typos that do not affect the score:\n\nLine 203: Jacobin -> Jacobian\n\nLine 287: decent -> descent\n\nLine 468: Differernt -> Different", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)