Group-robust Machine Unlearning
Authors: Thomas De Min
Subhankar Roy
Stéphane Lathuilière
Elisa Ricci
Massimiliano Mancini
Venue: Accepted by TMLR ()

Abstract: Machine unlearning is an emerging paradigm to remove the influence of specific training data (i.e., the forget set) from a model while preserving its knowledge of the rest of the data (i.e., the retain set). Previous approaches assume the forget data to be uniformly distributed from all training datapoints. However, if the data to unlearn is dominant in one group (e.g., ethnicity, gender), we empirically show that performance for this group degrades, leading to fairness issues. To perform unlearning while preserving fairness, this work addresses the overlooked problem of non-uniformly distributed forget sets, which we refer to as group-robust machine unlearning. We formalize the problem and present a simple and effective exact unlearning strategy that mitigates the performance loss in dominant groups via sample distribution reweighting. Moreover, we present MIU (Mutual Information-aware Machine Unlearning), the first approach for group robustness in approximate machine unlearning. MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degradation in the dominant group of the forget set. Additionally, MIU exploits sample distribution reweighting and mutual information calibration with the original model to preserve group robustness. We conduct experiments on three datasets and show that MIU outperforms standard methods, achieving unlearning without compromising model robustness. Source code available at https://github.com/tdemin16/group-robust_machine_unlearning

Published in Transactions on Machine Learning Research (12/2025)
Group-robust Machine Unlearning
Thomas De Min University of Trento, Italy deoman.demin@eeee @
Subhankar Roy
University of Bergomo, Italy
Stéphane Lathuilière
Inria Grenoble, Univ. Grenoble Alpes, Pronce
Elisa Ricci
University of Trento, Italy
Pondazione Bruno Kessler, Italy
Massimiliano Mancini
University of Trento, Italy
Reviewed on OpenReview: https: // openreviee.. net/ forum? id= 7mpS7mpUVV
Abstract
Machine unlearning is an emerging paradigm to remove the influence of specific training data (i.c., the forget set) from model while preserving its knowledge of the rest of the data (i.e., the remaining set). Previous approaches assume the forget data to be uniformly distributed from all training datapoints. However, if the data to unlearn is dominant in one group (e.g., ethnicity, gender), we empirically show that performance for this group can degrade, leading to fairness issues. To perform unlearning while preserving fairness, this work addresses the overlooked problem of non non-uniformly distributed forget sets, which we refer as group-robust machine unlearning. We formalize the problem and present a simple and effective exact unlearning stralegy that mitigates the performance loss in dom- inant groups via sample distribution reweighting. Moreover, INFORMATION-AWARE MACHINE UNLEARNING), the first approach for group robustness in approximate machine unlearning. MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degrada- tion in the dominant group of the forget set. Additionally, MIU exploits sample distribution reweighting and mutual information calibration with the original model to preserve group robustness. We conduct. experiments three datasets and show that MTC outperforms standard methods, achieving unlearning without compromising model robustness. Source code available at https://ggtie www.://eemnnl///roopoppoeoo rottss/t machine_unlearnne
present MIU (MUTUAL
1 Introduction
In several countries, recent regulations grant greater control over their digital privacy, explicitly recog- nizing their right to request the removal of personal data, known as the "right to be forgotten" (Mantelero, 2013
Voigt & Von dem Bussche 2017). Therefore, to comply with such regulations, machine learning sys- tems should be able to forget specific training data upon user request- challenge addressed by machine unlearning (Kurmanji et al. 2024 Chundawat et al. 2023a)..
In the machine unlearning literature, the standard setup assumes that a group of individuals requests the removal of their data. A machine unlearning algorithm then fulfills this request. by making the model "forget"

---

Published in Transactions on Machine Learning Research (12/2025)
Group Membership
Unlearning
Unlearning
Drop group accuracy
Group-robust Unlearning
Unaltered group accuracies
Uniformly distributed forget set
Non-uniformly distributed forget set
Unaltered group accurncies
Figure 1: Comparing unlearning approaches. Previous works assume the forget set to be uniformly distributed. However, real-life unlearning requests do not comply with the uniform distribution assumption [Bertram et al. 2019 If the forget set distribution is predominant in some groups (e.g., old males), it can lead to performance degradation in such dominant. forget groups (i.e., the blue group in the figure). Group-robust. Unlearning prevents this from happening.
the designated data (i.e., the forget set) while maintaining its utility (Kurmanji et al. 2024 The naive solution, called eract unlearning, consists of retraining the model without the forget data; however, this is computationally prohibitive for large models (Zhao et al. 2024b). Approzimate unlearning methods (Jia
al. 2023 Fan al., 2024b; Kurmanji et. al. 2024) overcome this limitation by unlearning the model with
significantly fewer resources but with less unlearning guarantees.
Most approaches assume that the forget sets are uniformly sampled from the training data (Jia et al.) 2023:
Fan et al.] 2024b Cheng & Amiri 2023 Shen et. al. 2024 He el al. 2024 Zhao et al.] 2024b uang el al.
2024j. However, studies (Bertram et al. 2019 Zhang et al. 2024) show that individuals from different social
and cultural backgrounds (i.e., groups) request to be forgotten at varying rates: Bertram et al. (0019) reports that 44.4% of unlearning requests to news websites relate to professional wrongdoing or crimes, while Zhang et al. (0244) finds that wealthier, highly educated individuals
more likely to request unlearning. Existing methods overlook this imbalance, potentially degrading the accuracy of dominant groups in the forget set and leading to unfair outcomes (see Figs. and 2h. This can be critical in scenarios requiring high accuracy across all groups. In a recommendation system, if unlearning requests predominantly come from a specific group (c.g., young males), the system's 's quality for that group may degrade, potentially making it unnsable. Thus, machine unlearning algorithus must account for distribution shifts.
We target this unexplored scenario, which we name group-robust machine unlearning, that aims to unlearn the user's data while minimizing the performance detcrioration of groups dominating the forget set. Differently from previous works, we tackle both eract and approrimate unlearning by (i) finding a retraining strategy that preserves the original group robustness (Sagawa et al. 2020). and (ii) proposing an approximate unlearning method that efficiently forgets data while preserving robustness for unbalanced forget sets.
For (i), we show that reweighting the sampling distribution during retraining compensates for information loss with minimal robustness impact. We validate this strategy (called REWEIGHT) against GROUP-DRO (Sagawa al., 2020), a popular group-robust optimization method, showing that REWEIGHTT better preserves model group robustness in exact unlearning. For (ii), we introduce approximate unlearning method called MACHINE UNLEARNING), leveraging mutual information minimiza- tion BBllgazi et al. 2018) and calibration to unlearn the forget set while preserving model robustncss. By minimizing the mutual information between forget-set features and ground-truth group annotations, we decorrelate unlearning from spurious attributes (Liu et al.) 2021), mitigating performance loss for dominant groups. To prevent affecting other groups, we calibrate the unlearned model's mutual information to match the original one. Coupled with REWEIGHT, MIU outperforms established unlearning approaches (L1-SPARSE Jia et al. 2023), SALUN (Fan al. 2024b), and SCRUB (Kurmanji et. al. 2024)) CelebA
MIU (MUTUAL INFORMATION-AWE
[Liu et. al.
2

---

Published in Transactions on Machine Learning Research (12/2025)
2015), Waterbirds (Sagawa et al. 2020 and FairFace Karkkainen & Joo 2021 p and preserved group robustness!
in both unlearning efficacy
Contributions. In summary, our contributions are:
We are the first to identify the issue of group robustness in approximate unlearning, showing how existing unlearning algorithms degrade model robustness in this setting.
We propose a simple and effective sample distribution reweighting strategy to mitigate the group accuracy degradation in exact unlearning.
We introduce MIU, the first approximate unlearning approach tailored for this task that minimizes the mutual information on the forget set while calibrating it to match the group robustness of the original model.
We benchmark existing baselines and MTU on group-robust machine unlearning using CelebA Liu et al. 2015), Waterbirds (Sagawa et al. 2020), and FairFace (Karkkainen & Joo 2021). and showing that MIU outperforms existing methods in this task.
2
Related work
Machine unlcarning. Exact machine unlcarning methods (Bourtoulc et al. 2021 Yan et al. 2022 Aldaghri
et al. 2021 guarantee that sensitive data is removed. However, they are impractical (Chundawal. el al.
2023a:
Kurmanji et al.) 2024] as retraining (part of) the model is prohibitive (Nguyen et al. 2022). Instead,
approximate unlearning concentrates on computational feasibility by relaxing the guarantees constraints
KKrmanji et al.
2024
Chundawat et al. 2023a Jia al. 2023 Fan et al., 2024b Chen al.)
2023) Most
works focus random
(Ja et al. [023 Fan et al.) 2024b; Cheng & Amiri
2023 Shen al.
2024
He al.
2024 Zhao et al.
2024b;
Huang al.) 2024 and class (Chen et al. 2023
Chundawat et al.
2023a
Hoang
et al. 2024 Cheng et al.
2024 Zhao el al.) 2024a) unlearning, respectively forgetting i.i.d. data points, and
all data points from a single class. While the former lowers the forget-set accuracy to match that of the model retrained without the unlcarning data, the latter aims at scoring zero accuracy on the unlcarned class.
A few works (Chundawal el al. 2023a Poster et al. 2024 Cheng el al. 2024) explore subclass unlearning, where a subset of a class (e.g., subclass car from superclass vehicles) is removed from model weights. While subelass and group-robust unlearning share similarities, the latter focuses on preserving group accuracies, whereas subclass unlearning alters the model to behave if the target subclass was never in the training set. Related to our research, Fan et al. (2024a) suggests a bi-level optimization to sample adversarial forget sets that are difficult to unlearn. Also related, Chen et al. (2024) proposes machine unlearning as an efficient debiasing technique. Instead, [Zhang et al. (2024) investigates how uniform and non-uniform data sampling affect fairness in MLPs and tabular data, limiting the evaluation to exact unlearning.
Group-robust unlearning can be seen as generalized case of random unlearning, where we drop the i.i.d. assumption of the forget set. This has implications for the design of proposed methodologies. As existing approaches assume uniform distribution of the forget set during evaluation, they do not account for group imbalance in the forget distribution, possibly degrading the model's robustness.
Group-robust learning. Methods for group-robust optimization train deep learning models to be robust to spurious correlations. Algorithms are categorized based on their access to group information. Within those that assume access to group annotations (Sagawa et al. 2020) Goel et al., 2020 Zhang al., 2021; Idrissi et al. 2022; Kirichenko et al., 2023), group-DRO (Sagawa et al. 2020) dynamically reweights the misclassificatiin penalty for each group to optimize the worst-group accuracy. Instead, Idrissi et al. (2022) proposes a simple baseline that subsamples each group to match the size of the smallest one. While these works usually achieve better results, accessing the group information can be challenging (c.g.. annotation
cost.) Within methods agnostic to group information (Liu el al. 2021 Zhang et. al. 2022 Tdrissi et al. 2022
Sohoni et al., 2020 Nam et al. 2020), JTT (Liu et al. 2021 increases the sampling probability of wrongly classified data to improve worst-group accuracy. Correct-N-Contreed
(Zhang et al.) 2022)
uses a contrastive
We robustness and rooup--ss interchangeshly in this paper.
3

---

Published in Transactions on Machine Learning Research (12/2025)
loss to pull correctly and misclassified samples of the same class while pushing apart wrongly classified data points of different categories. However, methods that require group information also show state-oftthe-art performance on groups discovered from data (Kim et al. 2024 D'Incà et al., 2024).
This paper is the first to study the intersection between group robustness and machine unlearning. For this reason, we assume complete access to the group information.
3 Method
This section formulates the machine unlearning task (Sec. 3.1 and the group-robust machine unlearning problem (Sec. 3.2 Then, it introduces the proposed sample distribution reweighting strategy (Sec. 3.3 showing its effectiveness in group-robust machine unlearning. Finally, Sec. 3.4 describes MIU, our approxi- mate unlearning method tailored for group-robust unlearning.
3.1 Machine unlearning
Let o X Y be learnable function (or model), where fo(-) : Z is non-linear feature extractor parameterized by 0, and h-)-) 2 + Y is linear classifier parameterized by 4,, mapping inputs from the image X to the target space Y. Let D.r {(x, y) be a training dalaset of size No:; where x; is an image, and y1 its target label (e.g., age). Let model trained on Drr with algorithm T be denoted as he. fa. (or PRETRAIN). A machine unlearning algorithm U scrubs the influence of desired forget set Dj C Der from the pretrained model by outputting scrubbed weights {Pu. Ow): such that lip se. is as close as possible lo the exact unlearning model h2g o Se. (or RETrAIN), trained solely on the remaining set DD = Der Dj with
algorithm T Kurmanjji et al. 2024 Fan et al. 2024b Zhao et al. 2024b
3.2 Group-robust machine unlearning
In group-robust machine unlearning, we consider the forget data non-uniformly distributed. There- fore, let us re-define the training dataset as DDr = {(x, y, ) where x; and Yr are defined as in Sec. 3.1 and a; is the protected or sensitive attribute (e.g.: gender, ethnicity). Now, let G Y x A be the set of all groups, defined the cartesian product between the target label set Y and the protected attribute set A (Sagawa et al.) 2020 Kirichenko eet al. 2023). We denote the i-th datapoint group as 9: = (yi, az). Each target-sensitive attribute pair (e.g., males, between the ages of 20-29) identifies a
PAETRANS
Rereate
Gisransi
SALOX
SCRUB
Accuracy on hlond males
Accuracy on non-hlonde females
100
Accurazy (?)
30
75
20
0.5 bloed males non-blonde fomados being untenrewd
0.9
Figure Unlearning non-uniformly distributed data. We test standard model reiraining, and pop- ular approximate unlearning methods (L1-SPARSE, SALUN, and SCRUB) in group-robust unlcarning. The more samples of the least represented group (blond males) are unlearned from CelebA, the lower the model accuracy on that group. On the contrary, the most represented one blondblodde females) is less affected.
unique group 2
If the forget set is non-i.id., the group distribution in the remaining set changes compared to the origi- nal training set. As the proportion of group in the remaining set lowers, its resulting accuracy also de- creases, which can harm the model's generalization performance on the dominant group of the forget set. Figure shows the accuracy degradation caused by jointly removing different percentages of the least and most represented groups, i.e., blond males and non-blonde: females. While the most represented group is mildly affected as its ratio drops from 44% to 7% (over four groups), the least represented one (dropping from 0.8% to 0.1%) suffers substantial accuracy degradation of ~00% Further analysis is provided in Appx.
Taarget and protected attributes color-coded.
4

---

Published in Transactions on Machine Learning Research (12/2025)
Therefore, the objective of group-robust machine unlearning differs from that of machine unlearning. While the latter minimizes the discrepancy with the retrained model (gold standard), the former also preserves the original model's performance on each group.
3.3 Frustratingly easy group-robust unlearning
Section 3.2 introduces the group-robust machine unlearning task and shows the extent to which the accuracy of the forget set dominant group(s) drops after unlearning. Unlike prior works (see Sec. [. we aim unlearning non-uniform forget data while preserving the model accuracy on the dominant group of the forget set. Therefore, this section proposes exact unlearning strategy tailored to this task. To mitigate the performance degradation of the dominant group of Dy, argue that reweighting the data distribution (RWG in Idrissi et al. (2022)) to account for the removed samples is simple and effective bascline that retrains the model with minimal performance drop. Contrary to RWG (Idrissi et al. 2022), which enforces uniform sampling distribution from group-perspective, we suggest instead to reweight the sampling distribution to match the original dataset statistics. Intuitively, increasing the sampling likelihood for partly unlearned groups rebalances the remaining set group statistics to match those of the training datasct, recovering the original robustness.
Formally, let P(x:) Nor be the probability of sampling Xi, let Var, 14 E NG respectively be the group frequencies of the training and remaining datasets. We reweight P(x;) according to the ratio #r: P(x:) Σj,v;" where ag. the adjusted group weight for sample x,. We denote the data distribution reweighting strategy (or REWEIGHT) as w(-), i.e., w(,,) denotes the reweighting strategy applied to the remaining dataset.
We validate REWEIGHT by comparing the model retrained with REWEIGHT and PROUP-DRO (Sagawa et al. 2020). Figure summarizes the outcome of our analysis. As a byproduct of strongly optimizing worst-group accuracies, we notice that GROUP-DRO (Sagawa et al. 2020) can also increase the forget set accuracy. This issue makes approximate unlearning evaluation more difficult RETRAIN + CROUP-DRO is used as the gold standard. Assuming a hypothetical original forget-set accuracy of 70,, if an approximate unlearning algorithm targeting such gold standard leads to higher accuracy (e-g., 80%), then the knowledge unlearned if foryet-set accuracy increased? Answering this question is non-trivial, as the retrained model cannot be accessed for real applications. Furthermore, retraining with GROUP-DRO (Sagawa el. al. 2020 causes test accuracy drop (-5.8%) in FairFace (Karkkainen & Joo 2021]. On the contrary, REWEIGHT does not suffer from these issucs.
3.4 Group-robust unlearning without retraining
Section 3.3 shows that sample reweighting
RETHAIN
RETRAIN cDRO
RETTHAI
simple and effective approach to machine unlearn- ing that preserves the original model's robustncss. However, model retraining is inefficient (Chundawat et al. 2023a; Kurmanji et al. 2024) and unpracti- cal [gguyen et al. 2022). Therefore, this section proposes MIU, our approximate machine unlearn- ing algorithm that jointly tackles unlearning while decorrelating unlcarning and group robustness with- out retraining the entire model.
Group Acc. Gap
Test Acc. Gap
Gap with PRETRAIN
100
90
99.3
99.5
93.8
94.2
80
77.5
70
CelebA
FairFace
As scrubbing unlearning data affects the forget set dominant group accuracy, we propose a unified ob- jective for jointly unlearning while preserving orig- inal group robustness. We use mutual informa- tion between output features and group annotation, which, upon minimization on forget data, jointly un- learns and mitigates the performance degradation the dominant group of the forget set Ragonesi et al.
Figure 3: REWEIGHT GROUP-DRO. RETRAIN REWEIGHT achieves better test and group accuracy alignment with the original model (higher is better). Thus, it better preserves the original performance after unlearning.
5

---

Published in Transactions on Machine Learning Research (12/2025)
2021j. Formally, let G[ I[Z (Y, AJ) be the mutual information between random variables Z and G, associated with model features z = fo(x) and group y = (v. a), then:
G(Z = [ P(z.a)(2 log
F(z.o1(2.9) dz dg, Pz(2)PP(9))
(1)
where (z,,)) is the joint pdf of Z and G, and Pz, PG are marginal pdfs of Z and G. Minimizing [ G) roduces the dependency between Z and Y while also reducing it between Z and A, jointly unlearning the network and disentangling features and protected attributes. However, computing the mutual information for continuous variables is generally intractable (Paninski 2003) Belghazi et al. 2018). Therefore, we follow Belghazi et al. 20118) and estimate the mutual information with an MLP Te::
M(D; v. 0) = ((x.))DTo(Zz.)) - log (x.)) e Tv(()))))
(2)
where Te is a two-layer MLP with ReLU as non-linear activation, outputting a single scalar, (x.g) D refers to sampling from the joint distribution, (x. D refers to sampling from the product of the marginal distributions. To sample from the product of the marginal distributions we sample twice from the joint distribution: (x, g) s.t. (x,9) ~ D, (x,g) ~ D, and keep only x and as in Belghazi et al. (2018). We now use this definition to derive MIU (MUTUAL INFORMATION-AWARE MACHINE UNLEARNING), our proposed method.
Unlcarning term. We minimize the mutual information between forget set features and their group label to unlearn the forget set D, while maintaining a good trade-off between information removal and robustness preservation. Therefore, we denote the unlearning term as:
(3) This term is high when the forget features correlate with the group information. Intuitively, we want this term to be low as this implies that we are unlearning the relation between forget-set features and labels while decorrelating the unlearning process from group information. Formally, minimizing this term achieves unlearning while preserving group robustness due to the following proposition:
M(Dy; 0).
Proposition 1. Mutual information minimization equivalent to cross-entropy loss maximization on a group classifier.
Proof. Let us rewrite Eq. in terms of entropy and conditional entropy:
1(; = H(G)- - H(G & -H(G)2),
(4)
where group entropy H(G) is constant, thus, it can be ignored during optimization. Let lg(z)) be a group classifier, namely linear layer that classifies image groups using features z, and let @ be the random variable associated with estimated groups. We can now write the cross-entropy loss for classifying groups using entropy notation as:
H(G: GIZ) = Z( + DKL (GI
(5)
Following Boudiaf et al. 2020 i.e., first optimize DKL. ZZ) and then H(G In other words, the first step fixes the backbone weights 0 and optimizes the group classifier weights a, while the second only optimizes model weights 0, keeping the classifier weights frozen. As the classifier is fixed in the latter, then maximizing the cross-cntropy with respect to encoder weights corresponds to maximizing HG)) i.e., minimizing T(Z; G). Thus, minimizing T(Z; G) (as in Eq. (3)) is equivalent to computing gradient ascent on group information, functionally disentangling label and spurious attributes from image fcatures.
Lemma we can relate Eqs. 4 and (5) by decoupling Eq. 5 into two steps,
Calibration term. As unlearning might also affect other groups, we designed an extra term to improve group performance retention. Thus, we minimize the mutual information discrepancy between the original and unlearning model on the reweighted remaining dataset w(D,):
(Dr)) (M) M(w 0,).1
(6)

---

Published in Transactions on Machine Learning Research (12/2025)
where (D,(D)): (..: estimates the mutual information using features computed by the original backbone (.) Equation 1 encourages the unlearned model to mimic the original robustness, preserving its behavior across groups.
Retaining term. Since unlearning generally causes performance degradation Kurmanji et al. 2024 Fan
et al. 2024b; Jia et al., 2023), we perform gradient descent steps on the remaining data to ensure that the original model group accuracy preserved after unlearning. Additionally, we use REWEIGIT to limit the degradation of group robustness. Therefore, optimize the cross-entropy loss the reweighted remaining
dataset (D,))
(D(,) F, ((x)) CCE
Putting all together. The retaining term in MIU preserves the model's original discriminative capabilities. The unlearning lerm removes Dj from model weights while minimizing performance loss for dominant, groups in the forget set. Finally, the calibration term ensures that the unlearned model maintains its original robustness. Therefore:
Pu: 02 = arg min (2(()))) + M(D + A v,, Calibration
(8)
$9,0
Retaining Unlearning term
MIU pscudocode. Algorithm presents MIU pscudocode in Porrch-like (Paskke et al. 2019 style.
2024b) and compute unlearning and retaining We follow previous works (Kurmanji et al. 2024 Fan et al. steps separately. Thus, we alternate between computing an unlearning epoch using Eq. (3) and retaining one using Eqs. D and (7). Like SCRUB (Kurmanji al.] 2024), we find it beneficial to stop performing unlearning steps after predefined number of epochs (i.e., out of 10). Contrary to [Ragonesi al. (2021), do not update the mutual information represenlation at every step. Instead, we empirically observed that updating it via 100 gradient updates for the first epoch and 10 updates for the remaining iterations is sufficient to achieve satisfactory results and limit the required resources. As we keep the optimization steps fixed, the overall mutual information estimation overhead depends on the dalaset size. For a small dataset like Waterbirds (Sagawa et al. 2020), we estimate an overhead of about 1.80x. Instead, for CelebA {Liul et al. 2015) and FairFace (Karkkainen Joo 2021 we estimate an increase of approximately 1.03x in unlcarning time, which is negligible.
Algorithm 1 PyTorch-like MIU pscudocode.
def (model, mine, mine_ores siie__riggial, optim, train_daatt dataloader, mamiiiinggddttlcadeee forget prnget_dattloaderr
unlearned - copy deeepcopyymmodil
for epoch in renoche::
Update mine (mutual information neural estinator) Belghazi et al. 2018
tune mine (unlearned, mine, train_ dataloader))
Use unlearn the forget_datased for the first forget_epocke epochs epoch forget_epochs:
image, group barget_datalone group_bar torch.randint Leindint_likeegrree (group, num_grome num_groups) loss mine eadeearneediiaage group, group_bar) update_modell (loss, optim)
Apply fine-tuning steps at every epoch using the reveighted sampler for image, target, group in reweight aaamaining_dttloadeee group_bar
randint, like (group, num, cum_gsr loss - F. F.crosss_eee =F.crosseentropy (tead(unlerreed(mnne))))) target) loss *- lambda P_mss_lose(
mine canleulleredddee (inage), group, group_bar)
mine_ mine_origing oww.ooiiiiaaaoooo.ooooo (model (inage), group, group bar) (etach
update_model model (loss, optim)
unlearned

---

Published in Transactions on Machine Learning Research (12/2025)
4 Experiments
This section describes the experimental protocol (Sec. 4.1 and compares MIU with multiple methods in group robust unlearning (Sec. 1.2). We then compare existing approaches and ours when varying the un- learning ratio of the forget set dominant group (Sec. 4.3) and when sampling the forget set from multiple
groups (Sec. 4.4. Finally, Sec. 4.5 shows a complete ablation study of MIU's components.
4.1 Experimental protocol
Datascts. We follow established works in group-robust optimization (Sagawa et al. 2020 Idrissi ct al.
2022
Liu et al. 2021 Park et al.
2022 Park & Byun 2024) and conduct experiments on CelebA (Liu et al. 2015
and Waterbirds (Sagawa et al.
2020) datasets, setting blonde and male as the target and protected attributes
for CelebA (Liu et al.) 2015). while setting waterbirds and land as the target and sensitive attributes for
Waterbirds (Sagawa et al. 2020p. We additionally experiment with FairFace (Karkkainen & Joo 2021 given its numerous annotations, using age as the downstream task and randomly choosing the class 20-29 and the cthnicity afro-americon as target and protocted attributes, respectively. Unless stated otherwise, forget-set images are sampled from groups described by the above (arget-sensitive attribute pairs. Moreover, unless stated otherwise, the forget set simulates the worst-case scenario where single group is responsible for the unlearning request, loading to high forget distribution imbalance. The unlearning ratio is defined as the proportion of samples from that particular group that have been unlearned. After unlearning, the model must have unlearned the forget data and maintained its original robustness.
Hardware and training details. We obtain PRETRAIN and RETRAIN by fine-tuning via empirical-risk minimization a ResNet-18 (He et al.] 2016) pre-trained on ImageNet (Russakovsky et al.] (2015) for 30 epochs, using SGD with 0.9 moment um and weight, decay. The learning rate is decayed with cosine annealing scheduler for the entire training. We additionally warm-up the learning rate for the first two epochs using linear scheduler. We apply standard data augmentation techniques, namely, random resized crop, random horizontal flip, and input normalization (He et al. 2016). We limited fine-tuning to 10 epochs for approximate unlearning methods, searching for the optimal configuration for the other hyperparameters. The À parameter of MIU is set between and 10 (see Appx. B.5 All experiments ran on a single A100 Nvidia GPU, using Py PyTorcch (Paszke et al. 2019).
Baselines. We compare MIU against three state-of-the-art machine unlearning approaches. LI-SPARSE (Jia et al. 2023) forgets sensitive information by fine-tuning the original model on the remaining set with a sparsity regularization term. SALUN (Fan et al., 2024b) proposes a saliency-based unlearning that forgets data via random labeling. SCRUB (Kurmanji et al. 2024) minimizes the divergence between the unlearned and original model on the remaining set while maximizing it on the unlcarning data. Following previous works (Kurmanji et al.) 2024 Jīa el al.] 2023). we report PRETRAIN, and RETRAIN, which are computed by fine-tuning an ImageNet (Russakovsky al. 2015) pre-trained ResNet-18 (He al. 2016). We also report RETRAIN + GROUP-DRO (Sagawa et al. 2020 to validate the proposed RETRAIN + REWEIGHT.
We reimplemented all three baselines following the existing codebases. For L1-SPARSE Jía et al. 2023 we perform 10 fine-tuning epochs on the remaining set with a linearly decaying L1 regularization that follows this rule: 7 = (1-1/T)) where i is the epoch, T is the total nunber of iterations, and 7 the initial penalty. For SALUN (Fan et al. 2024b) we followed the scale" implementation of the original codebase (i.e., the implementation for CIFAR-10 (Krizhevsky et al.] (200)) and SVHN (Netzer et al., (011) as is meant for datasets with few classes. We first compute the saliency mask using the gradient information from the forget set, pruning 50% of the network weights. We tune the pruned weights for 10 epochs by alternating a full pass the forget set and full pass the remaining set. Also, SCRUB (Kurmanji et al,, 2024) is implemented by separating forget and retaining steps, following the original code. The forget step maximizes the KL divergence between the original and the unlearned model in the forget data set. Following the original paper, we stop computing it after 3, 5, or 7 epochs. Instead, the retaining step minimizes the lincar combination between the cross-entropy loss and the KL divergence between the original and unlcarned model, respectively scaled by 0.99 and 0.001, as reported in (Kurmanji et al. (2024). We note that all methods use the same dataset splits; therefore, they must unlearn the same forget set.
8

---

Published in Transactions on Machine Learning Research (12/2025)
Table 1: Group-robust machine unlearning in CelebA (Liu et al., 2015). We build the forget set by sampling data points from a single group. The unlcarning ratio is set 0.5. We compare MIU against LI-SPARSE QJia et al.] 2023), SALUN (Fan et al. 2024b), and SCRUB (Kurmanji el al. 2024). The avg. gap and deltas are computed against RETRAIN + REWEIGHT, and we bold the methods that achieve the smallest discrepancy with it. Other reference models are in light gray.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
96.2
41.9
95.9
0.9
24.2
40.6
RETRAIN
96.5
31.3
95.9
1.6
27.0
3144
RETRAIN + GDRO
95.8
67.4
95.1
13.2
RETRAIN
96.3
39.7
95.8
2.2
23.9
41.3
LI-SPARSK Jia al.
2023
95.7 (0.5)
29.0 (10.7)
95.4 (0.3)
1.5 (1.3)
28.5 (4.6)
30.4 (10.9)
95.3
SALUN Fan et aL 20216
96.2 (0.2)
29.3 (10.3)
95.8 (0.1)
0.7 (1.5)
29.1 (5.2)
30.6 (10.7)
95.3
SCRUB Kurman MIU
2024
96.5 (0.3)
35.1 (4.6)
93.9 (0.2)
0.6 (1.7)
(2.8)
(5.4)
97.3
96.4 (0.3)
36.3 (3.4)
95.9 (0.1)
1.0 (1.3)
26.1 (2.2)
36.3 (5.0)
98.0
L1-SPARSK Jia al.
2023
95.6 (0.6)
37.3 (3.6)
95.4 (0.4)
0.7 (1.5)
26.7 (2.8)
34.8 (6.5)
97.4
SALUN Fan et a. 2024h
96.1 (0.2)
42.9 (8.2)
95.8 (0.1)
0.6 (2.0)
23.9 (3.4)
41.1 (9.8)
96.1
SCRUB Kurman et al MIU
2024
96.1 (0.3)
43.5 (3.8)
96.0 (0.2)
0.7 (1.5)
23.7 (0.7)
43.0 (1.7)
98.6
96.3 (0.3)
43.2 (3.5)
96.0 (0.2)
1.2 (1.3)
24.0 (0.6)
41.3 (0.4)
99.0
Metrics. To evaluate unlearning and group-robustuess, we rely on six different metrics. The first three are remaining (RA), forget (UA), and test (TA) accuracy. We also report the membership inference attack (MIA) Yeom et al., 2018), which measures the MIA-Efficacy as described in (Jia al. 2023 Fan et al. 2024b)- Finally, assess the change in group robustness by looking at equalized odds (FO) (Hardt et al.
2016,, and the test accuracy of the forget-set dominant group (GA).
GA measures the ratio of correctly classified test samples that belong to the same dominant group(s) of the forget set. Therefore, let D. = {(x; Vi, a.) 95 9i = (yi, az) A (x,, Vi, az) E Die} be the subset of the test set composed of all samples of group gy, then GA is computed as follows:
D
GA
1 [(k o fo)(xx) Vil.
(9)
D
where Il is the indicator function, returning if the argument is True, and 0 otherwise. Like other accuracy metrics, the closer GA is to 1 (or 100%), the better the model is at classifying data of the dominant group of the forget set.
Finally, to case the interpretation of six metries, follow previous works (Jia et al. 2023 Fan et al. 2024b)
and compute the average gap (avg. gap) and per-metric deltas with the gold standard S Following them, we do not report whether metrics should be maximized or minimized as the machine unlearning objective is to reduce the discrepancy with the gold standard on each metric, except for avg. gap, which must be maximized. We use RETRAIN + REWEIGHT as the gold standard since it better reduces the gap with PRETRAIN in TA, EO, and GA, compared to RETRAIN and RETRAIN + GROUP-DRO (Sagawa et al. 2020). Further details are in Appx.
4.2 Results on group unlearning
Tables I
3 show results for group-robust unlearning on CelebA (Liu et al. 2015), Waterbirds (Sagawa
ct al.
2020 and FairFace (Karkkainon & Joo 2021 using an unlcarning ratio r of 0.5.
CelebA. RETRAIN + REWEIGHT achieves the best trade-off between original performance preservation and unlearning, showing the best gap with UA (-2.2), EO (-0.3), and GA (+0.7). The forget accuracy (UA) is very close to that of PRETRAIN, which might seem strange at first glance. Yet, UA and GA share similar values for both PRETRAIN and RETRAIN. This suggests that the model achieves a low generalization error, as the forget accuracy aligns with that of unseen samples (from the same group) regardless of whether the forget set was part of the training data. Additionally, as CelebA [Lu al. 2015 counts numerous images,
Deltas are computed per each seed and then averaged over three runs.
9

---

Published in Transactions on Machine Learning Research (12/2025)
Table 2: Group-robust machine unlearning in Waterbirds (Sagawa et al., 2020). We build the forget set by sampling data points from a single group. The unlcarning ratio is sct 0.5. We compare MIU against LI-SPARSE (Jia et al., 2023), SALUN (Fan et al., 2024b), and SCRUB (Kurmanji et al., 2024 The avg. gap and deltas are computed against RETRAIN + REWEIGHT, and bold the methods that achieve the smallest discrepancy with it. To avoid confusion, other reference models are in light gray.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap f
PRETRAIN
98.9
84.5
87.7
33.3
26.2
56.6
RETRAIN
98.7
52.4
86.5
51.8
30.1
19.4
RETRAIN + GDRO
94.7
89.3
91.6
21.4
7.3
RETRAIN
99.0
59.5
87.2
53.6
28.3
51.6
LI-SPARSK Jia al.
2023
99.0 (0.2)
59.5 (7.1)
85.6 (1.6)
44.0 (9.5)
32.2 (4.1)
(11.))
94.4
SALUN Fan et aL 20216
100.0 (1.0)
50.0 (9.5)
81.8 (5.
90.5 (36.9)
38.7 (10.4)
39.3 (12.3)
87.4
SCRUB Kurman MIU
2024
98.8 (0.3)
60.7 (10.7)
86.9 (U.7)
(10.7)
(4.3)
41.7 (9.8)
93.9
100.0 (1.0)
53.6 (8.3)
86.1 (1.2)
58.3 (7.1)
28.3 (3.0)
53.8 (7.5)
95.3
L1-SPABSK Jia al.
2023
98.7 (0.2)
64.3 (11.9)
85.0 (2.2)
46.4 (7.1)
30.6 (2.3)
53.7 (8.3)
94.7
SALUN Fan et a. 2024h
100.0 (1.0)
47.6 (16.7)
81.1 (6.1)
91.7 (38.1)
39.0 (10.7)
39.0 (12.5)
85.8
SCRUB Kurman et al MIU
2024
98.9 (0.2)
66.7 (11.9)
87.0 (0.6)
11.0 (9.5)
30.9 (3.4)
41.3 (7.3)
94.3
99.9 (0.9)
54.8 (4.8)
85.8 (1.4)
59.5 (6.0)
28.3 (1.8)
53.7 (4.0)
96.9
Table 3: Group-robust machine unlcarning in FairFace (Karkkainen & Joo, 2021 ) We build the forget set by sampling data points from & single group. The unlearning ratio is set 0.5. We compare MIU against L1-SPARSE (Jia et al. 2023), SALUU (an et al., 2024b), and SCRUB (Kurmanji et al.) 2024 The avg, gap and deltas are computed against RETRAIN + REWEIGHT, and bold the methods that achieve the smallest discrepancy with it. To avoid confusion, other reference models are in light gray.
method
RA
UA
TA
MTA
GA
avg. gap 1
PERTRAIN
65.6
79.0
57.2
0.2
5.4
71.2
RESRAIN
66.8
57.8
56.5
0.9
9.2
58.7
RETRAIN + .DRO
61.7
56.3
51.4
10.2
2.3
57.4
RISTHAIN
66.7
69.3
56.7
0.7
5.6
69.6
LI-SPABSE Tia al.
2023
64.0 (2.7)
74.1 (4.8)
56.9 (0.5)
0.2 (0.7)
6.1 (0.9)
69.4 (0.4)
98.3
SALUN Fan et all 202-h
66.3 (0.3)
66.6 (3.0)
55.9 (0.8)
0.3 (0.6)
9.0 (3.4)
60.3 (9.3)
97.1
SCRUB Kurman MIU
2024
66.9 (0.3)
(3.9)
56.7 (0.6)
1.0 (0.5)
(1.3)
61.3 (8.3)
97.0
66.7 (0.1)
74.7 (5.4)
57.2 (0.8)
0.3 (0.5)
6.0 (1.1)
66.1 (3.5)
98.1
L1-SPARSE Jia al.
3023
64.4 (2.3)
72.9 (3.6)
56.0 (0.9)
0.3 (0.4)
6.1 (2.0)
67.0 (7.1)
97.3
SALUx [Fan el al 2024b
65.1 (1.5)
69.8 (5.6)
54.8 (1.8)
0.3 (0.4)
6.6 (1.7)
63.7 (5.9)
97.2
SCRUB Kurman et al MIU
2024
66.7 (0.2)
73.4 (4.1)
57.2 (0.0)
0.7 (0.3)
6.2 (0.7)
70.2 (1.8)
98.7
64.7 (1.9)
71.6 (2.3)
57.1 (0.5)
0.3 (0.3)
5.8 (1.5)
70.3 (1.2)
98.7
REWEIGHT easily preserves original group accuracies, thus, recovering the original model performances and showing the same UA of PRETRAIN. Retraining the model with GROUP-DRO (Sagawa et al. 2020) generally leads to unwanted unlearning behaviors, i.e. the UA increases by 25.5 points. Plain RETRAIN, instead, suffers performance degradation in the dominant group of the forget set, as we highlight in Fig. 2 (GA lowers by 6.2 points). Adopting the proposed reweighting strategy leads to better trade-off in group-robust unlearning.
MIU achieves the best performance preservation (Tab. D by scoring the highest GA aligmment both when using (A0.4) and not using (A5.0) REWEIGHT. These results highlight that mutual information improves group performance retention. Instead, existing approaches must rely on REWFIGHT to close the gap GA and EO with MIU as they are agnostic group-robust machine unlearning. We also notice that using REWEIGITT increases the UA for all tested algorithms. Yet, despite the method used, coupling REWEIGHT with approximate unlearning always recovers most of the original GA. We visualize this in Fig, 4 which shows the same experiment as Fig. g while highlighting the REWEIGHT contribution. However, as MIU is strictly designed for this task, it generally achieves better robustness than existing approaches, scoring the best avg. gap-
Waterbirds. Similarly CelebA (Liu et al. 2015) experiments, RETRAIN + REWEIGHT achieves the best discrepancy (Tab. 2 in terms of TA (-0.5), EO (+2.1), and GA (-5.0), better preserving original group robustness. RETRAIN + GROUP-DRO (agawa et al. 2020 instead, increases the UA by 4.8% above
10

---

Published in Transactions on Machine Learning Research (12/2025)
PRETRAIN and by 36.9% above RETRAIN, which negatively affect the unlearning evaluation if used the gold standard for approximate machine unlearning. Compared to the CelebA [Liu et al. 2015) case, RETRAIN achieves a better
preseevvtitneettiioonn trade-off. Nonetheless, RETRAIN + REWEIGHT is still a better candidate for group-robust unlearning, always achieving the best calibration with the original model.
In Waterbirds (Sagawa et al. 2020 our method outperforms existing methods both with and with- out REWEIGHT. Yet, REWEIGHT does not pro- vide substantial improvement get accuracy (from delta of 8.3 to 4.8) and the EO (from delta of 7.5 to 4.0) get significantly enhanced. Instead, REWEIGHT substantially in- creases L1-SPARSE (Jia et al. 2023) and SCRUB [Kurmanji ct al., 2024) average UA (+4.8 and +6.0), achieving higher values than RETRAIN + REWEIGHT (59.5%) Nonetheless, it also improves the GA, showing better dominant forget group preservation. We argue that the subtle improve- ment provided by REWEIGHT caused by the lim- ited dataset size of Waterbirds (Sagawa et al. 2020). By increasing the sampling likelihood of the few dominant forget group images left, the network fits those few samples, limiting robustness preser- vation. Even without REWEIGHT, MIU outper- forms all baselines, regardless of whether they use REWEIGHT, further validating our design choices.
PRETRAIN
Li-SPARSK
SCRUB
No RW
RETRAIN
SaLUK
MIU (Ours)
REWEIGHT preserves GA
only the for-
40
GA
20
0.1
0.5
0.9
Unlearning ratio
Figure 4: REWEIGHT for group-robust unlearn- ing. As in Fig. D we test different methods and REWEIGHT in group-robust unlearning on CelebA. Darker colors are used for methods without the reweighting, while lighter ones correspond to meth- ods coupled with REWEIGHT. As the unlearning ra- tio grows, the methods' GA degrade. Instead, adding REWEIGHTT restores the original GA.
FairFace. Differently from the other datasets, here RETRAIN CROUP-DRO (Sagawa al. 2020) struggles to preserve original model accuracy (Tab. 3 in both the dominant forget group (-13.8) and the test. set (-5.8). We ascribe this behavior to numerous FairFace [Karkkainen & Joo 2021 groups that make the group-robust optimization challenging. Instead, RETRAIN + REWFIGHT overcomes this issue as it simply reweights group sampling probabilities to match the original training dataset statisties, achieving better GA (-1.6) and a 'A (-0.5). Importantly, our experiments highlight a key advantage of REWEIGHT, which functions effectively "off the shelf" with the original model hyperparameters. Unlike RETRAIN + GROUP-DRO (Sagawa al. 2020), it requires no additional tuning.
All methods show good alignment RETRAIN + REWEIGHT, even without reweighting. TA and GA are generally preserved, with SALUN Fan et al., 2024b) scoring lowest at 55.9% (delta 0.8) and 60.3% (delta 9.3) respectively. However, methods without REWEIGHT show higher UA than RETRAIN, indicating partial scrubbing of the forget set. Regardless, Sec. 4.5 shows that the high UA of MIU is not caused by poor un- learning but the calibration lerm, which recovers the original group robustness. REWEIGHT further enhances model robustness, reflected in higher GA and lower EO. Finally, MIU + REWEIGHT and SCRUB (Kurmanji et al. 2024) + REWEIGHT achieve the same avg. gap, but while our method shows better UA and GA alignment with RETRAIN + REWEIGHT, SCRUB (Kurmanji al.) (2044) obtains a better alignment in EO terms. Although REWEIGHT does not fundamentally improve the avg. gap, it generally enhances GA and EO, promoting performance retention.
Discussion. UA strongly correlates with GA after unlearning in all three datascts. A high UA-GA cor- relation suggests that the forget set behaves as unseen data of the same group, indicating that the forget set was properly unlearned. Moreover, MIU consistently approximates RETRAIN-REWEIGIIT REWEIGHT better than existing methods or is on par, while REWEIGITT reliably preserves performance without drawbacks in model retraining or approximate unlearning.
11

---

Published in Transactions on Machine Learning Research (12/2025)
MTU nw (Ours)
SALUN + my SCRUB lo
L1-SPARSE +
MIU (Ours)
SALUN SCRUB
Sampling Dy from multiple groups
L1-SPARSE
CelebA
Waterbirds
Avg. Gap (1)
100.0
100
Avg. Gap (1)
98
97.5
90
98.7
98.7
99
98.6
08.6
98.5
98.3
98.3 98.3
98.0
97.3
97.3
97.4
977
80
96
9
25
0.1
0.5
0.9
0.1
0.5
0.9
Unlearning ratio
Unlearning ratio
Number of Groups
Figure 5: Group-robust unlearning across dif- ferent unlearning ratios. We compare L1- SPARSE (Jia al. 2023). SALUN (Fan et al., 2024b), and SCRUB (Kurmanji al., 2024 against our ap- proach while using the REWEIGHT strategy methods. MIU achieves overall the best avg. gap when varying the unlcarning ratio.
Figure 6: Sampling the forget set from multi- ple groups. We evaluate our method against L1- SPARSE (Jia et al. 2023), SALUN (Fan al. 2024b), and SCRUB (Kurmanji et al. 2024) when the forget set is sampled from multiple FairFace (Karkkainen) & Joo 2021] groups. MIU is more consistent across experiments, always achicving the best result.
all
4.3 Impact of different unlearning ratios
Figure analyzes methods avg. gap across different unlearning ratios. On CelebA [Liu et al. 2015), all methods show consistent discrepancies from RETRAIN + REWEIGHIT across different unlearning ratios, likely because REWEIGHT recovers a large portion of the lost robustness. Nonetheless, MIU consistently outper- forms existing approaches, scoring 99.4%, 99.0%, and 98.8% unlearning ratios 0.1, 0.5, and 0.9. Instead, all algorithms struggle at 0.1 unlcarning ratio Waterbirds (Sagawa et al. 2020). where the small forget set size (i.e., five samples) causes high fluctuations in the UA, as each sample counts as 1/5 of the total clas- sification error. Furthermore, the reduced forget set size makes the gradient and BatchNorm (Loffe 2015) estimation noisy. As the unlcarning ratio grows (i.c., 0.5 and 0.9), MIU outperforms basclines by a growing margin. At unlearning ratio 0.9, MIU achieves 97.2% on Waterbirds (vs. 93.6% of SCRUB (Kurmanji et al. 2024h). Furthermore, MIU remains more consistent across unlearning ratios, confirming that our design choices effectively narrow the avg. gap with RETRAIN + REWEIGHT. Full Fig. 5 tables are reported in Appx.
4.4 Multi-group unlearning
This section compares MTU and existing approaches multi-group unlearning, i.c., when the forget set data is sampled from multiple groups (group composition in Appx. B.3) Figure shows the outcome of this experiment
FairFace (Karkkainen & Joo 2021), given its numerous groups, with a varying number of groups in the forget set and unlearning ratio fixed to 0.5. All methods achieve high scores and good discrepancy with RETRAIN + REWEIGHT, though SCRUB (Kurmanji et al. 2024] performs worst (97.1% of avg. gap). Performance gaps shrink as more groups are included, as forget-set statistics align with the original training distribution, reducing non-uniform sampling effects. Nonetheless, MIU is the most consistent, highlighting its effectiveness group-robust machine unlearning.
4.5 Ablations
Table A shows the ablation of MIU components all three datasets by systematically adding each element to understand its contribution. We mark with "y" when the component used in the experiment and "x" when is not. From left to right, we list retaining term, un- learning term, calibration term, and REWRIGHT. In the first row, consider method when
12

---

Published in Transactions on Machine Learning Research (12/2025)
only the unlearning term and the retaining term used. We highlight how the UA is low for all three datasets, demonstrating that mutual information minimization can be used to unlearn.
This baseline already achieves a remarkable avg. gap with RETRAIN + REWEIGHT, scor- ing a 97.5% CelebA (Liu et al. 2015), which is the best among methods that do not use REWEIGITT. Adding our calibration term (Eq. ()). leads to an increase in GA in all three datasets, with FairFace (Karkkainen & Joo 2021] showing a growth of 6.9%. We high- light that the forget accuracy also grows. Yet, this increase is caused by Eq. (6) which cal- ibrates the mutual information to match the original group robustness. Therefore, the high CA cannot be blamed on the poor unlearning. Although most previous approaches exploit
Table 4: MIU ablations. We compute MIU ablations on each of the three investigated datasets. From left to right, report the investigated dataset, retaining term, unlearn- ing term, calibration term, and REWEIGHT. We mcasure performance using UA, GA, and avg. gap. The configura- tion that corresponds to MIU + REWEIGHT is highlighted.
dataset
Eq.
Eq.
Eq. 6
RW
UA
GA
gap
35.5
35.4
97.5
CelebA
36.3
36.3
98.0
27.2
28.5
95.2
43.2
41.3
99.0
x
47.6
51.1
92.5
Waterbirds
53.6
53.8
95.3
16.7
16.8
81.9
retaining term /Chundawat et al. 2023a Kur-
54.8
53.7
96.9
manji et al., 2024 Jía et al. 2023 Fan et. al. 2024b), we also ablate this component for com- pleteness. As previous works suggest (Kur-
63.1
59.2
96.1
FairFace
74.7
66.
98.
87.1
81.1
93.0
71.6
70.3
98.7
manji al., 2024 Chundawat et al. 2023a), removing the retaining term negatively impacts model utility, resulting in the lowest performance overall (e.g., 81.9% in Waterbirds). Similarly, when adding REWEIGHTT, the avg. gap gets improved in all three datasets (e.g., 96.9% in Waterbirds), as we also highlight in Fig. 4 These results highlight the contribution of each of MIU components that allow for unlearning (Eq. (3) while preserving forget set dominant group performance (Eq.
5 Conclusion
This paper is the first to address the performance degradation that can be caused by non-uniformly dis- tributed forget sets in both model retraining and approximate unlearning. We show that adopting simple data distribution reweighting (REWEIGHT) for retraining the model is simple and better allernative than retraining with GROUP-DRO (Sagawa et al. 2020). Moreover, we propose the first approximate unlearning method (MIU) that unlearns personal data while reducing the risk of degradation of the forget set domi- nant group. Our evaluation demonstrates that RETRAIN + REWEIGHT consistently improves over a simple RETRAIN while MTU perforrs existing baselines group-robust machine unlearning.
Limitations. One limitation of work is the assumption that group annotations known, which may not hold in world applications where such labels are difficult to obtain. Therefore, natural follow-up of this work is group-agnostic methodology that preserves model robustness as achieved by REWFIGHT. A naïve approach towards this direction would be to discover groups from data {Kim et al. 2024 D'Incà et al. 2024), and applying approaches presented in this paper. Furthermore, our evaluation restricted to the classífication setting. Applying the proposed techniques and baselines to other domains might. be non- trivial, and there is no guarantee that unlearning offectiveness and accuracy will transfer directly. Despite these limitations, our work is an important first step toward understanding and mitigating the accuracy degradation caused by group- unalanenal forget sets.
Broader impact statement. Machine unlcarning is designed to remove user data from trained model. While its primary goal is to preserve privacy, it can also be misused for malicious purposes. Targeted unlearning of specific groups may lead to biased models with harmful consequences, However, the proposed group-robust machine unlearning socks to minimize performance degradation for dominant groups in the forget set. Thus, MIU and REWEIGHT counteract biases introduced by unlearning, mitigating the negative societal impacts of its misuse.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Acknowleddgements. We acknowledge the CINECA award under the ISCRA initiative for the availability of high-performance computing resources and support. Flisa Ricci and Massimiliano Mancini are supported by the MUR PNRR project FAIR Future AI Research (PE00000013), funded by Nexterarntio EU. Elisa Ricci also supported by the EU projects AI4TRUST (No.101070190) and ELIAS (No.01120237). Thomas De Min is funded by NexiGeneration EU. This work also supported by the EU project ELLIOT (101214398) and by the French National Research Agency (ANR) with the ANR-20-CE23-0027, We thank Olivier Laurent for his valuable feedback and insightful suggestions.
References
Nasser Aldaghri, Hessam Mahdavifar, and Ahmad Beirami. Coded machine unlearning. IEEE Access, 2021.
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. Mine: mutual information neural estimation. In ICML, 2018.
Theo Bertram, Elie Burszlein, Stephanie Caro, Hubert Chao, Rutledge Chin Feman, Peter Fleischer, Albin Gustafsson, Jess Hemerly, Chris Hibbert, Luca Invernizzi, et al. Five years of the right to be forgotten. In SIGSAC, 2019.
Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. In ECCV, 2020.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In SP, 2021.
Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In CVPR, 2023.
Ruizhe Chen, Jianfei Yang, Huimin Xiong, Jianhong Bai, Tianxiang Hu, Jin Hao, Yang Feng, Joey Tianyi Zhou, Jian Wu, and Zuozhu Liu. Fast model debias with machine unlearning. In NeurIPS, 2024.
Jiali Cheng and Hadi Amiri. Multimodal machine unlearning. arxie, 2023.
Xinwen Cheng, Zhehao Huang, and Xiaolin Huang. Machine unlearning by suppressing sample contribution. arXiv, 2024.
Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using incompetent teacher. In AAAI, 2023a.
Vikram Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhall.. Zero-shot machine un-
learning. In TIFS, 2023b.
Moreno D'Tncå, Elia Peruzzo, Massimiliano Mancini, Dejia Xu, Vidit Goel, Xingqian Xu, Zhangyang Wang, Humphrey Shi, and Nicu Sebe. Openbias: Open-sct bias detection in text-to-image generative models. In CVPR, 2024.
Chongyu Fan, Jiancheng Liu, Alfred Hero, and Sijia Liu. Challenging forgets: Unveiling the worst-case forget sets in machine unlearning. In ECCV, 2024a.
Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Empowering machine unlearning via gradient- tradeent-based weight saliency in both image classification and generation. In ICLR, 2024b.
Jack Foster, Stefan Schoopf, and Alexandra Brintrup. Fast machine unlcarning without retraining through selective synaptic dampening. In AAAI, 2024.
Karan Goel, Albert Gu, Yixuan Li, and Christopher Ré. Model patching: Closing the subgroup performance gap with data augmentation. arXiv, 2020.
14

---

Published in Transactions on Machine Learning Research (12/2025)
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In NeurIPS, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Zhengbao He, Tao Li, Xinwen Cheng, Zhehao Huang, and Xiaolin Huang. Towards natural machine un- learning. arXiv, 2024.
Tuan Hoang, Santu Rana, Sunil Gupta, and Svetha Venkatesh. Learn to unlearn for deep neural networks: Minimizing unlearning interference with gradient projection. In WACV, 2024.
Mark He Huang, Lin Geng Foo, and Jun Liu. Learning to unlearn for robust machine unlearning. In ECCV, 2024.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive wonst-group-accuracyy Conference on Causal Learning and Reasoning, 2022.
Sergey Ioffe. Batch normalization: Accelerating deep network training by reducing internal covariate shift.
In ICML, 2015.
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsity can simplify machine unlearning. In NewrIPS, 2023.
Kimmo Karkkainen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In WACV, 2021.
Younghyun Kim, Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jaeho Lee, and Jinwoo Shin. Discovering and mitigating visual biases through keyword explanation. In CVPR, 2024.
Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In ICLR, 2023.
Alex Krizhevsky, Gcoffrey Hinton, al. Learning multiple layers of features from tiny images. Toronto, ON, Canada, 2009.
Meghdad Kurmanji, Peter Triantafilouu, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. In NeurIPS, 2024.
Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In NewrIPS, 2017.
Evan Liu, Behzad Haghgoo, Aunie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. TCML, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
Alessandro Mantelero. The eu proposal for general data protection regulation and the roots of the 'right to be forgotten'. Computer Law & Security Review, 2013.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learning from failure: De-biasing classifier from biased classifier. In NewrIPS, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NewrIPSW, 2011.
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee- Chung Liew, Hongzhi Yin, and Quoe Viet Hung Nguyen. A survey of machine unlearning. arXiv, 2022.
Liam Paninski. Estimation of entropy and mutual information. Neural computation, 2003.
15

---

Published in Transactions on Machine Learning Research (12/2025)
Sungho Park and Hyeran Byun. Fair-vpt: Fair visual prompt tuning for image classification. In CVPR, 2024.
Sungho Park, Jewook Lee, Pilhyeon Lee, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. Fair contrastive learning for facial attribute classification. In CVPR, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeeiing Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In NewrIPS, 2019.
Ruggero Ragonesi, Riccardo Volpi, Jacopo Cavazza, and Vittorio Murino. Learning unbiased representations via mutual information backpropagation. In CVPRW, 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. IJCV, 2015.
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In ICLR, 2020.
Shaofei Shen, Chenhao Zhang, Yawen Zhao, Alina Bialkowski, Weitong Tony Chen, and Miao Xu. Label- agnostic forgetting: A supervision-fres unlearming in deep models. In ICLR, 2024.
Nimit Sohoni, Jared Dunnmon, GeofTrey Angus, Albert Gu, and Christopher Ré. No subclass left behind: Fine-grained robustness in coarse saasse-griied classification problems. In NeurIPS, 2020.
Paul Voigt and Axel Von dem Bussche. The general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 2017.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In ACI, 2018.
Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. Arcane: An efficient architecture for exact machine unlearning. IJCAI, 2022.
Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somosh Jha. Privacy risk in machine learning: Ana- lyzing the connection to overfitting. In CSF, 2018.
Dawen Zhang, Shidong Pan, Thong Hoang, Zhenchang Xing, Mark Staples, Xiwei Xu, Lina Yao, Qinghua Lu, and Liming Zhu. To be forgotten or to be fair: Unveiling fairness implications of machine unlearning methods. AT and Ethics, 2024.
Jingzhao Zhang, Aditya Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar, and Suvrit Sra. Coping with label shift via distributionally robust optimisation. In ICLR, 2021.
Michael Zhang, Nimit S Sohoni, Hongyang R Zhang, Chelsea Finn, and Christopher Re. Correct---contrast: a contrastive approach for improving robustness to spurious correlations. In JCML, 2022.
Hongbo Zhao, Bolin Ni, Junsong Fan, Yuxi Wang, Yuntao Chen, Gaofeng Meng, and Zhaoxiang Zhang, Continual forgetting for pre-trained vision models. In CVPR, 2024a.
Kairan Zhao, Meghdad Kurmanji, Getrge-Octavian Bärbulescu, Eleni Triantafillou, and Peter Triantafillou. What makes unlearning hard and what to do about it. In NeurIPS, 2024b.
16

---

Published in Transactions on Machine Learning Research (12/2025)
A Metrics
This section provides further details on the adopted metrics, i.e., the RA, UA, TA, the cawabbershi-iinffeeen attack (MIA) (Yoom et al. 2018); the dominant forget group accuracy (GA), the equalized odds (EO), and the average gap (avg- gap) [Fan et al., 2024b).
R.A, UA, and TA. We evaluate the model on the remaining, forget, and test sets to compute the remaining, forget, and test accuracy. Therefore, report the ratio of correctly classified samples for each of these subsets of the dataset.
MIA. To compute the membership inference attack, we follow previous works (Jia et al. 2023 Fan et al. 2024b) and train a model on remaining and validation losses to predict membership. Therefore, we assign a different binary label to remaining and validation losses and train the model to discriminate between them. After model training, we use such a model to infer the membership of forget set data. In all tables, we report the MIA-Efficacy (Jia al. 2023) that is computed as follows:
MIA-Efficacy = TN
(10)
[Dyl'
where TN are the true negatives, i.e., the number of samples the MIA predicted as non-members. Instead of training a support vector machine as in (Jia et al. 2023 Fan et al. 2024b), we used a random forest as the accuracies are comparable with an SVM. Moreover, training is faster since it can easily parallelized. The higher the MIA- All--fficacy, the better the model's privacy protection.
EO. Equalized Odds (Hardt al. 2016) measures model fairness prediction dependencies on protected attribules. To compute the EO, measure model performance discrepancy by varying the sensitive attribute value and averaging over different target labels. Formally, the EO for binary classification model is computed as follows (Hardt et al. 2016):
EO = Σ P y.A P( = 1 1 Y= = v.A= 2
(11)
Y=
where Y, Y, A are random variables describing model predictions, target attributes, and sensitive attributes. EO measures the absolute difference in outputting a positive prediction when the protected attribute equals and 0, averaging over the two target attributes. In the FairFace (Karkkainen Joo 2021 case, we set all classes and protected attributes that do not match those of the dominant group the forget set as y = 0 and a = 0, as FairFace counts more than two classes and more than two attributes. The lower the EO, the better the model fairness.
avg. gap. Following previous works (Jia et al. 2023 Fan el al. 2024b), compute the avg. gap to simplify the comparison among different methodologies. avg. gap is computed as the average metric discrepancy be- tween the unlearned model and the retrained gold standard. Formally, let M = (RA, UA, TA, MIA, EO, GA} the set of all metrics used in this paper, then avg. gap is computed as follows:
E.eg m 02) m( 05)
avg. gap-
(12)
where m(p (%) and m(pp;, 0²) are calculated using unlearned and retrained model weights, the ex- periment seed and Σ is the average. The closer avg. gap is to (or 100 in the tables), the better the approximation of the retrained model.
B Additional results
This section contains the additional results that could not be included in the main paper. Appendix B.1 intuitively shows the accuracy degradation caused by non-uniformly sampled forget sets in all three investi- gated datascts. Appendices [B2 and B.3 report tables associated with experiments on different unlcarning ratios and sampling the forget set from multiple groups. Appendix B.5 further expands MIU ablations. Finally, Appendix B.6 further evaluates MTU's and REWEIgHT's fairness and robustness preservation.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Remra
SAIEN
SURIm
CalebA misority gomp
OelebA nujarilt group
Waterbink minority group Waterhinis mujorily group
FairPuoe minority grouge
PairFFos majoeilly groups
100
1100 bmb
50
20
50
50
0.9
Ralão of bllod undhe and non-lakanle Junndies loing unhotrssnd
Figure 7: Unlearning non-uniformly sampled data. We lest standard model retraining, and popular approximate unlearning methods (L1-SPARSE (Jia et al., 2023). SALUN (Fan al.) (0244)) SCRUB (Kur- manji et al., 2024)) in group-robust unlearning, The more samples of least represented groups are unlearned, the lower the model accuracy on such groups. On the contrary, the most represented ones are less affected.
Table 5: Group-robust machine unlearning in CelebA (Liu et al., 2015) with 0.1 unlearning
ratio. We build the forget set by sampling data points from single group. The unlearning ratio is set to 0.1. We compare MIU against L1-SPARSE (Jia et al.
2023 SALUN (Fan et al.
2024b), and SCRUB
Kurmanji et al. 2024h. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap f
PRETRAIN RETRAIN
96 06.2+000
24.0+2.7
95.9+0.1
0.5
23.4+0.5
41.9=177
96.220.0
39.9:1.0
95.810.00
0.2+0.33
21.60033
39.6=0.7
RISTRAIN
96.2+0.0
44-422.1
95.9+0.00
0.740.66
23.4+0.77
42.4-2.2
LI-SPARSE Jia et al.
2023
95.4 0..
36.013.9
95.4.10.1
0.7:10.0
27.610.7
31.3-444
95.7:11.2
SALUN Fan
2024b
96.24:0.0
36.2+1.0
95.8+00.0
0.0::0.0
26.9+0.6
34.4=2.3
96.6+1
SCRUB Kurmanji el al. MIU
2024
96.1
42.8:2211
96 06.0::0.0
0.5:t0.77
21.5+0.3
40.6=0.8
98.8:00.8
96.2:50.0
43.0042.44
95.91:0.0
0.2:00.3
99.1::0.3
L1-SPARSE (Jia et al
2023
95. 0.00
38.9.1..
95.4.10.0
1.0.10.3
26.3
36.5
97.311
SALUN Faa al 2024b
96.11:000
40.145.99
95.910.00
0.7::066
25.0::1.9
38.7=6.1
97.2+00.9
SCRUB Kurman MIU
al
2024
96.4+0.0
44.7+3.0
96.0+00.0
0.5::0.3
23.9 23.9+00.4
42.0=1.3
99.0=0.44
96.2100.0
44.9:12.7
95.9:10.0
0.2.10.3
22.9:10.5
43.0-0.7
99.4110.1
B.1 Unlearning non-uniformly sampled data
Figure 2 of the main paper shows how the forget set dominant group accuracy drops for least represented groups when increasing the unlcarning ratio, limiting the analysis to ColobA (Liu et al., 2015) due to space constraints. This section reports the same experiment on all three investigated datasets for completeness.
Figure 7 shows accuracy variations when unlearning blond males (minority) and non-blonde females (ma- jority) in CelebA (Liu et al.] 2015). waterbirds on land (minority) and landbirds land (majority) in Waterbirds (Sagawa et al. 2020), and more than 70 y.o. Middle Easterns (minorityy. 60-69 y. Caucasians (minority), 20-29 y.o. A African Americans (majority), and 30-39 y.o. Southeast. Asians (majority) ) in FairFace (Karkkainen & Joo 2021), with different unlearning ratios: 0.1, 0.5, 0.9. While the accuracy drop is more evident on CelebA (Liu et al. 2015), it is also visible in the other two datasets. Interestingly, some unlearning methods are more robust to this performance drop than others (e.g., SCRUB (Kurmanji al. 2024) scores 56.25% in FairFace (Karkkainen & Joo 2021) with unlcarning ratio 0.9 vs. 38.54% of SALUN (Fan et al. 2024b) in minority groups, where the latter is the second best).
Although some methods more robust, argue that the quality of the unlearning process influences the accuracy of the dominant group of the forget set. The less the forget set was unlearned, the more the performance relention. As an example, Sec. 4.2 highlights that likely SCRUB (Kurmanji et al.) 2024) fails to effectively unlearn the forget set in the FairFace (Karkkainen & Joo 2021) experiment, showing higher minority group accuracy (56.25%) than the RETRAIN (31.25%). Thus, to better investigate the relationship between unlearning effectiveness and performance retention, Appx. B.2 reports tables associated with Fig. using all investigated metrics.
18

---

Published in Transactions on Machine Learning Research (12/2025)
Table 6: Group-robust machine unlearning in CelebA (Liu et al., 2015) with 0.5 unlearning
ratio. We build the forget sot by sampling data points from a single group. The unlearning ratio set to 0.5. We compare MIU against L1-SPARSE (Jia et al.
2023 SALUN (Fan et al.
2024b), and SCRUB
Kurmanji et al. 2024b. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
96 2.22
41.940.8
95.940.11
0.940.3
24.241.0
40.6=2.1
RETRAIN
96.510.00
31.310.3
95.910..0
1.610.4
27.011..4
34.4=0.9
RETRAIN
96.320.1
39.7
95.810.00
2.2:1122
23.910.8
41.3=166
L1-SPARSE Jia et al
2023
95.71:0.1
29.0433.1
95.4.10.1
15:1066
28.5.10.6
30.4-2.9
95.3:10.7
SALUN et al. 2024h
0.
29.347.7
96.8400.1
0.74:0.2
19.1
30.6=7.5
95.342.5
SCRUB Kurman
20241
96 0.2
35.1+1.7
95.9+0000
0.6::0.2
10.11111
35.9=2.0
97.5+0.1 4
MIU
96 0.
36.3::0.9
95.94:0.1
1.040..3
26. 100.8
36.3=2.0
98.0:2..4
L1-SPARSE (Ji ct al
2025
95.61.0.0
37.3.138
95.4.10.1
0.7.10.33
26.710099
34.8 4.8
97.411..
SALUN (Fan et al 2024b
96. 0.1
42.94:10.5
95.8+0.00
0.6::0.5
23.9422.8
41.1=9.8
96.1+22.0
SCRUB Kurmanj MIU
2024
96.4+0..2
43.5+0.11
06.0+
0.7+0.11
23.7+0.7
43.0=0.9
98.6
96.3+002
43.2+0.6
96.0+0.0
1.240.1
21.0+0.5
41.3=11
99.0=+0.2
Table 7: Group-robust machine unlearning in CelebA ratio. We build the forget set by sampling data points from a single group. The unlearning ratio is set
(Liu et al. 2015)
with 0.9 unlearning
to 0.9. We compare MIU against LI-SPARSE (Jia et al.
2023 : SALUN (Fan et al.
2024b), and SCRUB
Kurmanji et al. 2024b. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap
PRETRAIN RETRAIN
96.4+0.1
44.0#6.6
95.9+0.11
0.7+0.1
23.5+11.
41,1=5,2
96.72:0.0
20.241.0
95.910.00
7.241.4
51.6
25.4=1.9
RETRAIN
96.4+0.1
33.6#1.1
95.7+0.11
4.3:1.3
25.6+0.7
35.9=1.9
LI-SPARSK Jia al.
2023
95.940.11
21.64666
95.44.0.1
3.941..0
31.511.6
23.9-7.1
94.742..6
SALUN (aa et al
20216
96 0.1
20.8+0088
95.9+0.0
2.6
31.6+006
25.0=2.4
94.6+00.8
SCRUB Kurmanj et al. MIU
2024
96. 7.10.1
27.2:11.2
95.91-0.0
3.14:0.5
31..
26.3=0.7
96.0:+0.6
9..51.0.1
32.6:10.9
95.8100.1
1.510.2
27.910..3
30.9
98.1100.7
L1-SPARSE JJa
gl. 2023
95.9100.0
33.2.12.9
95.2110.1
2.6:1.77
28.3110.5
33.1
98.4:10.6
SALUN Fan et al.
20241
96.4::0.1
38.116677
95.8+:0.2
1.9::1.2
25.91:1.1
36.9=7.2
97.11:0.4
SCRUB Kurman et al MIU
2024
96. 96.74:0.1
41.911.5
95.9
1.740.1
21.9+002
39.3=0.9
97.1+0.2
96.740.1
35.242.0
95.8400.0
4.24
26.94E1.4
34.3-2.7
98.8:10.4
B.2 Impact of different unlearning ratios
For completeness purposes, this section reports all tables associated with Figs. 2 and 7 For these exper- iments, the main paper summarizes the results to reduce the occupied space and simplify the interpretation (i.e., limiting the reported metrics to one). Therefore, Tabs.
10 and 13 present experiments in
CelebA (Liu et al. 2015), Waterbirds (Sagawa et al. 2020)), and Fair Face Karkkainen & Joo 2021] datasets, with unlearning ratios of 0.1, and 0.9. Additionally, Tabs, g and 12 report the standard deviations of
Tabs. to
Overall, we notice that in CelebA Liu et al. 2015), the higher the unlearning ratio, the lower the forget accuracy (from 43.0% to 32.6% using MIU), with the gap being reduced when REWEIGHT is included in the retaining step (from 44.9% to 35.2% with M. RA, TA, and MIA remain stable across different unlearning ratios. Instead, FO behaves similarly GA and UA, with high values at high unlearning ratios (e.g., MTU scores 22.9% vs. 26.9% with 0.1 and 0.9 unlcarning ratios).
In Waterbirds (Sagawa et al. 2020), metrics do not show global trends, except for EO and GA, which worsen as the unlearning ratio grows. Therefore, methods lose accuracy the dominant group of the forget set the ratio of unlearned samples grows, with SCRUB (Kurmanji et al. 2024) showing the highest drop (from 44.3% to 25)1%). Similarly, GA and EO worsen even when adding REW EIGHT. As the unlearning ratio grows, the number of forget set dominant group samples left in the remaining set lowers. Given the small size of Waterbirds (Sagawa et al. 2020) (4795 samples, of which 56 are in the smallest group), REWEIGHT strongly increases the sampling likelihood of restricted mumber of samples to preserve original accuracy, causing overfitting. Thus, the overall benefit of REWFIGHT is reduced (GA increases by only 1.3 with RETrAIN).
19

---

Published in Transactions on Machine Learning Research (12/2025)
Table 8: Group-robust machine unlearning in Waterbirds (Sagawa et al., 2020 with 0.1 un-
learning ratio. We build the forget set by sampling data points from a single group. The unlearning ratio is set to 0.1. We compare
MIU against LI-SPARSE (Jia et al.
2023 SALUN Fan al.
2024b), and
SCRUB (Kurmanii et al. 2024h. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
99 0+001
73.3+18.9
88.020.5
53.3=9.4
25.940.7
57.6=2.0
RETRAIN
99.010.22
46.71:2.9
87.110.8
60.0118.3
27.411.7
$7.4-2.9
RETRAIN
98.720.3
46.74:24.9
87.3
66.7:124.9
26.74113
57.6=4.8
L1-SPARSE Jia et al
2023
99.01:0.1
60.0:116.3
85.3.0..8
46.71:24.9
20.211..
67.9-2.6
92.3:33.0
SALUN ot al. 2024h
100.0=0.00
53.349.4
78.3433.
66.7=9.4
42.745.11
33.3=7.3
81.646.7
SCRUB Kurman
20241
98 0+1.1
33.3+9.1
86.9+004
53.3+18.9
31.1#111
11.3=3.5
91.3+2.0
MIU
99. 140.0
60.04 16.3
87.1
33.3 18.9
26. 121
59.2=5.3
91.249.1
L1-SPARSE (Ji ct al
2025
99.01.0.1
73.3111.9
8..
53.3.124.9
20.112288
58.2.3.8
92.2113.0
SALUN (Fan et al 2024b
99 0.1
33.349.4
76.7422.9
86.7=9.4
45.944..7
29.6=7.4
$1.3:2299
SCRUB Kurman MIU
2024
98 0.2.2
53.3+9.4
86.8+0.7
60.0+16.3
31.5+0.8
42.8=3.5
89.8+0.4
100.0=0.0
73.3+18.9
87.3+0.3
13.3+219
26.2+0.3
61.7=0.8
93.3+0.7
Table 9: Group-robust machine unlearning in Waterbirds (Sagawa et al., 2020) with 0.5
learning ratio. We build the forget set by sampling data points from single group. The unlearning ratio is set to 0.5. We compare MIU against LI-SPARSE (Jia et al. 2023 SALUN (Fan et al.
2024b and
SCRUB (Kurmanji et al. 2024]. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
98 08.920.3
81,5177
77.7
33.3=6.1
26.2+:99
56.6-6.0
RETRAIN
98.710.3
52.418.9
86.5110.2
54.8=9.4
30.410.5
1..4=1.6
RETRAIN
09.010.11
59.511188
87.210.3
53.6.1.77
28.312.00
51.616.0
LI-SPARSH Jia et al
2023
99 0
59.548.99
85.620.4
44.0
32.24:.8
48.8-7.4
94.4
SALUN Fan al.
20210
100.0=0.0
30.0+5.1
81.40.4
90.3=3.4
38.7#12
39.3=3.3
3.5
SCRUB Kurmanji et al. MIU
2024
98 0.2
60.717.7
86.910.6
8.9
$1.9118
41.7=1.7
93.91114
100.0=0.0
33.647.7
86.101.0
38.3=8.9
28.71177
53.8=2.6
95.3+0.7
LI-SPARSK Jia et ml.
2028
08.74001
64.315.8
85.0
46.4111..7
30.6
53.7=4.3
94.7
SALUN Fan et al.
100.011.00
47.6.14.5
81.1199
91.7-7.3
39.0112.2
39.0-1.6
85.8114.2
SCRUB Kurman MIU
2024
98.920.2
66.7#1.7
87.0+0.5
41.0=8.9
30.9+113
41.3=311
915#13
99.9
54.8114.77
85.8
59.5:112.1
28. 28.3122.9
53.7=3.8
96.9411.6
Also FairFace (Karkkainen & Joo 2021) shows a general drop in UA, which grows when REWEIGHTT is applied. EO and GA also behave like in CelebA (Liu et al. 2015). with enhanced degradation at higher unlearning ratios. However, MTU shows good robustness even woth REWFIGHT, scoring F.O and GA that are close to RETRAIN-REWEIGITT e.g., 10.4% vs. 11.9% in EO and 56.8% vs. 53.9 in GA with a 0.9 unlearning ratio (Tab. 13
B.3 Multi unlearning
This section further expands the experimental protocol of Sec. 1.4. highlighting the sampling composition and reporting all investigated metrics. For the 9 groups experiment, we sampled the forget set data with either 20-29 y.o., 50-59 y.o., and 3-9 y.o. target attributes, and Afro-American, Kalino-Hispant and Caucasian sensitive attributes. Instcad, for the 25 groups experiment, we additionally sample from the more than 70 y.o. and 30-39 y.o. target attributes, and East Asian and Middle Eastern sensitive altributes. The choice of groups is random, and the unlearning ratio is fixed to 0.5.
Full results for unlearning multiple groups in group-robust unlcarning are reported in Tabs. 14 and 15 notice overall the same trend as in Fig, G REWEIGHT contribution gets less important as the number of groups from which the forget set sampled decreases, highlighted by unchanged UA,, EO, and GA. The GA, when using MIU, e.g. increases by nearly 5% in the groups experiment (Tab. 14p. while it remains unchanged in the 25 groups one (Tab. 15p. Test accuracies are better preserved when unlearning 1 9 groups, while we notice drop (about 1.5%) in the 25 groups experiment. We argue that this decline is
We
20

---

Published in Transactions on Machine Learning Research (12/2025)
Table 10: Group-robust machine unlearning in Waterbirds (Sagawa et al.; 2020) with 0.9 un-
learning ratio. We build the forget set by sampling data points from a single group. The unlearning ratio MIU against LI-SPARSE (Jia et al.
is set to 0.9. We compare
2023 SALUN [an al.
2024b), and
SCRUB (Kurmanji et al. 2024h. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
98 6+0.6
76.019.1
86.5+0.4
44.7=4.11
28.
55.9=5.2
RETRAIN
98.910.2
41.345.7
84.310.3
68.7-6.8
36.411.4
41.7=3.9
RETRAIN
98.910.1
41.3:42.5
85.710.2
62.7=3.4
33.5+1.3
43.0=2.9
L1-SPARSE Jia et al
2023
98.9100.2
60.0433.3
82.9 1.2
50.7=5.0
35.3110.4
4..9=4.6
92.9:1 1.0
SALUN ot al. 2024h
100.0=0.0
40.0::4.3
81.3:40.9
92.7=3.4
1..41..
30.8=2.2
89.642.0
SCRUB Kurman
20241
97 8+0.1
30.7+1.9
86. 10..5
52.7=3.1
36.6+1.0
25.1=1.5
92.7=0.7
MIU
100.0:0.0
66.725.7
86.740077
58.7=4.7
12.15:155
49.8=3.3
93.4
L1-SPARSE (Ji ct al
2025
99.010022
59.31110.6
84.6.10.6
6..
31.500.7
5..0
91.5.14.1
SALUN (Fan et al 2024b
100.0+0.0
45.342.5
80.3:00.7
87.3=1.9
41.8+0.5
31.9=188
90.9:1.0
SCRUB Kurmanj MIU
2024
98 08.1
33.343.4
86.240.77
54.7=5.2
35.9+1.5
28 28.0=3.4
93.6#1.2
98.9+0.2
44.7=3.1
83.1+1.3
65.3=3.4
35.742.
45.0=1.7
97.2
Table 11: Group-robust machine unlearning in FairFace (Karkkainen & Joo,
2021 with 0.1
unlearning ratio. We build the forget set by sampling data points from single group. The unlearning ratio set to 0.1. We compare MIU against LI-SPARSE (Jia al.) 2023), SALUN (Fan et al.
2024b), and
SCRUB (Kurmanji et al. 2024h. The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
BO
GA
gap 1
PAETRAIN
66.2100.7
79.242.5
57.210.11
0.7.10.2
5.820.2
71.6-2.1
RISTRAIN
67.310.1
71.740.8
07.0+0.4
10+0..8
5.=1.11
69.0=2.8
RETRAIN
66.8
72.011.7
56.810..4
0.9.0..5
4.3.0.6
7..1
LI-SPARSE Jia et al 2025
63.710.3
78.943.5
56.14.0.8
0.0::0.0
5.5
69.7=2,2
97.3
SALUN Fan ct al. 2024h
65.9100.8
73.913.9
55.111.11
0.5.10.0
2.911.1
69.8
97.8.00.9
SCRUB Kurmanji MIU
2024
68.410..5
78.7400.8
67.5+-0.3
0.2::0.2
5.7=0.2
70.4=1.5
97.9:10.5
66.920.3
81.3::022
57.3
0.2:00.2
5.3=0.6
70.1005
97.8
LI-SPARSE (Jia
al 2023
64 61.020.3
72.74:0.7
56.4+-0.6
0.2+0.22
5.3=1.2
69.1=0.7
98.6:0.33
SALUN (Fan et an 20211
66.220.4
80.1#1.5
56.3+0044
0.2+0.2
4.7=0.9
73.3=1.6
97.1
SCRUB (Kurmanji al. MIU
68.4.10.5
79.241.0
57.5100.4
0.2:0..2
5.621.1
70.9-1.7
97.8:10.6
67.4:10.5
82.341.3
57.6110.3
0.0::0.0
6.0=0.7
71.2=0.5
97.6100.8
cansed by the larger forget set, which reduces the number of available samples in the remaining set. Overall, MIU approximates RETRAIN + REWEIGHTT better than baselines, consistently achieving the best avg. gap.
B.4 Unlearning on MultiNLI
Table 16 shows the result for group-robust unlearning on MultiNLI (Williams al. 2018) using an unlearning ratio of 0.5. REWEIGHT allows successful recovery of the original group accuracy (-0.5), while preserving the test accuracy (-0.6). Compared to PRETRAIN, the EO are worse (14.4 vs. 16.2) but better compared to plain RETRAIN (16.2 vs. 19.3), thus, REWEIGHT partly recovered also the equalized odds. MIU achieves the best performance preservation overall by scoring the highest GA alignment, both when using (96.0) and not using (95.8) REWEIGHT. Particularly, MIU is less influenced by REWEIGHT, and most of the alignment comes from its unique loss formulation. SCRUB (Kurmanji et al., 2022]] achieves the second-best result without REWEIGHT (95.3), but struggles when used (92.5), as forget accuracy unexpectedly grows high values (82.1). Instcad, SALUN (Fan et al.] 2024b) scores the best avg. gap after MIU (95.9), but shows high variance and low results without REWEIGHT.. Overall, MIU is the most consistent among the two configurations and generally shows the lowest variance across metrics.
B.5 Extended ablation study
Scction 4.5 shows comprehensive ablation of MIU's components. However, Tab. limits the analysis only the UA, GA, and avg. gap to reduce space usage. Thus, Tab. reports all metrics investigated for completeness. Although Tab. metrics are limited, chose subset that shows great variance along
21

---

Published in Transactions on Machine Learning Research (12/2025)
Table 12: Group-robust machine unlearning in FairFace (Karkkainen & Joo. 2021) with 0.5
unlearning ratio. We build the forget set by sampling data points from single group. The unlearning ratio is set to 0.5. We compare MTU against L1-SPARSE (Jia et al. 2023), SALUN (Fan et al.
2024b), and
SCRUB Kurmanjii et al. 2024 The avg. gap is computed against RETRAIN + REWEIGHT.
method
UA
TA
MIA
EO
GA
avg. gap
PRETRAIN
65 65.620.7
79.041.2
57.2+0.4
0.2+0.11
5.4=1.7
71.2=2.4
RETRAIN
66.8100.4
$7.813.3
56.5.10.1
0.9.10.2
9.2.0.9
58.7
RETRAIN
66.720.2
69.340.5
56.740.2
0.7:0.5
5.6=1.5
69.6=0.7
LI-SPARSE (Jia a1
2023
0+:033
74.1+1.2
56.940.55
0.240.1
6.1=0.7
69.1-0.7
98. 98.3+0.2
SALUN (Fab et 20201
66 0.44
66.643.4
55.9+0.6
0.3+0.1
9.0=0.5
60.3=2.4
97.1+1.1
SCRUB Kurmanji i et al.
2024
66 0.1
65.4111.6
56.71:0.7
1.0.0.00
9.9
61.3-225
97.0.10.5
MIU
66.7:10.2
7.27.112
57.210.77
0.3:10.0
6.0
66.1-:.4
98.1
LI-SPARSK JJi a1 2023
64.440.1
72.942.1
56.0+-0.9
0.340.1
6.1-2.1
67.0-6.8
97.340.3
SALUN Fan ct 2024h
65.1110.4
69.816.33
54.810.6
0.3:10.2
6.6.2.1
63.7
97.210.4
SCRUB Kurmanji et al
2024
66.74:0.1
73.4+22.
57.25:0.5
0.7+00.3
6,2-1.1
70.2-2.7
98.71007
64.7+0.3
71.642.8
67.130033
0.3+0.22
5.8=0.4
70.3=1.6
98.7+0.8
Table 13: Group-robust machine unlearning in FairFace (Karkkainen & Joo, 2021 with 0.9
unlearning ratio. We build the forget set by sampling data points from single group. The unlearning ratio sot to 0.9. We compare MIU against L1-SPARSE (Jia al.
2023), SALUN (Fan al.
2024b), and
SCRUB (Kurmanji et al. 2024h. The avg. gap computed against RETRAIN + REWEIGHTT.
method
RW
RA
UA
TA
MTA
BO
GA
1
PRETRAIN
66.010..0
77.542..1
56.6110.4
0.2:10.0
5.521.1
69.0-2.8
RESRAIN
67.310.6
38.
06.01:0.4
2.7.0.3
23.1
37.1=2.1
RETRAIN
67.1#0.4
33.6#1.2
56.6+0.4
1.840.1
11.9111
53.9=0.7
LI-SPARSE (Jia al 2023
64.510.2
37.14:2.1
55.210.66
0.4::0.1
13.020.7
51.0=1.9
97.710.33
SALUN Fan et a.. 2024h
65.7110.5
46.5.16.2
53.9110.1
0.5.10.1
15.311.0
4..8-
95.2.11.8
SCRUB
2024
60.24
52.74:4.4
63.31:0.5
2.4.:0.7
15.611.4
48.7=4.9
95.7:10.6
MIU
68.2.10.3
64.41225
56.5.10.5
0.5.10.1
10.410.7
56.8-..6
97.0111.0
LI-SPARSE (Jia
2023
61.020.4
74.5#330
55.94004
0.3:0022
5.6=0.6
69.8=4.6
91.911155
SALUN (aan et al 2021h
65.520.5
66.
56.3+00.2
0.540.3
7.2=1.2
62.8=1.9
94.911.7
SCRUB Kurmanji
65.54335
54.510.3
1.310.1
9.5-0.9
64.4-2.7
MIU
64.7:5022
67.1
56.7400.2
0.5:50.1
8.8=0.2
63.5=1.6
94.9:10.6
Table 14: Group-robust machine unlearning in FairFace (Karkkainen & Joo, 2021 by sampling
from groups. We build the forget set by sampling data points from groups. The unlearning ratio is set to 0.5. We compare MIU against L1-SPARSE (Jia et al.
2023 SALUN (Fan et al.
2024b and SCRUB
Kurmanji et al. 2024). The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap
PRETRAIN
64.610.55
81.5.0..5
57.410.3
0.4.0.00
1.110.1
72.5
RETRAIN
66.5+0.55
60.1
55.44:0.4
1.100.1
5.620.4
60.4=1.7
RETRAIN
64.41.1.
72.112.2
56.3.10.3
0.5.1.00
2.0.0.6
71.8
LI-SPARSR Jia et al.
2023
63.5410.6
69.943.9
55.3410.1
0.5:10.0
2.621.0
64.4
9.
SALUN Fan
64.31..55
64.6:11.2
54.010.11
0.210.11
3.620.3
59.7=1.5
96.0:10.6
SCRUB Kurmanj al. 2024
67.2
74.34-0.7 7
56.9-10.2
0.3.0..1
1.820.7
65.6-0.6
97.7-10.3
66.3+0.4
74.2:10.4
66.8+00.5
0.340.11
1.720.4
65.7=0.6
97.920.4
LI-SPABSR
Jia et al.
2023
63.74.0.1
75.2410.6
56.3.10.1
0.2.10.1
1.320.3
69.611.2
98.6:10.2
SALUN al. 202411
03.7400.8
7..44:155
65.540.4
0.440..0
2.3-0.3
69.0-1.6
98.4+10.2
SCRUB Kurmanj
2024
66.710.4
80.5.10.1
$7.410.55
0.4.10.1
1.7.0.3
71.3
97.410.33
MIU
63.420.4
73.2:20.3
56.7450.3
0.4:0..1
1.3=0.5
70.3=1.0
99.0::0.1
22

---

Published in Transactions on Machine Learning Research (12/2025)
Table 15: Group-robust machine unlearning in FairFace (Karkkainen & Joo.
2021 by sampling
from 25 groups. We build the forget set by sampling data points from 25 groups. The unlcarning ratio is
set to 0.5. We compare MIU against L1-SPARSE (Jia et al.
2023), , SALUN (Pan et al.
2024b), and SCRUB
Kurmanji et al. 2024 The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
PRETRAIN
65. 00.13
70.640.55
56.940.3
0.340.1
1.60066
62.2-1.4
RETRAIN
1.0
55.5.2..1
54.8100.7
0.8:1.11
1.940.3
56.0=2.1
RETRAIN
66.110.6
60.5.10.7
65.5:10.4
0.7::0.1
2.3=0.3
60.5=0.5
L1-SPARSE Jia et al
2023
65.5.10.8
61.810.33
55.5.10.5
0.5.10.1
1.620.8
87.0=0.9
98.7110.2
SALUN et al. 2024h
64.5 0.5
60.043.8
00.241.0
0.540.11
1.1004
67.0=3.8
98.0+0.77
SCRUB Kurman
20241
06.7 7.66
62.4+1.0
55.4+-0.3
0.3+0011
1.7=0.9
35.2=0.6
98.1+0.2
MIU
66.8 0.2
64.8::0.5
66.42:0.5
0.3:0.1
1.7=0.7
57.5=1.1
98.3:0..2
L1-SPARSE (Ji ct al
2025
6..3
66.7.10.4
65.81.0.4
0.4.10.1
2.1
1.0
98.3100.3
SALUN (Fan et al 2021b
62.5+11.4
64.6422.0
54.6+0.88
0.340.1
2.2=0.4
60.1=2.0
98.0+0.3
SCRUB Kurman MIU
2024
65.3 00.3
71.4+0.6
67.0+-0.3
0.2:0.00
1.9=0.9
63.7=0.7
97.1#0.1
66.2 1.6
63.0+1.9
54.3+0.6
0.9::0.3
3.3=0.9
37.6=3.3
98.3+0.1
Table 16: Group-robust machine unlcarning in MultiNL (Williams et al., 2018) with 0.5 un-
learning ratio. We build the forget set by sampling data points from a single group. The unlearning ratio is set to 0.5. We compare
MIU against LI-SPARSE (Jia et al. 2023), SALUN (aan al. 2024b). and
SCRUB (Kurmanji et al. 2024h.
The avg. gap is computed against RETRAIN + REWEIGHT.
method
RW
RA
UA
TA
MIA
EO
GA
gap 1
PRETRAIN
94.742..5
81.848.4
1
31.4=0.9
14.4
59.3-2.7
RETRAIN
1.19
33.0+3.7
82.3+0.1
47.6=3.9
19.3+0.6
50.8=3.1
RETRAIN
95.3
59.810.7
81.710.0
43.3-2.3
16.210.3
58.8=2.0
LI-SPARSI Jia et
2023
84.8+2.0
46.6+44.8
79.0+0.7
34.1=2.3
23.3+114
42,0=2.1
10.1
SALUN Fan ct a. 2024
76.411:0.33
36.511259
63.84:21.4
36.11225.6
20.2110.8
35.1122..
80.8-1 19.0
SCRUB Kurman MIU
2024
9.94+1.9
56.844.66
82.4+-0.2
46.7=5.5
22.6+11.7
47.6=3.3
1.3
99.2
60.3:1.6
79.2::0.2
$1.922.8
18.8:0.7
55.6=2.0
95.820.3
LI-SPARSH (Jia et al
2023
84 84.9+22.0
35.443.8
79.0+00.6
31.7+14
19.3
19.2=1.6
93.0#11.2
SALUx Faa et al 2021b
96 0..
34.3+3.6
78.3+0.2
48.1-2.0
18.5+19
51.9-2.6
95.9 3
SCRUR Kurmanji al. MIU
2024
82.147.8
82.41.0.2
32.2-8.9
14.04:3.8
62.4-6.3
92.541..8
99.910.0
67.0:12.6
81.5.0.22
51.1-2.
18.010.4
56.6 1.2
96.0110.8
different components and is more interesting to evaluate. For instance, EO variations are more muanced
compared to GA (e.g., it drops by 4.5 in FairFace The MIA shows subtle oscillations in CelebA (Liu
(Karkkainen & Joo 2021 while GA grows by 11.1).
CelebA
Waterbirds
FairFace
et al. 2015) and FairFace (Karkkainen Joo (2021)
Aug. Chp (1)
99
while drops when adding the colibration term in
90
Waterbirds (Sagawa et al., 2020), getting closer to
RETRAIN + REWEIGHT value (59.5% of MIU vs.
5
X
53.6% of RETRAIN + REWEIGHT). Finally, TA and RA are relatively stable across components, with RA showing a negative trend (64.7% using all three components vs. 65.6% of PRETRAIN).
Figure 8: Ablating parameter X. MIU avg. gap when varying parameter in CelebA (Liu et al,, 2015). Waterbirds (Sagawa et al. 2020 and FairFace [Karkkainen & Joo 2021). While À is optimal in
Fig. instead, reports the ablation study on pa- rameter X. We show results for À E {0, 1, 5, 10} and average each experiment over three different seeds. Overall, all three datasots benefit from the calibra- tion term. However, while CelebA (Liu el al. 2015) and FairFace (Karkkainen & Joo 2021) ) achieve bet- ter results when À= = (i.c., an avg. gap of 99.2 and 98.7), Waterbirds (Sagawa et al., 2020) minimizes the gap when À 10 (i.c., an avg. gap of 96.9). Nonetheless, results are generally stable, and when in doubt, we suggest setting x= it always shows alignment improvement with RETRAIN REWEIGHT.
CelebA (Liu et al., 2015) and FairFace Karkkainen & 2020 benefits
Joo 2021). Waterbirds (Sagawa et al.
from higher lambdas.
23

---

Published in Transactions on Machine Learning Research (12/2025)
Table 17: MIU ablations. We compute MIU ablations on each of the three investigated datasets. From left to right, we report the investigated dataset, the retaining term, the unlearning term, the calibration term, and REWEIGHTT. We measure performance using all metries. The configuration that corresponds to MTU + REWFIGHT is highlighted.
dataset
Eq.
Eq.
Eq.
6
RW
RA
UA
TA
MIA
EO
GA
avg. gap 1
X
96. 96.4110.1
35.51.0.9
95.910.00
1.110.1
26.71:0.9
35.41:2.3
97.5:10.5
CelebA
1+0.1
36.3+0.9
95.9+0.1
1.20..3
26.1+0.8
36.342.0
98.0#0.4
95.8+0.5
27.8+3.7
95.3+0.4
0.9-0.22
25.3+0.8
28.5+6.9
95.2422.5
90.3+0.2
13.220.6
96.0=0.0
1.240.1
24.0=0.5
41.3#1.1
99.0=0.2
x
100.0:0.0
47.647.3
85.040.6
73.84:3.4
32.3
$1.110.6
92.5:14.6
Waterbirds
100.0::0.0
53.617..7
86.111.0
58.3188.9
28.31:1.7
53.8122.6
95.3.10.7
93.0133.3
16.719.4
80.3.12.9
64.3112.7
3..8
16.8.8.9
$1.9.15.5
99.9
54.8:14.7
85.8+0.7
59.5+11.11
28.3+229
53.743.8
96.9:116
65 2+0.1
03.1116
56.9+0.3
0.3+0.0
10.3#0.8
59.2+1.9
96.1+0.8
66.74:0.2
1.2
87.240.7
0.3+0.0
6.0=2.0
66.14
18.144
FairFace
59. 14.13311 1
87. 1+6.8
54.5#18
0.0:0..0
3.1=0.5
81.126.2
93. 03.0#3..1
64.7.00.3
71.612.8
37.110.3
0.310.2
5.8.0.44
7..3:11.6
98.7.10.8
Table 18: Additional Fairness Metrics. datasets (using the same splits as Tabs. 1 to
Fairness metrics are computed on each of the three investigated From left to right, we report the method, DP, EP, EO, and
WG. MIU + REWEIGHT is highlighted.
method
DP CelebA
EP
EO
WG
(Liu
2015)
PRETRAIN
18.7
15.3
21.2
40.6
RETRAIN
18.9 (0.1)
51.0 (5.7)
27.0 (2.9)
34.4 (-6.1)
RETRAIN HWW
18.8 (0.1)
44.6 (-0.7)
23.9 (-0.3)
41.3 (0.7)
MIU
18.8 (0.1)
45.0 (-0.3)
24.0 (-0.1)
41.3 (0.7)
Waterbirds
(Sagawa et al.
2020)
PRETRAIN
20.6
30.1
26.1
56.6
RETRAIN
23.2 (2.6)
43.4 (7.3)
30.1 (1.3)
49.4 (-7.2)
RETRAIN + aW
21.3 (U.9)
40.6 (4.5)
28.3 (2.2)
51.6 (-5.0)
MIU
22.9 (2.3)
35.1 (2.0)
28.3 (2.2)
53.7 (-2.9)
Face
Karkkainen & Joo
2021
PRETRAIN
2.0
1.6 17.9 (10.3)
5.4
9.4
RETRAIN
(3.3)
9.2 (3.8)
8.3 (-1.1)
RETHAIN aW
3.8 (1.8)
7.6 (0.0)
5.6 (0.2)
6.1 (-3.3)
MIU
1.1 (-0.9)
8.0 (0.4)
5.8 (0.4)
16.3 (6.9)
B.6 Additional Fairness Metrics
To further study the method's fairness after unlearning. We investigate three additional fairness metrics alongside the Equalized Odds (EO) HHardt et al. 2016), namely, Demographic Parity (Kusner et al. 2017 7
Equal Opportunity (Hardt et al. 2016), and Worst Group Accuracy (WG) (Sagawa et al. 2020 Liu et al, 2021). To satisfy the Demographic Parity, the model's probability of outputting a positive prediction must be independent of the sensitive attribute. Therefore, we measure it DP IP(Y
A 0) P( A 1)}- Similarly, to satisfy Equal Opportunity (EP), the model true positive rate must be independent of the sensitive attribute: EP IP(Ý = - = 1, 0) P( 1 Y A 1)). Instead, the Worst Group Accuracy measures the average accuracy of the worst group of the test set. Thus, the Worst Group Accuracy is computed as: WG = min { D E.D
2 1 [[h o fx) = =il]. where D is the subset of images of group 9: of the test set. The lower the first two metrics (DP and EP), the better the model fairness, while the higher the WG, the higher the model robustness.
Table 18 reports our evaluation using the additional metrics on CelebA Liu et al. 2015 Waterbirds Sagawa et al. 2020), and FairFace Karkkainen & Joo, 202I). While plain model retraining generally shows performance degradation on almost all fairness metrics (c.g., +5.7 EP, +2.9 EO, and -6.1 WG, in CelebA
Liu et al. (2015) ), using REWEIGHT recovers the original model fairness and overall robustness (i.c., -0.7 EP, 0.3 E.O, and +0.7 WG, in CelebA (Liu et al. 2015)). Similarly, the proposed MTU preserves PRETRAIN
24

---

Published in Transactions on Machine Learning Research (12/2025)
robustness by approximating RETRAIN + REWEIGHT (i.e., -0.3 EP, -0.1 EO, and +0.7 WG, in CelebA et al. 2015
Liu
25


Reviews:
[{"review_id": "tbvOORUYVa", "reviewer": "Reviewer_KH27", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a problem of group-robust machine unlearning. It first proposes to use a reweighting strategy to maintain robustness in exact unlearning (model retraining), and also introduces an approximate unlearning method based on mutual information minimization called MIU. The authors conduct experiments comparing MIU to prior works in unlearning on image classification datasets (CelebA, Waterbirds, FairFace). MIU achieves strong results in both preserving model generalization and closely matching exact unlearning model properties.\n\nOverall, the proposed approximate unlearning method MIU is well-motivated and shows strong empirical performance. I also appreciate the presence of ablation studies on MIU.\nHowever, I have a few concerns which would require a revision of the paper:\n(1) clearly defining goals and the problem setting of machine unlearning and group-robust unlearning, (2) overall presentation clarity, and (3) over-claiming contribution of a novel exact unlearning method.", "claims_and_evidence_yn": "No", "claims_explanation": "1. Issue with problem formulation of machine unlearning and group-robust machine unlearning setup.\n\na) For machine unlearning in Section 3.1: There are contradictions in whether the goal is (1) to forget $D_f$ while maintaining performance on the remain and test sets, or (2) to approximate the exact SGD retraining on the remain set as exactly as possible. These are two distinct goals, and this needs to be clarified.\n\nb) In Section 3.2, for group-robust machine unlearning, similarly, is the goal to unlearn $D_f$ and (1) make the model as group robust as possible / maximum worst-group accuracy, (2) make it close to exact retraining with SGD on the remain set, (3) make it close to exact retraining with group robust learning on the remain set, or (4) minimize the accuracy losses on the dominant group in $D_f$, (5) make the accuracy of specific dominant group in $D_f$ close to the model with exact retraining …, etc? Please make this very clear in the problem statement. \n\n2. The novelty of the exact retraining method is exaggerated.\n\nThe method described in Section 3.3 is not a novel contribution, and it is misleading to present it as such. Training with imbalance groups e.g. using reweighting is exactly what standard group robustness is doing, and it is not specific to unlearning [see Idrissi et al, Chaudhuri et al, among others]. If we just have $D_r$ with some group underrepresented, we know the strategist to get a group robust model with it from group robustness literature. I suggest that the authors present reweighting as a baseline for robust exact retraining but I don’t see a technical contribution there if I understand everything correctly.\n\n\n[Idrissi et al https://arxiv.org/abs/2110.14503] \n\n[Chaudhuri et al https://arxiv.org/abs/2205.11672]", "audience_yn": "Yes", "audience_explanation": "The paper proposes an interesting problem formulation where we want to maintain group robustness during machine unlearning. The proposed approximate unlearning algorithm is novel and claims around the method MIU are empirically supported. When the authors incorporate the suggested changes and address the concerns regarding the formulation and novelty of exact retraining, the paper will be a great addition to TMLR, for both machine unlearning and group robustness communities.", "requested_changes": "1. (Most important) Improving clarity of the goals as discussed in “Issue with problem formulation of machine unlearning and group-robust machine unlearning setup.” above. Reframe the exact training method with reweighting as a baseline, not novel contribution, as discussed in “The novelty of the exact retraining method is exaggerated.” above.\n\nAdditional questions about the group-robust unlearning assumptions:\n- Do you assume uniform or non uniform group distribution in the original train dataset?\n- You assume that the forget set is skewed towards one group: but what if that group was overrepresented and now the retain set is balanced — does it change anything for the problem formulation? Isn’t it more important what group distribution of the retain set is, than what group distribution the forget set has? \n- Do you assume that your original Pretrain model is group robust or not (i.e. is it trained with SGD, reweighting or GDRO)? E.g., in Section 3.4, page 5 “While preserving original group robustness” sounds like you do assume the original model is group robust?\n\n\n2. Retraining baseline: the argument about Group DRO inflating forget set accuracy is confusing, since retraining and original training should align in methodology. It would be helpful to have baselines of Pretrain and Retrain models which align in training strategy. Specifically, have a Pretrain and Retrain pair both trained with SGD, GDRO, or reweighting.\n\nThe paper doesn’t have line numbers, so I’ll copy over a paragraph which was confusing to me:\n“As a byproduct of strongly optimizing worst-group accuracies, we notice that group-DRO (Sagawa et al., 2020) can also increase the forget set accuracy. This issue makes approximate unlearning evaluation more difficult if Retrain + group-DRO is used as the gold standard. Assuming a hypothetical original forget-set accuracy of 70%, if an approximate unlearning algorithm targeting such a gold standard leads to higher accuracy (e.g., 80%), then was the knowledge unlearned if forget-set accuracy increased?”\n\nPlease clarify this, since it sounds like GDRO leads to better generalization, and not using an algorithm which leads to better generalization / group robustness seems really confusing.\n\n3. Overall clarity, questions and suggested changes:\n* Please add error bars to the reported results. \n* Provide full experimental context for Figures 2, 3: training setup, data splits, target, models, group distributions, forget set details. What does y-axes shoes? Can you add group accuracies / average accuracy / WGA? What’s the difference between left and right subplot in Fig 3?\n* In Experiments section, which groups do you use for froget set?\n* Generally report worst-group accuracy (WGA).\n* Define all metrics / terms clearly: e.g., “average gap”, “unlearning ratio” (wrt group size, forget set size, or full dataset?).\n* Eq. (2): define $T_{\\psi}$.\n* Section 4.4: report group distribution and size of forget set.\n* typo: “This section describes the experimental protocol (Sec. 3.1)” -> should be 4.1.\n* Figures/Tables: indicate which group forms the unlearning set.\n* Ideally, if possible, change the CelebA attribute from “attractiveness” to a more neutral/academic choice like “smiling.”\n* Please clarify this claim: “Intuitively, increasing the sampling likelihood for partly unlearned groups rebalances the retain set group statistics to match those of the training dataset.”\n* Please elaborate on this observation at page 11: “Instead, all algorithms struggle at 0.1 unlearning ratio in Waterbirds (Sagawa et al., 2020), where the small forget set size causes UA fluctuations and affects the BatchNorm (Ioffe, 2015) estimation”.\n* Section 4.4: what is the group distribution and size of the forget set?", "broader_impact_concerns": "Ideally, if possible, change the CelebA attribute from “attractiveness” to a more neutral/academic choice like “smiling.” (see Requested Changes above)", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QcdU4Jbool", "reviewer": "Reviewer_vvqX", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies group-robust machine unlearning, where forget requests are uneven across demographic groups and can degrade accuracy for dominant groups; it formalizes this setting (assuming group labels) and considers both exact and approximate unlearning. It proposes (i) reweighting the retain-set sampling during retraining to preserve the original group robustness better than group-DRO, and (ii) MIU, an approximate method that minimizes mutual information between features and group labels with a calibration term; on CelebA, Waterbirds, and FairFace, these approaches better maintain group/test accuracy than prior unlearning methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "This paper has a list of clear strengths:\n- The problem is significant and novel, with a clear, well-motivated formulation.\n- The paper is well-written and easy to follow.\n- The method is intuitive, and the demo-code is provided for reproducibility.\n\nHowever, potential weaknesses for this paper are:\n- The method assumes access to oracle group labels, which is understandable for a first work, but should be complemented by discussion or simulations of mitigation strategies for missing group labels from the standard spurious-correlation literature.\n- The evaluation is limited to vision datasets; adding text benchmarks (e.g., MultiNLI, CivilComments) would better demonstrate generalizability.\n- The paper lacks a hyperparameter sensitivity analysis.", "audience_yn": "Yes", "audience_explanation": "Clearly, researchers from both machine unlearning and (group)-robust learning will be interested in this paper.", "requested_changes": "Aforementioned in the weakness part.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NaaQIYHQxA", "reviewer": "Reviewer_xfyW", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies the intersection of machine unlearning and group robustness, namely when the forget set consists primarily of data with a particular sensitive attribute. Benchmarks are provided which show that typical unlearning methods disproportionately harm group accuracy. Two methods are proposed to rectify this: in the exact unlearning setting, a sample reweighting scheme is shown to be sufficient, whereas in the approximate unlearning setting, a mutual-information-based algorithm called MIU is proposed. Experiments and ablations are then provided across three vision datasets.\n\n**Summary of strengths:**\n1. The problem is well-motivated by recent scholarship in AI ethics.\n2. This paper has potential to bring two distinct research communities closer together.\n3. The proposed MIU method is clearly explained, and the analysis and ablation studies are well-done.\n\n**Summary of weaknesses:**\n1. The benchmarking study in Section 3.2/App. B.1 should be expanded.\n2. The claim of an iid forget set assumption in the literature is insufficiently supported.\n3. The attractiveness classification task is ethically concerning.", "claims_and_evidence_yn": "No", "claims_explanation": "For the most part, the claims are well-supported, especially the analysis and ablation studies in Section 4. There are two claims which need more evidence, detailed below.\n\n**Retain Set Group Distribution Harms Group Accuracy**\n\nThe following claims in Section 3.2 are insufficiently supported: “The more samples of a group are removed, the lower the resulting group accuracy. Intuitively, removing non-uniformly distributed data changes the retain set group distribution, ultimately harming the model’s generalization performance on the dominant group of the forget set.”\n\nFirst, the supporting experiments in Figure 7 consist of learning only a single group in each of the three datasets, and each of these are minority groups/underrepresented in the training dataset (to my knowledge). It would be helpful to repeat these experiments for more groups (say, 4) in each dataset, distributed among both majority and minority groups.\n\nSecond, the claim “the more samples of a group are removed, the lower the resulting group accuracy” may not hold in situations where samples of *multiple groups* are being unlearned (c.f. the results in Section 4.4/Figure 6). Consider the following example: dataset D has a majority group A with 900 samples and a minority group B with 100 samples. If 100 samples from group A are unlearned, the experiments suggest the accuracy on group A will decrease. But if 200 samples from group A are unlearned and 100 samples from group B are unlearned, I would hypothesize it is possible for the accuracy on group A to *increase*, as the retain set would comprise solely group A.\n\nThird, it is unclear whether the distribution of data across the groups in the retain set is the primary causation of the harm to generalization on the dominant group of the forget set (as opposed to, for example, less data overall in the retain set). To support this claim, I would expect controlled experiments varying the retain set group distribution while keeping confounding factors constant, with, e.g., correlation coefficients between group balance metrics and GA. Since this claim is not critical to the narrative, I may advise reducing the claim for this third point.\n\n**IID Forget Set Assumption**\n\nIt is unclear to what extent the iid forget set assumption is actually prevalent in the machine unlearning literature, and it would be helpful to clearly delineate which methods specifically rely on this assumption. To my understanding, it seems that many of the cited methods in Section 2 actually allow for *arbitrary* forget sets, rather than requiring forget sets to be drawn randomly over the dataset. For example, SCRUB [1] studies two different settings, one where an entire class is forgotten and one where iid samples from a class are forgotten, and neither of these require the forget set to be sampled iid across all groups or the entire dataset.\n\nWith the tenuous nature of the iid forget set assumption in mind, I would argue that the primary contribution of this paper is actually its revision of the machine unlearning objective function. While previous unlearning methods prioritize metrics such as average accuracy on the retained data and average generalization [1], this work proposes to use group accuracy instead. Indeed, the calibration term of MIU explicitly enforces preservation of the base-model group accuracies, particularly on the worst-group and forget-set group.\n\nOverall, I would encourage the authors to clarify their primary contribution, specify the extent of the iid forget set assumption, and consider repositioning the work as a revision of the machine unlearning accuracy objective.\n\n[1] Kurmanji et al. “Towards Unbounded Machine Unlearning.” NeurIPS 2023.", "audience_yn": "Yes", "audience_explanation": "A strength of this paper is its potential to bring two distinct research communities closer together, namely in machine unlearning and group robustness. Each community would likely be interested in this paper: the machine unlearning community for its interesting challenge and proposal of robustness-preserving unlearning algorithms, and the group robustness community for the application of debiasing techniques in a new application area. There is potential for future work in this direction, especially towards the question at the end of Section 3.3 -- “Was the knowledge unlearned if forget-set accuracy increased?” -- which may motivate sample-by-sample metrics of unlearning and examination of the interaction between unlearning and Group DRO.", "requested_changes": "**Critical for acceptance**\n1. Expand the benchmarking study (see Claims).\n2. Clarify the primary contribution of the paper with respect to the iid forget set assumption (see Claims).\n3. Reconsider the attractiveness classification task (see Broader Impact Concerns).\n\n**Would strengthen the work:**\n1. Formally define the “unlearning ratio”. I assumed that it is the proportion of samples from a particular group which are to be unlearned (e.g., the forget set is 100% waterbirds on land, which comprises 90% of waterbirds on land in the train set), but it could also be interpreted as the proportion of samples from a particular group in the forget set (e.g., the forget set is 90% waterbirds on land and 10% other groups).\n2. Idrissi et al. [1] extensively discuss a version of REWEIGHT, called by them RWG, but this paper only mentions their subsampling method in Section 2. It would be helpful to contrast REWEIGHT with RWG and discuss how this paper’s insights compare to [1]. In general, contextualization with the literature on reweighting methods would be helpful given REWEIGHT’s massive importance in obtaining good performance (c.f. Figure 4).\n3. It took me a couple re-reads of the paper to understand that the most desired metric, which MIU is designed to maximize, is the *dominant forget-group accuracy (GA)*. The presentation is somewhat muddled by the discussion around Group DRO (which actually targets WGA) and the 5 other metrics in the tables. I would recommend bringing the definition of GA (Eq. 11) out of the appendix into Section 3 and increasing clarity around targeted metrics. \n4. Various improvements to the writing (see Additional Comments).\n\n[1] Idrissi et al. “Simple data balancing achieves competitive worst-group-accuracy.” CLeaR 2022.", "broader_impact_concerns": "No broader impact statement is provided.\n\nI have an ethical concern regarding the primary application on the CelebA dataset, namely the use of machine learning to classify facial attractiveness. Attractiveness classification is a fundamentally problematic and outdated task which reinforces culturally biased and highly subjective standards of beauty. It risks amplifying social inequities with respect to gender, age, race, etc and does not align with modern standards of responsible AI development -- especially for a paper which focuses on group fairness. Moreover, substantial scholarship exists regarding the biased nature of the “attractive” attribute in CelebA specifically, e.g., [1, 2]. I strongly encourage the authors to reconsider the inclusion of this task; one alternative is to keep CelebA but switch the target attribute to hair color (which is more commonly used in the spurious correlations literature).\n\n[1] Prabhu et al. “Covering up bias with Markov blankets: A post-hoc cure for attribute prior avoidance”. ICML 2019 Workshop on Invertible Neural Networks and Normalizing Flows.\n\n[2] Rajabi et al. “Through a fair looking-glass: mitigating bias in image datasets.” HCII 2023.", "additional_comments": "1. The color-coding of the target and protected attributes is a nice touch but somewhat inconsistent, e.g., it is missing from the appendix.\n2. I found the heavy use of both the terms “retain” and “retrain” confusing. I often had to re-read sentences to confirm the context, and it generally interrupted the flow of the writing. I would recommend changing one of these terms to something more distinct.\n3. Typo in Section 3.3: “reweighing” should be “reweighting”?\n4. In Algorithm 1, it would be nice to explain some acronyms in the caption. I assumed “dl” stands for “dataloader” and “mine” stands for mutual information neural estimation.\n5. I recommend removing or rewording the following sentence: “The popularity of group-DRO (Sagawa et al., 2020) in group-robust optimization suggests it should minimize drops in the forget-set dominant group accuracy.” There is no justification for the popularity of a method being correlated with its performance on a particular task.\n6. The bolding in the tables doesn’t make much sense, as the bolded numbers are often neither the lowest or highest in each column. It would be helpful to describe what is bolded in the caption and keep it consistent throughout the paper (bolding is also missing in the appendix tables). Relatedly, it would be helpful to include up-arrows and down-arrows in the tables to signify whether each metric is preferably large or small.\n7. Section 3.4 is very well-written; I appreciate the step-by-step motivation of each term and inclusion of theoretical justification when appropriate. The pseudocode is also helpful.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)