Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks
Authors: Piyush Tiwary
Atri Guha
Subhodip Panda
Prathosh AP
Venue: Accepted by TMLR ()

Abstract: Owing to the growing concerns about privacy and regulatory compliance, it is desirable to regulate the output of generative models. To that end, the objective of this work is to prevent the generation of outputs containing undesired features from a pre-trained Generative Adversarial Network (GAN) where the underlying training data set is inaccessible. Our approach is inspired by the observation that the parameter space of GANs exhibits meaningful directions that can be leveraged to suppress specific undesired features. However, such directions usually result in the degradation of the quality of generated samples. Our proposed two-stage method, known as 'Adapt-then-Unlearn,' excels at unlearning such undesirable features while also maintaining the quality of generated samples. In the initial stage, we adapt a pre-trained GAN on a set of negative samples (containing undesired features) provided by the user. Subsequently, we train the original pre-trained GAN using positive samples, along with a repulsion regularizer.  This regularizer encourages the learned model parameters to move away from the parameters of the adapted model (first stage) while not degrading the generation quality. We provide theoretical insights into the proposed method. To the best of our knowledge, our approach stands as the first method addressing unlearning within the realm of high-fidelity GANs (such as StyleGAN). We validate the effectiveness of our method through comprehensive experiments, encompassing both class-level unlearning on the MNIST and AFHQ dataset and feature-level unlearning tasks on the CelebA-HQ dataset. Our code and implementation is available at: https://github.com/atriguha/Adapt_Unlearn.

Published in Transactions on Machine Learning Research (02/2025)
Adapt then Unlearn: Exploring Parameter Space Semantics for Unlearning in Generative Adversarial Networks
Piyush Tiwary
pigushtioorry@iiscoo..h
Department of Electrical Communication Engineering Indian Institute of Science
Atri Guha
atri foi_2001cc08@iitp.com
Department Electrical Engineering Indian Institute of Technology Patno
Subhodip Panda
subhodipp wwwpcdippppiccc...oo
Department of Electrical Communication Engineering Indian Institute of Science
Prathosh A.P.
prathosh @risc.
Department Electrical Communication Engineering Indian Institute of Science
Reviewed on OpenReview: https: // hpt///vepnrre....// forun? id= JAHEBÁvObO
Abstract
Owing to the growing concerns about privacy and regulatory compliance, it desirable to regulate the output of generative models. To that end, the objective of this work is to prevent the generation of outputs containing undesired features from a pre-trained Gener- ative Adversarial Network (GAN) where the underlying training data is inaccessible. Our approach is inspired by the observation that the parameter space of GANs exhibits meaningful directions that can be leveraged to suppress specific undesired features. How- ever, such directions usually result in the degradation of the quality of generated samples. excels at unlearning
Our proposed two-stage method, known as 'Adapt-then-Unlee such undesirable features while also maintaining the quality of generated samples. In the initial stage, we adapt a pre-trained GAN on a set of negative samples (containing unde- sired features) provided by the user. Subsequently, we train the original pre-trained GAN using positive samples, along with repulsion regularizer. This regularizer encourages the learned model parameters to move away from the parameters of the adapted model (first stage) while not degrading the generation quality. We provide theoretical insights into the proposed method. To the best of our knowledge, our approach stands as the first mothod addressing unlcarning within the realm of high-fidelity GANs (such as StyleGAN). We validate the effectiveness of our method through comprehensive experiments, encom- passing both class-level unlearning on the MNIST and AFHQ dataset. and feature-level unlearning tasks on the CelebA-HI dataset. Our code and implementation is available https://github. com www./ttiiuuaaaaaaaaapapoao.oo//a Unlearn.
1 Introduction
1.1 Unlearning
Recent advancements deep generative models such as Generative Adversarial Networks (GANs) (Good- fellow et al., 2014; Arjovsky et al., 2017; Karras et al., 2018b;a; 2020) and Diffusion models (Ho et al., 2020:

---

Published in Transactions on Machine Learning Research (02/2025)
Song& & Ermon, 2019; Song et al., 2021) have showcased remarkable performance in diverse tasks, from gen- erating high-fidelity images (Karras et al., 2018 20; 2020; 2021) to text-to-image translations (Ramesh et al., 2021; 2022; Rombach et al., 2022). Consequently, these models find application in various fields, includ- ing but not limited to medical imaging (Celard et al., 2023; Varoquaux & Cheplygina, 2022; Tiwary et al., 2024a), remole sensing (Ball el al., 2017; Adegun et al., 2023), hyperspectral imagery (Jia et al., 2021; Wang et al., 2023), and many others (Choudhary et al., 2022; Yang & Xu, 2021; Liu et al., 2021; Tiwary et al., 2024b). However, the extensive incorporation of data with possible undesired features or inherent biases cause these models to generate violent, racial, or explicit content which poses significant concerns (Tommaai et al., 2017). Thus, these models are subject to regulatory measures (Voigt. & dem Bussche, 2017; Gold- man, 2020). Identifying and eliminating these undesired features from the model's knowledge representation poses a challenging lask. The framework of Machine Unlearning (Xu et al., 2020; Nguyen et al., 2022b) tries to solve this problem by removing specific training data points containing undesired feature from the pro-trained model. Specifically, machine unlearning refers to the task of forgetting the learned information (Sekhari el al., 2021; Ma et al., 2022; Ye et al., 2022; Cao & Yang, 2015; Golatkar et al., 2021; 2020a; Ginart et al., 2019; Golatkar et al., 2020b), or erasing the influence of specific data subset of the training dataset from a trained model in response to user request (Wu et al., 2020a; Guo et al., 2020; Graves et al., 2021; Wu et al., 2022; 2020b; Chourasia & Shah, 2023).
The task of unlearning can be challenging because we aim to 'unlearn' a specific undesired feature without negatively impacting the previously acquired knowledge. In other words, unlearning could lead to Catas- trophic Forgetting (Ginart et al., 2019; Nguyen et al., 2022a; Golatkar et al., 2020b) which would significantly deteriorate the performance of the model. Further, the level of difficulty faced in the process of unlearning may vary depending the specific features of the data that one is roquired to unlearn. For example, un- learning a particular class (e-g. class of digit '9' in MNIST) could be relatively easier than unlearning more subtle feature (e.g. beard feature in CelebA). This is because the classes in MNIST are quite distinct and don't necessarily share correlated features. Whereas, in the CelebA (Liu et al., 2015) datasot, the feature of having& a beard is closely linked to the concept of gender. So, unlearning this subtle feature while retaining other correlated features such as gender, poses an increasingly difficult challenge. is important to mention that re-training the model from scratch without the undesired input data is not feasible in this setting due unavailability of the training dataset.
1.2 Motivation and Contribution
In this work, we try to solve the problem of unlearning undesired feature in pre-trained generative adversarial networks (GANs) without having access to the training data used for pre-training the GAN. We operate under the feedback- baabback-bbsedd unlearning framework, where we start with a pre-trained GAN. A user is given a set of generated samples from this GAN. The user chooses a subset of generated samples and identifics them as 'undesirable' (negative samples). The beedback-based approach is similar to RLHF LLMs or human-in- loop settings in general (Ziegler et al., 2019; Christiano et al., 2017; Lambert al., 2022). The objective of the process of unlearning is to prevent the generation of undesirable characteristics, identified by the user. We propose to unlearn the undesired features by following two-step approach. In the first step, we adapt the pre-trained generator to the undesired features by using the samples marked as undesired by the user (negative samples). This ensures that the "adapted" generator exclusively generates samples that possess the undesired features. In the next step, we unlearn the undesired features from the original GAN by using the samples that weren't marked as undesired by the user (positive samples). While unlearning, we add a repulsion loss that encourages the parameters of the generator to stay away from the parameters of the adapted generator (obtained from first step) while also making sure that the quality of generated samples does not deteriorate. We provide theoretical justification for the proposed method by using a bayesian framework. Particularly, we show that the proposed method leads to contrastive-divergend kind of objective desired for unlearning. We call the proposed two-stage process Croapt-thenUnleaane An overview of the proposed method is shown in figure 1 (a).
Our approach hinges in realizing interpretable and meaningful directions within the parameter space of a pre-trained GAN generator, as discussed in (Cherepkov et al., 2021). In particular, the first stage of the proposed method leads to adapted parameters that exclusively generate negative samples. While the
2

---

Published in Transactions on Machine Learning Research (02/2025)
ON 7 vv)
Bungs
Feedback
Hats
A
Negalive Samples
Positive Samples
Cats
Goyg
Gig
Gop
Dogs
Lidy
Lndopt
Gropel
(b)
Negative Adaptation
Unlearning
Madi Datiwst
Negezive Sumplo
Poaltive Sumploo
~ N
Grp
Pre-teained GAN
Negative Adepted GAN
Unlearnt GAN
Original GAN Sumples
Adaptal GAN Sumples Unlearse CAN Sumples
Stage-1: Nogative Adaptation Stage-2: Unlearning
(a)
(c)
Figure 1: (a) Block diagram of the proposed method: Stage-1: Negative Adaptation of the GAN to negative samples received from user feedback and Stage-2: Unlearning of the original GAN using the positive samples with repulsion loss. (b) Illustrating linear interpolation and extrapolation in parameter space for unlearning undesired features. We observe that in the extrapolation region, undesired features are suppressed, but the quality of generated samples deteriorates. (c) An example of results obtained using our method on Mixture of Gaussian (MoG) dataset, where we unlearn two centers provided in negative samples.
parameters of the original pre-trained generator generate both positive well as negative samples. Hence, the difference between the parameters of adapted generator and the paramters of original generator can be interpreted as the direction in parameter space that leads to decrease in the generation of negative samples. Given this, is sensible to move away from the original parameters in this direction to further reduce the generation of negative samples. This observation shown in figure (b). However, it's worth noting that such extrapolation doesn't ensure the preservation of other image features' quality. In fact, deviations too far from the original parameters may hamper the smoothness of the latent space, potentially leading to a deterioration in the overall generation quality (see last columns of figure 1 (b)). Inspired by this observation, during unlearning stage, we propase to train the generator using adversarial loss while encouraging the generator parameters to be away from the parameters of the adapted generator by employing a repulsion regularization.
We provide a visual illustration of the proposed method on Mixture Gaussian (MoG) dataset with eight centers in figure (c). The first column shows the original training dataset and the samples generated by the pre-trained GAN. The second column shows the negalive samples provided during feedback and the samples generated by the adapted generator. We can that the adapted generator exclusively generates samples from the negative modes of MoG. Lastly, in the third column, see the positive samples and the samples generated after unlearning the negative modes. We clearly observe that after unlearning (via the proposed method), the generator unlearns the negative modes and generates samples from the rest of the modes. This gives proof-of-concep: for the proposed method.
We summarize our contribution as follows:
We introduce a two-stage approach for machine unlearning in GANs. In the first stage, our method adapts the pre-trained GAN to the negative samples. In the second stage, train the GAN using
3

---

Published in Transactions on Machine Learning Research (02/2025)
repulsion loss, ensuring that the generator's parameters diverge from those of the adapted GAN in stage 1.
By design, our method can operate in practical few-shot settings where the user provides very small amount of negative samples.
We provide theoretical justification for the proposed method by showing that the proposed regular- ization leads to contrastive-divereenne kind of objectives appropriate for unlearning.
The proposed method is thoroughly tested on multiple datasets, considering various types of un- learning scenarios such as class-level unlearning and feature-level unlearning. Throughout these tests, we empirically observe that the quality of the generated samples is not compromised.
2
Related Work
2.1 Machine Unlearning
Unlearning can be naively done by removing the unwanted data subset from the training dataset and then retraining the model from scratch. However, retraining is computationally costly and becomes impossible if the unlearning request comes recursively for single data points. The task of recursively 'unlearning' i.c. removing information of a single data point in an online manner (also known as decremental learning) for the SVM algorithm was introduced in (Cauwenberghs & Poggio, 2000). However, when multiple data points are added or removed, these algorithms become slow because they need to be applied to each data point individually. To address this, (Karasuyama & Takeuchi, 2009) introduced newer type of SVM training algorithm that can elficiently update an SVM model when multiple data points are added removed simul- taneously. Later, inspired by the problem of protecting user privacy (Cao & Yang, 2015) developed efficient ways to delete data from certain statistical query algorithms and coined the term "machine unlearning". The works of (Ginart et al., 2019) extended the idea of unlearning to more complicated algorithms such k-means clustering and also proposed the first definition of effective data deletion that can be applied to randomized algorithms, in terms of statistical indistinguushaabiiity. Depending upon this statistical indistinguishabile criteria machine unlcarning processes are widely classified into exact unlearning (Ginart et al., 2019; Brophy & Lowd, 2021) and approximate unlearning methods (Neel et al., 2021; Nguyen et al., 2020). The goal of exact unlearning is to exactly match the paramcter distributions of the unlearned model and the retrained model where as, in approximate unlearning, the distributions of the unlearned and retrained model's param- eters are close to some small multiplicative and additive terms (Neel et al., 2021). To extend the idea of unlearning or efficient data deletion for non-convex models such as deep neural networks (Golatkar et al., 2020b) proposed a scrubbing mechanism for approximate unlcarning in deep neural networks. A more of- ficient method of unlearning in deep networks is proposed by (Goel el al., 2022) where the initial layers of deep networks are frozen while the last few layers are finetuned on the filtered dataset. Further achieve the goal of exact unlcarning (Jia et al., 2023) exploit the model sparsification technique via weight pruning. Even though all of these methods achieve unlearning in supervised deep networks, the generalization of these methods for state-of-the-art high-fidelity GANs is unexplored.
Few methods like cascaded unlearning (Sun et al., 2023) and data redaction (Kong & Chaudhuri, 2023) try to prevent generation of undesired features in GANs, however, their methods operate primarily on very primitive DC-GAN as opposed to high-fidelity GANs like StyleGAN which the focus of this work. While Sun al. (2023) also show result on StyleGAN, there are several significant differences compared to the proposed method. First, there is a fundamental difference in the unlearning setting between Sun et al. (2023) and our method. To reduce the generation of undesired samples, Sun et al. (2023) proposes to forget undesired samples from the training datasct. Specifically, they assume access to samples from the training dataset. This is somewhat restrictive since users typically don't have access to the training data (Chundawatl et al., 2023; Graves et al., 2021). Further, in terms of methodology, Sun et al. (2023) propose to patch the latent space of the GAN with represontative samples. They suggest various strategies for generating these representative samples, such as using 'average samples' or 'other class samples' (cf. Section 4.3 of their paper). However, imposing such constraints on the latent. space may lead to suboptimal
4

---

Published in Transactions on Machine Learning Research (02/2025)
latent-space semantics, potentially harming the quality of generated images. To address this, we avoid manipulating the latent space directly. Instead, we focus on parameter-space semantics, where we identify generator parameters that produce undesired samples (Stage-1: Negative Adaptation), and then retrain the GAN to avoid these paramcters (Stago-2: Unlcarning Phase). Additionally, since the latent space naturally adjusts based on changes the parameter space (as shown in figure 1 (b)), we find sufficient to focus on paramcter-space semantics alone, as it automatically handles latent-space semantics as well. To the best of our knowledge, these insights into parameter-space semantics have not been explored in the context of unlearning, making approach novel.
2.2 Few-Shot Generative Domain Adaptation
The area of few-shot generative domain adaptation deals with the problem where a pre-trained generative model is adapted to a target domain using very few samples. A general strategy to do this is to fine-tune the model on target data using appropriate regularizers. Eg. Wang et al. (2018) observed that using a single tre-triined GAN for fine-tuning is good enough for adaptation. However, due to the limited amount of target data, this could lead to mode collapse, hence Noguchi & Harada (2019) proposed to fine-tune only the batch statisties of the model. Although, such a strategy can be very restrictive in practice. To overcome this issue, Wang al. (2020) proposed to append a 'miner' network before the generator. They propose two-stage framework, where the miner network is first trained to transform the input latent space to capture the target domain distribution then the whole pipeline is re-trained using target data. While these fine-tuning based methods give equal weightage to all the parameters of the generator, Li et al. (2020) proposed to finc-tune the parameter using Elastic Weight Consolidation (EWC). Particularly, EWC is used to penalize large changes in important parameters. This importance is quantified using Fisher-information while adapting the pre-trained GAN. Mo et al. (2020) showed that fine-tuning a GAN by freezing the lower layers of discriminator is good enough in few-shot sctting. Recently, string of work (Ojha et al., 2021; Xiao et al., 2022; Lee et al., 2021) focuses on few-shot adaptation by preserving the cross-domain correspondence. Lastly, Mondal et al. (2022) suggested an inference-time optimization approach where a they prepend a latent-learner, and the latent-learner is optimized every time a new set of images are to be generated from target domain.
As mentioned earlier, our approach involves an adaptation stage, where we adapt the pre-trained GAN to the negative samples provided by the user. In practice, the amount of negative samples provided by the user is very less hence such an adaptation falls under the category of few-shot generative domain adaptation. Hence, we make use of EWC (Li et al., 2020) for this adaptation phase (cf. Section 3.2 for details).
3 Proposed Methodology
3.1 Problem Formulation and Method Overview
Consider the generator Gng of pre-trained GAN with parameters 0g and an implicit generator distribution
rid Pv()). The GAN is trained using dataset D = [x, where x; ~ Pdeta(x). Using the feedback- based framework (Moon et al., 2023), we obtain a few negative and positive samples, marked by the user. Specifically, the user is provided with n samples S = {y: where y, are the generated samples from the pre-trained GAN, i.c., y; std po(y). The user identifies a subset of these samples S, (ythes., as negative samples or samples with undesired features, and the rest of the samples Sp = (ye)ies, as positive samples samples that don't possess the undesired features. Here, Sp and sn are index sets such that iid
sp Us, 21, and ss, 6. Formally, (yikees, iid pN (v) and hyse Pa (e), where, (y) is the implicit generator distribution on negative samples and PaiN (v) is the implicit generator distribution after removing support of negative samples. Given this, the goal of unlearning is to learn the parameters Op such that the generator Gop generates only positive samples. In other words, the parameters Op should lead to unlearning of the undesired features.
Our approach involves two stages: In Stage 1, we adapt the pre-trained generator Ge on the user-markod negative samples. This step gives the parameters Oy such that Goy generates only negative samples. Tn
5

---

Published in Transactions on Machine Learning Research (02/2025)
Stage 2, we unlearn the undesired features by training the original generator Ge on positive samples using the usual adversarial loss while adding additional regularization term that makes sure that the learned parameter is far from 0x. We call this regularization term repulsion loss as it repels the learned parameters from NN.
3.2 Stage-1: Negative Adaptation
The aim of the first stage of our method is to obtain parameter ON such that the generator Gay only generates negative samples (S,). However, one thing to note here is that the number of negative samples marked by the user (IS,l) might be much less in number (of the order of tens or a few hundrod). Directly adapting a pre- trained GAN with a much smaller amount of samples could lead to catastrophic forgetting (McClelland et al., 1995; McCloskey & Cohen, 1989). To address this issue, we employ a few-shot GAN adaptation technique, namely, Elastic Weight Consolidation (EWC) (Li al., 2020), mainly because of its simplicity and ease of implementation. EWC-based adaptation relies the simple observation that the 'rate of change' of weights is different for different layers. Further, this 'rate of change' is observed to be inversely proportional to the Fisher information, F' of the corresponding weights, which can used for penalizing changes in weights in different layers.
In our context, we want adapt the pre-trained GAN on the negative samples. Hence, the optimal parameter x for the adapted GAN can be obtained by solving the following optimization problem:
ØN, = arg min max Ladv + YLadapt
(1)
where, fodd x>px(())
Dlg Do +
E z>pz(=)
[log(1 Do(GG(z))))
(2)
0²
Ladopt > Σ F((0: 0G,,), E
Loo (Sn)
(3)
002
Herc, P2(2) is the standard Gaussian prior, and Lo (Sn) refers to the ligelikelihood function for the samples Sn generated by the GAN with parameters 0g. Specifically, Le (S.) log Pea (S.) which is the log-likelihood of the negative samples under the generator's distribution with parameter 6g. This term can be estimated by calculating the binary cross-entropy of the discriminator's output, (S,),,, as shown in (Li et al., 2020). In practice, we train multiple instances of the gencrator to obtain multiple 0x. Specifically, given the negative samples Sn, we adapt the pre-trained GAN & times to obtain )
3.3 Stage-2: Unlearning
In the second stage of our method, the actual unlearning of undesired features takes place. In particular, this stage motivated by the obscrvation that there exist mcaningful directions in the parameter space of the generator, shown in figure 1 (b). However, such extrapolation- schemes could lead to degradation in the quality of generated images.
Nevertheless, the above observation indicates that traversing away from Oy helps to crase unlearn the undesired features. Therefore, we ask the following question: Can we transverse in the parameter space of a generator in such a way the parameters remain far from 0N while making sure that the quality of generated samples doesn't degrade? To solve this problem, we make use of the positive samples S, provided by the user. Particularly, we propose to train the given GAN on the positive samples while incorporating a repulsion loss component that 'repels' or keeps the learned parameters away from 0y. Mathematically, we obtain the parameters after unlearning Op by solving the following optimization problem:
0p.0P = arg min max Lado + -rreputtion
(4)
where, Cude=
E
llgg D (x)) +
E 2xp2(x)
log(1 - (Go(()))
(5)
xxPoxN(*)
6

---

Published in Transactions on Machine Learning Research (02/2025)
Here, Erepulsion is the repulsion loss. The repulsion loss is chosen such that it encourages the learned parameters to be far from IN obtained from Stage-1. Further, Lods encourages the parameters to capture the desired distribution Pay, (z).. Hence, the combination of these two terms makes sure that we transverse in the parameter space maintaining the quality of generated samples while unlearning the undesired features as well.
We emphasize that Cadp is different from Lads used in Stage-1. Specifically, the two adversarial terms serve different purposes: (i) Ladv is utilized during the Negative Adaptation Phase (Stage-1) to adapt the original GAN the negative samples (Sn); and (ii) Ludv applied during the Unlearning Phase (Stago-2) to retrain the original GAN on positive samples (Sp), which are the samples not marked as undesired.
Note: Our method requires users to identify annotate negative samples, ie., those containing undesired features. This annotation serves to adapt the GAN to negative samples during the Negative Adaptation phase and subsequently retrain it on positive samples during the Unlearning phase. While human feedback is one approach for obtaining these samples, other methods, such as curating datasets of positive and negative samples, can also be employed. However, curating such datasets can be challenging, especially when the feature, concept, or class be unlearned subtle or complex and not readily annotated in standard datasets. In such scenarios, users may need to create custom dataset. By contrast, our current. approach leverages human feedback to annotate readily available samples generated by the GAN, reducing the need for external dataset creation. Nonetheless, if a pre-curated dataset of positive and negative samples is available, our method can be easily adapted to use it. The GAN can be trained on this dataset to obtain the negative parameters I,, which can then be utilized in the Unlearning phase. We demonstrate few results using such approach in Appendix.
Algorithm: Adapt-then- Unlearn
Stage-1:
Negative Adaptation
Stage-2: Unlearning
Required: Pre-trained parameters (0g. 6D), Neg- ative samples (S,), Number of adapted models (k:).
Required: Pre-trained parameters (0g. (D), Posi- tive samples (Sp), Adapted models (0N = (0 {
Initialize: j x
Initialize: 0, tg, Qr + ÓD
while j s k do
1: repeat
2:
0 - tc o t ÓD
2: Sample x ~ Sp and z ~ NN(0
3: repeat
3: L' +- log Dø(x) + log De
Sample Sn and a ~ -N(0
4: Choose Lrepudsion from Eq.
Ladv f- log (x log (1 De(G((z)))
5: 0+ 0 --nVoooooa + Erepulsion)
6:
Ladapt + X Σ, Fi(0; - -G..)
6: until convergence
7:
0- 0 - (Laad + Ladapt)
until convergence
0
0
10:
- -+j+1
11:
end while
3.4 Choice of Repulsion Loss
As mentioned above, the repulsion loss should encourage the learned parameter to traverse away from Ox obtained from the negative adaptation stage. Here, we note that the repulsion term operates in parameter- space of the generator. There is lineage of research work in Bayesian learning called Deep Ensembles, where multiple MAP estimates of a network are used to approximate full-data posterior (Levin et al., 1990; Hansen & Salamon, 1990; Breiman, 1996; Lakshminarayanan el al., 2017; Ovadia el al., 2019; Wilson & Ixmailov, 2020; D' Angelo Fortuin, 2021). However, if the members of an ensemble are not diverse enough, then the posterior approximation might not capture the multi-modal nature of full-data posterior. As consequence, there are several methods proposed to increase the diversity of the members of the ensemble (Huang al., 2016; Von Oswald et al., 2020; D' Angelo & Fortuin, 2021; Wenzel al., 2020; D' Angelo & Fortuin, 2021).

---

Published in Transactions on Machine Learning Research (02/2025)
Inspired by these developments, make use of the technique proposed in D D'Angelo & Fortuin (2021) where the members of an ensemble interact with each other through a repulsive force that encourages diversity in the ensemble. Particularly, we explore three choices for repulsion loss:
C1.
(Inverse f2 loss)
repulsion
|0-
Crepulsion
CNL -110 N 1
(Negative E2 loss)
(6)
repulsiion
C
repulsioon exp(-al16 @ A 1x)1 (Exponential negative f2 loss)
It can be seen that minimization of all these choices will force 9 to be away from 0N. consequently serving our purpose. In fact, in general, one can use any function of || 0x 1l that has global maxima at dN a choice for repulsion loss. In this work, work with the above mentioned choices. An algorithmic overview of Stage-1 Negative Adaptation is presented in Algorithms 11 and Stage-2 Unlearning is presented in Algorithm 6
4 Theoretical Discussion
In this section, we present theoretical insights into the proposed method. Inspired by the work in Nguyen al. (2020), operate in Bayesian setting for these claims and make of widely used Laplace approximation around relevant parameters. Specifically, regularization term combined with the adversarial term results in contrastive divergence-like objective (a difference of two divergence terms). This encourages the generator to capture the implicit distribution of the pre-trained generator without the support of negative samples while maximizing the divergence between the parameter distribution of the post-unlearning generator and that of the generator which produces negative samples (Theorem 1). This result is shown in Further, we show the relation between the parameter space divergence and data space divergence (Claim 1).
demonstrate that for optimal discriminator, the proposed
For this, let θ denote the parameter space of a generator network. Let OG E θ be the parameter of pre-trained generator with an implicit distribution (x) over the data space'. Further, consider two distributions (0) and p (0) over O, where the latter is learnable distribution and the former is such that for z (z) the corresponding generated samples from manifested generator Go(z) ~ (N(x). In other words, samples from pN (0) lead to the generation of negalive samples.
Theorem 1. Consider the distributions (0) and (0) to be Gaussian, i.e., PR(O) 2x exp ( - ~N) 0x)] and (0) = Panjaa exp [ 0p) 0p)], where E = I, IN and Op are the mean parameters and 0p is learnable. Then statements (1 3) hold for the following optimization problem:
min max 0p
E x>Pa)N(>)
[og Dz(x)] + E [og(1 DD + Crepulsion
(7)
+
0~pu
1. for Erepulsion
LL2 repulsion: solving Eq. 7 leads to pe that minimizes
DISD(P 1 [DKL [1
for Crepulsson
NL2 repulsion: solving Eq. 7 leads to p that minimizes
DisDlp ll r (xI(pÿ )
3. for Crepulsion = L EL2
repulsion? solving Eq. 7 leads to pt that minimizes
Drs 1 D ( PN)
For convenience, we superscript and to denote distribution in data space and parameter space respectively. We the data space distribution from previous section with superscript X this distinction.
8

---

Published in Transactions on Machine Learning Research (02/2025)
where, D (1-) and Du( II-) denote KL divergence and Hellinger divergence, and Go(z) ~ (t) denotes the implicil distribution of generalor when 2 pz and f ~ r.-
The proof for the aforementioned result is relatively straightforward; for the sake of completeness, include the proof in the Appendix. It is evident from the above result that the unlearning phase of the proposed method, assuming Gaussian parameter distribution, achieves two objectives: (a) minimizing the Jensen- Shannon Divergence between the learned data distribution (vê) and the implicit generator distribution after removing the support of negative samples N)), and (b) maximizing a suitable divergence measure between the parameter distribution during unlcarning (pL) and the parameter distribution that leads to the generation of negative samples (p While (a) is relatively straightforward, (b) provides valuable insights into the effect of the regularization term, which is particularly interesting. The regularization term ensures that the current parameter distribution moves away from the one responsible for generating undesired features, effectively aligning with the primary goal of unlcarning. Furthermore, this behavior has been shown to be crucial for unlearning in Bayesian settings, as demonstrated in Nguyen et al. (2020). Essentially, this aligns with the desired outcome of unlearning, ensuring that the model captures only the desired support of the distribution.
An interesting observation from the above result is that utilizing Crepulsion or C reppussion results in objective akin to contrastive divergence, ie., it entails the difference between two divergences. However, these two divergence metrics operate on distributions in distinct spaces: the first divergence operates in the data space, while the second operates in parameter space. This prompts natural question: how does the divergence in parameter space relate to the corresponding distribution in data space? We address this question in the following claim.
Claim 1. For any general f f-divergence Dy(-1 1-), and a given latent vector, the following inequality holds:
DıSDlPaNN 1 - Dy(pi ll 1xx) < Dusplran 11 DD(e0 ll (
(8)
Above result relies on simple application of data-proccssing incquality. The proof is provided in Appendix. Since, KL and Hellinger divergence are both instances of f-divergence, the above result holds for Statements 2 and 3 of Theorem 1. Hence, we see that the while using CNN12 repulsion or LEL2 reppulsion: the corresponding data space objectives act upper bounds to the parameter space objectives. With these insights, we end the thcoretical discussion.
5
Experiments and Results
5.1 Datasets
An unlearning algorithm should ensure that the generator should not generate images containing the unde- sired (or unlearnt) fcature. For our experiments, consider two types of unlearning settings: (i) Class-level unlearning and (ii) Feature-level unlearning. The primary difference between the two type of unlearning lies in the nature of the associations. In feature-level unlearning, an image can exhibit multiple features simultaneously, whereas in class-level unlearning, an image from one class cannot belong to any other class. In other words, if each feature is treated as a class, feature-level unlearning allows an image to belong to multiple classes, while class-level unlearning restricts an image to a single class. For instance, a person with bangs can be either male or female, but a digit labeled as fone' cannot simultancously belong to any other class.
We use MNIST dataset (LeCun et al,, 1998) and AFHQ dataset (Choi et al., 2020) for class-level unlearning. MNIST consists of 60, 000 28 x 28 dimensional black and white images of handwritten digits. For our experiments, we take three-digit classes: 1, 4, and 8 for unlearning. AFHQ consists of 15, 000 high-quality animal face images at 512 x 512 resolution with three categories: cat, dog and wildlife. We unlearn each class one at a time our experiments. Similarly, we use ColebA-HQ dataset (Liu et al., 2015) for feature-level unlearning. CelebA-HQ contains 30, 000 RGB high-quality celebrity face images of dimension 256 x 256. Here, we unlearn the following subtle features: (a) Bangs, (b) Hats, (c) Bald, and (d) Eyeglasses.
9

---

Published in Transactions on Machine Learning Research (02/2025)
Original Samples Extrapolation
CLL repulsion
P repulsiio
C
Cats
Dogs
Wild
Figure 2: Results of Unlearning different classes on AFHQ dataset.
5.2 Experimental Details
Training Details: We use one of the state-of-the-art and widely used high-fidelity StyleGAN2 (Karras et al., 2020) for demonstrating the performance of the proposed method on the tasks mentioned in previous section. The StyleGAN is trained on the entire MNIST, AFHQ and CelebA-HQ datasets to obtain the pre-trained GAN from which we desire to unlearn specific features. The training details of StyleGAN2 are given in Supplementary Section A.
Unlearning Details: In our experiments, we employ a pre-trained classifier as a proxy for human to obtain user feedback. Specifically, we pre-train the classifier to classify a given image as desired or undesired (depending upon the feature under consideration). We classify 1, 000 generated images from pre-trained GAN as positive and negative samples using the pre-trained classifier. The generated samples containing the undesired features are marked negative samples and the rest of the images are marked positive samples. These samples are then used in Stage-1 and Stage-2 of the proposed method for unlearning described in Section 3. We evaluate our result using all the choices of repulsion loss mentioned in Eq. 6. For reproducibility, we provide all the syper-parametent and training details in Supplementary Section A. The original FID of the GAN after training is as follows- MNIST: 5.4, AFHQ: 8.1, CelebA-HQ: 5.3. We mention these in caption of each table wherever necessary.
5.3 Baselines and Evaluation Metrics
Baselines: To the best of our knowledge, ours is one of the first works that addresses the problem of unlearning in high-fidelity generator models such as StyleGAN2. Hence, we evaluate and compare our method with all the candidates for repulsion loss presented in Eq. 6. Further, we also include the results with extrapolation
the parameter space as demonstrated figure 1 (b). We include recent unlearning baselines tailored for classification, easily adaptable generative tasks. Specifically, we incorporate EU-k, CF-k, and fl-sparse (Goel et al., 2022; Jia et al., 2023) for comparison, with detailed information in the Supplementary section A.4. Additionally,
assess method against GAN adaptation to positive samples, utilizing recent generative few-shot adaptation methods like EWC, CDC, and RSSA (Li et al., 2020; Ojha et al., 2021; Xiao et. al., 2022) as baselines. Apart from the above baselines, we also mention the results obtained from training a GAN from scratch on only desirable data present in the dataset. This model acts as the gold standard, however, due unavaiaaiility of underlying datasct, this not practical. Nonetheless, we mention in our tables for completeness. We evaluate the performance of each method across three independent runs and report the result in the form of mean + std. dev.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Original Samples Extrapolation
LLL C
C repulsion
repul
Bangs
Eyeglass lass
Bald head
Hats
Figure 3: Results Unlearning different features on CelebA dataset.
Evaluation Metrics: Various metrics have been devised for assessing machine unlearning methods (Xu et al., 2020). To gauge the effectiveness of our proposed techniques and the baseline methods, we utilize three fundamental evaluation metrics:
Percentage of Un- Learningg (PUL): This metric quantifics the extent of unlearning by measuring the reduction in the number of negative samples generated by the GAN post-unlearning compared to the pre-unlearning state. PUL is computed as: PUL (Sa)ag-(S,We -(S,)e x 100, where, (Sa)aa and (S.)ec
(S,)ap represent the number of negative samples generated by the original GAN and the GAN after unlearning respectively. We generate 15,000 random samples from both GANs and employ a pre-trained classifier (as detailed in Section 5.2) to identify the negative samples. PUL provides quantitative measure of the extent of the unlearning algorithm in eliminating the undesired fcature from the GAN.
2. Fréchet Inception Distance (FID): While PUL quantifies the degroe of unlearning, it does not assess the quality of samples generated by the GAN post-unlearning, Hence, calculate the FID (Heusel et al., 2017) between the generated samples and the original dataset without the undesired samples.
3. Retraining FID (Ret-FID): To resemble the retrained GAN, we compute the FID between the outputs of the GAN after unlearning and the GAN trained from scratch on the dataset obtained after climinating undesired features.
Please note that the original dataset unavailable during the unlcarning process. Consequently, the use of the original dataset is solely for evaluation purposes.
5.4 Unlearning Results
We present our results and observations on MNIST, AFHQ and CelebA-HQ in Table 1, 3 respectively. We observe that the choice of repulsion repulsion loss provides the highest PUT in most of the cases
11

---

Published in Transactions on Machine Learning Research (02/2025)
Table 1: PUL (t), FID (L), and Ret-FID (4) after unlearning MNIST classes. FID of pre-trained GAN: 5.4.
Method
Class
Class 4
Class 8
PUL
FID
Ret-FI1
PUL
FID
Ret-FID
PUL
FID
Ret-FID
Retraining
98.80 = 0.09
4.11
N/A
98.58 =: 0.28
=.21
N/A
99.72 = 0.011
1.11
N/A
CF-A
18.60 2.30
92.88 0.51
89.05 = 0.93
0.118
33.61 = 0.93
30.76 = 0.86
16.03 = 0.09
36.67 = 1.96
0.71
EU-É
31.77 1.56
15.34 0.01
14.05
4: 0.17
17.59 2: 3.31
8.80 = 0.08
7.84 1: 0.24
21.66 0.70
8.92 1: 0.77
7.10 0.30
(1-Sparse
91.81 0.55
22.17 0.14
17.87 = 0.42
91.16 0.06
23.21 0.59
17:25 0.22
95.12 0.01
22:23
16.83
EWC
90.93 = 0.46
7.18 + 0.08
5.10
82.78 + 1.02
9.34 = 0.08
6.91 + 0.04
92.70
0.81
9.33 + 0.04
6.03 + 0.04
CDC
90.20 0.16
20.85 = 0.49
17.51 +: 1.86
42.39 +: 0.66
17.82 + 1.99
12.30 +: 2.32
18.86 0.92
11.18 = 2.28
12.38 + 3.73
RSSA
38.36 0.09
12.70 0.17
13.54 +: 0:40
71.96 +: 0.21
25.41 + 0.06
19.08 0.59
79.54
0.32
34.30 0.26
25.33 0.30
Extrapolation
95.10
0.69
41.39
1.76
12,98 =: 0.68
91.50 4: 0.05
17.90 =: 0.35
27.81 =: 0.37
90,90 = 0.11
45.79 0..29
11,30 =: 0.40
Iipetaion (Ouns
97.85 2.25
9.60 a: 0.07
6.70 0.25
03.03 1: 0.70
10.50 a: 0.34
6.26 1. 0.12
97.92
0.67
9.95 1. 0.17
6.70 1. 0.18
(Ours
92.97 = 08
13.06 = 0.46
16.55 + 0.51
90.39 + 1.36
15.51 = 0.05
8.61 +: 0.90
98.28 + 0.55
9.72 +: 0.31
11.64 = 0.46
(Ours)
99.32 :: 0.43
9.65 +: 0.21
6.29 0.18
96.23 +: 0.54
10.24 2: 0.19
5,80 1: 0.04
93,22 0.55
8.89 1: 0.52
5.68 1: 0.30
Table 2: PUL (1), FID (L), and Ret-FID (L) after unlearning AFHQ classes. FID of pre-trained GAN: 8.1.
Method
Cat
Dog
Wild
PUL
FID
Ret-FID
PUL
FID
Ret-FID
PUL
FID
Ret-FID
Retraining
93.37 = 0.23
12.45 = 0.16
N/A
85,96 1: 0.21
5.71 0.43
N/A
88.60
0.22
13:24 = 0.24
N/A
CF-k
15.74 00.11
12:91
0.33
37.31 +: 0.29
13.12
61.50
0.58
63.72
16.40
0.35
35.56
34.47 0.91
EU-K
16.08 0.28
41.21
0.25
37.62
1 0.42
10.30 1. 0.13
32.69 1: 0.23
31.40 1 0.36
16.63
0.84
35.80 - 0.39
32.55 1 0.14
(1-Sparse
86.25 = 1.35
25.73
= 0.05
14.96
+ 0.11
68.21 = 0.15
19.51 = 0.53
15.10 = 0.85
84.05 = 0.45
37.63 = 0.32
21.80 =
EWC
90.58 1.54
15.30
0:98
8:29 0.01
08. = 0.08
10.38 = 1.08
8.50 0.35
T3.11 0.16
16.32
14.38 0.01
GDC
28.57 0.43
50.76
0.40
40.93 + 1.02
9.00 0.24
47.85 + 0.53
46,79 + 0.67
6.51 + 0.13
42.17 - 0.19
36.95 + 0.28
RSSA
66.86 0.22
0.35 59.88
16.80 = 0.0 62
54.54 + 36
55.06 = 0.20
53.24 = 0.31
43.99 = 0.72
00.95 = 0.45
14,20 0.32
Extrapolation
90.64 0.33
45.89
2:60
38.87 + 1.93
82.08 + 1.02
25.54 + 1.19
24.98 1.35
84.45 0.11
45.98 2.66
41.86 1.99
NLa
Piliaion (Ours)
94.28 0.17
16.29
0.06
8.03 1,23
75.33 + 0.35
8.62 0.08
5.96 0.45
82.82 0.77
17:60 0.18
14.89 + 0.13
cupmaton (Ours
90.93
01.1
20.69
0.04
9.86 0.08
78.53 d: 0.51
9.37 1.06
6.84 0.30
80.96 0.118
22.70 = 0.11
13.78 0.12
eppiciion (Ouns)
95.76 1. 0.25
16.50
0.12
8.17 1: 0.14
79.21 1. 0.18
9.31 0.14
7.13 J. 0,12
89.09 J. 0.25
19.67 0.69
14.90 0.65
Table 3: PUL (1), FID (4), and Ret-FID (4) after unlearning CelebA-HQ features. FID of pre-trained GAN:
5.3.
Mothod
Banga
Hat
Bold
Exeglasses
PUL
FTD
Ret-PID
PLL
P1D
Ret-FII
PUL
PID
Ret-FID
PUL
PID
Ret-FID
Recestning
84.07
7.58 5 0.05
N/A
95.65 à
: 0.10
N/A
72.13 8 0.07
7.48:
M/A
58.57 :1 0.04
0.50 0.77
N/A
CF-6
15.00 2:30
7,70
15.22
48.17 4:
9,30
7.47
0.49
EU-I
20.08 1.15
17.00
9.40 0.21
7.30 0.00
52.18 4: 0.53
9.08 0:00
0.02 0.00
1.63
0.39 0.04
E1-Sparaa
75.08 1.45
16:30
13.77 6.40
69.01
0:000
07:20 6
10.27 L 0.78
14.14 0.50
80,30 1.06
15.05 à:
12.96 2:
78.24
9.64
74.14
9.06
0.04
55.70 6:
9.32 0.07
6,77 0.04
0.10 0.04
CDC
34.00
3.04
24.67 0.02
65.20
0.32
17.40
10:30 6
26.05 0.00
17
0.00
0.31 20.21
36 05
32.07 0.58
17.79 0.00
15.00 0.08
22.70 8:
18.85 0.08
14.87
31.70 6:
0.34 20.55
18.03
0.01
22:24
15:00
Extrapolation
0.00
0.00
12.18:00000 0.12
0:01
04.44 0:34
26.30:: 0.30
10.50 :: 0.14
23:70
1.:10:00:00
2lta (Dun)
2155 (Uun)
8100 5 1:03
00.41 0.19
30.00
0.46
9:000
9.07
=:
03.00 1:
81.00 0:75
11.30 0.06
a:
7:25 0:114
0:44::0:111
07:13 1.42
NA.1 #: 2.15
14.70 0.55
12:94 0..0
9.08
0.00
76.25
2: 8.21
d- 6:35
13:12 :: 0.7%
12:01 a:
6.11 0.94
(Dun))
90.45 2 1.02
11.16 6 0.08
7.94 4 0:37
94.00 2: 2.19
9.45 6 0.M0
6:31 0-06
00.97 4 2.05
11.07 - 0.36
7.83 0.05
03.633 1:: 0.42
9.00 0.55
9.84 0.90
for both the datasets. Further, it also provides the best FID and Ret-FID as compared to other choices of repulsion loss. CNL2 repulsioo is stands out to be the second best these metrics for most of the cases. Further, we observe that across all datascts, the classification unlcarning basclines perform very poorly on all metries for unlearning in GANs. This tells us that methods proposed for unlearning in classification are not suited for unlearning in generative tasks. And lastly, we find that few-shot adaptation baselines, give relatively poor results when compared to the proposed method. This observation indicates that it is not enough to just adapt the GAN on the positive samples for unlearning, one needs to go further and use additional regularization to unlearn the undesired features.
For MNIST, we observe in Table 1 that the proposed method with repulsion as repulsion loss consistently provides a PUL of above 95% while giving the best FID and Ret-FID compared to other methods. We also observe that Extrapolation in parameter space leads to significant PUL albeit the FID and Ret-FID
CEL2
12

---

Published in Transactions on Machine Learning Research (02/2025)
are considerably worse compared to proposed method under different repulsion loss. This shows that the proposed method decently solves the task of unlearning at class-level.
We make similar observations for high-resolution AFHQ dataset as well. One can see that the proposed method provides highest PUL in all the cases, while maintaining the FID as well as Ret-FID. We observe highest PUT while unlearning the 'Cat' class while lowest. PUT is observed in 'Dog' class.
Lastly, for feature-level unlearning results on CelebA-HQ, it can be seen that the proposed method with C Creputsson as repulsion loss consistently provides a PUL of above 90%, illustrating significant unlearning of undesired features. Further, the FID and Ret-FID using CEL2 methods with significant PUL.
repulaion stand out to be the best among all the
We also observe some drop in FID across all dataset after unlearning. E.g., the FID of the samples generated by the unlearnt GAN (on Hats) using points while using repulsion and CIL2 repulsion as compared to the pre-trained GAN. On the other hand, Extrapolation in parameter space leads to a drop of 6.88 poinits in FID. This further validates the need of repulsion regularizer to maintain the generation quality. This observation is consistent across all datasets and features. This supports our claim that extrapolation might unlearn the undesired feature, however, it. deteriorates the quality of generated samples significantly.
C Prppusioon drops by about 4.15 points while it drops by 4.3 and 6.01
Another interesting observation from the above results is that the classification unlearning baselines consis- tently provide lower PUL, albeit with slightly better FID and Ret-FIID. This tells us that these baselines while capable of unlcarning in classification tasks, fail to nudge the generator appropriately for desired unlcarning lask. Leading to a suboptimal generator which still generates undesired samples, without compromising on the quality of the generated samples.
The visual illustration of these methods for AFHQ and CelebA-HQ are shown in figure 2 and figure 3 respectively. Here, we observe that the proposed hod effectively unlearns the undesired feature. Moreover, it can be seen that the unlearning through extrapolation leads to the unlearning of correlated features as well. E.g. Bangs are correlated with female attributes. It can be seen that the unlearning of Bangs through extrapolation also leads to the unlearming of female feature which is not desired. However, unlearning through the proposed method unlearns Bangs only, while keeping the other features as it is. Similar observations could be made for AFHQ, where extrapolation leads to some minor artifacts in generates samples, whereas proposed method generates plausible images without any artifacts. Similar visual results for MNIST is provided in Supplementary Section B. We also provide visualization of random samples generated from original GAN and the GAN after unlcarning in Supplementary Section F to give a qualitative idea of the generation quality.
Another aspect of unlearning that we explore in our work is the effect of unlearning on other features. Particularly, unlearning undesired feature should not disturb the other features. For this we generate samples from pre-trained and post-unlearning GANs. Then, we calculate the occurrence of specific features within the two GANs and report the percentage change in these numbers. These results could be found in Section E of Supplementary. As discussed in Section 3, the approach of first adapting the model to negative samples followed by applying the repulsion loss during the Unlearning phase can also be extended to scenarios where curated datasets of positive and negative samples are available. We demonstrate this Section D.
5.5 Comparison with DC-GAN Baselines
As previously mentioned, our proposed method operates with high-fidelity GANs. Nonetheless, in addition to the tailored baselines, few previous methods aim to unlearn undesired features in more primitive GANs, such as DC-GANs, on low-rcsolution images. Notably, the methods proposed in Sun et al. (2023) and Kong & Chaudhuri (2023) are highly relevant to our work. However, they primarily operate on DC-GAN. To ensure a fair comparison, we implement our method on DC-GAN and evaluate it against these baselines using the MNIST dataset (with CBL2 r Specifically, we compare our approach to the cascaded unicarning algorithm (CUA) from Sun et al. (2023) and the data redaction method using validity data (DRed) from Kong& & Chaudhuri (2023). Our findings are summarized in Table 4. The results indicate that our method
13

---

Published in Transactions on Machine Learning Research (02/2025)
Table 4: Comparison of the proposed method against DC-GAN baselines
Table 5: Effect on PUL (t), FID (4), and Ret-FID (4) with and without repulsion
loss.
Method
Metrics
Class
Class 4
Class
CTTA
PUL
RID)
12:3++00.48
06.12 1:11
06.00 0.37
11.01 1.81
07:34
15:00 + 1:03
Features
Metrics
Cate
CE1A
rapadaion
Ret-FID
11:02
11.57 0.79
10.0 + 1.21
PUL
79.80
0.49
90.45 1 1.01
PET
0.11
07.70 0:20
06.51+0225
Bangs
FID
10.06
0.24
11.16 0.08
DRed
11.72+1.29
10.32
Rei-FID
8.69 +: 0.04
7,94 1 0.32
Ret-F113
8.72
0.86
10:12 1.41
PUL
58.76
0.56
90.18
0.28
98.72
0.42
PUL
8168 3.89
94.40
2.19
Ours
PIT)
10.37
0.94
0.72
1.33
10.27
0.82
Hat
FID
9.00 0.16
9.45
Ret-FID
6.32
0.43
7.23
1:28
7.66
1.30
Ret-FID
6.15 + 0.08
6.04 +: 0.02
outperforms both baselines across all metrics in all scenarios. All methods performed well on PUL, with our method achieving the best PUL, followed by DRed and then CUA in most cases. A similar trend is observed in the FID scores. Although DRed is a close competitor to our proposed method, our approach yields significantly better Ret-FID than DRed, suggesting that the post-unlearning GAN using our method is closer to the gold standard compared to DRed.
5.6 Ablation Study
Lastly, we present the ablation study to observe the effect of repulsion loss. In particular, we see if adapting the pre-trained GAN only on the positive samples leads to desired levels of unlearning. Our observations CelebA-HQ for Bangs and Hats are presented in Table 5. Here, we use repuision as repulsion loss. Tt can be seen that only using adversarial loss doesn't 't lead to significant unlearning of undesired feature. E.g. using repulsion loss provides and increase of about 10.56% and 9.72% in PUL. The FID increases by minor 0.66 point on Bangs while it decreases by 0.21 points on Hats. Hence, we conclude that repulsion loss is indeed crucial for unlearning.
LEL2
6 Conclusion
We propose novel unlearning method designed for high-fidelity GANs. Our approach is distinguished via its unique ability to operate in zcro-shot scenario, entirely independent of the original data which GAN is trained. We operate under feedback-based framework in two stages. The initial stage adapts the pre-trained GAN on the negative samples whereas the later stage unlearns the undesired feature by adapting on positive samples along with a repulsion rogularizer. A notable advantage of our approach is its capability to conduct the unlearning process without significantly impacting other desirable features. We firmly believe that our work represents a substantial advancement the field of unlearning within deep generative models. This progress holds particular relevance in addressing critical societal concerns, particularly those related to the generation of biased, racial, or harmful content by these models.
Limitation and Future Work:
We note that our study does not address aspects like 'toxicity' due to the absence of annotated datasets with explicit characteristics. Despite this limitation, it's crucial to emphasize that our explored features are subtle and interconnected. For example, subtle details like hairstyles (e.g., bangs) are strongly linked to gender, and characteristics like baldness correlate with physical appearance, also associated with gender. Additionally, attributes such as hats and eyeglasses are tied to accessorics. Though not the primary focus for unlearning, these features hold potential utility for such purposes.
In future, aspire to provide rigorous theoretical guarantees to such methods making them more dependable and safe for deployment. Morcover, the approach proposed in this work is inherently generic and can be applied to any model exhibiting parameter-space semantics. This opens up the possibility of extending such techniques to more powerful generative models, such as Diffusion models and Flow-based models. However, applying this method to iterative models like diffusion models would require a thorough investigation of their parameter-space semantics. Additionally, developing an effective strategy for negative adaptation in these models remains an open research challenge, unlike GANs, where few-shot generative adaptation is relatively
14

---

Published in Transactions on Machine Learning Research (02/2025)
well-studied. Nonetheless, this represents promising avenue for modern generative models, and we leave this exploration for future research.
Broader Impact Statement
Machine unlcarning has emerged as a crucial tool for addressing privacy concerns and mitigating harmful biases in AT systems. Our method advances this field by enabling selective feature removal from pre-trained GANs without requiring access to the original training data. This capability is particularly valuable for correcting deployed models that may generate problematic ethical and socially responsible. While our approach currently focuses on GANs, its success demonstrates the potential for developing similar techniques for other generative models, contributing to the broader goal of creating Al systems that can be refined and corrected post-deployment to better serve society's needs.
biased content, making Al systems more
Acknowledgements
This work was supported (in part for setting up the GPU compute) by the Indian Institute of Science through start-up grant. Piyush is supported by Government of India via Prime Minister's Rescarch Fellowship. Atri contributed to this work as part of Uplink: IKDD Research Internship Program. Subhodip is supported by MOE Fellowship. Prathosh is supported by Infosys Foundation Young investigator award.
References
Adekanmi Adeyinka Adegun, Serestina Viriri, and Jules-Raymoot Tapamo. Review of deep learning methods for remote sensing satellite images classification: experimental survey and comparative analysis. Journal of Big Dala, 10(1):93, 2023.
TMartin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proc. of ICML, 2017.
John E Ball, Derek T Anderson, and Chee Seng Chan. Comprehensive survey of deep learning remote sensing: theories, tools, and challenges for the community. Journal of applied remote sensing, 11(4): 042609 042609, 2017.
Leo Breiman. Bagging predictors. Machine learning, 24:123 140, 1996.
Jonathan Brophy and Daniel Lowd. Machine unlearning for random forests. In Proc. of ICML, 2021.
Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning, In Proc. of IEEE Symposium on Securily and Privacy, 2015.
Gert Cauwenberghs and Tomaso Poggio. Incremental and decremental support vector machine learning. In Proc. of NIPS, 2000.
Pedro Celard, EL Iglesias, JM Sorribes-Fdez, Rubén Romero, Seara Vieira, and L Borrajo. survey on deep learning applied to medical images: from simple artificial neural networks to generative models. Neural Computing and Applications, 35(3):2291-23233 2023.
Anton Cherepkov, Andrey Voynov, and Artem Babenko. Navigating the gan parameter space for semantic image editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021.
pp.
Yunjey Choi, Youngiung Uh Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
Kamal Choudhary, Brian DeCost, Chi Chen, Amubhav Jain, Francesca Tavazza, Ryan Cohn, Chool Woo Park, Alok Choudhary, Ankit Agrawal, Simon JL Billinge, al. Recent advances and applications of deep learning methods in materials science. npj Computational Materials, 8(1):59, 2022.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Rishav Chourasia and Neil Shah. Forget unlearning: Towards true data-deletion in machine learning. In Proc. of ICML, 2023.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforce- ment learning from human preferences. Advances in neural information processing systems, 30, 2017.
Vikram S Chundawatl, Ayush K Taruul, Murari Mandal, and Mohan Kankanhalli. Zero-shot machine unlearning. IEFE Transactions on Information Forensics and Security, 2023.
Francesco D' D'Angelo and Vincent Fortuin. Repulsive deep ensembles are bayesian. Aduances in Neural Information Processing Systems, 34:3451-344 2021.
Antonio Ginart, Melody Guan, Gregory Valiant, and James Y. Zou. Making lorget you: Data deletion machine learning. In Proc. of NTPS, 2019.
Shashwat Goel, Ameya Prablu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru. Towards adversarial evaluations for inexact machine unlearning, arXiv preprint, 2022.
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Proc. of ECCV, 2020a.
Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proc. of CVPR, 2020b.
Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. Mixed- privacy forgetting in deep networks. In Proc. of CVPR, 2021.
E. Goldman, An introduction to the california consumer privacy act (cepa). Santa Clara Univ. Legal Studies Research Paper, 2020.
Tan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Proc. of NeuRIPS, 2014.
Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proc. of AAAI, 2021.
Chuan Guo, Tom Goldstein, Awni Hanmun, and Laurens van der Maaten. Certified data removal from machine learning models. In Proc. of ICML, 2020.
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE transactions on poltern analysis and machine intelligence, 12(10):993 1001, 1990.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash cquilibrium. Advances in neural infor- mation processing systems, 30, 2017.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Proc. of NeuRIPS, 2020.
Gao Huang, Yixuan Li, Gooff Pleiss, Zhuang Liu, John E Hoperoft, and Kilian Q Weinberger. Snapshot ensembles: Train 1, get m for free. In International Conference on Learning Representations, 2016.
Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsification can simplify machine unlearning. arXiv preprint, 2023.
Sen Jia, Shuguo Jiang, Zhijic Lin, Nanying Li, Meng Xu, and Shiqi Yu. A survey: Deep learning for hyperspectral image classification with few labeled samples. Nearocompuling, 448:179- 2049,,244--0 2021.
Masayuki Karasuyama and Ichiro Takeuchi. Multiple incremental decremental learning of support vector machines. In Proc. of NIPS, 2009.
Tero Karras, Timo Aila, and Samuli Laine. A style-based generator architecture for generative adversarial networks. In Proc. of CVPR, 2018a. Sa.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In Proc. of TCI.R, 2018b.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proc. of CVPR, 2020.
Tero Karras, Mika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34: 852-863, 2021.
Zhifeng Kong and Kamalika Chaudhuri. Data redaction from pre-trained gans. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 638 677. IEEE, 2023.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncer- tainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017.
Nathan Lambert, Louis Castricato, Leandro von Werra, and Alex Havrilla. Illustrating reinforcement learning from human feedback (rlhf). Hugging Face Blog, 2022.
hatpiicc..///////ll///aae blog/rlhf.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- based learning applicd to docu- ment recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Hyuk-Gi Gi Lee, Gi-Chcon Kang, Chang-Hoon Jeong, Han- Wool Sul, and Byoung- Tak Zhang. C.: Contrastive learning for cross-lomain correspondence in few-shot. image generation. In Proceedings of Workshop Controllable Generative Modeling in Longuage and Vision (CtrlGen) at NewrIPS 2021, 2021.
Esther Levin, Naftali Tishby, and Sara A Solla. A statistical approach to learning and generalization in layered neural networks. Proceedings of the IEEE, :8(10))1568-1577 1990.
Yijun Li, Richard Zhang, Jingwan (Cyuthia) Lu, and Eli Shechtman. Few-shot Image Generation with Elastic Weight Consolidation. In Proc. of NeurIPS, 2020.
Zhichao Liu, Luhong Jin, Jincheng Chen, Qinyu Fang, Sergey Ablameyko, Zhaozheng Yin, and Yingke Xu. A survey on applications of deep learning in microscopy image analysis. Computers in biology and medicine, 134:104523, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xisoou Tang. Deep learning face attributes in the wild. Proceedings of International Conference on Computer Vision (ICCV), 2015.
Zhuo Ma, Yang Liu, Ximong Liu, Jian Liu, Jianfeng Ma, and Kui Ren. Learn to forget: Machine unlcarning via neuron masking. In Proc. of IEEE Transactions on Dependaable and Secure Compuling, 2022.
James L McClelland, Bruce L McNaughton, and Randall C O'Reillly Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models learning and memory. Psychological review, 102, 1995.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 24:109-165, 165, 1989.
Sangwoo Mo, Minsu Cho, and Jinwoo Shin. Freeze the Discriminator: a Simple Baseline for Fine- Fine-Tuning GANs. In CVPR AI for Content Creation Workshop, 2020.
Arnab Kumar Mondal, Piyush Tiwary, Parag Singla, and AP Prathosh. Few-shot cross-domain image generation via inference-time latent-code learning. In The Eleventh International Conference on Learning Representations, 2022.
Saemi Moon, Scunghyuk Cho, and Dongwoo Kim. Feature unlearning for pre-trained gans and vacs. arXiv preprint, 2023.
Seth Neel, Aaron Roth, and Saced SharifiMallvajee Descent-to-delete: Gradient-basee methods for ma- chine unlearning. In Proc. of ALT,, 2021.
17

---

Published in Transactions on Machine Learning Research (02/2025)
Quoe Phong Nguyen, Bryan Kian Hsiang Low, and Patrick Jaillet. Variational bayesian unlearning, In Proc. of NIPS, 2020.
Quoc Phong Nguyen, Ryutaro Oikawa, Dinil Mon Divakaran, Mun Choon Chan, and Bryan Kian Hsiang Low. Markov chain monte carlo-basod machine unlearning: Unlearning what needs to be forgotten. In Proc. of ASIA CCS, 2022a.
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee- Wee-Chung Liew, Hongzhi Yin, and Quoe Viet Hung Nguyen. A survey of machine unlearning. arXiv preprint arXio:2209. 02299, 2022b.
A. Noguchi and T. Harada. Image Generation From Small Datasets via Batch Statistics Adaptation. Proc. of ICCV, 2019.
Utkarsh Ojha, Yijum Li, Cynthia Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, and Richard Zhang, Few-shot image generation via cross-domain correspondence. In Proc. of CVPR, 2021.
Yaniy Ovadia, Emily Fortig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshma Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainly? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pp. 8821 8831. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint Arx: 061255 1(2):, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Fsser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Ayush Sckhari, Jayadev Acharya, Gautam Kamath, and Ananda Theortha Suresh. Remember what you want. to forget: Algorithms for machine unlearnings. In Proc. of NewrIPS, 2021.
Yan Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Proc. of NeuRIPS, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederík P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poolc. Score-based generative modeling through stochastic differential equations. In Proc. of ICLR, 2021.
Hui Sun, Tianqing Zhu, Wenhan Chang, and Wanlci Zhou. Generative adversarial networks unlcarning.
arXiv preprint arXiv:2308.0000 09881, 2023.
Piyush Tiwary, Kinjawl Bhattacharyya, and Prathosh A.P. Cycle consistent twin energy-based models for image-to-image translation. Medical Image Analysis, 91:103031, 2024a. ISSN 1361-8415. doi: https:// doi.orrg/10.1011 mwtp:///..mmmm...m.. mwd.am.0023.103000000000 2033.103031. $1361841523002918
URL http://www..caacecceeeeee...ceec
Piyush Tiwary, Kumar Shubham, Vivek V Kashyap, and Prathosh AP. Bayesian pseudo-coresels via contrastive divergence. In The 40th Conference on Uncertainty in Artificial Intelligence, 2024b. URL http://wwwwvvveeeeeeeeer//////ee https://openreevev. wwt..//roruuttd-stssvvo...
Tatiana Tommasi, Novi Patricia, Barbara Caputo, and Tinne Tuytclaars. Advances in Computer Vision and Pallern Recognilion. Springer, 2017.
Gaäl Varoquaux and Veronika Cheplygina. Machine learning for medical imaging: methodological failures and recommendations for the future. NPJ digital medicine, 5(1):48, 2022.
P. Voigt and A. Von dem Bussche. The EU general data protection regulation (GDPR). Springer, 2017.
18

---

Published in Transactions on Machine Learning Research (02/2025)
Johannes Von Oswald, Seijin Kobayashi, Joao Sacramento, Alexander Meulemans, Christian Henning, and Benjamin F Grewe. Neural networks with late-phase weights. In International Conference on Learning Representations, 2020.
Xinya Wang, Qian Hu, Yingsong Cheng, and Jiayi Ma. Hyperspectral image super-resolution meets deep learning: A survey and perspective. IEEE/CAA Journal of Automatica Sinica, 10(8):1664-1687, 2023.
Yaxing Wang, Chenshen Wu, Luis Herranz, Joost van de Weijer, Abel Gonzalez-Garcia and Bogdan Radu- camu. Transferring GANs: generating images from limited data. In Proc. of ECCV, 2018.
Yaxing Wang, Abel Gorcalez-Garcia, David Berga, Luis Herranz, Fahad Shahbaz Khan, and Joost van de Weijer. MineGAN: effective knowledge transfer from GANs to targel domains with few images. In Proc. of CVPR, 2020.
Florian Wenzel, Jasper Snoek, Dustin Tran, and Rodolphe Jenatton. Hyperparameter ensembles for robust- ness and uncertainty quantification. Advances in Neural Information Processing Systems, 33:6514 -5271 2020.
Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generaliza- tion. Advances in neural information processing systems, 33:4697 4708, 2020.
Ga Wu, Masoud Hashemi, and Christopher Srinivasa. Puma: Performance unchanged model augmentation for training data removal. In Proc. of AAAI, 2022.
Yinjun Wu, Edgar Dobriban, and Susan B. Davidson. Deltagrad: Rapid retraining of machine learning models. In Proc. of ICML, 2020a.
Yinjun Wu, Val Tannen, and Susan B. Davidson. Priu: A provenance-base approach for incrementally updating regression models. In Proc. of SIGMOD, 2020b.
Jiayu Xiao, Liang Li, Chaofei Wang, Zheng-Jun Zha, and Qingming Huang, Few shot generative model adap- tion via relaxed spatial structural alignment. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11204-11221 11213, 2022.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transforma- tions for deep neural networks. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 1492-1500) 2017.
Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. Machine unlearning: A survey. ACM Computing Surveys Vol. 56, No. 1, 2020.
Biyun Yang and Yong Xu. Applications of deep-learning approaches in horticultural research: a review. Horticulture Research, 8, 2021.
Jingwen Ye, Yifang Fu, Jie Song, Xingyi Yang, Songhua Liu, Xin Jin, Mingli Song, and Xinchao Wang. Learning with recoverable forgetting. In Proc. of ECCV, 2022.
Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul F. Chris- liano, and Geolfrey Irving. Fine-tuning language models from human preferences. CoRR, abs/ 1909.08593. 2019. URL http:// htap:://aaxvv..ee/ perxiiv.ore org/ hatgg//b//19999 08593.
19

---

Published in Transactions on Machine Learning Research (02/2025)
A Training Details
Here, we provide the details pertaining to the proposed method. Specifically, we provide the details of the pre-trained GANs and pre-trained Classifiers used in the proposed method. We also provide details pertaining to the training strategy used during Unlearning. All the experiments are performed on RTX- A6000 GPUs with 48GB memory. Our code and implementation is available at: https://github.ce com/ atriguha/ http://pppppp_aarraaaaa Unlearn.
A.1 Details of Pre-trained GAN
As mentioned the main text, we use the famous StyleGAN2 architecture to obtain the pre-trained GAN, We use the open-source pytorch repository² for implementation. We resize the MNIST images to 32 x 32 and CelebA-HQ images to 256 256 to fit the StyleGAN2 architecture. The latent space dimension for MNIST and CelebA-HQ is consequently set to 128 x and 512 x 1. We train the GAN using the non-saturating adversarial loss along with path-regularization for training. We use default optimizers and hyperparameters as provided in the code for training. We train the GAN for 2 x 10y and 3.6 x 10y epochs for MNIST and CelebA-HQ respectively.
A.2 Details of Pre-trained Classifiers
We use pre-trained classifiers to simulate the process of obtaining the feedback. More specifically, the feedbacks (positive and negative samples) are obtained by passing the generated samples (from the pre- trained GAN) through these pre-trained classifiers. The classífier classifies the generated samples into positive and negative samples. Furthermore, the classifiers are also employed for obtaining the evaluation metrics discussed in Section-4.3 of the main text.
MNIST: We use simple LeNet model (LeCun et al., 1998) for classification among different digits of MNIST $ 3, The model is trained with a batch-size of 256 using Adam optimizer with a learning rate of x 10 -3, 81 = 0.9 and 32 = 0.999. The model is trained for a resolution of 32 x 32 same as the pre-trained GAN for 12 epochs. After training the classifier has an accuracy of 99.07% on the test split MNIST dataset.
CelebA-HQ: We Re model (Xie al., 2017) for classification among different facial attributes contained in C Note that we train the classifier normal CelebA the ground truth values are available for it. The classifier is trained with a batch-size of 64 using Adamax optimizer with learning rate of 2 x 10-3, 31 = 0.9 and 32 = 0.999. The model is trained for a resolution of 256 x 256 for 10 epochs. We also employ image augmentation techniques such as horizontal flip, image resize, and cropping to improve the performance of the classifier. The trained model exhibits a test accuracy of 91.93%.
A.3 Unlearning Hyper-parameter:
Here we mention the syper-parameters pertaining to the proposed negative adaptation and unlearning stages. As mentioned, we use an EWC regularizer during adaptation to avoid overfitting. The value of X (Eq.3) is set to 5 x 10 for all the experiments. Further, y (Eq.5) is chosen between 0.1, 1 and 10 when _ Preputsion and CNL2 Preputsion are chosen as repulsion loss. is varied between 10 and 500 when CEL2 Prepulsion is chosen as repulsion (Eq.7) is varied between 0.1 and 0.001. These values are chosen and adjusted to ensure that both the loss components C'do and Crepulsion are minimized properly.
loss. Further, the value of a for LEL2 Leppulsion
A.4 Details of Baselines
Here, we present the details pertaining to the baselines presented in Table 1, 2, 3.
ontter//uuthbb.om///ssvaaaaaaaa wotendd-yttoooottttoeeoeeeee /misst_ classifier
4 https://hhhabbommm//aaaaantttffo aaa.gattiioaanndcssssiaattttaoio
Classificationn-f Paw.iii----aaaaa-tt---a--aa Attribbtes
20

---

Published in Transactions on Machine Learning Research (02/2025)
EU-k and CF-k (Goel al., 2022) propose to train just the last k layers of the model from scratch (in E.U-)) or from pre-trained initialization (in CF-k) for unlearning on the positive samples. We employ the same strategy to GANs directly with k = 10 layers. Further, él-sparse (Jia et al., 2023) proposes to use sparse weights for fine-tuning to unlearn the undesired features. To this end, they propose to use f-1 regularization while fine-tuning. Hence, for our case, we fine-tune the model on positive samples by adding an {-1 regularization on weights of the network.
For the few-shot adaptation baselines, directly employ the provided open-source codebase of CDC$ and RSSA for adaptation positive samples to obtain results.
B MNIST Qualitative Results
We present visual illustration of images generated after unlearning using various methods in figure 4. Here, unlearn class of digits 1, 4, and 8. We observe that all the proposed methods effectively unlearn the undesired classes. Moreover, it can be seen that although extrapolation leads to unlearning, it does so at the expense of the quality of the generated images. In contrast, the quality of the generated images after unlea unlearning using the proposed method leads to unlearning with plausible image quality. We refer the reader the reader to main text for quantitative evaluation.
IL.2 repulsion
NL2
EL2 repulsion
Original Samples
Extrapolation
C repulsion
6 9 2
5 9 2
6 9 2
8 9 2
Class 8 Class 4 Class
7
4
2
7 4
2
7 4
Z
7 4 2
4
4
4
2
1 3 2 4
9
6
+ 9 7
9
6
1
9
6
|
9 6
4 4 H
2
6
1
6
4
6
/
8 2 4
8
8
8
7
9
4
9 3 4
7
8
8
8
|
7
7
1
9
1
5
7
1
9
Figure 4: Results of Unlearning undesired feature (class) via different methods. The undesired class contains digits (top row), 4(second row), 8 (bottom row).
C Proofs
Proof of Theorem 1. The said objective function is given by-
min max Bp
E x
[log D&(x))
E [log(1 Dz(GG(z))) frepulsion
&
@upi
min max E
[log (x))+ +
E
[log11 - D&(x))] + Crepulsion
(9)
Op
x
x
Since & depends only on the first two terms of Eq. 9, the optimal discriminator as obtained Goodfellow al. (2014) is given as Dø- et al. (2014) gives
Substituting this in Eq. 9 and using standard results from Goodfellow tox
min DiSD DJsp(powwllpp) w + Erepulsion 8p
(10)
httpes///ghhab..cm//uuaaasshhhh hhtt:s//gghhab.omm/Sseeeeeaaaa 999//RSA
21

---

Published in Transactions on Machine Learning Research (02/2025)
(rrepattton L11
Since (0)
| exp [2 0N) 10 0x)] and p0(() =
repulsioo
2 exp [2 0r) 0-))5-1(( 50 0r)] are both Gaussian distributions, then Dx (pe (PN) = 10p-0N11 (KL(PE1PRR)1
=
1 Ox11
1
C11.2
repplsioo Substituting in the above we get
min DUSD(PEAN (P 1--)
(11)
8p
Crepulsiion reppusioo CNL repulsionn
LN2
Similar to above argument, (x 10, -0v11 = -DKI (Kll()))R))
- -10pp 0x112
C
Substituting in the above we get
min DJsp(Powwlll - DKL (22
(12)
(Crepulsionn repulsion ): Again, since Pº and p follow Gaussian distribution, the Hellinger divergence
EL2
between two Gaussian distribution is a shifted negative Manhabolis distance between the means of the two distributions, i.c., Du (z(plllp)) = 1 cxp (-110p 0N (12) = 1 - DH (r = exp (-100p - 6N l1?) =
CEL2 repuusion Substituting in above we get
min DJSD(Pe DH(PillPR 0p
(13)
min DJSD Dusp DH (H(
(14)
8p
This completes the proof of all the three statements.
Proof of Claim 1. For a given latent vector, + Go(z) is a map from parameter space to generated sample in the data space. Hence, we can use data-processing incquality to obtain
Dy (p S (y(pilleN)
(15)
DasD(Poww(lb)) D2 pX) S DusD(Paww D
(16)
This completes the proof.
D Results with Curated Datasets
As explained in the main paper, our method requires users to identify or annotate negative samples under leedback-based framework. This annotation used to adapt the GAN to negative samples during the Negative Adaptation phase and subsequently retrain on positive samples during the Unlearning phase. While human feedback one approach to obtain these samples, curated datasets of positive and negative samples can also serve this purpose.
Curating datasets, however, can be challenging, particularly when the feature, concept, or class to be un- learned is subtle or complex and not readily available in standard datasets. In such cases, users may need to create a custom dataset. By contrast, our numan-foedbadde approach involves annotating samples generated by the GAN, reducing the need for external dataset creation. Nevertheless, if curated dataset of positive and negative samples is available, our method can be scamlessly adapted to utilize it.
To demonstrate this, conduct experiments all three datasets (MNIST, AFHQ, and CelebA-HQ) using curated dataset samples in both the Negative Adaptation and Unlearning phases. Specifically, for the Negative Adaptation phase, we use samples from the original dataset that were pre-annotated with the undesired feature or label. For instance, in the CelebA-HQ dataset, we selected all samples with a positive label for the undesired feature (e.g., Bangs). The GAN is then adapted to these samples using the training objective described in Section 3.2 (Eq. of the main paper) obtain the parameters 0y. the Unlcarning phase, the GAN was retrained on the remaining dataset samples using the objective described in Section 3.3 (Fq. 4).
22

---

Published in Transactions on Machine Learning Research (02/2025)
We performed these experiments MNIST (unlearning 'Class 8), AFHQ (unlearning 'Cat'), and CelebA- HQ (unlearning 'Bangs".. The results of these experiments are summarized in Table 6, comparing the performance of using curated dataset samples against GAN-generated samples.
Table 6: Comparison of results using curated dataset samples versus GAN-generated samples for negative adaptation. PUL (1), FID (1), and Ret-FID (L).
Method
MNIST (Class
AFHQ (Cat)
CelebA-HQ (Bange)
PUL
FID
Ret-FID
PUL
FID
Ret-FID
PUL
FID
Ret-FID 6.05
w/ dotaset samples
7.09 0.8
L08
10.04
w/ GAN sumples (using Areepalonn
96.22 + 0.34
8.50 1.8 0.10
06.66-0.66
0.12
KI7 0.14
90.451100 +.45+100 11.16 + +DB 7.34 = 0.32
As shown in Table 6, using curated dataset samples during the Negative Adaptation phase achieves per- formance comparable to that of GAN-generated samples. This demonstrates the flexibility of our method, allowing users choose the most convenient approach depending on the availability of datascts. These results will be included in the supplementary material provide further clarity on this point.
E Effect on other features after Unlearning
As previously discussed, the unlearning procedure aims to exclusively erase undesired features without impacting other features. Consequently, it becomes imperative assess whether the unlearning process exerts any influence other features. To address this concern, we introduce plots illustrating the percentage change in the presence of other features.
Specifically, we undertake the generation of 15, 000 random samples from both the pre-trained GAN and the GAN after the unlearning procedure. Subsequently, employing the pre-trained classifiers (for comprehensive details, please refer to the Supplementary), we calculate the occurrence of specific features within the two GANs. We report the percentage change in these numbers demonstrate how has the unlcarning process affected this feature. Hence, lower percentage change is better as it means that the other features are not affected after unlearning. This experiment is repeated across multiple features, and our findings after unlcarning Bangs in CelebA-HQ are depicted in figure 5. We observe that for all the features, unlcarning via extrapolation leads to significant changes in other features. Whereas, unlearning via the proposed method leads to minor changes in the features. For instance, since Bangs are highly correlated with gender, we observe that unlearning Bangs via extrapolation leads to an increase in Males, However, unlearning via the proposed method leads minor changes in the number of samples with Male features. This shows that the proposed method is effective in erasing the undesired feature while preserving other features. Tt can be observed that in majority of the cases, extrapolation leads to significant change in the features, indicating that unlearning via extrapolation leads to significant change in other features well. This also indicates that extrapolation leads to unlearning of several correlated features. On the other hand, we observe that unlearning using the proposed method with repulsioo gives the least change in most of the cases. This illustrates the efficacy of the proposed method preserving features other than the unlearnt feature.
23

---

Published in Transactions on Machine Learning Research (02/2025)
Extrapolation
Extrapolation
I
300
Nus
300
% change Features
% change features
Legps
Loup
100
100
Arched Eychrros
Artructive
Boga Under Eyes
Blungs
Big Lips
Big Nowr
Mlack Hair
Browen Ilire
Eyeglaoos
Gontoe
Gray Hairr
High Cheekbverss
Male
Slighdy Open
No Beard
Palk Skit
Stdehurns
Smilling
Straight Hair
Wearing Barrings
Wearing Lipolick
Young
Arched Eyehroos
Attructive
Bugg Unster Eyes
Bangs
Bite Lips
Big Noor
Macc Hair
Brooen Tinie
Fyeghaoon
Goatoo
High Cheekbvaes
Malo
Slighdy Open
No Board
Palk Skin
Stéchurns
Smillng
Straight Hair
Weating Earrings
Wearing Lipptick Young
Wary
Blood
Gray
Warr
(a) Unlearnt feature: Bald
(b) Unlearnt feature: Bangs
400
Extrapolation
Extrapolation
% change In Keatures
I
change features
200
200
%
Arched Kyebrows
Attraclive
Baga Under Rayes
Hanng
Neg Nose
Black Malr
Beown Hair
Fyaglooss
Gratee
Hair
High Rheekinnss
Male
Stightl Open
No Board
Pule Skin
Sileburns
Smiling
Striight Hair
Wearlng barringss
Wouring Ligaick Young
Arched Eyeboows
Attraclive
Bage Under Dyees
Hangg
Big Nove
Black Hair
Blond Hulr
Brown Hair
Fyeghwos
Goalee
Halr
High Bhekkunss
Male
Stightly Open
No Board
Pale Skin
Stleburns
Smiling
Straight Hair
Wearing karrisss
Wanring Ligatck
Wig
Blond
Wig
Wearing
(c) Unlearnt feature: Hats
(d) Unlearnt feature: Eyeglasses
Figure 5: Percentage (%) change in several features after unlearning specific features. A lower percentage change denotes that the method has suecessfully preserved that feature after unlearning. Here, clip the bar to 100% if the percentage change in that feature is more than 100% to make the plots legible.
F Visualization of Generated samples
Here, we provide visualization of random samples generated from the original GAN and the GAN after unlearning to give an intuitive/qualitativee idea of generation quality. We provide the results by using reppatsion as it gives the best performance in most of the cases. These results are presented for all the datasets in figure 6, 8. While the results in Section 5.4 provide quantitative idea of generation quality, these visualizations provide qualitative idea of the same.
CEL2
24

---

Published in Transactions on Machine Learning Research (02/2025)
0
/
0
0
0
9
0
6 6
3
8
4
3 0
0 0 5
2
4
3
0
7
7
9
2
2
3
5
0
7
1
4
6 2
0
6
2
4
8
9
6
2
4 9
5 9
6
2
7
0
0
2
2
8
1
0
6 773
2
2
2
3
9 3
6
6
0
0
9
6
6
1
5
4
5
7 4 2
6 9
5
2
7767
6
5 + 7
5
0 9
2
5
5
6
2
4 0 4
/
5
a
0
4
1 8
2
6 9 4
3
776
7
5
5
7
3
6
9
6
6
3
2
0
0
9
4
8
0 5
0
9 7
7
5
0
8
/
1 3
0
3
3
7 1
9
/
6
+
3
/
/
8
7
6
1
5
8
9
7
2
2
5
0
0
0
8
/
7
4
5
6
O
9
3
8
2
4
2
3
6
8
5
7
4
2
7
5
0
4
6
0
Original GAN
Class
Class
Class 8
Figure Visualization of random MNIST samples generated before and after unlearning.
Original GAN
Cat
Dog
Wild
Figure 7: Visualization of random AFHQ samples generated before and after unlcarning.
25

---

Published in Transactions on Machine Learning Research (02/2025)
Original GAN
Bangs
Hats
Bald head
Eyeglass
Figure 8: Visualization of random CelebA-HQ samples generated before and after unlearning.
26


Reviews:
[{"review_id": "Hjg1iPYQi9", "reviewer": "Reviewer_cEyE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- the method does seem to work well at its intended goal. The experiments reasonably convinced me of this. \n\nWeaknesses:\n\n- it seems to just be a bunch of existing methods glued together. however, if it works, I guess one shouldn't count this against the technique.\n\n- I didn't find any of the theoretical results to be very enlightening, although I didn't catch any errors.\n\n- I have no intuitive sense of \"how bad\" the decline in FID due to their method might be, in terms of actual perceived image quality.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors’ goal is “unlearning” in generative models, specifically GANs. This is not the usual idea of “unlearning”, meaning removing any trace of certain training data, but rather suppressing outputs with specific undesired features without overly distorting the other outputs.\n\nIn their model, a user will obtain some samples that show the undesired feature. These are used to adapt the base GAN using an off-the-shelf few-shot adaptation technique. The result is a GAN which tends to output negative features. Then, the base GAN is trained again, this time on positive samples, but with an additional regularization term to steer its parameters away from the negative-adapted GAN. Different choices of regularization term have slightly different interpretations in terms of which divergence the GAN training procedure is approximating.\n\nThe authors use StyleGAN2 on several standard datasets (MNIST, AFHQ, and CelebA-HQ). In these experiments, they train a simple classifier to act as a simulated human (this seems like a reasonable choice and doesn’t undermine their method).\n\nIn the experiments, their method works substantially better than baselines for unlearning, although at the cost of some quality as measured by FID. They also have an ablation study giving some evidence that their repulsion loss is important to performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The table of results is very tiny and hard to read. I would suggest that the authors split it up somehow to allow for bigger font. Fortunately, TMLR doesn't have page limits like CS conferences.", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "aCj3RjWiEQ", "reviewer": "Reviewer_H1FD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nThe paper is very well-motivated, organised, and written. The experimental section is throrough well-thought out and\nincludes detailed ablation studies that motivate the contribution of the method. It is compared to the relevant\nrelated work where it makes sense. The theoretical contributions offer interesting connections to other losses in\nthe GAN literature.\n\nWeaknesses\n\nThe main challenge for the paper is that the contribution is not novel. Sun et al. 2023 also\nintroduce a cascade based unlearning method which uses StyleGAN2. The difference seems to be\nmore that this paper uses one of several revulsion losses while Sun et al. use a different\nloss entirely. It would be better to highlight this difference as it seems both papers use\nthe same GAN architecture.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a new method for unlearning generative models so they do not generate images\nfrom a certain class or with certain features. The method is demonstrated to work on StyleGAN2 and\nworks in the few shot setting.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would be helpful to better explain how this work differs from Sun et al. in the related work section. It is already compared against in the experimental section.\n\nThe paper would be strengthened if it could be compared against other methods for feature unlearning. It seems all comparisons are only for class unlearning.", "broader_impact_concerns": "I have no ethical concerns about this work that would require adding an ethical impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Qso2NDIMb7", "reviewer": "Reviewer_ouU3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\nThe problem is hard and very interesting, and the experimental results are convincing.\n\nWeaknesses:\nIMO the paper has two main weaknesses:\n- 1 applicability of the proposed method, from what I understand, the method requires human feedback to annotate examples to remove, which can be tedious and costly (this is however standard in LLMs with RLHF). Maybe a more realistic/interesting would be to suppose that one has access to a dataset of the mode to remove (e.g only gods, or only harmful content), this would be another project, but out of curiosity would it be possible to adapt the proposed technique ti this setting?\n- 2 clarify. I am not an expert in unlearning, but I am not sure I fully understand the method, here is a list of clarification questions\n\n- why is $\\mathcal{L}_{adv}$ named thi was? is this an adversarial term?\n- From what I understand, the repulsion is done in the parameters space?\n- In Equation 3, could you clarify what does $\\mathcal{L}(\\mathcal{S}_n | \\theta_G)$ mean? It seems that you compare samples against parameters.\n- the technique seems quite generic, why make it specific to GANs? Could it be applied to diffusion/flow matching as well?\n\n\nMinor:\n- ¨fischer' >> ¨Fisher¨\n- ¨Francesco D’Angelo and Vincent Fortuin¨ is duplicated in the bibliography\n- In Thm 1, why not directly replace $\\Sigma$ by $I$?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a new ¨unlearning¨ technique, i.e., how to remove the given mode a distribution, i.e. how to re;ove one of the Gaussian in a mixture of Gaussian. This problem has obvious applications to prevent models from generating harmful content.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Clarify the proposed technique, maybe by writing the equations more exhaustively", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers believe that the paper meets the standards and criteria for acceptance to TMLR.
Decision: Accept (poster)