Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
Authors: Alvin Heng
Harold Soh
Venue: NeurIPS 2023 spotlight (2023)
Keywords: generative models
forgetting
TL;DR: We devise a method inspired by continual learning for selective forgetting of concepts in deep generative models.

Abstract: The recent proliferation of large-scale text-to-image models has led to growing concerns that such models may be misused to generate harmful, misleading, and inappropriate content. Motivated by this issue, we derive a technique inspired by continual learning to selectively forget concepts in pretrained deep generative models. Our method, dubbed Selective Amnesia, enables controllable forgetting  where a user can specify how a concept should be forgotten. Selective Amnesia can be applied to conditional variational likelihood models, which encompass a variety of popular deep generative frameworks, including variational autoencoders and large-scale text-to-image diffusion models. Experiments across different models demonstrate that our approach induces forgetting on a variety of concepts, from entire classes in standard datasets to celebrity and nudity prompts in text-to-image models.

Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
Alvin Heng', Harold Soh
Dept. of Computer Science, National University of Singapore Smart Systems Institute, National University of Singapore {alvinh, harold) mus edu.sg
Abstract
The recent proliferation of large-scale text-to-image models has led to growing concerns that such models may be misused to generate harmful, misleading. and inappropriate content. Motivated by this issue, we derive a technique inspired by continual learning to selectively forget concepts in pretrained deep generative models. Our method, dubbed Selective Amnesia, enables controllable forgetting where user specify how concept should be forgotten. Selective Amnesia be applied to conditional variational likelihood models, which encompass a variety of popular deep generative frameworks, including variational autoencoders and large-scale text-to-image diffusion models. Experiments across different models demonstrate that approach induces forgetting on variety of concepts, from entire classes in standard datasets to celebrity and nudity prompts in texl-to-image models. Our code is publicly available at amnesia
MNIST VAE
CIFARIO DDPM
Stable Diffusion
Forget 'O' class
Forget 'arpllan'' class
Brad wearing santa
0
0
2
235
3
3
2 3
3
3
4
4
5
5
4
4 6 8
5
5 7 9
7 8
7
6
8
9
8
Angelina Jolie wearing santa
Original
Ours
Original
Ours
photo of naked person
*
Original
Ours
Original
Ours
Forget 'airplane' class
STL10 DDPM
Added by authors for publication
Figure 1: Qualitative results of our method, Selective Amnesia (SA). SA can be applied to a variety of models, from forgetting textual prompts such specific celebrities nudity in text-to-image models to discrete classes in VAEs and diffusion models (DDPM).
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Introduction
Deep generative models have made significant strides in recent years, with large-scale text-to-image models attracting immense interest due to their excellent generation capabilities. Unfortunately, these models can also be misused to create realistic-looking images of harmful, discriminatory and inappropriate content [T. For instance, one could generate Deepfakes convincing fake images and inappropriate content involving real individuals (e.g., nude celebrities) 3 A naive approach to address this issue is to omit specific concepts or individuals from the training dataset. However, filtering datasets of billions of images is challenge in itself. Moreover, it entails retraining the entire model from scratch each time new concept is to be forgotten, which is costly in terms of compute and time. this work, our goal to retrain the model to only forget specific concepts, i.e., to induce selective amnesia.
Several efforts in this direction have been made in the field of data forgetting 4477. as well concept erasure in the context of texi-to-image diffusion models [ 8 9]. However, works in the former either focus on discriminative models, or require special partitioning of data and model during training. The few works in the latter nascent field of concept erasure target toxt-to-image diffusion models and work by exploiting specific design characteristics of these models. Here, we aim to develop a general framework that is applicable to variety of pretrained generative models, without access to the original training data.
Our key insight is that selective forgetting can he framed from the perspective of continual learning. Ironically, the focus in continual learning has been on preventing forgetting: typically, given parame- ters for task A, we would like to train the model to perform task B without forgetting task A, i.e., 0a- AA B. In our case, we have a model that is trained generate and B, and we would like the model to only generate B while forgetting A, i.e., AA,B- - 0g.
In this work, we show that well-known methods in continual learning can be unified into single objective function that can be used to train models to forget. Unlike prior works, our method allows for controllable forgetting, where the forgotten concept be remapped to a user-defined concept that is deemed more appropriate. We focus our scope on conditional variational likelihood models, which includes popular deep generative frameworks, namely Variational Autoencoders (VAEs) [TO) and Denoising Diffusion Probabilistic Models (DDPMs) [II]. To demonstrate its generality, we apply our method, dubbed Selective Amnesia (SA) to datasets and models of varying complexities, from simple VA trained on MNIST, DDPMs trained on CIFARIO 10 and STL10, to the open-source Stable Diffusion 12 text-to-image model trained on large corpus of internet data. Our results shows that SA causes generative models to forget diverse concepts such as discrete classes to celebrities and nudity in manner that is customizable by the user.
Our paper is structured as follows. We cover the relevant background and related works in Sec. We introduce Selective Amnesia (SA) in Sec. followed by in-depth experimental results in Sec. Finally, we conclude in Sec. 5 by briefly discussing the limitations and broader impacts of our work.
Background and Related Work
2.1 Variational Generative Models
Conditional Variational Autoencoders. Conditional Variational Autoencoders [10] are generative models of the form p(x, z|0, c) p(x ())p(2)0,0))) c), where is the data (c.g.. an image), is the concept/class, and p(z]0, c) is prior over the latent variables z. Due to the intractability of the posterior p(z c), VAEs adopt approximate posterior q x, c) and maximize the evidence lower bound (ELBO),
log 1o 10 c) 2 logp(x(0,2, 2, DKL(g(zlo, (lz)0 c)) ELBO(x(0,
Conditional Diffusion Models. Diffusion models [ are class of generative models that sample from a distribution through an iterative Markov denoising process. A sample xy is typically sampled from a Gaussian distribution and gradually denoised for T time steps, finally recovering a clean sample Xg. In practice, the model is trained to predict the noise cXx,, 1, clf) that must be removed from the sample x, with the following reweighted variational bound: ELBO(x (, c) Σ ||e(x,, f, c(f)
ella, where X, vāxxo + vI- ā,e for (O, 1), āe constants related to the noise schedule

---

in the forward noising process, Sampling from a conditional diffusion model can be carried out using classifier-free guidance [T].
2.2 Continual Learning
The field of continual learning is primarily concerned with the sequential learning of tasks in deep
neural networks, while avoiding catastrophic forgetting. variety of methods have been proposed to tackle this problem, including regularization approaches [14] 15), architectural modifications [T6 17). and data replay [8]. We discuss two popular approaches that will be used in our work: Elastic Weight Consolidation and Generative Replay.
Elastic Weight Consolidation. Elastic Weight Consolidation (EWC) [1] adopts Bayesian approach to model the posterior of the weights for accomplishing two tasks, DA and DB, given model 0 that has learnt DA- The Laplace approximation is applied to the posterior over the initial task DA, giving rise to quadratic penalty that slows down learning of weights that are most relevant to the initial task. Concretely, the posterior given by log p(0) DA, DB) = log p( D (f) EE 2- (0,- 0,)2, where F' the Fisher information matrix (FIM) and À is weighting parameter. In practice, a diagonal approximation F; = Epppje>) [(0; efficiency. F/ can be viewed as a sensitivity measure of the weight 6, on the model's output. For variational models, we modify the F; to measure the sensitivity of 0; the ELBO: F; (pixx(",cip(e]] [50 ELBO(x|0, c)
ao; log [] is adopted for computational
Generative Replay. Generative Replay (GR) [18 was proposed as a method where generative model can be leveraged to generate data from previous tasks, and used to augment data from the current task in the training of a discriminative model. More generally, it motivates one to leverage generative models for continual learning, whereby without needing to store any of the previous datasets, a model can be trained on all tasks simultaneously, which prevents catastrophic forgetting.
Our work leverages EWC and GR train a model sequentially forget certain classes and concepts. There have been several works utilizing these techniques for generative models, such as GANs [19] 20) and VA 21. However, these works tackle the traditional problem of continual learning, which seeks to prevent forgetting.
2.3
Data Forgetting
The increased focus on privacy in machine learning models in recent years, coupled with data privacy regulations such as the EU's General Data Protection Regulation, has led to significant advancements in the field of data forgetting. Data forgetting was first proposed in 4
statistical query learning problem. Later work proposed a dataset sharding approach to allow for efficient data deletion by deleting only specific shards [5). Alternative methods define unlearning through information accessible directly from model weights [6). while (7) proposed a variational unlearning method which relies on a posterior belief over the model weights. Wu al. [22] proposes method to remove the influence of certain datapoints from a trained model by caching the model gradients during training.
Our method only requires access to trained model and does not require control over the initial training process or the original dataset, making it distinct from 4 22 In addition, earlier methods are designed for discriminative tasks such as classification and regression 77. while we focus on deep generative models.
2.4 Editing and Unlearning in Generative Models
Several works have investigated the post-hoc editing and retraining of generative models. Data redaction and unlearning have been proposed for GANs (23 and Normalizing Flows (24] However, both methods exploit specific properties of the model (discriminators and exact likelihoods) which are absent from variational models, hence are not comparable to our work. Moon al. (25] implicitly assumes that a generator's 's latent space has disentangled features over concepts, which does not apply to conditional models (a given latent z can be used to generate all classes just by varying the conditioning signal c). Bau et al. [26] directly modifies single layer's weights in a generator to alter its semantic rules, such as removal of watermarks. The work focuses on GANs and preliminary experiments more drastic changes that forgetting necessities led to severe visual artifacts.
3

---

2.5 Concept Erasure in Text-to-lmage Models
Large-scale text-to-image models 12 27 28 can be misused to generate biased, unsafe, and inappropriate content []. To tackle this problem, Safe Latent Diffusion (SL.) M proposes an inference scheme to guide the latent codes away from specific concepts, while Erasing Stable Diffusion (ESD) g proposes a training scheme to crase concepts from a model. Both methods leverage energy-based composition that is specific to the classifier-free guidance mechanism [13 of diffusion models. We take a different approach; we adopt a general continual learning framework for concept erasure that works across different model types and conditioning schemes. Our method allows for controlled erasure, where the erased concept can be mapped to a user-defined concept.
Proposed Method: Selective Amnesia
Problem Statement. We consider dataset D that can be partitioned as D = Dj U D. = {(x]") of U {(x,")) cr ((m))
where D, and D, correspond to the data to forget and remember respectively. The underlying distribution of D is a joint distribution given by p(x, c) = p(x) We further define the distribution over concepts/class labels p(c) = - (.):(c) where Σief.r 6i = 1. The two concept/class distributions are disjoint such that py(c,) = where c, ~ (c) and vice-versa. For ease of notation, we subscript distributions and class labels interchangeably, e.g., Py(c) and p(cy).
)m )
We assume access to a trained conditional generative model parameterized by 6" arg max, Ep(x.e) log p(x(0, c), which is the maximum likelihood estimate (MLE) of the dataset D. We would like to train this model such that it forgets how to generate D |cy while remembering D,Ic,, cy. A key criteria is that the training process must not require access D. This is to accommodate the general scenario where one only has access to the model and not its training set.
A Bayesian Continual Learning Approach to Forgetting. We start from Bayesian perspective of continual learning inspired by the derivation of Elastic Weight Consolidation (EWC) [4]
log p(0 D5. D.) = log p( 1 + log + C.
log p(D, (0. (,, + log ( D.) - log (D))
For forgetting, we are interested in the posterior conditioned only on D,.
log D,) = Logr(D,l00 + logg(()) D,) +C
logp(x, cy) > Σ
F; (0. 6,) +C
(I)
where we use log logp( P(Dy10) = log =
+ log p(cc) ) so that the conditional likelihood is explicit, and substitute log ((g)( D,D,,,D,. with the Laplace approximation of EWC. Our goal is to maximize log p(0) D.)) to obtain a maximum a posteriori (MAP) estimate. Intuitively, maximizing Eq. lowers the likelihood log p(x, (,,,0,,,), cy), while keeping close to 0*
Unfortunately, direct optimization hampered by two key issues. First, the optimization objective of does not involve using samples from D,. In preliminary experiments, we found that without replaying data from Dr. the model's ability to generate the data to be remembered diminishes over time. Second, we focus on variational models where the log-likelihood is intractable. We have the ELBO, but minimizing a lower bound does not necessarily decrease the log- log-likelihood. In the following, address both these problems via generative replay and a surrogate objective.
3.1 Generative Replay Over Dr
Our approach unify the two paradigms of continual learning, EWC and GR, such that they can be optimized under a single objective. We introduce an extra likelihood term over D, that corresponds generative replay term, while keeping the optimization over the posterior of D, unchanged:
log (0gp(0))))
1
1ogp(x,(0,0m c0)
Σ
F (0. 2
0,)² + log 10, c,) + log
+ C.
2
(2)

---

Original
Naive
Ours
0
O
I
0 O
I
1
2
2
3
3
2
R
$
3
2
3
3
4
x 6
5 7 9
5
4
5 7
4
5
5 7
6
7
6 6
7
6 6
7 9
8
8
2
4
8
7
Figure 2: Illustration of training VAE forget the MNIST digit O. The 'original' column shows the baseline samples generated by the VAE. In the 'naive' column, we train the VAE to optimize Eq. 3 with Dy being the '0' class, while in the 'ours' column we train using the modified objective Eq.
A complete derivation is given in Appendix A.I The term log corresponds to a prior over the parameters 0. Practically, find that simply setting the uniform prior achieves good results, thus rendering it constant with regards to optimization. With the expectations written down explicitly, our objective becomes
C - ((x (lg - - >
Fi (0: 0,)2 (p(x(e)p-(e) [logp(x|0, c)]
(3)
As we focus on conditional generative models in this work, the expectations over p(x[c)py(c) and p(x[c)p-(c) can approximated by using conditional samples generated by the model prior to training. Similarly, the FIM is calculated using samples from the model. Thus, Eq. can be optimized without the original training dataset D. Empirically, we observe that the addition of the GR term improves performance when generating D, after training to forget D (see ablations in Sec. 4.
3.2 Surrogate Objective
Similar Eq. I Eq. suggests that we need minimize the log-likelihood of the data forget
Ex Px.e-p(xlc)py (c) log pl x (0, c)]. With variational models, we only have access to the lower bound of the log-likelihood, but naively optimizing E by replacing the likelihood terms with the standard ELBOs leads to poor results. Fig. Dillusttttss samples from a VAE trained to forget the MNIST digit 'O'; not only has the VAE failed to forget, but the sample quality the other classes has also greatly diminished (despite adding the GR term).
We propose alternative objective that is guaranteed to lower the log-likelihood of Dy, as compared to the original model parameterized by 0- Rather than attempting to directly minimize the log- likelihood or the ELBO, we maximize the ligeliihd of a surrogate distribution of the class to forget, (x # p(x(cy). We formalize this idea in the following theorem.
Theorem 1. Consider a surrogate distribution g(x|c) c) such that q(x(c:; ()))= * p(x(cç). Assume we have access the MLE optimum for the full dataset 6* arg max, Expxx,e) (log p(x c)] such that Eptc) [PKL(pxxee]lle (xpx) c]]]= = 0. Define the MLE optimum over the surrogate dataset as 09 arg max (g(x)c)py(C (c) loggpxx c)}. Then the following inequality involving the expectations of the optimal models over the data to forget holds:
Epix e)py(e) [log p(x (2x)0))c) c)] < (ppxlchpy(e)) [log ((x((( ,c]]
Theorem I tells us that optimizing the surrogate objective arg max, (gxx logp(xx/p c)] is guaranteed reduce Ep(x e)py(e) [log (0 c)}, the problematic first term of Eq. from its original starting point 0*.
Corollary 1. Assume that the MLE optimum over the surrogate, 89 arg max, (gt [log p(x(f,,)) is such that Epy(e) (Dxul(())c)lp((((@) the gap presented in Theorem
0. Then
Epixlcipy(c) plog(()(09,0) c) logp(x(0" c)] -Epy(o)
Corollary tells us that the greater the difference between g(x(cy) and p(x|ey), the lower the log- likelihood over D, we can achieve. For example, could choose the uniform distribution is
5

---

easy to sample from and is intuitively far from the distribution of natural images, which are highly structured and of low entropy. That said, users are free choose q(x(c,), e.g., induce realistic but acceptable images, and we experiment with different choices in the Stable Diffusion experiments (Sec|4.) 4.2)
Putting the above elements together, the Selective Amnesia (SA) objective is given by
L = (x((())py(e) [(x]0, c)] - Σ
F, (0, -0,) + + (p(xcc)p.(e) [logp(x|0,c) c)]
2
F; (0, - -,,² (c(x)c)p, [ELBO c)]
2 (c ELBO 6, c)] A Σ
(4)
2
where we replace likelihood terms with their respective evidence lower bounds. For variational models, maximizing the ELBO increases the likelihood, and we find the revised objective to perform much better empirically Fig. 2 (right) shows results of the revised objective when applied to the MNIST example, where we set x(x]cx]xxx)p)pc cy) uniform distribution over the pixel values, U'( 1]. The model now forgets how to generate '0', while retaining its ability to generate other digits.
Experiments
In this section, we demonstrate that SA is able to forget diverse concepts, ranging from discrete classes to language prompts, in models with varying complexities. For discrete classes, we evaluate SA on MNIST, CIFARIO and STL10. The former modeled by conditional VAE with simple MLP architecture, which is conditioned by concatenating a one-hot encoding vector to its inputs. The latter two datasets are modeled by conditional DDPM with the UNet architecture, which is conditioned using FiLM transformations [29] within each residual block. Class-conditional samples are generated with classifier- free guidance [13] We also experiment with the open-source text-to-image model
Stable Diffusion v v1.4 [12), where the model is conditioned CLIP (30) text embeddings using the
cross-attention mechanism. Further experimental details can be found in Appendix B
In addition to qualitative comparisons, we performed quantitative analyses using three types of metries for the discrete classes:
Image Quality Metrics. First, evaluate the image quality of the classes to remember using standard metrics such as the Fréchet Inception Distance (FID). Precision, and Recall [1 32). Ideally, would like SA have minimal effects the image quality of the classes remember.
Probability of Class to Forget. Second, using an external classifier, we evaluate generated samples from the class to forget to ensure that the class has been successfully erased. The probability of a class to forget is defined as (x(x(0,,,) from our trained model, and Pa x) is pretrained classifier. If we choose q( q(x(c,) to be an uninformative distribution, such as the uniform distribution, this should approach 1/N /Nedasses(=11100 for the datasets studied here) as the classifier becomes completely uncertain which class it belongs to.
(P [Pa
cy(x)), where the expectation is over samples generated
Classifier Entropy. This is the average entropy of the classifier's output distribution given x-, defined as H (y((())))) = -Eptx|o,e,) IΣ, Py = c, (x log (y = c,(x)]. When we choose q(xcc,) to the uniform distribution, all class information in the generated x, should be erased. The entropy should therefore approach the theoretical maximum given by - > 1 log To = 2.30, To
as the classifier becomes maximally uncertain and assigns probability of 1/ to every outcome.
4.1 MNIST, CIFAR10 and STL10 Results
In this experiment on MNIST, CIFARIO and STLI0, we attempt to forget the digit '0' in MNIST and the 'airplane' class in CIFAR10 and STL10. We choose q to be uniform distribution over the pixel values. In brief, the results suggest that SA has successfully induced forgetting for the relevant class, with minor degradation in image diversity of the classes to remember. Qualitative samples are shown Fig. [ where it clear that the classes to forget have been erased to noise, while the quality of the classes to remember remain visually indistinguishable from the original model. Additional samples are shown in Appendix D.T the following, we perform a quantitative comparison using metrics shown in T

---

Table 1: Quantitative results for forgetting on the MNIST, CIFARIO and STL10 datasets. H (Pooyyxx and Pa(y = C xyy indicate the entropy of the classifier's distribution and the probability of the forgotten class respectively. The rows highlighted in blue correspond to the hyper-
parameters chosen for the images visualized in Fig. |
The rows highlighted in orange are ablation
results for CIFAR1O.
FID (4)
Precision (t)
Recall (1)
(P xx)
Poly Cy xy)
Original
0.103
0.967
MNIST
À = 100
2.19
0.0580
Original
9.67
0.390
0.788
0.0293
0.979
Classes
9.46
0.399
0.783
À = 10
9.08
0.412
0.767
1.47
0.156
CIFARIO
A ==
19.3
0.286
0.770
0.977
0.700
À = 50
8.41
0.428
0.760
1.17
0.142
À 100
8.33
0.429
0.758
1.07
0.235
No GR (A = 10)
126
0.0296
0.268
0.893
0.737
Original
14.5
0.356
0.796
0,0418
0.987
STLI0
Classes
14.5
0.360
0.803
10
18.0
0.378
0.713
1.80
0.0189
First, we evaluate the information content left in the generated x by examining the rows in Table D that are highlighted in blue. On MNIST, there is a 96.7% probability of classifying the samples of the '0' class from the original model correctly, and correspondingly low entropy its distribution. However, after training with À = 100, the probability drops to 5.8%. while the entropy closely approaches the theoretical maximum of 2.30, indicating that any information in the generated x about the digit 'O' has been erased. We see similar result for the CIFAR10 10 and STL110 diffusion models, where the entropy increases significantly after training, although it does not approach the maximum value as closely as the VAE.
Next, we evaluate the image quality of the classes remember on CIFARIO and STL10. We compare with two baselines: the original models and version trained only on the nine classes to remember. Surprisingly on CIFARIO, training with À 10 actually improves FID slightly, which priori is unusual as the baselines should serve natural upper-bounds on image quality. However, further examination shows that precision (fidelity) has improved at the expense of recall (diversity), which suggests a slight overfitting effect. On STL10, there is similarly a slight improvement in precision, but with drop in recall, which overall resulted in a higher FID score. This can attributed to the fact that we chose the number of GR samples to be relatively small at 5000 samples, as sampling for diffusion models can be expensive. We hypothesize that this can alleviated by increasing the GR sample size, but we leave this to future investigation.
Ablations. We conduct ablations on the À parameter and on the generative replay term in our objective function, using DDPM trained on CIFARIO. Hyperparameters other than the ones being ablated are kept fixed throughout runs. The results are highlighted orange in Table [ Starting with ablations X, see that when À 1, image fidelity measured by FID and precision is significantly poorer than larger values X, showing that the FIM term is crucial in our training scheme. As À is increased, there is drastic improvement in fidelity, which comes at slight cost to diversity as measured by recall, although the changes are relatively minor across the tested range of A [10, 100). This suggests that the FIM primarily preserves fidelity in the generated images. When comparing classifier entropy, we see that increments beyond À = 10 decreases entropy further from the bound, which indicate some information leakage the forgotten samples being generated. Moving to generative replay, we find that all metrics suffer significantly when the term omitted. In summary, our ablation studies show that generative replay is crucial in our method, and intermediate values À E [10, 100] is sufficient for good performance.
4.2
Case Study: Stable Diffusion
With the potential of large-scale generative models to be misused for impersonations and deepfakes, we apply SA to the forgetting of famous persons with SD v1.4. We leverage the fact that with language conditioning, we can choose ((x(c)) to be represented by images that are appropriate substitutes of the concept to forget. For instance, we attempt to forget the celebrities Brad Pitt and Angelina Jolie, thus set C, "Brrd Pitt") } and cy= "Anngelina Jolie")
Forget Famous Persons.
(c))
7

---

L.. in grocery store
[-.] petting corgi
painting of (..] Van Gogh style
SD v1.4
SA (Durs)
SD v1.4
SA (Ouns)
SDD v1.4
SA (Ours)
Figure 3: Qualitative results of SA applied to forgetting famous persons. Within each column, the leftmost image represents SD v1.4 samples, the middle image represents SA with g(x(ey) set to "middle aged man/woman" and the rightmost image is SA with g(x(cy) set to "male/female clown". [..]] is substituted with either "Brad Pitt" or "Angelina Jolie".
Table 2: Quantitative results from the GIPHY Celebrity Detector. For SA, we use the variant with g(xle,) set to "middle aged man" or "middle aged woman" " for forgetting Brad Pitt and Angelina Jolie respectively. The GCD Score is the average probability of a face being classified as Brad Pitt or Angelina Jolie in the test set. Numbers brackets are standard deviations. Note that the standard deviations are typically much larger than the mean, which indicates highly skewed distribution, i.e.,
a majority of faces have very low probabilities, but a few have very large probabilities.
Forget Brad Pitt
Forget Angelina Jolie
Proportion of images
Proportion of images
GCD Score (1)
GCD Score (4)
without faces ())
without faces (4)
SD v1.4 (original)
0.104
0.606 (0.424)
0.117
0.738 (0.454)
SLD Medium
0.141
0.00474 (0.0354)
0.119
0.0329 (0. 129)
ESD-x
0.347
0.0201 (0.109)
0.326
0.0335 (0.153)
SA (Ours)
0.058
0.0752 (0.1
0.0440
0.0774 (0.213)
and represent (x[[cy) with images generated from SD v1.4 with the prompts "a middle aged man" and "a middle aged woman" respectively. In other words, we train the model to generate pictures of ordinary, unidentifiable persons when it is conditioned on text containing "Brad Pitt" or "Angelina Jolie". In this way, our model still generates semantically-relevaan pictures of humans, as opposed to uniform noise if we had chosen the same gxx(c,) f) as the previous section.
To demonstrate the control and versatility of SA, we conduct second set of experiments where we map the celebrities to clowns, by setting g(x(cy) to images of "male clown" " or "female clown" generated by SD v For SD experiments, we only train the diffusion model operating in latent space, while freezing the encoder and decoder. Our qualitative results shown in Fig. 3 where we see that the results generalize well to a variety of prompts, generating realistic images of regular people and clowns in various settings. Additional samples are shown in Appendix D.2
We compare our results against the following baselines: 1) original SD v1.4, 2) SLD Medium 1 and 3) ESD-x [8) training only the cross-attention layers. We generate 20 images each of 50 random prompts containing "Brad Pitt" and "Angelina Jolie" and evaluate using the open-source GIPHY Celebrity Detector (GCD) 33 We calculate two metrics, the proportion of images generated with no faces detected and the average probability of the celebrity given that a face is detected, which we abbreviate GCD Score (GCDS). Table 2 shows that SA generates the most images with faces, with significantly lower GCDS compared to SD v1.4. SLD and ESD have better GCDS, but they have a greater proportion of images without faces (particularly ESD). Looking at the qualitative samples in Figs. 4 and 5S ESD sometimes generates faceless and semantically unrelated images due to its lace training process. Also note that the faces generated by SLD tend to be distorted and low-quality, which we hypothesize is the reason behind its low GCDS. Visual inspection of the top-5 images in terms of GCDS in Fig. shows that, despite the high scores, the images generated by SA
'This demonstration is not meant to suggest that the celebrities are clowns. It is meant solely as test to examine the versatility of the method map the forgotten individual to alternative unrelated concepts.
8

---

Brad Pit
close of Brad
Brad Pin laughing
park
KH
(A
ESD
0.599
0,507
0.381
0.255
SLD
Sample Images
Top-5 GCD Score
Figure 4: Comparisons between SA with ESD and SLD in forgetting Brad Pitt. We use SA with ((x|ey) ) set to "middle aged man". Images the left are sample images with the prompts specified per column. Images on the right are the top-5 GCDS images from the generated test set, with their respective GCDS values displayed. Intuitively, these are the images with the highest probabilities that the GCD network classifies as Brad Pitt.
Angelina Jolie
Angelina Jolie
holding bouquct
Angolina Wearing a hat
in black drees
Bowers
0.995
0,995
O 9K7
D. PK9
0.978
AXX
OIeA
SA (Ounn)
0.997
0,997
0.995
0
0.987
ESD
0 0.991
0.955
0.917
01856
ILM35
SLD
Sample Images
Top-5 Score
Figure 5: Comparisons between our method with ESD and SLD in forgetting Angelina Jolie. We use the variant of SA with q(x(cy) set to "middle aged woman". Images the left sample images with the prompts specified per column. Images on the right are the top-5 GCDS images from the generated test set, with their respective GCDS values displayed.
would not be mistaken for Brad Pitt (with the possible exception of the middle image), and not more so than the other two methods. Similar observations can made for the Angelina Jolie samples in Fig. 5 We also investigate the effects celebrities other than the one being forgotten in Sec. of the appendix. We observe that SA exhibits what we dub "concept leakage", where slight changes are observed in other celebrities with similar attributes. We view this double-edged sword, also means that SA can generalize to related concepts. If desired, this effect can mitigated by tuning only the cross-attention layers of SD g We discuss this in greater detail in Sec.
Forget Nudity, We also attempt to tackle the problem of inappropriate concept generation by training SD v1.4 to forget the concept of nudity. Unlike the previous celebrity setting, nudity is "global" concept that can be indirectly referenced through numerous text prompts, such as styles of artists that produce nude imagery. As such, train only the unconditional (non-cross atttenttttttnoon

---

SLD
ESD
SA OOus))
Added by authors for publication
Figure 6: Sample images with the prompt "a photo of naked person" " from the three approaches.
layers in SID v1.4, as proposed in [8] In this scenario, we represent g(x(ey) with images generated from SD v1.4 with the prompt "a person wearing clothes". ", which is a semantically-reles antonym of the concept of nudity. We let our prompts to forget be c₁ =("nudity", "naked", "crotic", "sexual") and sample them uniformly during training.
We evaluate the I2P dataset M1, which is collection of 4703 inappropriate prompts. Our results are in Table the appendix. where we compare against SD v1.4, SLD, ESD-u (train unconditional layers only) well as SD 12.1, which is trained nudity content was detected using the NudeNet classifier (with default detection threshold of 0.6, which results in some false positives). Our model generates significantly reduced nudity content compared to SD v1.4 and SD v2.1. SLD and ESD achieve hetter scores, potentially hecause they are model-specific and leverage inductive biases of Stable Diffusion, namely ecore-function composition. Qualitative samples between the three approaches are shown in Fig. di Similar to the celebrity experiments, we find that ESD tends generate arbitrary images that are not semantically-relevaan to the test prompt, due to its uncontrollable training process. On the other hand, SA generates semantically related images, but did not forget how generate nude images to the same extent. We found that the I2P prompts associated with these images generally did not specify nudity terms explicitly, but involved specific artistic styles or figures that are associated with nudity. Additional evaluations shows SA to perform better on prompts with explicit nudity terms (Table 4 appendix). Combining the positive traits of SA, such as controlled forgetting, with the efficayy of ESD's global erasure capabilities would be interesting future work.
dataset filtered for nudity. The quantity of
5 Conclusion, Limitations, and Future Work
This paper contributes Selective Amnesia (SA), continual learning approach to controlled forgetting of concepts in conditional variational models. Unlike prior methods, SA is general formulation and can be applied to variety of conditional generative models. We presented unified training loss that combines the EWC and GR methods of continual learning, which when coupled to a surrogate distribution, enables targeted forgetting of a specific concept. Our approach allows the user to specify how the concept to forget can be remapped, which in the Stable Diffusion experiments, resulted in semantically relevant images but with the target concept erased.
Limitations and Broader Impacts. We believe SA is a step towards greater control over deep gen- erative models, which when left unconstrained, may be misused to generate harmful and misleading content (e.g., deepfakes). There are several avenues for future work moving forward. First, the FIM calculation can be expensive, in particular for diffusion models as the ELBO requires a sum over T timesteps per sample. Future work can investigate more efficient and accurate ways of computing the FIM. Second, SA appears be more proficient at removing "local" specific concepts, rather than "global" concepts (e.g.. nudity); more work is needed on general methods that work well for both types of information. Third, SA requires manual selection of appropriate surrogate distribution; method to automate this process would be an interesting future direction. Finally, human assessment can also be explored to provide more holistic evaluation of SA's forgetting capabilities. From broader perspective, SA could potentially be used to alter concepts inappropriately or maliciously, such as erasing historical events. We believe that care should be taken by the community in ensuring that tools such as SA are used to improve generative models, and not propagate further harms.
Note that we are deliberately conservative when censoring nudity in this paper. For instance, we censor all bare chests, though is socially acceptable to depict topless males in many cultures.
10

---

Acknowledgements
We would like to thank the anonymous reviewers for their comments and suggestions that have helped improve the paper. This research/project is supported by the National Research Foundation Singapore and DSO National Laboratories under the Al Singapore Programme (AISG Award No: AISG2-RP-2020-0177)
References
[1] Patrick Schramowski, Manuel Brack, Bjöm Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappropriate degeneration diffusion models. arXiv preprint arXiv:2211.51105, 2022.
[2] Yisroel Mirsky and Wenke Lec. The creation and detection of deepfakes: survey, ACM Computing Surveys (CSUR), 14(1):1-41, 2021.
[3] Luisa Verdoliva. Media forensics and deepfakes: an overview. IEEE Journal of Selected Topics in Signal Processing, 14(5))910-993) 2020.
[4] Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlcaming. 2015 IEEE symposium on security and privacy, pages 463-480. IEEE, 2015.
[5] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang. David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP). pages 141-159. IEEE, 2021.
[6] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Seleetive forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9304-9312, 2020.
[7] Quoc Phong Nguyen. Bryan Kian Hsiang Low, and Patrick Jaillet. Variational bayesian unlearning. Advances in Neural Information Processing Systems, 33:16025-16036 2020.
[8] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufina Kaufman, and David Bau. Erasing concepts from diffusion models. arXiv preprint vr.iiv2303.073:4 2023.
[9] Eric Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. arXiv preprint arXiv:2303. 17591, 2023.
[10] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
[11] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[12] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. Proceedings of the IEFF/VV Conference Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[13] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12590 2022.
[14] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Barnkka-aa,iikka,, et al. Overcoming catastrophic forgetting in neural networks. Proceedings the notional academy of sciences, 114(13):551-3326 2017.
[15] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference machine learning, pages 3987-3993 3995. PMLR, 2017.
[16] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Ko- ray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1606. 04671, 2016.
[17] Timothy Draelos, Nadine E Miner, Christopher C Lamb, Jonathan Cox, Craig Vineyard, Kristofor Carlson, William M Severa, Conrad D James, and James B Aimone. Neurogenesis deep learning: Extending deep networks to accommodate new classes. In 2017 international joint conference on neural networks (IJCNN), pages 526 533. IEEE, 2017.

---

[18] Hanul Shin, Jung Kwon Lee. Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
[19] Ari Seff, Alex Beatson, Daniel Suo, and Han Liu. Continual learning in generative adversarial nets. arXiv preprint arXiv:1005.08395, 2017.
[20] Timothée Lesort, Hugo Caselles-Dupré, Michael Garcia-Ortiz, Andrei Stoian, and David Filliat. Generative models from the perspective of continual learning. In 2019 International Joint Conference on Neural Networks (ICCN)) pages 1 8. IEEE, 2019.
[21] Cuong V Nguyen, Yingzhen Li. Thang D Bui, and Richard E Turner. Variational continual learning. arXiv preprint 1710 10628, 2017.
[22] Yinjun Wu, Edgar Dobriban, and Susan Davidson. Deltagrad: Rapid retraining of machine learning models. In International Conference on Machine Learning, pages 10355 10366. PMLR, 2020.
[23] Zhifeng Kong and Kamalika Chaudhuri. Data redaction from pre-trained gans. In 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 638-677. IEBE, 2023.
[24] Shimon Malnick, Shai Avidan, and Ohad Fried. Taming generative model. arXiv preprint
arXiv:2211.16488 2022.
[25] Saemi Moon, Seunghyuk Cho, and Dongwao Kim. Feature unlearning for generative models via implicit feedback. arXiv preprint arXiv:2303.052 0059999,,......... 2023.
[26] David Bau, Steven Liu, Tongzhou Wang, Jun- Yan Zhu, and Antonio Torralba. Rewriting deep generative model. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16, pages 351-369. Springer, 2020.
Jun-Yan
[27] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily I. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorcalistic text-to- image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479- 964944 2022.
[28] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.0 06125, 2022.
[29] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reason- ing with a general conditioning layer. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 32, 2018.
[30] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Cabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 9748-8763. PMLR, 2021.
[31] Mehdi SM Sajaadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018.
[32] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinea, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems, 32, 2019.
[33] Nick Hasty, Ihor Kroosh, Dmitry Voitekh, and Dmytro Korduban, 2019. URL httpt::t/tttttti// com/ Giphy/ hephy//ceeb-ooe celeb-detection.
[34] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot leamers. Advances in neural information processing systems, 33:1877-1900 2020.
[35] Jiaming Song, Chenlin Meng. and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
[36] Bedapudi Praneeth. Nudenet: Neural nets for nudity classification, detection and selective censoring, 2019.
12

---

A Proofs
A.1 Generative Replay Objective
Our Bayesian posterior over the set to remember is given by Eq. 1
log pog = 1ogp(xx)0, Cy) + log lo D.) +.
(5)
Let us introduce an extra likelihood term over D, on both sides as follows
log p()p D,) logp( D. (0) = lolgllxx (0, Cÿ) + log p() D,) + (Dg
(6)
The terms on the left hand side the equation can be simplified using Bayes rule
log (() log D, log (( D.) log p(0) (() log log (D,) log
2 p D,) l())(0) + C
We substitute this new form back to Eq. 6 and simplify to obtain
log (ggp(0)D, log logp(x,10,c/) log Dy) log (D,16) logp(0)) +C
(7)
logp(x,10,com log Dy) log 00, c,) log C
(8)
which gives us Eq. 2
A.2 Proof of Theorem
Before we prove Theorem T. we first prove two related lemmas.
Let us first formalize the original conditional MLE objective as KL divergence minimization:
Lemma 1. Given a labeled dataset p(x, c) and a conditional likelihood model p(x c) pa-
rameterized by 0, the MLE objective arg maxg Ep(x,e) log p(x(0,c) is equivalent minimizing Eptc) [Dxx ())xxx(
Proof.
arg max Ep(x(e)p(e) [logp(x(0,c).
arg max / ((x(c)pp() [log((((0c)))) logp(x/c)) dxde / logr))
arg max
wttpp//ittttttlllllll.ee........ c))cc
p(c)H(((())))dde
arg min Epte Dxg
where in the last line we use the fact that the entropy term is independent of 0.
Lemma D is straightforward generalization of the equivalence of MLE and KL divergence mini- mization the conditional case.
We assume the asymptoptic limit where the model, represented by a neural network with pa- rameters 8*, is sufficiently expressive such that the MLE training the full dataset results in Eytc) [Dx z. (p(x ()]]x(x(0"" c)] = 0: in other words, the model has learnt the underlying data dis- tribution exactly. Under this assumption, it straightforward to show that the model also learns the forgetting data distribution exactly, (p)(e) (Px))(()))]]
=0.
exists such that by Lemma 7 c)] 0. The class distribution is defined as p(c) = @spy(c)+ + &.Pr(C), where Øs.0, 6r > 0 and oy + Ør Then the model parameterized by 8* also exactly reproduces the conditional likelihood of the class to forget:
Lemma 2. Assume that the global optimum Eptc) (Dku(p(xic)lppxx///
Ep,(e) (Dkz(p(x(c) p(x(0", cc]
13

---

Proof.
0 = Epee) (DxI(((x)))](()))]))))))
= /eem///=ooo//)pprddodllllllll c))dc
o1 / |
or / p.lo)Daa c))dc
= pp(e) [Dxz(p(x c)) (p (Dxx,(/x(eelll(xxx(0..)
Since Ør > 0 and Dxr(-1)) 2 0 by definition, then for the sum of two KI. di- vergence terms equal 0, must mean that each individual KL divergence is 0, i.e., (p)(s) [DK((p(xee |p(x|0",c) cc] 0.
Finally, we are now able to prove Theorem We restate the theorem and then provide its proof.
Theorem 1. Consider a surrogate distribution q(x(c) such that g(x(cy) + p(((cç)) Assume have access to the MLE optimum for the full datasel 0* = arg maxg Ep(x.e) [logp(x c)] such that Ep(c) [D<Lpx((cc]llx((0 ,c)]] = 0. Define the MLE optimum over the surrogate dataset as 09 arg max, ((e)))))pp()) [log p(0(0 c)}. Then the following inequality involving the expectations of the optimal models over the data to forget holds:
E Ep(x c)py(c) [log p(000, c)] < Ep(xlc)py(e) p(x .c]]
Proof.
Exxee-p(xlepprre) [og [aa_]]]xxeexeeeeeeee exxex((x(e)ps(e) [ogr(x(00
= p(xxc)pr(c)logp((ppp) c)dxde. (c) c)dxde
/ p(x(c) log p(x099.)) c) p(x c) dx xx* dx
Epy(c)
/ p(xlc) log p(xlc)p(x(0'
(ey((e)
/ ((x))) log (x(c) dx
- -Eps(o) If p(xxc)) log p(x(c) dx p(x(0", c)
Epy(c)
p(x(0'
(pr(e) [Dx<(p((ce]pp((00 c))]
- -Epyle) (DxL(x())c)ll(xx ())
-Epyle) [
(apply Lemma 2) (non-negativity of KI.)
< 5
A.3 Proof of Corollary
Corollary 1. Assume that the MLE optimum over the surrogate, 09
arg max, g (g(xle)py(e)) [log p(x(0, c)] is such that Epy(e) [Dxu(q()cc)ll(x(000cc c)] 0. Then the gap presented in Theorem!]
Ep(xlcipy(e) [logr(x(09,c) logp(x(0", c)] EEpy(o) (Dxa.lxxleella(xxcc))
The proof follows straightforwarddy from Theorem
Proof.
Exconpixle)nn(()) [logp(x(0f. c) - log p00 c)] = -Epy(e) [Drr [
EEp [Dx[(xx|cc]l((<]c)]]
where the first line is directly taken from the proof of Theorem ( line makes use of the fact that the model p(x(0", c) g(x assumption.
(second last line), while the second
14

---

B Experimental Details
B.1 VAE and DDPM
MNIST VAE The VAE encoder and decoder are simple MLPs, both with two hidden layers of dimensions 256 and 512. The latent space z has dimensions We choose a Bernoulli distribution over the pixels as the decoder output distribution, and standard Gaussian as the prior. Class conditioning is performed by appending a one-hor hot encoding vector to the encoder and decoder inputs. The original VAE is trained for 100K steps, and the forgetting training is trained for 10K steps. We use learning rate of 10-4 and batch size of 256. As sampling with VAEs is cheap, we use 50K samples to calculate the FIM, and sample the replay data from frozen copy of the original VAE during forgetting training.
CIFARIO DDPM We adopt the same U-Net architecture unconditional DDPM in M. with four feature map resolutions (32 x 32 to 4 x 4) and self-attention blocks at the 16 x 16 resolution. We use the linear 3 schedule with 1000 timesteps, and train for 800K steps with a learning rate of 10-4 and batch size of 128. For classifier free guidance, we use the FiLM transformation at every residual block and drop the class embeddings 10% of the time. For sampling, we use 1000 timesteps of the standard DDPM sampler with a guidance scale of 2.0. As sampling with diffusion models is significantly more expensive than VAËs, we generate and store set of 5000 images, and subsequently use it both for calculating the FIM and as the replay dataset. For forgetting, we train the model with 20K training steps. We use a learning rate of 10 and batch size of 128. As the CIFARIO training set has 5000 images per class, when evaluating the image quality of the remaining classes, we generate 5000 images of each class for total of 45000 images, and compare them against the corresponding 45000 images in the training set. Experiments are run on 2 RTX A5000s.
STL10 DDPM We conduct our STL10 experiments by resizing the dataset to the 64 x 64 resolution. The experiments follow closely from our CIFARIO experiments, where we have five feature map resolutions (64 64 to 4) instead while keeping attention blocks at the 16 16 resolution. Due to the smaller size of the dataset, we combine the train and test sets to form a larger training set, resulting in 1300 images per class. We train for a total of 250K steps with a learning rate of 2 x 10-4 and batch size of 64. All other hyperparameters are kept identical to the CIFARIO experiments. For forgetting training, we train similarly for 20K steps with learning rate of 10 4 and batch size of 64. To evaluate the image quality of the remaining classes, we generate 1300 images of each class, for total of 1170 images, and compare them against the corresponding 11700 images in the training set. Experiments run on 2 RTX A5000s.
Classifier Evaluation In terms of classifier architectures and training, for MNIST, we train a simple two-layer CNN the original MNIST dataset for 20 epochs. As for both CIFAR10 and STL10, we finetune a ResNet34 classifier pretrained on ImageNet that was obtained from the torchvision library. All layers of the ResNet34 classifier are finetuned on the target dataset for 20 epochs. We calculate [xxx((,ey) (y cylx) and (PP (y (ylx))) by averaging over 500 generated images of the forgotten class from the respective models.
B.2 Stable Diffusion
Forget Celebrities We use the open-source SD v1.4 checkpoint as the pretrained model for all Stable Diffusion experiments with Selective Amnesia. We choose v1.4 opposed newer versions for fair evaluations as competing baselines, SL.D and ESD, are based on v1.4. Similar to the CIFARIO and STL10 experiments, we generate 5000 images from SD v1.4 and use it for both FIM calculation and GR. These images are conditioned on 5000 random prompts that were generated with GPT3.5 [34]. We use 50 steps of the DDIM [35] sampler with a guidance scale of 7.5 for all image generation with SD. For forgetting training, we set the prompt to forget as cy = (Brad Pitt") or ("Angelina Jolie") and train the model using the (x((c)) represented by 1000 images generated with prompts as specified in the main text. We train for a total of 200 epochs of the surrogate dataset with À = 50 and base learning rate of 10-5 (scaled by number of GPUs). We similarly generate the 50 test prompts using GPT3.5, and generate 20 images per prompt. Experiments are run on 4 RTX A6000s and training takes approximately hours with peak memory usage of around 40GB per GPU. Note that we did not optimize for computational efficiency in our reported experiments;
15

---

by tuning hyperparameters in preliminary experiments, we could achieve similar performance with 2 A6000s and hours of training. We believe additional performance gains can be achieved with further tuning and leave that for future work.
In terms of evaluation, Detector (33), which is trained to detect 2306 different celebrities. The classifier is composed of two stages: the first stage is face detector while the second stage is celebrity face classifier. a given image is found to have multiple faces, we only consider the face with the highest probability of the target celebrity. This to account for cases where multiple persons are in an image, but typically only one of them will be of the celebrity of interest. As for the baselines, for SLD Medium, we set the safety concept to either "Brad pitt" or "Angelina Jolie" during inference, while for ESD-x, we train the model to forget the prompts "Brad Pitt" or "Angelina Jolie".
evaluate the 1000 generated images with the open-source GIPHY Celebrity
Forget Nudity For forgetting nudity, we tume only the unconditional (non-cross-attentie layers of the latent diffusion model as proposed in []. We use the same set of samples for calculating the FIM and for GR. The prompt to forget is set as cy =("nudity". "naked", "erotic", "sexual"). We set x = 50 and train for 500 epochs. Experiments are run on the same resources as the celebrities experiments.
We evaluate on the I2P dataset by generating one image per prompt with the provided random seeds. The 4703 images are evaluated using the open-source NudeNet classifier [36], with the default probability threshold of 0.6 to count as a positive detection of a nudity instance. As NudeNet considers exposed and covered content separately, we only consider nudity content that are classified as exposed. Manual inspection showed the classifier to give false positives; for example, 10 of the 16 images generated by SA classified as showing Female Genitalia actually have this attribute. Likewise, some images classified as showing Female Breasts actually showed Male Breasts.
In terms of baselines, for SLD Medium set the safety concept to "nudity, sexual, naked, erotic". For ESD-u, we use the publicly available checkpoint from the official GitHub repository that was trained to forget nudity.
16

---

C More Results
C.1 Forget Nudity
Table 3: Quantity of nudity content detected using the NudeNet classifier on the I2P benchmark dataset (4703 images). As NudeNet classifies covered and exposed content separately, all nudity content considered here are classified as exposed.
Female
Female Genitalia
Male
Male
Armpits
Belly
Buttocks
Feet
Anus
Breasts
Breasts
Genitalia
SD vl.4
214
171
40
295
23
21
24
154
14
SL.D Medium
58
60
7
15
42
1
21
ESD-u
60
17
13
26
1
4
SA (Ours)
72
77
19
25
83
16
0
Upon manual inspection of I2P samples from SA, we observed that the NudeNet classifier has tendency to classify gender incorrectly. For instance, male breasts are often classified as female breasts. Hence, the number of female breasts presented for our method is an overestimation of the true number. There also a relatively high false positive rate for exposed female genitalia, as 6 of the flagged images for SA do not depict any exposed female genitalia.
Table Quantity of nudity content detected using the NudeNet classifier from 1000 sampled images with the prompt "a photo of naked person". Similar to I2P results, we only consider exposed content. Note that there a larger number of Armpits and Female Breasts for SD v1.4 than there images because NudeNet classifies multiple instances of each content per image separately.
Female
Female
Male
Male
Armpits
Belly
Buttocks
Feet
Anus
Breasts
Genitalia
Breasts
Genitalia
SD vl.4
1013
753
110
116
1389
453
8
858
149
685
201
SLD Medium
360
369
38
56
351
115
73
ESD-u
86
56
7
35
55
5
10
SA (Ours)
204
172
15
105
245
27
0
We conduct additional quantitative study on nudity by evaluating 1000 images sampled using the prompt "a photo of naked person" using the NudeNet classifier. We use the same setup as the I2P experiments for all models. The results shown in Table 4 Similar the I2P experiments, our model drastically reduces the amount nudity content compared to SD vl.4 and v2.1. ESD-u achieves the best scores overall. Our model outperforms SLD Medium, particularly sensitive content like Female Breasts and Female Genitalia.
17

---

D
Additional Samples
D.1 MNIST, CIEARIO, STL10
Original
= 100
0
0
3
0
0
0
/
1
/
1
}
1
1
1
1
2
2
2
2
2
2
2
2
3
2
2
2
2
2
2
2
2
2
3
2
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
4
1
4
4
4
4
4
9
4 4
4
4
4
4
4
4
4 5 6
5
5
5
5
5
5
5
5 5 6
5 5
5
5
5
5
5
5 666
6
6
6
6
6
6666
666
6
6 6
C
6
6
6
6
7
7
7
7
7
7
7
7
7
9
7 7
7
7
7
7
7
7
7
9 8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
8
9
9
9
9
9
9
9
8
8
9
9
9
9
9
9
9
9
9
9
9
Figure 7: Additional sample images comparing the original MNIST VAE versus ours with the digit "0" forgotten with A = 100 (with GR), which corresponds to the hyperparameters shown in Table
Original
À 10
Figure 8: Additional sample images comparing the original STL10 DDPM versus with the "airplane" class forgotten with = 10 (with GR), which corresponds to the hyperparameters shown in Table
18

---

Original
=
10 with GR
= 10 no GR
== with GR
Figure 9: Additional sample images comparing the original CIFARIO DDPM versus ours with the "airplane' class forgotten. We show three variants of SA, corresponding to the ablations shown in Table It is clear from inspection that the image quality of the classes remember is significantly impacted without the GR term (A = 10 no GR). When visually comparing À = and À = 10 (both immediately obvious to the naked eye, although the quantitative
with GR), the differences metrics show that À = 10 produces better results.
19

---

D.2
Stable Diffusion
SD v1.4
ESD
SLD
SA (Ours)
Figure 10: Sample images with prompt "Brad Pitt in a tuxedo". These are an extension of Fig. to provide the reader with more context as the qualitative differences between the various approaches. The bottom two rows are our method, where we set ((x(cy) to "middle aged man" and "male clown" for cy = ("Brad P.tt").
v1.4
IONL
ESD
7618
SLD
SA (Ours)
Figure 11: Sample images with prompt "a close up of Brad Pitt's face". These are an extension of
Fig. 4 The bottom two rows method, where we set q(x cf) to "middle aged man" and "male clown" for c = ("Brad P.tt").
20

---

SD,1
ESD
SLD
SA (Ours)
Figure 12: Sample images with prompt "Brad Pitt laughing in a park". These are an extension of Fig. The bottom rows our method, where we set q(x cy) to "middle aged man" and "male clown" for Cy ("Brad Pitt"). ).
v1.44
ESD
SLD
SA (Ours)
Figure 13: Sample images with prompt "Angelina Jolie in black dress". These are an extension of
Fig. 5, The bottom rows are method, where q(x (yy) to "middle aged woman" and
"female clown" for c ("Angelina Jolie"].
21

---

SD v1.4
ESD
SLD
SA (Ours)
Figure 14: Sample images with prompt "Angelina Jolie holding bouquet of flowers". These are an extension of Fig. 5 The bottom two rows are our method, where we (x((cy) "middle aged woman" and "female clown" for cy f*Angelina Jolie"}.
SD v1.4
ESD
SLD
SA (Ours)
Figure 15: Sample images with prompt AAngelina Jolie wearing a hat". These are an extension of
Fig.5. 5. The bottom rows are our method, where q(xccy) to "middle aged woman" and "female clown" for c ("Angelina Jolie"].
22

---

SD
SA (Ours)
Figure 16: Sample images with prompt "realistic portrayal Maleficient movie" from SD v1.4 and our method where we set q(x(ey) to "middle aged woman" and "female clown" for e, ("Angelina Jolie"). Even though the prompt does not explicitly mention Angelina Jolie, we observe that our method generalizes to the portrayal of the character Maleficient.
vD
SD v2.1
ESD
SLD
SA (Ours)
Figure 17: Sample images with prompt "a photo of naked person". These are an extension of Fig.
Note again that being conservative in censoring the image here.
SD
SA (Ours)
Figure 18: We also experiment with forgetting of art styles. Here we attempt to forget the artistic style of Vincent van Gogh by setting q(x(cy) to images of "pop art style" generated by SD v1.4 for Cj = l ('van gogh style"), J. The images shown are samples from the prompt "a painting of a lion in gogh style". SA successfully generates images that lacks the distinct gogh style (and instead contains elements pop art style).
23

---

E
Effects on Other Celebrities
Jennifer Anniston on stage
v1.4
SA-Xattn
SA-Full
Oprah Winfrey on stage
SD
SA-Full SA-Xattn
Jackie Chan stage
SD v1.4
SA-Full SA-Xattn
Figure 19: Sample images investigating the effects on celebrities other than the one being forgotten when using method. SA-Full indicates training of all layers and SA-Xattn indicates training of only the cross-attention layers. Both models are trained to forget y = ("Angelina Jolie") by setting to q(x|cy) to "middle aged woman". We use the prompt "[...] stage", where [...] is substituted with Jennifer Aniston, Oprah Winfrey or Jackie Chan.
In this section conduct qualitative study the effects on celebrities other than the one being forgotten. Ideally, the changes to other celebrities should be minimal. We revisit the case of forgetting Angelina Jolie by setting q(x(c,) )) to "middle aged woman". We train variants, training all layers (like in Sec. 4.2 of the main text on forgetting famous persons) and training only the cross-attention layers. We abbreviate them SA-Full and SA-Xattn respectively.
From Fig. 19 we see that SA-Full leads slight changes the depiction of Jennifer Aniston (compared to how she looks in person, or to SD vl.4), but minimal changes to Oprah Winfrey and

---

Jackie Chan. We hypothesize that this is due to Jennifer Aniston sharing greater similarities to Angelina Jolie than the latter two, they are both female and of similar ethnicity. leading the model to more strongly associate the two together. This is not an inherent limitation of SA; the effects can be minimized by tuning only the cross-attention layers, as seen SA-Xattn rendering Jennifer Aniston (and the other celebrities) which recommends tuning the cross-attention layers only if one wishes to forget concepts that are specified explicitly (e.g., celebrity names which are mentioned in the prompt).
accurately SD v1.4. This corroborates the findings in [8],
However, there are cases where celebrities can be rendered even without explicit mention of their names, for example in Fig. 16 In such cases, we observe anecdotally that tuning only the cross- attention layers limits the model's ability to generalize to such prompts. Recall that the unconditional (non-cross-atee ) layers are responsible for generalization to prompts without explicit mention of the concept to forget (ef. the nudity experiments Sec. [.2 of the main text), hence tuning only the cross-attention layers unsurprisingly limits generalization performance. As such, there is trade-off between generalization and interference of other concepts. We recommend tuning all layers the user wants good balance of generalization but with potentially slight interference of closely related concepts, and only the cross-attention layers if minimal interference other concepts is required, at the expense of generalization. We leave a more precise study of this trade-off to future work.
25


Reviews:
[{"review_id": "kLzXlVj3vo", "reviewer": "Reviewer_bzv7", "summary": "This paper presents a technique called Selective Amnesia, which enables controllable forgetting of concepts in pretrained deep generative models. The authors show that this technique can be applied to a variety of models, including text-to-image and VAEs, and can be framed from the perspective of continual learning. The paper discusses two popular approaches used in their work: Elastic Weight Consolidation and Generative Replay. The authors argue that this technique can be used to prevent the generation of harmful, misleading, and inappropriate content by deep generative models.", "questions": "Please refer to the above sections for detailed discussion.", "limitations": "Yes, there is a section at the end of the paper discussing limitations and future research opportunities.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The manuscript ventures to address a vital and emergent challenge — ensuring content safety within the ambit of generative artificial intelligence. This issue is of critical importance, given the increasing prevalence and capabilities of generative models, coupled with the potential risks and implications associated with their misuse. The authors' focus on such a topical concern is commendable and lends significant relevance and timeliness to their work.\n\n2. The authors introduce a model inspired by continual learning — a concept that, while established in the broader field of machine learning, presents a novel approach in the specific context of generative model safety. This inventive application of continual learning principles adds a fresh dimension to the discourse on generative model safety, thereby enhancing the manuscript's contribution to this field of study.\n\n3. The selection of datasets for the empirical evaluation is impressively diverse. The authors have conducted experiments on a broad spectrum of datasets, ranging from simpler, well-established ones such as MNIST to more complex, real-life image datasets. This range serves to validate the robustness and versatility of their model across different levels of complexity and in varied contexts, adding significant credibility to their findings. Furthermore, this choice reflects a meticulous approach to experimental design, allowing the model's performance to be tested and evaluated under a wide array of conditions.\n\n4. The quality of exposition is commendable. The paper presents a clear narrative, beginning with the intuitive premise, followed by a comprehensive description of the model, including a highly useful algorithmic representation. The logical sequencing of these sections facilitates reader comprehension. The authors have succeeded in presenting a complex topic in an accessible and cogent manner.", "weaknesses": "The manuscript is of high quality and is presented competently, with no major flaws readily apparent. However, my primary concern pertains predominantly to the scope and rigor of the quantitative evaluation.\n\n1. The current quantitative evaluation is exclusively conducted on relatively simple datasets. While this serves as a valid starting point, extending the quantitative evaluation to more challenging datasets would provide a more comprehensive perspective on the model's performance. Although this might present additional difficulties, I would encourage the authors to devise some form of quantitative measure, even if it requires human assessment for accuracy. Such a quantitative comparison would help to mitigate potential selection bias and lend greater credence to the results.\n\n2. A more thorough discussion on the chosen baseline models and the current state-of-the-art models would be greatly beneficial. By elucidating how these models relate to the proposed model, and where they diverge, the authors could provide a clearer context for their work. Such discourse would enhance the readers' understanding of the field's landscape and allow them to appreciate the distinctiveness and the value of the authors' contribution more deeply.\n\n3. The comparative analysis currently conducted is primarily qualitative in nature. However, this leaves a gap in the evaluation, as quantitative comparisons can provide unique insights that are not always captured by qualitative analysis. Quantitative comparisons can offer more objective, replicable, and measurable findings, which are highly valuable in establishing the proposed model's superiority over existing models.\n\nAddressing these concerns would add depth to the study, providing a more rigorous and holistic evaluation of the proposed model. Such improvements would be of significant value, helping to fortify the authors' claims and making a more compelling case for the paper's contributions to the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for acknowledging the strong motivations, novelty and quality of our work. Below we provide our responses to the concerns raised by the reviewer.\n\n> ...extending the quantitative evaluation to more challenging datasets would provide a more comprehensive perspective on the model's performance *and* ..even if it requires human assessment for accuracy\n\nQuantitative results for the Stable Diffusion experiments are shown in Tables 2, 3 and 4 in Appendix C (due to space constraints). We will make this clearer in the main text. \n\nIn brief, we evaluated these experiments with well-established pretrained classifiers: the GIPHY Celebrity Detector (GCD) and NudeNet nudity classifier; the latter was employed in prior works on SLD and ESD. We agree with the reviewer that additional quantitative measures, such as human evaluations, would provide a more holistic evaluation due to the many factors one can consider when forgetting concepts in Stable Diffusion. For instance, we have noted in Sec 4.2 of the paper that our model produces more semantically-relevant images (far fewer faceless images), and the faces that our model generates are of higher quality (particularly compared to SLD), despite having a higher GCD score (where lower is better) than ESD and SLD.  We will add to Sec. 5 that conducting human evaluations is part of our future work.\n\n> A more thorough discussion on the chosen baseline models and the current state-of-the-art models would be greatly beneficial.\n\nWe included discussion and comparisons with works from the machine unlearning literature as well as baselines in Stable Diffusion (ESD and SLD) in Sec. 2.3 and 2.4, respectively. Discussions on qualitative and quantitative results from experiments between our method and ESD and SLD are also included in Sec 4.2. We will include additional discussions in the related works section on feature unlearning, data redaction and model rewriting in deep generative models. We believe this expanded discussion will be reasonably comprehensive given space constraints. If the reviewer has specific suggestions for relevant works or comparisons, please let us know. \n\nWe hope that we have addressed your concerns. If so, we kindly request to consider raising your score. "}, {"review_id": "YKQQXvIVjK", "reviewer": "Reviewer_n35v", "summary": "In this work, authors introduce Selective Amnesia - a new method for selective forgetting of particular concepts in generative modeling without access to the training data. Authors propose to use Continual Learning methods - namely, Elastic Weights Consolidation and Generative Replay to retrain the base model with carefully selected additional data that promote forgetting of examples conditioned on particular conditional values without affecting the other ones. In particular, to enforce forgetting, authors introduce a surrogate objective that, instead of just minimizing the probability of generating examples from a distribution that needs to be forgotten, substitutes it with a different one e.g., random noise.\n", "questions": "- How precise is the forgetting presented in this method? What happens if the concept we want to forget is highly correlated with another one? E.g. if we take CelebA dataset, I think it might be very hard to forget the “wearing lipstick” class without forgetting “heavy makeup” one. From a more precise point of view, I believe this question relates to the problem of how adequate is the approximation of p(x|c)p_f(c) with generations with concepts that we want to forgot/remember?\n\n- What would happen if we simply skip the EWC regularisation term and continue model retraining with only two ELBOs with generative replay?", "limitations": "Authors address the main limitations of the work except from the one described in weaknesses/questions", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- This submission tackles an important problem crucial for the reliable use of recent ML applications.\n- The main contribution is clear, seems to be well thought and is well presented both in terms of intuition and details.\n- The proposed evaluation is extensive, convincing, and well-structured. The experiment results conducted on several datasets with multiple methods are impressive. Simple experiments were implemented with Variational Autoencoders, while the more extensive ones with high-quality datasets employed diffusion models.\n- The work is well-written and easy to follow.\n- Additionally to the main contribution, this work introduces an interesting reformulation of the EWC method.", "weaknesses": "The proposed method does not really enforce forgetting the particular concept but rather promotes replacing it with a different one. This might lead to fake associations between conditional values and generated outputs. For example, as presented in this work, forgetting “Brad Pitt” generations by learning to generate a “male clown” instead might also affect other generations with prompts similar to “Brad Pitt” - e.g. other actors. This problem should be more evident when trying to forget more general concepts by replacing them with different ones. To make the method more general, it would be interesting to find a technique that automatically finds the suitable surrogate dataset.\n\nSmall:\n- Small detail: \"most prior work on continual learning for generative models are applied to GANs [19, 20], while our work is primarily concerned with variational models.\" - This is not true, see, for example (1,2,3,4,5). Those methods tackle the problem of continual learning of variational models, so the fact that they already exist does not affect the novelty of this work. However, I think it would be beneficial to mention at least [1], that first introduced EWC for Variational Autoencoders. I don’t believe the rest should be cited, but I just wanted to point those to authors as potentially interesting for any future works.\n\n\n\n1. Nguyen, Cuong V., et al. \"Variational Continual Learning.\" International Conference on Learning Representations.\n2. Egorov, Evgenii, Anna Kuzina, and Evgeny Burnaev. \"BooVAE: Boosting approach for continual learning of VAE.\" Advances in Neural Information Processing Systems 34 (2021): 17889-17901.\n3. Achille, Alessandro, et al. \"Life-long disentangled representation learning with cross-domain latent homologies.\" Advances in Neural Information Processing Systems 31 (2018).\n4. Mundt, M., Pliushch, I., Majumder, S., Hong, Y., & Ramesh, V. (2022). Unified probabilistic deep continual learning through generative replay and open set recognition. Journal of Imaging, 8(4), 93.\n5. Deja, Kamil, et al. \"Multiband VAE: Latent Space Alignment for Knowledge Consolidation in Continual Learning.\" IJCAI 2022Limitation - how precise is the forgetting presented in this method? What happens if the concept we want to forget is highly correlated with another one? E.g. if we take CelebA dataset, I think it might be very hard to forget the “wearing lipstick” class without forgetting “heavy makeup” one. From a more precise point of view, I believe this question relates to the problem of how adequate is the approximation of p(x|c)p_f(c) with generations with concepts that we want to forgot/remember?\nWhat would happen if we simply skip the EWC regularisation term and continue model retraining with only two ELBOs with generative replay?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for acknowledging the strong motivation and contributions of our work, as well as our extensive experimental results. Below we provide our responses to the concerns raised by the reviewer.\n\n> What happens if the concept we want to forget is highly correlated with another one?\n\nWe thank the reviewer for raising this issue of “leakage” to correlated concepts. We have documented this effect in appendix E, where we show that forgetting “Angelina Jolie” leads to slight changes when generating “Jennifer Anniston”. We discussed a way to mitigate this issue, which is to tune the cross-attention layers only in Stable Diffusion. We view this “leakage” issue as a potential double-edged sword that could be beneficial in situations where we would like to forget highly correlated concepts. For instance, in the nudity experiments in Sec 4.2, one would prefer the remapping of clothed persons to generalize to related prompts, such as when the prompt includes artist styles that typically contain nudity. \n\n> ...it would be interesting to find a technique that automatically finds the suitable surrogate dataset\n\nThis is an interesting idea. In this work, our approach was to enable the user to pick $q$ as the surrogate distribution as its choice may be context-dependent (e.g., due to cultural differences). That said, the system can suggest potential $q$'s that are commonly accepted to be benign yet retain the generation capabilities of the model. We will add this discussion to the revised manuscript as future work. \n\n> ...see, for example (1,2,3,4,5)... I think it would be beneficial to mention at least [1], that first introduced EWC for Variational Autoencoders\n\nThank you for pointing out these related works and we will revise our discussion in Sec 2.2. As the reviewer notes, [1] addresses the continual learning problem and was applied to a VAE to sequentially learn MNIST classes.  The method uses variational approximation to calculate the posterior over the parameters of a model for the T-th learning task, given parameters for the T-1th learning task.  Our setting here is different in that we work on forgetting specific concepts rather than continual learning; we will amend our related works section to discuss the relationship. \n \n> What would happen if we simply skip the EWC regularisation term and continue model retraining with only two ELBOs with generative replay?\n\nAs suggested by the reviewer, we ran our method on DDPM to forget the airplanes class in CIFAR10 by turning off EWC completely. This is achieved by setting $\\lambda=0$. We kept all other hyperparameters identical to experiments in Table 1. After training, we evaluated the image quality on the remaining 9 classes. We obtained an FID of 45.7, a precision of 0.078 and recall of 0.803, which suggests that image fidelity significantly deteriorated without the EWC term. This result is overall in line with the ablations in Table 1, which show that image fidelity decreases as we decrease the strength of the EWC term.\n\n[1] Nguyen, Cuong V., et al. \"Variational continual learning.\" arXiv preprint arXiv:1710.10628 (2017).\n\nThank you for your positive review and we hope that we have addressed your remaining concerns. If there are any further issues, please let us know. \n"}, {"review_id": "LgADe04L6B", "reviewer": "Reviewer_VhMr", "summary": "The paper aims to forget selected concepts from deep generative models trained with variational inference with a focus on the trustworthiness such as malicious prompts. The paper introduces a continue learning approach that combines EWC and GR to maximize the likelihood (or ELBO) on the remembered set. Experimentally, the paper evaluates forgetting qualities for simple models on standard datasets and for large scale stable diffusion on real world datasets.", "questions": "What is the criteria of selecting $q$? Why does the selection of $q$ leads to mapping of concepts (line 262)? \n\nHow much computational resource (memory and time) do you need to continual-learn stable diffusion? Do you fine-tune a subset of layers or train all layers? \n\nWhat is the advantage of the proposed method compared to baseline methods for stable diffusion?", "limitations": "The paper addressed limitations in the last section.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The forgetting task is very meaningful and important as we are now facing such trustworthy challenges in the large model era.\n\nThe continual learning approach that combines EWC and GR is a good idea and solves several practical difficulties as discussed in the methodology section.\n\nThere is theory that could guarantee improved likelihood after using the surrogate loss.\n\nThe framework applies to several types of models trained with likelihood or ELBO.\n\nThe results on forgetting identity and malicious contents from stable diffusion are interesting and potentially useful.", "weaknesses": "The selection of $q$ is not well studied in experiments and seems arbitrary. There should be more detailed explanation on the selection of $q$.\n\nThe paper lacks analysis on the computational resources needed for the continual learning approach. \n\nThe experimental results for simple models on standard datasets are not compared to baseline methods. There are a few methods for similar goals in the wild including data redaction, feature unlearning, model rewriting, model taming, etc. \n\nThe experimental results for stable diffusion do not seem to be better than baseline methods, especially the \"erasing concepts from diffusion models\".\n\nSome minor points:\n\nI recommend the authors to discuss more related research (mentioned above) in the related work section so that readers can have a better sense of the literature. \n\nPlease check typos and unbalanced parentheses in the math equations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the motivations and usefulness of our work. Below we provide our responses to the reviewer's concerns.\n\n> ...detailed explanation on the selection of q\n\nThe choice of $q$ can be understood theoretically from Corollary 1-- a greater difference between $q$ and the distribution over $D_f$ means a lower likelihood over $D_f$. In general, we want $q$ to be far from $D_f$. For instance, in the experiments of Table 1, $q$ is chosen to be uniform noise as it is intuitively far from the distribution of natural images. However, we stress that Corollary 1 serves as a heuristic, and users are free to choose $q$ that is most relevant for their use case. In our SD experiments, we attempt to forget certain celebrities and a semantically-relevant choice is to have the model generate unrecognizable persons. \n\n> Why does the selection of q leads to mapping of concepts?\n\nSA trains the model to forget $D_f$ by instead generating concepts from $q$ when conditioned on $c_f$. For instance, in the experiments in Sec 4.2, as we chose $q$ to represent images of middle-aged men, the model generates unrecognizable men when prompted for Brad Pitt.\n\n> Do you fine-tune a subset of layers or train all layers?\n\nOur method does not make assumptions about network architecture, thus we tune all layers except in the nudity experiments, where we tune only unconditional layers of SD. One could also tune only cross-attention layers to minimize interference with other concepts (explored in appendix E).\n\n> ...analysis on the computational resources needed\n\nAppendix B mentions computational resources which we will expand to include more details. In brief, we used 2 RTX A5000s for DDPM experiments and 4 A6000s for SD experiments. Training with SA takes 4-5 hours on DDPM and 20 hours on SD. Memory usage in SD is approximately 40GB. Note that we did not optimize for computational efficiency in our reported experiments; by tuning hyperparameters in preliminary experiments, we could achieve similar performance with 2 GPUs and 6 hours of training. For comparison, the ESD baseline also uses 2 GPUs and around 2 hours of training. We believe additional performance gains can be achieved with further tuning. \n\n> ...results for simple models on standard datasets are not compared to baseline methods… including data redaction, feature unlearning, model rewriting, model taming, etc *and* ...discuss more related research (mentioned above) in the related work section\n\nThank you for the suggested keywords. We surveyed representative works but the methods are not applicable to forgetting concepts in conditional variational generative models, or require major extensions. We elaborate below, and will include these discussions in the revised manuscript:\n\n[1] and [2] pertain to GANs and Normalizing Flows, respectively. The former requires the discriminator as feedback for the generator for data redaction. [4] employs exact likelihood computation of NFs to reduce the likelihood over $D_f$. Variational models lack access to exact likelihoods, and Sec 3.2 shows that minimizing the ELBO over $D_f$ leads to poor results.\n\n[3] implicitly assumes that the generator’s latent space has disentangled features, which does not apply to our conditional models. For instance, a given latent $z$ can generate all ten digits of MNIST, $x_i=G_\\theta(z,c_i), i=0, …, 9$, by changing the conditioning signal. Hence, it is unclear how one would apply [3] to conditional models. \n\n[4] is work on image editing, such as removing watermarks from images, by directly modifying a single layer’s weights in a generator. The paper's experiments focused only on GANs and required finding the best layer to tune for specific applications. This method was not designed for forgetting concepts in conditional variational models and we are unsure whether it qualifies as a suitable baseline. Preliminary experiments with the provided code (on GANs) on more drastic changes that forgetting necessitates did not yield desired results, e.g, altering entire roofs of churches to domes led to severe visual artifacts.\n\n> The experimental results for stable diffusion do not seem to be better than baseline methods, especially the \"erasing concepts from diffusion models\"\n\nWe do not claim that our method is strictly better than the strong baselines, at least purely in terms of the classifier metrics. We emphasize that the metrics only provide a partial view of the results; the compared methods have qualitatively different behaviors, as discussed in Sec. 4.2 and App. C. ESD (and SLD) steers the model in arbitrary, uncontrollable directions away from the concept, which results in generated images lacking semantic relevance. For instance, ESD frequently produces images without faces or persons, such as houses or cars when prompted for Angelina Jolie (Fig 13 of app.), or mountains and forests in the nudity experiments (Fig 5). This is reflected in the high proportion of generated images without faces (over 30%) in Table 2 of the appendix. By choosing $q$ to be images of unrecognizable persons, SA produces fewer images without faces (about 5%). Compared to SLD, SLD tends to produce distorted faces with visual artifacts. We believe these aspects should be considered when comparing methods. \n\n> Please check typos and unbalanced parentheses in the math equations.\n\nWe thank the reviewer for pointing out the typos. We have corrected the unbalanced parentheses in Thm 1 and Corollary 1. \n\n[1] Kong, Z., and K. Chaudhuri. \"Data redaction from pre-trained gans.\" \n\n[2] Malnick, S., Shai A., and Ohad F.. \"Taming a Generative Model.\"\n\n[3] Moon, S., S. Cho, and D. Kim. \"Feature unlearning for generative models via implicit feedback.\"\n\n[4]  Bau, D., et al. \"Rewriting a deep generative model.\"\n\nWe hope that we have addressed your concerns. If so, we kindly request to consider updating your review and score."}, {"review_id": "lRCNIpCSfW", "reviewer": "Reviewer_ehLH", "summary": "This paper draws inspirations from continual learning and proposes a Selective Amnesia method to selectively forget concepts in pre-trained generative models. The proposed method is implemented in a widely-used framework of Bayesian continual learning, together with generative replay to optimize the objective. Experiments on several datasets demonstrate the effectiveness of the proposed method.", "questions": "My major concerns include the comparison of AFEC, more advanced baselines and computational cost of generative replay.  Please refer to the weakness.", "limitations": "The authors have discussed the limitations and potential negative impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-organized and easy to follow.\n2. The reverse application of the continual learning process to forget certain concepts is an interesting idea. \n3. The experimental results, especially for the case study on Stable Diffusion, seem to be remarkable. This may contribute to realistic applications in erasing harmful concepts and protecting privacy.", "weaknesses": "1. The idea of selective or graceful forgetting has been explored in continual learning. In particular, AFEC [1] also considers a similar Bayesian-based framework and controls a forgetting rate to incorporates a non-informative prior $p(\\theta)$. Interestingly, I find Eq.(2) also derive the prior $p(\\theta)$. Therefore, a comparison of the proposed Selective Amnesia and AFEC is necessary. \n2. The experiments only compare Selective Amnesia and Original. Although the results of selective forgetting seem to be remarkable, is it possible to compare with representative baselines of continual learning and/or machine unlearning?\n3. A primary motivation of this work is to avoid the computational overhead of retraining all old data. However, the proposed method relies on strong generative replay, which potentially deviates from the motivation. I encourage the authors to compare the computational overhead of generative replay, and also consider it when comparing with other baselines.\n\n[1] AFEC: Active Forgetting of Negative Transfer in Continual Learning. NeurIPS 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for acknowledging the quality, strong experimental results and interesting approach of our work. Below we provide our responses to the concerns raised by the reviewer.\n\n> Therefore, a comparison of the proposed Selective Amnesia and AFEC is necessary.\n\nThank you for bringing AFEC to our attention. We will include a discussion of AFEC in the revised manuscript. The major difference between AFEC and our work SA is that AFEC tackles the traditional setting of sequential tasks learning. AFEC proposes an adaptive posterior over prior tasks when learning a new task in a traditional continual learning setting, which allows conflicting weights from old tasks to be forgotten if they will negatively affect the new tasks. In our work, we would like complete forgetting of $D_f$. Applied to AFEC, this means setting $\\beta=1$ in Eq 6, i.e., completely ignoring the posterior over old tasks. This reduces to relying solely on catastrophic forgetting of $D_f$, which we tried in early experiments and found did not lead to sufficient forgetting of $D_f$ due to strong conditioning of conditional generative models. Regarding the prior term in Eq 6, $p(\\theta)$ appears from applying Bayes rule over the posterior of the task to remember. This resembles the prior in Eq 2 of our work, which arises from Bayes rule over $D_f$. Similar to us, it seems that the prior term in AFEC is left out of the optimization.\n\n> ...is it possible to compare with representative baselines of continual learning and/or machine unlearning.\n\nWe were unable to find suitable continual learning baselines as continual learning is concerned with *preventing* catastrophic forgetting, while we are interested in promoting forgetting.  As mentioned above, relying solely on catastrophic forgetting of $D_f$ led to suboptimal results. \n\nWe also could not find suitable machine unlearning baselines for our specific setting of forgetting classes in conditional variational generative models. Based on a recent machine unlearning review [1], the methods covered were applied mainly to discriminative models. To our knowledge, there are no machine unlearning works targeted at generative models. Next, we considered several representative methods in machine unlearning and sought to investigate if they could be extended to generative models. \n\nAs discussed in Sec 2.3, several prior works had limiting assumptions that prevented them from being applied to our setting. For instance, [2] requires altering the original training process of the model via dataset sharding. [3] requires a variational posterior over the original model weights, whereas we only have a MLE point estimate. [4] assumes that the weights of the model that has forgotten $D_f$ only differ from the original model by a small Gaussian error, and proposes to directly modify the weights of the network. As the method is demonstrated only on classification, it is unclear if the assumption holds in the generative case. Regardless, we ran preliminary experiments of [4] on the MNIST VAE, with the simplified Fisher forgetting method of Sec 4 of the paper (as SA also utilizes the FIM). We found this to be unstable due to the inversion of the FIM ($F^{-1/4}$ term). \n\nWe also considered [5], which we will include in Sec 2.3 of the revised manuscript. [5] proposes to tune a model to remove the influence of certain datapoints but requires the gradients during the original training process to be cached, which we do not have.\nIn general, it appears that methods in the machine unlearning literature [1] often have limiting assumptions and are more suitable for discriminative settings. In considering other avenues for baselines, we also considered other methods designed for generative models (which were pointed out by reviewer 3). However, we found that these methods will not work in our specific setting of conditional variational models, or were not appropriate for forgetting. Extending these methods to our setting would constitute major separate research. For additional details regarding these methods, please refer to our response to reviewer 3.\n\n> the proposed method relies on strong generative replay, which potentially deviates from the motivation.I encourage the authors to compare the computational overhead of generative replay, and also consider it when comparing with other baselines.\n\nWe thank the reviewer for the suggestion on the overhead of GR, which we will include in our revised manuscript. The cost of GR comes in generating $D_r$. In the case of SD, we use a representative set of 5000 images generated from prompts by ChatGPT, which takes approximately 6 hours on a single GPU. As these prompts are general, we reuse the same $D_r$ across all SD experiments. The GR overhead is far smaller than the computational resources needed to retrain SD from scratch (reported to require 256 A100s for 150,000 GPU hours). Also, the cost of generating $D_r$ can be amortized over many runs, making it negligible compared to the cost of training SA or baselines like ESD (which we discuss further in our response to reviewer 3). \n\nThank you again for your review and feedback. We hope our response has addressed your concerns. If so, we hope you will consider updating your review and score. \n\nReferences\n\n[1] Nguyen, T., et al. \"A survey of machine unlearning.\" \n\n[2] Bourtoule, L., et al. \"Machine unlearning.\" \n\n[3] Nguyen, Q.P., Low, B.K.H. and Jaillet, P.. \"Variational bayesian unlearning.\"\n\n[4] Golatkar, A., Achille A,, and Soatto, S.. \"Eternal sunshine of the spotless net: Selective forgetting in deep networks.\" \n\n[5] Wu, Y., Dobriban E., and Davidson S.. \"Deltagrad: Rapid retraining of machine learning models.\" \n"}]

Author rebuttal:
Thank you to the reviewers for their comments and feedback. We appreciate the positive comments and that the reviewers find our continual learning approach to concept forgetting to be meaningful and technically interesting. Based on the comments, we have revised the paper to include (i) additional comparisons to related work pointed out by the reviewers, (ii) details on computational cost, and (iii) text to clearly point out relevant portions of the Appendix (e.g., on concept leakage and additional quantitative results). Please see below for detailed responses to each reviewer. 

Meta-review: From the reviews, rebuttal and discussion, there was unanimous agreement that the problem being explored and technical contributions of the paper meet the standards for NeurIPS and further should be of broad interest to the community.
Decision: Accept (spotlight)