SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {x}, \mathbf {c} \sim p (\mathbf {x} | \mathbf {c}) p _ {f} (\mathbf {c})} [ \log p (\mathbf {x} | \theta^ {q}, \mathbf {c}) ] - \mathbb {E} _ {\mathbf {x}, \mathbf {c} \sim p (\mathbf {x} | \mathbf {c}) p _ {f} (\mathbf {c})} [ \log p (\mathbf {x} | \theta^ {*}, \mathbf {c}) ] \\ = \int p (\mathbf {x} | \mathbf {c}) p _ {f} (\mathbf {c}) \log p (\mathbf {x} | \theta^ {q}, \mathbf {c}) d \mathbf {x} d \mathbf {c} - \int p (\mathbf {x} | \mathbf {c}) p _ {f} (\mathbf {c}) \log p (\mathbf {x} | \theta^ {*}, \mathbf {c}) d \mathbf {x} d \mathbf {c} \\ = \mathbb {E} _ {p _ {f} (\mathbf {c})} \left[ \int p (\mathbf {x} | \mathbf {c}) \log \frac {p (\mathbf {x} | \theta^ {q} , \mathbf {c})}{p (\mathbf {x} | \theta^ {*} , \mathbf {c})} d \mathbf {x} \right] \\ = \mathbb {E} _ {p _ {f} (\mathbf {c})} \left[ \int p (\mathbf {x} | \mathbf {c}) \log \frac {p (\mathbf {x} | \mathbf {c}) p (\mathbf {x} | \theta^ {q} , \mathbf {c})}{p (\mathbf {x} | \mathbf {c}) p (\mathbf {x} | \theta^ {*} , \mathbf {c})} d \mathbf {x} \right] \\ = \mathbb {E} _ {p _ {f} (\mathbf {c})} \left[ \int p (\mathbf {x} | \mathbf {c}) \log \frac {p (\mathbf {x} | \mathbf {c})}{p (\mathbf {x} | \theta^ {*} , \mathbf {c})} d \mathbf {x} \right] - \mathbb {E} _ {p _ {f} (\mathbf {c})} \left[ \int p (\mathbf {x} | \mathbf {c}) \log \frac {p (\mathbf {x} | \mathbf {c})}{p (\mathbf {x} | \theta^ {q} , \mathbf {c})} d \mathbf {x} \right] \\ = \mathbb {E} _ {p f (\mathbf {c})} \left[ D _ {K L} \left(p (\mathbf {x} | \mathbf {c}) | | p (\mathbf {x} | \theta^ {*}, \mathbf {c})\right) \right] - \mathbb {E} _ {p f (\mathbf {c})} \left[ D _ {K L} \left(p (\mathbf {x} | \mathbf {c}) | | p (\mathbf {x} | \theta^ {q}, \mathbf {c})\right) \right] \\ = - \mathbb {E} _ {p _ {f} (\mathbf {c})} [ D _ {K L} (p (\mathbf {x} | \mathbf {c}) | | p (\mathbf {x} | \theta^ {q}, \mathbf {c})) ] \quad \text {(a p p l y L e m m a 2)} \\ \leq 0 \quad (\text {n o n - n e g a t i v i t y o f K L}) \\ \end{array}
$$
A.3 Proof of Corollary 1
The proof follows straightforwardly from Theorem 1.
$$
\begin{array}{l} \mathbb {E} _ {\mathbf {x}, \mathbf {c} \sim p (\mathbf {x} | \mathbf {c}) p _ {f} (\mathbf {c})} [ \log p (\mathbf {x} | \theta^ {q}, \mathbf {c}) - \log p (\mathbf {x} | \theta^ {*}, \mathbf {c}) ] = - \mathbb {E} _ {p _ {f} (\mathbf {c})} [ D _ {K L} (p (\mathbf {x} | \mathbf {c}) | | p (\mathbf {x} | \theta^ {q}, \mathbf {c})) ] \\ = - \mathbb {E} _ {p _ {f} (\mathbf {c})} [ D _ {K L} (p (\mathbf {x} | \mathbf {c}) | | q (\mathbf {x} | \mathbf {c})) ] \\ \end{array}
$$
where the first line is directly taken from the proof of Theorem 1 (second last line), while the second line makes use of the fact that the model  $p(\mathbf{x}|\theta^q,\mathbf{c}) = q(\mathbf{x}|\mathbf{c})$  by assumption.
B.1 VAE and DDPM
MNIST VAE The VAE encoder and decoder are simple MLPs, both with two hidden layers of dimensions 256 and 512. The latent space  $\mathbf{z}$  has dimensions 8. We choose a Bernoulli distribution over the pixels as the decoder output distribution, and a standard Gaussian as the prior. Class conditioning is performed by appending a one-hot encoding vector to the encoder and decoder inputs. The original VAE is trained for 100K steps, and the forgetting training is trained for 10K steps. We use a learning rate of  $10^{-4}$  and batch size of 256. As sampling with VAEs is cheap, we use 50K samples to calculate the FIM, and sample the replay data from a frozen copy of the original VAE during forgetting training.
CIFAR10 DDPM We adopt the same U-Net architecture as unconditional DDPM in [11], with four feature map resolutions  $(32 \times 32$  to  $4 \times 4)$  and self-attention blocks at the  $16 \times 16$  resolution. We use the linear  $\beta$  schedule with 1000 timesteps, and train for 800K steps with a learning rate of  $10^{-4}$  and batch size of 128. For classifier-free guidance, we use the FiLM transformation at every residual block and drop the class embeddings  $10\%$  of the time. For sampling, we use 1000 timesteps of the standard DDPM sampler with a guidance scale of 2.0. As sampling with diffusion models is significantly more expensive than VAEs, we generate and store a set of 5000 images, and subsequently use it both for calculating the FIM and as the replay dataset. For forgetting, we train the model with 20K training steps. We use a learning rate of  $10^{-4}$  and batch size of 128. As the CIFAR10 training set has 5000 images per class, when evaluating the image quality of the remaining classes, we generate 5000 images of each class for a total of 45000 images, and compare them against the corresponding 45000 images in the training set. Experiments are run on 2 RTX A5000s.
STL10 DDPM We conduct our STL10 experiments by resizing the dataset to the  $64 \times 64$  resolution. The experiments follow closely from our CIFAR10 experiments, where we have five feature map resolutions ( $64 \times 64$  to  $4 \times 4$ ) instead while keeping attention blocks at the  $16 \times 16$  resolution. Due to the smaller size of the dataset, we combine the train and test sets to form a larger training set, resulting in 1300 images per class. We train for a total of 250K steps with a learning rate of  $2 \times 10^{-4}$  and batch size of 64. All other hyperparameters are kept identical to the CIFAR10 experiments. For forgetting training, we train similarly for 20K steps with a learning rate of  $10^{-4}$  and batch size of 64. To evaluate the image quality of the remaining classes, we generate 1300 images of each class, for a total of 11700 images, and compare them against the corresponding 11700 images in the training set. Experiments are run on 2 RTX A5000s.
Classifier Evaluation In terms of classifier architectures and training, for MNIST, we train a simple two-layer CNN on the original MNIST dataset for 20 epochs. As for both CIFAR10 and STL10, we finetune a ResNet34 classifier pretrained on ImageNet that was obtained from the torchvision library. All layers of the ResNet34 classifier are finetuned on the target dataset for 20 epochs. We calculate  $\mathbb{E}_{p(\mathbf{x}|\theta ,\mathbf{c}_f)}P_{\phi}(\mathbf{y} = \mathbf{c}_f|\mathbf{x})$  and  $H(P_{\phi}(\mathbf{y}|\mathbf{x}_f))$  by averaging over 500 generated images of the forgotten class from the respective models.
B.2 Stable Diffusion
Forget Celebrities We use the open-source SD v1.4 checkpoint as the pretrained model for all Stable Diffusion experiments with Selective Amnesia. We choose v1.4 as opposed to newer versions for fair evaluations as competing baselines, SLD and ESD, are based on v1.4. Similar to the CIFAR10 and STL10 experiments, we generate 5000 images from SD v1.4 and use it for both FIM calculation and GR. These images are conditioned on 5000 random prompts that were generated with GPT3.5 [34]. We use 50 steps of the DDIM [35] sampler with a guidance scale of 7.5 for all image generation with SD. For forgetting training, we set the prompt to forget as  $\mathbf{c}_f = \{\text{"Brad Pitt"}\}$  or {"Angelina Jolie"} and train the model using the  $q(\mathbf{x}|\mathbf{c}_f)$  represented by 1000 images generated with prompts as specified in the main text. We train for a total of 200 epochs of the surrogate dataset with  $\lambda = 50$  and a base learning rate of  $10^{-5}$  (scaled by number of GPUs). We similarly generate the 50 test prompts using GPT3.5, and generate 20 images per prompt. Experiments are run on 4 RTX A6000s and training takes approximately 20 hours with peak memory usage of around 40GB per GPU. Note that we did not optimize for computational efficiency in our reported experiments;
by tuning hyperparameters in preliminary experiments, we could achieve similar performance with 2 A6000s and 6 hours of training. We believe additional performance gains can be achieved with further tuning and leave that for future work.
In terms of evaluation, we evaluate the 1000 generated images with the open-source GIPHY Celebrity Detector [33], which is trained to detect 2306 different celebrities. The classifier is composed of two stages: the first stage is a face detector while the second stage is a celebrity face classifier. If a given image is found to have multiple faces, we only consider the face with the highest probability of the target celebrity. This is to account for cases where multiple persons are in an image, but typically only one of them will be of the celebrity of interest. As for the baselines, for SLD Medium, we set the safety concept to either "Brad Pitt" or "Angelina Jolie" during inference, while for ESD-x, we train the model to forget the prompts "Brad Pitt" or "Angelina Jolie".
Forget Nudity For forgetting nudity, we tune only the unconditional (non-cross-attention) layers of the latent diffusion model as proposed in [8]. We use the same set of samples for calculating the FIM and for GR. The prompt to forget is set as  $\mathbf{c}_f = \{\text{"nudity", "naked", "erotic", "sexual"}\}$ . We set  $\lambda = 50$  and train for 500 epochs. Experiments are run on the same resources as the celebrities experiments.
We evaluate on the I2P dataset by generating one image per prompt with the provided random seeds. The 4703 images are evaluated using the open-source NudeNet classifier [36], with the default probability threshold of 0.6 to count as a positive detection of a nudity instance. As NudeNet considers exposed and covered content separately, we only consider nudity content that are classified as exposed. Manual inspection showed the classifier to give false positives; for example, 10 of the 16 images generated by SA classified as showing Female Genitalia actually have this attribute. Likewise, some images classified as showing Female Breasts actually showed Male Breasts.
In terms of baselines, for SLD Medium we set the safety concept to "nudity, sexual, naked, erotic". For ESD-u, we use the publicly available checkpoint from the official GitHub repository that was trained to forget nudity.
C.1 Forget Nudity
Table 3: Quantity of nudity content detected using the NudeNet classifier on the I2P benchmark dataset (4703 images). As NudeNet classifies covered and exposed content separately, all nudity content considered here are classified as exposed.
Armpits	Belly	Buttocks	Feet	Female
Breasts	Female
Genitalia	Male
Breasts	Male
Genitalia	Anus
SD v1.4	214	171	40	39	295	23	21	6	0
SD v2.1	191	124	24	30	154	14	12	6	0
SLD Medium	58	60	7	15	42	1	21	0	0
ESD-u	60	17	13	9	26	1	4	3	0
SA (Ours)	72	77	19	25	83	16	0	0	0
Upon manual inspection of I2P samples from SA, we observed that the NudeNet classifier has a tendency to classify gender incorrectly. For instance, male breasts are often classified as female breasts. Hence, the number of female breasts presented for our method is an overestimation of the true number. There is also a relatively high false positive rate for exposed female genitalia, as 6 of the flagged images for SA do not depict any exposed female genitalia.
Table 4: Quantity of nudity content detected using the NudeNet classifier from 1000 sampled images with the prompt "a photo of a naked person". Similar to I2P results, we only consider exposed content. Note that there is a larger number of Armpits and Female Breasts for SD v1.4 than there are images because NudeNet classifies multiple instances of each content per image separately.
Armpits	Belly	Buttocks	Feet	Female
Breasts	Female
Genitalia	Male
Breasts	Male
Genitalia	Anus
SD v1.4	1013	753	110	116	1389	453	8	3	0
SD v2.1	858	659	149	120	685	201	110	3	0
SLD Medium	360	369	38	56	351	115	73	1	0
ESD-u	86	56	7	35	55	5	10	0	0
SA (Ours)	204	172	15	105	245	27	0	0	0
We conduct an additional quantitative study on nudity by evaluating 1000 images sampled using the prompt "a photo of a naked person" using the NudeNet classifier. We use the same setup as the I2P experiments for all models. The results are shown in Table 4. Similar to the I2P experiments, our model drastically reduces the amount of nudity content compared to SD v1.4 and v2.1. ESD-u achieves the best scores overall. Our model outperforms SLD Medium, particularly on sensitive content like Female Breasts and Female Genitalia.
D Additional Samples
D.1 MNIST, CIFAR10, STL10
Figure 7: Additional sample images comparing the original MNIST VAE versus ours with the digit '0' forgotten with  $\lambda = 100$  (with GR), which corresponds to the hyperparameters shown in Table 1.
Figure 8: Additional sample images comparing the original STL10 DDPM versus ours with the 'airplane' class forgotten with  $\lambda = 10$  (with GR), which corresponds to the hyperparameters shown in Table 1.
$\lambda = 10$  no GR
Figure 9: Additional sample images comparing the original CIFAR10 DDPM versus ours with the 'airplane' class forgotten. We show three variants of SA, corresponding to the ablations shown in Table 1. It is clear from inspection that the image quality of the classes to remember is significantly impacted without the GR term ( $\lambda = 10$  no GR). When visually comparing  $\lambda = 1$  and  $\lambda = 10$  (both with GR), the differences are not immediately obvious to the naked eye, although the quantitative metrics show that  $\lambda = 10$  produces better results.
$\lambda = 1$  with GR
D.2 Stable Diffusion
Figure 10: Sample images with prompt "Brad Pitt in a tuxedo". These are an extension of Fig. 4 to provide the reader with more context as to the qualitative differences between the various approaches. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged man" and "male clown" for  $\mathbf{c}_f = \{\text{"Brad Pitt"}\}$ .
Figure 11: Sample images with prompt "a close up of Brad Pitt's face". These are an extension of Fig. 4. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged man" and "male clown" for  $\mathbf{c}_f = \{\text{"Brad Pitt"}\}$ .
Figure 12: Sample images with prompt "Brad Pitt laughing in a park". These are an extension of Fig. 4. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged man" and "male clown" for  $\mathbf{c}_f = \{\text{"Brad Pitt"}\}$ .
Figure 13: Sample images with prompt "Angelina Jolie in a black dress". These are an extension of Fig. 5. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman" and "female clown" for  $\mathbf{c}_f = \{\text{"Angelina Jolie"}\}$ .
Figure 14: Sample images with prompt "Angelina Jolie holding a bouquet of flowers". These are an extension of Fig. 5. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman" and "female clown" for  $\mathbf{c}_f = \{\text{"Angelina Jolie"}\}$ .
Figure 15: Sample images with prompt "Angelina Jolie wearing a hat". These are an extension of Fig. 5. The bottom two rows are our method, where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman" and "female clown" for  $\mathbf{c}_f = \{\text{"Angelina Jolie"}\}$ .
Figure 16: Sample images with prompt "realistic portrayal Maleficent movie" from SD v1.4 and our method where we set  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman" and "female clown" for  $\mathbf{c}_f = \{\text{"Angelina Jolie"}\}$ . Even though the prompt does not explicitly mention Angelina Jolie, we observe that our method generalizes to the portrayal of the character Maleficent.
Figure 17: Sample images with prompt "a photo of a naked person". These are an extension of Fig. 6. Note again that we are being conservative in censoring the image here.
Figure 18: We also experiment with forgetting of art styles. Here we attempt to forget the artistic style of Vincent van Gogh by setting  $q(\mathbf{x}|\mathbf{c}_f)$  to images of "pop art style" generated by SD v1.4 for  $\mathbf{c}_f = \{\text{"van gogh style"}\}$ . The images shown are samples from the prompt "a painting of a lion in van gogh style". SA successfully generates images that lack the distinct van gogh style (and instead contains elements of pop art style).
E Effects on Other Celebrities
Figure 19: Sample images investigating the effects on celebrities other than the one being forgotten when using our method. SA-Full indicates training of all layers and SA-Xattn indicates training of only the cross-attention layers. Both models are trained to forget  $\mathbf{c}_f = \{\text{"Angelina Jolie"}\}$  by setting to  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman". We use the prompt "[...] on stage", where [...] is substituted with Jennifer Aniston, Oprah Winfrey or Jackie Chan.
In this section we conduct a qualitative study on the effects on celebrities other than the one being forgotten. Ideally, the changes to other celebrities should be minimal. We revisit the case of forgetting Angelina Jolie by setting  $q(\mathbf{x}|\mathbf{c}_f)$  to "middle aged woman". We train two variants, training all layers (like in Sec. 4.2 of the main text on forgetting famous persons) and training only the cross-attention layers. We abbreviate them as SA-Full and SA-Xattn respectively.
From Fig. 19, we see that SA-Full leads to slight changes in the depiction of Jennifer Aniston (compared to how she looks in person, or to SD v1.4), but minimal changes to Oprah Winfrey and
Jackie Chan. We hypothesize that this is due to Jennifer Aniston sharing greater similarities to Angelina Jolie than the latter two, as they are both female and of similar ethnicity, leading the model to more strongly associate the two together. This is not an inherent limitation of SA; the effects can be minimized by tuning only the cross-attention layers, as seen in SA-Xattn rendering Jennifer Aniston (and the other celebrities) as accurately as SD v1.4. This corroborates the findings in [8], which recommends tuning the cross-attention layers only if one wishes to forget concepts that are specified explicitly (e.g., celebrity names which are mentioned in the prompt).
However, there are cases where celebrities can be rendered even without explicit mention of their names, for example in Fig. 16. In such cases, we observe anecdotally that tuning only the cross-attention layers limits the model's ability to generalize to such prompts. Recall that the unconditional (non-cross-attention) layers are responsible for generalization to prompts without explicit mention of the concept to forget (cf. the nudity experiments in Sec. 4.2 of the main text), hence tuning only the cross-attention layers unsurprisingly limits generalization performance. As such, there is a trade-off between generalization and interference of other concepts. We recommend tuning all layers if the user wants a good balance of generalization but with potentially slight interference of closely related concepts, and only the cross-attention layers if minimal interference of other concepts is required, at the expense of generalization. We leave a more precise study of this trade-off to future work.