Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models

## Abstract

Abstract The machine learning community is increasingly recognizing the importance of fostering trust and safety in modern generative AI (GenAI) models. We posit machine unlearning (MU) as a crucial foundation for developing safe, secure, and trustworthy GenAI models. Traditional MU methods often rely on stringent assumptions and require access to real data. This paper introduces Score Forgetting Distillation (SFD), an innovative MU approach that promotes the forgetting of undesirable information in diffusion models by aligning the conditional scores of “unsafe” classes or concepts with those of “safe” ones. To eliminate the need for real data, our SFD framework incorporates a score-based MU loss into the score distillation objective of a pretrained diffusion model. This serves as a regularization term that preserves desired generation capabilities while enabling the production of synthetic data through a one-step generator. Our experiments on pretrained label-conditional and text-to-image diffusion models demonstrate that our method effectively accelerates the forgetting of target classes or concepts during generation, while preserving the quality of other classes or concepts. This unlearned and distilled diffusion not only pioneers a novel concept in MU but also accelerates the generation speed of diffusion models. Our experiments and studies on a range of diffusion models and datasets confirm that our approach is generalizable, effective, and advantageous for MU in diffusion models. Warning: This paper contains sexually explicit imagery, discussions of pornography, racially-charged terminology, and other content that some readers may find disturbing, distressing, and/or offensive.

## 1 Introduction

Diffusion models, also known as score-based generative models (Sohl-Dickstein et al., ; Song and Ermon, ; Ho et al., ; Dhariwal and Nichol, ; Karras et al., ), have emerged as the leading choice for generative modeling of high-dimensional data. These models are widely celebrated for their ability to produce high-quality, diverse, and photorealistic images (Nichol et al., ; Ramesh et al., ; Saharia et al., ; Rombach et al., ; Podell et al., ; Zheng et al., ).
However, their capacity to memorize and reproduce specific images and concepts from training datasets raises significant privacy and safety concerns. Moreover, they are susceptible to poisoning attacks, enabling the generation of targeted images with embedded triggers, posing substantial security risks (Rando et al., ; Chen et al., ).

To address these challenges, we introduce *Score Forgetting Distillation* (SFD), a novel framework designed to efficiently mitigate the influence of specific characteristics in data points on pre-trained diffusion models. This framework is a key part of the broader domain of Machine Unlearning (MU), which has evolved significantly to address core issues in trustworthy machine learning (Lowd and Meek, ; Narayanan and Shmatikov, ; Abadi et al., ). Originating from compliance needs with data protection regulations such as the “right to be forgotten” (Hoofnagle et al., ), MU has broadened its scope to include applications in diffusion modeling across various domains like computer vision and content generation
(Gandikota et al., ; Fan et al., ; Heng and Soh, ). Additionally, MU aims to promote model fairness (Oesterling et al., ), refine pre-training methodologies (Jain et al., ; Jia et al., ), and reduce the generation of inappropriate content (Gandikota et al., ).
The development of SFD is aligned with these objectives, providing a strategic approach to mitigate the potential risks and reduce the high generation costs associated with diffusion models, thereby advancing the field of trustworthy machine learning.

MU methods are generally categorized into two types: exact MU and approximate MU. Exact MU entails creating a model that behaves as if sensitive data had never been part of the training set (Cao and Yang, ; Bourtoule et al., ). This process requires the unlearned model to be identical in distribution to a model retrained without the sensitive data, both in terms of model weights and output behavior. In contrast, approximate MU does not seek an exact match between the unlearned model and a retrained model. Instead, it aims to approximate how closely the output distributions of the two models align after the unlearning process. A prominent strategy in approximate MU utilizes the principles of differential privacy (Dwork, ). For instance, Guo et al. () introduced a certified removal technique that prevents adversaries from extracting information about removed training data, offering a theoretical guarantee of data privacy. However, these approaches typically necessitate retraining the model from scratch, which can be computationally intensive and require access to the original training dataset.
Efficient and stable unlearning has become crucial in MU. Techniques like the influence functions (Warnecke et al., ; Izzo et al., ), selective forgetting (Golatkar et al., ), weight-based pruning (Liu et al., ), and gradient-based saliency (Fan et al., ) have been explored, though they often suffer from performance degradation or restrictive assumptions (Becker and Liebig, ). These methods are primarily applied to MU for image classification tasks and do not adequately address the rapid forgetting and unlearning required for data generation tasks.

Given the prominence of diffusion models, there is a growing need to develop MU techniques that specifically cater to these models, ensuring efficient unlearning while maintaining generation capabilities (Gandikota et al., ; Fan et al., ; Heng and Soh, ). Our SFD framework efficiently distills the knowledge from a pre-trained diffusion model by optimizing two learnable modules—a generator network and a score network—guided by the frozen pre-trained model itself.
The score network is trained to optimize the score associated with the generator by minimizing a score distillation loss, which aims to match the conditional scores of the class to forget and the classes to remember with those of the pre-trained model.
The generator network learns to produce examples that are “indistinguishable” by the pre-trained score network and fake score network in terms of score predictions, utilizing a model-based cross-class score distillation loss.

This dual functionality facilitates both MU and rapid sampling, effectively bridging the gap in generation speed between diffusion-based models and one-step counterparts such as GANs and VAEs.
The forgetting process is seamlessly integrated into the model distillation, where we concurrently optimize the score-matching loss and the forgetting loss. This integrated approach offers a robust framework for achieving effective unlearning and fast generation, thereby providing a comprehensive solution for enhancing the efficiency and trustworthiness of diffusion-based generative modeling.

Our approach’s effectiveness is demonstrated through both class and concept forgetting tasks for diffusion models in image generation. The experiments conducted on class-conditional diffusion models pretrained on CIFAR-10 and STL-10 demonstrate that SFD effectively erases the target class while preserving the image generation quality for other classes. We also present extensive ablation studies that support the robustness and efficiency of our method, which achieves competitive performance on the key metric for class forgetting, namely Unlearning Accuracy (UA), and significantly improves several metrics for preserving generative quality and efficiency, including Fréchet Inception Distance (FID), Inception Score (IS), Precision and Recall, and generation speed measured by the number of function evaluations (NFEs).

Additionally, experiments conducted on Stable Diffusion reveal that SFD successfully erases concepts associated with specific text inputs. Our method outperforms the baselines in both celebrity forgetting and NSFW-concept forgetting tasks. Moreover, because our method operates in a completely data-free manner, it significantly reduces the privacy risks associated with the MU fine-tuning process. The development of SFD benefits from related works on MU, distribution matching, score matching, acceleration methods for diffusion sampling, and data-free diffusion distillation. A detailed review of these topics is provided in Appendix A.

Our key contributions are:

- •
Introducing SFD, a pioneering data-free approach for MU that utilizes cross-class score distillation in diffusion models to achieve not only effective forgetting but also fast one-step generation.
- •
Developing a robust and efficient technique to distill score-based generative models into one-step generators, incorporating the MU loss as a regularization element within the model-based score distillation framework to optimize both distillation and forgetting simultaneously.
- •
Validating the effectiveness of our method with experiments on not only class-conditional diffusion models based on DDPM and EDM, but also text-to-image diffusion models based on Stable Diffusion,
marking the first instance of accelerated forgetting in machine unlearning for diffusion models. This achievement demonstrates the potential of our method for broader applications and sets the stage for future advancements in the field.

Figure: Figure 2: Overview of score forgetting distillation (SFD). Some notations are labeled along with corresponding components. ‘Snowflake’ refers to the frozen (non-trainable), ‘Fire’ refers to the trainable, and ‘Combine’ refers to combining operation on input losses by arithmetic addition according to predefined weights.
Refer to caption: extracted/5909233/images/sfd_overview.png

## 2 Method

Diffusion models are celebrated for their superior performance in generating high-quality and diverse samples. However, their robust capabilities also introduce challenges, particularly the risk of misuse in generating inappropriate content. This concern highlights the ethical implications and potential negative impacts of their application. Additionally, these models have a significant drawback: slow sampling speeds. This inefficiency becomes particularly problematic in downstream tasks that require finetuning on synthetic data generated by these models. When access to real data is not feasible, the task of preparing a sufficiently large synthetic dataset can already become computationally prohibitive (Yin et al., ). This issue is especially acute in the context of MU and image generation, where access to real data often raises privacy concerns, making reliance on synthetic data crucial. Consequently, the slow sampling rate of diffusion models presents a critical bottleneck, necessitating improvements to enable effective data-free MU operations.

In this section, we introduce SFD,
a principled and data-free approach designed to address the MU problem while simultaneously achieving fast sampling for diffusion models. Building on recent advancements in data-free diffusion distillation for one-step generation (Luo et al., ; Zhou et al., ), we conceptualize MU in diffusion models as a problem of MU-regularized score distillation.

### 2.1 Problem Definition and Notations

Before diving into the specific MU problem, we will first establish the essential concepts and notations in diffusion modeling:
A diffusion model corrupts its data $x\sim p_{\text{data}}(x\mid c)$ during the forward diffusion process at time $t$ as $z_{t}=a_{t}x+\sigma_{t}\epsilon_{t}$,
where $\epsilon_{t}\sim\mathcal{N}(0,1)$, $c$ represents the given condition such as a label or text, and $a_{t}$ and $\sigma_{t}$ are diffusion scheduling parameters. The goal of pretraining a diffusion model is to obtain an optimal score estimator $s_{\phi}(z_{t},c,t)$ such that $s_{\phi}(z_{t},c,t)=\nabla_{z_{t}}\ln p_{\text{data}}(z_{t}\,|\,c)$. Let $x_{\phi}(z_{t},c,t)$ be the optimal conditional mean estimator such that for $x_{\phi}(z_{t},c,t)=\mathbb{E}[x\,|\,z_{t},c,t]$. Applying Tweedie’s formula (Robbins, ; Efron, ) in the context of diffusion modeling (Luo, ; Chung et al., ; Zhou et al., ), the optimal score and conditional mean estimators, $s_{\phi}$ and $x_{\phi}$, for the training data are related as follows:

$$ $\displaystyle\textstyle s_{\phi}(z_{t},c,t)=\frac{a_{t}x_{\phi}(z_{t},c,t)-z_{ t}}{\sigma_{t}^{2}},~{}~{}~{}~{}x_{\phi}(z_{t},c,t)=\frac{z_{t}+\sigma_{t}^{2} s_{\phi}(z_{t},c,t)}{a_{t}}.$ (1) $$

With this optimal score estimator, we can construct a corresponding reverse diffusion process, enabling us to approximately sample from the data distribution through numerical discretization along the time horizon (Anderson, ; Song et al., ).

A distilled one-step diffusion model is a one-step generator capable of producing samples from the generative distribution of a pretrained model in a single step. The generation process for this one-step generator is defined as $g_{\theta}(n,c)$, where $n\sim\mathcal{N}(\mathbf{0},\mathbf{I})$. Denote the generative distribution of $x$ given class $c$ as $\mathcal{D}_{\theta,c}$, and the optimal score estimator corresponding to the one-step generator $g_{\theta}$ as $s_{\psi^{*}(\theta)}(z_{t},c,t)$. The same as how $x_{\phi}$ and $s_{\phi}$ is related in Eq. 1, we have

$$ $\displaystyle\textstyle s_{\psi^{*}(\theta)}(z_{t},c,t)=\frac{a_{t}x_{\psi^{*} (\theta)}(z_{t},c,t)-z_{t}}{\sigma_{t}^{2}}.$ (2) $$

For class forgetting in class-conditional diffusion models, our goal is to unlearn a specific class by overriding it with another class while minimizing any negative impact on the remaining classes. We denote the class to forget as $c_{f}$, the remaining classes (classes other than $c_{f}$) as $\mathcal{C}_{r}\coloneqq\{c_{r}\mid c_{r}\neq c_{f}\}$, and the class for overriding $c_{f}$ as $c_{o}\in\mathcal{C}_{r}$. The distribution of the remaining classes is denoted as $\mathcal{D}_{r}$ over the set $C_{r}$, the sampling distribution of all classes after unlearning as $\mathcal{D}_{s}$, and the conditional distribution of samples from class $c$ generated by $g_{\theta}$ as $\mathcal{D}_{\theta,c}\coloneqq g_{\theta}(\mathcal{N}(\mathbf{0},\mathbf{I}),c)$. The class forgetting problem can be solved by aligning the model distribution of $x$ given $c_{f}$ under the generator $g_{\theta}$ with the original data distribution of $x$ given $c_{o}$, and by simultaneously ensuring that the distributions of $x$ given $c_{r}$ under both the model and the original data are matched.
Specifically, our objective is to forget $c_{f}$ and override it with $c_{o}$ by aligning the distributions such that $\mathcal{D}_{\theta,c_{f}}\stackrel{{\scriptstyle d}}{{=}}p_{\text{data}}(x
\mid c_{o})$, while preserving the remaining classes by ensuring $\mathcal{D}_{\theta,c_{r}}\stackrel{{\scriptstyle d}}{{=}}p_{\text{data}}(x
\mid c_{r}),~{}\forall c_{r}\in\mathcal{C}_{r}$.

In the problem setting of concept forgetting in text-to-image diffusion models, our goal is to unlearn the concepts associated with specific keywords, such as “Brad Pitt,” by substituting them with more generic terms like “a middle aged man,” as illustrated in Figure 1. This process aims to minimize any negative impact on the generation quality of other concepts, thereby maintaining the overall integrity and diversity of the images generated under text guidance.

### 2.2 Score Forgetting Distillation

In the problem of class unlearning, as described in Section 2.1, our goal is to align the conditional distributions of both the forgetting class and the remaining classes with those that would exist if the model had been retrained without the data from the forgetting class. By adapting the concept of data-free score distillation to the MU challenge, we aim to achieve this alignment using our proposed data-free MU process, SFD. Our method eliminates the need for access to the original training data and accelerates synthetic data sampling, effectively enabling the forgetting of a specific class while preserving the original generative capabilities for the other classes.

Specifically, for two arbitrary classes $c_{1}$ and $c_{2}$, we define a Score Forgetting Distillation (SFD) loss over the forward diffusion process of one-step generated fake data. The following analysis also applies when $c_{1}$ and $c_{2}$ refer to concepts.
We denote $z_{t},t,x\sim\mathcal{D}_{\theta,c}$ as a random sample generated as

$$ $z_{t}=a_{t}x+\sigma_{t}{\epsilon}_{t},~{}{\epsilon}_{t}\sim\mathcal{N}(\mathbf {0},\mathbf{I}),~{}t\sim\text{Unif}[t_{\text{min}},t_{\text{max}}],~{}x=g_{ \theta}(n,c),~{}n\sim\mathcal{N}(\mathbf{0},\mathbf{I}).$ $$

Taking the expectation over fake data generated by the distilled one-step generation model $g_{\theta}$ under class $c_{2}$ and subsequently corrupted through the forward diffusion process, we formulate this loss as:

$$ $\mathcal{L}_{\text{sfd}}(\theta;\phi,c_{1},c_{2})=\mathbb{E}_{z_{t},t,x\sim \mathcal{D}_{\theta,c_{2}}}\left[\omega_{t}\|s_{\phi}(z_{t},c_{1},t)-s_{\psi^{ *}(\theta)}(z_{t},c_{2},t)\|^{2}\right],$ (3) $$

where $\omega_{t}>0$ is a re-weighting function, and $\psi^{*}(\theta)$ represents the optimal solution to the model-based explicit SM (MESM) loss, which can be expressed as

$$ $\mathcal{L}_{\text{mesm}}(\psi;\theta,c)=\mathbb{E}_{z_{t},t,x\sim\mathcal{D}_ {\theta,c}}\left[\gamma_{t}\|s_{\psi}(z_{t},c)-\nabla_{x}\ln p_{\theta}(z_{t} \,|\,c)\|_{2}^{2}\right],$ (4) $$

where $\gamma_{t}>0$ is a re-weighting function.
In practice, the lack of the access to $\nabla_{x}\ln p_{\theta}(z_{t}\,|\,c)$ makes Eq. 4 intractable. However, we can alternatively optimize a denoising SM loss (Vincent, ) as

$$ $\displaystyle\textstyle\mathcal{L}_{\text{dsm}}(\psi;\theta,c)={\mathbb{E}_{z_ {t},t,x\sim\mathcal{D}_{\theta,c}}\left[\gamma_{t}\frac{a_{t}^{2}}{\sigma_{t}^ {4}}\|x_{\psi}(z_{t},c)-x\|_{2}^{2}\right],}$ (5) $$

which admits the same optimal solution as Eq. 4 and provides an estimation of the score of the generator $g_{\theta}$ at different noise levels. This setup allows us to tailor the SFD loss in Eq. 3 specifically for different class dynamics. When $c_{1}=c_{2}=c$, the SFD loss facilitates class-specific score distillation, optimizing the score to closely model that of the generator within the same class. Conversely, setting $c_{1}\neq c_{2}$ configures the SFD loss for score overriding, replacing the score $s_{\psi^{*}(\theta)}$ for class $c_{2}$ with the score $s_{\phi}$ for class $c_{1}$. This approach effectively addresses the dual objectives of class forgetting and targeted score modification, introducing two distinct losses to manage these scenarios:

- •
Distillation Loss: Enhances fidelity within a class by refining the generator’s score to closely match the true distribution of the class:
$\displaystyle\mathcal{L}_{\text{sfd}}(\theta;\phi,c_{r},c_{r})=\mathbb{E}_{z_{
t},t,x\sim\mathcal{D}_{\theta,c_{r}}}\left(\omega_{t}\|s_{\phi}(z_{t},c_{r},t)
-s_{\psi^{*}(\theta)}(z_{t},c_{r},t)\|^{2}\right).$
(6)
- •
Forgetting Loss: Alters the generator’s score to reflect characteristics of a different class, facilitating the effective forgetting of the original class attributes:
$\displaystyle\mathcal{L}_{\text{sfd}}(\theta;\phi,c_{o},c_{f})=\mathbb{E}_{z_{
t},t,x\sim\mathcal{D}_{\theta,c_{f}}}\left(\omega_{t}\|s_{\phi}(z_{t},c_{o},t)
-s_{\psi^{*}(\theta)}(z_{t},c_{f},t)\|^{2}\right).$
(7)

To summarize our approach, we now present the entire formulation as follows:

$$ $\displaystyle\min_{\theta}$ $\displaystyle\mathbb{E}_{c_{r}\sim\mathcal{C}_{r}}\mathcal{L}_{\text{sfd}}( \theta;\phi,c_{r},c_{r}),\text{ s.t. }\psi^{*}(\theta)=\arg\min_{\psi}\mathbb{ E}_{c\sim\mathcal{C}_{s}}\mathcal{L}_{\text{dsm}}(\psi;\theta,c),~{}\mathcal{L }_{\text{sfd}}(\theta;\phi,c_{o},c_{f})\leq C_{0}.$ $$

This formulation corresponds to a bi-level optimization problem (Ye et al., ; Hong et al., ; Shen et al., ), subject to an additional forgetting-based constraint. Solving this problem directly is challenging, so we initially relax the constraint specified by $\mathcal{L}_{\text{sfd}}$ in the above equation by integrating it into the distillation objective as an additional MU regularization term:

$$ $\displaystyle\min_{\theta}$ $\displaystyle~{}\mathbb{E}_{c_{r}\sim\mathcal{C}_{r}}\lambda\mathcal{L}_{\text {sfd}}(\theta;\phi,c_{r},c_{r})+\mu\mathcal{L}_{\text{sfd}}(\theta;\phi,c_{o}, c_{f}),\text{ s.t. }\psi^{*}(\theta)=\arg\min_{\psi}\mathbb{E}_{c\sim\mathcal{ C}_{s}}\mathcal{L}_{\text{dsm}}(\psi;\theta,c),$ $$

where $\lambda$ and $\mu$ are tunable constants that serve as control knobs to balance the distillation of the remaining classes and the unlearning of the target class.
Furthermore, we implement an alternating update strategy between $\theta$ and $\psi$. This approach mitigates the need to obtain the optimal score estimator $\psi^{*}(\theta)$ for each $\theta$, simplifying the computational process. We outline a practical implementation of this strategy in Algorithm 1. Specifically, generalizing the derivation in Zhou et al. (), we have the following Lemma, whose proof is provided in Appendix D:

###### Lemma 1 .

The Score Forgetting Distillation (SFD) loss in Eq. 3 can be equivalently expressed as

$$ (8) $$

A biased loss for $\theta$ can be derived by replacing $\psi^{*}(\theta)$ in either Eq. 3 or Eq. 8 with its SGD-based approximation $\psi$, and disregarding the dependency of $\psi^{*}$ on $\theta$ when computing the gradient of $\theta$. Empirical experiments by Zhou et al. () suggest that in the context of diffusion distillation without involving unlearning, Eq. 8 can be effective independently, while Eq. 3 may not perform as expected. This observation leads to a practical approach that involves subtracting Eq. 3 from Eq. 8. This strategy aims to sidestep detrimental biased gradient directions and potentially compensate for the overlooked gradient dependency of $\psi^{*}(\theta)$. We implement this approach in practice under the framework of SFD, defining the loss used in practice as follows:

$$ $\displaystyle\textstyle\hat{\mathcal{L}}_{\text{sfd}}(\theta,\psi;\phi,c_{1},c _{2},\alpha)$ $\displaystyle\textstyle=(1-\alpha)\omega_{t}\frac{a_{t}^{2}}{\sigma_{t}^{4}}\| x_{\phi}(z_{t},c_{1},t)-x_{\psi}(z_{t},c_{2},t)\|^{2}+$ (9) $\displaystyle\textstyle\omega_{t}\frac{a_{t}^{2}}{\sigma_{t}^{4}}(x_{\phi}(z_{ t},c_{1},t)-x_{\psi}(z_{t},c_{2},t))^{T}(x_{\psi}(z_{t},c_{2},t)-x),$ (10) $$

where $\alpha\geq 0$ is some constant that typically set as 1 or 1.2, $z_{t}=a_{t}x+\sigma_{t}{\epsilon}_{t},~{}x\sim\mathcal{D}_{\theta,c_{2}},~{}{
\epsilon}_{t}\sim\mathcal{N}(\mathbf{0},\mathbf{I}),~{}t\sim\text{Unif}[t_{
\text{min}},t_{\text{max}}]$.
In this paper, we follow Yin et al. () and Zhou et al. () to set $\omega_{t}=\frac{\sigma_{t}^{4}}{a_{t}^{2}}\frac{C}{\|x_{\phi}(z_{t},t,c)-x\|_
{1,\text{sg}}}$,
where $C$ is the data dimension and “sg” stands for stop gradient.

Similar to Eqs. 6 and 7, we have the following:

$$ $\displaystyle\text{Distillation Loss:}\quad\hat{\mathcal{L}}_{\text{sfd}}( \theta,\psi;\phi,c_{r},c_{r},\alpha),\quad\text{where }z_{t},t,x\sim\mathcal{D }_{\theta,c_{r}}$ (11) $\displaystyle\text{Forgetting Loss:}\quad\hat{\mathcal{L}}_{\text{sfd}}(\theta ,\psi;\phi,c_{o},c_{f},\alpha),\quad\text{where }z_{t},t,x\sim\mathcal{D}_{ \theta,c_{f}}$ (12) $$

where timestep $t$ is omitted for brevity. Intuitively speaking, our algorithm first trains the approximate score estimator $s_{\psi}$ to mimic the score of the generator $g_{\theta}$ at different time points $t$ of the forward diffusion process, and then uses both the pre-trained score estimator and the fake score estimator across these time points to instruct the generator itself. The alternate updating approach largely reduces the computational cost of obtaining an optimal score estimator for the generator while effectively passing an informative learning signal to the generator and helping the generation quality improve rapidly over time. It is worth noting that the whole training process require neither real data nor fake data synthesized by reversing the full diffusion process, and a pre-trained score network of a diffusion model is sufficient to provide proper supervision on distillation as well as machine unlearning. In other words, our method is data-free.

**Table 1: Class forgetting results on CIFAR-10 and STL-10. “SFD” refers to the DDPM model trained with Score Forgetting Distillation, while “SFD-CFG” refers to the SFD model trained with classifier-free guidance (as discussed in Section 3.2). UAs that exceed the testing recall rate of the forgetting class (96.60% for CIFAR-10 and 98.15% for STL-10) are highlighted in yellow.**
| Dataset | Model | UA ($\uparrow$) | FID ($\downarrow$) | IS ($\uparrow$) | Precision ($\uparrow$) | Recall ($\uparrow$) | NFEs ($\downarrow$) | Data-free |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| CIFAR-10 | Retrain | 98.5 | 7.94 | 8.34 | 0.6418 | 0.5203 | 1000 | ✘ |
| ESD (Gandikota et al., ) | 91.21 | 12.68 | 9.78 | 0.7709 | 0.3848 | 2000 | ✔ |  |
| SA (Heng and Soh, ) | 85.80 | 9.08 | - | 0.4120 | 0.7670 | 2000 | ✔ |  |
| SalUn (Fan et al., ) | 99.96 | 11.25 | 9.41 | 0.7806 | 0.3176 | 2000 | ✘ |  |
|  | SFD (Ours) | 99.64 | 5.35 | 9.51 | 0.6587 | 0.5471 | 1 | ✔ |
| STL-10 | Retrain | 97.54 | 26.52 | 8.30 | 0.5573 | 0.4526 | 1000 | ✘ |
| ESD (Gandikota et al., ) | 92.01 | 39.32 | 10.16 | 0.5229 | 0.2898 | 2000 | ✔ |  |
| SalUn (Fan et al., ) | 99.31 | 20.78 | 10.89 | 0.5713 | 0.5415 | 2000 | ✘ |  |
| SFD (Ours) | 99.02 | 18.82 | 10.93 | 0.5543 | 0.4054 | 1 | ✔ |  |
|  | SFD-CFG (Ours) | 99.64 | 15.32 | 11.46 | 0.5983 | 0.3551 | 1 | ✔ |

###### Lemma 1 .

## 3 Experiments

In our experiments, we thoroughly evaluate our method for class forgetting in diffusion models pretrained on two datasets, CIFAR-10 and STL-10, which have been commonly used for evaluating MU in previous studies.
We provide the details of them in Appendix B.
We also assess our method for concept forgetting tasks, such as celebrity forgetting, in text-to-image diffusion models.

#### Forgetting setups

We explore class forgetting in class-conditional image generation tasks using DDPM (Ho et al., ), and investigate concept forgetting in text-to-image generation tasks using Stable Diffusion (Rombach et al., ). Class forgetting aims to prevent class-conditional diffusion models from generating images of a specified class, while concept forgetting seeks to remove the model’s ability to generate images containing specific concepts, such as celebrities or inappropriate content.
Class-conditional and text-to-image sampling are achieved by inputting class labels and text prompts into the respective diffusion models, with fidelity further enhanced by classifier-free guidance introduced in Ho and Salimans (). Specifically, we approach unlearning by overriding a class or concept with another that is safe to retain. The class forgetting experiments were conducted on class-conditional diffusion models pre-trained on CIFAR-10 and STL-10, while the concept forgetting experiments were conducted on Stable Diffusion, including forgetting celebrities, specifically American actor Brad Pitt and actress Angelina Jolie, and forgetting a general NSFW (not safe for work) concept, $i.e.$, nudity. For DDPM baselines, we used the default 1000-step DDPM samplers to obtain FIDs for samples from the remaining classes, while for SD baselines, we used DDIM samplers with 50 steps. In contrast, our method requires only a single step for generation, making it 1,000 times faster than the DDPM baselines and 50 times faster in latent sampling than the SD baselines.

#### Evaluation

To quantitatively assess the effectiveness of class forgetting, we primarily focus on the success rate of forgetting the target class, and the generative capability on classes to retain. Specifically, we measure the success rate of forgetting by unlearning accuracy (UA) employing an external classifier trained on the original training set, which is essentially the mis-classification rate of the classifier on the generated samples from the target class. We measure image generation quality using Fréchet Inception Distance (FID) (Heusel et al., ) and sample diversity using Inception Score (IS) (Salimans et al., ). Additionally, we report Precision and Recall (Kynkäänniemi et al., ), and number of function evaluations (NFEs) for sampling. Following Fan et al. (), we compute and report generation quality metrics using generated samples, with the full training set from the remaining classes serving as the reference. For concept forgetting tasks including celebrity forgetting and “nudity” forgetting, we also provide quantitative evaluations as well as qualitative comparison. Specifically, we evaluate celebrity forgetting using a off-the-shelf celebrity face detector, while we assess the MU performance of our “nudity” forgetting model on the I2P benchmark ([https://github.com/ml-research/i2p](https://github.com/ml-research/i2p)). Please refer to Section B.2 for more details of the evaluation metrics.

#### Implementation details

Our main implementation of class forgetting experiments is based on DDPM (Ho et al., ),
where we utilize the codebase developed by Fan et al. (). Additionally, we implement our method using EDM (Karras et al., ) framework and the official codebase ([https://github.com/NVlabs/edm](https://github.com/NVlabs/edm)). For concept forgetting experiments, we implement our method for SD models based on the implementation of Zhou et al. ().
We adopt the same model configuration for both the generator $g_{\theta}$ and its score estimation network
$s_{\psi}$ and initialize the model weights according to the pre-trained score network $s_{\phi}$. This type of initialization prepares a good starting point for SFD.

#### SFD-Two Stage

In addition to initializing both the generator and the fake score network with the pre-trained score network, we also experimented on a different initialization, $i.e.$, initializing the generator with a pre-distilled generator model weights. Considering the nature of “first distilling then forgetting,” we named this variant “SFD-Two Stage.” For this variant specifically, we disabled exponential moving average (EMA) and adopted a more aggressive regularization with $\lambda_{\psi}=\mu_{\psi}=\lambda_{\theta}=\mu_{\theta}=1.0$. The rationale behind this configuration was that the first stage distillation would have prepared a solid foundation for the second stage forgetting, which enables fast forgetting by increasing the weight of forgetting loss and by further prioritizing it in the second stage.
We use Adam optimizer with $\beta_{1}=0$ and $\beta_{2}=0.999$ for all the experiments. The base learning rate for both DDPM and EDM models is set to $10^{-5}$, except that we slightly increase the learning rate for $s_{\psi}$ when distilling DDPM models. More details on the hyperparameter settings for the experiments can be found in Table 8.

Figure: Figure 3: Generated images on CIFAR-10 and STL-10 during the training of SFD. The upper panel shows $3\times 3$ grids of generated samples at different time steps, with fixed random seeds and class labels arranged from 1 to 9 (left to right, top to bottom). The same sequence of random seeds is used across all grids to ensure consistency. The lower panel illustrates the forgetting process for two examples from CIFAR-10 and STL-10.
Refer to caption: extracted/5909233/images/sfd_main.png

### 3.1 Experimental Results

#### Class forgetting

From the empirical results, the proposed method, SFD, can effectively unlearn unwanted content ($e.g.$, a class of objects) and converge rapidly towards the level of generation quality of the pre-trained model. Additionally, the models fine-tuned by SFD inherently enables one step generation. Figure 3 shows that the remaining classes were in fact intact during the MU-regularized distillation, the generation quality of class 1 to 9 were consistently improving as the number of generator-synthesized images, which were used by SFD for distillation and MU, went up. The FID between generated samples and training dataset decreased nearly exponentially fast as is captured by Figure 5. The forgetting class, on the other hand, was initialized to output airplanes and gradually forced to match the assigned class, $i.e.$, the class of automobile. The forgetting effect noticeably took place between 10k and 20k training steps. From Figure 5, we observe a steady increase of unlearning accuracy, reflecting the extent to which the generated Class 0 samples can no longer be correctly identified by the pre-trained image classifier.

On CIFAR-10, we observed that the SFD-Two Stage model (or Two Stage, for short), which involves first distilling the pre-trained diffusion model with 50,000 steps and then fine-tuning it using the SFD loss for the same number of steps, exhibited faster forgetting. In Figure 7, we report two performance metrics, FID and UA, during the unlearning stage, compared with the results from SFD. The results indicate that SFD consistently outperforms the two-stage approach in both metrics given sufficient training. Although the two-stage approach started with a lower FID than SFD, its performance fluctuated and declined over time. The UA initially increased rapidly, peaked, and then slightly decreased at the end. The gain in UA during the unlearning stage came at the cost of FID. In contrast, SFD effectively coordinated machine unlearning and distillation to forget specific classes while retaining the original generative capability for the remaining classes, thereby improving both FID and UA throughout finetuning and achieving better final results. Nonetheless, the two-stage approach remains practical, especially when forgetting requirements vary over time or when there is an urgent need, as it appears more flexible and efficient under such conditions.

Figure: Figure 4: FID between generated images and original dataset of remaining classes. The solid blue line and dot denote the training FIDs and final FID evaluated at the last checkpoint of one-step SFD generator; the dotted green line marks the initial FID of the pre-trained model using 1,000 sampling steps. The solid orange line and dot mark the training UAs and final UA evaluated at the last checkpoint of SFD; the dotted orange line marks the initial UA of the pre-trained model.
Refer to caption: extracted/5909233/images/df2_training_metrics.jpg

#### Celebrity forgetting

We provide both qualitative and quantitative results of celebrity forgetting tasks on two selected celebrities, $i.e.$, Brad Pitt and Angelina Jolie, where the concepts to forget are “bard pitt” and “angelina jolie”, respectively, and the corresponding concepts to override are “a middle aged man” and “a middle aged woman”, respectively. As is shown in Figure 1 and Table 2, we showcase the effectiveness of SFD for forgetting certain concepts in text-to-image diffusion models, such as removing the generative capability of celebrities.

**Table 2: Quantitative results of celebrity forgetting of two celebrities, $i.e.$, “Brad Pitt” and “Angelina Jolie.” Bold values indicate the best score in each column, while underlined values represent the second-best.**
| Model | Brad Pitt | Angelina Jolie |  |  |
| --- | --- | --- | --- | --- |
| Prop. | GCD ($\downarrow$) | Prop. | GCD ($\downarrow$) |  |
| w/o Faces ($\downarrow$) | w/o Faces ($\downarrow$) |  |  |  |
| SD v1.4 (Rombach et al., ) | 10.4% | 60.6% | 11.7% | 73.8% |
| SLD Medium (Schramowski et al., ) | 14.1% | 0.47% | 11.9% | 3.29% |
| ESD-x (Gandikota et al., ) | 34.7% | 2.01% | 32.6% | 3.35% |
| SA (Heng and Soh, ) | 5.8% | 7.52% | 4.4% | 7.74% |
| SFD-Two Stage (Ours) | 1.76% | 2.5% | 1.92% | 1.06% |

#### “Nudity” forgetting

In addition to the celebrity forgetting experiments, we conducted experiments on a broader concept forgetting task, namely, forgetting “nudity” as a concept. We note that “nudity” is a broader concept than individuals ($e.g.,$ celebrities) and forgetting “nudity” in general is much more challenging. Therefore, we adopted a slightly different strategy for this task to enhance the forgetting performance. In particular, we first created a list of 12 common human subjects (see Table 5) that can be potentially misused for generating “nudity”-related contents and randomly paired them with one of NSFW keywords (see Table 6) as prompts to forget. We further leveraged the negative prompting technique to match these prompts with their corresponding prompts to override. Specifically, we take the original text prompt as the conditional text input while using the concatenated NSFW keywords instead of an empty string as the unconditional text input. We notice this approach also has a concept forgetting effect on the original score distillation method, which is denoted as “SiD-LSG-Neg.” We report key MU performance metrics in Table 3. Sample images by baselines and SFD are displayed in Figure 6.

**Table 3: Quantitative results of “nudity” forgetting. Bold values indicate the best score in each column, while underlined values represent the second-best.**
| Model | Inapprop. Prob. ($\downarrow$) | Max. Exp. Inapprop. ($\downarrow$) |
| --- | --- | --- |
| SD v1.4 (Rombach et al., ) | 28.54% | 86.6% |
| SiD-LSG (Zhou et al., ) | 26.86% | 88.12% |
| SiD-LSG-Neg (Ours) | 20.97% | 81.64% |
| SLD Medium (Schramowski et al., ) | 14.10% | 71.73% |
| ESD-u (Gandikota et al., ) | 16.94 % | 69.68% |
| SFD-Two Stage (Ours) | 11.03% | 66.90% |

Figure: Figure 6: Generated images using different text-to-image diffusion models. The prompts used for generation follow the general form, “A photo of a <nudity keyword> <human subject>.” Sensitive body parts were manually censored after image generation.
Refer to caption: extracted/5909233/images/sfd_nudity.jpg

### 3.2 Ablation Studies

#### Ablation on the model architecture

EDM (Karras et al., ) is a state-of-the-art diffusion model with enhanced capability for generating high-quality images. To evaluate our method’s generalizability across different model architectures, we additionally conduct experiment using the EDM architecture. We adapted the codebase used by SiD (Zhou et al., ) and fine-tuned the pre-trained class-conditional CIFAR-10 EDM-VP model. Figure 5 shows that the FID results of our method can be further improve when based on a more powerful pre-trained model.

#### Abalation on the classifier-free guidance

Classifier-free guidance (CFG), first proposed by Ho and Salimans (), is a commonly-used strategy for conditional sampling. While typically adopted during inference to enhance class fidelity, it has also been shown to be useful for the training of score-based distillation (Yin et al., ; Zhou et al., ). We compare our models trained with and without CFG in Table 4. In our experiments on STL-10, we found that including classifier-free guidance during training improved the performance in terms of both FID and UA. However, we did not observe such improvements on the CIFAR-10 dataset; on the contrary, we noticed a degradation in the evaluation metrics. We speculate that the influence of CFG may be tied to the inter-class differences: when training data contain classes sharing similar features, such as automobile and truck in CIFAR-10, training with CFG may not be as beneficial as it is when the training dataset consists of more distinct classes.

**Table 4: Ablation study on classifier-free guidance during training and on the CIFAR-10 and STL-10 datasets. The percentages in green and red are the relative performance boost and degradation respectively when the model is trained without classifier-free guidance.**
| Model | FID ($\downarrow$) | UA ($\uparrow$) |
| --- | --- | --- |
| SFD | 5.35 | 99.64% |
| + CFG | 7.27 (+35.89%)0 | 99.62% (-0.02%) |

## 4 Conclusion

Our work demonstrates the benefits of the proposed score forgetting distillation (SFD), which achieves accelerated forgetting with score-based distillation, providing a unified and effective solution to diffusion-based generative modeling and machine unlearning. The generator trained by our method produces high-quality images of desired classes with a single step while the target class is effectively forgotten.
Our experiments show that the proposed strategy attains noticeable gains in performance on both CIFAR-10 and STL-10. We further conduct the detailed study with the SFD in different settings, *e.g.*, comparing SFD against baselines as well as different configurations of SFD in terms of UA, FID, and other metrics. Additionally, we provide qualitative results of concept forgetting for text-to-image diffusion models like SD.
To summarize, the forgetting method is effective and general, with the potential to be incorporated into existing models, such as text-to-image diffusion models.

## Appendix A Related work

#### Unlearning for Machine Learning Models

The study of MU can be traced back to classical machine learning models in response to data protection regulations
such as “the right to be forgotten” [Cao and Yang, , Hoofnagle et al., , Bourtoule et al., , Nguyen et al., ]. Due to its capability of assessing data influence on model performance, the landscape of MU has expanded to
encompass diverse domains, such as image classification [Ginart et al., , Golatkar et al., , Neel et al., , Sekhari et al., ], text-to-image generation [Gandikota et al., , Zhang et al., , Kumari et al., , Fan et al., ], federated learning [Halimi et al., , Che et al., ], and graph neural networks [Chen et al., , Chien et al., , Wu et al., ].
In the literature, ‘exact’ unlearning, which involves retraining the model from scratch after removing specific training
data points, is often considered the gold standard. However, this approach comes with significant computational
demands and requires access to the entire training set [Thudi et al., ]. To address these challenges, many research
efforts have shifted towards the development of scalable and effective approximate unlearning methods [Liu et al., , Chen et al., ]. In addition,
probabilistic methods with certain provable removal guarantees have been explored, often leveraging the concept of
differential privacy [Neel et al., , Sekhari et al., ]. Focusing on MU in diffusion-based image generation, this paper introduces a general data-free approach for rapid forgetting and one-step sampling in diffusion models, eliminating the need to access any real data.

#### Challenges in Machine Unlearning

In examining the challenges and strategies associated with diffusion models and MU, several key issues and methodologies have been identified. Diffusion models, particularly when trained on data from open collections, face risks of contamination or manipulation, which could lead to the generation of inappropriate or offensive content [Chen et al., , Schramowski et al., ]. Strategies to mitigate these include data censoring and safety guidance to steer models away from undesirable outputs [Nichol et al., ], and introducing subtle perturbations to protect artistic styles [Shan et al., ]. Despite these measures, challenges remain in fully preventing diffusion models from generating harmful content or being susceptible to targeted poison attacks [Rando et al., ]. Furthermore, the evaluation of MU presents unique difficulties, especially as conventional retraining benchmarks are often impractical. Empirical metrics for assessing MU include unlearning accuracy, the utility of the model post-unlearning, and the use of classifiers to gauge the integrity of generated outputs [Jang et al., ]. Unlike existing methods, our approach efficiently suppresses the generation of harmful content using a one-step diffusion generator that overrides ‘unsafe’ concepts with MU-regularized score-based distillation.

#### Distribution Matching and Score Matching

Generative modeling is a pivotal area in statistics and machine learning. Prior to the development of diffusion models and their associated denoising score matching (SM) techniques, effectively matching distributions in high-dimensional spaces—particularly those with intractable probability density functions—posed a significant challenge. Traditionally, deep generative models aimed to minimize discrepancies between data and model probability distributions using various distribution-matching related loss functions. These included Kullback-Leibler (KL) divergence [Kingma and Welling, , Yin and Zhou, ], Jensen-Shannon (JS) divergence [Goodfellow et al., ], and transport cost [Tanwisuth et al., , Zheng and Zhou, , Zhang et al., , Tanwisuth et al., ]. While VAEs and GANs developed under this framework have significantly advanced the field of generative modeling, they have exhibited limited capabilities in faithfully regenerating the original data.
More recent methods have utilized data-based Fisher divergence [Song and Ermon, , Ho et al., , Song et al., ] to compare noise-corrupted data with noise-corrupted model distributions. While directly minimizing Fisher divergence, $i.e.$, the explicit SM loss, is intractable, diffusion models have effectively transformed the problem into minimizing a data-based denoising SM loss [Vincent, , Sohl-Dickstein et al., ]. This transformation has allowed diffusion models to demonstrate exceptional capabilities in generating high-dimensional data that closely resemble the original distribution. However, the iterative denoising-based sampling inherent in these models is not only slow but also complicates efforts to further optimize the data generation process for downstream tasks. This issue becomes particularly challenging for tasks such as MU, which require the model to selectively forget specific concepts we are targeting in this paper.

#### Accelerated Diffusion Models

Classic score-matching-based diffusion models [Sohl-Dickstein et al., , Song and Ermon, , Ho et al., , Song et al., ] have become increasingly influential in developing generative models with high extensibility and sample quality [Dhariwal and Nichol, , Karras et al., , Ramesh et al., ]. However, standard Gaussian diffusion models, along with other non-Gaussian variants [Hoogeboom et al., , Austin et al., , Chen and Zhou, , Zhou et al., ], suffer from relatively slow sampling compared to traditional one-step generative models, such as GANs and VAEs.
Inspired by the success of applying diffusion processes to the training of generative models, Xiao et al. [] and Wang et al. [] were among the first to promote faster generation by leveraging both adversarial training techniques and diffusion-based data augmentation. However, these approaches inevitably reintroduce potential issues like training instability and mode collapse.
Closely related to the original score matching, Salimans and Ho [] proposed progressively halving the steps needed in the reverse generation process. Similarly, Song et al. [] presented the consistency model as a method for distilling the reverse ODE sampling process. Along this direction, much effort has been made by others [Xu et al., , Yin et al., , Luo et al., , Zhou et al., ] to improve both sample quality and diversity.

#### Data-Free Score Distillation

To address the slow sampling speed associated with traditional diffusion models, score distillation methods have been developed to harness pretrained score functions. These methods approximate data scores, facilitating model distribution matching under noisy conditions to align with the noisy data distribution governed by the pretrained denoising score matching function. These methods, as explored in several recent works [Poole et al., , Wang et al., , Luo et al., , Nguyen and Tran, , Yin et al., ],
primarily utilize the KL divergence, whose gradients can be analytically computed using both the pretrained and estimated score functions. Importantly, these KL-based methods do not require access to real data, as the KL divergence is defined with respect to the model distribution. While these approaches have successfully approximated the data distribution in a data-free manner, they often suffer from performance degradation when compared to the original, pretrained teacher diffusion model. Consequently, additional loss terms that require access to the original training data or data synthesized with the pretrained diffusion models are often necessary to mitigate this performance degradation. However, employing these terms voids the data-free feature of the process.

In response to these challenges, Score identity Distillation (SiD) has emerged as an effective data-free solution for matching distributions by minimizing a model-based Fisher divergence. Although directly computing this divergence is intractable, its minimization is effectively converted into a model-based score distillation loss. This data-free method facilitates the distillation of the pretrained score function from the teacher diffusion model into a potentially superior one-step student generator. Inspired by the success of this data-free score distillation, we are motivated to integrate its loss into our algorithm, SFD, to enhance its effectiveness and efficiency in generative modeling with data-free unlearning.

#### Evaluation of Machine Unlearning

When applying MU to classification tasks, effectiveness-oriented metrics include unlearning accuracy, which evaluates how accurately the model performs on the forget set after unlearning [Golatkar et al., ]. Utility-oriented metrics include remaining accuracy, which measures the updated model’s performance on the retain set post-unlearning [Song and Mittal, ], and testing accuracy, which assesses the model’s generalization capability after unlearning. For generation tasks, accuracy-based metrics use a post-generation classifier to evaluate the generated content [Zhang et al., ], while quality metrics assess the overall utility of the generated outputs [Gandikota et al., ]. A significant limitation of these metrics, particularly in measuring unlearning effectiveness, is their heavy dependence on the specific unlearning tasks [Fan et al., ]. To address this, we train an external classifier to evaluate *unlearning accuracy* (UA), ensuring that the generated images do not belong to the forgetting class or concept. Additionally, we use FID to evaluate the quality of image generations for non-forgetting classes or prompts.

## Appendix B Experimental Details

### B.1 Datasets for class forgetting tasks

For the class forgetting tasks, we utilize CIFAR-10 [Krizhevsky, ] at a resolution of $32\times 32$ and STL-10 [Coates et al., ] at $64\times 64$ resolution.
The CIFAR-10 dataset consists of 60,000 32$\times$32 color images in 10 classes, with 6,000 images per class. There are 50,000 training images and 10,000 test images. The dataset consists of 50,000 training images and 10,000 test images. It is organized into five training batches and one test batch, each containing 10,000 images. The test batch includes precisely 1,000 randomly-selected images from each class. The training batches, which hold the remaining images in random order, may have varying numbers of images from each class. The STL-10 dataset is another natural image dataset with 10 classes, each of which has 500 training data and 800 testing data. The image data has a higher resolution of 96$\times$96 in pixels and RGB color channels compared with CIFAR-10. The images were acquired from labeled examples on ImageNet [Deng et al., ]. During training time, the image data from STL-10 are resized to 64$\times$64. Due to the limited number of the original training data, both training and testing data were used in the experiments, making up 13,000 training images in total.

### B.2 Evaluation

#### Unlearning accuracy

For class forgetting tasks, we employed an external classifier to obtain unlearning accuracy (UA), ensuring that the generated images are not associated with the class or concept designated for forgetting. The UA is essentially the mis-classification rate of the classifier on the generated samples from the target class. A classifier with high test accuracy and low UA typically indicates effective forgetting, ensuring that the generated images are unlikely to belong to the target class or concept. For the external classifier, we fine-tuned ResNet-34 [He et al., ] for 10 epochs on both CIFAR-10 and STL-10 datasets using transfer learning, which is originally pretrained on ImageNet [Deng et al., ]. We adapted the original 1000-way classification model by replacing the last fully-connected layer with a customized fully-connected layer with 10 output dimension. The resulting classifiers achieved training and testing accuracies of 99.96% and 95.03% on CIFAR-10, and 100.00% and 96.20% on STL-10, respectively.

#### GCD score

For the celebrity forgetting task, we first generated 1,000 images generated from 50 different prompts per celebrity. We then utilized an open-source celebrity detector(^1^11https://github.com/Giphy/celeb-detection-oss) to calculate the proportion of images without human faces, referred to as probability without faces (“Prop. w/o Faces”), and the average probability of detecting specific celebrities in images that contain faces, referred to as the Giphy Celebrity Detection (GCD) score.

#### I2P metrics

We followed the Inappropriate Image Prompts (I2P) benchmark introduced by Schramowski et al. [] to assess the risk of generating NSFW images in text-to-image diffusion models. The I2P dataset consists of 4,703 text prompts covering a wide range of NSFW concepts, including “nudity.” For each prompt, we generated 10 images and applied both the NudeNet and Q16 detectors to identify inappropriate content. We report the sample-level inappropriate probability (referred to as “Inapprop. Prob.”) and the prompt-level inappropriate rate (referred to as “Max. Exp. Inapprop.”).

### B.3 SFD-Two Stage

We plot two main evaluation metrics for class forgetting experiments on CIFAR-10 for comparing SFD with SFD-Two Stage in Figure 7.

Figure: (a) FID
Refer to caption: extracted/5909233/images/cifar10_rfid_sfd_vs_2stage_forget1_noema.jpg

### B.4 Implementation Details

We implemented our techniques in a newly developed codebase, loosely based on the original implementations by [Karras et al., , Fan et al., , Zhou et al., ]. The pseudo-code is described in Algorithm 1. We performed
extensive evaluation to verify that our implementation produced exactly the same results as previous
work, including samplers, pre-trained models, network architectures, training configurations, and
evaluation. We ran all experiments using PyTorch with 4 NVIDIA RTX A5000 GPUs.

Figure: Algorithm 1 SFD: Score Forgetting Distillation

### B.5 Forgetting Celebrities

The text prompts used to train our model to forget “Brad Pitt” and “Angelina Jolie” were simply “brad pitt” and “angelina jolie,” which correspond to the overriding prompts “a middle aged man” and “a middle aged woman,” respectively.

### B.6 Forgetting Nudity as a Concept

We provide details of “nudity” forgetting experiments. Table 5 lists 12 common human subjects by category that were used for fine-tuning our model, while Table 6 lists 27 NSFW keywords, which often carry sexual connotations and suggest nudity. For the forgetting text prompts, we randomly combined one of the human subject with one NSFW keyword in two forms: “<NSFW keyword> <human subject>” and “<human subject>, <NSFW keyword>,” which correspond to a negative prompt composed of the comma-separated 27 NSFW keywords.

We also provide additional nudity detection results by NudeNet detector in Table 7. Upon reviewing the ’unsafe’ images flagged by NudeNet, we identified several false positives (see Figure 8 for examples) that were unrelated to the classification results, such as cases where no exposed anus was present in images generated by SFD.

Table: Table 5: List of 12 subjects potentially prone to abuse

**Table 6: List of nudity-related NSFW keywords used for negative prompts**
| Category | NSFW Keywords |
| --- | --- |
| Sexual Themes | nudity, sexual, explicit, erotic, inappropriate, indecent, suggestive, provocative, seductive, lewd |
| Clothing and Exposure | revealing, topless, bottomless, lingerie, bikini, underwear, skimpy, swimsuit, improper clothing, overexposure |
| Explicit Contexts | pornography, adult content, nsfw, intimate, mature themes, objectification, private setting |

**Table 7: Count of exposed body parts detected using the NudeNet classifier on the I2P benchmark dataset (4703 images).**
| Model | Armpits | Belly | Buttocks | Feet | Female | Female | Male | Male | Anus |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  |  |  |  | Breasts | Genitalia | Breasts | Genitalia |  |
| SD v1.4 | 214 | 171 | 40 | 39 | 295 | 23 | 21 | 6 | 0 |
| SD v2.1 | 191 | 124 | 24 | 30 | 154 | 14 | 12 | 6 | 0 |
| SLD Medium | 58 | 60 | 7 | 15 | 42 | 1 | 21 | 0 | 0 |
| ESD-u | 60 | 17 | 13 | 9 | 26 | 1 | 4 | 3 | 0 |
| SA | 72 | 77 | 19 | 25 | 83 | 16 | 0 | 0 | 0 |
| SFD (Ours) | 25 | 10 | 4 | 3 | 8 | 2 | 1 | 7 | 1 |

Figure: Figure 8: Detection results of SFD-generated images using NudeNet. False alarms are marked in red, while true positives are marked in green.
Refer to caption: extracted/5909233/images/nudenet_false_alarm.png

### B.7 Hyperparameter Settings

We list all the detailed hyparameter settings for training our DDPM, EDM, SD models in Table 8.

**Table 8: Detailed unlearned and distilled diffusion hyperparameter setting in for both DDPM, EDM, and SD model architectures**
| Scope | Hyperparameter | Model |  |  |
| --- | --- | --- | --- | --- |
| DDPM | EDM | SD |  |  |
| Training | batch size | 128 | 256 | 8 |
| #kimgs | 6,400 | 20,480 | 100 / 300 |  |
| Distillation | $\sigma_{\text{init}}$ | 2.5 | 2.5 | 2.5 |
| $t_{\text{min}}$ | 38 | 0 | 20 |  |
| $t_{\text{max}}$ | 712 | 800 | 980 |  |
|  | $\eta$ | 1.2 | 1.2 | 1.0 |
| Forgetting | $c_{f}$ | 0 | 0 | see B.5/B.6 |
| $c_{o}$ | 1 | 1 | see B.5/B.6 |  |
| $s_{\psi}$ | $\lambda_{\psi}$ | 1.0 | 1.0 | 1.0 |
| $\mu_{\psi}$ | 0.01 | 0.01 | 1.0 |  |
| optimizer | Adam | Adam | Adam |  |
| learning rate | $3\times 10^{-5}$ | $10^{-5}$ | $3\times 10^{-6}$ |  |
| $\beta_{1}$ | 0.0 | 0.0 | 0.0 |  |
| $\beta_{2}$ | 0.999 | 0.999 | 0.999 |  |
|  | ${\epsilon}$ | $10^{-8}$ | $10^{-8}$ | $10^{-8}$ |
| $g_{\theta}$ | $\lambda_{\theta}$ | 1.0 | 1.0 | 1.0 |
| $\mu_{\theta}$ | 0.01 | 0.01 | 1.0 |  |
| optimizer | Adam | Adam | Adam |  |
| learning rate | $10^{-5}$ | $10^{-5}$ | $10^{-6}$ |  |
| $\beta_{1}$ | 0.0 | 0.0 | 0.0 |  |
| $\beta_{2}$ | 0.999 | 0.999 | 0.999 |  |
|  | ${\epsilon}$ | $10^{-8}$ | $10^{-8}$ | $10^{-8}$ |

## Appendix C Limitations

There can be substantial disparities and biases between training and testing datasets in real-world settings. These discrepancies might result in models performing poorly and having unintended effects when applied to new, unseen data. To address these challenges and lessen the impact of biases, it is crucial to employ strategies like data preprocessing, augmentation, and regularization. Additionally, considerations around environmental and computational resource usage are important. Such measures will enhance the models’ usability and accessibility across diverse user groups.

## Appendix D Proof of Lemma 1

For a fixed timestep $t$, we have:

$$ $\displaystyle E_{g_{\theta}}\|s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2})\|^{2}$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{\theta}]$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-\int_{y}(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}\nabla_{y}p_{\theta} (y\mid c_{2})dy$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-\int_{y}(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}\nabla_{y}\left(\int _{x}p(y\mid x)p_{\theta}(x\mid c_{2})dx\right)dy$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-\int_{y}(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}\int_{x}\nabla_{y}p( y\mid x)p_{\theta}(x\mid c_{2})dxdy$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-\iint_{x,y}(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s(y\mid x)p_{ \theta}(x,y\mid c_{2})dxdy$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s_{ \phi}]-E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}s(y\mid x)]$ $\displaystyle=E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y,c_{2}))^{T}(s_{ \phi}+\sigma^{-2}(y-\alpha x))]$ $\displaystyle=\alpha\sigma^{-2}E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y, c_{2}))^{T}((\sigma^{2}s_{\phi}+y)/\alpha-x)]$ $\displaystyle=\alpha\sigma^{-2}E_{g_{\theta}}[(s_{\phi}(y,c_{1})-s_{\theta}(y, c_{2}))^{T}(x_{\phi}(y,c_{1})-x)]$ $$

where $g_{\theta}$ represents the joint distribution of $z,x$ and $z=\alpha x+\sigma{\epsilon},x\sim\mathcal{D}_{\theta,c_{2}},{\epsilon}\sim
\mathcal{N}(\mathbf{0},\mathbf{I})$. We can see that the equality holds for arbitrary $t$ up to some constant. Therefore, for any weighted sum or expectation of the losses w.r.t. $t$, we know the two expressions are equivalent.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Is it possible to provide the running time for the whole unlearning procedure compared with others?

Is it possible to provide some robustness evaluation on T2I unlearned models?
soundness: 3
strengths: Several strengths:

1. Novelty of the Approach: The paper introduces a novel method for machine unlearning in diffusion models using score distillation. To the best of current knowledge, this is the first work that approaches class and concept unlearning from the score distillation perspective. This innovative method opens up new avenues for efficient and effective unlearning in generative models.

2. Clarity of Presentation: The authors present their ideas and methods clearly, which helps readers understand both the existing literature and the proposed approach. The paper effectively explains complex concepts related to diffusion models and machine unlearning, making it accessible to a broader audience.

3. Extensive Experiments: The paper includes comprehensive experiments across various unlearning scenarios. These experiments not only demonstrate the effectiveness of the proposed method but also provide valuable insights into its applicability in different contexts. The thorough empirical evaluation strengthens the credibility of the findings
summary: The paper introduces Score Forgetting Distillation (SFD), a novel, data-free method for machine unlearning in diffusion models. SFD enables the efficient removal of specific classes or concepts from pretrained diffusion models without requiring access to real training data. It works by aligning the conditional scores of the undesired ("unsafe") content with those of desired ("safe") content within the model's score function.
weaknesses: Weakness:

Running Time Concerns: There is a concern regarding the overall runtime of the unlearning process proposed in the paper. While the method aims to accelerate forgetting and improve generation speed, it is important to evaluate the computational efficiency and practicality of the unlearning procedure. Without empirical data on the time required for the process, it's unclear whether the method is suitable for large-scale or real-time applications.

Lack of Robustness Evaluation Against Adversarial Prompts: Previous studies [1,2] have shown that adversarial text prompts can circumvent unlearning mechanisms in text-to-image (T2I) models. The paper lacks an evaluation of how robust the proposed method is against such adversarial inputs. Assessing the method's ability to prevent the generation of unlearned content when faced with intentionally manipulative prompts is crucial. Providing robustness evaluations or proposing solutions to address potential vulnerabilities would strengthen the validity and reliability of the unlearning approach.

1. Zhang Y, Jia J, Chen X, et al. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now[C]//European Conference on Computer Vision.

2.Pham M, Marshall K O, Cohen N, et al. Circumventing concept erasure methods for text-to-image generative models[C]//The Twelfth International Conference on Learning Representations. 2023.

---

confidence: 5
contribution: 2
initial_rating: 6
presentation: 2
questions: check Weakness section
soundness: 2
strengths: 1. Covers multiple diffusion model unlearning tasks, such as object classes, celebrity identities, and nudity.
2. Has a strong theoretical foundation.
summary: This paper introduces Score Forgetting Distillation (SFD), a method that enables diffusion models to unlearn unsafe content by aligning it with safe content, without needing real data. SFD improves generation speed and maintains quality for other concepts, supporting safer generative AI.
weaknesses: 1. There is no related work section to highlight the differences among existing works.
2. Evaluation under adversarial prompt attacks, such as UnlearnDiffAtk [1], is missing, even though it is a common evaluation tool for unlearned diffusion models.
3. A comparison of generation times is missing, which is essential to demonstrate the efficiency of the proposed one-step generator.
4. The baselines used in the experiments are weak, and stronger state-of-the-art methods, such as AdvUnlearn [2], should be considered.

[1]  To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images … For Now, ECCV’24
[2]  Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models, NeurIPS’24

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 4
questions: See weaknesses.
soundness: 3
strengths: 1. The paper presents a novel approach to machine unlearning in diffusion models by introducing Score Forgetting Distillation. Unlike previous methods that require access to real data or rely on stringent assumptions, SFD operates in a data-free manner and integrates unlearning directly into the distillation process. This creative combination of score distillation and machine unlearning extends to both class-conditional and text-to-image diffusion models.

2. The proposed method is thoroughly evaluated on multiple datasets and models, including CIFAR-10, STL-10, and Stable Diffusion. The experiments include both quantitative metrics (e.g., Unlearning Accuracy, FID, IS, Precision, Recall) and qualitative results (e.g., visual examples of forgetting effects). Ablation studies analyze the impact of different components of the method, enhancing the robustness of the evaluation.

3. The paper is well-written and organized, with clear explanations of the problem, methodology, and experimental results. Figures and tables are informative and effectively support the text. The inclusion of algorithm pseudo-code and detailed descriptions aids in understanding the implementation details.

4. Machine unlearning in diffusion models is crucial for enhancing the safety and trustworthiness of generative AI. By providing a data-free method that accelerates both unlearning and sampling, the paper contributes to making diffusion models more practical and secure for real-world applications.
summary: The paper introduces Score Forgetting Distillation (SFD), a data-free method for machine unlearning (MU) in diffusion models. The goal is to efficiently erase specific classes or concepts (e.g., "unsafe" or undesirable content) from pre-trained diffusion models without accessing the original training data. SFD achieves this by aligning the conditional scores of the target classes or concepts to forget with those of safe or desired ones, effectively overriding the unwanted information. The method incorporates a score-based MU loss into the score distillation process, which not only enables data-free unlearning but also accelerates the generation speed of diffusion models by distilling them into one-step generators. The authors validate the effectiveness of SFD through experiments on class-conditional diffusion models (e.g., CIFAR-10 and STL-10) and text-to-image models like Stable Diffusion, demonstrating that their approach can effectively forget specific classes or concepts while preserving the quality and diversity of the remaining content.
weaknesses: 1. While the paper introduces a novel method, the contributions may be perceived as incremental within the context of existing work on score distillation [1] and machine unlearning [2]. The proposed method builds upon recent advancements in data-free diffusion distillation, with the primary novelty being the integration of machine unlearning into this framework. Providing additional emphasis on the theoretical foundations or discussing broader implications could strengthen the overall contribution.

2. Although SFD reduces the number of sampling steps, this benefit appears to be a general characteristic of score distillation or matching methods. The authors do not sufficiently explain the specific advantages of integrating unlearning into score distillation. A more intuitive and necessary ablation study would involve comparing the results with the following baselines: (1) comparing SFD against existing unlearning methods applied to one-step diffusion models (e.g., SiD/DMD distilled models [1,3,4] or pre-trained consistency models [5,6]); (2) applying score distillation or matching methods to models that have already undergone unlearning. Additionally, the authors could compare their method with baseline unlearned models using fewer sampling steps (e.g., 50-step DDIM [7] or 10-step DPM-Solver++ [8,9]). Intuitively, reducing the number of sampling steps should primarily affect image quality rather than unlearning accuracy.

3. Machine unlearning generally focuses on methods for rapid modification of existing models. An essential aspect of evaluations is the algorithm's efficiency [10]. If the training time of the proposed algorithm is excessively long, one might question why not simply filter out all harmful data and retrain the model from scratch. However, the paper does not provide an analysis of the algorithm's runtime or computational efficiency. If this is a misunderstanding on my part, clarification would be appreciated.

4. A major application of machine unlearning is its direct deployment in existing products for hot updates to production models, which requires the algorithm to handle sequential unlearning requests effectively [11]. However, the SFD method presented by the authors has only been applied to models that have not undergone prior distillation. Does SFD perform equally well when applied to distilled models? For example, does it work effectively when sequentially forgetting classes such as airplane, bird, car, cat, and deer from the STL-10 dataset? In such scenarios, does it perform better than other unlearning methods?

5. There appear to be inconsistencies in the experimental results. For instance, in Table 1, the authors compare their method with ESD, SA, and SalUn, whereas in Tables 2 and 3, some of these baselines are missing. Could the authors briefly explain the reason for excluding them?

> [1] Zhou, Mingyuan, et al. "Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation." Forty-first International Conference on Machine Learning. 2024.
> 
> [2] Kumari, Nupur, et al. "Ablating concepts in text-to-image diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
> 
> [3] Yin, Tianwei, et al. "One-step diffusion with distribution matching distillation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
>
> [4] Yin, Tianwei, et al. "Improved Distribution Matching Distillation for Fast Image Synthesis." arXiv preprint arXiv:2405.14867 (2024).
>
> [5] Song, Yang, et al. "Consistency models." Proceedings of the 40th International Conference on Machine Learning. 2023.
>
> [6] Song, Yang, and Prafulla Dhariwal. "Improved Techniques for Training Consistency Models." The Twelfth International Conference on Learning Representations.
>
> [7] Song, Jiaming, Chenlin Meng, and Stefano Ermon. "Denoising Diffusion Implicit Models." International Conference on Learning Representations.
>
> [8] Lu, Cheng, et al. "Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps." Advances in Neural Information Processing Systems 35 (2022): 5775-5787.
>
> [9] Lu, Cheng, et al. "Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models." arXiv preprint arXiv:2211.01095 (2022).
>
> [10] Fan, Chongyu, et al. "SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation." The Twelfth International Conference on Learning Representations.
>
> [11] Zhang, Yihua, et al. "Unlearncanvas: A stylized image dataset to benchmark machine unlearning for diffusion models." arXiv preprint arXiv:2402.11846 (2024).

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Regarding the first weakness, could the authors provide justification for selecting Score Identity Distillation (SiD) over Diff-Instruct and DMD in this work? Alternatively, could Diff-Instruct and DMD be integrated into the SFD framework? I encourage the authors to include related discussions or experimental results in the revised paper to address these points.

2. How were the baseline results obtained? If they were sourced from original papers, could the authors specify the exact source for these baselines? It is unclear if the setting in Table 1 aligns directly with any cases from previous work such as [1]. Please clarify, and, if applicable, identify comparable results in [1] that correspond directly with Table 1.

3. Style forgetting and random data forgetting are standard choices for MU evaluation. Why were these not considered in this study?

4. Run-time efficiency (RTE), defined as the computation time for applying an MU method, is not reported here. Additionally, the computational cost associated with pretraining a score function is not addressed. I would appreciate relevant results or discussions on these aspects.

5. Another approach to achieving similar efficiency for an unlearned model could involve performing forgetting first, followed by distillation. Could the authors comment on this alternative?

[1] SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
soundness: 3
strengths: The exploration of machine unlearning (MU) using distilled diffusion models (DMs) is a novel approach, particularly given that distilled DMs enable significantly more efficient content generation compared to traditional DMs.
summary: This paper presents Score Forgetting Distillation (SFD), a novel approach to machine unlearning (MU) in diffusion models (DM) that aims to "forget" undesirable information by aligning the conditional scores of "unsafe" classes with those of "safe" ones. Departing from traditional MU objectives for diffusion models, the authors introduce a score-based MU loss built upon the score distillation objective proposed in [1], which facilitates one-step synthetic data generation and obviates the need for real data. Through empirical evaluations on pretrained label-conditional and text-to-image diffusion models, the authors demonstrate the efficacy of their method in achieving targeted unlearning without compromising model quality.

[1] Score Identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation
weaknesses: 1. The presentation of this paper could be improved. The authors have omitted a dedicated related works section, instead integrating substantial content from prior work [1] within the methodology section. This blending of information makes it challenging to clearly identify the paper’s unique contributions.

2. The empirical study is not as comprehensive as recognized work on the same topic [2]. See questions.

[1] Score Identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation
[2] SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation

Meta-review: This paper received mixed reviews, with both positive and negative reviewers maintaining their positions throughout the discussion. The positive reviewers praised the quality, significance, and empirical results of the work. In contrast, the negative reviewers raised concerns about the novelty of the method, the computational cost due to heavy training time, and the unclear motivation for performing distillation and unlearning simultaneously (led to the request for additional comparisons with baselines that first perform unlearning and then apply one-step distillation).

After carefully considering the discussion, I find myself more aligned with the positive reviewers for the following reasons:

Novelty and Technical Contribution: While the proposed method may appear to be an incremental combination of selective distillation (SiD) and an unlearning method, implementing it in practice requires substantial effort. For instance, framing the process of simultaneous distillation and unlearning as a bi-level optimization problem is non-trivial and involves careful design and relaxation. This level of technical depth adds value beyond a simple combination of existing techniques.

Motivation for Simultaneous Distillation and Unlearning: While the authors have not fully clarified the motivation behind combining these two objectives, I believe there is merit; the literature suggests that safety-aligned pre-trained models may lose their safety guarantees during subsequent fine-tuning [1,2]. This highlights a potential advantage of simultaneous distillation and unlearning, as it could prevent unexpected behavior when we try to first unlearn a diffusion model and then distill later. I encourage the authors to better articulate and empirically validate this motivation in their future work.

Training Time Concerns: The reviewers raised concerns about the computational cost of the proposed method. I think the authors' response to this is reasonable. Provided that the cost of the proposed method is reasonable compared to the sum of costs for the unlearning and distillation. 

Overall, despite some open questions and areas where the paper could provide more clarity, the contributions of this work are meaningful, and the method demonstrates potential for advancing the field. Therefore, I recommend acceptance, while strongly encouraging the authors to explicitly clarify the motivation for simultaneous distillation and unlearning.

[1] Qi et al., Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023.
[2] Kim et al., Safety Alignment Backfires: Preventing the Re-emergence of Suppressed Concepts in Fine-tuned Text-to-Image Diffusion Models, 2024.

Decision: Accept (Poster)