Paper: EraseDiff: Erasing Data Influence in Diffusion Models (ICLR-2024)

Reviewer comment (Experiments):
Design more experiments (and have more discussion) on random (or not so well-labeled) subsets to demonstrate model efficacy.

Author response:
Thanks for the suggestion. We conducted an additional experiment to perform sample unlearning on an unconditional DDPM with pre-trained weights from HuggingFace. Examples of generated images from the unscrubbed model and our scrubbed model are shown in Figure 21 on page 22. FID scores over 50K images are 10.85 and 10.99 for the unscrubbed model and our scrubbed model, respectively.  
We can see that the quality of generated images only drops by around 0.14 in terms of the FID score. It is true that it is hard to verify the unlearning effect in this setting. We'll investigate how to evaluate in a sample-wise unlearning setting in future work.

Reviewer comment (Evaluation):
Evaluating unconditional diffusion model utility with respect to specific sub-datasets can be hardly well-defined, so try to narrow down and specify the problem you want to tackle in a more guided way (such as only targeting some specific conditional diffusion models).

Author response:
Thanks for the suggestion. We clarified this in the conclusion of our revised manuscript.

Reviewer comment (Experiments):
Try more types of condition (or context).

Author response:
Thanks for the suggestion. More generated image examples with different settings can be found in the Appendix (unlearning different classes for CIFAR-10 and UTKFace, and different attributes on CelebA and CelebA-HQ). We have since added Table 7 on page 21 to show the performance in different settings.

Reviewer comment (Theory):
The formulation of the diffusion model unlearning problem in this work seems unconventional. Both the inner and outer objectives aim to optimize the model parameters. As such, it can be naturally defined as a multitask problem. One task seeks to preserve the utility of the diffusion model for the remaining data, while the other aims to eliminate the information related to the data slated for removal.

Author response:
**Q3.1 The diffusion unlearning can be naturally defined as a multitask problem.**

Indeed, this formulation is possible; however, it would not perform better than our proposal empirically.
To provide further insights, we performed extra experiments on the CIFAR-10 dataset using conditional DDIM (please see Tables 4 and 5, Figures 17 and 18 on page 20 in the revised manuscript). Formulation as a multi-task problem and solving it via scalarization as a single-level optimization is shown as SO in Table 4 and Figure 17. We observed that this would lead to a significant drop in the image quality on $\mathcal{C}_r$ when boosting to forget $\mathcal{C}_f$ by large.
As shown in Figure 17 ($\alpha=0.5$), when the generated images yield coarse information for $\mathcal{C}_f$, the quality of generated images for $\mathcal{C}_r$ drops significantly.
Overall, we believe formulating as vanilla multi-task problem is not beneficial. Thanks for the suggestion; further studies on effective multi-task formulation remain as future work, which we have reflected in the conclusion of our revised manuscript.

Reviewer comment (Theory):
The inner objective of this work strives to make the diffusion model incapable of generating meaningful images corresponding to C_f. The rationale behind defining sample unlearning in this manner is unclear. A more intuitive approach would be to ensure that the model, when trained with both the remaining and the forgetting data, has parameters equivalent to those obtained when trained solely on the remaining data after the unlearning process.

Author response:
**Q3.2 $\textbf{I.}$ The inner objective of this work strives to make the diffusion model incapable of generating meaningful images corresponding to $\mathcal{C}_f$. The rationale behind defining sample unlearning is unclear. $\textbf{II.}$ A more intuitive approach would be to ensure that the model, when trained with both the remaining and the forgetting data, has parameters equivalent to those obtained when trained solely on the remaining data after the unlearning process.**

Our objective is defined with respect to the forgetting data $\mathcal{D}_f$ and the remaining data $\mathcal{D}_r$, $\mathcal{D}_f$ could be randomly selected from the training data, i.e., sample unlearning.
We didn't evaluate sample-wise unlearning because, currently, it is hard to evaluate the unlearning effect when performing sample-wise unlearning for diffusion models, as Reviewer ${\color{orange}LbvE}$ mentioned.
Per your comment, we conducted an experiment to perform sample unlearning on an unconditional DDPM with pre-trained weights from HuggingFace (i.e., the forgetting data $\mathcal{D}_f$ is randomly selected from). Examples of generated images from the unscrubbed model and our scrubbed model are shown in Figure 21 on page 22 in the revised manuscript. FID scores over 50K images are 10.85 and 10.99 for the unscrubbed model and our scrubbed model respectively. We can see that the quality of generated images only drops by around 0.14 in terms of the FID score.

It's true that the scrubbed model with unlearning algorithms should be equivalent to the retrained model which uses the remaining data to retrain from scratch, which is the ideal unlearning algorithm. However, the removal of pertinent data followed by retraining diffusion models from scratch demands substantial resources and is often deemed impractical. Therefore, the model retrained solely on the remaining data is unknown to the unlearning algorithms. Instead, approximate unlearning algorithms (e.g., [1-2] and ours) achieve model scrubbing without the need for retraining. This is accomplished by updating the model parameters in a negative direction relative to the samples or concepts intended for erasure.

We evaluated the Weight Distance (WD) between the scrubbed models and the retrained model (please see Table 3 on page 8) in experiments.
Table 3 reveals that although the NegGrad method has a Weight Distance (WD) of 1.3533, it is unable to preserve model utility. In contrast, our approach attains the smallest WD among effective unlearning algorithms, i.e., 1.3534. Importantly, it successfully erases information about the forgetting classes $\mathcal{C}_f$, while concurrently preserving data regarding the remaining classes $\mathcal{C}_r$.

---
>[1] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto-Kaufman, and David Bau. Erasing concepts from diffusion models. arXiv preprint arXiv:2303.07345, 2023.
>
>[2] Heng, Alvin, and Harold Soh. "Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models." arXiv preprint arXiv:2305.10120 (2023).

Reviewer comment (Experiments):
The proposed method primarily focuses on class-wise unlearning and may have limitations when applied outside this specific context (e.g., nudity removal, artistic style removal).

Author response:
Our algorithm can perform unlearning outside of class-wise unlearning. Experiments on concept unlearning (unlearning the attribute 'Blond hair' or `Eyeglasses') can be found in Figure 5 (b) on page 9 in the main paper, in Figures 14 and 15 on page 18, and in Figure 16 on page 19 in the Appendix.
Thanks, we have since added descriptions to the manuscript for better clarity.

Reviewer comment (Evaluation):
How effective is the unlearning process when subjected to adversarial attacks?

Author response:
Thanks for the suggestion. We conducted an additional experiment to evaluate the unscrubbed model, the retrained model, and our scrubbed model when subjected to adversarial attacks. This experiment has been added to Figure 22 on page 22. We observed that as the magnitude of the perturbation increases, the quality of the generated images decreases for all models, and the generated images by our scrubbed model still contain no information about the forgetting classes $\mathcal{C}_f$.

Reviewer comment (Experiments):
I understand the assumptions made in the paper are that we do have access to $\mathcal{D}_r$. However, is this a reasonable assumption for large scale diffusion models? Often times, we have access to a pre-trained diffusion model and also to the forgetting data ($\mathcal{D}_f$) but assuming access to $\mathcal{D}_r$ which may be very large might not be reasonable. Have the authors thought about the case where we do not have access to the remaining data? How does the algorithm change? Is there significant impact on the results?

Author response:
**Q1.1 How does the algorithm change when we do not have access to the remaining data $\mathcal{D}_r$?**

In response to your comment and following the methodology outlined in [2], we conducted an additional experiment using Generative Replay (GR) to generate images $\mathcal{D}_r'$ as a substitute for $\mathcal{D}_r$ in the unlearning process. The findings from this experiment are presented in our revised manuscript, specifically in Table 6 and Figure 19 on page 21 in the revised manuscript. The results indicate that while image quality diminishes post-scrubbing when using generated images, our model still significantly outperforms other methods.

Additionally, we would like to highlight that in our initial submission, the unlearning process was conducted using only 8K data points, randomly selected from $\mathcal{D}_r$. Following your suggestion, we have not only clarified this in the main paper but also explored the impact of varying the number of samples from $\mathcal{D}_r$ on our algorithm, detailed in Figure 6 on page 15. We found that increasing the sample size can enhance the quality of images generated from the remaining classes, albeit at the price of slight detriment to the forgetting classes.

Reviewer comment (Evaluation):
The impact of unlearning seems to have affected the samples quality significantly.

Author response:
**Q1.3 Compare against Selective Amnesia [2] and FID score clarifications.**

In the current stage, we follow the same setting in [2] to forget samples belonging to the label '0' on CIFAR-10 and use the well-trained model from [2]. From [2], the FID score on the $\mathcal{C}_r$ is 9.67 for the original model and 9.08 for Selective Amnesia [2].
With an EMA decay of 0.99, our method achieves a FID score of 8.93 at 210 steps, 8.83 at 205 steps, and 8.90 at 200 steps. Generated examples are shown in Figure 20 on page 21 of the revised manuscript.

Regarding the FID score on CIFAR-10, results in Table 2 are reported for DDIM with inference time step $T=100$, and FID is computed over 5K images separately for each class, while results in Table 1 of [2] are reported for DDPM with inference time step $T=1000$ and FID is computed over 5K$\times 9$ images.
For your reference, the FID score computed over 40K images is around 8.18 in Table 2.

Reviewer comment (Theory):
The assumption in the methodology that they access to $\mathcal{D}_r$, hinders the use of the algorithm for large scale diffusion models such as Stable Diffusion.

Author response:
**Q1.1 How does the algorithm change when we do not have access to the remaining data $\mathcal{D}_r$?**

In response to your comment and following the methodology outlined in [2], we conducted an additional experiment using Generative Replay (GR) to generate images $\mathcal{D}_r'$ as a substitute for $\mathcal{D}_r$ in the unlearning process. The findings from this experiment are presented in our revised manuscript, specifically in Table 6 and Figure 19 on page 21 in the revised manuscript. The results indicate that while image quality diminishes post-scrubbing when using generated images, our model still significantly outperforms other methods.

Additionally, we would like to highlight that in our initial submission, the unlearning process was conducted using only 8K data points, randomly selected from $\mathcal{D}_r$. Following your suggestion, we have not only clarified this in the main paper but also explored the impact of varying the number of samples from $\mathcal{D}_r$ on our algorithm, detailed in Figure 6 on page 15. We found that increasing the sample size can enhance the quality of images generated from the remaining classes, albeit at the price of slight detriment to the forgetting classes.

Reviewer comment (Reproducibility):
There is no definition of the hyper-parameter $\lambda$. Does it control the balance between retaining and forgetting data?

Author response:
**Q1.4 Does hyper-parameter $\lambda$ control the balance between retaining and forgetting data?**

Yes, $\lambda$ controls the balance between retaining and forgetting data. An ablation study with respect to $\lambda$ is provided in Figure 6. Thanks, we have since added a description in the main text for clarity.

Reviewer comment (Novelty):
Can the authors clarify on the connections and similarity of their work compared to [1], [2] and [3]? From my understanding, [1] also fine-tunes the score network to minimize the generation probability of samples that can be labeled as a specific class. Also in [2] and [3] authors propose similar approaches for removing concepts. I believe the claim made in the introduction about this work being the first to study unlearning in diffusion models is incorrect.

Author response:
**Q1.2 Can the authors clarify the connections and similarity of this work compared to [1-3]?**

Thanks for the suggestion. All these methods aim to erase some information carried by the diffusion models.
However, [1] and [3] mainly focus on text-to-image models and high-level visual concept erasure ([3] also focus on debiasing).
Both the method proposed in [2] and ours aim to maximize the distance between the ground-truth backward distribution and the learnable backward distribution, to perform class-wise unlearning and concept erasure for diffusion models.
Thanks, we have since added this in the related work section and removed the statement in the introduction.

Reviewer comment (Experiments):
I believe it would be nice to be able to compare your method against Selective Amnesia [2]. In Table 1 of [2] there is results on CIFAR10 and the FID on the remaining classes seems to be much lower (9.08) than what you have reported in Table 2 (seems to be on average above 20). I would appreciate it if the authors provide more comparisons or clarifications with respect to [2].

Author response:
**Q1.3 Compare against Selective Amnesia [2] and FID score clarifications.**

In the current stage, we follow the same setting in [2] to forget samples belonging to the label '0' on CIFAR-10 and use the well-trained model from [2]. From [2], the FID score on the $\mathcal{C}_r$ is 9.67 for the original model and 9.08 for Selective Amnesia [2].
With an EMA decay of 0.99, our method achieves a FID score of 8.93 at 210 steps, 8.83 at 205 steps, and 8.90 at 200 steps. Generated examples are shown in Figure 20 on page 21 of the revised manuscript.

Regarding the FID score on CIFAR-10, results in Table 2 are reported for DDIM with inference time step $T=100$, and FID is computed over 5K images separately for each class, while results in Table 1 of [2] are reported for DDPM with inference time step $T=1000$ and FID is computed over 5K$\times 9$ images.
For your reference, the FID score computed over 40K images is around 8.18 in Table 2.

Reviewer comment (Presentation):
The texts on Figure 3 are not easily readable. What is the y axis (frequency)? How do we interpret this plot?

Author response:
**Q1.5 The texts on Figure 3 are not easily readable. What is the y axis (frequency)? How do we interpret this plot?**

The y-axis represents the number of occurrences of the specific value.
The diffusion process would add noise to the clean input $x$, consequently, the model output $\epsilon_T$ distribution given these noisy images $x_t$ tends to converge towards the Gaussian noise distribution $\epsilon$.
As shown in Figure 3, we can see that the distance between the output distribution with respect to $\mathcal{C}_r$ and the Gaussian distribution for our scrubbed model is close to that of the unscrubbed model, while the distance between the output distribution with respect to the $\mathcal{C}_f$ is further away from the Gaussian distribution for our scrubbed model than the unscrubbed model, indicating that our algorithm preserves the model utility over $\mathcal{C}_r$ while successfully scrub the information concerning $\mathcal{C}_f$.
Thanks, we have since revised the context for better clarity.