Paper: Machine Unlearning for Image-to-Image Generative Models (ICLR-2024)

Reviewer comment (Novelty):
Abstract: „primarily focused on classification models, leaving the landscape of unlearning for generative models relatively unexplored.“: Is this true? Are there any references which support this statement?

Author response:
We provide the following categories of references to support our claim that *’the landscape of unlearning for generative models is relatively unexplored’*:

- Many recent review/survey papers on machine unlearning. As shown in [1,2,3,4], there are only three papers targeting unlearning for generative models out of hundreds of cited papers in these papers.

- Open-source collections of machine unlearning papers [5]. Out of 254 papers shown in this collection, only eight papers focus on the unlearning for language generation and only 9 papers target the unlearning for image generation. Specifically, for unlearning on image generation models, most papers only explore GANs and none of them addresses the image-to-image generation. In contrast, our approach is generally applicable to GANs, MAE and diffusion models and is the first to unlearn on image-to-image generative models.

Reviewer comment (Experiments):
Intro: „Informally speaking, we define a generative model as having “truly unlearned” an image when it is unable to faithfully reconstruct the original image when provided with only partial information (see Figure 1 for an illustrative example where the partial information involves center cropping2).“: I wonder if you did any analysis if the „noise“ is fully uncorrelated with the part of the image before the unlearning? Maybe there are still some high level corrections left. In learning fair representations people do train secondary models on the representations and try to predict the sensitivity attribute from the fair representations. I wonder if such study can be used to check if a secondary autoencoders could not re-generate the cropped image from the generated noise.

Author response:
We appreciate for this valuable suggestion for a more thorough evaluation of our approach. To this answer this question, we conducted the following experiment for VQ-GAN:

- We first conduct the unlearning for VQ-GAN for ImageNet to obtain the *unlearned model*, under the same setup mentioned in Section 4.1.

- Given the center-cropped input images (cropping central $8\times 8$ patches), we then use this *unlearned model* to generate the images on both forget set and retain set (i.e., image upcropping/inpainting). Here we call them *FIRST* generated images.
- Given these *FIRST* generated images, we only keep the reconstructed central $8\times 8$ patches (cropping the outer ones) as the new input to the original VQ-GAN (i.e., the model before unlearning) and get the newly generated images (here we call them *SECOND* generated images). We then evaluate the quality of *SECOND* generated images for both forget set and retain set and report the results. 
- We also conduct the above process for the *original model* (i.e., before unlearning) as the baseline/reference. The results are given below:

| |FID: $D_R$|FID: $D_F$|IS: $D_R$|IS: $D_F$|CLIP: $D_R$|CLIP: $D_F$|
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
|Original Model|36.14|28.96|27.28|27.95|0.37|0.49|
|Unlearned Model|36.74|156.68|26.62|7.67|0.37|0.28|

> **NOTE:**  Lower FID, higher IS, and higher CLIP values indicate higher image quality.

As shown above, compared to the original model, given the *noise* (i.e., *FIRST* generated images on the forget set) from the unlearned model as the input, the *SECOND* generated images has very low quality in terms of all these three metrics. This means that the *noise* indeed are not correlated with the real forget images.

In contrast, the performance on the retain set is almost the same before and after unlearning. This indicates that our approach indeed preserves the knowledge on the retain set well.

We will add these results and provide some visualizations in the revised paper.

Reviewer comment (Theory):
Equation 3: why is alpha not between 0-1 so you have (1-alpha) x term0 - alpha term1?

Author response:
We would like to point out that these two methods are mathematically equivalent, in principle: $L_R+\alpha L_F$ and $(1-\alpha) L_R+\alpha L_F$. Specifically, we can rewrite $L_R+\alpha L_F$ as: 
 
$L_R+\alpha L_F=(1+\alpha) (\frac{1}{1+\alpha}L_R+ \frac{\alpha}{1+\alpha} L_F) = (1+\alpha) [(1-\beta) L_R+ \beta L_F]$
 
where $\beta=\frac{\alpha}{1+\alpha}$. Therefore, the main difference between $(1+\alpha) [(1-\beta) L_R+ \beta L_F]$ and $(1-\alpha) L_R+\alpha L_F$ is that there’s a scaling factor $(1+\alpha) $ for the former one. In practice, one can use either model based on personal preference.

Reviewer comment (Theory):
Theorem 1: Why is having the same decoder not a problem? Couldn’t the decoder not fully remember the images in some way or could you not extract some of the images by some information extraction techniques?

Author response:
This is a very insightful question. In fact, this kind of information extraction technique is highly correlated to another  important problem called Model Inversion Attacks (MIA) [6,7]. Briefly speaking, MIA tries to generate samples that are very similar to the original training samples. 

Currently, our approach cannot guarantee the robustness to MIA; in fact, even unlearning the entire model may still suffer from the vulnerability of MIA. We remark that like the unlearning of generative models, MIA for generative models, especially for diffusion models, is also an emerging problem [6,7]. We plan to explore the defense methods of the MIA in our future work .
 
Having said that, we also want to highlight a couple of important benefits that result from  focusing on the encoder only vs. unlearning the entire model:

- Efficiency: training the encoder only is much more efficient (and requires much less computational capability) compared to training the entire model. 

- Universal applicability. Unlearning the encoder enables us to propose a universal MSE-based loss instead of designing the different loss for distinct types of generative models.

Reviewer comment (Theory):
The main concern is that the paper assumption is ideal and its hyper parameter \sigma, which may not hold in practice.

Author response:
**Response** Regarding the assumptions used in our paper, we tried our best in making them align with practice. Specifically, one of the main assumptions is that the original model can do a perfect reconstruction for the cropped images. This assumption is somehow ideal, yet practically reasonable, especially given the recent progress in diffusion models [1,2]. We also note that since our work is the first to target unlearning for image-to-image generative models, we hope it will pave the way for follow-up works that will relax some of our assumptions.  The theoretical results we provide offer a solid basis for such future extensions.  

We also agree with Reviewer that, ideally, we should use the exact $\Sigma$ of the images from the forget set. However, there are some computational barriers to using the exact $\Sigma$ for a high-resolution image dataset. Specifically, consider a commonly used 256$\times$256 resolution for image generation tasks, the distribution of the generated images will have $256\times 256\times 3 \approx 2\times 10^5$ dimensions. The size of the covariance matrix $\Sigma$ for such a high-dimensional distribution is around $(2\times 10^5)^2=4\times 10^{10}$, which requires around 144GB memory if stored in float precision thus is not practical.


Consequently, to address the computational barrier of $\Sigma$, we use some approximated methods derived from some empirical observations on some small datasets.
Specifically, we compute the exact $\Sigma$ for some small-scale image dataset, including MNIST and CIFAR10/100.

- *Off-diagonal elements*: as shown in Figure C.6 at Appendix C.4, the values on the diagonal of exact $\Sigma$ for these datasets are generally much larger than the off-diagonal elements; hence, we truncate the off-diagonal elements to zero. 
- *Diagonal elements*: since the input images for deep networks are typically normalized, we may set the diagonal elements to one. 

In short, using $I$ to approximate $\Sigma$ is a practical approximation alternative due to the extremely high computational costs of $\Sigma$ for high-resolution images. 
For future work, given our theoretical analysis, we believe that our approach will achieve better results (lower image quality) on forget set if we can find a way to use the exact $\Sigma$. Hence, we plan to explore the potential to reduce the computation of exact $\Sigma$ with low-rank approximation thus enabling the use of more accurate data-driven $\Sigma$.

We will add the above clarifications to the revised paper. We hope our response can address your concerns. We look forward to further discussions if you have any other questions. 

[1] Song, Bowen, et al. "Solving inverse problems with latent diffusion models via hard data consistency." arXiv preprint arXiv:2307.08123 (2023).

[2] Wallace, Bram, Akash Gokul, and Nikhil Naik. "Edict: Exact diffusion inversion via coupled transformations." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Reviewer comment (Theory):
From Lemma 1, $\sigma$ should be from the forget set? In the appendix C.4, however it sets $\sigma=I$, which is not estimated from forget set?

Author response:
**Response** Regarding the assumptions used in our paper, we tried our best in making them align with practice. Specifically, one of the main assumptions is that the original model can do a perfect reconstruction for the cropped images. This assumption is somehow ideal, yet practically reasonable, especially given the recent progress in diffusion models [1,2]. We also note that since our work is the first to target unlearning for image-to-image generative models, we hope it will pave the way for follow-up works that will relax some of our assumptions.  The theoretical results we provide offer a solid basis for such future extensions.  

We also agree with Reviewer that, ideally, we should use the exact $\Sigma$ of the images from the forget set. However, there are some computational barriers to using the exact $\Sigma$ for a high-resolution image dataset. Specifically, consider a commonly used 256$\times$256 resolution for image generation tasks, the distribution of the generated images will have $256\times 256\times 3 \approx 2\times 10^5$ dimensions. The size of the covariance matrix $\Sigma$ for such a high-dimensional distribution is around $(2\times 10^5)^2=4\times 10^{10}$, which requires around 144GB memory if stored in float precision thus is not practical.


Consequently, to address the computational barrier of $\Sigma$, we use some approximated methods derived from some empirical observations on some small datasets.
Specifically, we compute the exact $\Sigma$ for some small-scale image dataset, including MNIST and CIFAR10/100.

- *Off-diagonal elements*: as shown in Figure C.6 at Appendix C.4, the values on the diagonal of exact $\Sigma$ for these datasets are generally much larger than the off-diagonal elements; hence, we truncate the off-diagonal elements to zero. 
- *Diagonal elements*: since the input images for deep networks are typically normalized, we may set the diagonal elements to one. 

In short, using $I$ to approximate $\Sigma$ is a practical approximation alternative due to the extremely high computational costs of $\Sigma$ for high-resolution images. 
For future work, given our theoretical analysis, we believe that our approach will achieve better results (lower image quality) on forget set if we can find a way to use the exact $\Sigma$. Hence, we plan to explore the potential to reduce the computation of exact $\Sigma$ with low-rank approximation thus enabling the use of more accurate data-driven $\Sigma$.

We will add the above clarifications to the revised paper. We hope our response can address your concerns. We look forward to further discussions if you have any other questions. 

[1] Song, Bowen, et al. "Solving inverse problems with latent diffusion models via hard data consistency." arXiv preprint arXiv:2307.08123 (2023).

[2] Wallace, Bram, Akash Gokul, and Nikhil Naik. "Edict: Exact diffusion inversion via coupled transformations." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Reviewer comment (Experiments):
It's unclear if the machine unlearning setup considered in this work is practical. For machine unlearning setups, the gold standard to mimic is a model trained only on the retain set. In this work, the approach performs this task by minimizing the embedding distance for the forget set to normal distribution. Does this lead to overall worse performance than the gold standard model (where only retain set is used to train from scratch)? It would be good to include this as a baseline for comparison.

Author response:
This is a very fundamental question, and we understand the reviewer’s concern to the definition/setup of our unlearning approach. Indeed, we did ask the same questions ourselves. First, we agree that most of previous works consider the training only on the retain set is the gold baseline for *classifications tasks*. However, recently it has been shown that this golden standard bear several concerns:

- Some recent works show that for *‘some unlearning algorithms can generate models indistinguishable from the golden baseline beyond any arbitrarily small threshold while still exposing the deleted data’* for some cases [1]. In other words, the indistinguishability of training on retain set only cannot guarantee the data has been totally removed from the model, which is the goal in our paper.


- Retraining these generative models from scratch on retain set is completely beyond our computation capacity. For the three models evaluated in our paper, it takes more than 10K GPU hours ($>$50 days on our 8-GPU server; estimated from the experimental setup in [2,3,4])  to train the model from scratch. 

- Given these considerations, our setup/definition primarily focuses on the practical scenarios other than the indistinguishability from the retrained model. For example, if the training set for the original model has some pornographic images by accident, our methods can make the unlearned model avoid generating pornographic images in a very efficient way other than expensive retraining the model from scratch. This way, the model's developers can avoid the potential violation of contents control. In short, our main goal is making the unlearned model not generate the samples from the forget set without hurting the performance on the retain set. Under this perspective, the performance drops on the forget set shown in our results is expected and dedicatedly designed. Moreover, the results show that on retain set, our approach has negligible performance drop compared to the model before unlearning. 

As for the performance comparison with the golden standard model, our method will have worse performance **on the forget set** than this golden baseline (the performance on the retain set is expected to be similar to the golden standard model), which is dedicatedly designed by our approach. 

Finally, we remark that the unlearning for generative models is still in its emerging stage; we do hope our works could contribute to a better setup/definition for this problem space. 



[1] Chourasia, Rishav, and Neil Shah. "Forget Unlearning: Towards True Data-Deletion in Machine Learning." ICML 2023. https://icml.cc/virtual/2023/poster/23753


[2] Li, Tianhong, et al. "Mage: Masked generative encoder to unify representation learning and image synthesis." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[3] He, Kaiming, et al. "Masked autoencoders are scalable vision learners." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

[4] Saharia, Chitwan, et al. "Palette: Image-to-image diffusion models." ACM SIGGRAPH 2022 Conference Proceedings. 2022.

Reviewer comment (Experiments):
The experimental setup for retain sample availability is not quite clear. Why are retain samples selected from the remaining 800 random classes? Also, it is confusing whether all the 1000 classes, or only 200 classes, are used for the main experiments to train the original model.

Author response:
First, out of 1000 classes from ImageNet-1K, our experimental setup selects 100 classes as the retain set $D_R$ and another 100 classes as the forget set $D_F$; the remaining 800 classes $D_O$ are not considered either as retain or forget. We evaluate our methods with two different setups:


- Use the images from exact 100 classes from $D_R$ to do the unlearning. This is what we called *real retain samples*.

- Suppose we don’t have access to the images from 100 classes from $D_R$, we use images from the other 800 classes $D_O$ to serve as the proxy retain set.

For both cases, the images from the exact classes of $D_F$ is needed. Moreover, the performance/test on the retain set is always conducted on the 100 classes from $D_R$. 


As for the original model, we download the open-source models pre-trained on ImageNet-1K as our starting point. We then fine-tune these models on these selected classes; this is the *original model* we used in this paper. 



Thanks again and we would be happy to provide more clarifications and answer any follow-up questions.

Reviewer comment (Experiments):
The study does not align with the foundational concept of machine unlearning, which typically necessitates a comparison between the unlearned and retrained models as per references [1-7]. Although the authors justify this divergence due to the high costs associated with retraining generative models, this deviates from the core goal of machine unlearning aimed at addressing privacy concerns. Particularly, the approach presented in this paper generates conspicuous Gaussian noise over the 'forgotten' data, which may inadvertently signal that the data was previously part of the training set, contradicting privacy preservation goals. A more compelling motivation might be found in text-to-image (T2I) scenarios [8] where the goal is to prevent the generation of inappropriate content, or in image-to-image (I2I) applications [9] that showcase practical utility. It would be more meaningful—and privacy-compliant—if the model could reconstruct unremarkable images that don't trace back to the original training data, rather than reconstruct images with evident distortions signaling prior data use.

Author response:
**Response 1** This is a fundamental question, and we understand the reviewer's concern about the definition/setup of our unlearning approach. Indeed, we did ask the same questions ourselves. First, for *classifications tasks*, we agree that most of previous work considers that the unlearned model should be indistinguishable to the model retrained only on the retain set (golden baseline). However, there are three points we want to bring to this reviewer’s attention: 


- First, even the indistinguishability to the golden standard may have some issues. Indeed, recent work has shown that *‘some unlearning algorithms can generate models indistinguishable from the golden baseline beyond any arbitrarily small threshold while still exposing the deleted data’*[10]. In other words, the indistinguishability of training on the retain set alone *cannot* guarantee the information is indeed removed from the model. 

- Second, retraining these generative models from scratch on the retain set is beyond our (and average user, in general) computational capabilities. More precisely, for the three models evaluated in our paper, it would take more than 10K GPU hours ($>$50 days on our 8-GPU server; estimated from the experimental setup in [11,12,13]) to train the model from scratch; this is clearly infeasible given the resources we have at hand and the tight deadline for this rebuttal period. 



- Given these considerations, our setup/definition primarily focuses on the practical scenarios of copyright protection and/or pornographic/violence content control, other than the indistinguishability from the retrained model. For example, if the original training set for the original model got some pornographic images by accident, our method can make the unlearned model not generate those pornographic images in a very efficient way (as opposed to expensively retraining the model from scratch). Moreover, the results show that on the retain set, our approach has negligible performance drop compared to the model before unlearning. Consequently, we believe our approach is sound for copyright protection and pornographic/violence content control hence very useful in real applications. 

**Response 2**:  This is a very insightful comment (*the approach presented in this paper generates conspicuous Gaussian noise over the 'forgotten' data, which may inadvertently signal that the data was previously part of the training set, contradicting privacy preservation goals.*). We note that, under our setup, given an image with noise distortions, one cannot conclude whether it is in the training set or not. For example, suppose our method makes the model forget the concept *car*; given any image of a car, the corresponding generated image typically will have noise distortions, but it is possible that this specific image is not used for training. Even though one can infer that some concepts are from the forget set, that still meets our main goal, i.e., the unlearned model does not generate the unwanted images.

Reviewer comment (Evaluation):
The evaluation process presented in the paper is incomplete with regard to established machine unlearning protocols. Typically, an unlearned model's performance is assessed using three distinct datasets: a test dataset to determine its generalization capability, a retained dataset to evaluate performance on non-forgotten data, and a forget dataset to check the efficacy of the unlearning process. The paper's Table 1 appears to only present results for the latter two, omitting the crucial evaluation on the general test dataset.

Author response:
**Response 4**: This is a very important question. We would like to clarify that we *actually performed* such a test (the first evaluation in the reviewer's question) in our paper, but simply described it in a slightly different way. 

Specifically, out of 1000 classes from ImageNet-1K, our experimental setup selects 100 classes as the retain set $D_R$ and another 100 classes as the forget set $D_F$; the remaining 800 classes $D_O$ are not considered either as retain or forget sets. We evaluate our method with the following setup: In Table 1 and  Table 2 (rows *Proxy* $D_R$) of our paper, we assume that we don’t have access to the images from 100 classes from $D_R$, and then we use images from the other 800 classes $D_O$ to serve as the proxy retain set for our unlearning approach. Moreover, the performance/test on the retain set is always conducted on the 100 classes from original $D_R$. 

In fact, for this setup, we can re-consider the original $D_R$ as the *general test set* and re-consider the $D_O$ as the new $D_R$. As shown in Table 1 and Table 2 (rows *Proxy* $D_R$), our approach is evaluated on the 100 classes from *general test set* and it shows negligible performance drop on the general test set.

Reviewer comment (Writing):
The scope of the study with respect to the application of machine unlearning in image-to-image (I2I) generative models appears to be inaccurately broad. The term "machine unlearning for I2I generative models" suggests a wide range of applications; however, the paper primarily focuses on the image inpainting task. It would be more precise to either expand the variety of I2I applications examined in the study or to specifically define the scope as "machine unlearning for image inpainting tasks" to reflect the content more accurately. This would ensure clarity in the paper's contributions and avoid overgeneralization of the results.

Author response:
**Response 5**: We appreciate this reviewer's suggestion. We will modify our title to 'Machine Unlearning for Image Completion Generative Tasks'.