Paper: Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR-2024)

Reviewer comment (Theory):
The paper assumes that all prompts can be modified or that users will be willing to modify them. In real-world scenarios, some prompts might be non-negotiable, and changing them might not be an option.

Author response:
We acknowledge your concerns regarding the potential complexity of modifying prompts for some users. However, prompt interaction might be a potential feature. A notable example of this is OpenAI's DALL-E 3, which allows users to iteratively refine their prompts to achieve the desired results. This advanced level of user interaction illustrates both the practicality and the growing trend of user familiarity with such interactive processes. We anticipate that, in the future, users will become increasingly adept at interacting with the models.

Addressing memorization is crucial for users. If the model's outputs contain copyrighted images, users who further utilize these images might also encounter legal issues. Additionally, for memorized prompts, the generated content is often limited to the memorized images. Therefore, modifying the prompt is beneficial for the users to achieve more diverse generations.

Furthermore, for situations where users find it cumbersome to modify prompts or encounter hard-to-alter tokens, the automatic inference-time mitigation method proposed by our paper could be an alternative. This option provides flexibility, catering to different user preferences and capabilities.

Overall, our proposed method offers a new perspective on designing user interfaces and addressing the issue of memorization. We believe that our approach contributes valuable insights for future developments in the field.

Reviewer comment (Evaluation):
While the paper suggests that the method is computationally efficient, implementing the strategies during the training and inference phases might still introduce computational or operational overheads for model owners.

Author response:
We agree that there is a small computational overhead for model owners. Specifically, as mentioned in our paper, our detection method does not require additional computation since the text-conditional noise prediction is calculated during inference. Meanwhile, the inference-time mitigation incurs around 6 seconds for optimization, and a 10% increase in time for training-time mitigation. These costs are negligible compared to potential lawsuits due to memorization. Also, we believe that there is no free lunch to mitigate memorization. Therefore, a minor cost to achieve effective memorization reduction is acceptable.

Reviewer comment (Evaluation):
Is there any way to quantify memorization in the diffusion models that the future method could use to benchmark? It might be good to have a discussion in this direction.

Author response:
Yes, in our work, we adopt the definition of memorization from previous research [1], where we measure the SSCD similarity between training and generated images using the memorized prompts dataset from [2].

However, we acknowledge the potential for developing a more comprehensive benchmark in the future. For instance, categorizing memorization into different types of legal copyright infringement could offer more practical insights. This approach would allow for a comparative analysis of methods under various memorization categories. Although establishing such a benchmark may be labor-intensive, we believe it is crucial and advantageous for advancing the development of reliable generative AI.

Reviewer comment (Evaluation):
Are certain tokens more susceptible to triggering memorization than others? How were these trigger tokens identified, and is there a taxonomy or classification for them?

Author response:
This problem was well studied by [1]. The susceptiblility of triggering memorization really depends on the repetition of a certain word associated with the same image in the training data. However, if under the same repetition rate, [1] found in generating more specific captions are easier to memorize due to their uniqueness.

Reviewer comment (Experiments):
Were there any adversarial tests done to ascertain if an attacker could still exploit the memorization tendencies, even after applying the proposed mitigation strategies?

Author response:
We have yet to discover any adversarial methods capable of evading our proposed detection and mitigation strategy. One trial we tried was to utilize PEZ[3] to optimize a new prompt with the CLIP model. However, we observed that significant text-conditional noise prediction occurs whenever memorization is involved, affirming the effectiveness of our detection and mitigation approaches.

Reviewer comment (Evaluation):
How is the metric computed with multiple generations in Table 1?

Author response:
For each prompt, the final metric is calculated as an average across all generations.

Reviewer comment (Evaluation):
In Table 1, what does the column First 10 steps indicate, is it the AUC, TPR values calculated with the average of metric values for the first 10 steps of the diffusion process?

Author response:
Your interpretation is correct. It is calculated based on the average metric obtained during the initial 10 steps of the denoising process.

Reviewer comment (Reproducibility):
What is meant by the following sentence in the "An effective inference-time mitigation method" — "Thus, a perturbed prompt embedding e* is obtained as t=0 by minimizing Eq (5)" — Is this minimization done via gradient descent, what is the data on which this minimization is performed?

Author response:
We initiate the process with the original prompt embedding and employ gradient descent using the Adam optimizer. The objective function for this optimization is detailed in Equation 5 of our paper. It is crucial to note that this optimization process does not involve any data and is conducted independently for different prompts.

Reviewer comment (Experiments):
Was this done on the 200 LAION data points? If yes, what is Figure 4(a) and 4(b) plotted over?

Author response:
Regarding Figures 4(a) and 4(b), the results were derived from an evaluation of 5 x 200 data points from the LAION dataset, with '5' representing five different random seeds. The detailed experiment setting can be found in section 4.4.

Reviewer comment (Experiments):
In terms of the experimental setting, I do believe that performing experiments to see how the memorization ratio changes with repetitions in the data set might be a great way to further solidify if the method works. In particular, this could follow directly from the setup of Somepalli et al.

Author response:
We are grateful for your suggestion to investigate the memorization ratio with varying data repetitions. We have conducted additional experiments with different data duplication times: 50, 100, 150, 200, 250, and 300 times. The results with the SSCD similarity scores before and after applying the inference-time mitigation are presented in the table below. The proposed method is effective in all situtations with the same target loss for mitigation. We believe this is a good add-on for the future version.

| duplication times |   50  |  100  |  150  |  200  |  250  |  300  |
|:---------------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|
|  No Mitigation  | 0.544 | 0.556 | 0.559 | 0.561 | 0.562 | 0.570 |
| With Mitigation | 0.328 | 0.341 | 0.342 | 0.350 | 0.351 | 0.367 |

Reviewer comment (Evaluation):
Most pertinently, I see that the mitigation strategies lead to a significant drop in CLIP score. In particular, if you were to look at the region on the plot between the model initialization before fine-tuning and the final fine-tuned model, it is evident that, especially when you contrast with Figure B where all the points are between 0.29 and 0.3, the inference time mitigation leads to a significant drop in CLIP score. It is unclear if this method is actually useful in that regard. It suggests that we are unable to reach the same performance as that of a model that was never fine-tuned. I am curious what the authors feel about this particular observation. In particular, a model that was not fine-tuned had a higher CLIP score on the prompts, but the method using mitigation achieved a much lower CLIP score. I am not able to position these results with the overall setup.

Author response:
We recognize your concerns regarding the observed decrease in CLIP score to the inference-time mitigation. This issue primarily arises from the model's significant overfitting to memorized data points, making it challenging to achieve high CLIP scores during inference without additional model retraining or fine-tuning. However, we want to emphasize that in practice, the model owner can deploy our proposed detection (section 3.3) and mitigation (section 4.2) strategy at the same time. Therefore, the mitigation will be only applied to memorized prompts, and, in non-memorization cases, the model performance will not be impaired.

Meanwhile, we believe that a slight drop in CLIP score is still beneficial to the company when compared to the issues of copyright infringement or privacy violations due to outputting memorized training data.

We hope our response can solve your concern regarding our paper. Please let us know if you have any more questions.