Paper: On Memorization in Diffusion Models (ICLR-2024)

Reviewer comment (Novelty):
The scientific contribution of this submission is limited. The problem of memorisation in diffusion models was already noticed in several works (as mentioned in the related works section). The observation that in the theoretical optimum of diffusion models they can only replicate training data is new, but it is quite expected since the simplified training objective of DDPMs [Ho et al. 2020] is to directly denoise all training examples with a simple MSE loss. The main contribution of this work is therefore, a set of experiments that measure the memorization across different model sizes, widths, dataset sizes etc. Except for one experiment described in the strengths sections the results are rather intuitive and expected (e.g. diffusion models memorize more examples from smaller datasets, or when trained with wider models), and are presented in a form of report without in-depth analysis of the root-causes of memorization. The interesting hypothesis is proposed only for the analysis of the class conditioning influence, but it is denied right away in the next paragraph.

Author response:
We would like to clarify the primary objective of our work: investigating under what conditions the diffusion models (undesirably) approximate the optimal solution represented by Eq.(2). On the one hand, such an optimal model can only replicate training data, and may even lead to privacy/copyright issues. On the other hand, such an optimal model is indeed what is theoretically expected. Therefore, it becomes imperative to gauge the extent of this theoretical hazard in typical diffusion model training scenarios. This awareness is vital for mitigating adverse consequences and refining the practical utility of these models, thus requiring quantitative studies like our work. 

Furthermore, our research offers extensive and empirical guidances for understanding the memorization in diffusion models and training diffusion models while preventing large memorization. Our findings aim to delineate which factors have a substantial impact on memorization and which contribute more subtly. As such, our paper primarily presents an empirical analysis, rather than a theoretical one, of the interrelationships between these factors. In addition, our study reveals surprising findings, e.g. the significant effects of random labels, which may inspire the theoretical practitioners for further exploration.

Reviewer comment (Experiments):
The evaluation is performed using only one dataset (CIFAR10). It would be interesting how diffusion models memorize more detailed dataset e.g CelebA.

Author response:
Thank you for your valuable suggestions. In addition to CIFAR-10, we have conducted a series of additional experiments using the FFHQ dataset [1], which is a higher-dimensional face dataset similar to CelebA. These new experiments have been included in **Appendix D: More empirical results on FFHQ** of our revised paper. Due to the time constraints, our additional experiments focused on investigating the impact of data dimension / time embeddings / conditioning, on the memorization of diffusion models. It is noteworthy that the outcomes of these recent experiments support our initial findings derived from the CIFAR-10 dataset.  

Dear Reviewer gEHa,

Sorry for bothering you, but the discussion period is coming to an end soon. Could you please let us know if our responses and additional experiments have alleviated your concerns? Besides the experiments on FFHQ added in our initial rebuttal, we have included new experiments on Artbench-10 [b] in scenarios of fine-tuning Stable Diffusion [a]. We believe that these additional experiments on the two new datasets will significantly strengthen our work. If there are any further comments, we will do our best to respond.

---

**References:**

[a] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.\
[b] Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The artbench dataset: Benchmarking generative models with artworks. arXiv preprint arXiv:2206.11404, 2022.

Reviewer comment (Evaluation):
What is the statistical significance of all of the experiments? In some plots, there is small difference between different setups, it is unclear if it should be taken into consideration.

Author response:
In our experiments, we noted that the variance in memorization ratios across repeated trials is negligible. For instance, as depicted in Figure 2(b), with the training data size of $|\mathcal{D}|=1$k and two classes ($C=2$), the memorization ratio for the trained diffusion model stands at $94.59\pm0.19$\% over three different random seeds. Similarly, for $C=5$, the memorization ratio is $92.32\pm0.14$\%.

Throughout our research, we have trained several hundred diffusion models, necessitating substantial GPU hours. Consequently, it is impractical to execute each experiment with varying random seeds. Given that the observed variance in memorization ratios is minor (less than $0.2$%), we postulate that this minimal fluctuation is unlikely to impact the statistical significance or alter the overarching conclusions of our study.

Reviewer comment (Experiments):
What was the performance of the model when trained with large weight-decay values? What is the trade-off between the quality of samples and memorization?

Author response:
We note that the model performance also deteriorates when large weight decay is applied during the training process.

In reference to the tradeoff you mentioned, here we make more clarifications regarding the relationship between the quality of generated samples and memorization. When a significant proportion of generated samples are replicas of the training data, their image quality is inherently high. Additionally, this leads to a generation distribution close to that of the training data, resulting in a low FID score. The FID score is conventionally used to evaluate the quality and diversity of generated images. For instance, the optimal diffusion model, which can only replicate training data (thus exhibiting a memorization ratio is 100%), achieves an FID score of 0.56. This is substantially compared to the 1.96 FID score attained by the state-of-the-art unconditional diffusion model, EDM. 

Given that the objective of our research is to explore when diffusion models memorize in terms of our definitions of EMM for a training recipe, we have consistently employed a memorization metric throughout our study.

Reviewer comment (Novelty):
Lack of discussion of SOTA text-to-image diffusion models, e.g., stable diffusion.

Author response:
***Weakness: Lack of experiments on stable diffusion.***

We would like to clarify the primary objective of our work: investigating under what conditions the diffusion models (undesirably) approximate the optimal solution represented by Eq.(2). On the one hand, such an optimal model can only replicate training data, and may even lead to privacy/copyright issues. On the other hand, such an optimal model is indeed what is theoretically expected. Therefore, it becomes imperative to gauge the extent of this theoretical hazard in typical diffusion model training scenarios. This awareness is vital for mitigating adverse consequences and refining the practical utility of these models, thus requiring quantitative studies like our work.

To advance this research, we have trained hundreds of EDMs from scratch. However, it is computationally intractable and not feasible to similarly train such a huge number of stable diffusion models. Notably, text-to-image diffusion models, which also adhere to the objective of denoising score matching, possess the similar theoretical optimum and are likewise expected to exhibit substantial memorization of training data when meeting EMM. As updated in **Appendix D: More empirical results on FFHQ** and **Appendix E: Discussions on memorization criteria** of the revised paper, our new investigations reveal consistent findings when changing the dataset to FFHQ [1] or altering the memorization ratio to KNN distance. Consequently, we conjecture that similar memorization behaviors are likely observable in state-of-the-art text-to-image diffusion models. 

---

Thank you for your feedback.

We have extended to fine-tuning stable diffusion [a] on Artbench-10 [b], a high dimensional art dataset with image resolution of $256\times256$. We have explored the effects of training data size and conditioning on memorization. The results have been included in **Appendix F: More empirical results on stable diffusion** in our revised paper. We note that the observations align with our previous experiments on CIFAR-10 and FFHQ.

Please kindly let us know if there is any further concern, and we will do our best to respond.

---
**References:**

[a] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.\
[b] Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The artbench dataset: Benchmarking generative models with artworks. arXiv preprint arXiv:2206.11404, 2022.

Reviewer comment (Evaluation):
I appreciate the thorough empirical study. I am curious is there any memorization mitigation strategy given the empirical results?

Author response:
***Question: Any stategies to mitigate memorization?***

Thank you for your inquiry. Our experiments offer empirical and quantitative guidance for practitioners aiming to train their diffusion models from scratch while preventing large memorization. Generally, it is advisable to select a training recipe, encompassing aspects of data distribution, model configuration, training procedure, conditioning that exhibits a lower EMM when other performance metrics are comparably maintained. Opting for such a training recipe is advantageous as it has lower risk of memorizaton.

---

In response to the inquiry on mitigation strategies, from our empirical analysis, we suggest to adopt random fourier features as time embeddings in the architecture of DDPM++ or incoporate weight decay or smaller batch size during the training as these strategies typically have smaller EMMs. 

Please kindly let us know if there is any further concern, and we will do our best to respond.

Reviewer comment (Experiments):
The work focuses on a simple toy setup using a subset of CIFAR-10. While such simple setups are useful for analysis, presented in this work it does leave a taste for more. It would be good to ablate setups that plague large datasets, such as dataset duplication which was discussed to be a cause for memorization in diffusion models [3, 4, 5].

Author response:
***Weakness 1 and 2: Lack of experiments or analysis on other datasets.***

We have also conducted a series of new experiments on the FFHQ dataset [6], which is a higher-dimensional face dataset. These experiments have been meticulously updated in **Appendix D** of the updated paper, titled "**More empirical results on FFHQ**". Given the time limit, the investigation primarily focused on revisting the effects of data dimension / time embeddings / conditioning, on the memorization of diffusion models. It is noteworthy that the outcomes of these recent experiments corroborate the findings previously observed on the CIFAR-10 dataset.

Our objective is to determine the specific training size when diffusion models demonstrate similar memorization behaviors as the theoretical optimum. This specific training size is defined as EMM in our paper. However, dataset duplication will cause the ambiguity in this definition.

Reviewer comment (Experiments):
I also expected to see at least a few of these analyses on another simple dataset such as SVHN or CIFAR-100.

Author response:
***Weakness 1 and 2: Lack of experiments or analysis on other datasets.***

We have also conducted a series of new experiments on the FFHQ dataset [6], which is a higher-dimensional face dataset. These experiments have been meticulously updated in **Appendix D** of the updated paper, titled "**More empirical results on FFHQ**". Given the time limit, the investigation primarily focused on revisting the effects of data dimension / time embeddings / conditioning, on the memorization of diffusion models. It is noteworthy that the outcomes of these recent experiments corroborate the findings previously observed on the CIFAR-10 dataset.

Our objective is to determine the specific training size when diffusion models demonstrate similar memorization behaviors as the theoretical optimum. This specific training size is defined as EMM in our paper. However, dataset duplication will cause the ambiguity in this definition.

Reviewer comment (Evaluation):
The results don't discuss other relevant metrics, such as quality of generations or loss convergence. For example, high weight decay in this work is shown to have a large effect on memorization but it isn't discussed how much it comes at the cost of quality of generations.

Author response:
***Weakness 3: Lack of analysis on other metrics related to generation. For example, the effects of weight decay on quality metrics.***

We would like to clarify the primary objective of our work: investigating under what conditions the diffusion models (undesirably) approximate the optimal solution represented by Eq.(2). On the one hand, such an optimal model can only replicate training data, and may even lead to privacy/copyright issues. On the other hand, such an optimal model is indeed what is theoretically expected. Therefore, it becomes imperative to gauge the extent of this theoretical hazard in typical diffusion model training scenarios. This awareness is vital for mitigating adverse consequences and refining the practical utility of these models, thus requiring quantitative studies like our work. Consequently, the memorization metric was the focal point of our investigation. 

FID score is conventionally employed for assessing the quality and diversity of generations. However, in the context of our research, a low FID score is expected when diffusion models extensively memorize training data. This is because a large amount of training data replicas in generated samples would naturally result in a generation distribution close to the training distribution. Furthermore, it is anticipated that image quality metrics would also show enhanced performance, given that replicas of training data are typically of high quality. 

It is crucial to note that the introduction of weight decay greater than zero alters the training objective, leading to a divergence from the original theoretical optimum, which can only replicate training data. This divergence becomes apparent for large weight decay. However, as aforementioned, the primary focus of this study is not on these other metrics but rather on the memorization aspect.

Reviewer comment (Novelty):
Several results presented in this paper, especially regarding dataset and model complexity, are generally expected based on previous work on other generative and discriminative models [1, 2].

Author response:
***Weakness 4: “Several results presented in this paper, especially regarding dataset and model complexity …”***

The relationship between learning outcomes and the complexity of data and models represent a topic of enduring interest within the machine learning community. Nevertheless, the literature has not adequately elucidated the relationship between the memorization of diffusion models and various influencing factors. Our study aims to address this gap via a thorough analysis, examining the role of data distribution, model configuration, training procedure, and conditioning. 

Moreover, the motivations underpinning our research diverge significantly from those in previous studies, particularly those outlined in references [1] and [2]. While [1] delved into memorization within GANs, and [2] investigated similar phenomena in discriminative models, our focus is distinctly oriented towards diffusion models. Unlike GANs and discriminative models, which possess infinite optimal solutions, diffusion models are characterized by a closed-form solution that exclusively memorizes training data without generalization. This distinct attribute propels our inquiry into the memorization discrepancies between the trained diffusion model and its theoretical optimal solution. In contrast to [1], which concentrated on dataset size and complexity, our experimental framework extends to encompass the effects of data distribution, model configuration, training procedure, and conditioning.

Reviewer comment (Evaluation):
Results regarding weight decay and EMA aren't very informative. Is the model convergence much worse when weight decay is set high? I would suggest discussing this in more detail.

Author response:
***Question 2: More explanations on experimental results regarding EMA and weight decay.***

The primary aim of our study is to systematically analyze the influence of various factors—namely, data distribution, model configuration, training procedure, and conditioning on the value of EMM. Our exploration on EMA is motivated by its substantial impact on the FID score and overall image quality. This observation prompted an investigation into whether it similarly exerts a considerable effect on memorization. Our empirical findings reveal that while EMA substantially influences image quality, its effect on memorization is marginal, a conclusion that is not trivial.

Our exploration on weight decay is motvated by that the training objective of diffusion models is altered when introducing weight decay. Consequently, the optimal solution is also different from Eq. 2 of our main paper. This alteration in the training objective raises the question of how weight decay affects the deviation of trained diffusion models from the optimal solution. Our experiments have shown that with a small weight decay, diffusion models are still capable of demonstrating memorization behaviors.

Reviewer comment (Experiments):
It would be interesting to show how noise schedule in diffusion models impacts memorization.

Author response:
***Question 3: New explorations on noise schedule.***

Thank you for your suggestion. We have prioritized experiments involving the FFHQ dataset. Nonetheless, if time allows, we will endeavor to incorporate your suggestion regarding the investigation of noise schedules in diffusion models and include the results in the revised version of our paper.

Reviewer comment (Presentation):
Skip connection results figures could be better. In Figure 4a & 4c some markers are too close to understand. While the main observation comes out clearly, the results about the number of skip connections are hard to parse from the figure.

Author response:
***Minor question 1: Better demonstrations/visualizations regarding skip connection results.***

Your feedback is highly appreciated. We have amended the figures in our revised paper to enhance clarity and better illustrate the results.

Reviewer comment (Writing):
The memorization criterion can be easily explained in words. I had to look up the referenced paper, as the notation using $j$-th closest sample was taking a while to parse.

Author response:
***Minor question 2: Better word descriptions on memorization ratio.***

Thank you for your suggestion. We have incorporated detailed word descriptions on the memorization metric we used into the revised version of our paper.

Reviewer comment (Novelty):
There is no detailed comparison with related work in these areas, despite the fact that the effect of various factors on memorization in diffusion models has been discussed in the literature before.

Author response:
Thank you for the question. First, the foundational motivations of our research diverge significantly from those of studies [1], [2], [3]. We aim to investigate under what conditions the diffusion models (undesirably) approximate the optimal solution represented by Eq.(2). On the one hand, such an optimal model can only replicate training data, and may even lead to privacy/copyright issues. On the other hand, such an optimal model is indeed what is theoretically expected. Therefore, it becomes imperative to gauge the extent of this theoretical hazard in typical diffusion model training scenarios. This awareness is vital for mitigating adverse consequences and refining the practical utility of these models, thus requiring quantitative studies like our work.

Secondly, our paper conducted a comprehensive and quantitative examination of various factors influencing the memorization of diffusion models. These factors span a wide range, including data distribution, model configuration, training configuration, and conditioning. In contrast, [1] and [3] primarily focused on demonstrating that diffusion models may replicate training data and proposing frameworks for detecting or extracting such replication. [2] investigated the impact of various factors on the memorization in text-to-image diffusion models, particularly those fine-tuned on new datasets. Compared to [2], our experiments predominantly engage with unconditional diffusion models trained from scratch, and the variables we examine differ from those in [2]. Our findings offer new insights, e.g. time embedding / random conditions / skip connections, into how these factors affect memorization in diffuion models, as detailed in our paper.

We wound also like to make more clarifications on how our research different from [1] regarding dataset size and [2] regarding text conditioning and dataset complexity. 

- In [1], the authors conducted a comparative analysis of the memorization tendencies in diffusion models trained on datasets of varying sizes (specifically, 300 versus 3,000 samples). In contrast, in our research, we showed when diffusion models memorize in terms of a novel metric EMM. This specific training data size reflects the capacity of model and algorithm, etc, and discloses the interactions among different factors. Additionally, we monitored the memorization ratios throughout the training procedure and showed that the memorization becomes apparent after a sufficiently extended training duration, particularly when the size of the training data is sufficiently small. 

- The authors in [2] undertook a comparative analysis of diffusion models conditioned on various types of captions. Our research, however, diverges in the notable discovery that random conditions can effectively induce the memorization of class-conditioned diffusion models. We also find that the number of classes plays an important role in the memorization. In terms of dataset complexity, [2] compared models trained on LAION-10k and Imagenette datasets, attributing the higher memorization observed in the latter to the structural complexity of its images. Here the dataset complexity is assessed on an instance-level basis. While in our experiments, we meticulously constructed a series of training datasets, each varying in the number of classes or the intra-class diversity, while keeping the other factor constant. In our research, the number of classes and intra-class diversity serve as population-level metrics to assess the dataset complexity.

Finally, we introduced a novel metric for memorization: EMM. This metric is designed to determine the conditions under which trained diffusion models exhibit memorization behaviors akin to those of the optimal solution.

Reviewer comment (Novelty):
The effect of text conditioning and dataset complexity is also discussed in [2].

Author response:
We wound also like to make more clarifications on how our research different from [1] regarding dataset size and [2] regarding text conditioning and dataset complexity. 

- The authors in [2] undertook a comparative analysis of diffusion models conditioned on various types of captions. Our research, however, diverges in the notable discovery that random conditions can effectively induce the memorization of class-conditioned diffusion models. We also find that the number of classes plays an important role in the memorization. In terms of dataset complexity, [2] compared models trained on LAION-10k and Imagenette datasets, attributing the higher memorization observed in the latter to the structural complexity of its images. Here the dataset complexity is assessed on an instance-level basis. While in our experiments, we meticulously constructed a series of training datasets, each varying in the number of classes or the intra-class diversity, while keeping the other factor constant. In our research, the number of classes and intra-class diversity serve as population-level metrics to assess the dataset complexity.

Reviewer comment (Experiments):
How do the findings discussed in the paper help us understand memorization in diffusion models happening in real world settings where datasets are huge?

Author response:
Firstly, we have run a series of new experiments on the FFHQ dataset [4] during the rebuttal period, which has been updated in **Appendix D: More empirical results on FFHQ** in the revised paper. The new experimental results support our findings on CIFAR-10.

Secondly, we would like to emphasize that our objective is to gauge the extent of the theoretical hazard in typical diffusion model training scenarios instead of understanding the memorization behaviors of diffusion models trained on large data. When diffusion models trained on large amounts of data, they generally do not memorize training data in a pixel-by-pixel manner. This also aligns the research findings in [3]. The authors in [3] (Table 1) mentioned that only 200~300 training images out of 1 million generations sampled by DDPM [5] and its variant [6] are extracted succussfully. The above two models are trained on a dataset of only 50k CIFAR-10 images. Therefore, much larger training data size is out of our research scope.

Reviewer comment (Evaluation):
First, the analysis of memorization is done in complete isolation of the model's generalization or analysis of aspects of image generation or image quality such as inception score or pressure distance. And I do not think that any analysis on memorization can purely happen in the absence of the latter because we might end up analyzing models that do not make any sense for practitioners.

Author response:
Thank you for pointing this out. We would like to clarify the primary objective of our work: investigating under what conditions the diffusion models (undesirably) approximate the optimal solution represented by Eq.(2). On the one hand, such an optimal model makes no sense to practitioners (as also mentioned in the review), and may even lead to privacy/copyright issues. On the other hand, such an optimal model is indeed what is theoretically expected. Therefore, it becomes imperative to gauge the extent of this theoretical hazard in typical diffusion model training scenarios. This awareness is vital for mitigating adverse consequences and refining the practical utility of these models, thus requiring quantitative studies like our work.

Further, studying memorization behavior also helps the understanding of generalization. A recent work [1] suggests that diffusion models with memorization has potential adverse generalization performance. Therefore, it is expected that an increase in the memorization ratio within a diffusion model implies a diminution in its generalization capability.

As for the image quality, it has interleaving relationship with memorization. When a significant proportion of generated samples are replicas of the training data, their image quality is inherently high. Additionally, this leads to a generation distribution close to that of the training data, resulting in a low FID score. For instance, the optimal diffusion model, which can only replicate training data (thus exhibiting a memorization ratio is 100%), achieves an FID score of 0.56. This is substantially lower compared to the 1.96 FID score attained by the state-of-the-art unconditional diffusion model, EDM. Therefore, we do not emphasize metrics of image quality in our experiments.

Reviewer comment (Experiments):
Second, the experiments are performed on very small datasets and it is unclear how these findings actually take shape in real scenarios where you have huge datasets and you are training on millions of samples with almost similarly sized models.

Author response:
As elucidated in the above, our objective is to gauge the extent of the theoretical hazard in typical diffusion model training scenarios instead of understanding the memorization behaviors of diffusion models trained on millions of images. Through our extensive experiments, we find that the EMMs for training recipes of diffusion models are generally small. This provides explanations why diffusion models in real scenarios demonstrate low memorization ratios.

In addition to CIFAR-10, we have conducted a series of additional experiments using the FFHQ dataset [2], which is a higher-dimensional face dataset. These new experiments have been included in **Appendix D: More empirical results on FFHQ** of our revised paper. Due to the time constraints, our additional experiments focused on investigating the impact of data dimension / time embeddings / conditioning, on the memorization of diffusion models. It is noteworthy that the outcomes of these recent experiments are in alignment with our initial findings derived from the CIFAR-10 dataset.

Reviewer comment (Experiments):
Third, I don't think that the authors should perform a set of experiments where they try to fine-tune a stable diffusion model on a small dataset which may still be a reasonable analysis where people might want to use a custom stable diffusion model on a particular style by further fine-tuning it on a certain type of data. However, the setting that the authors discuss while it is very helpful in creating the analysis that they do is also very, very restrictive and does not generalize to realistic settings that practitioners actually care for. And I would encourage the authors to explore that.

Author response:
Thank you for your suggestion on stable diffusion. We have extended to fine-tuning stable diffusion [a] on Artbench-10 [b], a high dimensional art dataset with image resolution of $256\times256$. We have explored the effects of training data size and conditioning on memorization. The results have been included in **Appendix F: More empirical results on stable diffusion** in our revised paper. We note that the observations align with our previous experiments on CIFAR-10 and FFHQ. 

Please kindly let us know if there is any further concern, and we will do our best to respond.

---
**References:**

[a] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.\
[b] Peiyuan Liao, Xiuyu Li, Xihui Liu, and Kurt Keutzer. The artbench dataset: Benchmarking generative models with artworks. arXiv preprint arXiv:2206.11404, 2022.

Reviewer comment (Evaluation):
Fourth, a lot of the paper is about showcasing a finding but does not actually explain the reasons for why a finding actually makes sense. For instance, in particular, the section on the type of embedding was rather weak in my opinion in terms of explaining the effect. Similarly, the section on why data diversity does not influence memorization so much was pretty weak and this paper can significantly be strengthened if the authors actually discuss the results in more detail and why they should happen in a particular way. And I would say that this is true for most of the sections where currently this paper reads as a reporting of a result rather than a scientific discussion of a phenomenon.

Author response:
Our main body of work was developed as a comprehensive and empirical guidance towards the effects of various factors from perspectives of data distributions / model configuration / training procedure / conditioning on memorization in diffusion models. Our findings aim to delineate which factors have a substantial impact on memorization and which contribute more subtly. Therefore, our paper is positioned at an empirical analysis instead of a theoretical analysis of how different factors interact. Additionally, we provide several surprising findings, e.g. the significant effects of random labels, which may inspire the theoretical practitioners for further exploration.