Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Authors: Kulin Shah
Alkis Kalavasis
Adam Klivans
Giannis Daras
Venue: ICML 2025 poster (2025)
Keywords: diffusion
memorization
corrupted data
limited samples
generative models
ambient diffusion
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: We study to what extent is possible to train powerful generative models without memorizing the training set.

Abstract: There is strong empirical evidence that the stateof-the-art diffusion modeling paradigm leads to models that memorize the training set, especially when the training set is small. Prior methods to mitigate the memorization problem often lead to decrease in image quality. Is it possible to obtain strong and creative generative models, i.e., models that achieve high generation quality and low memorization? Despite the current pessimistic landscape of results, we make significant progress in pushing the trade-off between fidelity and memorization. We first provide theoretical evidence that memorization in diffusion models is only necessary for denoising problems at low noise scales (usually used in generating high-frequency details). Using this theoretical insight, we propose a simple, principled method to train the diffusion models using noisy data at large noise scales. We show that our method significantly reduces memorization without decreasing the image quality, for both text-conditional and unconditional models and for a variety of data availability settings.

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Kulin Shah
Alkis Kalavasis
2
Adam Klivans'
Giannis Daras
Abstract
et al., 2023), especially applications where the training set contains sensitive or copyrighted information (Chambon et al., 2022b;a). Carlini et al. (2023) conjectures that the im- proved performance over alternative frameworks may come from the increased memorization (Lee et al., 2024). This raises the following question:
There is strong empirical evidence that the state- of-the-art diffusion modeling paradigm leads to models that memorize the training set, especially when the training set is small. Prior methods to mitigate the memorization problem often lead to decrease in image quality. possible to obtain strong and creative generative models, i.e., mod- els that achieve high generation quality and low memorization? Despite the current pessimistic landscape of results, we make significant progress in pushing the trade-off between fidelity and mem- orization. We first provide theoretical evidence that memorization in diffusion models only nec- essary for denoising problems at low noise scales (usually used in generating high-frequency de- tails). Using this theoretical insight, we propose a simple, principled method to train the diffusion models using noisy data at large noise scales. We show that our method significantly reduces memo- rization without decreasing the image quality, for both text-conditional and unconditional models and for variety of data availability settings.
Can we improve the memorization of diffusion models without decreasing the image generation quality?
Prior work has shown that the optimal solution to the diffu- sion objective is a model that merely replicates the training points (Scarvelis et al., 2023; Biroli et al., 2024; De Bortoli, 2022; Kamb & Ganguli, 2024; Benton et al., 2024). Several papers have tried to understand the reason behind memo- rization and transition from memorization generalization (Kadkhodaie al., 2023; Kamb & Ganguli, 2024). For ex- ample, Kamb & Ganguli (2024) showed that the creativity in diffusion modeling happens when the models fail to per- fectly minimize their training loss. As the training dataset becomes smaller, overfitting becomes easier, memorization increases and output diversity decreases (Kadkhodaic et al., 2023; Somepalli et al., 2023; Daras et al., 2023c; Gu et al., 2023). Text-conditioning is also known to exacerbate mem- orization (Carlini et al., 2023; Somepalli al., 2023; 2022) and text-conditional diffusion models are known to memo- rize individual training points even when trained on billions of image-text pairs (Carlini et al., 2023; Daras et al., 2023a).
1. Introduction
Related work. Several methods have been proposed to re- duce the memorization in diffusion models (Somepalli et al., 2023; Daras al., 2023c; Wen et al., 2024; Daras al., 2024a; Kazdan et al., 2024; Chen al., 2024). A line of work proposes sampling adaptations that guide the genera- tion process away from training points (Kazdan et al., 2024; Wen et al., 2024; Chen et al., 2024). Wang et al. (2025); Ku- likov et al. (2023) propose decreasing the receptive field of the generative model to avoid memorization. Another line of work corrupts the images (Daras et al., 2023c; 2024b) the text-embedding in text-conditioned image models (Somepalli et al., 2023). Another way of tackling memo- rization is by using privacy based methods (c.g., differential privacy based methods (Dockhorn et al., 2022; Lin et al., 2023; Wang et al., 2024a) ). These methods, while effective in reducing memorization and improving the privacy, often
Diffusion models (Song& & Ermon, 2019; Ho et al., 2020; Song et al., 2020) have become widely used framework for unconditional and cext-conditional image generation. However, recent works (Carlini et al., 2023; Daras et al., 2023c; 2024b; Somepalli et al., 2022; 2023; Ross et al., 2024) have shown that the trained models memorize the training data and often replicate them at generation time. This issue has raised important privacy and ethical con- cemss (Somepalli et al., 2022; Tramèr et al., 2022; Appel
'Equal contribution 'University of Texas at Austin Yaae Uni- versity "Massachusetts Institute of Technology. Correspondence to: Kulin Shah <kulinshah @ utcx.s.cdu>>.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Training image
(Wen al., 2024)
(Wen al., 2024)
Trade-off betweeo FLD and Memorization PFLIQ with 300 Training Images
+ Ours
26
Data Potats
Pareto Pronter
24
= 0.5
Frochet Inceptionnnn Distance
22
20
"RISE 24" TriFecta Dishwasher
18
=10 0.0
16
0, 0.25
0.1
0
10 20
30
40
50
Percentage generated images that training data duplicates
Figure 2. (FID, Memorization) pairs for different values of dt. used our proposed Algorithm (presented in Section 3) for training diffusion models from limited data. The standard DDPM objective corresponds to ot, = 0 and it is not in the Pareto frontier. Setting dt low ior. Values for ot, E [0.4, 4] strike different balances between memorization and quality of generated images. The models this Figure are trained only 300 images from FFIIQ.
CANYON CARGO Outdoor shorts dark moss
high reverts back to the DDPM behav-
entire subpopulation and decrease the generalization error by (11/n). Hence, the optimal classifier should memorize these unique examples to avoid paying Case 2 in the error.
Western Chief Down Hill Trot (Black) Women's 's Rain Boots
The key assumption above seems to break when noise is added to the images. That is because different subpopu- lations start to merge and the heavy-tails of the weights' distribution disappear. Interestingly, diffusion models learn the (score of the) distribution different levels of noise. This indicates that, in principle, it is feasible avoid mem- orization in the high-noise regime (without sacrificing too much quality). Despite that, regular diffusion model train- ing, e.g., the DDPM (Ho et al., 2020) objective, results to score functions that have attractors around the training points, even for highly noisy inputs, as shown in Figure 3.
Figure 1. Qualitative results for reducing the memorization of Sta- ble Diffusion 2. Combining our method with (Wen et al., 2024) helps generate novel samples for the above prompts. See Section for our method and Section 5.2 for more details on the experiment.
decrease the image generation quality. Feldman (2020) the- oretically showed strong trade-offs between memorization and generalization by showing that memorization is nec- essary for (optimal) classification. This raises the natural question of whether this trade-off also applies to generative modeling.
The discussion above suggests that should be possible to train high-quality diffusion models that do not memorize in the high-noise part of the diffusion. It has been empirically established that this part controls the structural information of the outputs and hence the diversity of the generated dis- tribution (Dieleman, 2024; Li & Chen, 2024). To avoid memorization in the high-noise regime, we propose a sim- ple, principled framework that trains the diffusion model only with noisy data at large noise scales. We give theoret- ical evidence that the noisy targets used for learning leak much less information about the training set and further they are harder to memorize since they are less compressible.
The need for memorization in (Feldman, 2020) is associated with the frequencies of different subpopulations (e.g., cats, dogs, etc.) that appear in the dataset. The key observation is that the distribution of the frequencies is usually heavy- tailed (Zhu et al., 2014), i.e., roughly speaking in dataset of size n, there will be many classes with frequency around 1/n. This means that the training algorithm will only ob- serve single representative from those subpopulations and cannot distinguish between the following two cases: Case 1. If the unique example comes from an extremely rare subpopulation (with frequency < 1/n). then memorizing has no significant benefits, Case 2. If the unique exam- ple comes from a subpopulation with 1/n frequency. then memorizing it will probably improve the accuracy on the
Our contributions:
We propose simple framework to train diffusion mod-
2

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Input images
where the map o, == (() is the noise schedule and X is drawn from the clean distribution po- We remark that our framework extends to other noise schedules, diffusion models (Song & Ermon, 2019; Daras et al., 2023b; Bansal et al., 2022; Karras et al., 2022) and flow matching (Lipman et al., 2022; Liu et al., 2022; Albergo et al., 2023).
σ 17
8
Outputs of diffusion model trained on 52k images
Our ultimate goal is sample from the unknown distri- bution po- The key idea behind diffusion modeling is to learn the score functions, defined V Vlog pc()), for differ- ent noise levels I, where Xx Pr. The latter is related to the optimal denoiser E[zo Xx, = xel through Tweedie's formula (Efron, 2011):
Outputs of diffusion model trained on 300 images
Outputs of Ambient Diffusion trained on 300 images
V p((x)) = v of
PR Fo X. Nz. 21.
The conditional expectation is typically learee from the available data with supervised learning over some paramet- ric class of models H. = {hg : E θ). using the training objective:
Figure 3. Comparison of denoised images under different noise levels and training conditions. Standard diffusion modeling leads to overconfident predictions (row 3) even for very highly noised inputs when it is trained on small datasets. Our algorithm (row 4), has similar behavior (blurry outputs) to a model trained with significantly more data (row 1), indicating less memorization.
((0) Ezg B(e..t) æo (1,,, xull?]
(3)
Post training, the score function V pl is approxi- mated by plugging the optimal solution of (3) to (2). Alter- natively, one can train directly for the score function using the noise prediction loss (Ho et al., 2020; Vincent, 2011):
els that achieve reduced memorization and high-quality sample generation even when trained on limited data. We experimentally validate our approach on various datasets and data settings, showcasing significantly re- duced memorization and improved generation quality compared to natural baselines, in both the uncondi- tional and text-conditional settings.
(()) Exa.zei so( t) v1 of20 It
of
(4)
Given access to the score function for different times 4 one can sample from the distribution of po by running the process (Song et al., 2020):
On the theory side, we adapt the theoretical frame- work of (Feldman, 2020) for studying memorization to diffusion models. Based on that, argue about the necessity of memorizing the training set in differ- ent noise scales indicating that memorization is only essential at the low-noise regime.
(dor/dl)o v logp:(a)) dt.
d. = -I
(5)
1
2.2. Memorization in Diffusion Models
The first expectation of (3) is taken over the distribution of zo. The underlying distribution of zo is continuous, but in practice we only optimize this objective over finite distri- bution of training points. Prior work has shown that when the expectation is taken over an empirical distribution Po. the optimal score can be written in closed form (Scarvelis et al., 2023: Biroli et al., 2024; De Bortoli, 2022; Kamb Ganguli, 2024; Benton et al., 2024). Specifically, the opti- mal score for the empirical distribution, which corresponds to a finite amount of examples S, can be written as:
We quantify the information leakage of our proposed algorithm in the high-noise regime showing significant benefits over the standard diffusion modeling objective.
2. Background and Related Work
2.1. Diff'usion Modeling
The first step in diffusion modeling is to design corruption process. For the ease of presentation, we focus on the widely used Variance Preserving (VP) corruption (Ho et al., 2020; Song et al., 2020). We define sequence of increasing corruption levels indexed by E [0, 1], with:
1
8. (1 1)
o7 EioEsN(TH v 07
Σ no ES
(V 0x - -x1) N(x;; /1 02 0
X, = v 2xxo + xx0
Z ~~N(0,1. (1)
3

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusson
Chen, 2024). Hence, if we can avoid memorization in this regime, it is highly unlikely that we will replicate training examples at inference time, even if we memorize at the low- noise part. Our training algorithm can "copy" details from the training samples and still produce diverse outputs.
Intuitively, each point 20 in the finite sample S (i.c., the empirical distribution Po) is pulling the noisy iterate 2t towards itself, where the weight of the pull depends on the distance of each training point to the noisy point. The above solution will lead to a diffusion model that only replicates the training points during sampling (Scarvelis et al., 2023; Kamb & Ganguli, 2024). Hence, any potential creativity that is observed experimentally in diffusion models comes from the failure to perfectly optimize the training objective.
Our training framework is presented in Algorithm 1. works by splitting the diffusion training time into two parts, S tn and & > f.. where en (t-nature) is a free parameter to be controlled. For the regime, t S fu. we train with the regular diffusion training objective, and (assuming perfect optimization) we know the exact score, which is as given in Section 2.2. To train for t >> f.. we first create the set St, which has one noisy version of each image in the training
2.3. Ambient Score Matching
One way to mitigate memorization is to never see the train- ing data. Recent techniques for training with corrupted data allow learning of the score function without ever seeing a clean image (Daras et al., 2024b; 2023a; Kawar et al., 2023; Bai et al., 2024; Wang et al., 2024b; Rozet et al., 2024). Consider the case where we are given samples from a noisy distribution Pt. (where in stands for t-nature) and we desire to learn the score at time for t t.. The Ambient Score Matching loss (Daras et al., 2024b), defined as:
set. Then, we train using the set St, and the Ambient Score Matching loss introduced in Section 2.3.
It is useful to build some intuition about why this algorithm avoids memorization and at the same time produces high- quality outputs. Regarding memorization: 1) the learned score function for times t 2 t, does not point directly towards the training points since Ambient Diffusion aims to predict the noisy points (recall that the optimal DDPM solution points towards scalings of the training points) and 2) the noisy versions Tt, are harder to memorize than To. since noise is not compressible. At the same time, if the dataset size were to grow to infinity, both our algorithm and the standard diffusion objective would find the true solution: the score of the underlying continuous distribution. In fact, Algorithm 1 learns the same score function for times S tn as DDPM. This contributes to generating samples with high-quality details, copied from the training set.
J Jambien: (0) = Exrt, T.
Tie
ho ho(
of
2
1 0 D1
It,
(6)
o
o
can learn the conditional expectation E[zg [ar]] (similar to Equation (3)) without ever looking at clean data from Po- The intuition behind this objective is that to denoise the noisy sample Z4, we need to find the direction of the noise and then rescale it appropriately. The former can be found by denoising to an intermediate level in and the rescaling en- sures that we denoise all the way to the level of clean images. Once the conditional expectation B[zo| [2)] is recovered, we get the score by using Tweedie's Formula.
4. Theoretical Results
4.1. Information Leakage
In this section, we attempt to formalize the intuition of why our proposed algorithm reduces memorization of the dataset. We start by showing the following Lemma that characterizes the sampling distribution of our algorithm for ( I.
We remark that this objective can only be used for t > t. While there are ways to train for t < tn without any clean data (c.g. see (Daras et al.. 2023a: 2024b; Bai et al., 2024; Wang et al., 2024b; Rozet et al., 2024)), this leads to performance deterioration unless a massive noisy dataset is available (Daras et al., 2024a). For what follows, we refer to (3) as the DDPM training objective and to (6) the Ambient Diffusion training objective for noisy data.
Lemma 4.1 (Ambient Diffusion solution at ty)) Let St., be the noisy training set as in LI of Algorithm 1. For a fixed St,, let Pt, be the distribution at time arises by using the score of Algorithm I in the reverse process of (5) initialized at (o, It holds that Pr.. [St.l 1sss111 Exe,Es,, 8(x Itt.)
tn that
For the proof, we refer to Section D.1.I. This Lemma extends the result of Kamb & Ganguli (2024) from the stan- dard diffusion objective of Eq. to the training objective of (6). Given this result, we can compare the information leakage of Ambient Diffusion at time in compared to the
3. Method
We are now ready to present our framework for training diffuusio models with limited data that will allow creativity without sacrificing quality. Our key observation is that the diversity of the generated images is controlled in the high- noise part of the diffusion trajectory (Dieleman, 2024; Li &
We often use the symbol n for sample size; the notation f, unrelated to the size #.
attraction to 2t weight of attraction
4

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
optimal distribution It.. learned by DDPM at that time.
to differential privacy and stability in learning (Feldman, 2020; Russo & Zou, 2019; Xu & Raginsky, 2017: Bousquet & Elisseeff, 2002; Steinke & Zakynthinou, 2020; Bassily et al., 2018). Our work studies this trade-off in diffusion models, inspired by the work of (Feldman, 2020).
Lemma 4.2 (Information Leakage). Consider point 2o Np, Σ), set A of size m generated i.i.d. by Pt.. (optimal ambient solution at time in with inpul ro) and a set D of size m generated i.i.d. by qt (optimal DDPM solution at time In with imput To)- Then the mutual information satisfies: [(; mo) = m I(A; zu) = m log det( Σ 1+11 I).
Section Overview. To facilitate the understanding of this Section, we provide an overview of the results. In Section 4.2.1, we define the distribution be learned as a mixture of distributions of subpopulations (e.g., dogs, cats, etc.) with unknown mixing weights. This distribution is learned given finite set Z of size and we are interested in the general- ization error of the trained model (at some fixed noise scale ot). In Theorem 4.3 we express this generalization error into two terms, one of which is the error of the algorithm for populations that are seen only once during training. We consider that the trained model "memorizes" when the error of these rare examples is small. Due to the error decompo- sition, generalization is related to the memorization error and its multiplying constant 7i that appears in Theorem 4.3. In Section 4.2.3 we analyze how this constant changes for different noise levels under the assumption of (Zhu et al., 2014; Feldman, 2020) that the mixing weights are heavy- tailed. We argue that when the noise level is small, Ti is large and due to the decomposition, the only way to achieve good generalization is to memorize. For high noise levels, ti becomes smaller and hence it is in principle possible to achieve generalization without excessive memorization.
1- of
2
For the proof, we refer to Section D.1.2. The above means that DDPM leaks much more information about the train- ing point compared to Ambient Diffusion, when asked to generate a collection of samples from the model at time tu. Another way to see it, is that given m samples from DDPM time lu, one can get estimator for 2o with error poly(1/m), while with Ambient Diffusion, no consistent estimation is possible. As expected, as ot. ~ O, then no noise is added to create St, and hence the mutual informa- tion blows up. On the other extreme, as or, R 1, then the models reveal no information about the original point. the dataset contains multiple points, similar results about the mutual information can be obtained (see Section D.1.3).
The above indicate that Ambient Diffusion can only mem- orize the noisy images. Our justification for the improved performance in practice is that memorizing noise is much harder since noise is not compressible. Even if the noisy images are perfectly memorized, they do not contain enough information to perfectly recover the training set (as shown above) and hence creativity will emerge. A possible con- jecture is that under reasonable smoothness assumptions the concatenation of Ambient Diffusion (i.e., of non- memorized trajectory (up to (n)) and of DDPM (i.e., of a memorized one (from tu to 0)) will not lead to memorized outputs. Under this conjecture, controlling the high noise case is all you need to decrease memorization, and this is what our algorithm achieves. Showing some non-trivial lower bound between the distribution learned by our algo- rithm and the distribution learned by DDPM is a challenging theoretical problem that remains to be addressed.
1.2.1. SUBPOPULATION MODEL OF FELDMAN (2020)
Let us consider a continuous data domain X S E (c.g.. images). We model the data distribution N fixed distributions M1, .... MN. where each component corresponds to subpopulation (e.g., dogs, cats, etc.). For simplicity, we follow Feldman (2020) and assume that each component Mj has disjoint support X; (this can be relaxed, see Remark B.3). Without loss of generality, let X = U.
mixture of
We will now describe the procedure of (Feldman, 2020) that assigns frequencies to each subpopulation of the mixture.
4.2. Connections to Feldman (2020)
1. Consider list of frequencies π = (mT, X2, NN).
2. For each component E [N] of the mixture, select randomly and independently an element p. from π.
In the previous section, we discussed ways to reduce the memorization. In this section, we consider what is the price to pay for reduced memorization, i.e., we analyze the trade off between memorization and fidelity.
3. Finally, to obtain the mixing weights, we normalize the elements P1...,,N PN, i.e., the weight of component i is D2 Ese(x Ps
While there significant amount of empirical research on connections between memorization and generation for diffu- sion models, our rigorous theoretical understanding is still lacking. In terms of theory, there are many works studying memorization- memorizztionnggencalizaaiio trade-offs for machine learn- ing algorithms (Feldman, 2020; Feldman & Zhang, 2020; Brown et al., 2021; Attias et al., 2024; Cheng et al., 2022: Livni, 2024; Brown et al., 2022) with several connections
We denote by Dr, the distribution over the mixing coeffi- cients tuple (D), Dx). A sample D ~ D, is just a list of the normalized frequencies of the N subpopulations. If D ~ D,,, then we can define the true mixture as
MD(x) Σ Dj M(x)) [E[N
5

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
The above random distribution corresponds to the subpopu- lations model introduced by Feldman (2020).
parameter that relates the two quantities is the coefficient TI If the coefficient ti is large, it means that if the model does not fit the "rare examples" of the dataset, it will have to pay roughly T1 in the generalization error. As shown by (Feldman, 2020), TI is controlled by how much heavy- tailed is the distribution of the frequencies of the mixture model. This the topic of the next section, where we also investigate the effect of adding noise to the training set.
4.2.2. ADAPTATION TO DIFFUSION
As explained in the Background Section 2. one way to train a generative model in order to generate from the target MD is to estimate the score function v log MD, for all levels of noise indexed by t. For the analysis of this Section, we consider the case of a single fixed t. We define learning algorithms A as (potentially randomized) mappings from datasets Z to score functions 8p AZ).
4.2.3. HEAVY TAILS AND THE ROLE OF NOISE
In this section, we are going to formally explain what it means for the frequencies of the original dataset to be heavy- tailed (Zhu et al., 2014; Feldman, 2020). This heavy-tailed structure will then allow us to control the generalization error in Theorem 4.3. We will be interested in subpopula- tions that have only one representative in the training set Z (these are the examples that will cost roughly тi in the error of Theorem 4.3). We will refer to them single subpop- ulations. For this to happen given that |Z| = it should be roughly speaking the case where some frequencies D, are of order 1/n. The quantity that controls how many of the frequencies D; will be of order 1/n is the mass that the distribution (aa) Pro = al assigns to the interval [1/(2)), 1/n]. Typically, we will call list of frequencies heavy-tailed if
As in Feldman (2020), we are interested about the expected error of A conditioned on dataset being equal to Z E X as
C A| =
A(Z))rTM (s0).
where D ~~ Dr is a (random) collection of mixing weights and CrTaN, (50) = Eug [(so; To) for some loss function L is the expected loss of the score function 89 under the true population MD. The results we will present shortly are agnostic to the choice of I., but the reader should think of L as the noise prediction loss used in (4) for fixed time t.
The quantity err (T, A|z) measures the generalization error of the score function of the learning algorithm A conditional on the training set being Z. We will show that the population loss of an algorithm given dataset Z is at least:
1 2n 1/m)
1/ml)=
weight
2(1)
(7)
1. its loss on the unseen part of the domain, i.e., the popu- lation loss in X Z plus
In words, there should be a constant number of subpopula- tions with frequencies of order O(1/n). This definition is important because it can then lower bound the value 71 in Theorem 4.3 and hence it can lower bound the generaliza- tion loss of not fitting single subpopulations.
2. its loss on the elements of 7 that belong to subpopu- lations that are represented only once in Z (i.e., the dataset contains a single image of a dog or single image of a car). This loss, denoted by errz (A, 1). is scaled up by coefficient Tı, which expresses the "likelihood" of having such subpopulations.
Lemma 4.4 (Informal, see Lemma B.4 and Lemma 2.6 in (Feldman, 2020)). Consider dataset of size n and assume that π is heavy-tailed, as in (7). Then T (11/m).
Typically, we define:
On the contrary, when π is not heavy-tailed, Ti will be small and hence generalization is not hurt by not memo- rizing (see Lemma B.5). Next, we are going to inspect how the noise scale affects the heavy-tailed structure of the frequencies and hence the value of 77. For illus- tration, we will consider the most standard model, that of mixture of Gaussian subpopulations (similar results are expected for more general population models; we note that the previous results can be naturally adapted for the GMM and other cases, see Remark B.3 and the dis- cussion in (Feldman, 2020)). Let us consider a density 90 = Ei
E EanT [a - 1)==1]
T1 E Ean [a(1- - (al1-)m--1 a)
where म is the marginal distribution F(a) PrD(D, = a]. Note that, because the random process of picking the mixing weights is run independently for any i E [N], the marginal is the same across different i's (and hence we omit the index i from T). We are now ready to present our result.
Theorem 4.3 (Informal, see Theorem B.2). It holds that
eTT TT, A|Z) 2 eTT umseneeen (T, A(2) + (A,
ii (,,V 1) -E, -,, wyNg. We will say that two components NG,NN are c-separated if TV(N,, Ng) > 2c and can be e-merged if (VN,, S E. If N;; and Nf, are merged, weringnsidigtthal that chesnow codfficiintiss at class,i Lemma 4.5 (Informal, see Section B.4). Consider the GMM density qo and let qe be the density of the forward diffusion
The above result can be extended to subpopulations repre- sented by 2 or more examples in Z (see Appendix B). The above inequality relates the population error of the model with its loss on some parts of the training set. The crucial
6

---

Does Generation Require Memorization? Creative Diffusson Models using Ambient Diffusion
tion Distance (Heusel al., 2017) (FID) between 50,000 generated samples and 50,000 dataset samples measure of quality. Following prior work (Somepalli et al., 2022; 2023; Daras et al., 2023c), we measure memorization by computing the similarity score (i.e., inner product) of each generated sample to its nearest neighbor in the embedding space of DINOv2 (Oquab et al., 2023). For all these experi- ments, we compare the performance of Algorithm I against the regular training of diffusion models (see E3)(
Table 1. FID and Memorization results comparing DDPM and Algorithm 1. Memorization is measured as DINOv2 similarity be- tween generated samples and their nearest training neighbors. We achieve same or better FID with significantly lower memorization.
# Train Images
300
1k
3k Ours
DDPM
Ours
DDPM Ours
DDPM
CIFAR- 10
FID
25.1
23.91
10.46 10.36
14.73
14.26
S>0099
78,96
44.84
75.86
69.08
53,40
52.24
Choice of ln- Our method has single parameter in that needs to be controlled. We argue that there is an interval (fmins mmax that contains reasonable choices of tn- Setting tu too low, i... (tn, I tmin), essentially reverts back to the original algorithm that produces memorized images of good quality. But also, setting in too high, i.e., in 2 mmax will also lead to memorization as there is more time in the sampling trajectory (the interval [0, mmax)) where we use the memorized score. Values in the range (fmin, (max) achieve low memorization and strike good balances in the maliiittmmoiizttionn trade-off.
S>0.9255
67.2
20.22
57.98
47.26
11.92
11.36
8>0.95
56.56
9.64
43.44
26.34
0.08
0.06
FID
16.21 15.05
12.26
11.3
6,42
6.46
FFHQ
S>0.85
63.38
49.68
55.36
32.08
21.58
20.08
S>0.875
55.48
40.01
17.48 43.82
4.98
4.53
S>0099
47.86
29.86
33.92
7.52
0.46
0.42
ImageNet
FID
50.2
47.19
40.66
39.87
S>0,9
54.72
26.68
32.86
28.40
8>0..255
41.66
15.56
12.32
9.44
$>0.95
25.86
5.54
6.08
4.02
process at time t with schedule of E 1]. Consider any pair of components N3 in qo with total variation Ciş for some absolute constant Cigy and let NY be the associ- ated distributions in qt-
Decreasing memorization without sacrificing quality, Most of the prior mitigation strategies for memorization often decrease the image generation quality. Here, we ask: how much do we need to memorize to achieve a given im- age quality? To answer this, we tune the value f, to train models using Algorithm 1 that match the FID obtained by DDPM, and we measure their memorization levels. To report memorization, we use three thresholds in the similar- ities of DINOv2 embeddings that semantically correspond to: i) potentially memorized image, ii) (partially) memo- rized image, and, iii) exact copy of an image in the training set. The thresholds are tuned separately for each dataset to express these semantics. We present analytic results for 300, 1k and 3k training images from CIFAR-10, 10, FFHQ and (mine)-mmageNer in Table 12. As shown, for the same or better FID, our models achieve significantly lower memo- rization levels. This leads to the surprising conclusion that models learned by the DDPM loss are not Pareto optimal for small datasets. That said, the benefit from our algorithm in both FID and memorization shrinks as the dataset grows.
(Low Noise) If ai S - (2e/ then NG,N are e-separated.
(High Noise) If 2 VI (e/C,)) then Ni NN are e-merged with coefficient w. + wy.
For more formal treatment, we refer to Section B.4. The above Lemma has the following interpretations. If the noise level is small, the originally separated subpopulations (at t = 0) will remain separated. This implies that if the fre- quencies (i.e., the mixing weights) were originally heavy- tailed (as in the above discussion), they will remain heavy- tailed even in the low-noise regime, i.e. Lemma 4.4 applies (T1 is large). On the other side, as we increase f, the clusters start to merge and the heavy-tailed tailed distribution of the mix- ing coefficients becomes lighter (until all the clusters are merged into a single one). Hence, will be small. This con- ceptually indicates that there is no reason for memorizing the training noisy images i2 (and hence the original images To which do not appear during training).
Other points in the Pareto frontier. So far, our goal was to reduce memorization while keeping FID the same as DDPM. However, by appropriately tuning the value tn we can achieve other points in the Pareto frontier that achieve varying trade-offss between memorization and quality of generated images. We present these results for model trained on 300 images from FFHQ in Figure 2. We see that setting or,, E [0.44 4] corresponds to Pareto optimal points, while setting the value of i, too low or too high brings us
5. Experiments
5.1. Memorization in Unconditional Models
We start experimental evaluation by measuring the mem- orization and performance of unconditional diffusion mod- els in several controlled settings. Specifically, we train models from scratch on CIFAR- 10, FFHQ, and (tiny) Im- ageNet using 300, 1000 and 3000 training samples. For each one of these settings, we compute the Fréchet Incep-
FFor tiny ImageNet, we do not report results in the 300 samples setting since there are 200 different classes and so for some of the classes we do not observe any samples.
7

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Table Comparison between DDPM, our Algorithm and results obtained by training with only corrupted data (masking or additive Gaussian noise). As shown, our algorithm achieves low memorization since it uses noisy data in the high-noise regime, but it also achieves low FID (contrary to the algorithms only using corrupted data) as it can copy the freuurnyyy details from the training samples.
# Training Images
Metric
300
1k
3k
DDPM
Masking Noise
Ours
DDPM
Masking
Noise
Ours
DDPM Masking
Noise
Ours
FID
16.21
23.40
27.92
15.05
12.26
15.73
25.37
11.3
6.42
7.44
16.28
6.46
Sim > 0.85
63.38
53.73
29.12
49.68
55.36
38.74
14.83
32.08
21.58
19.74
12.08
20.08
Sim > 0.875
55.48
41.37
18.73
40.01
43.82
22.94
9.37
17.48
4.98
4.56
3.32
4.53
Sim > 0.9
47.86
30.34
10.60
29.86
33.92
10.08
6.49
7.52
0.46
0.43
0.36
0.42
back the DDPM performance, as expected. For or., = 4, we almost match the FID that DDPM gets with 1000 images, while we only use 300 images for training, establishing our Algorithm as much more data-efficient than DDPM. We also compute the Frechet distance using Dinov2 and report the result in 3. The result shows that our method improves the quality of generated images.
strategies that reduce memorization in the unconditional set- ting. These methods are known to achieve lower memoriza- tion but at the expense of FID. We compare with model trained on linearly corrupted data (random inpainting), as in the work of (Daras et al., 2023c), and a model trained with only noisy data as in (Daras et al., 2024b). We present the results Table 2. As shown, our algorithm produces superior behavior it achieves lower memorization for the same or better FID. The superior performance comes from the ability our method has to generate high-frequency de- tails, contrary to the existing methods that only use solely noisy data and are not capable of such behavior.
Table FDnimoy2 Scores for Baseline and Our Method
Method
FD DinoV2
Baseline
353.19
Ours (r=0.1)
347.21
Ours (d=0.25)
344.60
5.2. Memorization in Text-Conditional Models
Ours (r=0.3)
358.23
We continue our evaluation in text-conditional models. Here, the primal source of memorization is the text-conditioning itself. Wen et al. (2024) observe that for certain trigger prompts, the prediction of the network always converges to the same training point, independent of the image ini- tialization. Our method mitigates image memorization by training with noisy images, so by itself, it cannot mitigate memorization that arises from the text- text-conditioning. How- ever, we will show that when we combine our method with strategies that mitigate the impact of text memorization, we achieve state-of-the-an results in memorization reduction while keeping the quality of the generated images high.
Ours (r=0.4)
371.81
Ours (r=0.5)
382.03
Mean and median similarity of generated samples. To understand the average similarity of the generated samples to the training samples, we also calculate the mean and median of the similarity score of 50,000 generated samples in the same setting in Figure 2 (300 training samples from FFHQ). We report our results in Table 4. We see that the mean and median similarity of generated samples is significantly lower for our method.
Following prior work (Somepalli al., 2023), we finetune Stable Diffusion on 10k image-text pairs from a curated subset of LAION (Schuhmann et al., 2022) and we measure image quality and memorization of the resulting models. We compare with existing state-of-the- methods for reducing memorizing arising from the rext-conditioning. Specifically, we compare with the work of Somepalli et al. (2023) where corruption is added to the text-embedding during training and with the work of Wen et al. (2024) where the model is explicitly trained to pay attention to the visual content (for details, we refer the reader to the associated papers).
Table 4. Comparison of Mean and Median Similarity Values
Method
Mean
Median
DDPM
0.8826
0.8931
Ours (a = 0.1)
0,8854
0.8748
Ours (a = =0.25)
0,8518
0,8491
Ours (a =0.3)
0.8473
0.8475
Ours (a =0.4)
0.8287
0.8292
Ours (a 0.5)
0.8060
0.8069
Comparison with other mitigation strategies. For com- pleteness, we include comparisons with two other mitigation
We include all the results in Table 5. As shown, the combi- nation of our work with existing methods achieves otate-of-
8

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Impact Statement
Table Memorization and FID results for text-conditional models. Sim denotes the average similarity between generated sample and nearest neighbor in the dataset, while 95% is 95% percentile of the similarities distribution. CLIP measures the image-text alignment. The combination of our method with existing methods from Somepalli et al. (2023) (S23) and Wen al. (2024) (W24) achieves strong CLIP/FID results with reduced memorization.
This paper presents work whose goal is to advance the field of Machine Learning. Machine learning models that memo- rize training data pose significant privacy and security risks, especially when trained on sensitive copyrighted datasets. Our methods improve the existing memorization-geneeee trade-off's Hence, this paper contributes to safer and more ethical deployment of ML systems.
Method
Sim
95%
CLIP
FID
Without text mitigation:
Baseline
0.378
0.649
0.306
18.18
Ours
0.373
0.636
0.305
18.34
Text mitigation:
S23
0.319
0.573
0.302
20.55
S23+ ours
0.308
0.547
0.306
21.30
W24
0.208
0.300
0.293
21.44
W24+ ours
0.192
0.267
0.293
20.74
the-art memorization performance while performing on par in terms of image quality. As expected, without any text- mitigation our algorithm fails to improve significantly the memorization since the model remains heavily reliant on the text-conditioning, effectively ignoring the visual content.
6. Limitations, Conclusion and Future Work
Our work provides a positive note on the rather pessimistic landscape of results regarding the memorization-quai qualtiy trade-off in diffusion models. We manage to push the Pareto frontier in various data availability settings for both text- conditional and unconditional models. We further provide theoretical evidence for the plausibility of generation of diverse structures without memorization. We remark that our method does not come with any privacy guarantees or optimality properties and that despite some encouraging first theoretical evidence an end-to-end analysis for the pro- posed algorithm is currently lacking. We believe that these constitute exciting research directions for future research.
Acknowledgements
Kulin Shah and Adam Klivans are supported by the NSF AI Institute for Foundations of Machine Learning (IFML).
Alkis Kalavasis is supported by the Institute for Foundations of Data Science at Yale (FDS). Giannis Daras is supported by the NSF AI Institute for Foundations of Machine Leamn ing (IFML) and the Computer Science Artificial Intelligence Laboratory at MIT (CSAIL).

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
References
USENIX Security Symposium (USENIX Security 23), pp. 5253-5270, 2023.
Albergo, M. S., Boffi, N. M., and Vanden-Eijnden, E. Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint WrXiv:230300879) 2023.
Chambon, P., Bluethgen, C., Delbrouck, J.-B., Van der Slujjs, R., Polacin, M., Chaves, M. Z.,, Abraham, M., Purohit, S., Langlotz, C. P., and Chaudhari, A. Roent- gen: vision-language foundation model for chest x-ray generation. arXiv preprint arXiv:2211. 2022a.
Appel, G., Neelbauer, J., and Schweidel, D. A. Generative ai has an intellectual property problem. Harvard Business Review, 2023.
Chambon, P., Bluethgen, C.,, Langlotz, C. P., and Chaudhari, A. Adapting pretrained vision-language foundational models to medical imaging domains. arXiv preprint arXiv:2210.04133, 2022b.
Arbas, J., Ashtiani, H., and Liaw, C. Polynomial time and private learning of unbounded gaussian mixture models. In International Conference Machine Learning, pp. 1018-1040. PMLR, 2023.
Chen, C., Liu, D., and Xu, C. Towards memorization- free diffusion models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8425 8434, 2024.
Attias, L.. Dziugaite, G. K., Haghifam, M.. Livni, R., and Roy, D. M. Information complexity of stochastic convex optimization: Applications to generalization and memo- rization. arXiv preprint 2402::2400..9322,, 2024.
Cheng, C., Duchi, J., and Kuditipudi, R. Memorize to generalize: on the necessity of interpolation in high di- mensional linear regression. In Conference on Learning Theory, pp. 5528 05580.56..055 PMLR, 2022.
Bai, W., Wang, Y. Chen, W., and Sun, H. An expectation- maximization algorithm for training clean diffusion models from corrupted observations. arXiv preprint arXiv:2407.01014, 2024.
Daras, G., Dagan, Y., Dimakis, A. G., and Daskalakis, C. Consistent diffusion models: Mitigating sampling drift by learning be consistent. arXiv preprint wwxh:2332.0905:1 2023a.
Bansal, A., Borgnia, E., Chu, H.-M., Li, J. S., Kazemi, H., Huang. F., Goldblum, M., Geiping, J., and Goldstein, Inverting arbitrary image transforms 2022.
T. Cold diffusion:
without noise. arXiv preprint arXiv:2208.0923
Daras, G., Delbracio, M., Talebi, H., Dimakis, A., and Milanfar, Soft diffusion: Score matching with gen- eral corruptions. Transactions on Machine Learning Research, 2023b. ISSN 2835-8856. URL https:
Bassily, R., Moran, S., Nachum, Shafer, J., and Ychuday- off, A. Learners that use little information. In Algorithmiic Learning Theory, pp. 25-55. PMLR, 2018
Benton, J.. Bortoli, V., Doucet, A., and Deligiannidis, G. Nearly d-linear convergence bounds for diffusion models via stochastic localization. 2024.
/ // //openeeview....mm net (tte//oormmmddn9
Daras, G., Shah, K., Dagan, Y., Gollakota, A., Dimakis, A., and Klivans, A. Ambient diffusion: Learning clean distributions from corrupted data. In Thirt- Conference Neural Information Processing Systems, 2023c. URL http::ttt///o //tps////e/eeeeeeew/ net forum? i id=wBJBLy
Biroli, G., Bonnaire, T., De Bortoli, V., and Mézard, M. Dynamical regimes of diffusion models. Nature Commu- nications, 15(1):9957, 2024.
Bousquet, O. and Elisseeff, A. Stability and generalization. The Journal of Machine Learning Research, 2:499-526, 2002.
Daras, G., Cherapanamjeri, Y., and Daskalakis, C. How much is noisy image worth? data scaling laws for ambi- ent diffusion. arXiv preprint arXiv:2411.02700 2024a.
Brown, G., Bun, M., Feldman, V., Smith, A., and Talwar, K. When memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53rd annual ACM SIGACT symposium on theory of computing, pp. 123-132, 2021.
Daras, G., Dimakis, G., and Daskalakis, C. Consistent diffusion meets tweedie: Training exact ambient diffusion models with noisy data. arXiv preprint arXiv:2404. 10177 2024b.
Brown, G., Bun, M., and Smith, A. Strong memory lower bounds for learning natural models. In Conference on Learning Theory, pp. 4989-5029. PMLR, 2022.
De Bortoli, Convergence of denoising diffusion mod- els under the manifold hypothesis. arXiv preprint arXiv:2208.053/1 033004 2022.
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramer, F., Balle, B., Ippolito, D., and Wallace, E. Extracting training data from diffusion models. 32nd
Dieleman, S. Diffusion is spectral autoregression, 2024. URL https: Uttrrsss///adeee.///2///// wttp/ddr.aa//22/////00// spectral appttrall- html.
10

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Dockhorn, T., Cao, T,, Vahdat, A., and Kreis, K. Dif- ferentially private diffusion models. arXiv preprint arXiv:2210.09929, 2022.
Le, Y. and Yang, X. S. Tiny imagenet visual recognition challenge. CS231N Course Report, Stanford University, 2015. URL
Efron, B. Tweedic's s formula and selection bias. Journal of the American Statistical Association, 106(496):1602- 1614, 2011.
y eep_jrecctt
Lee, T., Yasunaga, M., Meng, C., Mai, Y., Park, S., Gupta, A., Zhang, Y., Narayanan, D., Teufel, H., Bellagente, M., et al. Holistic evaluation of rext-to-image models. Advances Neural Information Processing Systems, 36, 2024.
Feldman, V. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pp. 954-959, 2020.
Li, M. and Chen, S. Critical windows: non-asymptotic theory for feature emergence diffusion models. arXiv preprint arXiv:2403.01633, 2024.
Feldman, V. and Zhang, C. What neural networks mem- orize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881 28911, 2020.
Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, Differentially private synthetic data via foundation model apis I: Images. arXiv preprint arXiv:2305. 2023.
Gu, X., Du, C., Pang, T., Li, C., Lin, M., and Wang, Y. On memorization in diffusion models. arXiv preprint arXiv:2310.02664, 2023.
Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. arXiv preprint ar arXi::2210.0274: 2022.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Advances in Neural Information Process- ing Systems, 33:6840-6851, 2020.
Livni, R. Information theoretic lower bounds for informa- tion theoretic upper bounds. Advances in Neural Infor- mation Processing Systems, 36, 2024.
Kadkhodaie, Z., Guth, F., Simoncelli, E. P., and Mal- lat, S. Generalization in diffusion models arises from gcometry-adaptive harmonic representations. arXiv preprint arXiv:2310.02557 2023.
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F,, El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.071933 2023.
Kamb, M. and Ganguli, S. An analytic theory of creativ- ity in convolutional diffusion models. arXiv preprint arXiv:2412.20292 2024.
Ross, L.,, Kamkari, H., Wu, T., Hosseinzadeh, R., Liu, Z., Stein, G., Cresswell, J. C., and geometric framework for understanding memorization in generative models. arXiv preprint arXiv:2411.00113, 2024.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 06565-265777 2022.
G.
Kawar, B., Elata, N., Michaeli, T., and Elad, M. Gsure- based diffusion model training with corrupted data, arXiv preprint ar 2rXiv:2305.131
Rozet, F., Andry, G., Lanusse. F., and Louppe, G. Learning diffusion priors from observations by expectation maxi- mization. arXiv preprint 17X1 2024.
2023.
Kazdan, J., Sun, H., Han, J., Petersen, F., and Ermon, S. Cpsample: Classifier protected sampling for guard- ing training data during diffusion. arXiv preprint arXiv:2409.07022 2024.
Russo, D. and Zou, J. How much does your data explo- ration overfit? controlling bias via information usage. IEEE Transactions on Information Theory, 66(1):330 ):302- 323, 2019.
Kulikov, V., Yadin, S., Kleiner, M., and Michaeli, T. Sinddm: A single image denoising diffusion model. Interna- tional conference on machine learning. pp. PMLR, 2023.
Scarvelis, C., Borde, H. S. d. O., and Solomon, J. Closed- form diffusion models. arXiv preprint arXiv:2310.12395 2023.

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b; An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35: 25278-25294 2022.
Xu, A. and Raginsky, M. Information-theoreti analy- sis of generalization capability of learning algorithms. Advances in neural information processing systems, 30, 2017.
Zhu, X., Anguelov, D., and Ramanan, D. Capturing long- tail distributions of object subcategories. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 915 022, 2014.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Diffusion art or digital forgery? investigat- ing data replication in diffusion models. arXiv preprint arXiv:2212.03880) 2022.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Understanding and mitigating copying in diffusion models. arXiv preprint arXiv:2305.20086, 2023.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint OrXiv:2011.13455 2020.
Steinke, T. and Zakynthinou, L. Reasoning about general- ization via conditional mutual information. In Conference on Learning Theory, pp. 3437-3452. PMLR, 2020.
Tramèr, F., Kamath, G., and Carlini, N. Position: Consider- ations for differentially private learning with large-scale public pretraining. In Forty-first International Conference on Machine Learning, 2022.
Vincent, P. A connection between matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Wang, H., Pang, S., Lu, Z., Rao, Y., Zhou, Y., and Xue, M. dp-promise: Differentially private diffusion probabilis- tic models for image synthesis. In 33rd USENIX Secu- rity Symposium (USENIX Security 24), pp. 1063-1080, 2024a.
Wang, W., Bao, J., Zhou, W., Chen, D., Chen, D., Yuan, I.., and Li, H. Sindiffusion: Learning diffusion model from single natural image. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
Wang, Y., Bai, W., Luo, W., Chen, W., and Sun, H. In- tegrating amortized inference with diffusion models for learning clean distribution from corrupted images. arXiv preprint 4rXiv:2407.1 11 162, 2024b.
Wen, Y., Liu, Y., Chen, C., and Lyu, L. Detecting, explain- ing, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learning Repre- sentations, 2024.
12

---

Compute average loss. Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
A. Our Algorithm
In this section, we present the pseudo-code our algorithmic approach, which combines DDPM and Ambient Diffusion. Details about the approach appear in Section 3.
Algorithm Algorithm for training diffusion models using limited data.
Require: untrained network has set of samples S, noise level In, noise scheduling (1), batch size B, diffusion time
1: St. - -(11 o "0 + Te. = (x0))) E S ~ (N0
& Noise the training examples at a specified level.
2: while not converged do
3: Form batch B of size B uniformly sampled from S U St.
4: loss <- 0
Initialize loss.
5:
for each sample I E B do
6:
e~ e~~N ()
Sample noise.
7:
: E St, then
8:
t, f x
>> We are dealing with a noisy sample. Sample diffusion time for noisy sample.
9:
t ul (ta. T)
Fe +-
1-0%, Ft +
1-07
d--00 o
T-o
Add additional noise.
11:
loss loss
07
0
h (0,,1) 1)+
ho
of
1-07
Fta
Ambient Score Matching loss.
0in
oi
1-
12:
else
13:
To + x
D We are dealing with a clean sample. b Sample diffusion time for clean sample. Add noise. Regular Denoising Score Matching loss.
14:
t ~ (1,
15:
Kt + /I - 0700 + o0E
16:
loss +- loss + [1,l t) zoll
17:
end
end for
loss loss
B
20:
0 +- 0 77 Vloos
> Update network parameters via backpropagation.
21:
end while
B. Subpopulations Model and Connections to Diffusion Models
In this section, we present a more extensive exposition of the framework of the work (Feldman, 2020). Moreover, we adapt this framework to diffusion models.
B.1. Subpopulations Model of Feldman (2020)
Let us recall the subpopulations model of (Feldman, 2020). Let us consider a continuous data domain X C Rdd We model the data distribution as mixture of fixed distributions My...., My, where each component corresponds to a subpopulation. For simplicity, we follow Feldman (2020) and assume that each component M, has disjoint support X; (we relax this condition, Remark B.3). Without loss of generality, let X U, We will now describe the procedure of (Feldman, 2020) that assigns frequencies to each subpopulation of the mixture.
1. First consider a (fixed) list of frequencies æ = (mi, T2, ..,..., N.). N.
2. For each component E [N] of the mixture, we select randomly and independently an element p; from the list æ.
3. Finally, to obtain the mixing weights, we normalize the weights Pi.... PN. i.e. the weight of component i is D,
Pi
E Ps
We summarize the above as follows:
13

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Definition B.1 (Random Frequencies (Feldman, 2020)). For the mixing weights, first consider list of subpopulation frequencies = (TT,,... #x)- The procedure is the following: randomly pick pi from the list for any index i [N] and then normalize (p./ Σ5 Pj denotes the frequency of subpopulation i). We denote by Dr the distribution over probability mass functions on [N] induced by the above procedure.
A sample D ~ D is just list of the frequencies of the N subpopulations.
We also denote by ग the resulting marginal distribution over the frequency of any single element i, i.e.,
((a) = Pr
[Di
a).
(8)
D-D,
Hence, if D Dr, then can define the true mixture
Mn(x)= Σ D,M(()). iE[N]
The above random distribution corresponds to the subpopulations model introduced by Feldman (2020). Intuitively the choice of the random coefficients for the mixture corresponds to the fact that the learner does not know the true frequencies of the subpopulations.
B.2. Adaptation of (Feldman, 2020)'s Result to Diffusion Models
As explained in the Background Section 2, one way to train a generative model is to estimate the score function V log MD, for all levels of noise indexed by t. For the analysis of this Section, we consider the case of a single fixed t. We define learning algorithms A as (potentially randomized) mappings from datasets Z to score functions sg ~ A(Z). We further define the expected error of A conditioned on dataset being equal E X (eventually Z will be drawn i.i.d. from MD) as
CIT( (T, Z EDDD. A(7 (so).
where Drr is a random list of frequencies according to Definition B.1 and ert (se) Ezo~ Mi ((s;; for some loss function T.. The results will present shortly are agnostic the choice of loss function T., but the reader should think of L as the noise prediction loss used in (4) for fixed time
We remark that the quantity बा ( Al measures the generalization error of the output score function (according to loss function I.) of the learning algorithm conditional on the training set being Z. We will relate this generalization error with the loss in the training set. Recall that any subpopulation i E [N] of the mixture is associated with a domain X, (and X; XX = 0 for + j).
Let n be the training set size. For any l [n], consider all the subpopulations 1e S. [N] such that X, Z l for í Ig in words, E I if there exactly l representatives of cluster i in the dataset Z. We can define Ze = (x X I2) C Z. Note that the sets Zi,..., ZN partition the training set Z. For E [m], we define
errnz (A, = Ego~A(Z)
Σ xEZ:
Mr, (x)
99)
Here i, E [N] is the unique index of the component whose support contains In words, (A, is the loss of the algorithm A evaluated on the elements of the training set Z that belong to subpopulations will exactly & representatives in Z.
We show the following result, which is an adaptation of a result of (Feldman, 2020) and relates the population loss with the empirical losses errn z (A, ....n,, (A, n..
Theorem B.2. Fix a number of samples n. Let {MilsetN) be densities of subpopulations over disjoint subdomains (Xi)veN))- Let n be the fixed list of frequencies as in Definition and let N the marginal distribution of (8). For any learning algorithm A and any fixed dataset Z X", holds that
CTT T, AlZ) = eTT unseen (T, AIZ) +
Σ Te errnz (A,6). (rrnz(A,)) EE(m)
(10)
where
14

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
1. unseen T. A corresponds the expected Z-conditional loss of the algorithm A on the points that do not appear in the training set Z.
2. тe is a coefficient that corresponds to the weight of having subpopulations with exactly E representatives. Given Dr and l E [n], we define
Eaw (ar +11 (1 a))-9
= Ban la (1 a)m-
For the proof we refer to Section D.2.1. The optimal algorithm A (without any bound on the complexity of the score function class) is the one that minimizes the RHS in Theorem B.2, which means that
for any E, errn (A", , !) = 0 and
eTTunsern (T,, A Z) is as small as possible.
The above general form relates the population error of the model with its loss on the training set. The crucial parameters that relate the two quantities are the coefficients Ti Tn- If the coefficient T1 is large, it means that if the model does not fit the training examples that appear once in the dataset ("rare examples"), it will have pay roughly TI in the generalization error. As shown by (Feldman, 2020), is controlled by how much heavy-tailed is the distribution of the frequencies of the mixture model. This is the topic of the next section, where we also investigate the effect of adding noise to the training set. Remark B.3 (Gaussian Mixture Models). Subpopulations are often modeled as Gaussians. If the probability of the overlap between the subpopulations is sufficiently small (the means far), then can reduce this case to the disjoint one by modifying the components M, to have disjoint supports while changing the marginal distribution over Z by at most 5 in the TV distance.
B.3. Heavy-Tailed Distributions of Frequencies
In this section, we going to formally explain what it means for the frequencies of the original dataset to be heavy-tailed (Zhu et al., 2014; Feldman, 2020). This heavy-tailed structure will then allow us to control the generalization error in Theorem 4.3. Following Feldman (2020), we will assume that the mixing coefficients Dı...., Dy are drawn from a heavy-tailed distribution since this is the case in most datasets (Feldman, 2020; Feldman & Zhang, 2020). We will be interested in subpopulations that have only one representative in the training set Z (these are the examples that will cost roughly Ti in the error of Theorem 4.3). We will refer to them as single subpopulations. For this to happen given that |Z] n, it should be roughly speaking the case where some frequencies D; are of order 1/n.
The quantity that controls how many of the frequencies D; will be of order 1/n is the marginal distribution <aa) Pro[D;- = a]. We first note that the expected number of singleton examples is determined by the weight of the entire tail of frequencies below 1/n in V. In particular, one can show (see (Feldman, 2020)) that the expected number of singleton points least
weight(Tr, [0, 1/ml),
where
(r, 1/m]) = EDED
Σ KE[N]
Da(D, e [0,1//l) N Eaw E 10,1/n])]-
The above weight function essentially controls how heavy-tailed our distribution over frequencies is. Typically, we will call a list of frequencies heavy-tailed if
weight (
1 2n 1/m
2(1).
In words, there should be constant number of subpopulations with frequencies of order 1/n. This definition is important because it can then lower bound the value T2 in Theorem 4.3 and hence it can lower bound the generalization loss of not fitting single subpopulations.
Lemma B.4 (Lemma 2.6 in (Feldman, 2020)). For any n, it holds that Ti 2 5n weight I1 2.D.
As an illustration, π is the Zipf distribution and the number of clusters N 2 n then 72 = Q(1/n) and weight [0, 1/n]) = 2(1) (see (Feldman, 2020) for more examples). On the contrary, when æ is not heavy-tailed, 7i will be small.
15

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusson
Lemma B.5 (Lemma 2.7 in (Feldman, 2020)). Let π be frequency prior such that for some S 1/(2n), weight(s,, (0,t/nl) = 0, where t = (0))) 3 weight [0,0]) Then TI 20.
The above lemma indicates that when the frequencies are not heavy-tailed then 7j is small (and hence generalization is not hurt by not memorizing).
B.4. The Effects of Noise
In this section we analyze the effect of adding noise to the training set. We distinguish two cases: the low noise regime and the high noise regime.
Low Noise Regime. When the noise level is small, the originally separated subpopulations (at = 0) will remain separated. This implies that if the frequencies of the subpopulations were originally heavy-tailed (as in the above discussion), they will remain heavy-tailed even in the low-noise regime. This will imply that some clusters will be represented by singletons (f=1) 1) and any algorithm that satisfies errnz (A, 1) 7 0 has to pay TI -rrnz (A, 1) in the population error with TI being lower bounded as in Lemma B.4. We interpret errnz (A, 1) = 0 as evidence for memorization. To be more concrete, we will need the following definition that is a smooth generalization of single representative of a subpopulation.
Definition B.6. We will say that a subpopulation C has an e-smoothed single representative in a set of points S belonging to C' if for any x, x E S, holds that ||= x'1l S E.
Intuitively this means that if there are more than one images in the training set Z from C, they are all very close to each other. This will be the case in diffusion with low noise.
Lemma B.7 (Subpopulations Remain Heavy-Tailed), Consider an example no E Rdd that the unique representative of a subpopulation E [N] in the training set Z with |sol| S poly(d). Consider yn noisy copies (at)vetmi of zo at noise level t: x 11 07 + oe 2] ~ NN00, Ia). Then the subpopulation j has poly(1/d) (/dd/smoohose single representative in the ser {xi [im] for oe = poly(1/d) with probability at least 1 - m mxexp(-1]
The proof appears in Section D.2.2. The above lemma implies that if the original dataset contains various well separated images (in the sense that correspond to representatives of single subpopulations) then after adding noise to each one of them (and even if we create multiple copies for each example), the clusters will remain separated when o; is small. This implies that the single subpopulations remain and Lemma B.4 applies (T is large).
For an illustration, let us consider the GMM density function q == En (P,,,, I). It is a standard calculation to see that at time t, the pdf of the forward diffusion process is q: > W,N(Vi of pq I), which means that the clusters starting to concentrate around 0 as f. + 1 and the images from different subpopulations are starting to look more and more indistinguishable (since the TV distance between the components is contracting with t). We will say that two components N,,'' are e-separated if (V , > 2c.
Lemma B.8 (Clusters Are Separated in Low Noise). Any pair of Gaussians with original total variation C = 1/600 will be e-separated noise scale a+ < / (2e/C)2.
For the proof, see Section D.2.3.
High Noise Regime. As we increase t, we add more and more noise to the images. This means that the clusters start to merge and the heavy-tailed distribution of the mixing coefficients becomes lighter (until all the clusters are merged into a single one). To illustrate this phenomenon, we will consider a mixture of Gaussians, which is the standard model for clustering tasks (we expect similar behavior for more general mixture models). Let us again consider the density function q En w:N(u, (M;, 1). Also, let the pdf of the forward diffusion process be q: E w:N(( of 1). We will say that two components N, N can be m-merged if TVW,,N") <.
Lemma B.9 (Clusters Merge in High Noise). Any pair of Gaussians with original total variation C = 1/600 will be e-merged at noise scale ot 2 /1 - (e/C/2
For the proof, see Section D.2.3. As the clusters are getting merged, then their coefficients are added up and their distribution is no more heavy-tailed. Hence, Lemma B.5 implies that 7ı will be small. This conceptually indicates that there is no reason for memorizing the training noisy images 2; (and hence the original images To which do not appear during training).
Given the above discussion, we reach the conclusion that if the frequencies of the original subpopulations are heavy- tailed then, in the low-noise regime, the training set will have single subpopulations and, in that case, fitting these single
16

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
representatives is required for successful generalization. However, in the high-noise regime, the noisy training set does not have isolated examples and, in principle, there is no original set). We believe that this discussion sheds some light the nature of memorization needed for optimal generative modeling and motivates our training Algorithm I that avoids memorization only in the high-noise regime.
memorize its elements (and hence even elements of the
C. Noisy data training of stable diffusion using v-prediction
The variance-preserviee forward process defines the following transition probability distribution:
p(x; = xz Xx) = (xx axx, of and p(X, = xx) = NN (az/as)Xs; R..
where o/. (1
o of )o? Let in be the noise scale corresponding to the noisy data and the noisy data Tt,, from the clean data zo has the probability distribution p(,,, Ir, (Xo) (ares a Xo, of f). In this case, the following Lemma holds.
Lemma C.I. [XX ot Tilte X| +
and X
0f
Proof. Let (c) denote the probability density of the random variable X,, Observe that X, ox +0, Using Tweedie's formula, we have
V Vo = ag X X X - X2
of
Additionally, the random variable X, = (ore/or..))) Xt, + 0 Z. Using Tweedie's formula, we can write the score function
Vlogp+ (ar/ar,, [E(,, Xx] Xx
iita
Using the above equations, have
(at/ar, [[x,, - Xt xxE[Xo Xt] - Xt
of
0t Tit
On Xx at 01,
onto ofan,
X X1] =
(a.E(Xo|x,) - X1) +
Xx)
Xt
of
2 (Xe,
X.)
Lemma C.2. Predicting o - ax
ofo, oin Prit
gives us that the optimal u-prediction.
D. Proofs
D.1. Technical Details about Information Leakage
1.1.1. PROOF OF LEMMA 4.
Proof. The distribution of the training data conditioned on the dataset St, is (z(x) = 4 Ex,,E,. iterates time f, we add additional noise points Ftt, St.- Particularly, the following relation holds for any t (In, T):
ES,, 5(x-It,)) To obtain
o? of
o o e N(0.1).
X
Amb
X1, +
1 o
This induces a distribution for each time d:
q (x)
Σ
N(o:
1 - oi
0² 0 2
I).
1
o²
Ze,, EScn
17

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
The score of the Gaussian mixture ye is given by
I- -2- 1) T-ot Tha
N(x; V
of o
T-a
Amb (x) St,)
Tt - x
of - 1- iin
1-07 yr T
09-02m- T
Men
ESt
Eyest, Maz
Since the reverse flow of Eq)(5) provably reverses the forward diffusion (Song et al., 2020), the distribution % equals the empirical data distribution Qo. which a sum of delta functions on the noisy training set St,,-
D.1.2. PROOF OF LEMMA 4.2
Proof. For two random variables X, Y, recall that I(X;Y) - H(X) - where (X) is the entropy of X and HXX, Y) is the joint entropy X and Y. Without loss generality, let μt = O. Let zn ~ (0, Σ). For the Ambient Diffusion at time lns conditional on the noisy point being It,,, the optimal distribution learned 8(x Tr,,), where
Ft
o7 xo + o,, Z. Note that i.i.d. draws from this distribution (denoted by A) are identical and hence
I(A; 2o) [xxt, 20)
Now observe that
20 ~ ~N(
and
Ita ~ (1- o221 + o. I).
Σ
v 1
Moreover, for the random column vector [30,, Fit. JT, we have that EC(C
V 1 of (1 o2 of
Now it remains control the mutual information of Gaussians. Given that E-1 exists, we note that deti [E(c(")
(E) det(o? I). We can hence write
1
det(B[202g (m xI
log oet((1-0f,11) JE of,
1 (00,,500) 2
log
(11)
(E((CT))
del
This simplifies to
1 2 log det 1+ 1 o
t
where 1-o of corresponds to the signal-to-noise ratio. (An equivalent way to see the above, by taking the conditional distribution v,. (.a which has covariance of I, and hence directly get (11).)
On the other side, for DDPM, the learned distribution is NV W of zo. of I). Let X, be single draw from that distribution. Hence, i...d. draws S from that measure correspond to mutual information
1(5 = m 1(X1;z0) = m (0)
This concludes the proof.
D.1.3. ADDITIONAL BOUNDS ON MUTUAL INFORMATION FOR AMBIENT DIFFUSION
The following lemma gives a bound on the mutual information of a generated set of size m from Ambient Diffusion at time in given training set of size N. On the other side, the mutual information of the DDPM solution at time t-nature should be m times larger.
Lemma D.1. Consider a dataset S of size N drawn i..dd from (H,,) I). Consider the optimal ambient solution at time ta
with input S. Consider set A of size m generated i...d by that distribution. Then T(D; S mdd22 1og(1 /
18

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Proof. Let X; be N i...d. draws from (n, I). For each let Y, -1-o oi,) ot, Z3 for some independent normal 74 NO, I). We know that the optimal ambient solution is the empirical distribution
P(u) N Σ - Yi),
conditioned on the realization of the noisy dataset Y {Yi .... YN}. By the data processing inequality, we know that [(A;8) v :( Since Y, are generated all in the same way and independently, we can write
I(Y:))= - x() mI
We have that [(Y;:X,) = H(Y) note that Y1 ~ [(11 oi,),,, I). These imply that
- - ( Recall that X1 ~ N 1) and X1 ~ (V(11172))) Moreover,
H(Y) = 2
log(2Te)
(since it has identity covariance) and
d H(Y]XX1) 2
log 1og(2teo7?.)
This means that
X1); = 1og(1/o²
This concludes the proof.
D.2. Technical Details about the Subpopulations Model
D.2.1. PROOF THEOREM B.2
Proof. For each subpopulation with exactly l representatives, we put in the set Xz#e those representatives. Observe that the collection of sets (X2#t) partitions Z for é fl...., n). Set X2 = Ube[m] X2 The unseen points correspond to the set X X
With this notation hand, we define
errn 2 (A,E) = Beg~
Σ =EX2
Ms, ()).
where ig the index of the unique component whose support contains z. We have that
err(m, AlZ) (12) Σ
Mn(x) L(80,2).
xEX
We now decompose
-x Xz#0 and write
बाT TT, A|Z)
Σ
ED.., x))
Σ
ED.co L(sg. x)
zENz
#EXZ#o
Let us first deal with the second term. For any Xz#0. it holds
ED (MD(I) L(so, x)] DD (Es
because the way we choose is independent of the random variable L(s0, x)) which only depends on the way the algorithm picks the score function given the dataset.
19

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Set plz, 2) -E (12) Hence, from the elements that do not appear in Z, get contribution
Σ
p(a, 2) A()
(12)
Let us now deal with the elements appearing in Fix [n]. For any I E Xz#e, we have that
Ep. [MD(2)) L(sa, x)] = ED-DX (1n) (12 (x)] Eeg (88,,))
since the random variables L(so. x) and (D(x) are independent given Z. By Lemma 2.1 in (Feldman, 2020) and since the supports of the components M,,..., My are disjoint, we know that ED= (-12) [z)]] (i MM, (x)] (i,(i,))M (x) TeMi,(x), where iz is the index of the component whose support contains z. Hence, we have that
Σ xEX2
(lM)(x) L[80, 2)]
Σ RE(n] xEX2#1
Σ
Te . Mi, (x) ((00
Σ Te E
(e ((80,))
#EX2#1
In total, have shown that
CIT(T,A|Z)
Σ
Te (r,,,(A, + err unseen (TT, AlZ).
8E(m]
This completes the proof.
D.2.2. PROOF OF LEMMA B.7
Proof of Lemma B.7. We have that the i-th noisy example can be written as of v 07 + 0,2 Let us set or =
o(1 o(1/[[=ol)) = poly(1/d) Using Taylor's approximation for v1 around x = 0 (V1 x 1 -x/2 o( xz) we can write
| (1 poly(1/d))=o poly(1/d)=f1l <,
for some poly(1/d) sufficiently small. This means that
|| - xoll S + poly(1/d) + poly(1/0) |/)]|
By Gaussian concentration, we have that
Pr[l| > Vdl < exp(-d/2)
Let define the bad event Em which corresponds "subpopulation j does not have poly(L/d)-smooth)) single representative in the set [a]) for o, (dd)" A union bound over the m noisy examples gives that
Pr [Em] m exp(-d/2).
D.2.3. PROOFS OF LEMMA B.8 AND LEMMA B.9
Proofs of Lemma B.8 and Lemma B.9. The proof relies on the fact that when the total variation distance between two adentity-covarianee Gaussians is smaller than an absolute constant, then the total variation is up to constants characterized by the distance between the means (Arbas et al., 2023). When the original total variation is at most 1/600, (Arbas et al., 2023) shows that
TV(N(u, (), = 0(1 (('1))-''))
The lemmas follow by noting that the densities of N(u, (H,) I) at noise scale og (denoted as Nc, satisfy
Nr = L of., 1), No: N ofpl
20

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Since vI of < 1, the means are contracting and TV(N,, < 1/600. Also:
TV(N,,N)) & 1 121l/V2= = / - of 1
1110 1112.
If want make this quantity at most 6, it suffices take oi 2 2/te/lm--m'l] 1'
For the other side, by (Arbas et al., 2023), TV(N,, 2 I| μ4ر 11/0000 /1 - of ll 1 //200. (we assume that the original variation is smaller than 1/600 and we contract it by adding noise). this should be least 2e, then it should be that o2 S / 200 (2e/]lu 1l1))². This concludes the proof.
E. Experimental Details.
For all of our experiments regarding unconditional generation, we use the Adam optimizer with a learning rate of 0.0001, betas (0.9, 0.999), an epsilon value 1e-8, and a weight decay of 0.01. The model for FFHQ and CIFAR-10 is trained for 30,000 iterations with batch size of 256 and the model for Imagenet trained for 512 batch size for 80,000 iterations. For experiments the Imagenet dataset, we train a class-conditional model.
For FFHQ and CIFAR-10 experiments, we randomly sample 300, 1000 and 3000 samples from the complete dataset to create the dataset with limited size. We use Tiny Imagenet dataset which consists of 200 classes (Le & Yang, 2015). We sample 5 images randomly from each class to create a dataset consisting of 1000. Similarly, we sample 15 images from each class to create a dataset consisting of 3000 images. For the unconditional and conditional generation experiments, we used with the implementation of (Karras et al., 2022) and default parameters of the implementation.
For text-conditioned experiments, we use the implementation of (Somepalli et al., 2023) and implement additional baseline LAION-10K dataset train
(Wen al., 2024) and method in the implementation. Similar to previous works, the stable diffusion v2 model for 100000 number of iterations using batch size 16. We use the final checkpoint after the complete training to evaluate the memorization, clipscore and fidelity. For the text-conditioned experiments, we tried adding nature noise at noise scale {25, 50, 100} and chose the model with best image quality.
F. Images generated using our method
21

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Figure 4. Images generated using a model trained with our method on 300 samples
22

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Figure Images generated using a model trained with our method on 1000 samples
23

---

Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion
Figure Images generated using a model trained with our method on 3000 samples
24


Reviews:
[{"review_id": "wgeYlGGhat", "reviewer": "Reviewer_pjLr", "summary": "In this paper, the authors propose a simple but effective method for the diffusion model to generate creative images rather than memorizing the training data. Their method is motivated by previous work of (Feldman, 2020) about the generalization in classification problems, where they showed that the model tends to memorize when the distribution of the frequencies is heavy-tailed. So they conjecture that the high noise added to the images in the diffusion model training process would force different subpopulations to start to merge and the heavy tail to disappear. Based on this intuition, they show that the memorization in diffusion model is only necessary in the low-noise region, and they use noisy data to train the model in the large-noise region. Some theory and experiments are provided to support their claims.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their Review and suggestions.\n\n**On the blurry output of the diffusion models**: Figure 3 contains a noisy image $x_t$ and the learned model’s approximation of $E[x_0 | x_t]$. Intuitively, this expectation represents an average over all possible clean images $x_0$ that could have produced the noisy image $x_t$. As a result, the predicted image is expected to appear blurry. This is validated by the second row (outputs of a diffusion model trained on 52k images): a model trained on the full dataset has blurry predictions, as it should. On the other hand, when a model memorizes the training data, it is not modeling correctly the conditional expectation – instead, it is overconfident, i.e., it believes that one of a few clean images could correspond to a given noisy input $x_t$. In this case, it outputs a sharper (cleaner) image, which reflects memorization. The figure shows that our algorithm (Row 4) matches the blurriness level of the oracle model (52k training images) even though our model is only trained with 300 images.\n\n**On combining two existing algorithm and novelty**: The novelty of the algorithm is not in the loss, but in the way we create the data we apply the loss to. Let us clarify further. For times $t\\geq t_n$, we replace the original dataset with a noisy version of it where each datapoint has been replaced (*once*) with a noisy realization. \nThis step corresponds to the creation of the set $S_{t_n}$ at step 1 of Algorithm 1 (Line 612, Appendix Page 12). The fact that we create $S_{t_n}$ once before the launch of the training (instead of recreating at each epoch) is very important as it ensures that there is reduced information about the clean distribution for the learning that happens at times $t\\geq t_n$ (see also Lemma 4.2). This step in the algorithm leads to decreased memorization.\n\nComparison with existing works:\n- In the papers “Consistent Diffusion Meets Tweedie” and “How Much is a Noisy Image Worth?”, the authors are dealing with corrupted datasets and they are using Ambient Score Matching loss to train with the corrupted data.\n- In DDPM, clean data is used for all times using the regular objective.\n- In this paper, we have clean data, but we intentionally corrupt them for some times to reduce memorization. At the same time, we do not want to reduce performance, so we still use the clean data for certain diffusion times. The idea of intentional corruption (and the way its done in Algorithm 1) so that we achieve both reduced memorization and good performance is the algorithmic novelty of this work.\n\n\n**On the related work and additional baselines based on those works**: We want to bring to the Reviewers’ attention that **two out of the three papers that we are being asked to compare against were released last month** (26 Feb'25, and 5 Feb'25). We submitted this work in January 2025, so we do not think it is fair to compare against these two baselines. Besides, **neither of these two works has experiments with images**. That said, we are happy to cite and discuss these works in the updated version, and we will do so.\n\nWe thank the Reviewer for bringing the paper “Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention” to our attention and will cite it in the updated version. We attempted to compare against this method, but the code for training-time memorization mitigation is not provided in the official repository. We even tried implementing the approach ourselves, but couldn’t replicate their results. We contacted two of the authors about the issue, but as of today we have not received a reply. If the code becomes available, we will gladly compare.\n\n**On integrating the method with other diffusion methods**: Our method can work with any training architecture, training loss type (e.g., x0-prediction, epsilon-prediction, flow-matching, etc), and sampling method (EDM sampler, DDIM, DDPM sampler, etc). All the results in the paper are obtained by using the EDM codebase for all these components (architecture, training loss, sampler).\n\nAs per Reviewer’s request, we experimented with other pipelines, and we present the results for (FID, Memorization) below when the training is performed using 300 training images for the FFHQ data set. \n\n| Method | FID | Memorization (S > 0.9) | Memorization (S > 0.875) | Memorization (S > 0.85) |\n|-------------------------------|-------|:------------------------:|:-------------------------:|:-------------------------:|\n| Baseline(ddim) | 16.34 | 41.62 | 49.8 | 58.14 |\n| Ours (ddim) | 16.48 | 23.76 | 33.58 | 43.60 |\n| Baseline (iddpm) | 16.83 | 45.54 | 53.74 | 61.82 | \n| Ours (iddpm) | 16.46 | 26.94 | 37.70 | 47.94 | \n| Baseline (ddpm (sde sampler)) | 16.73 | 42.37 | 51.91 | 59.48 |\n| Ours (ddpm (sde sampler)) | 16.35 | 24.34 | 36.28 | 46.04 |\n\nIn the light of these new experiments and clarifications, we hope that the Reviewer will support the acceptance of our submission.", "claims_and_evidence": "Most of their claims are supported by evidence. But I found some claims confusing:\n* In the title of Figure 3, why does the \"blurry\" output stand for generalization? Can we view this as a quality degradation? (The authors have addressed this on their rebuttal.)", "methods_and_evaluation_criteria": "I think the proposed method and the evaluation make sense for investigating the memorization of the diffusion model. Specifically, they validate the method on both the unconditional generation task and text-conditioned generation. The CIFAR-10, FFHQ, (tiny) ImageNet, and LAION are used to test the memorization.", "theoretical_claims": "No, I didn't carefully check the proof in the appendix. But their explanation in the main paper make sense and they provide some intuition to help me understand.", "experimental_designs_or_analyses": "Yes, I reviewed the algorithm and the experimental results.", "relation_to_broader_scientific_literature": "The proposed method involves two parts:\n* For t>tn, they train the diffusion model using the Ambient Score Matching loss (Daras et al., 2024b)\n* For t<tn, they train the diffusion model using DDPM.\nIt seems that you combine two existing methods and apply them on different noisy scales. Could you reclaim your algorithm novelty?", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Please refer to other parts.", "other_comments_or_suggestions": "I think the authors should have more discussion on related work in mitigating the memorization of diffusion models. There have been many interesting works in mitigating memorization, and you should discuss them in your work, although you have different techniques with them.\n\nSome works in understanding and/or mitigating memorization that I know:\n* On the Interpolation Effect of Score Smoothing. https://arxiv.org/abs/2502.19499\n* Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention. https://arxiv.org/abs/2403.11052v1\n* Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization. https://arxiv.org/abs/2502.03435\n\nBesides, I also suggest that you compare your method with those baselines in your experiments to validate your superiority.", "questions_for_authors": "Can your method be integrated with other diffusion methods such as EDM and DDIM? If yes, I would strongly recommend the authors to also apply your method on those diffusion piplines and verify the performance gain.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AjSGFq9B5V", "reviewer": "Reviewer_Einp", "summary": "The paper investigates the links between good performance of generative models (i.e., low FID) and memorization of the training set. in particular, the paper investigates the Pareto front. of performance vs memorization. Based on a recent \"ambient score matching loss\" [1], the paper introduces a new diffusion technique with good performance and less memorization.\n\n\n[1] DARAS, Giannis, DIMAKIS, Alexandros G., et DASKALAKIS, Constantinos. Consistent diffusion meets tweedie: Training exact ambient diffusion models with noisy data. arXiv preprint ICML 2024", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable reviews and suggestions!\n\n**On no of duplicates in figure 2**: By the no of duplicates, we mean the percentage of generated samples whose similarity to their nearest neighbor in the training set is greater than 0.9. The nearest neighbor is found using the cosine similarity in the DINO space.\n\nAs per the reviewer’s request, we also calculated the mean and median of the similarity in the same setting as Figure 1 (300 training samples from FFHQ). We will include these results in the updated version.\n\n| Method | Mean | Median |\n|------------------------|:-------:|:-------:|\n| Baseline | 0.8826 | 0.8931 |\n| Ours ($\\sigma=0.1$) | 0.8854 | 0.8748 |\n| Ours ($\\sigma=0.25$) | 0.8518 | 0.8491 |\n| Ours ($\\sigma=0.3$) | 0.8473 | 0.8475 |\n| Ours ($\\sigma=0.4$) | 0.8287 | 0.8292 | \n| Ours ($\\sigma=0.5$) | 0.8060 | 0.8069 |\n\n**On FID using Dinov2**:  As per the reviewer’s suggestion, we performed additional experiments to evaluate the quality of our model with FID using Dinov2. We will add these results with a plot in the updated version.\n\n| Method | FID | FD_{DinoV2} |\n|------------------------|:-------:|:-------:|\n| Baseline | 16.21 | 353.19 |\n| Ours ($\\sigma = 0.1$) | 14.94 | 347.21 |\n| Ours ($\\sigma = 0.25$) | 15.05 | 344.60 |\n| Ours ($\\sigma = 0.3$) | 16.14 | 358.23 |\n| Ours ($\\sigma = 0.4$) | 19.55 | 371.81 |\n| Ours ($\\sigma = 0.5$) | 23.73 | 382.03 |\n\n**On clarification about the sampling process**: Our method is a training-time mitigation strategy, and therefore, no modifications are made during the sampling. For $t\\geq t_n$, we replace the original dataset with a noisy version of it where each datapoint has been replaced (*once*) with a noisy realization. This step corresponds to the creation of $S_{t_n}$ at step 1 of Algorithm 1 (Appendix page 12). The creation of $S_{t_n}$ is very important as it ensures the reduced information about the clean distribution for the learning at $t\\geq t_n$ (see Lemma 4.2). This step leads to decreased memorization. For $t\\leq t_n$, we train as usual. We will clarify this in the updated version.\n\n**On theoretical results**: We will do our best to improve the presentation of our results, using the one additional page for the Camera Ready. Here we provide an intuitive explanation of the results, and will include such a discussion in the updated version to improve the readability. \n\nLemma 4.1 proves that the optimal score learned at time $t_n$ is a gradient field that points towards the noisy training set. This lemma is formal and can be seen as the analogue of the optimal score solution for standard DDPM in a noiseless dataset. \n\nLemma 4.2 motivates why this solution ``leaks’’ less information at time $t_n$ compared to running vanilla DDPM from time $T$ until time $t_n$. This property is formalized using the mutual information. Additionally, we needed the Gaussianity assumption to make it formal since for the Gaussian, we can derive explicit expressions for mutual information.\n\nSection 4.2 explores connections between our work and the work of Feldman, which focuses on the multiclass classification. Theorem 4.3 presents the analogue of Feldman’s result with a more general loss function. However, the message is essentially the same: the more heavy-tailed the distribution of the frequencies, the higher the penalty of not memorizing (in our setting, memorization corresponds to making the score objective small). This is formally presented in Lemma 4.4, which comes from the work of Feldman and shows that the penalty term \\tau_1 depends on the tail of the distribution of the mixing coefficients. Finally, we illustrate in Lemma 4.5 that the noise scale of the diffusion process affects the parameter \\tau_1 since, as the noise increases, the subpopulations merge and the mixing coefficients add up (hence becoming lighter). To make this intuitive argument fully formal, we provide the result for the most standard mixture model, namely Gaussian mixture, and the result should extend to any mixture family since noise addition is a contracting operation. \n\n**On larger datasets**: For the unconditional image generation setting, memorization becomes negligible for larger datasets. Per Reviewer’s Request, we trained on a slightly bigger dataset. For FFHQ 5k samples, we get the following results:\n\n| Method | FID | S > 0.85 | S > 0.875 | S > 0.9 |\n|---------|:-----:|:---------:|:---------:|:---------:|\n| Baseline | 6.4 | 21.58 | 4.98 | 0.46 |\n| Ours | 6.4 | 20.08 | 4.53 | 0.42 |\n\nAs seen, our method is still a little better in terms of memorization with the same FID as the Baseline, but the improvement is smaller compared to more data-limited settings.\n\nFor larger datasets, memorization is still an issue if there is text-conditioning. Our results in Section 5.2. show improvements over the baselines for this regime as well.\n\nWe hope our rebuttal clarifies the Reviewer's concerns and that the Reviewer will strongly support the acceptance of our work.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "In figure 2, how exactly do you define the \"number of duplicates\"? In [2] they use the cosine similarities between the synthetic image and the closest one from the training set. Maybe the mean/median of this cosine similarity could be a better than \"number of duplicates\", as done in hte experiment section as I understand.\n- Would it be possible to have the same kind of plot with the Dinov2 FID between the generated set and the test set?\n\n[2] Kadkhodaie, Zahra, et al. \"Generalization in diffusion models arises from geometry-adaptive harmonic representations.\" ICLR (2024)\n\n\n- **I did not understand the proposed algorithm**. How does the sampling process work exactly? You have 2 different time scales depending on how advanced the denoising process is.", "theoretical_claims": "- IMO the theoretical results lacks formality: I was not able to follow Section 4. It might be useful to properly encapsulate in an environment and explain one highlighted result.", "experimental_designs_or_analyses": "In the experimental results, the dataset ranges from 300 to 3k, does the proposed method also provide improvement for larger datasets? I.e. with the full dataset?", "relation_to_broader_scientific_literature": "Good", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "NA", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NgBAkijt3k", "reviewer": "Reviewer_pCuc", "summary": "This paper addresses the issue of memorization in diffusion models, proposing a framework to reduce memorization while maintaining high-quality image generation. The authors introduce a simple method that utilizes noisy images to learn the initial portion of the generation trajectory, followed by high-quality images for the final portion. This approach effectively prevents memorization when forming the high-level structure of the generated image, while still allowing for high-quality, detailed features from the training images.\n- The authors support their method with theoretical results that quantify the information leakage from the training set, demonstrating it to be smaller than standard approaches. They also extend existing results on memorization and generalization error from the classification literature to the context of diffusion models. \n- They also empirically validate their approach, showing that it achieves a better tradeoff between memorization and quality compared to standard DDPM, and does a better job at preserving the quality of generated images compared to existing approaches for preventing memorization in diffusion models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for careful reading, for their feedback and suggestions!\n\n**Lemma 4.2 being about a single point**: We preferred to present the statement for 1 point since it already presents the advantage of ambient diffusion compared to vanilla DDPM: Given a generation of m points from each of the two models, the mutual information is larger. We agree with the comment that a larger training set is desirable, and this is why we present such a result in the Appendix (we refer to it right after Lemma 4.2). \n\n**On the difference from Feldman’s work**: We believe that our main contribution in this part is the general observation that the tail of the distribution of the frequencies depends on the noise scale. This observation is specific to the way that diffusion models are trained and does not appear in Feldman’s work because it focuses on the multiclass classification. To make this observation more rigorous, we adapted Feldman’s framework with a more general loss function (which should be independent of the mixing coefficients): in fact, this adaptation is not technically challenging and we do not claim novel technical challenges; it is instead a generalization of the previous proof. However, the novelty comes from a conceptual perspective. As the noise level increases, the penalty \\tau_1 in the Informal Theorem 1 is decreased because the tails become lighter. \n\nTo summarize, this part of the paper illustrates our conceptual contribution on why for the high-noise regime, memorization could be avoided.\n\n**On the readability of appendix and main body**: We thank the reviewer for that comment. Indeed, the lack of space made the presentation a bit dense. We will use the extra page in the Camera Ready to improve the presentation of the work.\n\nWe hope that our response addresses any remaining concerns and that the Reviewer will support the acceptance of our submission. We remain at the Reviewer's availability in case there are additional concerns to be addressed.", "claims_and_evidence": "- The claims made in the submission are generally supported by clear evidence. The authors provide theoretical results, including Lemmas 4.1 and 4.2, which characterize the sampling distribution and compare mutual information between their method and standard diffusion. I have read the proofs of Lemmas 4.1, 4.2, and Theorem 4.3, and they seemed sound. \n- The experimental results demonstrate the effectiveness of their approach in reducing memorization while maintaining image quality, as measured by FID scores and similarity metrics.\n- While their theory does not completely speak to the success/guarantees of their particular method (for instance the information leakage guarantees hold only for ambient diffusion, and not their two-stage concatenated method), the authors are careful to point out these limitations.", "methods_and_evaluation_criteria": "The methods and evaluation criteria appear appropriate and well-justified for the problem at hand. The authors use established metrics such as FID for image quality and nearest neighbor similarity scores for measuring memorization. The authors explore the effects of different t_n values and also compare against existing baselines for memorization prevention, in both cases looking at datasets of varying sizes, thus providing a thorough evaluation of the performance of their method.", "theoretical_claims": "- The information leakage result (Lemmas 4.1/4.2) is a nice accompaniment for the proposed method, though I found it a bit unclear at first that the statement is about only a single point, and would have preferred that to be made more clear before the lemma was presented. \n- The results in Section 4.2 make sense, but upon inspection of the proofs, it doesn't seem like they use anything specific to diffusion models. It would be useful for the authors to comment on if/how the proofs are substantially different from Feldman's results. \n- In a similar vein, I found the statement of Theorem 4.3/B.2 to be a little hard to parse at first, especially when trying to compare to the analogous statement from Feldman, which compares the error to the optimal generalization error. Based on my understanding, the result implies this, but I think it might make it clear to add some discussion and/or changing how the theorem is presented.", "experimental_designs_or_analyses": "The experimental designs appear sound, thorough, and well-explained. The authors thoroughly investigate the memorization-quality tradeoff of their method along a number of axes, i.e. tuning the t_n parameter, varying dataset size, comparison to existing anti-memorization baselines, and extending their method to text-conditional generation.", "relation_to_broader_scientific_literature": "This work builds upon and extends existing literature on memorization in machine learning, particularly Feldman's work on heavy-tailed distributions. It also contributes to the growing body of research on diffusion models and their properties. It is notable that their algorithms produce state-of-art results in terms of the trade-off between memorization and quality for both image generation and text-conditional generation compared to prior work.", "essential_references_not_discussed": "I am not aware of any essential references that are missing.", "other_strengths_and_weaknesses": "- For the most part, the paper is clearly written and easy to follow. \n- I appreciate that their approach is simple, yet achieves convincing improvements wrt the tradeoff between memorization and quality. \n- I found the appendix discussion of the theoretical results far easier to follow than the more informal discussion presented in the main portion. I understand that this may be due to a space constraint, but it would be worth taking a second look at the main portion to see how the discussion can be clarified.", "other_comments_or_suggestions": "- Several instances of straight quotes (\") instead of proper LaTeX quotes (`` and '') appear throughout the paper, resulting in backwards-facing opening quotes. \n- There are some grammatical issues that should be addressed, such as the phrase \"How much heavy-tailed is the distribution\" on lines 325-326.\n- Lemma D.1 seems to have a typo as I assume the statement should be about I(A;S) rather than I(D;S).", "questions_for_authors": "- As noted above, I would appreciate hearing from the authors in what ways the analysis presented in Section 4.2 is substantially different from [Feldman, 2020]", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iqJPfrqNxI", "reviewer": "Reviewer_SVEz", "summary": "This paper explores the use of noisy images to mitigate memorization issues in diffusion models, examining the tradeoff between fidelity and memorization. Under the assumption of normality in the data distribution, the authors analyze the problem of information leakage. They also investigate the memorization issue within a framework similar to that proposed by Feldman (2020). Empirical studies are conducted to better understand the fidelity–memorization tradeoff by introducing noise at different scales to the samples.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "From the questions raised, it appears that there are some major misunderstandings. In what follows, we do our best to clarify them, and we urge the reviewer to reread some part of our work and reassess their evaluation. In our end, we will do our best to improve the presentation based on the Reviewer's feedback.\n\n**\"Have you used any clean images?\"**: Yes, we use clean data but only for times $t\\leq t_n$ of the diffusion, where $t_n$ is a hyperparameter to be controlled. For times $t\\geq t_n$, we only use corrupted data and the Ambient Score Matching objective. The corruption to time $t_n$ happens once, before training, to ensure that there is no information leakage about the clean images at these times.\n\n**\"what is S?\"**: The meaning of S is explained on page 7, Lines 377-382 in the first column. To reiterate, for each generated image, we measure its similarity, S, (inner product), with the nearest neighbor in the dataset. Tables 1 (and Table 2) report percentages of the dataset that have similarity S greater than certain thresholds. In our rebuttal, we further report mean and median values -- see response to Reviewer Einp.\n\n**\"the relationship between FID and memorization issues and the selections of $t_n$ remains unclear\"**. Figure 1 shows the achieved FID and memorization for different values of $t_n$. There is an extended discussion on the choice of $t_n$ on page 7 of our submission.\n\n**\"but the true values of t_n are not provided\"**: this is a critical misunderstanding. We have access to clean data, and we intentionally corrupt it for some diffusion times to avoid memorization. Please refer to Algorithm 1 and Section 3 (Method, page 4) of our submission, where we verbally explain this algorithm. See also Choices for $t_n$ on page 7. \n\n**About differential privacy-based methods**: This is a great remark. We will add this discussion. Briefly, the notion of privacy is stronger than the notion of memorization. Privacy guarantees would mean that any adversary that has access to the model could not extract samples used for the training of that model. Our method does not guarantee that: we show that sampling with our model leads to outputs that resemble less to the training points. That does not exclude the possibility that an adversary with access to our model could recover the training points. Since we have relaxed expectations/guarantees, we also enjoy high performance. \n\n**On the Gaussian assumption in Lemma 4.2**: The goal of Lemma 4.2 is to motivate why the ambient optimal score solution ``leaks” less information at time $t_n$ compared to running vanilla DDPM from time $T$ until time $t_n$. While we believe that it holds for more general distributions, generalizing the result comes with some technical challenges and does not add much additional value to our goal of motivating why the ambient optimal score solution leaks less information.  \n\n**Section 4.2**: Building a theoretical framework for understanding memorization in machine learning is quite challenging, and there is limited amount of work trying to address it. The seminal work of Feldman provides a natural instance (with some arguable caveats) of the learning problem where optimal classification requires memorization. \n\nOur work aims to take a step in understanding memorization in diffusion models, not by solely using Feldman’s framework but by taking a two-step approach: \n\nFirst, we adapt the setting of Feldman (which is a well-accepted) to more general loss functions (capturing e.g., standard score matching losses). The message of this general framework is roughly speaking the following: if the distribution of the frequencies of the subpopulations is heavy-tailed, then to achieve optimal generalization loss, the algorithm has to make the loss at all the points seen exactly once zero; otherwise, it pays a penalty \\tau_1 for any not fitted point.\n\nSecond, to make use of this general framework, we rely on how diffusion models are trained. We view the above general framework in many noise scales. As we explain e.g., in Lemma 4.5, even if the original dataset has the heavy-tailed structure of Feldman’s model, as the noise scale increases (which is the core idea of diffusion-based generation), the tails will become lighter since the populations start to merge. This is where our framework is specialized in diffusion models and not in Theorem 4.3. \n\nCombining the above two observations, the theoretical part of Section 4.2 gives evidence that avoiding memorization in the high-noise regime is feasible. Our main contribution in this part is the general observation that the tail of the distribution of the frequencies depends on the noise scale, and this is a property that depends crucially on how diffusion models are trained. This illustrates our conceptual contribution on why, for the high-noise regime, memorization could be avoided.\n\nWe hope this rebuttal clarifies things and we remain at the Reviewer's availability if there are further concerns.", "claims_and_evidence": "All the theoretical results are accompanied by proofs. And the empirical studies' conclusions are aligned with the provided empirical results.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I checked those in section 4.1, which seem right. I am not familiar with the framework proposed by Feldman in section 4.2.1 but the results seem to be aligned with the intuition.", "experimental_designs_or_analyses": "The results in Table 1 are not well explained. For example, what is S. And for S>0.9, are they percentages of samples above the threshold? How are they computed? In addition, you mentioned that you played with t_n to obtain comparable FIDs of DDPM and yours, but the true values of t_n are not provided. In addition, could you also include more details on how the models are trained? E.g., have you used any clean images, etc. Currently, the relationship between FID and memorization issues and the selections of t_n remains unclear. Also, the Pareto frontier is not defined in the paper. Overall, it seems to me that the empirical studies are not comprehensive enough to give readers a good understanding of what the authors tried to present. Maybe more raw data and discussions on them should be made.", "relation_to_broader_scientific_literature": "While the model is trying to address the memorization problem of the diffusion models, the work can be framed as using corrupted images to train a diffusion model to estimate the ground truth data distributions. This problem can be seen as a special case of the more general deconvolution theory in statistics. \n\nAlthough not discussed in this paper, the memorization problem can also be tackled through the differential privacy-based method.", "essential_references_not_discussed": "It would be beneficial to include some work discussing the use of differential privacy-based methods to train generative models.", "other_strengths_and_weaknesses": "Strengths: Solving the memorization problems of the generative models is crucial. Solving this problem can potentially address copyright infringement problem, data providers' privacy concerns, etc. \n\n\nWeakness:\n1. I do not think the paper is well-organized and gets its theoretical results hinged with its empirical studies. The authors should have provided more intuitive descriptions of their theoretical work and how they together justify training diffusion models using generative models to tackle the memorization problems. In the current version, this kind of discussion is lacking, and it is difficult for the readers to see how the theoretical results are connected and how the empirical results support the theoretical claims. \n\n2. Regarding the theoretical work, Lem 4.2 assumes that x0 ~ N(mu, sigma), which might be too strong. While I understand that the proof might be very challenging for a general data distribution, the authors should discuss why the current selection is sufficient to give a good understanding of the general case. \n\n3. For the results in section 4.2, the assumed settings are not justified. In particular, I do not see the benefits of a mix of N distributions in understanding the memorization problem of the diffusion models. Besides, the derived results are not tailored for the diffusion model (i.e. the losses); as a result, it is somewhat questionable to what extent the theoretical results could characterize the diffusion models' memorization problem.", "other_comments_or_suggestions": "1. Eq (5) has the leading term -x. It seems like some discounting factor is missing. \n2. The appearance of the formula at the bottom right of page 3 should be optimized. \n3. Alg 1 should be included in the main text. (or a concise version should be included at least.)\n4. Section 4.2 could be reorganized. Currently, readers might find it challenging to understand the main purpose of the section, even if a section overview has been provided.\n5. Table 1 and Table 2 require more clarification.", "questions_for_authors": "Please the Other Strengths And Weaknesses section.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper tackles memorization in diffusion models by proposing a framework that preserves generation quality while mitigating overfitting. The authors introduce a simple strategy that uses noisy images to guide the early stages of generation and high-quality images for the final stages. This approach prevents memorization of high-level structures while still capturing fine-grained details from the training data.

Most reviewers find the work well presented and interesting, which studies an important problem of memorization in diffusion models.
The paper is well supported by both empirical and theoretical studies. Please incorporate the reviewers' feedback into the revision.
Decision: Accept (poster)