On the Stability of Iterative Retraining of Generative Models on their own Data
Authors: Quentin Bertrand
Joey Bose
Alexandre Duplessis
Marco Jiralerspong
Gauthier Gidel
Venue: ICLR 2024 spotlight (2024)
Keywords: Generative Models
Iterative Training
Diffusion
Primary area: generative models
TL;DR: We provide a theoretical framework to study the behavior of iterative retraining of generative models on their own data

Abstract: Deep generative models have made tremendous progress in modeling complex data, often exhibiting generation quality that surpasses a typical human's ability to discern the authenticity of samples. Undeniably, a key driver of this success is enabled by the massive amounts of web-scale data consumed by these models. Due to these models' striking performance and ease of availability, the web will inevitably be increasingly populated with synthetic content. Such a fact directly implies that future iterations of generative models will be trained on both clean and artificially generated data from past models. In this paper, we develop a framework to rigorously study the impact of training generative models on mixed datasets---from classical training on real data to self-consuming generative models trained on purely synthetic data. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough and the proportion of clean training data (w.r.t. synthetic data) is large enough. We empirically validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the-art diffusion models on CIFAR10 and FFHQ.

Published as a conference paper at ICLR 2024
ON THE STABILITY OF ITERATIVE RETRAINING OF GENERATIVE MODELS ON THEIR OWN DATA
Quentin Bertrand*
Avishek (Joey) Bose McGill and Mila
Université de Montréal and Mila
Alexandre Duplessis ENS, PSL
Marco Jiralerspong
Gauthier Gidel Université de Montréal and Mila
Université de Montréal and Mila
ABSTRACT
Deep generative models have made tremendous progress in modeling complex data, often exhibiting generation quality that surpasses typical human's 's ability to dis- cern the authenticity of samples. Undeniably, key driver of this success is enabled by the massive amounts of web-scale data consumed by these models. Due to these models' striking performance and ease of availability, the web will inevitably in- creasingly populated with synthetic content. Such fact directly implies that future iterations generative models will be trained on both clean and artificially gen- erated data from past models. In this paper, develop framework to rigorously study the impact training generative models on mixed datasets-fron classical training real data self-consuming generative models trained on purely syn- thetic data. We first prove the stability of iterative training under the condition that the initial generative models approximate the data distribution well enough and the proportion of clean training data (w.rt. synthetic data) is large enough. We empiri- cally validate our theory on both synthetic and natural images by iteratively training normalizing flows and state-of-the sere--ofthh-arr art diffusion models on CIFAR10 and FFHQ.
Fully synth.
Half' synth.
No synth.
(a) 0 retrain.
(b) 5 retrain.
(c) 10 retrain.
(d) 15 retrain.
(e) 20 retrain.
Figure 1: Samples generated from EDM trained on the FFHQ dataset. As observed in Shumailov al. (2023); Alemohammad al. (2023), iteratively retraining the model exclusively on own generated data yields degradation of the image (top row). On the other hand, retraining mix of half real and half synthetic data (middle) yields a similar quality as retraining on real data (bottom).
"Coresponding authors: ltw.eiin..eettrrdiiil...co
Canada Cifar AT Chair

---

Published as a conference paper at ICLR 2024
INTRODUCTION
One of the central promises of generative modeling is the ability to generate unlimited synthetic data indistinguishable from the original data distribution. Access such high-fidelity synthetic data has been one of the key ingredients powering numerous machine learning use cases such as data augmentation in supervised learning (Antreas et al., 2018), de novo protein design (Watson et al., 2023), and text-generated responses to prompts that enable suite of natural language processing applications (Dong et al., 2022). Indeed, the scale and power of these models-in particular large language models (Brown et al., 2020; OpenAI, 2023; Chowdhery et al., 2022; Anil et al., 2023)-have have even led to significant progress, often exceeding expert expectations (Steinhardt, 2022), in challenging domains like mathematical reasoning, code assistance (Bubeck et al., 2023).
In addition to the dramatic increase in computational resources, a key driver of progress has been the amount and the availability of high-quality training data (Kaplan al., 2020). As result, current LLMs and text-conditioned diffusion models like DALL-E (Ramesh et al., 2021), Stable Diffusion (Stability AI, 2023), Midjourney (Midjourney, 2023) are trained on web-scale datasets that already potentially exhaust all the available clean data on the internet. Furthermore, a growing proportion of synthetic data from existing deep generative models continues to populate the web to the point that even existing web-scale datasets are known to contain generated content (Schuhmann et al., 2022). While detecting generated content in public datasets is a challenging problem in its own right (Mitchell et al., 2023; Sadasivan et al., 2023), practitioners training future iterations of deep generative models must first contend with the reality that their training datasets alreudy contain synthetic data. This raises the following fundamental question:
Does training on mixed datasets finite real data and self-generated data alter performance?
Our Contributions. We study the iterative retraining of deep generative models on mixed datasets composed of clean real data and synthetic data generated from the current generative model itself. In particular, we propose theoretical framework for iterative retraining of generative models that builds upon the standard maximum likelihood objective, extending it to retraining on mixed datasets. encompasses many generative models of interest such as VAEs (Kingma and Welling, 2019), normal- izing flows (Rezende and Mohamed, 2015), and state-of-the-an diffusion models (Song et al., 2021).
Using theoretical framework, models by proving the existence of a fixed point (Theorem 1) under the following conditions: 1.) The first iteration generative model is sufficiently "well-trained" and 2.) Each retraining iteration keeps a high enough proportion of the original clean data. Moreover, we provide theoretical (Proposition 1) and empirical (Figure 1) evidence that failure to satisfy these conditions can lead to model collapse (i.e., iterative retraining leads the generative model to collapse to outputting single point). We then prove in Theorem that, with high probability, iterative retraining remains within neighborhood of the optimal generative model in parameter space when working in the stable regime. Finally, we substantiate our theory on both synthetic datasets and high dimensional natural images on a broad category of models that include continuous normalizing flows (Chen et al., 2018 8) constructed using a conditional Row-matching objective (OTCFM, Tong et al. 2023), Denoising Diffusion Probabilistic Models (DDPM, Ho et al. 2020) and Elucidating Diffusion Models (EDM, Karras et al. 2022).
show the stability of iterative retraining of deep generative
We summarize the main contributions of this paper below:
We provide theoretical framework to study the stability of the iterative retraining of likelihood- based generative models on mixed datasets containing relf-generated and real data.
Under mild regularity condition on the density of the considered generative models, we prove the stability of iterative retraining of generative models under the condition that the initial generative model is close enough to the real data distribution and that the proportion of real data is sufficiently large (Theorems and 2) during each iterative retraining procedure.
We empirically validate our theory through iterative retraining on CIFAR10 and FFHQ using powerful diffusion models in OTCFM, DDPM, and EDM.
2 ITERATIVE RETRAINING OF GENERATIVE MODELS
Notation and convention. Let Pdata E P(R)) be a probability distribution over Rd. An empirical
distribution sampled from Paata in the form of training dataset, Dreat (x:) is denoted Pdata-
2

---

Published as a conference paper at ICLR 2024
Algorithm 1 Iterative Retraining of Generative Models
input .Deaal == }= A // True data, learning procedure (c.g., Eq. 2) param: T. À // Nunber of retraining iterations, proportion of gen, data 20, = -A(Dreal) // Learn generative model on true data for t in 1, ....., T do
| with &; ~ Po... // Sample [A-n] synthetic data points Dayuth ((DrrelUDgye syuth) // Learn generative model on synthetic and true data pe, return po.
We write pe to indicate a generative model with parameters 0 while θ is the entire parameter space. A synthetic dataset sampled from a generative model is written as Daynth The optimal set parameters and its associated distribution are denoted as 0 and po-. We use subscripts, e.g., 0,, Dr, to denote the number of iterative retraining steps, and superseripts to indicate the number of samples used, e.g, 0 generative model pe, trained on Pdata. The asymptotic convergence notation th, = O(p²) where p > O, refers to We > 0, 3C > such that u, S C(p + +)*, Vt 2 0, i.e., 4t almost converges at a linear rate p²_ The Euclidean norm of vectors and the spectral norm of matrices is denoted ||1|2. Let S+ be the set of symmetric positive definite matrices, for S1 and S2 E S+, S 2 S2 if S - S2 ビ O.
uses n training samples at iteration L. Moreover, we use the convention that 0, is the initial
2.1 PRELIMINARIES
The goal of generative modeling is to approximate Pdata using a parametric model pg. For likelihood-based models, this corresponds to maximizing the likelihood of paata. which is an empirical sampling of data points from Paata and leads to the following optimization objective:
O argmax Ex-opdata [log po (x)]
(1)
d'c@
Note that the argmax operator defines set of equally good solutions to Equation (1). We follow the convention of selecting 0 = argmingree, ||0'| as the canonical choice. After obtaining the trained generative model pe.. we are free to sample from it and create a synthetic dataset |A-al where x² ~ pe., and À >> 0 is hyperparameter that controls the relative size of Deyuth with respect to Dreal- We can then update our initial generative model on any combination of real data Pdata and synthetic data drawn from pe, obtain po, and repeat this process ad infinitum. We term this procedure iterative retraining of a generative model (Algorithm 1). In the context of maximum likelihood, the iterative retraining update can be formally written as:
Deyuth (x())_1 J==1
0 local-argmas Exxxpaan log po (x)) + AER~re [log po (x)] @'e@
The notation local-argmax corresponds to any local maximizer of the loss considered." | The formulation Equation (2) precisely corresponds to training model on the empirical dataset Dreal U Deyoth of size [n + A-J.. Note that 1/+1 may be a set; we thus need to provide a tie-breaking rule pick 0 E Oz Since most training methods are local search method, we will select the closest solution the previous iterate 0
0 9 (0
argmin 8'e87
100' 0, 10'-0,11
Setting A O corresponds to only sampling from Pdata and is thus equivalent to training pe, for more iterations. The main focus of this work is studying the setting where À > 0 and, more specifically, analyzing the discrete sequence of learned parameters 0% - 0 + - 0 and thus their induced distributions and characterize the behavior of the generative model in reference to Pdata As result, this means that iterative retraining at timestep & 1-i.e, 07 9 resumes from the previous iterate 0;- In practice, this amounts to finetuning the model rather than retraining from scratch.
Given function f continuously twice differentiable, sufficient condition for locally maximize f (0) 0 and f() - Forr simplicity of presentation, assume that such 0.+1 is uniquely defined which will be the case under the assumptions of Theorems I and 2.
3

---

Published as a conference paper at ICLR 2024
2.2 WARM-UP WITH MULTIVARIATE GAUSSIAN
As warm-up, we consider multivariate Gaussian with parameters N₁ obtained from being fit on Pdata ~N No. This example corresponds to special case of Equation (2), where + oo (i.e. we do not reuse any training data). We analyze the evolution of the model parameters when iteratively retraining on its own samples. For each retraining, multivariate Gaussian model learned from finite samples. In the case of single multivariate Gaussian, the unbiased update of the mean and covariance parameters has a closed-form formula. For all 2
Sampling step:
{ = Mr / E with by
iid. No.1, 1 isisn.
(4)
Learning step:
|1+1 = - E+11 = - ( (xf (x/-Hei)
M4+1)
Proposition states that retraining a multivariate Gaussian only on its generated data leads to its col- lapse: its covariance matrix vanishes to O linearly as function of the number of retraining.
Proposition 1. (Gaussian Collapse) For all initializations of the mean po and the covariance En, iteratively learning a single multivariate Gaussian solely its generated data yields model collapse. More precisely, if b4 and En follows Equations (4) and (5), then, there exists a < 1,
B(VE, Σ.) I a Eg
(6)
The proof Proposition is provided in Appendix A. Interestingly, the convergence rate on goes to 1 as the number of samples n goes to infinity. In other words, the larger the number of samples, the slower the model collapses. Proposition 1 is a generalization of Shumailov et al. (2023); Alemohammad et al. (2023), who respectively proved convergence to 0 of the standard deviation of a one-dimensional Gaussian model, and the covariance of a single multivariate Gaussian, without rates of convergence. While Proposition states that learning simple generative model retrained only on its own output yields model collapse, in Section 3, we show that if a sufficient proportion of real data is used for retraining the generative models, then Algorithm is stable.
3 STABILITY OF ITERATIVE RETRAINING
We seek to characterize the stability behavior of iterative retraining of generative models on datasets that contain mixture of original clean data as well as self-generated synthetic samples. As deep generative models are parametric models, their performance is primarily affected by two sources of error: 1.) statistical approximation error and 2.) function approximation error. The first source of error occurs due to the sampling bias of using Pdata rather than Pdata- For instance, due to finite sampling, Pdata might not contain modes of the true distribution Pdata. leading to an imperfect generative model irrespective of its expressive power. The function approximation error chiefly arises due to mismatch of the model class achievable by 0. and consequently the induced distribution pe and Pdata-
Recent advances in generative modeling theory have proven the universal approximation abilities of popular model classes, e.g., normalizing flows (Teshima et al., 2020) and diffusion models (Oko et al., 2023). In practice, state-of-the-art. generative models such StableDiffusson (Stability AI, 2023) and MidJourney (Midjourney, 2023) exhibit impressive qualitative generative ability which dampens the practical concern of the magnitude of the function approximation error in generative models. Consequently, we structure our investigation by first examining the stability of iterative training in the absence of statistical error in Section 3.1, which an idealized setting. In Section 3.2, we study the stability of iterative retraining of practical generative models where the statistical error is non-zero, which is reflective of the setting and use cases of actual SOTA generative models the wild.
3.1 ITERATIVE RETRAINING UNDER NO STATISTICAL ERROR
We first study the behavior of iterative retraining under the setting of no statistical error. As the primary source of statistical error comes from using Pdata we may simply assume full access paata
4

---

Published as a conference paper at ICLR 2024
instead and infinite sampling budget. In this setting, define the optimal generative model pe. with parameters 0 as the solution to the following optimization problem.
l E argmax Expepaata log po px)(
0'c0
Compared to Equation (1), infinitely many samples are drawn from pdata Equation (7); hence, there is no statistical error due to finite sampling of the dataset. As the capacity of the model class (pe ( is limited, the maximum likelihood estimator pa- does not exactly correspond to the data distribution Pdata We note is the Wasserstein distance (Villani et al., 2009) between Pø- and Pdata
dw (pg- Pdata) =
sup fE(Lip(()$1
Ex-po: [f(x]] - Ex-
(8)
where (Lip(f) < 1 is the set of I-Lipschitz functions. We assume the Hessian of the maximum log-likelihood from Equation (7) has some regularity and 0 is strict local maximum.
Assumption 1. For 0 close enough to 0", the mapping
V po()) is L-Lipschitz.
We use this assumption to show that if pę- is close to Pdata (i.e., small), then the Hessians of the maximum likelihood with respectively real and synthetic data close to each other.
Assumption 2.
The mapping 0 - Ex-pasta [lo po (x)] is continuously twice differentiable locally around 0 and Ex-ppiata V lo Pa- (x) -II < 0.
Assumption 2 imposes local strong concavity around the local maximum 0- and Assumption implies Hessian Lipschitzness when close to 0*. For example, Assumptions and are valid for the Gaussian generative model described in Section 2.2.
Proposition Let po x + (x) c- v(x-x= E I (xx-1(((//)))) / (2x) Σ) with μ E Rd, Σ 0. (n, "),, then (xxx Vi (x)) x and + V log pe (x) 1-Lipschitz.
A direct consequence of the proposition is that the class of generative models (N₂,)) follows As- sumptions and 2. Thus, it proves that under these assumptions, finite À is necessary for the stability of iterative retraining. Now, we study what conditions sufficient for the stability of iterative retraining with À 0, i.e., training by sampling from
I#X + 14xPe- which corresponds to,
gx ( local-argmax Exxppata [log po (x) +λ Ex-po Dog po (x))
0
:=H;(0")
(0.00)
ties between the maximizers are broken as Equation (3), we can show that the solution of Equation (9) is locally unique, ensuring that G (0) is well defined and differentiable.
Propasition 3. (The Local Maximum Likelihood Solution is Unique) Let 0 be solution of Equation (7) that satisfies Assumptions and eL < a, then for all and 0 in a small enough neighborhood V around 0*, there exists unique local maximizer goo (0) in U.
The proof of Proposition is provided in Appendix C. Interestingly, the solution 0 of the maximum infinite sample likelihood Equation (7) is fixed point of the infinite sample iterative retraining procedure via maximum likelihood in Equation (9).
Formally, 0i-
(0) 30=aagae argmingrca ||0' 01), 0 local-arggaaggee (0,000')

---

Published as a conference paper at ICLR 2024
Proposition 4. (The Optimal Parametric Generative Model a Fixed Point) For a given data distribution Pdata any 0 solution of Equation (7), and for all X >> 0 we have,
0 -9x (0*)
(10)
The proof of Proposition 4 can be found in Appendix D. In the zero statistical eтror case, 0* corresponds to the perfectly trained generative model obtained after the first iteration of training exclu- sively on real data (see Equation (1)). The main question the stability of iterative retraining around r Even with Pdata. one can consider small sources of error such as optimization error numerical error floating point precision leading to an initial 0n R 0 only approximately solving Equation (7).
For our stability analysis, given 0, we decompose the maximum likelihood loss as Hi (0,00)) := Hi (0') + XH2 0¹) := Ex-pant [log pe (x)] + AEx=pp [log pe (x)]. With this decou- pling, we can view H2 regularizer injecting synthetic data and X as the amount of regularization. Our first main result states that iterative retraining of generative models is asymptotically stable the amount of real data is "large enough" with respect to the amount of synthetic data.
Theorem 1. (Stability of Iterative Retraining) Given 0 as defined in Equation (7) that a < 1/2,, then the operator norm of the
follows Assumptions and we have that, if x + Le Jacobian is strictly bounded by 1, more precisely,
Xa +L)
( a- a-Ma <1
(11)
Consequently, there exists a R > 0 such that for all 0n E 0 that satisfy 110 - 0'1l S R, then
starting at Og and having 0 0,41 9 (0.) we have that 0g + 0r and
((-aattt M(++ ) 100 (00-111l
110, - 00 10
(12)
X(a + +eT))
Proof of Theorem I can be found in Appendix E. Theorem establishes the stability property of generative models' iterative retraining and quantifies the convergence rate to the fixed point 0* Interestingly, setting À small enough is always sufficient to get [| important to notice that, prove that the local maximizer of 0' + H 0') is unique, we needed to assume that e.. <, Similarly, we speculate that with additional assumptions on the regularity of H, the neighborhood size for the local convergence result of Theorem I could be controlled with e.
< 1. However, it is
The limitation of Theorem is that it is based on using infinite samples from Pdata and only provides a sufficient condition for stability. Informally, we show that if dw Paata) and À are small enough-i.e,, our class of generative model can approximate well enough the data distribution and we re-use enough real data-, then, the procedure of iterative training is stable. On the one hand, remains open question to show that similar conditions the ones of Theorem I are necessary for stability. On the other hand, Proposition implies that the general case under Assumptions and 2, finite value for is necessary since for multivariate Gaussian, regardless of the value of dw (Po=.Paata). if X infinite (i.e., we only use generated data) then we do not have stability and collapse to a generative model with vanishing variance.
Our theoretical results point to a different conclusion than the one experimentally obtained by Alemohammad et al. (2023, Fig.3). In particular, stability may be ensured without injecting more 'fresh data'. Instead, sufficient condition for stability is to have good enough initial model (s small) and sufficient fraction of 'fixed real data' (A small enough) for each iterative training step.
3.2
ITERATIVE RETRAINING UNDER STATISTICAL APPROXIMATION ERROR
We now turn attention to iterative retraining for generative models under finite sampling. Mo- tivated by generalization bounds deep generative models (Yang and E, 2021a;b; Ji et al., 2021; Jakubovitz et al., 2019), we make the following additional assumption on the approximation capability

---

Published as a conference paper at ICLR 2024
of generative models as function of dataset size: suppose have generalization bound for our class of models with a vanishing term as the sample size increases.
Assumption 3. There exists b, EOPT 0 and a neighborhood U of 0 such that with probability 1 - over the samplings, we have
b
vo E U. E N, (0) - (0)1 EOPT
log
(13)
vn
"We could have kept more general bound as (nn, 6), and this choice comes without loss of generality.
The constant EOPT is usually negligible and mainly depends on the (controllable) optimization error. As for the term a/ v Vlog b/ō, it vanishes to 0 when increasing n. Intuitively, this assumption means that, in the neighborhood of 0 the practical iterate 9 (0) can get as close as desired to the ideal parameter O (0) by increasing the size of the sample from the dataset Dr used for retraining at iteration t and thus decreasing the optimization error. On the one hand, it is relatively strong to assume that one can approximate J (0) in the parameter space (usually such bounds regard the expected loss). On the other hand, the fact that this assumption is local makes it slightly weaker.
We now state our main result with finite sample size, with high probability, that iterative retraining remains within a neighborhood of the fixed point 0"
Theorem (Approximate Stability) Under the same assumptions of Theorem I and Assump- tion 3, we have that there exists O < p < 1 and R > O such that if 1 0- S R then, with probability 1
|00 11 EOPT +
a
bl log
Σ p/ + 101 - VIEN 1=0
(14)
Vn
Proof can be found Appendix F. The main takeaway on Theorem is that the error between the iteratively retrained parameters 0 and 0' can be decomposed in three main terms: the optimization error SOPT D/ o p', the statistical error a/ V /log bt/ō 2f p² and the iterative retraining error p'110, - 0-1, Interestingly. the error between 0 and 0" can be controlled in the regime where the log of the number of retraining is much smaller than the number of samples, i.e., in the regime log t << n.
4 RELATED WORK
Generative Models trained on Synthetic Data. The study of generative models within feedback loop has been a focus recent contemporary works by Alemohammad et al. (2023); Shumailov et al. (2023). Both these works exhibit the model collapse phenomenon in settings where the generative model iteratively retrained from scratch (A (Alemohammad et al., 2023) and finetuned (Shumailov al., 2023). Similar in spirit our work, Alemohammad al. (2023) advocates for the injection of real and even to add more fresh data-to reduce model collapse but falls short of providing theoretical stability guarantees which we provide. The effect of generative models trained on web- scale datasets contaminated with synthetic samples has also been investigated; corroborating the degradation of generated sample quality (Martínez al., 2023a;b; Hataya al., 2023).
Performative Predicition. The nature of the stability results in Theorem I and Theorem 2 bear connections with the literature on "performative prediction" " in supervised classification (Perdomo et al., 2020). In both cases, the outputs of the models affect the sampling distribution, which influences the objectives themselves. As such, we seek to characterize the existence and nature of fixed points. However, we note that performative prediction focuses supervised learning, and the required assumption in iterative stability for generative models is weaker than those to ensure global convergence in the performative prediction literature. For instance, Perdomo et al. (2021) require strong convexity with respect to 0, while Mofakhami et al. (2023) need stronger continuity properties of the mapping 0 Po- In contrast, we only require assumptions on po around optimal point 0
7

---

Published as a conference paper at ICLR 2024
5 EXPERIMENTS
We now investigate the iterative retraining of deep generative models in practical settings'. As our real-world models ingest datasets sampled from empirical distributions, they necessarily operate under statistical error arising from finite sampling. Other sources of eπror that inhibit learning the perfect generative inrepeivrreeeeetivee of the expressiveness of the architectur--include optimization errors from the learning process and other numerical emror from lack of numerical precision. Our goal with this empirical investigation is to characterize the impact on the performance of these models as a function of and iterative retraining iterations while holding the size of the real dataset constant.
Datasets and Models. We perform experiments on synthetic toy data as found in Grathwohl et al. (2018), CIFAR-I0 (Krizhevsky and Hinton, 2009), and FlickraacsHHQ 64 x 64 (FFHQ- 64 Kartas et al. 2019) datasets. For deep generative models, we conduct experiments with con- tinuous normalizing flows (Chen et al., 2018) constructed using a conditional flow-matching loss (CFM Lipman et al. 2022; Tong et al. 2023) and two powerful diffusion models in Denois- ing Diffusion Probabilistic Models (DDPM, Ho et al. 2020) and Elucidating Diffusion Models (EDM, Karras et al. 2022) where we relied on the original codebases torch- (https: / //github. c at ong01 /cod//oniitiiii coooniiiinal--i floww matching), github. - wtt.ct/./tttcccccooo dddpmttoooooooo faithful implementations.
(https: and edm (https: http//gttttttt/t www.tttuw...o/////...ee wwtp////o//////b//////eeaa for
Results on Synthetic Data. In Figures 3 and 4 in Appendix G we illustrate the learned densities of DDPM (Figure 3) on the 8 Gaussians dataset and conditional normalizing flows (Figure 4) on the two moon datasets. On the one hand, we observe model collapse if the generative models fully retrained on their own synthetic data (top rows). On the other hand, if the model is retrained on mix of real and generated data (bottom rows), then the iterative process does not diverge. In Figures 3 and 4, the model is iteratively retrained on as much generated data as real data. For Figure 3 we iteratively retrain DDPM diffusion model for 200 epochs on 10 samples. For Figure 4 we iteratively retrain a conditional flow matching model for 150 epochs on 10 samples.
Experimental Setup. For EDM we use the code, pre-trained models, and optimizers from the official implementation (specifically the dápmpp architecture with unconditional sampling) and a batch size of 128 for FFHQ-64 and 256 for CIFAR-10. For OTCFM, we used the to torch implementation. All other hyperparameters are those provided in the official implementations. After each training iteration, we generate a set of images of the same size as the training set, )² X; ~ Pa. This corresponds to 5 10a images for CIFAR-10 and 7 10 images for FFHQ-64. At each iteration, we compute the FID (Heusel et al., 2017) as well as precision and recall (Kynkäännin correlated with fidelity and diversity) between the real data and the full set of generated data. All the metrics are computed using the FLS package (https: / //gtttttiggg hub. com/ ma roo jo pirr///// fls, Jiralerspong et al. 2023, ee Appendix G for further details). Then, following Algorithm create a dataset which is a mix of all the real data Dreat and a fraction of the generated data Deyuth- = (x [A-]]_ After each retraining iteration, the network and the optimizer are saved, and the next retraining step resumes from this checkpoint: CIFAR-10 and FFHQ-64 models are finetuned for 10th and 1.4. 10 images, which is equivalent to 20 epochs on the real dataset. Note Dem
et al. 2019, which
that since the created datasets Dreat U {xx)
do not have the same size, i.e., their size depends on i=1 = n + [X nI,, we train constant number of images and not epochs which is necessary for a fair comparison between multiple fractions À of the incorporated generated data.
X: |Dreal U {xx]]11
Analysis of Results. Iterative retraining of OTCFM and EDM are displayed in Figure In particular, we report FID, precision, and recall as a function of the number of retraining iterations for various values of X. Note that the dashed black line with circles corresponds to the baseline of retraining without synthetic data x = 0, while the solid yellow line À = 1 is retraining with equal amounts of synthetic and original real data. The dashed red line with triangles corresponds retraining generative models only on their own data. Every other line is an interpolation between the settings of the black and yellow lines. As predicted by Theorem 2, if the proportion of real data is large enough, i.e., the value of À is small enough, then the retraining procedure is stable, and models do not diverge (see for instance, the CIFAR-10 and FFHQ-64-EDD
rows of Figure 2. If the fraction of used generated data À is too large, e.g., x= = 1. we observe that FID might diverge. This provides empirical evidence
*Code he found https gi thub. comb.ccm//// /00/m///3///// hasa//en_mmooe @ddnt_goo_add.
8

---

Published as a conference paper at ICLR 2024
Synth. (A
A 0.7
Synthetic
FID
Precision
Recall
10
0.60
CIFAR-10. - OTCFM
0.69-
0.55-
0.67
0.65
0.50
CIFAR- 10 EDM
0.65
0.60
0.60
0.55
0.55
12-
0.6
0.70
FFHQ-8 B1QQ-64- - EDM
0.5+
0.65
2
0.60
0.4
0
5
10
20
10
15
20
20
# retraining
# retraining
# retraining
Figure 2: FID, precision, and recall of the generative models as function of the number of retraining for multiple fractions À of generated data, D = Deeal U (x, ).Ar), X Pe.. For all models and datasets, only training synthetic data (dashed red line with triangles) yields divergence. For the EDM models on CIFAR- -00 (middle row), the iterative retraining is stable for all the proportions of generated data from X 0 to À = 1. For the EDM FFHQ-64 (bottom row), the iterative retraining is stable if the proportion of used generated data small enough (A 0.5).
in support of the model collapse phenomenon in both Alemohammad et al. (2023); Shumailov et al. (2023) where the generative models diverge and lead to poor sample quality. Interestingly, for fractions À used create the iterative training dataset D,, the precision of EDM models decreases with the number of retraining while their recall increases. It worth noting we observe the inverse trend between precision for OTCFM CIFAR-10.
CONCLUSION AND OPEN QUESTIONS
We investigated the iterative retraining of generative models their synthetic data. Our main contribution is showing that if the generative model initially trained on real data is good enough, and the iterative retraining made on a mixture of synthetic and real data, then the retraining procedure Algorithm 1 is stable (Theorems 1 and 2). Additionally, we validate our theoretical findings (Theorems and 2) empirically on natural image datasets (CIFAR-10 and FFHQ-64) with various powerful generative models (OTCFM, DDPM, and EDM). One of the limitations of Theorems 1 and 2 that the proposed sufficient condition to ensure stability may be necessary. Precisely, under Assumptions 1 and 2, there is gap to close between the necessity of < +oo and the sufficiency X(1 + Le) < for local stability. Another avenue for future work is to understand better the impact of the finite sampling aspect (described in Section 3,2) on iterative retaining stability.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
QB would like to thank Kilian Fatras for providing guidance on the conditional flow matching experiments, Nate Gillman for fruitful discussions yielding proof clarifications, and Samsung Electronics Co., Ldt. for partially funding this research. The authors would like to thank Nvidia for providing computing resources for this research.
10

---

Published as a conference paper at ICLR 2024
REFERENCES
S. Alemohammad, J. Badri-Rediigee,, L. Luzi, A. I. Humayun, H. Babaci, D. LeJeune, A. Siahkoohi, and G. Baraniuk. Self-consuming generative models go mad, 2023.
R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305 10403, 2023.
A. Antreas, S. Amos, and Harrison. Data augmentation generative adversarial networks. 2018.
D. Barfoot. Multivariate gaussian variational inference by natural gradient descent. arXiv preprint arXiv:2001.10025, 2020.
Boyd and Vandenberghe. Convex optimization. Cambridge university press, 2004.
Brown, B. Mann, N. Ryder, M. Subbiah, D. Kaplan, Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, al. Language models are few-shot learners, Advances in neural information processing systems, 2020.
S. Bubeck, Y. Cherapanamjeri, G. Gidel, and R. T. des Combes, A single gradient step finds adversarial examples on random two-layers neural networks. Advances Neural Information Processing Systems, 14:10081-1009,, 2021.
S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke. E. Horvitz, E. Kamar, Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4, arXiv preprint ar Aiv::2333. 12712, 2023.
R. Chen, Y. Rubanova, Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung,
C. Sutton, S. Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint MXXi::224.022111 2022.
Q Dong, L. L. Li, Dai, C. Zheng, Z. Wu, B. Chang, X. Sun, Xu, and Sui. A survey for in-context learning. arXiv preprint arXiv:2301.002. 2022.
Duchi. Probability bounds. URL: http://www. cs. berkeley. eluz/lachhfprieccsspoobbblll bounds. pdf. 2009.
W. Grathwohl, R. T. Chen, J. Bettencourt, I. Sutskever, and D. Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint 111.0:1810.01367, 2018.
R. Hataya, H. Bao, and H. Arai. Will large-scale generative models corrupt future datasets? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20555-20565, 2023.
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 6851, 2020.
D. Jakubovitz, R. Giryes, and M. R. Rodrigues. Generalization emror deep learning. In Compressed Sensing and Its Applications: Third International MATHEON Conference 2017, pages 153-193. Springer, 2019.
K. Ji, Y. Zhou, and Y. Liang. Understanding estimation and generalization emoor of generative adversarial networks. IEEE Tronsactions on Information Theory, 67(5):3114-3129 2021.
M. Jiralerspong, A. Bose, Gemp, C. Qin, Bachrach, and G. Gidel. Feature likelihood score: Evaluating generalization of generative models using samples, 2023.
11

---

Published as a conference paper at ICLR 2024
J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.0825 2020.
Karras, S. Laine, and T. Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
pages
Karras, M. Aittala, T. Aila, and Laine. Elucidating the design space of Liffusion-basse generative models. Advances in Neural Information Processing Systems, 35:26565-2265 266677,,677,--- 2022.
D. P. Kingma and M. Welling. An introduction variational autoencoders. Foundations and Trends@ in Machine Learning, 12(4):307-392, 2019.
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. 2009.
Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila. Improved precision and recall metric for assessing generative models. Advances in Neural Information Processing Systems, 32, 2019.
S. Lang. Fundamentals of differential geometry. Springer Science & Business Media, 1999.
Y. Lipman, R. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling, Oct. 2022.
A. W. Marshall, I. Olkin, and B. C. Aroll. Inequalities: theory of majorization and its applications. 1979.
G. Martínez, L. Watson, Reviriego, J. Hernández, M. Juarez, and Sarkar. Combining generative artificial intelligence (ai) and the internet: Heading towards evolution or degradation? arXiv preprint arXiv:2303..m 230.00/255 2023a.
G. Martínez, L. Watson, Reviriego, A. Hernández, M. Juarez, and Sarkar. Towards un- derstanding the interplay of generative artificial intelligence and the internet. arXiv preprint arXiv:2306.066 06100, 2023b.
Midjoumny. https: /www www.midjourneg.. cww/home// 2023. Accessed: 2023-09-09.
E. Mitchell, Y. Lee, Khazatsky, C. D. Manning, and C. Finn. Detectgpt: Zero-shot machine- generated text detection using probability curvature. arXiv preprint arXiv:2301./1300. 2023.
M. Mofakhami, I. Mitliagkas, and G. Gidel. Performative prediction with neural networks, 2023.
K. Oko, S. Akiyama, and T. Suzuki. Diffusion models are minimax optimal distribution estimators. arXiv preprint arXiv:2303.01S5 2023.
OpenAl. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023.
Perdomo, T. Zrnic, C. Mendler-Dünner, and M. Hardt. Performative prediction. In International Conference Machine Leurning, pages 7599-7609. PMLR, 2020.
J. C. Perdomo, T. Zmic, C. Mendler-Düinner, and M. Hardt. Performative prediction, 2021.
A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever. Zero-shot text-to-image generation. International Conference on Machine Learning. PMLR, 2021.
D. Rezende and Mohamed. Variational inference with normalizing flows. International conference on machine learning, pages 1530-1538. PMLR, 2015.
V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and S. Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303. 2023.
M. S.-M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly. Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018.
C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta,
C. Mullis, M. Wortsman, et al. LAION-SB: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 2022.
12

---

Published as a conference paper at ICLR 2024
1. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson. The curse of recursion: Training on generated data makes models forget, 2023.
Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. ICLR, 2021.
Stability AI. https:: http://saaaiittt ai http:/abbb.tt.....aaaaaeeeeeeeea
2023. Accessed: 2023-09-09.
Steinhardt. Al Forecasting: One Year In ftttss:/bbbunded
io/
ai- Sl-forrcaasingg one- year- in/, 2022. Accessed: 2023-09-09.
T. Teshima, I. Ishikawa, K. Tojo, K. Oono, M. Ikeda, and M. Sugiyama. Coupling based invertible neural networks are universal diffeomorphism approximators. Advances in Neural Information Processing Systems, 33:3362-3373, 2020.
A. Tong, N. Malkin, G. Huguet, Y. Zhang, Dector-Brooks, K. Fatras, G. Wolf, and Y. Bengio. Improving and generalizing flow-based generative models with minibatch optimal transport. arXiv preprint 2302. 302.00482, 2023.
C. Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahemn A. J. Borst, R. J. Ragotte, L. F. Milles, al. De novo design of protein structure and function with rfdiffusion. Nature, pages 1-3, 2023.
H. Yang and W. E. Generalization error of gan from the discriminator's ' perspective, 2021a.
H. Yang and W. E. Generalization and memorization: The bias potential model, 2021b.
13

---

Published as a conference paper at ICLR 2024
A PROOF OF PROPOSITION 1
Sampling step:
x Mr + VEri,
with Z hid No,r, 1 15j5 S ~
(4)
MLt1 [x E+1== nー - (x)- ( Het1) (x² Mo+1
Learning step:
(5)
Proposition 1. (Gaussian Collapse) For all initializations of the mean pio and the covariance Eo. iteratively learning a single multivariate Gaussian solely on its generated data yields model collapse. More precisely, if bu and En follows Equations (4) and (5), then, there exists < 1,
B( E 3
Eg
0
(6)
For exposition purposes, we first prove Proposition for a single one-dimensional Gaussian (Ap- pendix A.I). Then prove Proposition for single multivariate Gaussian in Appendix A.2.
A.1 ID-GAUSSIAN CASE.
In the one-dimensional case, the sampling and the learning steps become
Sampling step:
nnj = #4 + 0,27, with 24 iid. No.1, isisn,
(15)
Learning step: pe+1 = - of = - x [o) - (t-1)
(16)
Proposition 1 can be proved using the following lemmas. First, one can obtain a recursive formula for the variances of (Lemma A. i)). Then key component of the proof take the expectation of the standard deviation oe (not the variance of, which stays constant because we used an unbiased estimator of the variance), this is done Lemma A.1 ii). Finally, Lemma A.1 iii) shows how to upper-bound the expectation of the square root of x2 variable.
Lemma A.1.
i) If (at) and follow Equations (15) and (16), then, for all po, do.
0
Σ
Σ
Z3
(17)
ii)
vS.
V 2 1,0
Ti SS ~ Xn-1
(18)
iii)
Vt 1, a'op
(19)
E
2
V
with a= a(n)
<1.
Vn
Proof. (Lemma A.1 it). Let 1.
Pt+1
1 Σ X
1
Σ
Ime (ue + zf..)
(20)
n
n
14 + a: Σ
マj
at
(21)
j
14

---

Published as a conference paper at ICLR 2024
hence
Σ
(x} - Met1)²
(22)
71
2
Σ
Mt + 2 0t
(23)
71
5'
2
Σ
- Σ
oi
(24)
Proof. (Lemmaa A.1 ii) Cochran theorem states that S
E
Xn-1
2
Com-
bined with Lemma A.I i), this yields
VS.
Vt 2 1,0
Vn - 01
(25)
where vS Xn-1-
Proof. (Lemma A.1 ti)). The independence of the 2/ implies independence the St, which yields that for all t 21
VS.
E(0r-100))=
dr
(26)
11
V 15
= E
Te because S, IL oj
(27)
Vn
V/S
B(0+41)= = E
E(a;) because E(E(o++1lot) B(0+41)
(28)
The independence of the Z' yields independence of the S,. Combined with the strict Jensen inequality applied to the square root, yields
S
E
(29)
n
<
(30)
1 because IE(S.) 1 since S; X2-1
(31)
The strict Jensen inequality comes from the fact that the square root function is strictly convex, and S, x a constant random variable. Note that is strictly smaller than 1 and is independent of the number of retraining. However a depends on the number of samples n, and that a(n) + 1. when n DC.
A.2 MULTI-DIMENSIONAL CASE.
Sampling step:
x/ ui / Eizi, with zi Lid. ((0,1),.550
(4)
|1i41 = Ix E+1 = - x - m4z2) (xt - m4t1) T
Learning step:
(5)
15

---

Published as a conference paper at ICLR 2024
The generalization to the multi-dimensional follows the same scheme the unidimensional case: the key point is to upper the expectation of the square root of the covariance matrix
Lemma A.2.
i) If (Ei) and (He) follow Equations (4) and (5), then, for all po: Eo.
Ex+1
Σ
Z
Zizs
z
Σ
E. (32)
11
1
n
s
ii) Let
S s= -=Σ
Σ Z
Σ Zzz
(33)
72
n2
then
VE(S) = -1a
(34)
ii Then
B(VS) 3 avn 11. VE(S)
(35)
with 0 Amax
B(vS) vn-I
1I
iv) Finally
E (VEn+1) 3 3aE (VE,
(36)
with 0 < a < 1.
Proof. (Lemma A.2 i)) Rearranging the learning and the sampling steps (Equations (4) and (5)) yields
x p:: +
Eiz
(37)
1 Σ xi
P++1
(38)
P+t = #: + v Σ Σ z
hence
(39)
x pi+1 E,
z
Σ
z
(40)
Plugging Equation (40) in the learning step (Equation (5)) yields
)
Σ 2
Σ
VIL
VIL
(41)
n1
n
n
16

---

Published as a conference paper at ICLR 2024
Proof. (Lemma A.2 ū)) Since Z lid (O, I), then
E(S) = E
Σ
Σ
Z3
Σ
(42)
=
=ΣB E
2 Σ 3
z - Σ
Z3
(43)
Σ
E
1
Σ Z
I
(44)
3
8'
2
E
Σ
% 1 n
Zi
I
(45)
3'
expectation of unidimensional x
variable
1n-1)-I
(46)
L
Proof. (Lemma A.2 iii)) The matrix square root operator is strictly concave (Marshall et al., 1979,
Chapter 16, Example E.7.d), in addition, is not a Dirac, hence the strict Jensen inequality yields
B(vS) < B(S)
(47)
< vn Ia (by Lemma A.2 ii))
(48)
3 av n2 1- 11
(49)
with Amax
3(yS) vn-I
<1 1.
Proof. (Lemma A.2 iv))
Taking the matrix square root of Equation (41) yields
V E1t1
1
Σ
x
xf
1
Σ
Z
Σ
(50)
11
8'
=VS
1 n
E
Σ
S
(51)
Taking the expectation of Equation (51) conditioned on E, yields
E ((V2+i Ee)
1
E
(SSE2.
Σ
(52)
vn
Σ VSS Σラ
(because S 11. Σ₁)
(53)
Vn
--1 I
1
Σ aav
E ES) (by Lemma A.2 iii)
(54)
vn
1
(n 1
(55)
av
Σ+1
(56)
E (VEE++1)
3 aE (VE.) (because E [ (VEnHI =E (VE++1)
(57)
17

---

Published as a conference paper at ICLR 2024
B PROOF OF PROPOSITION 2
Proposition 2. Let po x + (xx)(() = (x u) x- -- 4) /v (2 [ with μ E Rd. E >
0 (u, ))),, then [x-xo(x) (x) V lo (x)) 0 and V log () is 1-Lipschitz.
(Proof of Proposition 2.) Following the computation of Barfoot (2020), and using the fact that Vy- log dE) = Σ (Boyd and Vandenberghe, 2004, Sec 1.4.1) one has
-log (x))
log det(2-1) + 7(x (x=x))= M - 'x ()) + log(2x) hence
(58)
E -- 1-xx x)
(u,)) logpo(x)
2x 1)(( (1x-1)) 14) - then
(59)
Σ-1
(1 x) La
(m. lx)
(60)
Ia 2 (μ xx)
2 { 2 Σ)
Equation (60) shows that x
Vi (x) is 1-Lipschitz. In addition, since E 0, then
E| V (u log px() > 0.
C PROOF OF PROPOSITION 3
Proposition (The Local Maximum Likelihood Solution is Unique) Let 0* be a solution of Equa- tion (7) that satisfies Assumptions I and 2. If cL a, then for all X 0 and 0 in small enough neighborhood U around 8*, there exists a unique local maximizer gx (0) in U.
(Proof of Proposition 3). Let be solution of Equation (7). Assume that 0 follows Assumptions 1 and 2. We recall that
: (0')
(0.00')))
1(0 8') == Excepista Pdata [log p (x) + A Ex flog po (x) and
(61)
9 (0) = argmax H0,
(62)
Using Proposition 4, since l is solution of Equation (7), then V H(0 0-) In addition.
Assumption 2 ensures that 0 E local-argmaxgr .'00'0'' 0') and that V2 "00000000" is invertible. Hence, by the implicit function theorem (Lang, 1999, Theorem 5.9), for
neighborhood of
or we have that there exists a continuous function such that g(0)) 0 and V (0,,000)( 0.
Finally, in order to show that g0) a local maximizer of 0' + H(O, 0') we just need to show that Vg H.( 0-) < O. V8 100, 0-) Vg. (0') AV O H², (1++
(63)
(1))
+X(V? H2(0,00) V2HH1
(64)
I- (using Assumption
1 2-(1 + A)aI+ + (Ex~po 2
log po (x) Ex 22 log po (x)]
(65)
(using Assumption
1 -a(1 + A)Ia + AELLa
(66)
20 i 2 EL
(67)
D PROOF OF PROPOSITION 4
Proposition 4. (The Optimal Parametric Generative Model is Fixed Point) For given data distribution Pdatas any 8¹ solution of Equation (7), and for all X >> O we have,
0- g (0*)
(10)
18

---

Published as a conference paper at ICLR 2024
Proof. We recall the definition of 0* and 9 (Equations (7) and (9)):
0 E argmax B Expata log po (x)]
(7)
O'c0
X (0) E llocal-argaax
(xx-paata Dog (x) +1 Ex-po Dlo po (x))
(9)
0'
:=N110")
(0.00
Since 0* is a solution of Equation (7)
vO' E 1, H1") = Exe- Pooger por (x)]
(68)
Exxepat [lo po. (x)]
(69)
H1(
(70)
Gibbs inequality yields
vO' O, H2 0') Ex [log po (x)]
(71)
s Ex [log po- (x)
(72)
H² 00))
(73)
H4 and H2((0,,)) are both maximized in 0 hence
g3ºh (0*) = argmax (0") + Xf2 0¹) = 0
(74)
0.
19

---

Published as a conference paper at ICLR 2024
E PROOF OF THEOREM 1
0- E argmax B Exupaata log po (x)]
(7)
0'E0
g (0) E llcall-argee argma-- Pxre log por (x)
+1 Ex [log po (x)])
(9)
0.00
(6")
(0.0')
Assumption 1. For 0 close enough 0*, the mapping V log Po (x) L-Lipschitz.
Assumption 2. The mapping O - Ex po is continuously twice differentiable locally around 0 and Ex-paana Vi log Pa- (x) 3 -lld
O.
Theorem 1. (Stability of Iterative Retraining) Given l* as defined in Equation (7) that follows As- sumptions and 2, we have that, if X(1 + Ls) < 1/2, then the operator norm of the Jacobian is strictly bounded by 1, more precisely,
X(a
|Jog (0))2 a M(x + EEL
(11)
Consequently, there exists a R > O such that for all 0u that satisfy 110, 0 S R, then starting at On and having 0,+1 - (0.) we have that 0 +
0r and
Maa +E)) (laxx(a+Ei) +EL)
10: - 0 -0
100 - -000
(12)
The main goal of Theorem to show that the operator norm of the Jacobian of the fixed-point operator is strictly bounded by one. We prove Theorem I with the following steps: using the Implicit Function Theorem, Schwarz theorem, and analytic manipulations (Lemmas E. i) and E.1 ii)) managed obtain simple formula for the Jacobian of G at 0- Using Kantorovich-Rubenstenn duality, we manage to bound the Jacobian of the fixed-point operator at 0' (Lemma E. ii)), and thus provide a condition for which [ (02)
These steps are detailed formally in Lemma E.
Lemma E.1. With A Vg. . Hi (0*) and B Vg. (f 0*), under the assumptions of Theorem I:
There exists an open set U O containing 0r such that v0 E u.
TG(0) = - (V (0) + AV o (2 (22002(0.000)
(75)
ii)
= VoroH2(0',00) Vo Vo (2"0"00"
(76)
and thus Jucobian at 0 can be writlen
Tg(0") (Ia + XA- AA AB
(77)
iii) The spectral norm of A 'BB and B - A can be bounded
Le || | 51+
(78)
a
and using Kantorovich-Rubeene Rubenstein duality theorem
[B
Alla S Ldw (po., (po-,Pdata) Pdata)
(79)
Proof. (Lemma E.1 ()) The definition of & yields
V ((())(())
(80)
20

---

Published as a conference paper at ICLR 2024
Differentiating Equation (80) using the chain rule yields
Vo H((0, (
(81)
which implies
JG(0) - (v ((0))0)0)))) V 22,02((00000))
(82)
Since Hi does not depend on 0
VoHi =
(83)
Combining Equations (82) and (83) and the fact that G())) 0' yields
TG(0') A Vo Hi + AVo or H2 0²1) -1 Vo. fo.00t21 (2'00(200000
(84)
Proof. (Lemma E.1 ūi)) First, let's compute V H20,00))
H2(0,0') = J log Po (x) (x) which yields
(85)
8 H2( J Ve log (x) and thus
(86)
V T (2
Ve log po
(using Schwarz theorem).
(87)
x
This yields
V/ (0"000000) / Vo log pa- (x) (x)dx
(88)
/ Voluo- (x) Vo log pe- (x) dx
0.0 log po- (x) (x) (x)dx
(89)
Napo- (x))
-V 0' H2 (-)
/ Vo (x)dx - Var H220000
(90)
V pe- (x) 0
(91)
=0
V2 0' o (2"00000))
(92)
Consequently, with A = V2. 211 (0" ,0") and B H220" 0*) we get
-9(0') = (A + AB) AB
(93)
= (Ia+ AA -B)) -1
BB) XA -B
(94)
Proof. Now, we have that
|A - | -B A) + Lal
(95)
11 A
|B
All
(96)
< Ls
(97)
21

---

Published as a conference paper at ICLR 2024
where used the triangle inequality, the submutippicattiity of the matrix norm and Assumptions 1 and yield
|B All
Vo.0 H2 00 Vg. 21, (0"))
(98)
v po. pa. (x) (x)dx / Vă logpe- (x)Paata
(99)
x
Bx-po
V log (x)] Exepp
V log pe (x)]
(100)
& Ldw (po. Pdata)
(101)
by Kantonovich-Rubensede duality theorem, where L is the Lipschitz norm of x V log (x()))
Proof. (Theorem 1) Using Lemma E.1 iii), one has that A(1 +1/a) < 1, then |AA MA-BB < 1 and thus
(I + AA AA-
(-AA-
(102)
k=0
Combining Lemma E.1 ii) and (102) yields
20
J9(0') = Σ (-AA 'B)*
(103)
Thus, by the triangle inequality and the submutliplicativity of the matrix norm,
|AA BB
|IA IBA
(104)
1- AA- -B
JIA 'Bl
To conclude, one simply needs provide sufficient condition for DA BB (1 be) 1/2.
which is
22

---

Published as a conference paper at ICLR 2024
F PROOF OF THEOREM 2
Assumption 1. For close enough to 0", the mapping x + V log (x) is I-Lipschitz.
Assumption 2. The mapping + Exce po()) is continuously twice differentiable locally around 0" and Ex-paan V log po- (x) -Id <
Theorem 2. (Approximate Stability) Under the same assumptions of Theorem and Assumption we have that there exists 0 < < and R > 0 such that if i 001 S R then, with probability 1 8,
100 -0'11 SOPT +
a
log
bt x
Σ
o' 100 - 00 VLEN".
(14)
VTi
100
Assumption 3. There exists a, b, EOrT 2 0 and a neighborhood U of 0* such that with probability
1 over the samplings, we have"
b
AO U,V. EN ( 9x (0)1 S EOPT
log
(13)
n
8
Proof. (Theorem 2)
|0 - 001|| $ 10 0,1 + 0 - 0-
(105)
< ||xX (07) - 9Λ (0") ||| + || - G(0
(106)
Using Assumption with probability 1 1-6 we have that
|| (0)) - 9x ( S coPT + Vn
a log 6
(107)
Moreover Theorem states that there exists a constant
< 1 such that
[[ G(0*) S pl 0-
(108)
This yields that with probability 1
| 10
0'11
<OOTT
log
p 100
0-1
(109)
A recurrence and the conditional independence of the successive samplings yield
AI VEN 5 E [0, [10: - -001 S
EOOTT
a
log
b 5
p² + p² 110 - -011
2 (1-5)²
(110)
As done in Duchi (2009) in Bubeck et al. (2021, Proof sketch of Prop. 2), the change of variable = 5 yields
Vt N 8' E t. |10 - 001
(EOTT+ + Vn
bt log 8
pt + pt 110 Σxt i=0
i
8-1| 110--011 2 (1 81-8
21-8 (111)
In particular, since 1, Equation (111) implies
bt log
WEN 8' E [0. IP |10 001l s
EOPT
Σe
Ep+ + p¹ 110 - -001 > 2 (1-88/t)
Vn
21-8'
(112)
'ee could have kept more general bound C(n, 8), and this choice without loss of generality.
23

---

Published as a conference paper at ICLR 2024
G ADDITIONAL EXPERIMENTS AND DETAILS
G.1 EIGHT GAUSSIANS DDPM
Ground Truth
No retraining
#retrain.
#retrain.
#retrain.
Collapsing
Stability
Figure 3: Stability vs. collapsing of iterative refraining of generative models on their own data. Each model's density is displayed function of the number of retraining steps. The first two columns correspond to the true density and the density of diffusion model trained the true data. As observed in Shumailov al. (2023); Alemohammad et al. (2023), iteratively retraining row) yields a density that collapses: samples very near the mean of cach mode are sampled almost exclusively after 100 iterations of retraining. Contrastingly, retraining on mixture of true and generated data (bottom row) does not yield a collapsing density.
model exclusively on own generated (top
G.2 TWO MOONS
FLOW MATCHING
Ground Truth
No retraining
#retrain. 10
#retrain. =0
#retrain. =5
Collapse
Stability
Figure 4: Stability vs. collapsing of iterative retraining of generative models on their own data. Each model's density is displayed function of the number of retraining steps. The first two columns correspond to the true density and the density diffusion model trained on the data respectively.
24

---

Published as a conference paper at ICLR 2024
G.3 CIFAR-10 DDPM
No Syrth.
Data (A
A 0.01
A
A- 0.6
A 0.8
A 0,001
A- 0.1
A- 0.3
A 0.5
A- 0.7
A 03
Fully Synthetic
FID
Precision
Recall
CIFAR-10 - DDPM
10.0
0.67
0.55
7.5-
0.66-
0.50
5.0
0.65
# retraining
# retraining
# retraining
Figure 5: FID, precision, and recall of the generative models as function of the number of retraining for
multiple fractions of generated data, D = Dreat U [xx] [A-n] & cv po.- Only training on synthetic data (dashed red line with triangles) yields divergence.
For DDPM, since the optimizer was not provided in the checkpoints, we first train the model from scratch for 1000 epochs and use it pretrained model.
25

---

Published as a conference paper at ICLR 2024
G.4 FULL GRAPHS FOR FULLY SYNTHETIC CIFARIO-OT SEARIO-EDM, AND FFHQ-EDM
No Synth. Data (A = 0)
A 0.01
A 0.2
=m U4
A
A 0.8
A
A 0.00L
0.1
==
=
=
A 0.9
Fully Synthetic
FID
Precision
Recall
0.75
0.6
CIFAR-10 OTCFM
0.4-
30-
0.70-
0.2
0.65
CIFAR- 10 EDM
0.65
5
0.60-
0.60
0.55
0.55
0.6
0.70
FFHQ-66 FTHQ-64-2 - EDM
0.4
0.65-
10-
2
0.60
0.2
0
15
20
10 15
20
0
5
10 15 20
retraining
# retraining
retraining
Figure 6: FID, precision, and recall of the generative models as function of the number of retraining for multiple fractions À of generated data, D = Dreat U (x, (x))),m X; ~ Pe.. For models and datasets, only training on synthetic data (dashed red line with triangles) yiclds divergence. For the EDM models on CIFAR-10 (middle row), the iterative retraining is stable for all the proportions of generated data from X 0 to A = 1. For the EDM FFHQ-64 (bottom row), the iterative retraining is stable if the proportion of used generated data small enough (A 0.5).
G.5 DETAILS ON THE METRICS: PRECISION AND RECALL
High Level Idea The notion of precision and recall used for generative models is different from the one used standard supervised learning. was first introduced in "Assessing Generative Models via Precision and Recall" (Sajjadi et al., 2018) and refined in "Improved precision and recall metric for assessing generative models" (Kynkäänniemi al., 2019), which we use in practice. Intuitively, at the distribution level, precision and recall measure how the training and the generated distributions overlap. Precision can seen as a measure of what proportion of the generated samples are contained in the "support" of the training distribution. On the other hand, recall measures what proportion of the training samples are contained in the "support" of the generated distribution. the training the same, then precision and recall should be perfect.
distribution and the generated distribution there is no overlap- i.c. disjoint between the training distribution and the generated distribution, then one should have precision and zero recall.
26

---

Published as a conference paper at ICLR 2024
Implementation Details In order adapt these metries for empirical distributions that have finite samples customary of training deep generative models, multiple numerical tricks are required. Specifically, for empirical set of samples, the support of the associated distribution is approximated by taking the union of balls centered at each point with radius determined by the distance to the k-th nearest neighbor. This is generally done in network, Heusel et al. 2017). For FID, we follow the standard of comparing 50k generated samples to the 50k samples in the training set. For precision and recall, with the same of samples, we follow the methodology of (Kynkäänniemi et al., 2019) with & = 4. We used the FID, precision and recall implementations of Jiralerspong et al. (2023) (https: / g github. com marco 1 fira/ fls).
meaningful feature space (here use Inception V3
G.6 SAMPLE VISUALIZATION
Fully Synth.
1=
4=03 0.5
= 0.1
No synth. data
(a) 0 retrain.
(b) 5 retrain.
(c) 10 retrain.
(d) 15 retrain.
(e) 20 retrain.
Figure 7: EDM samples, with generative models iteratively retrained on their own data, only on the synthetic data (top), and mix of synthetic and real data.
27

---

Published as a conference paper at ICLR 2024
G.7
QUANTITATIVE SYNTHETIC EXPERIMENTS
Fully Synthetic
= 0.0
0.5
À = 1.0
Wasserstein- distance
Wasserstein-2 distance
0.8-
0.6-
0.6
0.4-
0.4
0.2
0
10
10
# retraining
# retraining
Figure 8: Stability collapsing of iterative retraining of generative models their data. The Wasserstein-1 and distances between the true data distribution and the generated one displayed as function the number retraining.
distances are averaged over runs, the corresponds to the mean, and shaded area to the standard deviation. When the model is fully retrained only on its own data (Fully Synthetic, dashed red line with triangles), the distance to the true data distribution diverges. When the model is retrained on mixture of real and synthetic data 0, 0.5, 1), then the distance between the generated samples and the true data distribution stabilizes.
The setting of Figure the same as for Figure 4: we used the two-moons datasets, and learn the samples using OT-CFM (Tong et al., 2023), a state-of-the art flow model, which corresponds to a continuous normalizing flow, which is an exact likelihood. The dataset has 1000 samples, and all the hyperparameters of the OT-CFM Tong et al. (2023), https: ptttps:/gitthen /ttpttgtttt/t////////////t com/ atongol ionmdiiiionll-fooo/. ching/ bl ho/mmai//// exampllll es hot.eoookk///22 2D example e. ipynb.
the default ones from the implementation of
28

---

Published as a conference paper at ICLR 2024
G.8
PARAMETER CONVERGENCE
Fully Synthetic
A=
0=
0
==
A=
Random Init.
No Synth. Data (A= ()
Am 02
Am
2-
A-1
CIFAR-10 - OTCFM
CIFAR-10 - EDM
FFHQ-64 EDM
10-
P
10
10
Boll
10
10 2
10
10 10-3
10
10
20
10
20
10
20
# retraining
# retraining
# retraining
Figure 9: Convergence in the Parameter Space. The figure displays the Euclidean norm of the difference between current parameters at retraining 0 and the parameters O.o the initial network used to finetune. In order to have comparable scales between the models and the dataset, the norm is rescaled by the number of parameters of each network. The dashed black line shows the norm of the difference between the parameter t. of the initial network used to finetune, and random initialization. Each column displays different combinaison can that fully retraining on synthetic data yields to larger distance the initial parameters, than retraining on mix synthetic and real data.
dataset algorithm.
The setting of Figure 9 is the same as for Figure 2. we finetune the EDM (Karras et al., 2022) and OTCFM models (Lipman al., 2022; Tong et al., 2023) for 20 epochs and 20 retraining on CIFAR-10 and FFHQ.
29


Reviews:
[{"review_id": "Oyp6e1M9tt", "reviewer": "Reviewer_pzxF", "summary": "The authors propose an analysis of iterative retraining of generative models on their own data. This is indeed a very tempting approach used in various fields. \n\nThe analysis begins in a simple Gaussian case that enables the authors to provide intuition on the behavior of such a training method. \n\nThe analysis then addresses two other cases: under no statistical error assumption were under inifinite sampling the stability and convergence can be retrieved.", "questions": "My overall understanding of the field is limited. However i have a couple of questions:\n\n1. How likely is assumption 3 ? can the authors provide example where such a bound stands ?\n2. How is precision and recall computed in the experimental section ? what classifier is used here ?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is clearly presented, the objectives are well stated and i believe that the authors propositions and lemma clearly answers their problematic.\n\nThe experiments seem conclusive.", "weaknesses": "see questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OpPzDWlMsE", "reviewer": "Reviewer_bfKK", "summary": "In this work authors focus on the problem of retraining the generative models with the combination of real and synthesised data coming from the previous state of the same model. With a series of theoretical derivations authors show that such a process is stable (under some assumptions). The theoretical theorems are also evaluated with additional empirical studies that seem to align with the main claims.", "questions": "Did you consider evaluation how other sampling procedures that minimises the statistical approximation error might influence the findings presented in this paper? Maybe instead of drawing random samples, but for example those that best cover the data distribution (e.g. using herding alogrithm) could prevent the model from collapsing, or at least slow it down when retrained with bigger portion of sampled data?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- To my knowledge this is the first work to study the theoretical stability of generative models when retrained on its own data. It is an interesting practical problem as soon we might struggle with distinguishing true training examples from fake synthetic images. This problem is well motivated in this submission.\n- In the submission authors  provide theorems with proofs that shed a new light into the topic of continual retraining of generative models with self-generated data, showing that this process might be stable under the assumption of retraining the model with sufficient share of true data samples.\n- This work might have some impact on theoretical fields of ML such as continual learning of generative models and practical aspects such as deployment of big diffusion models.\n- This is one of the best written theoretical papers I have ever read. Everything is extremely clear and easy to follow. It reads as a good crime!", "weaknesses": "- The contribution of the theoretical part has limited significance as it mostly concerns unfeasible setups with normally hard or impossible to achieve assumptions  (e.g. an infinite number of rehearsal samples generated by the model).\n- The empirical evaluation of two simpler models is limited to 2 visualisations without quantitative measurements\n- For Diffusion models, the evaluation is performed on 3 datasets, which is sufficient. However if I understand the setup correctly the whole analysis is performed on a single training seed. The differences presented in the plots are extremely small so it is unclear whether they are statistically significant, and therefore whether the main claims hold.\n\nSmall not important detail:\nPage 8 the end of Experimental Setup section, I spend some time trying to understand what emphconstant means :D - please correct a typo", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vJHIcYImT9", "reviewer": "Reviewer_9HHH", "summary": "The authors study the problem of generative models that are recursively trained with parts of the dataset are generated by an earlier version of the generative model. They provide theoretical proofs that suggest that if the share of generated data w.r.t. the original data is small enough, this training can still be stable, whereas otherwise it collapses.", "questions": "1. Have the authors tried running the experiment for more than 5 steps? \n\n2. Can the authors give standard deviations for the plots in Figure 3?\n\n3. What do the recall and precision metrics in Figure 3 mean?\n\n4. Can the authors clarify the >-operator for matrices?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**Well-structured.** The structure of the paper seems fine and I like how the simple cases are considered first to build an intuition before more complex cases are presented. Although the matter of the paper is quite technical, the formal introduction of the problem and the notation is well executed. Apart from one point (see \"clarity\" below), I think the overall presentation is good.\n\n**Interesting and Timely problem.** the problem studied in this work is interesting and can be considered timely as indeed, content by generative AI is flooding the internet, which in turn is the key source of training data for generative models.\n\n**Technical rigor.** I have the overall impression that the technical part is rigorously executed and I did not find any significant flaws. However, as pointed out below, I was not able to verify all steps of the proofs.", "weaknesses": "**Clarity.** I am a bit puzzled by the use of $\\succcurlyeq$ for matrices. When you write $\\Sigma \\succcurlyeq 0$, I suppose it is the usual condition for Sigma to be positive definite and not that each individual element of sigma should be larger than zero. This would rule out certain covariance matrices. On the other hand, the authors write $\\nabla^2 f = H_f \\preccurlyeq -\\alpha I_d$, comparing two matrices (supposing that $I_d$ is the $d$-dimensional unit matrix). I suppose that it now constrains each element. Can the authors please clarify?\n\nMeasures: What is meant by precision and recall in the evaluation section? Is there a discriminator deployed that tries to differentiate between the real and fake samples? This needs to be clarified. I don’t currently see how generative models can have recall/precision.\n\n**Clarity of the Proofs.** Unfortunately, some proofs in the paper offer room for improvement of clarity and were inaccessible for verification in their current form. For instance:\nIn appendix A.1., I can follow the proof of Lemma 1 and Lemma 2. However, it is not obvious how equation (13) implies the form of $\\alpha(n)$ using the $\\Gamma$-function. This should necessarily be clarified as there is not a word on how the form of $\\alpha(n)$ comes into play.\n\n**Some assumptions made in this work may not reflect reality well and may be oversimplifying.** While I know that certain assumptions are required to make the problem amenable to theoretical analysis, I have concerns that they may be overly restrictive in practice. In particular, we basically assume convexity of the loss function through assumption 1 and 2. As far as I can see, due to the Lipschitz constant on the Hessian being L (assumption 1) and the Hessian's eigenvalues being smaller than $-\\alpha$ (interpreting the operator that way, please correct me in case this is not correct), in the ball of $\\epsilon = \\alpha/L$ around $\\theta^*$, the Hessian will be negative definite, implying convexity. As the Theorems only state the existence of a radius $\\delta > 0$ in which the convergence properties hold, we basically consider the convex part of the problem. As we know, modern loss landscapes are far from convex. It is highly unlikely that both the initial value and the optimum $\\theta^*$ lie in a ball in which the loss function is convex.\nAnother impractical assumption may be the assumption of no approximation error. This is usually only shown for infinite-parameter models. I would be fine with these approximations if the empirical results would confirm the assumptions and the analysis that follows from them. However the evaluation results seem to rather confirm the doubts.\n\n**The evaluation does not confirm the claims.** As far as I can tell, the point of departure for this work is as follows: Previous works [1,2] have already established that model training on solely generated data is unstable or collapses. On the other hand, training models again and again on the same dataset is stable (otherwise the models we currently have wouldn’t work at all, as they are trained epoch by epoch with the same data). The key claim of this work is therefore that there is some value $\\lambda > 0$, i.e., a certain amount of generated data can be injected, such that model training remains stable. \n\nUnfortunately, the experimental results are not very convincing in that regard. Indeed, the leftmost column of Figure 3 shows that the resulting FID curve is almost a linear interpolation between $\\lambda=0$ (stable) and $\\lambda=1$ (unstable). This suggest that for every $\\lambda>0$ training will be diverge in the end. Even for the smaller lambdas, we see that the training FID gradually increases. For \\lambda=0.001, I guess one would require more than 5 steps of training to observe a statistically significant effect (as we are discussion the case of infinite retraining). Furthermore, there are no measures of disparity (e.g., standard deviations) displayed, would could solidify the empirical evaluation.\n\n**Minor points**\n\nRelated work: I am aware of the fact that the studied problem is different from mode collapse in generative models, but I have the impression that there seem to be some connections. Maybe the authors can add a discussion on this point.\n\nWrite-up: Missing parenthesis in the last line of before the statement of the contributions “(DDPM…”\n\nLast point of the contribution section: “Using two powerful diffusion models in DDPM and EDM” (“in” seems unexpected at this place)\nThere are many unclarities in the proofs:\n\n“Since most methods are local search method[s]” (use the plural form here)\n\nProof in Appendix C. There are some formatting errors below eqn. 33. (theta is not properly displayed). \n\nProof in Appendix D. Equation (39) theta’ is multiply defined, first by the outer quantor, then below the max after the $\\leq$ sign. Consider using $\\theta’’$ or similar in this case.\n\n\nThe PDF seems to render very slowly. Maybe the authors can check some of the vector graphics again to increase the overall accessibility.\n\n**Summary:** Overall, this is an interesting work. While I do not contest the main results, I was not able to verify all proofs either as I wasn’t able to follow the arguments at some points. Furthermore, the empirical evaluation is almost contradictory to the theoretical claims in this paper. I will be willing to increase my rating to an accept-score, if the authors can clarify their proofs such that the validity of their results can be easily verified and convincingly show that values of $\\lambda>0$ exist, where stable retraining is possible for a larger amount of retraining iterations (5 iterations are insufficient when considering an infinite regime).\n\n\n-----------------------------\n\n**References**\n\n[1] S. Alemohammad, J. Casco-Rodriguez, L. Luzi, A. I. Humayun, H. Babaei, D. LeJeune, A. Siahkoohi, and R. G. Baraniuk. Self-consuming generative models go mad, 2023.\n\n[2] I. Shumailov, Z. Shumaylov, Y. Zhao, Y. Gal, N. Papernot, and R. Anderson. The curse of recursion: Training on generated data makes models forget, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zUaE7Km5sO", "reviewer": "Reviewer_LQsW", "summary": "This article studies the stability of iteratively training generative models with generated samples as part of the training set. In this article, it is proven that under some regulatory and optimality assumptions, the model after many rounds of training will still be close to the original one. This article considers both the infinite-sample and finite-sample cases, and provide error bounds on model distances. Then, the article studies several popular deep generative models on standard benchmarks to investigate what happens in practice.", "questions": "Please refer to the weakness section for questions on experiments.\n\n\n--------------\n\n**After rebuttal** the authors have improved or added experiments that fully addressed my concerns. The results make the paper much stronger than the first draft. I think this paper is novel, sound, enlightening, and opens a new window to look at the current challenges of modern generative models.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The iterative training formulation has become a very important problem today as there are many powerful deep generative models and their generated samples are used to train or finetune other models. This article presents a very concise and elegant way to describe this task, which is able to leverage previous theory on deep generative models by its nature. \n- The assumptions are mild and the theoretical results are good. It is not surprising that with a small enough $\\lambda$ the iterative training will be stable, but it is encouraging that $\\lambda$ can be as large as $1/4$ in Thm 1. \n- The presentation of this paper is very clean. It is very easy to follow from background and preliminaries to assumptions, theorems, and proofs.", "weaknesses": "The main weakness of this article is that its experiments cannot fully justify the theoretical analysis. All experiments on the high-resolution image tasks are based on diffusion models, where there might be some shift from the theoretical analysis on maximum likelihood training as diffusion models optimize variants of ELBO. There should be experiments on models trained with the exact likelihood (and on real world datasets), such as flows and autoregressive models. While flows may have a larger $\\epsilon$ due to their capacity issues, autoregressive models might be a better objective to look at. I'd like to see some experiments on this. \n\nRegarding Fig 3, the trends are not clear enough, and I'd like to see results for more iterations so that the trends become clear. The differences between different runs are very small, so the authors should run multiple experiments with different random seeds to reduce the effect caused by randomness. In addition, I'd like to see results on  $\\parallel \\theta_t - \\theta^* \\parallel$ for more direct comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)