Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World

## Abstract

Abstract The increasing presence of AI-generated content on the internet raises a critical question: What happens when generative machine learning models are pretrained on web-scale datasets containing data created by earlier models?
Some authors prophesy model collapse under a ‘ replace ’ scenario: a sequence of models, the first trained with real data and each later one trained only on synthetic data from its preceding model. In this scenario, models successively degrade. Others see collapse as easily avoidable; in an ‘ accumulate ’ scenario, a sequence of models is trained, but each training uses all real and synthetic data generated so far. In this work, we deepen and extend the study of these contrasting scenarios.
First, collapse versus avoidance of collapse is studied
by comparing the replace and accumulate scenarios
on each of three prominent generative modeling settings;
we find the same contrast emerges in all three settings. Second, we study a compromise scenario; the available data remains the same as in the accumulate scenario – but unlike accumulate and like replace , each model is trained using a fixed compute budget; we demonstrate that model test loss on real data is larger than in the accumulate scenario, but apparently plateaus, unlike the divergence seen with replace .
Third, we study the relative importance of cardinality and
proportion of real data for avoiding model collapse.
Surprisingly, we find a non-trivial interaction between real and synthetic data, where the value of synthetic data for reducing test loss depends on the absolute quantity of real data.
Our insights are particularly important when forecasting whether future frontier generative models will collapse or thrive, and our results open avenues for empirically and mathematically studying the context-dependent value of synthetic data.

## 1 Introduction: Model Collapse & Why It Matters

With each passing day, the internet contains increasingly more AI-generated content (Altman, ). What is the impact of this for future of deep generative models pretrained on web-scale datasets containing data generated by their predecessors?
Previous work forewarned that such model-data feedback loops can exhibit model collapse, a phenomenon whereby model performance degrades with each model-fitting iteration such that newer models trend towards useless (Shumailov et al., ).
This prophecy is deeply concerning because society is increasingly relying on these deep generative models (Bommasani et al., ; Reuel et al., ; Perrault & Clark, ; Kapoor et al., ), and model collapse threatens that future models will be made useless as society’s current data practices pollute the pretraining data supply.

However, the model collapse literature is replete with different experimental methodologies and different mathematical assumptions of different generative models, with different papers reaching different conclusions (Taori & Hashimoto, ; Hataya et al., ; Martínez et al., ; Shumailov et al., ; Alemohammad et al., ; Martínez et al., ; Bohacek & Farid, ; Guo et al., ; Bertrand et al., ; Briesch et al., ; Dohmatob et al., ; ; Gerstgrasser et al., ; Seddik et al., ; Marchi et al., ; Padmakumar & He, ; Chen et al., ; Ferbach et al., ; Veprikov et al., ). These mixed methods and findings make assessing the probability and harm of model collapse difficult.

In this work, we extend the accumulate workflow studied in Gerstgrasser et al. () to cover several settings previously not studied in the literature to exhibit its effectiveness over the replace workflow. We begin by testing the following hypothesis – that model collapse emerges in a scenario where models are trained on evolving datasets built by deleting past data en masse; and model collapse is avoided in a scenario where the training datasets instead accumulate all real and synthetic data. We consider whether these claims hold in three new generative model task settings pointed to by recent prominent work; we find the claims hold.
We then compare three clear dataset evolution scenarios, focusing on a new middle ground where data accumulate over time but each model is trained under a fixed compute budget; in this middle ground, we find that losses on real test data climb faster than without a compute budget but plateau to lower values than if data are deleted en masse after each model-fitting iteration. These results are consistent across five different generative modeling settings.
Lastly, we investigate, in a specific situation, whether the proportion or cardinality of initial real data matters more for preventing model collapse and discover a non-trivial interaction between real and synthetic data: when real data are scarce, an appropriate amount of synthetic data reduces the test loss on real data, whereas when real data are ample, synthetic data increases the test loss on real data.
Altogether, our work provides valuable comprehensive insights for predicting likely futures of deep generative models pretrained on web-scale data.

## 2 Testing Two Model Collapse Claims in Three New Generative Modeling Settings

Gerstgrasser et al. () recently made two claims about model collapse:

- 1.
Many previous papers induced model collapse by deleting past data en masse and training largely (or solely) on synthetic data from the latest generative model, and
- 2.
If new synthetic data are instead added to real data, i.e., data accumulate over time, then model collapse is avoided.

These two claims are important for forecasting the future of generative models because, if correct, model collapse is then less likely to pose a realistic threat since accumulating data over time is a more realistic modeling assumption; as a partner at Andreessen Horowitz elegantly explained, deleting data en masse is “not what is happening on the internet. We won’t replace the Mona Lisa or Lord of the Rings with AI generated data, but the classics will continue to be part of the training data set" (Appenzeller, ).

However, these claims have not been tested in three new generative modeling settings recently introduced by prominent work (Shumailov et al., ) for studying model collapse:

- 1.
Multivariate Gaussian Modeling: Multivariate Gaussians are repeatedly fit to data and then used to sample new synthetic data for future Gaussian fitting.
- 2.
Kernel Density Estimation: Kernel density estimators are repeatedly fit to data and then used to sample new synthetic data for future kernel density estimators.
- 3.
Supervised Finetuning of Language Models: Language models are finetuned in a supervised manner and then used to sample new synthetic text for future finetuning.

In this section, we ask and answer:

> In these three new generative modeling settings, is model collapse caused by deleting data en masse and avoided by instead accumulating data?

In all three settings, we empirically find and (when possible) mathematically prove the answer is yes.

### 2.1 Model Collapse in Multivariate Gaussian Modeling

Figure: Figure 1: Model Collapse in Multivariate Gaussian Modeling. Top: Previous work (Shumailov et al., ) proves model collapse occurs if one iteratively fits means and covariances to data and then samples new data from a Gaussian with the fitted parameters (left). We demonstrate that if one doesn’t delete all data after each model-fitting iteration - i.e., if data accumulate - then model collapse does not occur (right). Number of Samples Per Iteration: $316$. Note: We visualize the fit Gaussians as zero-mean for easy comparison of the fit covariances across model-fitting iterations. Middle: If data are replaced, then the empirically fit means drift away from the original data’s mean with increasing model-fitting iterations, but if data instead accumulate, then the empirically fit means stabilize. Bottom: If data are replaced, then the empirically fit covariances collapse compared to the original data’s covariance, but if past data are not discarded, then the fit covariances stabilize quickly and collapse is avoided. Note: Rows 2 and 3 correspond to $d=31$ dimensional data.
Refer to caption: x1.png

We consider repeatedly fitting multivariate Gaussians to data and sampling from the fit Gaussians. We begin with $n$ real data drawn from a multivariate Gaussian with mean $\mu^{(0)}$ and covariance $\Sigma^{(0)}$:

$$ $X_{1}^{(0)},...,X_{n}^{(0)}\sim_{i.i.d.}\mathcal{N}(\mu^{(0)},\Sigma^{(0)}).$ $$

For model fitting, we compute the unbiased mean and covariance of the most recent data:

$$ $\displaystyle\hat{\mu}_{\text{Replace}}^{(t+1)}$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{n}\sum_{j=1}^{n}X_{j}^{(t)}$ (1) $\displaystyle\hat{\Sigma}_{\text{Replace}}^{(t+1)}$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{n-1}\sum_{j=1}^{n}(X_{j}^{(t)}-\hat{\mu}_{\text{Replace}}^{(t+1)})(X_{j}^{( t)}-\hat{\mu}_{\text{Replace}}^{(t+1)})^{T}$ (2) $$

For model sampling, we sample $m$ new synthetic data using the fit Gaussian parameters:

$$ $X_{1}^{(t)},...,X_{n}^{(t)}\;\Big{|}\;\hat{\mu}_{\text{Replace}}^{(t)},\hat{ \Sigma}_{\text{Replace}}^{(t)}\quad\sim_{i.i.d.}\quad\mathcal{N}(\hat{\mu}_{ \text{Replace}}^{(t)},\hat{\Sigma}_{\text{Replace}}^{(t)}).$ (3) $$

Under the above data-model feedback loop, Shumailov et al. () prove that

$$ $\hat{\Sigma}_{\text{Replace}}^{(t+1)}\overset{a.s.}{\rightarrow}0\quad;\quad \mathbb{E}[\mathbb{W}_{2}^{2}(\mathcal{N}(\hat{\mu}_{\text{Replace}}^{(t+1)}, \hat{\Sigma}_{\text{Replace}}^{(t+1)}),\mathcal{N}(\mu^{(0)},\Sigma^{(0)}))] \rightarrow\infty\text{ as }t\rightarrow\infty,$ (4) $$

where $\mathbb{W}_{2}$ denotes the Wasserstein-2 distance.
This result states that the fit covariance will collapse to 0 0 and that the Wasserstein-2 distance will diverge as this model-data feedback loop unfolds.
Note that the Wasserstein-2 distance diverges not because the covariance collapses to 0 0 but because the distance between the $t$-th fit mean $\hat{\mu}_{\text{Replace}}^{(t)}$ and the true mean $\mu^{(0)}$ diverges.

However, this result assumes that all data are deleted after each model-fitting iteration. As discussed above, this assumption is likely unrealistic because society does not delete earlier content from the internet and replace it with new model-generated content after fitting each state-of-the-art model. What happens if data instead accumulate across model-fitting iterations?
To study this, we instead consider fitting to all previous real and synthetic data:

$$ $\displaystyle\hat{\mu}_{\text{Accumulate}}^{(t+1)}$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{n(t+1)}\sum_{i=0}^{t}\sum_{j=1}^{n}X_{j}^{(i)}$ (5) $\displaystyle\hat{\Sigma}_{\text{Accumulate}}^{(t+1)}$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{n(t+1)-1}\sum_{i=0}^{t}\sum_{j=1}^{n}(X_{j}^{(i)}-\hat{\mu}_{\text{ Accumulate}}^{(t+1)})(X_{j}^{(i)}-\hat{\mu}_{\text{Accumulate}}^{(t+1)})^{T}$ (6) $$

Data are then sampled using these fit Accumulate parameters rather than the fit Replace parameters.

Empirically, we find that deleting all data after each model-fitting iteration causes model collapse (Fig. 1 Left), whereas accumulating data across model-fitting iterations prevents model collapse (Fig. 1 Right). More specifically, we find that if data are deleted, the squared error between the fit mean $\hat{\mu}_{\text{Replace}}^{(n)}$ and the initial mean $\mu^{(0)}$ diverges (Fig. 1, Middle Left), and the fit covariance $\hat{\Sigma}_{\text{Replace}}^{(n)}$ relative to the initial covariance $\Sigma^{(0)}$ collapses to 0 0 (Fig. 1, Bottom Left), as measured by the ratio between the trace of $\hat{\Sigma}^{(t)}$ and the trace of $\Sigma^{(0)}$. In contrast, if data accumulate, the squared error between the fit mean and the initial mean plateaus quickly (Fig. 1, Middle Right), as does the fit covariance relative to the initial covariance (Fig. 1, Bottom Right).

Additionally, in the univariate case, we mathematically characterize the limit distribution:

###### Theorem 1 .

For notational efficiency, for a univariate Gaussian, let $\hat{\mu}^{(t)}$ and $\hat{\sigma}^{(t)}$ denote $\hat{\mu}^{(t)}_{\textrm{Accumulate}}$ and $\hat{\Sigma}^{(t)}_{\textrm{Accumulate}}$. Suppose that the mean and covariance are updated as in Eqns.  5 and 6. Then

$$ $\displaystyle\mathbb{E}\left(\sigma_{t}^{2}\right)=\sigma_{0}^{2}\cdot\prod_{k =1}^{t}\left(1-\frac{1}{k^{2}n}\right)\quad$ $\displaystyle\xrightarrow{t\rightarrow\infty}\quad\sigma_{0}^{2}\cdot\left( \frac{\sin(\pi/\sqrt{n})}{\pi/\sqrt{n}}\right)$ (7) $\displaystyle\mathbb{E}[(\mu_{t}-\mu_{0})^{2}]=\sigma_{0}^{2}\cdot\left(1- \prod_{k=1}^{t}\left(1-\frac{1}{k^{2}n}\right)\right)\quad$ $\displaystyle\xrightarrow{t\rightarrow\infty}\quad\sigma_{0}^{2}\cdot\left(1- \frac{\sin(\pi/\sqrt{n})}{\pi/\sqrt{n}}\right).$ (8) $$

See Appendix Sec. A for the proof. This reveals two key differences when data accumulate: the covariance no longer collapses, and the mean no longer diverges, meaning model collapse is mitigated.

Figure: Figure 2: Model Collapse in Kernel Density Estimation. Left: We consider 4 standard datasets from sklearn: Blobs, Circles, Moons and Swiss Roll. Center: For all four datasets, deleting data en masse causes the negative log likelihoods (NLL) of real test data to increase with each model-fitting iteration. Right: For all four datasets, accumulating data avoids model collapse. Interestingly, for specific pairs of datasets and number of samples per iteration, training on real and accumulating synthetic data can yield lower loss on real test data than training on real data alone.
Refer to caption: x4.png

###### Theorem 1 .

### 2.2 Model Collapse in Kernel Density Estimation

We next turn to the second generative modeling setting for studying model collapsed introduced by Shumailov et al. (): kernel density estimation (KDE). Similar to multivariate Gaussian modeling, we begin with $n$ real data points drawn from an initial probability distribution $p^{(0)}$: $X_{1}^{(0)},...,X_{n}^{(0)}\sim_{i.i.d.}p^{(0)}$. We then iteratively fit KDEs to the data and sample new synthetic data from these estimators. In the Replace setting, we fit the KDE to $n$ data samples from the most recently fit model, whereas in the Accumulate setting, we fit the KDE to all data points from all previous iterations, with the number of points growing linearly as $n(t+1)$:

$$ $\displaystyle\hat{p}_{\text{Replace}}^{(t+1)}(x)$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{nh}\sum_{j=1}^{n}K\Big{(}\frac{x-X_{j}^{(t)}}{h}\Big{)}$ (9) $\displaystyle\hat{p}_{\text{Accumulate}}^{(t+1)}(x)$ $\displaystyle\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}\frac{ 1}{nh(t+1)}\sum_{i=0}^{t}\sum_{j=1}^{n}K\Big{(}\frac{x-X_{j}^{(i)})}{h}\Big{)}$ (10) $$

where $K$ is the kernel function and $h$ is the bandwidth parameter. We consider a standard Gaussian kernel. For sampling, at each iteration, we draw $n$ new synthetic data points from the fitted kernel density estimators. We evaluate the performance using the negative log-likelihood (NLL) on real held-out test data; lower NLL indicates better performance. For data, we use four standard synthetic datasets from sklearn (Pedregosa et al., ): blobs, circles, moons, and swiss roll.

We again observe the same general difference between replacing data and accumulating data (Fig. 2): replacing data causes a rapid increase in NLL as the number of model-fitting iterations increases, indicating that the KDEs are becoming increasingly poor at modeling the true underlying distribution.
In contrast, when data accumulate across model-fitting iterations, we observe that the NLL remains relatively stable, suggesting that accumulating data helps maintain the quality of the KDEs.

Despite the apparent empirical similarities to the iterative Gaussian fitting shown in Figure 1, Gaussian KDEs are theoretically distinct. Unlike Gaussian fitting, regardless of whether we accumulate or replace, the test NLL for a Gaussian KDE asymptotically diverges in theory. There are two interesting caveats: (1) when one begins with a small bandwidth, iteratively fitting KDEs can cause the NLL to initially decrease before it diverges due to the effective kernel bandwidth increasing with model-fitting iterations; (2) although accumulating data causes the NLL to diverge asymptotically, this occurs at a rate so glacial that it doesn’t pose a practical concern. If one wishes to prevent the eventual divergence, one can do so by fitting at each iteration with the optimal bandwidth for the number of data, which should be of the form $c(in)^{-1/5}$ in the $i$th model-fitting iteration as long as data accumulates at a constant rate. Practically speaking, one chooses the bandwidth for KDEs based on the number and characteristics of the data, implying that conscientious practitioners should never witness severe model collapse for KDEs in the accumulate case. For details, see Appendix Sec. B.

We also note a surprising discovery: accumulating data can yield NLLs that decrease with additional model-fitting iterations, meaning that training on real and synthetic data yields lower test loss on real data than training on real data alone. While synthetic data has been shown valuable elsewhere, e.g., Jain et al. (), we were surprised to discover this behavior in such a simple setting.
This behavior is analogous to Mobahi et al. (), which demonstrated how self-distillation of linear models can initially improve model performance by acting as increasing regularization in Hilbert space, but if too many iterations take place, the predictor is regularized towards 0 0 and performance deteriorates.

### 2.3 Model Collapse in Supervised Finetuning of Language Models

Figure: Figure 3: Model Collapse in Supervised Finetuning of Language Models. Finetuning Google’s Gemma2 2B on Nvidia’s HelpSteer 2 dataset demonstrates that model collapse occurs if previous data are replaced after each model-fitting iteration (left), whereas model collapse is avoided if new synthetic data instead accumulate with previous real and synthetic data (right).
Refer to caption: extracted/5944931/figures/model_collapse_in_new_settings/sft_lm/y=eval_loss_x=model_fitting_iteration_col=setting.png

We now turn to the third setting for studying model collapse introduced by Shumailov et al. (): supervised finetuning of language models. We begin with an instruction following dataset – Nvidia’s HelpSteer2 (Wang et al., ) – and finetune a language model before sampling new text data from it. We choose Google’s Gemma2 2B model (Team et al., ) because it is high performing and relatively small. For Replace, we fine-tune the $n$-th language model only on data generated by the $(n-1)$ language model. For Accumulate, we instead fine-tune the $n$-th language model on the starting real data plus all the synthetic data sampled from all previous models; thus, the amount of data for Replace is constant $\sim 12.5k$, whereas the amount of data for Accumulate grows linearly $\sim 12.5k*t$. Consistent with our results and with Gerstgrasser et al. (), we find that deleting data after each iteration leads to collapse whereas accumulating data avoids collapse (Fig. 3).

Figure: Figure 4: Model Collapse Under a Fixed Compute Budget. We compare deleting data after each model-fitting iteration (Replace) and accumulating data after each iteration (Accumulate) with a new fixed-compute data paradigm Accumulate-Subsample. In Accumulate-Subsample, real and synthetic data accumulate but are then subsampled so that each model is trained on a constant number of data. Accumulate-Subsample’s test loss on real data deteriorates more quickly than Accumulate’s loss but more slowly than Replace’s loss, and frequently converges, albeit to a higher plateau than Accumulate. These results hold across five settings: multivariate Gaussian modeling, language model instruction finetuning, kernel density estimation, linear regression and language model pretraining.
Refer to caption: x6.png

## 3 Model Collapse Under a Fixed Compute Budget

Thus far, we have focused on two data paradigms: Replace and Accumulate. As discussed in Sec. 2, Replace is unlikely to be an faithful model of reality because we do not delete the internet after pretraining each model, but one might also argue that Accumulate is similarly unfaithful because Accumulate requires that every new model is trained on (linearly) more data and thus requires more compute than its predecessor. Whether this criticism is valid in practice is unclear, since newer models are trained on increasing data (e.g., 1.4T tokens for Llama 1, 2T tokens for Llama 2, 15T tokens from Llama 3) and increasing GPUs (e.g., 2k GPUs for Llama1, 4K for Llama2, 16k for Llama3 (Goyal, )).
Nevertheless, for the sake of understanding the space of possible outcomes and predicting likely outcomes for future generative models, we ask and answer:

> Does model collapse occur when data accumulate but models are trained under a fixed compute budget?

We call this data paradigm Accumulate-Subsample because data accumulate but are then subsampled to ensure constant data and thus constant compute at each model-fitting iteration. To study whether model collapse occurs in Accumulate-Subsample, we use the same three generative modeling settings we’ve studied (multivariate Gaussian modeling, supervised finetuning of language models and kernel density estimation) plus two new generative modeling settings studied by prior work (Mobahi et al., ; Dohmatob et al., ; Gerstgrasser et al., ): linear regression and pretraining language models on a GPT3.5/GPT4-generated dataset of kindergarten-level text (Eldan & Li, ).

To explain how linear regression can be used as a generative model, we briefly here and direct the reader to prior work (Mobahi et al., ; Dohmatob et al., ; Gerstgrasser et al., ) for a more thorough description. We begin with our real covariates $X\in\mathbb{R}^{n\times d}$ and true linear relationship $w^{(0)}$. Initializing $\hat{w}^{(0)}=w^{(0)}$, we sample the regression targets as:

$$ $y^{(t)}\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}X\hat{w}^{(t )}+E^{(t)}\quad;\quad E^{(t)}\sim\mathcal{N}(0,\sigma^{2}I_{d})$ (11) $$

Assuming $X^{T}X$ is full rank, e.g., $n\gg d$, we fit the next linear model using ordinary least squares:

$$ $\hat{w}^{(t+1)}\operatorname{\stackrel{{\scriptstyle\text{def}}}{{\;=\;}}}(X^{ T}X)^{-1}X^{T}y^{(t)}$ (12) $$

Following Gerstgrasser et al. (), we additionally pretrain sequences of small variants of common large language models – GPT (Radford et al., ; Brown et al., ) and Llama (Touvron et al., ; ) – on TinyStories (Eldan & Li, ), a synthetic dataset of simple short stories; this combination of models, parameters and data was chosen to faithfully study model collapse in as realistic a setting as possible, subject to our limited computational budget.

Across all five generative modeling settings, we find that Accumulate-Subsample’s test loss on real data lies between the test losses of Replace and Accumulate (Fig. 4 Center). Specifically, Accumulate-Subsample (Fig. 4 center) exhibits higher test loss than Accumulate (Fig. 4, Right) but lower test loss than Replace (Fig. 4 Left), showing that the fixed compute budget imposes some cost.
In a qualitative difference, test losses on real data typically plateaus for both Accumulate-Subsample and Accumulate, whereas test losses for Replace typically diverge in an apparently unbounded manner.
These results collectively tell a consistent story: under more realistic conditions, where data accumulate and compute is bounded, model performance on real test data is unlikely to diverge.

## 4 Cardinality of Real Data vs Proportion of Real Data in Mitigating Model Collapse

Figure: Figure 5: The Value of Synthetic Data in Supervised Finetuning of Language Models. Finetuning Google’s Gemma 2 2B on Nvidia’s HelpSteer 2 dataset on different combinations of real and synthetic data demonstrates that loss grows with the number of synthetic data. Our results suggest that the test loss depends on both the proportion (p-value$=3.84\times 10^{-16}$) and the cardinality (p-value$=3.54\times 10^{-8}$) of real data. We plot the test loss against the number of real datapoints in the training set (top left). The hue represents the number of synthetic datapoints. Additionally, we display a heatmap demonstrating the effect of the number of real and synthetic datapoints on test loss (top right). We provide a graph of the test loss versus the fraction of real data, where the hue represents the cardinality of the real data (bottom left). Finally, we plot the test loss against the fraction of real datapoints, where the hue represents the number of synthetic datapoints (bottom right).
Refer to caption: extracted/5944931/figures/cardinality_vs_proportionality/sft_val_loss_v_num_train.png

We conclude by turning to a question asked by Gerstgrasser et al. () that, to the best of our knowledge, remains open:

> Which matters more for avoiding model collapse: the cardinality of real data or the proportion of real data? Relatedly, how does the value of synthetic data for reducing test loss on real data depend on the amount of real data?

These questions are highly pertinent to researchers sampling from web-scale data in order to pretrain or finetune language models.
We conduct our investigation of this question as follows:
First, we perform SFT on the HelpSteer2 dataset for Google’s Gemma 2 2B model. We sample 100k completions from the finetuned model and filter for those that are fewer than 512 tokens in length. This leaves us with over 55,000 remaining completions. We aggregate datasets containing various numbers of real and synthetic synthetic data, which are given in Figure 5, and perform SFT on these datasets starting from the original Gemma 2B model. We record and display the final test loss from this process.

This experiment provides several insights. First, both the number and proportion of real data have an impact on the test loss following SFT.
To assess this, we first transformed the number of real datapoints $n$ as $\frac{1}{n^{1/2}}$, in keeping with intuitions from classical statistics on how the log likelihood scales with the number of data points. Then, based on observation of the data, we computed

$$ $\log\left(\frac{\textrm{real data}}{\textrm{real data}+\textrm{synthetic data} }\right)$ $$

to best capture the relationship between the fraction of real data and the log likelihood. We measured $R^{2}$ values of $0.59$ for the transformed number of real data and $0.34$ for the proportion of real data. We then computed $F$-statistics for the one-term versus two term models involving each of these covariates, which gave us $p$-values of $6.9\times 10^{-25}$ and $4.6\times 10^{-25}$. These statistics suggest that both the proportion and the cardinality of real data have a statistically significant effect on the test loss, and explain a sizable fraction of the variance in the test loss.

Second, we find a difference in the effect that synthetic data has on test loss in high versus low real data regimes. In our experiments specifically, when the number of real data is 1024 or lower, we find that there is an *small but non-zero amount of synthetic data that improves the test loss when it is included*.
This suggests that practitioners fine-tuning with insufficient amounts of real data should consider supplementing with synthetic data to improve model quality.

On the other hand, when real data are plentiful, we find that more synthetic data almost always harms final model quality when the number of real data is held constant. In some cases, datasets containing only real data prove to be more valuable than datasets that contain ten times more real data mixed with synthetic data.

Although these results are preliminary, they raise some interesting questions about the role of synthetic data in SFT that deserve further exploration. In some of our experiments, we achieve better results by removing all synthetic data from the training set than by doubling the amount of real training data. More generally, when constructing datasets subject to cost constraints, these results suggest that the value of removing synthetic or low-quality data can sometimes exceed that of collecting greater volumes high-quality data.

## 5 Discussion

Our work sought to clarify and unify understanding of model-data feedback loops.
We demonstrated in three new generative modeling settings that accumulating data over time avoids model collapse, whereas replacing data over time induces model collapse.
We then demonstrated in five generative modeling settings that even when each model is trained on a fixed compute budget with a mixture of real and synthetic data, model performance does deteriorate more, but still tends to plateau.
The consistency of these results across different model types and datasets suggests that this distinction is a general phenomenon, and is not specific to any particular model, dataset, or learning algorithm.
Lastly, we explored the value of synthetic data for reducing the test loss on real data and found two different regimes: when real data are plentiful, synthetic data is harmful, but when real data are scarce, there exists an optimal amount of synthetic data that are helpful.

In our view, the data paradigm in which synthetic data accumulates from a host of models in conjunction with a constant influx of real-world data is more realistic.
Under such dynamics, where new synthetic data are added to existing real and synthetic data, model collapse appears unlikely.
Our experiments take a pessimistic viewpoint, in the sense that our experiments pay no attention to the quality of data, whereas in practice, engineers heavily filter data based on various indicators of data quality, e.g., (Brown et al., ; Lee et al., ; Wettig et al., ; Penedo et al., ; Li et al., ; Sachdeva et al., ); for a recent review, see Albalak et al. ().

## 6 Future Directions

An especially interesting future direction is how to combine synthetic data generation with filtering techniques to enable performant and efficient pretraining at scale using synthetic data. As we saw in kernel density estimation (Fig. 2) and in language model pretraining on TinyStories (Fig. 4), training on accumulating real and synthetic data can yield lower loss on real test data than training on real data alone. Identifying under what conditions, and why, this is possible is a tantalizing prospect.

Our results in Section 4 suggest that removing low-quality synthetic data from model training sets *can improve test loss more than gathering additional high-quality data*. Developing efficient removal techniques for detrimental data would streamline the model fine-tuning process and produce better alignment. Future work that further studies and quantifies the harm done by lower-quality synthetic data across broader settings could greatly improve the efficiency of model training.

## Appendix A Iterative Gaussian Model Fitting: Mathematical Results and Proofs

### A.1 Setup

###### Lemma 2 .

Using the notation of Theorem 1, we can express $\mu_{t}=\sum_{r=1}^{t}\sigma_{r-1}\frac{\overline{z_{r}}}{r}+\mu_{0}$.

###### Proof.

Note that $X_{i,t}=\mu_{t-1}+\sigma_{t-1}z_{i,t}$, where $z_{i,t}\sim\mathcal{N}(0,1)$. Therefore,

$$ $\displaystyle\mu_{t}$ $\displaystyle=\frac{1}{nt}\sum_{r=1}^{t}\sum_{i=1}^{n}X_{i,r}$ $\displaystyle=\frac{t-1}{t}\mu_{t-1}+\frac{\mu_{t-1}}{t}+\sigma_{t-1}\frac{ \overline{z_{t}}}{t}$ $\displaystyle=\mu_{t-1}+\sigma_{t-1}\frac{\overline{z_{t}}}{t}.$ $$

Therefore, $\mu_{t}=\sum_{r=1}^{t}\sigma_{r-1}\cdot\frac{\overline{z_{r}}}{r}+\mu_{0}$.
∎

###### Lemma 3 .

Under the setup described in Theorem 1, $\mathbb{E}[\frac{\sigma_{t}^{2}}{\sigma_{0}^{2}}]=\prod_{k=1}^{t}\left(1-\frac
{1}{nk^{2}}\right)\xrightarrow{t\rightarrow\infty}\frac{\sin(\pi/\sqrt{n})}{
\pi/\sqrt{n}}$.

###### Proof.

Using the recursive expression for $\mu_{t}$ in Lemma 2, we can rewrite

$$ $\displaystyle\sigma_{t}^{2}$ $\displaystyle=\frac{1}{nt}\sum_{r=1}^{t}\sum_{i=1}^{n}\left(X_{i,r}-\mu_{t} \right)^{2}$ $\displaystyle=\frac{1}{nt}\sum_{r=1}^{t}\sum_{i=1}^{n}\left(X_{i,r}-\overline{ X_{r}}+\overline{X}_{r}-\mu_{t}\right)^{2}$ $\displaystyle=\frac{1}{nt}\sum_{r=1}^{t}\left(\sum_{i=1}^{n}\left(X_{i,r}- \overline{X_{r}}\right)^{2}+n(\overline{X_{r}}-\mu_{t})^{2}\right)$ $\displaystyle=\frac{1}{t}\sum_{r=1}^{t}\left(\sigma_{r-1}^{2}S_{r}^{2}+(\mu_{r -1}+\sigma_{r-1}\overline{z_{r}}-\mu_{t})^{2}\right).$ $$

In the last line, we define $S_{r}^{2}=\sum_{i=1}^{n}(z_{i,r}-\overline{z_{r}})^{2}$. The term

$$ $(\mu_{r-1}+\sigma_{r-1}\overline{z_{r}}-\mu_{t})^{2}=\left(\sigma_{r-1} \overline{z_{r}}-\sum_{k=r}^{t}\sigma_{k-1}\cdot\frac{\overline{z_{k}}}{k} \right)^{2},$ $$

so

$$ $\displaystyle\sigma_{t}^{2}$ $\displaystyle=\frac{1}{t}\sum_{r=1}^{t}\left(\sigma_{r-1}^{2}S_{r}^{2}+\left( \sigma_{r-1}\overline{z_{r}}-\sum_{k=r}^{t}\sigma_{k-1}\frac{\overline{z_{k}}} {k}\right)^{2}\right)$ $\displaystyle\Rightarrow t\sigma_{t}^{2}$ $\displaystyle=\sum_{r=1}^{t}\left(\sigma_{r-1}^{2}S_{r}^{2}+\left(\sigma_{r-1} \overline{z_{r}}\left(1-\frac{1}{r}\right)-\sum_{k=r+1}^{t}\sigma_{k-1}\frac{ \overline{z_{k}}}{k}\right)^{2}\right).$ $$

We now compute the conditional expectations of the terms in this sum. Where $\mathcal{F}_{i}$ denotes the $i$th filtration,

$$ $\mathbb{E}[\sigma_{r-1}^{2}S_{r}^{2}|\mathcal{F}_{t-1}]=\begin{cases}\sigma_{r -1}^{2}S_{r}^{2}&r<t\\ \sigma_{t-1}^{2}\cdot\left(\frac{n-1}{n}\right)&r=t.\end{cases}$ $$

For $r=t$, we find that

$$ $\displaystyle\mathbb{E}\left[\left(\sigma_{r-1}\overline{z_{r}}\cdot\left(1- \frac{1}{r}\right)-\sum_{k=r+1}^{t}\sigma_{k-1}\cdot\frac{\overline{z_{k}}}{k} \right)^{2}|\mathcal{F}_{t-1}\right]$ $\displaystyle=\sigma_{t-1}^{2}\left(1-\frac{1}{t}\right)\cdot\frac{1}{n}.$ $$

On the other hand, when $r<t$,

$$ $\displaystyle\mathbb{E}\left[\left(\sigma_{r-1}\overline{z_{r}}\cdot\left(1- \frac{1}{r}\right)-\sum_{k=r+1}^{t-1}\sigma_{k-1}\cdot\frac{\overline{z_{k}}}{ k}-\sigma_{t-1}\cdot\frac{\overline{z}_{t}}{t}\right)^{2}|\mathcal{F}_{t-1}\right]$ $\displaystyle=\sigma_{t-1}^{2}\cdot\frac{1}{t^{2}}\cdot\frac{1}{n}+\left( \sigma_{r-1}\overline{z_{r}}\cdot\left(1-\frac{1}{r}\right)-\sum_{k=r+1}^{t-1} \sigma_{k-1}\cdot\frac{\overline{z_{k}}}{k}\right)^{2}.$ $$

Therefore,

$$ $\displaystyle\mathbb{E}[t\sigma_{t}^{2}|\mathcal{F}_{t-1}]$ $\displaystyle=(t-1)\sigma_{t-1}^{2}+\sigma_{t-1}^{2}\cdot\left(1-\frac{1}{n} \right)+\sigma_{t-1}^{2}\cdot\left(\frac{t-1}{t}\right)\cdot\left(\frac{1}{n} \right)+\sigma_{t-1}^{2}\cdot\left(1-\frac{1}{t}\right)^{2}\cdot\left(\frac{1} {n}\right)$ $\displaystyle=\sigma_{t-1}^{2}\left(t-1+1-\frac{1}{n}+\frac{1}{tn}-\frac{1}{t^ {2}n}+\frac{1}{n}-\frac{2}{tn}+\frac{1}{t^{2}n}\right)$ $\displaystyle=\sigma_{t-1}^{2}\left(t-\frac{1}{tn}\right).$ $$

It follows that

$$ $\mathbb{E}[\sigma_{t}^{2}|\mathcal{F}_{t-1}]=\sigma_{t-1}^{2}\left(1-\frac{1}{ t^{2}n}\right)<\sigma_{t-1}^{2}$ $$

for all $t$. Thus, $\{\sigma_{t}^{2}\}_{t}$ is a supermartingale, and

$$ $\sigma_{t}^{2}\xrightarrow{a.s.}\sigma_{\infty}^{2}$ $$

because $\sigma_{t}^{2}$ is bounded below by 0 0. Therefore, we still have convergence. Next, letting $m_{t}=\mathbb{E}[\sigma_{t}^{2}]$, we have

$$ $m_{t}=m_{t-1}\left(1-\frac{1}{t^{2}n}\right)=\cdots=\sigma_{0}^{2}\prod_{k=1}^ {t}\left(1-\frac{1}{k^{2}n}\right),$ $$

so

$$ $\displaystyle\mathbb{E}[\sigma_{t}^{2}]=\sigma_{0}^{2}\prod_{k=1}^{\infty} \left(1-\frac{1}{k^{2}n}\right).$ (13) $$

By a theorem of Euler, this is equal to

$$ $\sigma_{0}^{2}\frac{\sin(\pi/\sqrt{n})}{\pi/\sqrt{n}}.$ (14) $$

∎

Observe that by performing a variable replacement and using L’Hospital’s rule, it is clear that $\lim_{n\rightarrow\infty}\mathbb{E}[\sigma_{t}^{2}]=\sigma_{0}^{2}$.

Finally, we are able to compute $\mathbb{E}[(\mu_{t}-\mu_{0})^{2}]$.

###### Corollary 4 .

The expected error in the mean

$$ $\mathbb{E}[(\mu_{t}-\mu_{0})^{2}]=\sigma_{0}^{2}\left(1-\prod_{k=1}^{t}\left(1 -\frac{1}{k^{2}n}\right)\right).$ (15) $$

###### Proof.

Using the recursion from Lemma 2 and the expression for the variance in Lemma 6, we can rewrite

$$ $\displaystyle\mathbb{E}[(\mu_{t}-\mu_{0})^{2}]$ $\displaystyle=\sum_{k=1}^{t}\frac{\mathbb{E}[\sigma_{k-1}^{2}]}{nk^{2}}$ $\displaystyle=\sigma_{0}^{2}\sum_{k=1}^{t}\frac{1}{k^{2}n}\prod_{\ell=1}^{k-1} \left(1-\frac{1}{\ell^{2}n}\right)$ $\displaystyle=\sigma_{0}^{2}\sum_{k=1}^{t}\left(\prod_{\ell=1}^{k-1}\left((1- \frac{1}{\ell^{2}n}\right)-\prod_{\ell=1}^{k}\left(1-\frac{1}{\ell^{2}n}\right )\right)$ $\displaystyle=\sigma_{0}^{2}\left(1-\prod_{k=1}^{t}\left(1-\frac{1}{k^{2}n} \right)\right).$ $$

∎

Therefore,

$$ $\lim_{t\rightarrow\infty}\mathbb{E}[(\mu_{t}-\mu_{0})^{2}]={\sigma_{0}^{2}} \left(1-\frac{\sin(\pi/\sqrt{n})}{\pi/\sqrt{n}}\right).$ $$

###### Lemma 2 .

###### Proof.

###### Lemma 3 .

###### Proof.

###### Corollary 4 .

###### Proof.

## Appendix B Iterative KDE Fitting: Mathematical Results and Proofs

In this section, we prove that the NLL diverges when iteratively fitting KDE’s regardless of whether one accumulates or replaces data from previous iterations.

###### Theorem 5 .

In the replace setting described in Section 2.2, as long as one holds the bandwidth constant, the NLL asymptotically diverges.

###### Proof.

Define $f_{0}$ as the density function for the data distribution from which the original data $x_{1},...,x_{n}$ are sampled. Define $K_{h}$ to be the Gaussian kernel function with fixed bandwidth $h$. One can rewrite the fitted distribution at iteration $t$ as

$$ $D_{t}=K_{h}*D_{t-1}$ $$

where $*$ denotes the standard convolution of densities.

By a simple recursion, it is clear that $D_{t}=K^{*t}*D_{0}$. When two Gaussian kernels with bandwidths $a$ and $b$ are convolved, a basic calculation shows that the resulting effective bandwidth is $\sqrt{a^{2}+b^{2}}$. Consequently, by an inductive argument, the effective bandwidth of $K^{*t}$ is $h\sqrt{t}$. Therefore,

$$ $\lim_{t\rightarrow\infty}K^{*t}*D_{0}=\lim_{t\rightarrow\infty}K_{h\sqrt{t}}*D _{0}=0$ $$

because as the bandwidth goes to $\infty$, the likelihood of any point goes to 0 0. Hence, regardless of the choice of test data, the negative log likelihood diverges to $-\infty$.
∎

The same conclusion holds when one accumulates rather than subsampling data:

###### Theorem 6 .

For any non-trivial kernel (i.e. a kernel whose Fourier transform is not $1$), 2.2, the NLL diverges.

###### Proof.

We adopt the same notation as in Theorem 5, except this time $K$ denotes a general kernel $K$ that doesn’t necessarily need to be Gaussian. In this instance, it is more convenient to work in frequency space, where convolution in probability space corresponds to multiplication.

Define $\varphi_{0}$ as the Fourier transform (FT) of $f_{0}$, also called the characteristic function.
Let $\kappa$ denote the FT of $K$. Then

$$ $\varphi_{t}=\kappa\cdot\varphi_{t-1}$ $$

where $\cdot$ denotes standard complex multiplication. Define $\delta_{t}=\frac{\phi_{t}}{\phi_{0}}$ so that $\varphi_{t}=\delta_{t}\cdot\varphi_{0}$. Define $d_{t}=\varphi_{t}/\varphi_{0}$, and let $a_{t}=\frac{1}{t}\sum_{i=0}^{t}d_{i}$. Using this notation,

$$ $\displaystyle d_{t}$ $\displaystyle=\kappa\cdot a_{t-1}$ (16) $\displaystyle a_{t}$ $\displaystyle=\left((t-1)a_{t-1}+d_{t}\right)/t.$ (17) $$

We see that $a_{t}=L_{t,K}(a_{t-1})$ is an affine map with slope $((t-1)+\kappa)/t$ and intercept 0 0. Suppose that the characteristic function of the density converges to $\varphi_{\infty}$. Then the map $a_{t}$ has a fixed point. As long as $\kappa\neq 1$, this fixed point must satisfy the equation

$$ $\displaystyle\varphi$ $\displaystyle=((t-1)+\kappa)\varphi$ $\displaystyle\Rightarrow 0$ $\displaystyle=\left((t-1)+\kappa)/g-1\right)\varphi$ $\displaystyle\Rightarrow 0$ $\displaystyle=\left(-1+\kappa\right)\varphi\Rightarrow\varphi=0.$ $$

Note that if $\varphi_{\infty}=0$, its inverse FT is a function that has 0 0 probability density everywhere in probability space. Equivalently, the variance of $f_{t}$ diverges to $\infty$.

∎

Although the NLL eventually diverges in the accumulate case, it is clear from the expression for $a_{t}$ that this divergence occurs very slowly.

For a Gaussian kernel, both the replace and accumulate case offer an interesting shared insight. Throughout the iterative fitting process, regardless of whether we accumulate or replace, the bandwidth monotonically grows. Therefore, when one starts this process with a very small bandwidth smaller than the optimal bandwidth for the density being fit, one could initially observe a decrease in the negative log likelihood as the bandwidth approaches its optimum.

Finally, model collapse, while inevitable with a fixed bandwidth, can be avoided in all cases by shrinking the bandwidth at a sufficiently fast rate. Since practitioners typically optimize their bandwidth according to the amount of the data that they have, the bandwidth should have the form $c(tn)^{1/5}$ where $c$ is a constant. In this setting, model collapse is avoided entirely.

###### Theorem 7 .

Suppose that data accumulates as in Section 2.2 for a Gaussian kernel. Let the bandwidth at the $n$th model-fitting iteration be $c(tn)^{-1/5}$ for a constant $c$. Then the asymptotic variance of the limiting KDE is finite.

###### Proof.

Let $K_{c(tn)^{-1/5}}$ denote the kernel at the $t$th model-fitting iteration. Let $f_{0}$ denote the original distribution, and define $f_{t}$ to be the distribution of the KDE at the $t$th iteration.

We can write

$$ $\displaystyle f_{t}$ $\displaystyle=\frac{1}{t}\sum_{i=1}^{t}f_{i-1}*K_{c(in)^{-1/5}}$ $\displaystyle=\left(1-\frac{1}{t}\right)\cdot\left(\frac{1}{t-1}\sum_{i=1}^{t- 1}f_{i-1}*K_{c(in)^{-1/5}}\right)+\frac{1}{t}f_{t-1}*K_{c(tn)^{-1/5}}$ $\displaystyle=\left(1-\frac{1}{t}\right)f_{t-1}+\frac{1}{t}f_{t-1}*K_{c(tn)^{- 1/5}}$ $\displaystyle=\left(\left(1-\frac{1}{t}\right)K_{0}+\frac{1}{t}K_{c(tn)^{-1/5} }\right)$ $$

where $K_{0}$ is the identity kernel, or equivalently the Gaussian kernel with 0 0 bandwidth.

Therefore, we find that

$$ $f_{t}=f_{0}*\stackMath\mathbin{\stackinset{c}{0ex}{c}{0ex}{\ast}{\bigcirc}}_{i =1}^{t}\left(\left(1-\frac{1}{i}\right)K_{0}+\frac{1}{i}K_{c(in)^{-1/5}}\right).$ $$

Define $W_{i}$ to be a random variable that is $K_{c(in)^{-1/5}}$ with probability $\frac{1}{i}$ and $K_{0}$ with probability $1-\frac{1}{i}$.
We can rewrite $X_{t}$, a random variable drawn at the $t$th fitting iteration as

$$ $X_{t}=X_{0}+\sum_{i=1}^{t}W_{i}.$ $$

All of $X_{0},W_{1},...,W_{t}$ are independent. The variance is given by

$$ $\displaystyle\textrm{Var}(X_{t})$ $\displaystyle=\textrm{Var}(X_{0})+\sum_{i=1}^{t}\textrm{Var}(W_{i})$ $\displaystyle=\textrm{Var}(X_{0})+\sum_{i=1}^{t}\frac{1}{i}\times\frac{c}{(in) ^{2/5}}$ $\displaystyle=\textrm{Var}(X_{0})+\frac{c}{n^{2/5}}\sum_{i=1}^{t}\frac{1}{i^{7 /5}}.$ $$

As $t\rightarrow\infty$,

$$ $\textrm{Var}(X_{t})\rightarrow\textrm{Var}(X_{0})+\frac{c}{n^{2/5}}\sum_{i=1}^ {\infty}\frac{1}{i^{4}}<\infty.$ $$

Therefore, when the kernel size is appropriately adjusted, the variance of the KDE under accumulate converges.
∎

###### Theorem 5 .

###### Proof.

###### Theorem 6 .

###### Proof.

###### Theorem 7 .

###### Proof.

## Appendix C Experimental Results: Sweep Configurations

### C.1 Model Collapse in Multivariate Gaussian Modeling

To study model collapse in multivariate Gaussian modeling, we ran the following YAML sweep:

Seeds were swept from 0 to 99, inclusive.

### C.2 Model Collapse in Kernel Density Estimation

To study model collapse in multivariate Gaussian modeling, we ran the following YAML sweep:

Seeds were swept from 0 to 99, inclusive.

### C.3 Model Collapse in Kernel Density Estimation

To study model collapse in kernel density estimation, we ran the following YAML sweep:

Seeds were swept from 0 to 99, inclusive.

### C.4 Model Collapse in Linear Regression

To study model collapse in linear regression, we ran the following YAML sweep:

Seeds were swept from 0 to 99, inclusive.
Note: We ran this sweep as 9 separate sweeps; to understand why, see [this GitHub issue.](https://github.com/wandb/wandb/issues/8549)

## Appendix D Additional Experimental Results for Model Collapse Hyperparameters

Due to space limitations in the main text, we can oftentimes only present a subset of runs corresponding to a subset of hyperparameters. We present additional figures with a wide range of hyperparameters here for completeness.

### D.1 Additional Results for Model Collapse in Linear Regression

Figure: Figure 6: Linear Regression for Data Dimension $d=1$ and variance $\sigma^{2}=0.10$.
Refer to caption: x13.png

Figure: Figure 7: Linear Regression for Data Dimension $d=1$ and variance $\sigma^{2}=1.00$.
Refer to caption: x14.png

Figure: Figure 8: Linear Regression for Data Dimension $d=1$ and variance $\sigma^{2}=10.0$.
Refer to caption: x14.png

Figure: Figure 9: Linear Regression for Data Dimension $d=3$ and variance $\sigma^{2}=0.10$.
Refer to caption: x9.png

Figure: Figure 10: Linear Regression for Data Dimension $d=3$ and variance $\sigma^{2}=1.00$.
Refer to caption: x15.png

Figure: Figure 11: Linear Regression for Data Dimension $d=3$ and variance $\sigma^{2}=10.0$.
Refer to caption: x15.png

Figure: Figure 12: Linear Regression for Data Dimension $d=10$ and variance $\sigma^{2}=0.10$.
Refer to caption: x16.png

Figure: Figure 13: Linear Regression for Data Dimension $d=10$ and variance $\sigma^{2}=1.00$.
Refer to caption: x17.png

Figure: Figure 14: Linear Regression for Data Dimension $d=10$ and variance $\sigma^{2}=10.0$.
Refer to caption: x17.png

Figure: Figure 15: Linear Regression for Data Dimension $d=32$ and variance $\sigma^{2}=0.10$.
Refer to caption: x18.png

Figure: Figure 16: Linear Regression for Data Dimension $d=32$ and variance $\sigma^{2}=1.00$.
Refer to caption: x19.png

Figure: Figure 17: Linear Regression for Data Dimension $d=32$ and variance $\sigma^{2}=10.0$.
Refer to caption: x19.png

Figure: Figure 18: Linear Regression for Data Dimension $d=100$ and variance $\sigma^{2}=0.10$.
Refer to caption: x20.png

Figure: Figure 19: Linear Regression for Data Dimension $d=100$ and variance $\sigma^{2}=1.00$.
Refer to caption: x21.png

Figure: Figure 20: Linear Regression for Data Dimension $d=100$ and variance $\sigma^{2}=10.0$.
Refer to caption: x21.png

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 8
presentation: 3
questions: - Can the authors please clarify their novelty a bit more?
- Can the authors please discuss how they expect their results to hold when training significantly larger models, or changing the quality of the synthetic data?
soundness: 3
strengths: - *Important Problem Setting.* The problem of model collapse is a pressing problem and I found the authors investigation into the more nuanced issues of the model-data feedback loop compelling: that likely parts of synthetic and real data will be aggreagated, the budget assumptions etc. 

- *Thorough Experiments.* The authors conduct well-designed experiments on a variety of models and settings, including Gaussian models, kernel density estimators, linear regression and language models on a variety of datasets. While some of these models and datasets are fairly simple, they nonetheless help suport the author's claims. 

- *Theoretical Insights.*  In addition to the empirical insights, the authors are able to proud some theoretical guarantees. This helps complement the empirical results with simpler models.
summary: Update: I appreciate the authors efforts to address my concerns, and thus would like to raise my score. Ideally, I would change it to a 7 b/c I feel it does deserve acceptance but not spotlight/oral. 

This paper studies how training on synthetically generated data in the model-data feedback loop. In particular the authors claim to focus on a more nuanced setting than previous papers on the topic by considering settings where (i) parts of the real-data and synthetic data are used together (instead of discarding the real-data entirely in favor of synthetic), (ii) only a fixed budget is available for updating the model, and (iii) consider how important the proportion vs. cardinality is when updating the model. The authors demonstrate that keeping some of the real data can help prevent collapse, that the right amount of synthetic + real data can improve performance, and in some cases, no amount of synthetic data is outperform real data. The authors provide some proofs to demonstrate their claims too.
weaknesses: - * Novelty.* The novelty of this work is fairly limited since it builds quite a bit on existing work. It starts by addressing some of the claims from Gertgrasser et al, and then uses the model/settings in Shumailov et al. The settings the authors tried out (not replacing data en-masse. etc) are important but do seem a bit incremental conceptually.

- *More Real-settings.* The authors motivate their work a fair bit from the perspective of language models. I would like to have seen more experiments on this setting under a couple of different model strengths and synthetic data quality.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. the paper starts with MULTIVARIATE GAUSSIAN MODELING, can the findings still hold for other model, like VAE, diffusion model, or Transformer. Somehow,  I feel the paper make too strong claim since there are many generative model.

2. the reprodubility is unclear at this time.

3. the math proof in Appendix is mainly for Guassina model, it is unclear how it can generalize to more generative models.
soundness: 3
strengths: 1. The research topic is very interested. It tries to study the model collapse issue when the world have more and more sythetic data due to generative model.

2. The paper contain a lot of experiemts to valdiate their claim.

3. The finding in this work is interesting. they find a non-trivial interaction between real and synthetic data, where the value of synthetic
data for reducing model test loss depends on the absolute quantity of real data.
summary: this work study the model collapse with the sythetic data.
weaknesses: 1. the paper starts with MULTIVARIATE GAUSSIAN MODELING, can the findings still hold for other model, like VAE, diffusion model, or Transformer. Somehow,  I feel the paper make too strong claim since there are many generative model.

2. the reprodubility is unclear at this time.

3. the math proof in Appendix is mainly for Guassina model, it is unclear how it can generalize to more generative models.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: 1. This paper presents iterative settings in which performance either suffers from model collapse (replace) or avoids it (accumulate and accumulate+subsample). Have the authors explored ways to identify the breaking point between these two modes? For example, would the results hold if, instead of adding $n$ samples at each iteration, we add an increasing amount of synthetic data (say, $2n$ each time)?

2. Another question is about the relative quality of the synthetic data. Presumably, there is a connection between how well the model learns the task and whether its generated data can effectively train a second iteration. Do the authors have comments on this?


3. Regarding the observations in the blob experiment, the authors note a tendency: "Interestingly, for specific pairs of datasets and numbers of samples per iteration, training on real data while accumulating synthetic data can yield lower loss on real test data than training on real data alone." For these specific settings where the true distribution is known (I believe that it is the case?), it would be valuable to report the NLL value of the true distribution as a reference point.

4. Have the authors considered extending the result from Theorem 1 to the accumulate-subsample setup? Or relating this result to their experimental results? This could strengthen the contribution of this work.
soundness: 4
strengths: 1. Motivation. The research question is clearly defined and contextually relevant. Exploring the impact of training with synthetically generated data is important for advancing the field of machine learning.

2. Experiment section. The experiments are thorough, relevant, and insightful. The authors verify that the hypotheses they are testing hold across a wide range of models and data generation processes. They present interesting experiments that control for different parameters, offering clear insights into the scope of the problem. The authors clearly list the questions they aim to answer and provide solid experiments to support their conclusions, along with clear and detailed explanations of their experimental setup.

3. Writing and structure. The paper is well written and follows a logical and clear structure.
summary: The paper investigates the impact of iteratively training a generative model on a combination of real and synthetic data generated from a previously trained model. This is an important area of research to explore and understand, as exclusively training models on synthetic data will eventually fail, as highlighted by prior work. 
The authors conduct experiments to validate previous findings, offering new evidence to clarify cases in which model collapse can be avoided. They also explore additional experimental settings in which they control for the ratio of synthetic/real data and the absolute quantity of real data to gain further insight into the limits of training with synthetic data.
weaknesses: 1. Contribution. The majority of the experimental section (4/9 pages) validates the findings of previous work (Gerstgrasser et al., 2024). The experiments verify that retraining with a mix of synthetic and real data can avoid model collapse in three additional settings. Although it is valuable to reproduce and verify previous findings, this should not constitute the main contribution of a novel work. Emphasizing and clarifying the unique contributions and findings of this paper would strengthen its impact.

2. Presentation. Overall, the presentation of the work could be improved in several ways. First, a clearer discussion of related work to better position this paper's contributions is needed. In the introduction, various papers on model collapse are listed with the comment that they have differing methodologies and conclusions. What are the differing methodologies and conclusions? A brief summary of these differences and a clearer statement of this work's position relative to others is required. This should be easy to achieve as the paper is one page short of the limit, so I am not sure what justifies the almost nonexistent discussion of the listed previous work. Second, the presentation of figures could be improved (e.g., readability of axis labels, font size, relative scaling of the plots). Some figures use notation like 1.2×10^0 instead of simply 1.2, and there is a lot of wasted empty space in the figures. Lastly, there are several errors in the references, as several published works are cited as arXiv references. This gives the impression of an unpolished work.

Minor

1. Some of the writing is needlessly sensationalized. For example, describing "that model collapse is caused by deleting past data en masse and avoided by instead accumulating real and synthetic data" as a "provocative pair of claims" feels excessive. It’s clear that using real data works and that purely generated data does not, so it is not surprising that a middle ground exists. Especially since this finding was already made by previous work. 

2. The use of tweets and LinkedIn posts as references gives the paper an unserious tone, though this may be a matter of personal preference.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. The study's empirical analysis is limited to a 2B parameter language model. Given the prevalence of larger models like Llama 3 and Mistral, do the authors plan to extend their experiments to these more widely used models to assess the generalizability of their findings?

2. This work primarily explores synthetic data quantity, but the quality of synthetic data is also a significant factor. Would lower-quality synthetic data exacerbate collapse, and could filtering or curating synthetic data mitigate this effect?
soundness: 2
strengths: 1. The studied problem of model collapse under increasing AI-generatd data is prompt and interesting.

2. The theories and empirical studies round up the good work.

3. This paper offers actionable guidance on how to balance real and synthetic data in training.
summary: This paper addresses the risks of training generative models on datasets increasingly dominated by AI-generated (synthetic) data, studying "model collapse"—where model performance deteriorates as synthetic data accumulates. Through theoretical and empirical analyses, the authors explore when and how this collapse can be mitigated. They test three data-handling approaches (replace, accumulate, and accumulate-subsample) across various generative model settings. Findings suggest that model collapse is avoidable if synthetic data is accumulated alongside real data, and highlight how different ratios of real to synthetic data influence model performance.
weaknesses: 1. Some of the theoretical proofs hinge on idealized assumptions (e.g., independence and Gaussian distribution in certain proofs). These may not fully apply in practical, real-world datasets.

2. The "accumulate-subsample" paradigm offers a practical perspective on fixed compute budgets but is tested under simplified conditions. Additional real-world constraints (e.g., dynamic memory handling) are not considered in depth.

3. While language models are likely the most relevant generative model for readers, the theoretical analysis does not extend to this model type and is instead focused on Gaussian and kernel density estimation models. Additionally, the size of the language model used in this study is quite small. While contemporary models like Llama range from 7B to 70B parameters, the paper’s experiments are limited to Gemma2 with only 2B parameters.

Meta-review: This paper studies model collapse, an important shortcoming of generative models when they are trained on synthetic data. The major concern that reviewers have, including public reviewers from the model collapse community, is that the paper is of limited novelty and doesn't fully contextualize itself in the broader model collapse literature. The reviewers judged this paper fails to properly discuss key papers making similar contributions. There has been a long exchange between the authors and the reviewers. My recommendation for rejection is based on reading the paper and on the authors' failure to address all the reviewers' concerns when it comes to previous work.

Decision: Reject