Model Collapse Demystified: The Case of Regression
Authors: Elvis Dohmatob
Yunzhen Feng
Julia Kempe
Venue: NeurIPS 2024 poster (2024)
Keywords: Model Collapse
Kernel Ridge Regression
High dimensional asymptotics
Synthetic Data
Scaling Laws
Primary area: learning_theory
TL;DR: Exact characterization of Model collapse (retraining on data generated from prior models) in a theoretical framework; exact formulae for bias and variance decay; new phenomena emerging.

Abstract: The era of proliferation of large language and image generation models begs the question of what happens if models are trained on the synthesized outputs of other models. The phenomenon of "model collapse" refers to the situation whereby as a model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e. the model collapses. In this work, we investigate this phenomenon within the context of high-dimensional regression with Gaussian data, considering both low- and high-dimensional asymptotics. We derive analytical formulas that quantitatively describe this phenomenon in both under-parameterized and over-parameterized regimes. We show how test error increases linearly in the number of model iterations in terms of all problem hyperparameters (covariance spectrum, regularization, label noise level, dataset size) and further isolate how model collapse affects both bias and variance terms in our setup. We show that even in the noise-free case, catastrophic (exponentially fast) model-collapse can happen in the over-parametrized regime. In the special case of polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast to slow rates. We also propose a simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.

Model Collapse Demystified: The Case of Regression
Elvis Dohmatob Yunzhen Feng? Julia Kempelt
FAIR, Meta
Center for Data Science, New York University
Courant Institue of Mathematical Sciences, New York University *Correspondence to dohmatob@meta.com com
Abstract
The era of proliferation of large language and image generation models begs the question of what happens if models are trained on the synthesized outputs of other models. The phenomenon of "model collapse" refers to the situation whereby as model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e. the model collapses. In this work, we investigate this phenomenon within the context of high-dimensional regression with Gaussian data, considering both low- and high-dimensional asymptotics. We derive analytical formulas that quantitatively describe this phenomenon in both and wwer-parameterized regimes. We show how test eπrоr increases linearly in the number of model iterations in terms of all problem hyperparameters (covariance spectrum, regularization, label noise level, dataset size) and further isolate how model collapse affects both bias and variance terms in our setup. We show that even in the noise-free free case, catastrophic (exponentially fast) model-collapse can happen in the over- weer-paraetrrzeed regime. In the special case of polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast slow rates. We also propose simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.
Introduction
Model collapse describes the situation where the performance of large language models (LLMs) large image generators degrade more and more Al-generated data becomes present in their training dataset [44] Indeed, in the early stages of the generative Al evolution (e.g the ChatGPT-xyz series of models), there is emerging evidence suggesting that retraining a generative AI model on its own outputs can lead to various anomalies in the model's later outputs. This phenomenon has been particularly observed in LLMs, where retraining on their generated content introduces irreparable defects, resulting in what is known as "model collapse", the production of nonsensical or gibberish output Though several recent works demonstrate facets of this phenomenon empirically in g 9 21]. theoretical understanding is still missing.
various settings 23 32
33
In this work, initiate theoretical study of model collapse in the setting of high figh-dimensional supervised-laaming with linear regression. This is equivalent to kernel regressior which serves effective proxy for neural networks in various regimes, for instance in the infinite-widtt limit 37 49 25 28 or the lazy regime training [12.. 11] characterize the power-law generalization error of regularized least-squares kernel algorithms, assuming a power-decay spectrum of the kernel (capacity) and of the coefficients of the target function (source). Source and capacity power decay
'we present the linear regression setting for ease, the extension to the kerel setting straightforward.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

37
10-
13"
40
18
10
20
167
12
(a) Isotropic covariance spectrum E= Ta. Here, we show the evolution of test error for different sample
(h)
Power-law covariance spectrum. Refer Eqn 23 The setup is: d = 300, To = 600, o = do = 1, diag(A), A2), where A x k k-2 Left
size (7), different levels of ridge-regularizatio (A), and training data from different generations (n) of fake data, The setup is: input-dimension d = 300, sample size fake data generator To = 600, noise levels a 0.1 and du = 0.2. Left plot is for T = 1000 and different values of X. Notice the U-shape of the curves for large values of indicating the existence of sweet spot (optimal regularization parameter). Right plot is for À = 10-3 and different values T. The broken lines correspond to the theoretical result established in Theorem 4.1
plot corresponds to T 10, 000 and Right plot corresponds to adaptive regularization À f
where À = X(T') as proposed in [14]. See Section Difor details. The broken curves are as predicted by our Theorem 5. Though l = feree is optimal in classical case, is not in the setup of model collapse. In fact here, the test error diverges with sample size T. Our theory proposes a corrected value this exponent which gracefully adapts to synthesized data. See F (Appendix) for results on MNIST [[6]
Figure 1: Demystifying model collapse. Refer to Appendix D for details on the experimental setup.
capture properties of the data and the model that give rise to power law scaling of test error in terms of data set size and model capacity, as empirically observed e.g. in [26] 24 More recently, scaling laws have been shown for kernel models under the Gaussian design, e.g. in [46 13 [4] for regression and [15) for classification.
33 43
3
study scaling laws for regression the random feature model.
Summary of Main Contributions. Following the rich tradition in prior works outlined above, we study the Gaussian design where the input is sampled from multivariate zero-mean Gaussian (O, M) and labels y are determined by a linear ground truth function with independent label noise e as y = x "D + c (we present the linear regression setting for ease, the generalization to the kernel setting is straightforward). At each generation step, an approximation up is learned from the data, and used to generate new, fake /synthetic labels for the next generation. Note that the machine learner has no control over the fake data generation process. It only sees data from a stage n of this process, which is then used to fit a downstream predictor. Our main findings can be summarized as follows:
(I) Exact Characterization of Test Error under Iterative Retraining on Synthesized Data. In Section 4 (Theorem 4.3), we obtain analytic formulae for test error under the influence of training data with fake/ / synthesized labels. For n-fold iteration of data gettreeene,,,on, this formula writes
Exest =
Eelean + A Bias + n CA,(A, T,To,
(1)
where Edean is the usual test error of the model trained on clean data (not Al-generated) and of the label noise level in the clean data distribution. The non-negative term P precisely highlights the effects of all the relevant problems parameters: the feature covariance matrix >, sample size T, original data size To, label noise level in the fake data distribution o², and regularization X. The non-negative term A Bias is increase in bias brought about by the iterative synthetic data regime (Corollary (4.4). if each
generation process. This term disappears in the mnder-parametric stage in the process was fitted sufficiently many samples To compared to the input dimension d (i.e if To 2 d). In the over-paramelrized case where To < d, this term is either a constant (Theorem 4.5) or an increasing function of n. depending on whether the design matrix stays the same or is resampled across different generations (Theorem(4.6) Notably, even in the case of noiseless labels (when do- = 0), the downstream model converges to Gaussian process around zero exponentially fast with the number of iterations n, leading to "catastrophic" model collapse.
A direct consequence of is that, as the number of generations becomes large, the effect of re-synthesizing will make learning impossible. We note that the multiplicative degradation in scaling with the number of generations memory models and their variants and empirically observed there. Illustration in Figures La and 2
completely analogous to what has been shown in [88 for infinite
(2) Modified Scaling Laws. Turning to the special case of power-law spectra of the covariance matrix Σ, which allows to derive test-error scaling laws 1T 46 14 29], we obtain in Section (see Theorem
5. precise new scaling laws of the test error that quantitatively highlight the negative effect of training

---

Goveratee
62-
10-
1.3
02-
61
Geneationn
16 12"
20
4x
(a) Identical intermediate design matrices X,, Xo Vnn
(b) Independent intermediate design matrices.
Figure 2: Model collapse in the case of noiseless over- peerpparametrized synthetic data generator. Here d = 300. the sample sizes for the different versions of the fake data generator are equal, i.e T, To = d/2 for all and noise levels do = and 0.1. Everything else is as in the setting of Figure Ta Broken lines correspond the theoretical estimates given Theorem 4.3 As predicted by our theory, the test error of the model fitted on synthetic data (n 2 1) increases (relative to the baseline n = O. corresponding to training on clean data). The model collapse here, even in the absence of noise (oe = 0), is due to the fact that the synthetic aata-generator does not have access enough data capture the true labelling function. (a) Importantly, and in accordance theory, the amount of model collapse in the case X,, Xo is due an increase in bias error of the model and does not depend on the number of generations n as long as n 2 1. (b) In contrast, for the
of the test
case where the X, "s are independent, the increase in bias term grows with nt, leading to "catastrophic" model collapse (Theorem 4.6 Refer to Appendix D for the experimental setup.
on synthetically generated data. Further exploiting analytic estimates, obtain (Corollary 5.2) the optimal ridge regularization parameter À as a function of all the problem parameters (sample size, spectral exponents, strength of fake data-generator, etc.). This new regularization parameter corresponds to a correction of the the value proposed in the classical theory on clean data [14), and highlights novel crossover phenomenon where for an appropriate tuning of the regularization parameter, the effect of training on fake data is degradation of the fast error rate in the noiseless regime [14 [1] much slower error rate which depends
the amount of true data which the
fake data- getaaeeeerrtor was trained in the first place. On the other hand, a choice of regularization which optimal for the classical setting (training on real data), might lead to catastrophic failure: the test error diverges. See Figure [b] an illustration.
Related Work
Current LLMs [17 30 10 47, including GPT-4 1 were trained on predominantly human-generated
text; similarly, diffusion models like DALL-E 40). Stable Diffusion [42), Midjourney (35) are trained on web-scale image datasets. Their training corpora already potentially exhaust all the available clean data on the internet. A growing number of synthetic data generated with these increasingly popular models starts populate the web, often indistinguishable from "real" data. Recent works call attention to the potential dramatic deterioration in the resulting models, an effect referred to as "model collapse" " (44).
Empirical evidence of model collapse has been reported across various domains 23
44 attribute collapse to finite sampling bias and function approximation errors in the (single) Gaussian case but only provide lower bounds without detailed analytic expressions. (7] analyze the training process the distribution level using both clean and synthetic data and provide stability results. However, these results do not account for finite samples and are only valid locally in parameter space, making them more relevant to fine-tuning rather than training from scratch. 2 examine "self-consuming loops" in the Gaussian case by assuming a sampling bias that reduces data variance with each generation- (martingale) assumption that we do not require. These studies lack a comprchensive theoretical framework to quantify model collapse and its impact on scaling laws. Our work addresses these gaps by providing an analytic theory that captures how model collapse emerges from training on synthetic data, providing deeper understanding that goes beyond merely identifying the collapse. concurrent study by [18] demonstrate that model collapse in foundation models can be attributed to a breakdown in scaling laws 26 24 where increasing the sample size eventually fails to improve model performance. This finding, theoretically shown for discrete data in variants of the infinite memory model, complements our analytical results on how synthetic data alters the rate of scaling laws, as discussed in Scction
Some theoretical studies [4 2 18 have begun exploring this phenomenon.
3

---

3 Theoretical Setup
We now present setup which is simple enough be analytically tractable, but rich enough exhibit a wide range of regimes to illustrate a range of new phenomena that emerge with model collapse.
Data Distribution and Synthetized Data. Consider the distribution o on R x R given by (Input) x (0, E), (Noise) e [N0, o²), indep. of x (Output/Label) y - x wo + e.
The positive integer d is the input-dimension, the vector wn E IR defines the ground-truth labelling function z + x Wo. the matrix Σ E Rdxd is the covariance structure of the inputs. The scalar o² is the level of label noise. Here, we consider the linear case for clarity. We describe the extension to the kernel setting in appendix[[] Thus, in classical linear regression, given sample YY {(11, B1)..... (Er, 3r)3 of size T Irom Pr 22 one seeks a linear model w E IRd with small test error
Etest (w) == Ez,y[(<' T @ - y)
(3)
where (ar. y) ~ Px, o. a random clean test point. In our setup for studying model collapse, the training data (X, Y) is sampled from iterative loop where each generation of the model serves as the labeller for the data for the next generation. This process is described below.
Structure of the Synthesized / Fake Data Generator. Consider sequence of data distributions
wo + E + -... + Ps. ....
(4)
where T.. 's is defined recursively by T. = Wo, and
wn
n-11, for 1
(5)
where Yn == m F. and Fit B) = OLS(A,, B): ABB is ordinary-least squares (OLS). The design matrices (Xn)n20 are of shapes Tù x d, each with iid rows from N (0, 2))
The sequence of noise vectors (En) forms an independent collection, which is independent of the (Xn)n20+ each En E fn has iid components εn,i from N(O, o;)) Refer to Figure 3
Fubo Syntiisized Data Gerevaeer
G W0
6,
Gn
Bise A Biii t0 Staling
Modts Canssaton ig Modi fruning
- Xetty Eules)
F. Xjū. Ei(es)
X, E,le)
Figure 3: Illustration of the theoretical framework. The process begins with the original model Wo (wo) and the original dataset Yo). synthetic data generators w1 tn are iteratively fit on data labelled by the previous model with label noise do. using To samples each. We evaluate the test error (with respect to the ground truth labels from wo) of @ trained on (X, = (Xn, Yn) using T' samples with label noise a and regularization coefficient X.
Thus, in summary, each Wn results from fitting a model on a dataset of size Ta-1 from P.. for every generation index n > 1.
The Downstream Model: Ridge Regression. For number of iterations 2 0. noise levels dg and a, dataset sizes To and T, and regularization parameter À 2 0, let W pred n. T.To.ag.TTo..
Rdd be the ridge predictor constructed from an iid sample (211, V1).... (IT, vr)} of size T from the n-fold fake data distribution Ps. o., where for ease of presentation of our results we will assume that
T--1 =.... T, = To, Ta T and an-1 = =... = 01 oo: on = a.
(6)
For an n-fold fake data generator Pe,o,,z, we denote with = X, E Txx the design matrix
with iid rows from NO0, E), with E : En E RT the stage-n label-noise vector with components in N (O, oï), and Y Y, X F. R.. the labels generated by Pε, Let = XTX/T X/T
pd is the sample covariance matrix, and R = R(A) = ( + Ala) denote its resolvent, so that upred RX Y/T for X 0, and uired X xty for À 0.

---

We are interested in the dynamics of the test error Etest ( (Wrred) of this linear model. Importantly. the evaluation of the model is performed on the true data distribution ox even though the model is trained on the fake data distribution pr. Note that for n = 0. Exeer := Etest (upred) corresponds to the usual test error when the downstream model is trained on clean data. Importantly, the downstream model has no control over this process. It will only see training data from a given version P but evaluation will be on the true distribution Px.wp.08
The mental picture is follows: each generation wn can be seen as proxy for specific version of ChatGPT, for example. The sample size To used to create the fake labelling functions Wn is proxy for the strength of the fake data-generator thus constructed. Other works which have considered model collapse under such self-looping training process include 44
18
4 Exact Test Error Characterization
In this section we establish generic analytic formulae for the test error of the downstream model wpreed (7) trained on n-fold fake data-generation as outlined in Section 3 The fully general technical key Theorem F. detailing formula (I) with trace expression for p. (as well as proofs) are given in Appendix F consult part [. for an exposition. Notations are standard (summarized in Appendix E
4.1 Warm-up: Ordinary Least Squares on Isotropic Data
For a start, let us first consider the case of unregularized regression, where X = 0 in Equation 7 Theorem 4.1. For n-fold fake data generation process with To 2 d + 2 samples, the test error for the linear predictor Whred in Equation (7) learned on T 2 d (d+2 2 samples, with = 0, is given by
σ²6
nonou
d
Ereet(W (predd (w
1 + 1- 00
with Φ =
Φ To
(8)
T
where the notation f(T) ~ (()) means f(T)/9(T) + 1, for large T.
The first term Frest
(1) (ograd) = 0² o - の) in the above decomposition corresponds to the usual error when the downstream model is fitted on clean data (see (22], for example). The additional term nog (1 Φ proportional to the number of generations n, is responsible for model collapse.
Model collapse versus more training data. Note that the linear degeneration in test error highlighted by Equation (8) is a direct consequence of using the same dataset size To across the fake data generator. Of course, if the underlying synthetic generating process has access to a larger data budget across generations, this decay can be significantly alleviated. For instance, if fake data increases gradually with the number of generations m 2 2 as Tm = (m log" m)To (and, to simplify, a = σo) a trivial extension of Theorem 4.1 yields
Frest(@pred
+1+ 2log22
+ Exest (aees(@@rre) (mpred) ~ Erest (W
3 3log²3
which will keep collapse at bay at the expense of largely increased training data (144) also has similar formula). This does not avoid model collapse; rather, it trades additional data generation and training effort against deterioration from generations of fake data. Thus, while for clean data increasing the dataset size n-fold leads to better scaling, with synthetic data, we forfeit this improvement. Also, note that we do not assume access to samples from any of the intermediate generation steps Wo in we only train the downstream model @pred on data from the last step S..
Model Collapse Change of Scaling Laws. In the low-dimensional regime (fixed d), Theorem 4.1 already predicts a change of scaling law from - to o + noooTo Thus, the sample size T' is scaled up, the test error eventually plateaus at the value no and does not vanish. This phenomenon, also established in [18] in the context of large language models, is clearly visible in Figure Ia] In the rest of this section and also in Section 3 we shall establish an analogous picture for high-dimensionss regimes (d + 20).
Mitigation via Regularization. Note that the test error of the null predictor Woall 0 is Frest (wait) = || and so
Frest Crest(Wprree (ippred)
n
d
Exest ( (Wmult)
SNR 1
SNR, 1 - 00
5

---

where SNR := | and SNR, := -1 We deduce that if n > /N - 1). then the learned model is already much worse than the null predictor! This suggests that a possible strategy for mitigating the negative effects on learning on Al-generated data is regularization, as empirically illustrated in Figures Ta
Tb 2
and also in of Appendix [
Furthermore, in Section [ we shall establish that the optimal regularization parameter established in [1], in the case of polynomially decreasing spectra (a regime which is relevant to wide neural networks), must be modified in the presence of synthetic training data in order to prevent the generalization error to diverge to infinity (i.e catastrophic failure).
4.2 Bigh--Dimensioall Regimes
In order to analyze the trace term p appearing in Equation T (and spelled out (32) in Appendix F.T
need some tools from RMT, and ultimately obtain analytic formulae for Etest (mpred) in Theorem
Such tools have been used extensively to analyze anisotropic ridge regression 41 22
Random Matrix Equivalents. For any sample size T 2 1 and À 2 0, define k(>, T) implicitly by
<(A, T)
s(A, T) df, (m(A, T))/T,
where, for any À 2 0 and m E N.. df (A) is the mth order "degree of freedom" of the covariance matrix Σ is given by df (A) df (A; ) == tr E (E ( Ala)
The effect of ridge regularization level A 2 is improve the condition of the empirical covariance matrix E; what the x-function does is translate this into regularization on Σ at level <(A, T), so as control the capacity of the former, i.e. the "effective dimension" of the underlying problem. Quantitatively, there is equivalence of the form df, (A; Σ) R df (m(A, T); E). Roughly speaking, RMT is the business of formalizing such a relationship and derivatives (w.r.t X) thereof. standard reference on the subject is (5)
Example: Isotropic Data. As an illustration, note that df, (A) = d/(1 + A) (polynomial decay) in the isotropic case where Σ = Ia- Consequently, we have
Ø x<(,, T) r)), with Φ d/T. In this case, is easy to obtain the following well-known formula for x = k(>, T):
<(,, T)
(A + 2 with := 1, A Δ
(10)
2
which is reminiscent of the celebrated Marchenko-Pasee Pastur law
Asymptotic Regime. We shall work in the following so-called proportionate asymptotic scaling regime which is standard analysis based on random matrix theory (RMT):
T,d co, d/T + Φb || || 1lop 0(1).
(11)
Later in Section 5 when we consider power-law spectra, this scaling will be extended to account for the more realistic case where and T are of the same order on log scale, i.e
T,d oc, //c STSC, 1Bllop
"1lopp11>" Ilop O()).
(12)
for some absolute constant C > 1. Such non- crn-proportionate settings are covered by the theory developed in 427 For clarity of presentation, even in this more general regime of Equations (22 we will still continue write Φu := d/ To and Φ == d/T.
Bias- Variance Decomposition. With everything now in place, let us recall for later use, the classical bias-variance decomposition for ridge regression (for example, see [41] 22
Proposition 4.2. In the RMT limit (12). the test error of a ridge predictor w(A) based on T iid samples from the true data distribution Ps is given by
Frest (w)) E (|(e(A) wol 13.133,-0 Bias + Var, x 21 x
(13)
Var 0² d2
with Bias
(14)
df2(m)/T
T
1- dd2((K))
where K K(A, T) is given in Equation

---

4.3 Analytic Formula for Test Error
The following result gives the test error for the downstream ridge predictor Wored defined in Equation (7) in the context of fake training data, and will be heavily exploited later to obtain precise estimates different regimes. Define generic Var and Bias by:
Var 3 R E E/TIB = o² tr E
Bias = E [| - ww E
and note that Eelenn == Bias + Var, for standard ridge regression fitted on clean data from the true data distribution o² (e.g., the orthogonal projection unto the subspace of E spanned by the rows of Xm and define
see Hastie al. (22]). Let Qn- Pn ... Po where m is
ABias == E E wo)ll 2
(15)
Theorem 4.3. For an n-fold sake data- cola-generation process, the test error of a ridge predictor wpred based on sample of size T with regularization parameter X, is given in the RMT limil 12) by
Freat (orrdd) Bias + Var + no
(16)
where p is as given in Theorem R.1 and Bias satisfies
Bias 2 Bias + ABias 2 Bias (with equality To 2 d),
and A Bias as given in 15
Furthermore, one of the following conditions holds
To 2 d OR Xn, = Xo for all n 2 1,
(17)
then, we have the following explicit formula for P
tr (E + kod) (X kI)
x22 Σ2 ( + NO - +[+++T)++
df2(k) T - df22(xx)
+
(18)
To (
Tu - du-df2 (Ko)
with K x<(,, 1") and Ki == x(0, To) are as given in Equation
Instructively, the term ABias measures how biased the synthetic data- data- generation process away from ground-truth model wo. This term disappears if the generator was fitted on sufficiently many samples (i.e. if To 2 d). More quantitatively, when To < and X, = Xo. it is easy to see that Biasu, where Bias, == E || Pawy wull2 measures the inability due
Bías 2 E [|| [| to lack of enough data, of the first generation (n 1) to reliably estimate wy even in the absence of noise (on = 0) in the data-generating process. This gap propagates over to higher generations of the process. The situation is illustrated in Figure [ In the case where To < d and the X,, 's independent, we shall see Section 4.5 that this increase in bias actually grows with n, even in the case of fake data generation without label noise (i.e. on- 0).
4.4 Model Collapse in the Case of
Fake Data-Generator
We now consider the scenario of under- parameterization, where To 2 d, indicating that the number of data points exceeds the number of dimensions. This condition typically results in unique solution for the regression. this case, Po
La a.s., leading Bias Bias (given in formula I4 ), and
Ko = 0 in (18), and so Theorem 4.3 gives
df dk2(x)
x² tr (E + kl) df2(x) T - (f)(m))
p To
To d
(19)
We have the following corollary to Theorem
4.3
Corollary 4.4. Consider the setting of Theorems
4.3 and F. I 2 d additionally, then it holds in
the RMT limit (2) that Frest fiest((ppa (mpped)) = Bias + ar + no60, where Bias and ar are as given in
formula 14 and p is as given in Equation (19).
Moreover, in the special case of features, it holds that
x²2 0221wol13 113 z20
do
1
1
Ok?
Bias + Var
(1+
p =
2o
k:)2 k1+
(1
K)2 (1 x22 1
with Φ = d/T, 6n = d/To. and x(A as in Equation
(TPp.
Note that Theorem 4.1 is special case of the above result corresponding to À = 0 and Φ 2 1. A result like Corollary 4.4 gives us the needed analytical handle for understanding n-fold model collapse in terms of all problem pyrametars (covariance spectrum, regularization, label-noise level, etc.).
7

---

4.5 Model Collapse in the Absence of Label Noise
We now consider the over- weerrparametrizaee regime, where the different iterations of the synthetic data- generator (refer to the illustration in Figure 3 are fitted on insufficient data. For simplicity of exposition, we restrict our presentation to isotropic covariance Σ Ia- Since we will be focusing on the possible increase A Bias above the bias (defined in Equation (14)) due to n. 2 1 generations as predicted by Theorem 4.3 we further restrict ourselves to the noiseless regime where the fake data-generating process has no label noise, i.e. do = 0. Thanks to Lemma F.4 we know that the generation-n fake labelling vector wn (defined in Eqn. (5) is given explicitly as a series of projections
V.p nn Pa-1PPP- Paui.
(20)
Further, for simplicity will assume T Tn d, i.e the downstream model has access to enough data. We shall focus on two important special cases.
The Dependent Case. We first consider the case where Tm To < d and Xm Xo for all m S n 1. It is clear that Equation (20) reduces to Wn = Powo, with rank Po =o < d.
Theorem 4.5, In the limit x 0* and d. To - oo with d/To + @u > 1, holds that
| 11 = ||00012 A ABias = || (1 - 1/00)-
(21)
We see that in this setting, the increase in bias A Bias ~ data is a positive constant which does not grow with the number of generations n 2 1. This increase in bias (i.e compared training on clean data) is due the fact that, with probability 1, the random subspace of Rd spanned by Xo does not contain the ground truth model wp- The expression is nothing but RMT estimate of Powt wall 212 i.e. the squared norm of the projection of wo onto the orthogonal complement of this subspace. The result is illustrated in Figure 2aa..
|| brought about by synthetic
The Independent Case. For our second example, we remove the assumption that T'm = To and Xm = X, for all m S n 1 considered in the previous case (Theorem 4.5). We instead assume that (A) The Xm are assumed to be independent, and (B) are in the following high-dimensional limit
A + 0+,
Tn-1 + 00, d/Tm -> 6m, for some $1,..., Pn-1 > O.
(22)
Define 1) = Imm= min(1 (1/mm:1mmmmmmm 1) E (0, 1). We have the following theorem.
Theorem 4.6. In the limit (22). it holds that [| 11? = || 1 and A Bias = 1 11) In particular, if n oo with infinitely many &m. > 1, then Wn and ABas + [wol
The theorem predicts that a sequence of ever-parametrizze zero (and thus, effectively escapes from the ground truth model wg).. Consequently, the downstream model @ered convergences to a Gaussian process around zero, instead of the true model wo, leading to an increase in the bias term of the test error!
fake data-generators (ūæ)n collapses to
For example if dh = on > 1, then Theorem 4.6 predicts that A Bias = (1 A "|l 12, which grows exponentially fast towards ||w 1| the test error of the null predictor: This compounding effect is due to the fact that in (20), each projection P.. spins the fake data labelling vector Wn further away from the ground-truie truth wo. The result is illustrated in Figure 2 b).
Comparing the dependent case and the independent case, 4.6 shows that the increase in bias is proportional to 1 - 1o7 ... 7/-11, which is typically much larger than 1 Too which is the increase in the dependent case. Sampling different design matrices results in more pronounced model collapse.
5 The Case of Heavy Tails (Power Law)
Neural scaling laws 266 relate a model's test error to the sample size, model size, and computa- tional resources, and are critical tools for practitioners in strategically allocating resources during the design and implementation of large language models. Previous theoretical works [1] 41 [4] have examined scaling laws in our tractable setting of linear regression with Gaussian design in the context of a power-law covariance spectrum. Now we explore how synthetic data alters these scaling laws in this setting.
8

---

Let the spectral decomposition of the covariance matrix E be E = A +... + Paaapy where A1 2 2 Ad 2 0 are the eigenvalues and ....... da E RR are the cigenvectors. For any feature index j E [d], define coefficient çj == wi 0y. i.e the projection of wh along the jth eigenvector of E. We shall work under the following well studied spectral conditions
(Capacity Condition) j for all j E [d],
(23)
(Source Condition) || = O(1,
where 3 1 and r > 0. The parameter r measures the amount of dispersion of wo relative to the spectrum of E; large value of r means wy is concentrated only along few important eigen-directions (i.e. the learning problem is easy). For later convenience, define 6, I, and c by
8 1 1+ 3(2r 1) E I,, r :: min(r, 1) E (0, 1), c: 228/(28r + 1) E (0,1).
(24)
As noted in [4], the source condition in 23 is satisfied if og = j j- for all j E [d]. Consider adaptive ridge regularization strength of the form
xT
(25)
for fixed é 2 0. The case where / = 0 corresponds to non-adaptive regularization; otherwise, the level of regularization decays polynomially with the sample size T. Define
Lerst 3/(2Br 1).
(26)
In [1], KRR under normal circumstances (corresponding to n = 0, i.e no fake data) was considered and was shown that this value for the regularization exponent (25) minimax minimax-optimal for normal test error in the noisy regime (a > 0), namely Eteat (ig (Gggedd T This represents a crossover from the noiseless regime where it was shown that the test error scales like Erest ( (wpred) x T-22r, a much faster rate. In the context of training on fake data, which is the object of this manuscript, we shall establish new scaling laws which paint a drastically different picture.
A "Collapsed" Scaling Law. The following result shows that model collapse is modification of usual scaling laws induced by fake data. All proofs of this section can be found in Appendix HH Here, for simplicity of presentation, we restrict to the case To 2 d 2 to make the results easier to present. This condition can be removed as in Theorem
Theorem 5.1. Consider n-fold fake-data generation with sample size To 2 d + 2. For a ridge predictor Wpredd given in Equation 77) based a fake data sample of size T, with regularization parameter X = X(T) tuned adaptively as in Equation
(25) with exponent [0, B), the test error
satisfies the following scaling law in the RMT limit
Erest Fwest@@rree (Wgred) max(o² T b) T (1-5)
no
max (T/To, Φ0) T (1-1))
(27)
do
We now provide an instructive interpretation of Theorem 5.1 and outline the effect of regularization.
The Noiseless Regime. First consider the case = 0 (or equivalently, exponentially small in 7`) and do E (0, fixed, and consider number of generations n such that not x T where 0 < - f/B 1. Note that a = O corresponds to a constant number of generations. Also take To = Tb, for some constant E (0, o.). According to Theorem 5.1 if we want to balance out the model-collapsing negative effect of training on fake data, we should chose l so to balance the second term n(T = T (b-é/8 and the first term T - We have the following:
Corollary 5.2. In the setting of Theorem 5.1 with To x T and n Tb, the optimal exponent of the ridge regularization parameter in Equation 25 is l = E., where
l, = min(mi eersss B),
(28)
and lorit is as in Eqn. 26 with corresponding optimal test error infr2o Erest (Wpped) ) x - (b-a)e
Observe that when (b a)e 28r, which the case when 0(1), r 2 1 and < a + this corresponds to the condition T Z To- The above result represents a crossover from the fast rate Erest Stesr(@pred) (upred) x 25 in the case of training on clean data [4. to a much slower rate

---

Etest (wpred) x (b-aa) attained by the adaptive regularization À x T which is optimal in this setting. Furthermore, in this setting if we still use X x T-leree as proposed in 14 in the clean
data setting, Corollary 5.2 predicts that
Exest (wpred)
T (b-lon//-a))) TT (cttbba--1)
which diverges to infinity if a + c. empirically illustrated in Figures Tb anddd
This is a catastrophic form of model collapse, and is
The Noisy Regime. This discussion can be found in Appendix
Remark. In all the analyses above, we quantitatively demonstrate how model collapse manifests as change in scaling laws within setting commonly used to understand scaling behavior in current foundation models 46 3
Our results indicate that, in the presence of synthetic data, scaling laws with respect dataset size slow down (i.e., exhibit smaller exponents), meaning much larger sample size is needed to achieve the same reduction in test eπrоr as with real data. Furthermore, the optimal scaling law with synthetic data requires different regularization; the optimal settings for real data could lead to catastrophic model collapse. Related findings are reported in Dohmatob et al. [ in the setting of discrete data for infinite memory models and their variants.
Experiments
To further support our theoretical findings, we conduct experiments using kernel ridge regression on the MNIST dataset [16], as detailed in Appendix [D.2] Our experiments validate the theoretical pre- dictions for both RBF and Polynomial kernels, demonstrating the parallels between linear regression and kernel regression, and highlighting the relevance of our theory to more complex settings.
We also explore the behavior of real neural networks by training two-layer networks in two different settings: fixing the first layer training both layers (see Appendix D.3). Consistent with our theoretical insights, we observe a linear pattern of model collapse when the first layer is fixed. However, more severe, nearly quadratic model collapse observed when both layers are trained, with our theory providing a lower bound for this behavior. These results reinforce the ability of our theory to capture the dynamics of model collapse across varying complexities. Full experimental details and results are provided in Appendix D.
7 Concluding Remarks
As we navigate the "synthetic data age", our findings signal a departure from traditional test error rates (e.g. neural scaling laws), introducing novel challenges and phenomena with the integration of synthetic data from preceding Al models into training sets. Our work provides a solid analytical handle for demystifying the model collapse phenomenon modification of usual scaling laws caused by fake synthesized training data.
On the practical side, our analysis reveals that Al-generated data alters the optimal regularization for downstream models and changes the scaling laws. Drawing from the insight that regularization
mirrors early stopping [3]. our study suggests that models trained on mixed real and Al Al-generated data may initially improve but later decline in performance (model collapse), necessitating early detection of this inflection point. To preserve model quality when scaling laws are altered, it is essential to employ data filtering and watermarking techniques to distinguish real data from synthetic content. Recent studies have also explored methods for data selection [19 and correction [20]. These observations prompt a re-evaluation of current training approaches and underscores the complexity of model optimization in the era of synthetic data.
Acknowledgments
YF and JK are supported by the National Science Foundation under NSF Award 1922658, Part of this work was done while JK and YF were hosted by the Centre Sciences de Données at the École Normale Supérieure (ENS) in 2023/24, whose hospitality they gratefully acknowledge. This work was partially supported through the NYU High Performance Computing (HPC) resources, services, and staff expertise.
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidr, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0877/ 2023.
[2] Sina Alemohammad, Josue Casco Cadro-Rodr,guez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi, and Richard G. Baraniuk. Self-consuming generative models go mad. arXiv preprint arxiv:2307.01850 2023.
[3] Alnur Ali, J. Zico Kolter, and Ryan J. Tibshirani. A continuous-iing view of early stopping for least squares regression. In Kamalika Chaudhuri and Masashi Sugiyama, editors, Proceedings of the Swenty-Secont International Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pages 3378-338. PMLR, 16-18 Apr 2019.
[4] Francis Bach. figh-dimensional analysis of double descent for linear regression with random projections. 2023.
[5] Zhidong. Bai and Jack W. (Jack William) Silverstein. Spectral analysis of large dimensional random matrices. Springer series statisties. Springer, New York ;, 2nd ed. edition, 2010. ISBN 9781441906601.
[6] Raphaël Berthier, Francis R. Bach, and Pierre Gaillard. Tight nonparametric convergence rates for stochastic gradient descent under the noiseless linear model. CoRR, abs/2006.08212, 2020. URL http::ppssaaxi..o/////////////// 200/ 200.200000000222
[7] Quentin Bertrand, Avishek Joey Bose, Alexandre Duplessis, Marco Jiralerspong, and Gauthier Gidel. On the stability of iterative retraining of generative models on their own data. arXiv preprint 13xiv:2310.00429, 2023.
[8] Matyas Bohacck and Hany Farid. Nepotistically trained generative-ai models collapse, 2023.
[9] Martin Briesch, Dominik Sobania, and Franz Rothlauf. Large language models suffer from their own output: An analysis of the self-consuming training loop, 2023.
[10] Tom Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Perbert-Voss.. Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc., 2020.
[11] Andrea Caponnetto and Ernesto de Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7:331 1-38,, 2007.
[12] Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training in differentiable program- ming. Advances in neural information processing systems, 32, 2019.
[13] Hugo Cui, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborova. Generalization error rates in kernel regression: The crossover from the noiseless to noisy regime. In A. Beygelzimer, Dauphin, Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[14] Hugo Cui, Bruno in kernel regression: the crossover from the noiseless to noisy regime. Journal of Statistical Mechanies:
Loureiro, Florent Krzakala, and Lenka Zdeborová. Generalization error rates
Theory and Experiment, 2022(1 11:2111:114411 14404,, nov 2022.
[15] Hugo Cui, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborová. Error scaling laws for kernel classification under source and capacity conditions. Muchine Learning: Science and Technology, 4(3):035033, August 2023. ISSN 2632-2153.

---

[16] Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142. 2012.
[17] Jacob Devlin, Ming-
Min Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1xXiv:8110004055 2018.
[18] Elvis Dohmatob, Yunzhen Feng, Pu Yang, François Charton, and Julia Kempe. tale of tails: Model collapse as change of scaling laws. In Forty-first International Conference on Machine Learning, 2024. URL caattps://pperreee wwwp:/ewwwwww...eeeeeeeeeeeeeeee www.//eee/foniimee wow.rrrmmnmwiwwwkuuua
[19] Yunzhen Feng, Elvis Dohmatob, Pu Yang. Francois Charton, and Julia Kempe. Beyond model collapse: Scaling up with synthesized data requires reinforcement. arXiv preprint arXiv:2406.075155 2024.
[20] Nate Gillman, Michael Freeman, Daksh Aggarwal. HSU Chia-Hong, Calvin Luo. Yonglong Tian, and Chen Sun. Self-correcting self-consuming loops for generative model training. In Forty-first International Conference on Machine Learning, 2024.
[21] Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. The curious decline of linguistic diversity: Training language models on synthetic text, 2023.
[22] Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J. Tibshirani. Surprises in high- dimensional ridgeless least squares interpolation. The Annals of Statistics, 50(2), 2022.
[23] Ryuichiro Hataya, Han Bao, and Hiromi Arai. Will large-scale generative models corrupt future datasets? In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 20555-20563 October 2023.
[24] Jordan Hoffmann. Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. arXiv preprint 12355:2203.15556, 2022.
[25] Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Prucessing Systems, volume 31. Curran Associates, Inc., 2018
[26] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001. 08361, 2020.
[27] Antti Knowles and Jun Yin. Anisotropic local laws for random matrices. Probability Theory and Related Fields, (69)53237-335, 2017.
[28] Jaehoon Lee, Yasaman Bahri, Roman Novak, Samuel S. Schoenholz, Jeffrey Pennington, and Jascha Sohl-Dickstein. Deep neural networks as gaussian processes. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May
3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
[29] Tengyuan Liang and Alexander Rakhlin. Just interpolate: Kernel "Ridgeless" regression can generalize. The Annals of Statistics, 48(3), 2020.
[30] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint 1rXiv:190..11662 2019.
[31] Alexander Maloney, Daniel A. Roberts, and James Sully. A solvable model of neural scaling laws, 2022.
12

---

[32] Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Mare Juarez, and Rik Sarkar. Combining generative artificial intelligence (ai) and the internet: Heading towards evolution or degradation? arXiv preprini arxiv: 2303.01255, 2023.
[33] Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Heráandez, Mare Juarez, and Rik Sarkar. Towards understanding the interplay of generative artificial intelligence and the internet. arXiv preprint arxiv: 2306.06/30. 2023.
[34] V.A. Marčenko and Leonid Pastur. Distribution of eigenvalues for some sets of random matrices.
Math USSR Sb, 1:457-483, 01 1967.
[35]
Midjoumny.. Midjourney ai, 2023. URL http::://www.mdfooonnneeee com/
[36] Hossein Mobahi, Mehrdad Farajtabar, and Peter Bartlett. Self-distillation amplifies regulariza- tion in Hilbert space. In Advances in Neural Information Processing Systems, volume 33, pages 3351-3361. Curran Associates, Inc., 2020.
[37] Radford M. Neal. Priors for infinite networks. In Bayesian Learning for Neural Networks, pages 29-53. Springer, New York, 1996.
[38] Loucas Pilvied-Vivien, Alessandro Rudi, and Francis R. Bach. Statistical optimality of stochas- tic gradient descent on hard learning problems through multiple passes. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi and Roman Garnett, editors, Advances in Neural Information Processing Systems 31; Annual Conference on Neural Information Processing Systems 2018, pages 8125-8135, 2018.
[39] Ali Rahimi and Benjamin Recht. Weighted sums random kitchen sinks: Replacing minimiza- tion with randomization in learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2008
[40] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Iya Sutskever. Zero-shot text-to-image generation. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8821-8831 PMLR, 18-24 Jul 2021.
[41] Dominic Richards, Jaouad Mourtada, and Lorenzo Rosasco. Asymptotics of ridge(less) regres- sion under general source condition. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research. PMLR, 2021.
[42] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695 June 2022.
[43] Alessandro Rudi and Lorenzo Rosasco. Generalization properties of learning with random features. In Advances in Neural Information Processing Systems. Curran Associates Inc., 2017. ISBN 9781510860964
[44] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Ander- son. The eurse of recursion: Training generated data makes models forget. arXiv preprint arxiv:2305. /7493, 2023.
[45] James B. Simon, Madeline Dickens, and Michael Robert DeWeese. Neural tangent kernel eigenvalues accurately predict generalization. 2021.
[46] Stefano Spigler, Mario Geiger, and Matthieu Wyart. Asymptotic learning curves of kernel methods: empirical data versus sacher-student paradigm. Journal of Statistical Mechanics: Theory and Experiment, 2020(12), 2020.
[47] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
13

---

[48] Alexander Wei, Wei Hu, and Jacob Steinhardt. More than toy: Random matrix models predict how real-world neural representations generalize. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research. PMIR, 2022.
[49] Christopher Williams. Computing with infinite networks. In M.C. Mozer, M. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9. MIT Press, 1996.
14

---

Appendix/ / Supplementary Material for Model Collapse Demystified: The Case of Regression
Contents
Differences to Self-Distillation
15
B Related work on Kernel Ridge Regression with Gaussian Design
16
Extension to Kernel Methods
16
Details of Experiments
17
DJ Simulated Data
17
D.2 Real Data: Kernel Ridge Regression on MNIST
17
D.3 Neural Networks on MNIST
17
Notations
19
Exact Characterization of Test Error Under Model Collapse
19
A General Formula for Test Error
19
Proof of Theorem 4.
Rigeless Regression)
20
Proof of Theorem
Ridge Regression + General Covariance)
21
Proof of Theorem 4.3
22
H5
Proof of Corollary 4.4
23
F6
A Note on Proposition
24
Proof of Theorem 4.5 and Theorem (Model Collapse in the Absence of Label Noise)
24
The Noisy Regime for Power Law Spectra
26
H
Proof of Results for Power-Law Covariance Spectrum
26
Proof of Theorem|5.1
26
Representation of Clean Test Error
26
Auxiliary Lemmas
27
Differences to Self-Distillation
important point we wish make that the fake data generation process that we analyse should not be confused with self-distillation formulated in Mobahi et al. [36] for example. Our setting
15

---

is inspired by the model collapse phenomenon, where increasingly vast amounts of synthetic data generated by users is posted online, and will necessarily enter the training set of the next foundation model. In this case, we do not have ground truth labels, nor is the generation of synthetic data not controlled by us, but by other users. Therefore, we adopt the setting of solely synthetic labels with added noise.
Specifically, in our setup, at generation > O, we do not have access the true labels Yo fo( + noise for the training samples X. but rather to some Y, (X) + noise, where fr is an unknown function, which synthesizes fake labels iteratively; the integer n is the number of iterations. In our work, we make the structural assumption that fn is obtained by iterative / successive regressions on a true dataset Do = (Xo, Yo). We do not have any control over the creation of these labels, which is reflected by the noise injected at each stage.
In the self- eeff-distillation setting, the data generation process actually helps performance of the down- stream model. The model has access to training labels from the true data distribution Y, but decides to fit a model on this data, and then use its outputs as the new labels Y, := (X, Y), iterating this process possibly over severable steps. Thus, self-distillation has control over the data generating pro- cess, which is carefully optimized for the next stage training. Specifically, [36] study self-distillation in the same Gaussian regression model underlying our analysis, but in each distillation generation are able to tune the regularization parameter for downstream performance as a function of the original data labels (with the data being the same at each generation). In the setting of model collapse, there is no control over the data generation process, since constitutes synthesized data which typically comes from the wide web.
Self-distillatio for linear regression would amount to very special instance of our analysis where (1) Xo = X1 =... = Xn-1 = Xx, = X and (2) og = 0n = That is, there is exactly one design matrix which is used in the data generation process and in the downstream estimator, and also no additional source of label noise is present at the end of each generation.
In the general setup considered in our work, (1) is not imposed. We typically assume that Xo, XI,..,,,,............. X,.,, Xx, with Xn, = X, are all independent random matrices. An exception is line 247 ("The Dependent Case") of Section 3.5, where we assume Xm.. Xo for all S n 1, and independent of X, X. That setup (considered for the purposes of showing that model collapse can still occur in the absence of label noise) also assumes om = 0 for all m;; the analytic picture which emerges (Theorem 3.5) is already drastically different from what one would get from self ceff-distlattion (corresponding to additional assumption that X Xp).
B Related work on Kernel Ridge Regression with Gaussian Design
This model has been studied by a vast body of works. For example, Richards et al. [41, Hastie et al. 22) Bach [ analyze the classical bias-variance decomposition of the test error for ridge regression in the high dimensional setting where dataset size and dimension diverge proportionately. using tools from Random Matrix Theory (RMT). In Section 4 we significantly extend this type of analysis to training on iteratively generated synthetic data. This model is also particularly attractive because allows to analyze important trade-off: the relative decay of the eigenvalues of the kernel (capacity) and the coefficients of the target function in feature space (source). Sizeable effort has been dedicated to characterize the influence
the decay rate of the test error as a function of these two relative
decays (aka power laws) [1 38
46 [4] 15]. In Section 5 we extend these efforts, in particular which has given full characterization of all regimes and test error decay that can be observed at the interplay of noise and regularization, characterizing a crossover transition of rates in the noisy setting. Our work uncovers fascinating new effects as a result of iterative training on synthetic data.
6 41
based on works of Cui et al. 13 [4]
C Extension to Kernel Methods
Though we present our results in the case of linear regression in Rd for clarity, they can be rewritten
in equivalent form in the kernel setting. Indeed, as in [TI 45 14 29), suffices to replace x with a feature map induced by a kernel K, namely <b(x)= == K2 E Hx- Here, Hx is the reproducing kernel Hilbert space (RKHS) induced by K. In the data distribution (2). we must now replace the
Gaussian marginal distribution condition ~ (O, E) with (xx) (0, E). The ground-truth
16

---

labeling linear function in 2 is now just a general function fo E L2. The predictor 7 is then given by (Representer Theorem) f (xred()) K x) with Tn (G + XT E R", where x(x,2)= == ()), ((rr,))) r)), and G = K(X, E R is the Gram matrix.
D Details of Experiments
We perform the following experiments both simulated and real data to empirically validate our theoretical results.
D.1 Simulated Data
We consider ordinary / linear ridge regression in R. for d = 300 and different structures for the covariance matrix Σ of the inputs: isotropic (i.e Ta) and power-law (23). with (3,r) r) = (2, 0.375). For each value of n (the generation index), the fake data-generator is constructed according to the process described in (4). Then, for different values of (between and 1000, 000), a sample size T is drawn from this fake data-generator and then a downstream ridge model 7 is fitted. The test set consists of 100, 000 clean pairs (x, y) form the true data distribution ox This experiment is repeated 10 times to generate error bars. The results for the isotropic setting are shown in Figure la and the results for the power-law setting are shown Figure Tb Figure shows the over-parametrized setting.
D.2 Real Data: Kernel Ridge Regression on MNIST
As in Cui et al. [4], Wei et al. 48 we consider a distribution on MNIST [6]. a popular dataset in the ML community. The classification dataset contains 60, 000 training and 10, 000 test data points (handwritten), with labels from 0 to 9 inclusive. Like in Cui al. [4], we convert the labels into real numbers (i.e regression problem) as follows: y = label mod 2 + noise where the variance of the noise is o² = (for simplicity, we also set oe = 1). The test set consists of 10, 000 pairs (zr,y). with the labels y constructed as described in the previous sentence. The fake data used for training generated as the previous experiment, but via kernel ridge regression (instead of least squares) with the RBF kernel (bandwidth = 10-1) and the polynomial kernel (degree = 5, bandwidth =10-3). Note that it was empirically shown in Cui et al. [14] that these datasets verify (23) with (B,r) & (1.65, 0.097) in the case of the aforementioned RBF kernel, and (8, (1.2, 0.15) in the case of the polynomial kernel. Then, for different values of I (between and 1000). a sample of size T's is drawn from this fake data-generator and then a downstream kernel ridge model is fitted. Each of these experiments are repeated 10 times to generate error bars (due to different realizations of label noise). The results are shown in Figure
D.3 Neural Networks on MNIST
We now further examine model collapse in two-layer neural networks on the MNIST dataset, beyond the linear setting and Gaussian data. We consider two scenarios:
learning with a random features (RF) model, where the first layer is fixed randomly, and only the second layer is trained, and
learning with a fully trainable neural network.
For the two-layer network with the first layer fixed, our theory predicts a linear increase in test error as a function of the number of iterations n. This is because such models belong to the lincarized regimes finite-width random feature models and can be approximated by kernel regression 225 45) For fully-trained neural networks, our theory does not directly apply. However, we anticipate that the general trends uncovered by our asymptotic theory will hold true- for example, more parameters are expected to lead to greater model collapse, as shown in Theorem 4.1
Specifically, the models were trained using stochastic gradient descent (SGD) with a batch size of 128 and learning rate of 0. We employed regression setting where labels were converted to one-hot vectors, and the model was trained using mean squared ervor for 200 epochs to convergence. When generating the synthetic data, Gaussian label noise with a standard deviation of 0.1 added. The test error is consistently evaluated on the test set using clean labels.
17

---

À =
x
2x10
Real data
Gen. d2 Gen. #3
Test error
10a
10a
6 6x
6 10
4x
10
10a
10a
10a
10a
10>
Sample size
Sample size
(a) RBF kernel (bandwidth 10-")
= = far
A = A=T1.
10"
109
6x
Real data
Test
Gen. #11
x 10-
Gen. #2
4 10-1
Gen. 43
10a
102
10a
10a
102
10a
Sample size
Somple size 7
(b) Polynomial kernel (degree = 5, bandwidth = 10
Figure Model collapse in kernel ridge regression (power-law covariance spectrum) MNIST. Here, we use adaptive regularization T for different values of the exponent e 2 0 (see Section D for full experimental setup). Top row: RBF kernel. Bottom row: polynomial kernel. In each plot, we show test eтrоr curves as a function of sample size T, from different generations (n) of fake data. The broken vertical line corresponds data distribution) which was used to train the label faker. The value of the exponent regularization f l, (broken curves) is the optimal value the presence of iterative data relabeling, while ferit (solid curves) corresponds to the optimal value without iterative re-labelling (i.e 0) proposed in Cui et [44 (see (26)). Specifically, we take = -(b- a) Eesrt = blerit, where = log To/ log T (so that To Th), as proposed Theorem 5.1 formula (28). Notice how the effect of fake data makes the test error become non decreasing in sample size T. This is effectively collapse of the learned model.
To. where To is the number of samples (from the true
The results for RF models width (i.e number of hidden dimensions) & of 20,000 are presented in Figure We observe that, with the exception of the first two generations, the decay in MSE loss generally follows linear trend, which is consistent with the predictions of our theory.
Next, we consider the scenario of training the entire neural network. By varying the width k, we adjust the number of parameters to further explore the theoretical predictions how the number of parameters influences model collapse.
Observations. From Figure 6 can observe that
More parameters (wider neural networks, i.c large k) lead to increased model collapse. This observation is consistent with our results proved the linear regime (Theorem I.1). For linear models, the number of parameters is proportional to (the input dimension), whereas in two-layer neural networks, the number of parameters of order kd (i.e proportional to the width k)).
The dependence of model collapse on the number of iterations is linear for small values of n (with n in our experiments), and becomes superlinear (possibly quadratic) for larger values of (with 2 4). Recall that O corresponds to training on clean data from the data distribution. Thus, possibly, model collapse neural networks appears to even more severe than in linear regression.
18

---

Random Feature Model
Two-layer Neural Networks
0.040
Mean
0.25
Num_samples 100
Deviation
0.10
Num_samples 200
Mean squared error
0.015
Mean squared error
Num_ Num_samples
0.25
Sum_samples
0.20
0.15
0.020
0.30
0.05
0.05
5 6
generations
Number of iterations
Figure 5: Performance RF model on MNIST, Figure The performance of two-layer neural
with one-hidden layer NN (width A: 20, 000). network on MNIST with varying hidden dimen- sions.
Standard deviation calculated using 10 seeds.
E
Notations
The set of integers from through denoted [d]. Given variable z (which can be the input
dimension d or the sample size T, etc.) the notation f(z) S I(z) means that f(z) < Cg(z) for
sufficiently large and absolute constant C, while [(z) g(z) means f(z) S g(z) f(z).
Further, f(z) ~ g(2) means f(z) (1 + o(1))g(2), where o(1) stands for quantity which tends to zero in the limit = + oc. We denote with A the Moore-Penrose pseudo-inverse any matrix A, and by ||Allop is operator norm, while the trace of square matrix A is denoted tr A. Finally, ||u||s == Vu² Eu is the Mahalanobis norm induced by a positive-definite matrix Σ.
F Exact Characterization of Test Error Under Model Collapse
F1 A General Formula for Test Error
now consider the case general ridge penalty O, and drop the requirements T d + 2 and To 2 d 2. Recall the definitions of X,Y, E and the random matrices R and Σ appearing 7 For later reference, define
Bios E [| wp 1
(29)
Ver = |RX E/T1 o² II
(30)
These respectively the bias and variance terms in the classical bias- bias-variance decomposition
Eelean = Bias + Var,
(31)
for standard ridge regression fitted on clean data from the true data distribution Ps (e.g., Hastie et al. (2)).
Theorem F.1. For an n-fold fake data generation process, the test error of a ridge predictor Gpred based on sample of size T 2 with regularization parameter X is given by
Etest (wpred) Bias + Var no.p.
Bias EE R_ ww woll
(32)
1
E tr C. In. L ERERE,
m=0
where Var is as given in 30 and Ck. = Qk Lrm for Qum Qk. vo Qk.
PaPk-I... P.. Qi == Qk PaPu-1... Po. with Pm Xt being the orthogonal pro- jection matrix onto the subspace of Rd spanned by the rows of X
19

---

In particular, To 2 d d+2 (under-paramettrize) dula-generator), then Bius Bias in 29 and
Fitest (ppred)
=
Felean "test
+ noop;
(33)
p
- EREER.
E Et
To
In the second part of the theorem, the term Er (introduced earlier in (31 corresponds to the usual test error when the downstream model is trained on real (not fake) data, for which well-known formulae exist in variety of scenarios (see Proposition
Remark F22 We show in Theorem 4.3 that Bias 2 Bias + ABias, where ABias 2 0 in the appropriate asymptotic limil, with equality if 2 d + (the under-parametrized regime). Thus, apart from the variance term, the bias term of the test error of downstream models. contrast, an under-paut padamerrreeetizzed synthetic data- generator (To 2 d d 2) only harms the variance. The increase in bias suffered in the over- parametrized regime is precisely quantified in Section and shown to be an increasing function of the number of generations
over-parametrized (To < <d + 2) synthetic data-genereen generateerrrr harms
The test enror decomposition in Theorem F.1 is thus of the promised form (T) This additional term means that there competition between the usual test error Eeleon and the additional term induced by the fake labeling process. Understanding the interaction of these two terms is key to demystifying the origins of model collapse.
Low-Dimensional Limit. Observe that if dl is fixed and T - co, then the empirical covariance matrix Σ converges to itts its population version E, and for To 2 d + 2, have
p (
(22 To d'
To - d
where for any À 2 0 and m N., df (A) is the mth order "degree of freedom" of the covariance matrix E is given by
df = dr tr (1
Note that df,,, (A) S always. In the high-dimensional setting (where can grow beyond To), the precise analysis of p will be carried out via random matrix theory (RMT).
F.2 Proof of Theorem 4.1 Rigeless Regression)
The proof is by induction the number of generations of fake data. For n have
Erest (ūgred) E Wopred wo E W wol1
= Xo) - Eall
d
0²0
(34)
- Xo) σ² T
where o= d/T E (0, 1) and the last step has made use Lemma result for the test error linear regression in the under-paramee pader-parametrized regime, without any Al pollution (fake / synthesized training data).
R3 below. This is well-known
Pe.g. weakly, w.r.t operator-norm.
20

---

Analogously, for 1 one computes the test error after the first generation of fake data as follows
Etast Bast(@@) (Tppeed)
= E Elpprree Wo 1 = E ( Wp woll
E Ellopred W1 + úi mol1
[ (1) x E1 + W we
E ||wo T6
11 R8 |1 1 X1) - x Fill2
ofd
Etest (Wgred)
+
Ti 1
od
Etest
+ To
-
0²6
0y
Φ0
where 6o d/To E (0, 1). Continuing the induction on we obtain the result. Lemma F.3. Let Xo be an To random matrix with idd rows frosfrom N(0, E). 18T 2 d + then the empirical covariance matrix Eo = =Xo Xo/To is invertible a.s and
To
E E
To d
F.3 Proof of Theorem F. (Ridge Regression + General Covariance)
F33.1 Representation of 8h and Gareed
We first obtain explicit formulae for the labelling vectors wh used in the fake-data generation process
(5) For any integer m 2 O, define Pm. = X m mm the orthogonal projection matrix onto the subspace of gd spanned by the rows of Xm. Observe from (5) that
wn X Y R-1 xt (X (Xn-10n-
En-1) - P X En-1
+ En 2) + 1 PR-1XX 2 2 2
PP Pa x En-2 X 1 Bn-1
(35)
PR-1Pn-1 ... Powo + P PIX|E P
R-1
P Powp +
Σ
P. Pa-iPa- ... Pm
We get the following result.
Lemma F.4. For any 2 0, the following formula holds
Wp:
if = 0,
@n
(36)
Q2-1w0 + >013
Qn 1m Em.
if 2
where Qk.m Qu X Qk. PyPe-1.... Pm and Qx = Qx.0 PaP Po. Moreover, wn Im Pn-1 as soon n In particular, under the simplifying condition 117 holds that
2 1.
wo:
in 0, in 2 1.
Wn
Pows + E E Im Po.
(37)
where F.n-1 == Rh-1 independent of Xo. Moreover, Wn E Im Po as soon > 1.
m- Foms a random vector of length To, with iid entries from N(0, not), and
21

---

Note that the second part of the result uses the elementary linear-algebraic fact that Pm = = In the special case where To d, we have Po I a.s., and Wn we E Otherwise, even in the absence generator noise (Øn = 0), the fake data labeller Un = Powp drifts away from the truth Wo, into subspace of R spanned by the rows of Xg.
Next, let us obtain decomposition for the downstream predictor wa defined in 7 As usual, let E = X X/T be the empirical covariance matrix with resolvent R = ( + AT)_ and observe that the downstream model writes
upreed
R. V (X)/ RX (Xw, + +E/TT
n-1 Qn nn 1,m Em + E\/T
= RX (XQ +X
(38)
m=0
RÉQQ0-10 + RX E/T + RE
Σ Qn-1,m
m==0
F.3.2
Proof of Theorem
E1
Using the decomposition
38
for the downstream model wreed, we deduce that
Fstest mared] E I wo E
E |REPow + RX DI-XXEETT RE Σ
Qn Em
woll
m=0
E |REPowo wp RX B/T RS
1 Em11
(39)
woll ElIREPowo w + E RXX E/T
+E
RE Σ
Q 1.
Em
172 m=00
Bias Var + nopp.
where = X/T, Bias, Var, and are given in the theorem. On the second line, we have used independence (of X. Xo. E, and En- and the fact that E and En- are centered Gaussian random vectors, with iid components variances o² and no respectively.
E.4 Proof of Theorem 4.3
Analysis Bias-like Term. An exact analysis the Bias term appearing in Theorems F. and 4.3 is presumably a treacherous enterprise given dependency on (via R and Σ) and X, (via Po). In place of such analysis, shall settle for the following result which gives instructive lower-bound.
Proposition F.5. In the RMT limit 112 it holds that
lim Bias lim Bias 2 lim E P - RÉwoll 2 0.
Thus, training on fake synthesized data increases the bias term of the downstream model's test error!
Proof. Letting = RÊ, one computes
Bías Bias
||Pow woll 1/2 ||Awo woll3
||Pooo Awo Aw w/} - ||Awo wo 1/3
||APowo Awo 1/ + 2w0 (A PoA)E(1 - A)wo
||APowo Awn 13 + 2wo (I - Po) A)wO
then suffices to observe that, in the RMT limit 12 it holds that
lim wo (1 Ao) A)wo 2 0, can be seen from repeated application of Propositions and of Bach 4
22

---

Analysis of Term. Define a d x random psd matrix H := ÊREER. Under the simplifying
assumption (17), the matrices 2u.m defined in the theorem all equal Qo X It follows that the p-term in 32 then writes
Qn-1m H = E ftr H1
R
( # R (xx (x() H H]. (40)
Now, one computes the conditional expectation follows
E ttr (xx)
H H] E ftr X (Xuxx)) XxHH H]
lim -0+
E tr X (Xu + - XoH H).
To aλo
Furthermore, defining : /1/2211 /2 and Zo = Xo 1/2, we have
tr X.J (Xo. (oTol)-1Xplh tr / 2J (ZoEZ) -
= Atr (ZoZZn) + -
We deduce from Proposition 2 of Bach [ that
Bir (XxoX + AoTool)- XoH H = Ir A + ,(>n; To)
= tr H + x(Ap 1) E.
Differentiating w.r.t. Ao and letting this parameter tend to zero from above gives
E [tr (1xt) H I1
1 lim
- E
X
(Xaxg
+ Xoll H)
To do +0+
aλn
1
Oms(d, To)
0
lim
tr (()
To do
Oλo
Öt
(=x(>o:26)
tr H/ + Kn -
To - df (RO)
where ко x(0,To and we have made of Lemma I.2 Combining with (40) and then applying
Proposition of Bach [ to compute Eh H(E + k)T) 25 Ex tr EREER(E + Ko - gives the following result. Proposition F.6. In the RMT limit 12 it holds for any À > 0 that
2r kol) xod)- 222 xf)
x² 22 (2 + kol) 2(2 kl)
df2(k)
+
(41)
To dl2(Ko)
To - -(2((0)
T dl2(k))
where := =(>o, To) and r(),T).
In particular, To 2 then
dk)(n)
(1+ x²2 tr(E + xx)-
(42)
T df2(k)
To df2
This result completes the proof of Theorem
F.5 Proof of Corollary 4.4
For the first part, we know from Theorem F.I that
Frast ( ppeeed
Frtss ( 10
pred
+ +no with
(43)
E tr
E AT) E X 1
(44)
To
23

---

The Etest ( ( term is taken care of by Proposition 4.2 since this corresponds to generalization
eтror on clean training data, For the p term, we use Proposition of Bach with A Σ and B = E to get
tr( ((I++RI)22
12 tr + kT)
tr tr(E + +x)=+
+
To d
To d
T- -f
dfa(k)
x² ir(2 + kl) dl2(k)
+
To d
To --d d
T -d)
which proves the first part of the result.
For the second part, note that df2(k) = d1(1 + x)² when E I, TO) holds, and so
d
(1 - 1--1/00pp- ~
(1 + 13)2
(1 k]+ T d/(1 x)²
x²
K))²
+ x3 1 6/(1+ + k)²
2k²
(1+
(1+ K)²
+ (1+ Φ"
and the result follows.
F.6 A Note on Proposition 4.2
As mentioned in the main text, the result is classical Richards al. 41. Hastie al. 12]. Bach
4). Only the second part needs a comment which we now provide. Indeed, the second part of the result follows from the first we now see. Indeed, wa E(E + kl) wwo = r² 221(1 + x)?
(x) = d/(1 + x)² and so we deduce from the first part that
1
0²d
Var = [1 + R22 - // + x²)
(1 + K)2
2 12
k²2 12 3
Bias ~ k 1wo
11+))11-
+ K
+ k)
from which the result follows.
We now need to estimate 5T H8 for deterministic psd matrix H. Observe that
T H8 = (Qx wp- wo) H(Qn- - wo)
(45)
T HQn 2w0 QTT Hwn + wg T Hwn-
w/o x-
F.7 Proof of Theorem
4.5
and Theorem 4.6 (Model Collapse in the Absence of Label Noise)
We first prove Theorem 4.6
Note that since we are in the isotropic case, ABaas defined in (15) is
now given by A Bias := E
[S((Qn- ww wn))l where Qa-1 == P P ... Po. Moreover,
since T > d and À 0 by assumption, we have ER Ia, and we further have ABias E ElQ2-1u0 ww0 - Wo I2. Now, one computes
IQn wg- woll² = ||0ol|2 - 2w0 Qn- + wp QT 15-1Qn-1w0
||ol|| - wio Qn-1w1
1-1
|1oll 2| wo
II m-=
(I + kmm)) w0
(46)
| - -[ 1
min (1/dms 1),
where on the 2nd line we have used the fact that QT Qn-1 Q because the P., 's are
projections; the 3rd line we have used Lemma E.7 with Σ = I and u Wo: on the 4th line
24

---

we have used the fact that Km == Tm) = max( 1, max( 1) 1. This completes the proof of Theorem 4.6
The proof of Theorem 4.5
is completely analogous, with Qn-1 replaced with Qn-
Lemma F.7. Let Xu..... Xn-1 be independent random matrices of shapes Tm x d for m O,..., n 1, 1, with rows iid from N(, E), and let Qn-1 := Pn-1P Po. where P.. X the orthogonal projection onto the subspace of I2 spanned by the rows of Xm. Then, in the limit d, To. ,.....TTT...........
+ oc such that d/To + Φo E oo)..... d//nn-1 - Qn-1 E (0, oo) with
|>]] op. | 1lop = (1), holds that for deterministic Le-bounded sequences of vectors and
u Qn-
II m==
( + kmm) v,
(47)
where Km = =(0, (m)) Tm) is as defined in (9)
Proof. The prof is by induction u Qov - u T Pou lim u Xo (XoXo +11) +(1)- Xoo
2 1. For 1, we have Qn- Qo = Po. Thus,
(48)
lim un ( mlt, T) -1 u/ J< - kol) -o,)- 0
where No := x00 T) and we used Proposition 2 of Bach 4 at the beginning of the 2nd line. Now, suppose the claim holds for n, and let's prove that it holds for + 1. Indeed,
R-1
u Q = u PaQn-1u T P
(m+ a T
II m=0
(E + Km) kmm) v,
m=0
where the second step is an application of the induction hypothesis with P. in place of u.
The following lemma can be used to compute
||Q n- wo
woll., in the case of anisotropic E.
Lemma F.8. Under the hypothesis of Lemma
holds that
u QQ- g"
II m==
( + Kim kiml)-1
(49)
u T Q EQn-10 E"
II Am v, m=0
(50)
2 + n² dzz dfz((mm)
(
with Am 2 Km
(51)
dl 2l
where Km ;= k(0, Tm) as defined in 9
Proof. The first formula follows directly from Lemma E7 with replaced with Eu. For the second formula, we can write
u. TTQTT MQ.- - PoP. MP PoPi
So we really only need to prove the result for n = 1; the general case will follow by induction and due to multiplicativity. Indeed, defining A = 22/2 11/2 B = 1// and Zo- = XgE-1/2 1/2, we have
where ūn-1 ==
u. PoMPou lim u X. (XoX. +11) X.MX (XoX) + Xov
++++
= lim tr AZo AZo + (-0+
(Z + +11) Zo
Ar + 1) -1B() +(o x A(E -) tr BE kol)
T - df2(Fo)
1 2 X + kgl) -2. MMEE + Ko) -2
= u (E+
T - dl2
= u EAgr for M- E,
where the 3rd line is an application of Proposition 2 of Bach (4.
25

---

G The Noisy Regime for Power Law Spectra
Here we discuss the consequences of Theorem 5.1 for the noisy regime.
Now fix o² * 0 and Φo E (0, In this regime, Theorem 5.1 predicts that consistency (i.e. Etest (eree) T 0) is only possible if I l.. First consider values of & for which the clean variance o TT (1 (1-1/8) is less than the clean bias T 2nd in (27) i.c. 0 S < ferst- We get
Eteat (Ggred) x T T-r TT
which is minimized by taking l = min(e,, forit). For other values of E, variance dominates, giving Breet ( x (1-(00) + (0-1//-))) x (--11
where y min miin -)). This is minimized by taking = Lerit, leading to
a).
Erest (uppeed) (1-10111222
This tends to zero with T oc only if a + 1/
H Proof of Results for Power-Law Covariance Spectrum
H.1 Proof of Theorem
From Theorem
F.I need to analyze the quantity
df2(k(2)) + x(A)² tr (E + k(A)la) -2
df2(m(())
(52)
P To d
To
T (-df2(m(A))
Now, for small x== s(A) is small and can compute
AmT
(f) Σ
(m 1/8) -1/8
(A: + +mm
(1+6-18) jm
where we have used Lemma 1.1 with D x_1 and n m in the last step. On the other hand, we can use some of the results Appendix (Section of 14 do the following. It can be shown (see aforementioned paper) that
If 8, then x T and so df (k) for all m 2 1.
If é < B, then x x and so df (n) T o(T) for all m 2 1.
For < B, plugging this into 52 gives
T//3
To d T IT-1///5 + Φu (1-(1))
d T
p 0
To d
do
max (T/To, Φ0) ( (1-111))
where do = d/To. Combining our Theorem F.1 with 57p.
we get the claimed result.
H.2 Representation of Clean Test Error
make a small digression to present the following curiosity: with a slight leap faith, the main results of [14] can be obtained in few lines from the tools developed in [4, namely Proposition
This is significant, because the computations in [4] were done via methods of statistical physics
(replica trick), while [4] is based on RMT.
Indeed, for regularization parameter - given in 25 we have k(A) X. Thus
x T df2(k) X x 1/8
(53)
26

---

Now, since A x xi (capacity condition) and (w)²2)2
uz)2 = ct x i- (source condition), we deduce
c
x²2 w/
+ xx) uwo = wio
Σ
(As + K-- 120 miyl
mg Σ
(A: + h x;)
(54)
X
Σ (A, + 11,32 x Σ
(Az + K 11)
where 7 (2.inn2211+))) 6/8 1/3) min (2) 2r, with r min 1). The exponent is so
because 8 = + (2r 1), and 6 1/3 2r by construction. The estimation of the last
sum in 54 is thanks to Lemma L.] applied with
D = x
8/8, and = 2. Therefore,
invoking Proposition 4.2 gives
x² 22 + 2 wo
T T
Bias
T -
(55)
df2(k)/T
(-
Var (n)
1
T
T 1 1-o(1) X x02--(1-111)
(56)
T
1 df2(x)/T
We deduce the scaling law
Erest ~ Bias + Var x T 2c (1-1/9)) x max 2, (19)) (1--
(57)
which is precisely the main result of 14
Low-Noise Regime. In the low noise regime where o² (5r-25r), one may take l 3; the variance is then much smaller than the bias, and one has the fast rate
Exest x T9-25
(58)
High-Noise Regime. Now, consider the case where x² @()) Setting 26r 1 -l/3 balance out the bias and variance gives = ferit, where
larit == 23r 1
E (0,
(59)
With this value of the exponent f, we get the error rate
2
Frest T Forr T with
(60)
2,3r
which is precisely the main result of (4,, known be minimax optimal (de Vito etc.)
I Auxiliary Lemmas
Lemma L.1. Let the sequence (A+) of positive numbers be such that A X k- for some constant O, and let m, n 2 0 with nš > I. Then, for > 1, it holds that
x
flog D.
if m m 1/3,
D
(61)
DAR
1,
else,
where = min(m, 1/3) 2 0.
Proof. First observe that
/ " X min(1, (D^+) "))
X
= t ng
if DA < 1, ieifk > D218
m \ (m- (m--)
D m (mm
else.
We deduce that
X
Dmm
D m
Σ
k(m-nij n)B +
Σ nB
(62)
1 D
I
KVD
27

---

By comparing with the corresponding integral, one can write the first sum in 62 as
D1/a
D-m
Σ I
R {m-n))
x
D
(m-n) du
(D =D-(n-1/0)
if 1/3 < m, if m 1/3. else.
D
log D,
1,
D 1/8),
if 1/3 < m,
D log D,
if 1/3, 1/3,
else.
D log D, 1,
m else,
where is given in the lemma. Analogously, one can write the second sum in (62 as
Σ k k-n
nd nu du (DV0))-n =D- D-(0-1/0)
D1/2
and the result follows upon putting things together.
Lemma 1.2. For x = k(2,1) T) defined as in 2 holds that
ŌN
ax df2(k)/T 2 1.
(63)
The formula given in the above lemma is useful because can be combined with the identities
ŌR
Bias = we ΣX{E KT)
wwo
(64)
ax
Var df2(x) (x) ŌR
(65)
T ax
The RHS of 664 is usually referred to as the omniscient risk Hastie al. [2]. Cui et al. 13, Wei
et al. 48
Proof of Lemma 1.2
By definition of k, we know that
K kdf, (x)/T e + /T.
Let = 0X
Differentiating the above identity w.r.t. À gives
R' - 1- K' (Ir E(E + =1 lrr Σ(Σ + x) //T ni' l (E+ =(d[((k)//
and the result follows upon rearranging. Note that we have used the identity
1 -- 1- = = I(E+
rewrite E + xz) xE -d)
2
28

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We explicitly describe the setting, as well as the conclusions of our theory in various regimes.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We clearly state that our theory applies to the case of kernel ridge regression. All conclusions to different models are by analogy only, as is always done when this type of model is studied.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
29

---

Answer: [Yes]
Justification: All assumptions are clearly stated in the main text, and all proofs can be found in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We clearly give all experimental details, including parameters. These are sufficient to reproduce all our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
30

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: We only use one publicly available dataset. MNIST, and no idiosyncratic model. Thus, we provide neither dataset nor code, as the dataset is publicly available, and the experiments are easy to reproduce from their description.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
publitc/guudes//odettttiii oww.ssabbsisiooooiooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/mipss.cc/pulli//uuiiee htt///CooSSSeiiaaeeeeeee sisssssssiniyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We precisely describe all our (simple) linear or kernel methods in sufficient detail. They can casily be reproduced from these.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Error bars are given and we describe the number of trials over which they obtained. We believe these are chosen suitably.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
31

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: Compared to most contributions, our experiments, which accompany our theory, do not require any significant compute whatsoever.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: http:///warrrga.cooobaacacaaaaaa
Question:
Answer: [Yes]
Justification: our research does not involve human participants, and never uses private data or models.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: Our paper provides toy theory that underlines the point that retraining data generated from currently available models might lead vicious loop of model collapse, which might have societal impact. We call for attention to this matter.
32

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not release any models or new data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The only public dataset we use is MNIST [16]. which we properly credit. The MNIST dataset is made available under the terms of the Creative Commons Attribution-
Justification:
Share Alike 3.0 license.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
33

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the palasets.papersstt paperssi thcode. com con/datases package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release any new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not involve crowdsourcing nor research on humon subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not involve crowdsourcing nor research on humon subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
34

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
35


Reviews:
[{"review_id": "8F1Cf2KpiF", "reviewer": "Reviewer_16H4", "summary": "This paper studies the model collapse problem in the context of ridge regression. The authors train a sequence of linear models which are trained on Gaussian data and characterize the test error under various settings. Their results are intuitive and theoretically support the ongoing research area of model collapse in generative models. For example, they show that the test error increases more for larger number of generations n in the ordinary least squares settings. They heavily use random matrix theory to derive their results.", "questions": "None", "limitations": "The work is mostly on linear models with Gaussian data.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "1. The theoretical setup is sound and a good way to incorporate supervision in a generative model context in a way that is fitting for theoretical study.\n2. The results are interesting and intuitive. In fact they reflect what we have seen empirically in several of the previous works.\n3. Interestingly enough, since you use a supervised version of the model collapse, the martingale principle used in [2] does not apply. This makes this setting very interesting to study and I suggest that you mention this in your paper as it is a contribution which I don’t think that you claim.", "weaknesses": "Here are some things to be improved on in decreasing order of importance:\n1. The main weakness of the paper is that it seems only tangentially related to the generative model collapse problem. This is for several reasons but most importantly because the kind of experimental and theoretical setup explained in this paper is not a complex generative model. The model collapse problem occurs with generative models which are quite complex (not linear models on gaussian data). Additionally, the results in the paper are all asymptotic, which don’t tell us too much about non-asymptotic behavior which we see in practice. That being said, the work done in this paper is actually really good and definitely is really polished and should be published. Therefore, experiments using deep learning models supporting the theoretical thesis of this paper are very important for this work to appear applicable to the generative model collapse problem.\n2. The abstract is very vague and doesn’t describe what actually happens in the paper. Here are several areas of improvement:\n- It is not mentioned in the abstract that the paper seems to focus primarily on Gaussian data, an important point. \n- Moreover, “... quantitatively outline this phenomenon in a broad range of regimes.” Please say what regimes you are talking about otherwise this statement doesn’t really tell the reader almost any information.\n- You mention “how model collapse affects both bias and variance terms” which is a very general statement but does this happen no matter what the model is? Or only with your specific setup? The evidence in the paper points to the latter and thus should be mention so that a reader does not think that this is a very general result.\n- Results are pretty much all asymptotic. This is kind of hinted at by the mentioning of “scaling laws” but needs to be more explicit in order to reflect the work of the paper.\n3. It is not clear what was mystified and now has been cleared up…. This is an important point because it is literally in the title of the paper.\n4. Formatting:\n- The axes in Figure 1 and 2 are way too small and hard to see. \n- The flowchart in Figure 3 has tiny font too.\n- You don’t have to change these, but I recommend not starting sentences with citations as in lines 35, 94, 98.\n- On line 213 you mention that the formula for rho is in the appendix. I would not do this because then the reader has to go to the appendix to understand what rho is. Especially since you actually state what rho is in a special case on Equation (17) after redirecting the reader to the appendix.\n- The conditions in Equation (22) need to be defined\n5. Typos and mistakes:\n- It seems that between equation (4) and (5) that there is a mistake in defining \\widehat w_0\n- For large d but actually for d->inf\n- There is supposed to be no space on the section title on line 173", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your thoughtful comments, and are delighted you appreciate the technical heavy lift that was required to prove our results and paint a picture of the regimes of \"model collapse\" in the setting of ridge regression. Thank you for pointing out that we should highlight what other technical arguments would fail (like martingale arguments) - we are adding this point to the next version!\n\n\n1. > The main weakness of the paper is that it seems only tangentially related to the generative model collapse problem ... Therefore, experiments using deep learning models supporting the theoretical thesis of this paper are very important for this work to appear applicable to the generative model collapse problem. \n\nWe addressed it in the general response.\n\n2. > The abstract is very vague and doesn’t describe what actually happens in the paper. \n> It is not mentioned in the abstract that the paper seems to focus primarily on Gaussian data, an important point.\nMoreover, “... quantitatively outline this phenomenon in a broad range of regimes.” Please say what regimes you are talking about otherwise this statement doesn’t really tell the reader almost any information.\nYou mention “how model collapse affects both bias and variance terms” which is a very general statement but does this happen no matter what the model is? Or only with your specific setup? The evidence in the paper points to the latter and thus should be mention so that a reader does not think that this is a very general result.\n\nThank you for the suggestion! We have modified the abstract accordingly, with the differences marked in **bold**.\n\nThe era of proliferation of large language and image generation models begs the question of what happens if models are trained on the synthesized outputs of other models. The phenomenon of \"model collapse\" refers to the situation whereby as a model is trained recursively on data generated from previous generations of itself over time, its performance degrades until the model eventually becomes completely useless, i.e. the model collapses. In this work, we investigate this phenomenon within the context of high-dimensional regression **with Gaussian data**, considering both low- and high-dimensional asymptotics. We derive analytical formulas that quantitatively describe this phenomenon in **both under-parameterized and over-parameterized regimes**. We show how test error increases linearly in the number of model iterations in terms of all problem hyperparameters (covariance spectrum, regularization, label noise level, dataset size) and further isolate how model collapse affects both bias and variance terms **in our setup**. We show that even in the noise-free case, catastrophic (exponentially fast) model-collapse can happen in the over-parametrized regime. In the special case of polynomial decaying spectral and source conditions, we obtain modified scaling laws which exhibit new crossover phenomena from fast to slow rates. We also propose a simple strategy based on adaptive regularization to mitigate model collapse. Our theoretical results are validated with experiments.\n\n> Results are pretty much all asymptotic. This is kind of hinted at by the mentioning of “scaling laws” but needs to be more explicit in order to reflect the work of the paper.\n\nWe mentioned that \"In this work, we study this phenomenon ..., under low- and high-dimensional asymptotics\".\n\n3. > It is not clear what was mystified and now has been cleared up…\n\nWe have discussed how our theory compares with previous theories in the general response. Generally speaking, the concept of model collapse, as popularized by prior works, still lacks a theoretical framework to support empirically-backed findings. Our work addresses this gap by developing an analytic theory that clearly demonstrates how model collapse quantifiably emerges from training on synthetic data. This is reflected in the 'demystified' aspect of the title of our paper. We will incorporate this comparison into the paper.\n\n4. > Formatting\n\nThanks for the suggestion. We will change the font size. \n\n5. > Typos and mistakes:\n\nThanks a lot! We have corrected them.\n\nWe would like to thank you once again for reviewing our work and helping us improve its presentation, especially with regard to weakness No.2. Please let us know if you have any more questions. If there are no outstanding concerns, we would kindly ask you to consider raising your score which would substantially help in reaching a reviewer consensus. Thank you.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dk4DvioYH4", "reviewer": "Reviewer_Xs1n", "summary": "Authors analyze the problem of ´model collapse´, i.e., what happens when a generative model is retrained on data generated by itself. The authors study a slightly different and simplified setup: linear regression, where the target is iteratively generated by previous estimates of the coefficients. Based on random matrix theory tools authors provide sample complexity bounds on the estimation of the oracle regression coefficients.", "questions": "- What´s the point of figure 1a? Illustrating the degradation of the models as they are retrained on synthetic data? What is the difference betzeen the solid and the dashed curves?\n- What is $T$ exactly? Is this the number of synthetic samples at each step? The size of the initial dataset? Appendix did not provide clarifications to me on this question. This information seems vital to understand Figures 1a and 1b.\n- I have a clarification question on the the theoretical setup: is a `semi-supervised` setup, where you assume that you have access to $Y_0, X_1, \\dots, X_n$?\n\n- On Theorem 3 (Equation 15), it seems that there is a remaining error $n \\\\sigma\\_0 \\\\rho$, even if the model is perfect at step $0$. In other words, is it normal that if the model is perfect at step $0$, then the test error still increase with the number of retraining $n$?", "limitations": "No", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper tackles an interesting and timely topic, the theoretical results seem pretty sophisticated and sharp", "weaknesses": "It seems that the provided analyses, in some kind of semi-supervised learning, do not match the motivations.\nThe motivation is clear, but the paper is overall hard to follow and often consists of a succession of arid theoretical results.\nEither my area of expertise is too far, or the paper needs to be rewritten.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback, especially on the presentation of our material, which is challenging, given the amount of technical work required for our ((necessarily lengthy) proofs, and the difficulty of partitioning this content petween the main body and the appendix. Our aim for the main body of the paper was to present and to explain and connect the various technical results which are very hard to state in a simplified fashion. We take your point and will add a content overview style paragraph/picture to the paper (or the appendix if space does not permit this).\n\n1. > What´s the point of figure 1a? Illustrating the degradation of the models as they are retrained on synthetic data? What is the difference betzeen the solid and the dashed curves?\n\nThe figure illustrates three key aspects: the degradation with respect to the number of generations, the effect of regularization on accuracy, and the scaling law curve concerning the amount of data. The dashed curves represent theoretical predictions, while the solid curves depict experimental results.\n\n2. > What is $T$ exactly?\n\n$T$ represents the number of samples generated by the generator to train the subsequent model, while $T_0$ denotes the number of samples used to train the previous generators. The complete definition can be found in Equation (6) and please see Figure 3 for an illustration of all our notation. We will enhance the caption of Figure 1 - thank you for pointing out the possible unclarity.\n\n3. > I have a clarification question on the the theoretical setup: is it a semi-supervised setup, where you assume that you have access to $Y_0, X_1, X_2, ..., X_n$?\n\nNo. We consider the fully supervised setting. We use $X_i$ to denote the input data at iteration $i$, and we only have $\\bar Y_i, X_i$. In the paper, we define the data and labels used to train the subsequent models in Equation (5) within the box, and we further elaborate on this in Figure 3.\n\n4. > On Theorem 3 (Equation 15), it seems that there is a remaining error $n \\sigma_0 \\rho$, even if the model is perfect at step 0. In other words, is it normal that if the model is perfect at step 0, then the test error still increase with the number of retraining?\n\nYes. Consider the groud truth data as generated from a perfect model, and our theory still apply. The test error of consequent models still increase since there are finite sample bias and label noises.\n\nWe thank you again for the appreciation of our work and are at your disposal if there are more questions. We hope our replies find your approval and would incite you to raise your score. Thank you.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B7tmawtGqB", "reviewer": "Reviewer_Z9jq", "summary": "This paper provides a theoretical analysis of the \"model collapse\" phenomenon that can occur when machine learning models are trained on synthetic data generated by other AI models. The key contributions include:\n\n1. A mathematical framework for studying model collapse in high-dimensional regression settings.\n\n2. Exact analytic formulae for how test error increases as a function of the number of synthetic data generations, regularization, sample sizes, and other parameters.\n\n3. Demonstration of how model collapse can lead to a change in scaling laws, with test error potentially plateauing or even diverging rather than continuing to decrease with more data.\n\n4. Analysis showing that even in noiseless settings, overparameterized synthetic data generators can lead to catastrophic model collapse.\n\n5. Insights on how to potentially mitigate model collapse through adaptive regularization.\n\nThe authors provide theoretical results as well as empirical validation through experiments. Overall, the paper offers a rigorous mathematical treatment of model collapse, providing insights into how and why performance degrades when training on iteratively generated synthetic data.", "questions": "1. While you propose adaptive regularization as a potential solution to mitigate model collapse, could you elaborate on other practical strategies that might be effective? For instance, how might techniques like data filtering, model ensembling, or continual learning potentially address the issues you've identified?\n\n2. Given your findings on how model performance can degrade with synthetic data, what implications do you see for AI safety and robustness, especially as AI-generated content becomes more prevalent on the internet? How might your work inform strategies for maintaining model quality and reliability in an ecosystem increasingly populated by AI-generated data?", "limitations": "1. The analysis primarily focuses on linear regression models, which may not fully capture the complexities of advanced machine learning architectures like deep neural networks or transformers used in modern AI systems.\n\n2. The theoretical framework relies on idealized data distributions (e.g., Gaussian) and noise models, which may not accurately represent the diverse and complex nature of real-world datasets used in training large AI models.\n\n3. While the paper includes experimental validation, it primarily uses synthetic datasets. The absence of experiments on large-scale, real-world datasets limits the immediate applicability of the findings to practical scenarios.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a solid mathematical framework for analyzing model collapse, offering precise analytical formulae rather than just empirical observations.\n\n2. It covers various scenarios, including different dimensionality regimes, noise levels, and spectral conditions, providing a broad understanding of the phenomenon.\n\n3. The work reveals new phenomena, such as the change in scaling laws and the potential for catastrophic collapse even in noiseless settings, advancing our understanding of model behavior with synthetic data.\n\n4. The analysis of regularization effects and the proposal for adaptive regularization offer actionable insights for practitioners dealing with synthetic data.\n\n5. While primarily theoretical, the paper validates its findings with experiments, strengthening the connection between theory and practical applications.", "weaknesses": "1. The analysis primarily focuses on regression tasks, which may not fully capture the complexities of more advanced machine learning models like large language models or image generators.\n\n2. The theoretical framework relies on certain simplifying assumptions (e.g., Gaussian data distribution, linear models) that may not always hold in real-world scenarios.\n\n3. While the paper includes experiments, they are primarily on synthetic datasets. Validation on real-world, large-scale datasets could further strengthen the findings.\n\n4. While the paper proposes adaptive regularization as a potential mitigation strategy, it doesn't extensively explore or compare other possible solutions to model collapse.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review and positive evaluation of our work, as well as your acknowledgment of its strengths.\n\n1. > The analysis primarily focuses on regression tasks, which may not fully capture the complexities of more advanced machine learning models like large language models or image generators. The theoretical framework relies on certain simplifying assumptions (e.g., Gaussian data distribution, linear models) that may not always hold in real-world scenarios. While the paper includes experiments, they are primarily on synthetic datasets. Validation on real-world, large-scale datasets could further strengthen the findings.\n\nWe addressed it in the general response.\n\n2. > While you propose adaptive regularization as a potential solution to mitigate model collapse, could you elaborate on other practical strategies that might be effective? For instance, how might techniques like data filtering, model ensembling, or continual learning potentially address the issues you've identified?\n\nScaling laws tend to deteriorate when incorporating synthetic data. As demonstrated in our Theorem 4.1, adding more synthetic data can either break scaling laws or significantly worsen them. To achieve improvements in the next generation of foundational models, it is crucial to preserve these scaling laws. Data filtering is certainly beneficial in this regard. Additionally, watermarking synthetic data is becoming increasingly important as it aids in distinguishing high-quality real data from synthetic counterparts.\n\nAdaptive regularization in Corollary 4.2, akin to early stopping, suggests that early intervention during training can prevent model collapse. Furthermore, using the current model to filter and bootstrap the data could be a viable solution. This approach can help maintain the integrity of the training process by ensuring that only the most relevant and high-quality data, whether real or synthetic, is used.\n\nWe will add the discussion to the paper.\n\n3. > Given your findings on how model performance can degrade with synthetic data, what implications do you see for AI safety and robustness, especially as AI-generated content becomes more prevalent on the internet? How might your work inform strategies for maintaining model quality and reliability in an ecosystem increasingly populated by AI-generated data?\n\nThe key takeaway is that scaling alone is insufficient when synthetic data is involved; these data can alter traditional training paradigms, particularly as optimal regularizations change. Consequently, new algorithms with appropriate regularizations are necessary. Maintaining the quality and reliability of models requires rigorous validation processes focused on data quality. More effort must be directed toward data curation to ensure the integrity of training sets. Additionally, developing tools and methodologies for detecting AI-generated content, such as watermarking, is essential to distinguish synthetic data from real data and to maintain model reliability.\n\nWe remain available to address any further questions you may have. We hope our responses address your questions and encourage you to consider raising your evaluation score. Thank you.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fk2dLrLMrI", "reviewer": "Reviewer_jLLV", "summary": "This paper analyses a ridge regression model of the recently described phenomenon of model collapse in various settings (unregularised linear regression, large dimensional RMT limit regression), acquiring analytical expressions for scaling limits in the case of heavy tails under capacity and source conditions, which are validated against numerical experiments.", "questions": "The connection to self-distillation work is briefly mentioned the appendix, however it is mentioned that it stands apart from it. I am not clear how this is the case? This is especially not true in the noiseless label setting, when design matrices are fixed to be the same. When either of those isnt true, we still find ourselves in the distillation setting, alas not exactly the model considered in Mobahi et al. Could you clarify this either for me or in the paper? \n\nSection 4 ends very abruptly, and no in-depth discussion of the model is presented. I believe this to be a significant limitation, as relevance of the model becomes unclear.\n\n\nFurther comments: \n* line 30 - kernel regression is not analysed in this paper\n* line 47 - you do not mention what \\epsilon is\n* line 54 - Insights from this formula needs to be related to existing mathematical models of model collapse.\n* line 118 - \\hat{\\omega} is a vector input to E_{test}? Or is it a random variable for expectation? \n* It does not seem to be clarified what the effect of sampling different design matrices is.\n* around line 128 - notation seems a bit overloaded, with n subscripts being dropped, even though you may not have the exact same variables across iterations? E.g. X_n can not always be denoted by X?\n* line 155 - the whole paragraph seems to be describing almost the same result as Shumailov et al., but does not mention it.\n* line 173 missing citation\n* I am slightly unclear why section 3.4 is underparametrized - could you clarify?", "limitations": "See above responses", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The model presented in the pair is well-presented, and put into context in terms of extra analytical term additions that arise due to recursive re-training on synthetic data. This is particularly interesting for analysing the phenomenon of model collapse, as it highlights that scaling laws must be adjusted when synthetic data appears in the training data of the model, and further highlights the necessity of adjustable regularization - directly deriving how the optimal regularisation choice changes due to synthetic data. This places the paper in a very good spot as a first step towards understanding how to mitigate model collapse.", "weaknesses": "There are a number of weakness that this paper does not address in extensive detail. To begin with, the model considered here is extremely simplistic and places itself much closer to the distillation literature, rather than generative literature. I believe too much emphasis is made on the linear regression, which does not seem to provide significantly more insights beyond that provided in the original model collapse paper by Shumailov et al. There may be more, which I may not be seeing, but they dont appear to be mentioned in the paper explicitly. The particularly interesting section establishing the scaling laws, which goes significantly beyond the model in Shumailov et al. seems to be only briefly discussed, and the novel insights from this analysis are not discussed beyond the paragraph in the introduction. \n\nWhile it may be interesting as a generic model, the potential insights it might bring into the process of model collapse are much more important, and for this it needs to be put into context of the limitations of previous proposed models of the phenomenon. \n\nIn addition, the paper multiple times claims to analyse the kernel ridge regression model. However, *none* of the models described are in the kernel regression setting. While it may be an uncomplicated extension, as discussed in Appendix B, the current version of the paper does not perform this extension - and none of the results are presented in this setting. Thus, currently the paper is overselling its contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and the dedicated time and effort you invested. We genuinely appreciate your recognition of the key strengths in our work.\n\n1. > To begin with, the model considered here is extremely simplistic and places itself much closer to the distillation literature, rather than generative literature ...\n\nThanks for the question. We have addressed it in the general response, and will include a discussion comparing our results with previous theoretical results in the paper.\n\n2. > The particularly interesting section establishing the scaling laws, which goes significantly beyond the model in Shumailov et al. seems to be only briefly discussed, and the novel insights from this analysis are not discussed beyond the paragraph in the introduction.\n\nThe purpose of Section 4 to which the reviewer is referring is to quantifiable show how model collapse manifests as a **change of scaling laws**, in a setting which is close to neural networks (in certain regimes) and large language models. We recall that scaling laws (Kaplan et al. [26] Hoffmann et al. [24]), which relate the test error of a model to the sample size, model size, and amount of compute used, are one of the main instruments used by practitioners in strategic allocation of resources in the design and implementation of large language models (ChatGPT, Llama, etc.). Indeed, the results of Section 4 predict that the scaling laws w.r.t dataset slow down (i.e smaller exponents) in the presence of **synthetic data**: a much larger sample size might be required to achieve the same decrease in test error as with real data. Moreover, the test error for models trained on **synthetic data** will eventually plateau as a function of sample size, in sharp contrast to the idealized classical picture [24, 26] where the test error for models trained on **real data** is predicted to decrease forever. These theoretical results are illustrated in Figure 1(b) and Figure 4.\n\nWe will add the discussion to the paper.\n\n3. > While it may be interesting as a generic model, the potential insights it might bring into the process of model collapse are much more important, and for this it needs to be put into context of the limitations of previous proposed models of the phenomenon.\n\nWe have addressed the point raise here in the general response and will include it in the paper.\n\n4. > The kernel ridge regression model. \n\nThank you for the question. The main difference between linear regression and kernel regression lies in the use of the kernel feature mapping. In our setting, generalizing from linear to kernel regression is straightforward. We chose to use linear regression because it is simpler in terms of both notation and understanding. As noted at line 47, \"we present the linear regression setting for simplicity; the generalization to the kernel setting is straightforward.\" and we have discussed it in more details in Appendix B. We will also add a footnote at line 30.\n\n5. > The connection to self-distillation.\n\nThank you for raising this point, which caused us some grief to formulate succintly in the paper. Allow us to expand here, to address your very important question (Model collapse versus self-distillation):\nOur setting is inspired by the model collapse phenomenon, where more and more there will be vast amounts of synthetic data generated by users posted online, which will necessarily enter the training set of the next foundation model. In this case, we do not have the groud truth label, and the generation of synthetic data is not controlled by us, but by other users. Therefore, we adopt the setting with only synthetic labels with added noise.\n\nSpecifically, in our setup, at generation $n>0$, one doesn't have access to the true labels $Y_0=f_0(X) + noise$ for the training samples $X$, but rather to some $\\hat Y_n = \\hat f_n(X) + noise$, where $\\hat f_n$ is an unknown function, which synthesizes fake labels iteratively; the integer $n$ is the number of iterations. $f_0$ is the ground-truth labelling function (unknown). In our work, we make the structural assumption that $\\hat f_n$ is obtained by iterative / successive regressions on a true dataset $D_0=(X_0,Y_0)$, and then new noise is injected every single generation. The fake labels $\\hat Y_n$ are like labels for images $X$ on internet, or outputs from a LLM like ChatGPT. One doesn't have any control over the creation of these labels, especially because of the noise injected at each stage. In practice, one doesn't even know that they're not real labels. The final model is learnt on the dataset $(X,\\widehat Y_n)$.\n\nIn the **self-distillation** setting, one has access to the true labels $Y$. One decides to replace $Y$ with some $Y_n := F_n(X,Y)$. Here, $f_0$ is the ground-truth labelling function (i.e $f_0(x) := x^\\top w_0$ in the case of linear models). The final model learned is via regression on the dataset $(X,Y_n)$. It is up to the practitioner to chose what the number of iterations $n$, and in fact Mobahi et al. showed that there is an optimal value of $n$ depending on the distribution of the data and the model class over which the regression is done. This is what we mean by having control over the data synthesization process. Thus, mathematically, the difference boils down to the absence of fresh noise when data is regenerated in SD, which leads to completely different behavior to our setting, where we do not control the noise injected at each generation. **Allow us to use a metaphore here. Model collapse is like the rain, while SD is like the shower. The latter can be controlled by the user (for example, turned on / off), while the user has to cope with the former.**\n\n6. > Section 4 ends very abruptly, and no in-depth discussion of the model is presented. \n\nThank you for the question. We agree that additional explanation is necessary. We have provided a discussion in response to question 2 and will include it in the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We appreciate the time and effort the reviewers have dedicated to reviewing our paper. We are delighted to see that overall reception has been positive, with all reviewers acknowledging our theoretical contributions, particularly in analytically characterizing model collapse across a wide range of settings and our findings on scaling laws. Building on extensive work in the classical (non-iterative) case [1], we demonstrate how previously understood phenomena related to scaling and optimal regularization are altered with synthetic data. These insights, derived from comprehensive analyses using random matrix theory, align with empirical observations [2, 11], despite the simplicity of our model and the use of Gaussian data (with general covariance). The findings on scaling laws are particularly relevant to practitioners, as they are foundational for large language models. We hope this work serves as a catalyst for further studies in the field.

One common concern among reviewers is whether our results, based on Gaussian data and linear regressions, generalize to modern generative models for images and languages. We would like to clarify our focus on theoretical aspects. In the area of model collapse, there exists a substantial body of literature [3, 4, 5], that provides theoretical analyses using Gaussian data. For example, [6] examines the mixing effects of combining real and synthetic data in generative models using a linear model, Gaussian data, and asymptotic analysis. Additionally, [7] explores scaling laws in linear regression with Gaussian data, revealing phenomena that align with empirical observations. Thus, the use of Gaussian data and linear regression is a standard approach that can offer valuable insights even for large models.

Moreover, regression is not limited to traditional applications; current large language models (LLMs) are also utilized in labeling tasks. Many tasks, such as question-answering (QA), code generation, and mathematical problem-solving, involve inputs in the form of questions x and responses y. This provides a rationale for extending our results to real-world applications, as we have observed strong alignment with findings from other empirical studies. Furthermore, there is ample evidence [8, 9, 10] suggesting that LLMs exhibit linear behavior in their handling of concepts. Thus, regression with complex kernel functions serves as a good theoretical model.

Our work specifically focuses on high-dimensional linear regression on multivariate data because it allows us to: (1) achieve a solvable setting, and (2) explore a sufficiently rich model (through the covariance matrix $\Sigma$ and the relative scaling of sample size $n$ and and dimensionality $d$). This approach allows us to abstract away all the nitty gritty of neural networks and large language models, while enabling us to present a clear analytic picture which explains previously reported empirical findings on model collapse and provides an effective theory for this interesting phenomenon. Also, we outline fundamental limitations of synthetic data, the impact of label noise, and the degrees of over-/under-parameterization, etc.

Compared to prior works, our theoretical results offer a more detailed analysis. Prior studies have shown that model collapse occurs, but often without fine-grained analysis or only within a narrow range of settings. For instance, [3] shows model collapse with increasing variance, which approximates our results in an over-parameterized setting. [5] provides stability analysis at the distribution level with infinite data, focusing only locally, aligning with fine-tuning rather than training from scratch. We appreciate reviewer 16H4's observation that [4] relies on martingale techniques at the expectation level. In contrast, our work applies to general cases, providing analytic results to fully understand model collapse. We demonstrate how scaling laws change as a consequence, how optimal regularization shifts, and that model collapse can occur even without noise, expanding beyond the theories presented in [3]. These insights provide a clear understanding that was not achievable with previous approaches.

Some reviewers also suggested adding more empirical experiments. As discussed, numerous papers have already documented empirical observations of model collapse. Our primary contribution lies in the theoretical domain, where we are the first to fully characterize model collapse to this extent. Our goal is to comprehensively understand the factors contributing to model collapse and their effects on scaling laws and regularization. Given the depth of our theoretical analysis, we believe additional empirical results would be superfluous. Our findings on model collapse and scaling laws are consistent with the experiments reported in [7, 8], which explored generative models in both mathematical and natural language contexts, and in [4, 5], which investigated generative models for images.

In conclusion, we are grateful for the constructive feedback provided by the reviewers. Our work not only addresses significant theoretical questions but also aligns closely with empirical findings, providing a general framework for understanding complex phenomena in model collapse. 

[1] Cui, Hugo, et al. "Generalization error rates in kernel regression: The crossover from the noiseless to noisy regime." NIPS 2021.

[2] Elvis Dohmatob, et al. "A Tale of Tails: Model Collapse as a Change of Scaling Laws." ICML 2024.

[3] Ilia Shumailov, et al. "AI models collapse when trained on recursively generated data." Nature 2024.

[4] Sina Alemohammad, et al. "Self-Consuming Generative Models Go MAD." ICLR 2024.

[5] Quentin Bertrand, et al. "On the stability of iterative retraining of generative models on their own data." ICLR 2024.

[6] Ayush Jain, et al. "Scaling laws for learning with real and surrogate data."

[7] Licong Lin, et al. "Scaling Laws in Linear Regression: Compute, Parameters, and Data."

Meta-review: This paper provides a theory for the model collapse phenomenon in the high dimensional regime of linear regression and Gaussian data. In addition, the paper examines the mitigation of model collapse by adjustable regularization. Although the examined regression setting has conceptual differences from model collapse in generative models, the contributions are of sufficient interest and significance and can be considered for publication. 

During the rebuttal and Authors-Reviewers discussion, the authors significantly improved their paper following detailed comments and discussion with Reviewers 16H4 and jLLV. This improvement includes better description of details and claims regrading the contributions in this paper. Moreover, the original submission did not consider the connection of its linear model theory to nonlinear models. In the rebuttal, following comments by Reviewer 16H4, the authors added experiment results showing that their theoretical principles appear (to some extent) in nonlinear two-layer neural networks. By this, the authors have adequately addressed this concern. The authors are requested to update their paper according to their discussions with the reviewers. 

Additional requests for the authors to implement in their camera-ready version of the paper: 
1. The authors claim in the paper that their results for linear regression can be *straightforwardly* extended to kernel regression. Reviewer jLLV commented about this and discussed this with the authors.  At the end of this discussion, the authors wrote that `` We agree with the reviewer that the statement about kernel regression should be a remark that our analysis can be straightforwardly extended to the case of kernel regression. This will be rectified.’’ The authors mentioned that they discuss the extension to kernel regression in Appendix B --- There they explain that it is *straightforward* to extend their results to kernels by applying the Gaussian assumption on the feature vector instead of the input vector. But this seems too simplistic, because the extension should still consider a Gaussian input and then for a nonlinear feature map the feature vector will not be Gaussian. Their assumption that the feature vector is Gaussian can hold in overly simple cases, for example, if the input is Gaussian and the feature map is linear. Alternatively, it might be that this assumption could be justified by another, more complex explanation than what currently available in the paper. Accordingly, the authors are requested to remove from the camera-ready version the claim that their results are *straightforwardly* extended to kernel regression, as well as the explanation in Appendix B. This request is also supported by further discussion I had with reviewers. Importantly, the removal of this claim from the paper does not reduces its contribution and significance.
2. The authors are requested to further search for typos and fix them. This includes the appendices and the proofs. For example, please check the proof in Appendix E.2 (the equations after line 625), as there might be a typo in the subscript numbering in the equality between the last expression of the first line of equations to the first expression in the second line of equations.

Based on reading the paper, the reviews, the authors’ rebuttal and their discussion with the reviewers, I find this paper of sufficient quality and significance for the theory of contemporary machine learning. My recommendation is therefore to accept this paper.
Decision: Accept (poster)