What happens when generative AI models train recursively on each others' outputs?
Authors: Hung Anh Vu
Galen Reeves
Emily Wenger
Venue: ICLR 2026 Poster (2026)
Keywords: model collapse
generative AI
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: When generative AI models train on each others' generated outputs, they benefit by learning from other models' distributions but can become homogeneous.

Abstract: The internet serves as a common source of training data for generative AI (genAI) models but is increasingly populated with AI-generated content. This duality raises the possibility that future genAI models may be trained on other models'  generated outputs. Prior work has studied consequences of models training on their own generated outputs, but limited work has considered what happens if models ingest content produced by other models. Given society's increasing dependence on genAI tools, understanding such data-mediated model interactions is critical. This work provides empirical evidence for how data-mediated interactions might unfold in practice, develops a theoretical model for this interactive training process, and experimentally validates the theory. We find that data-mediated interactions can benefit models by exposing them to novel concepts perhaps missed in original training data, but also can homogenize their performance on shared tasks.

Published as a conference paper at ICLR 2026
WHAT HAPPENS WHEN GENERATIVE AI MODELS TRAIN RECURSIVELY ON EACH OTHER'S S OUTPUTS?
Hung Anh Vu, Galen Reeves, & Emily Wenger* Department of Electrical and Computer Engineering Duke University
ABSTRACT
The internet serves as a common source of training data for generative (genAl) models but is increasingly populated with Al-generated content. This duality raises the possibility that future genAl models may be trained on other models' generated outputs. Prior work has studied consequences of models training their own generated outputs, but limited work has considered what happens models ingest content produced by other models. Given society's increasing dependence on genAl tools, understanding such data-mediated model interactions is critical. This work provides empirical evidence for how data-mediated interactions might unfold in practice, develops theoretical model for this interactive training process, and experimentally validates the theory. We find that data- mnda-meddated interactions can benefit models by exposing them to novel concepts perhaps missed in original training data, but also can homogenize their performance on shared tasks.
INTRODUCTION
Since the release of ChatGPT in 2022, generative AI (genAl) models have exploded in popularity. Now capable generating highly realistic text, images, and videos, these models have been widely adopted for various cases, from creative idea generation (Ali Elfa & Dawood, 2023) to healthcare support (Reddy, 2024) to national security settings (Harding, 2024; GAO, 2024). Given the significant uptick in genAl use across numerous industries, this technology is clearly here to stay. Consequently, interrogating potential ways genAl models could evolve- in positive or harmful ways-is is critical.
With few exceptions, today's large-scale genAl models trained massive datasets sourced from the internet. Widely-accepted scaling laws for model performance say that training on more data aids learning (Kaplan al., 2020), and the internet provides rich, cheap, and ever-evolving source of training data. Although whitepapers for more recent genAl models withhold details about training set pntpoosttooooooenttti
due ongoing litigation about copyright concerns- eeveerns-videnne from earlier whitepapers indicates that scraped data was used to train models like Llama, Gemini, Phi, the GPT series, Claude, and others (Dubey et al., 2024; Achiam al., 2023; Team al., 2024a; Jiang et al., 2023; coh, 2024; Anthropic, 2023; Abdin et al., 2024).
Beyond privacy and copyright concerns, training on scraped data could have other downsides. Prior work has noted that genAl models trained recursively on their own generated outputs "collapse," becoming unable to generate meaningful content (Shumailov et al., 2024; Hataya et al., 2023; Martínez al., 2023; Alemohammad al., 2024). This scenario is feasible, since Al-generated content abounds online (Sun et al., 2025) and could be part of future training datasets. However, subsequent work has proposed ways to mitigate collapse via reuse of non-Al-generated data in subsequent training iterations (Dey & Donoho, 2024; Kazdan et al., 2025; Dohmatob et al., 2025; Feng et al., 2024). Model collapse remains an activate research area (Schaeffer et al., 2025).
Yet, prior work studying the dynamics of model collapse has overlooked another reality: the internet teems with content from many genAl models. Today's most popular models have millions of users (Reuters, 2024; Handa et al., 2025; AI, 2024a), who leverage generative Al tools to create online content like web pages and social media posts (gen, 2022). Recent work showed that up to 40% of content on popular sites like Quora is now Al-generated (Sun et al., 2025). Given the
"Corresponding author; emi 1y. wyweeger@dukee

---

Published as a conference paper at ICLR 2026
increasing availability of these models for variety of public-facing uses, Al Al-generated content from many different models will continue to proliferate.
The standard practice of training scraped internet data and the increasing prevalence of Al- generated content online suggest the strong possibility that future generative Al models will be trained on other models' outputs. Yet, this aspect of model training has received relatively little attention. Given the widespread adoption of generative Al models in critical settings like healthcare and national security, this phenomenon ought to be investigated ensure models remain helpful and trustworthy.
Contributions. To address this need, this work theoretically derives and experimentally evaluates the long-term evolutionary behavior of generative models trained on each other's data. Specifically, we
Develop a framework describing
interactions between genAl models. Derive concise formulas describing the dynamics of interactive training under varied regimes.
Run experiments on large language models to understand how iata-medi interactions affect model performance in practice.
Key findings. Both our theoretical analysis and experiments show that when training with a mixture of real and synthetic data, the implicit interaction between heterogeneous models and datasets can have both positive and negative impacts. Specifically, interactive recursive training can help models learn from each other's private data but risks homogenization. At high level, well known concepts in statistical learning theory anticipate this: recursive training on the same data is bad (e.g., overfitting and model "collapse") but training on novel data, even if synthetic, can boost performance (e.g., transfer learning). Our experimental results provide concrete evidence that these phenomena can occur simultaneously. Future work should further study these dynamics.
2 RELATED WORK
Model collapse is recently observed phenomenon in large-scat scale- generative text and image models. It referred- its earliest form-to the phenomenon of models performing much worse after generations of training on their own generated outputs (Shumailov et al., 2024; Peterson, 2025; Wang et al., 2024; Alemohammad et al., 2024; Feng et al., 2024; Hataya et al., 2023; Dohmatob et al., 2025; Martínez et al., 2023). Theoretical and empirical results from these works show that models. if trained on generated outputs from their prior versions, slowly degrade in performance as generations progress. One way this manifests is in models forgetting the tails of their original (real) training data, since generated content tends not to contain rare content from the original training data. Training repeatedly on truncated, synthetic data leads the model forget the richness of its original distribution, resulting in degraded performance (at best) and total failure (at worst).
Mitigating model collapse. Despite the dire predictions of these papers, subsequent work has proposed simple mitigation strategy: instead of discarding all prior (human-generated) training data, retain some fraction of this while augmenting it with generated data. Numerous works have observed that this choice to augment instead of discard the original training dataset results in a bounded error in future models, avoiding collapse (Kazdan et al., 2025; Gerstgrasser al., 2024; Marchi et al., 2024). Although most of these results were discovered on small models, recent work claims that the observed bound in error n² 7/6 exists for all models (Dey & Donoho, 2024). Further work (Schaeffer et al., 2025) summarizes current research on collapse.
Transfer learning and other model interactions. Significant prior work has studied transfer learning, in which information learned by one model passed to another, often by reusing the trained weights of a "teacher" model to initialize a "student" model (Zhuang et al.. 2020). Some prior work has further considered the use of synthetic data in transfer learning (Tian Shen, 2025; Kim et al., 2022: Brinner et al., 2025). Our work is distinct from transfer learning due to its focus unintentional mata-mediated interactions between models. Furthermore, limited work has examined long-term effects of models training on each other's data. Zhang et al. (2024) consider the setting where generative model is trained on data generated by other models, but does not consider long-term effects of such interactions among multiple models. Jain & Krishnamurthy (2025) study interacting Large Language Model agents through the lens of Bayesian social learning and microeconomics, but do not focus specifically on mediated interactions between models.
2

---

Published as a conference paper at ICLR 2026
Table 1: Evidence from LLama, GPT, and Phi suggests reuse of old training and collection of additional data to train new model generations. Datasets reused across models and generations are highlighted. We start with Phi 1.5, the first version of Phi designed for general NLP tasks. Phi I was designed for coding tasks.
Model
v2
v3
v4
Llama
Asxi Rooks. Common Crmw C4, Wikipedin, StackFcchnge
(Touxron al., 2025a):
Coonoron al., 2023b); "A new publicly available online data."
"A variety data sources containing knowledge unril and 20233"
(Dubyy al., 2024);
licensed information from
(AL, 2025);
publicly available.
Meta's is products and services."
GPT
(Radffond et al., 2018):
Booku
(Radford al., 2019| WebTexk
CommonCrawl... WebText2 Books, Books Wikipedia
(Brown et al., 2020)
(Achium et al., 2023: No info poovied
[Abdin et al., 2023):
(Li 2023);
The Stack,, |Stnck Overtlow
(Ahdin al., 2024y: 'publicly manilable web datn...
Phi
The Stock.. Stack Overflow synthetic "textbock" data
synthetic "textbook" data. filtere Commmon Crawl
and synthetc LLM generased data'
N/A
3 HOW TODAY'S S LARGE-SCALE GENERATIVE AI MODELS ARE TRAINED
We first establish why we believe that data Mata-mediated interactions between models e.g. instances of models training on each other's generated outputs-arr realistic and worthy of study. To do this, we comb through academic literature and whitepapers describing today's ' large-scale genAl models to understand how models are trained, what data they are trained on, and how data is collected and used for model updates. This sets the stage for the formalization and experiments in the rest of the paper.
We find that most of today's models follow a 3 step update process. First, models are pretrained on a large corpus of data; then they are fine-tuned to teach specific behaviors and/or to align them with human preferences. Finally, they are later updated, either to teach new behaviors or update knowledge. As we describe these steps in detail below, we highlight specific realities or assumptions that have been largely overlooked or not made explicit by prior work.
Step 1: Pretraining. Following well-established scaling laws linking model performance and dataset size (Kaplan al., 2020), large-scale generative AI models trained massive, internet-scrape datasets. Early versions of GPT, Llama, and PaLM all report being trained on scraped datasets like Common Crawl, ArXiv, Github, Wikipedia, and/or Stack Exchange (Touvron al., 2023a; Chowdhery et al., 2023; Brown et al., -ee Table 6 in Appendix for an overview.
Another striking fact emerges from the categorization of training data in Table 6: large-scale model training datasets overlap. For example, GPT. Jamba, Llama, PaLM, and Phi are all trained on subsets of CommonCrawl (Crawl, 2025), while GPT, Llama, and PaLM are all trained on Wikipedia and Books datasets. Several other models have other points of training data overlap.
Step 2: Fine-tuming. Variously called fine-tuning or alignment, this phase leverages proprietary methods or data to tweak model behaviors in ways model providers believe are helpful. For example, the LLama fine-tuning phase (Dubey et al., 2024) involves many rounds of reinforcement learning with human feedback (RLHF) to stamp out model negative behaviors, while Phi (Abdin et al., 2024) was fine-tuned proprietary synthetic data to patch "gaps" in its mathematical reasoning abilities.
Step 3: Model updates. A key assumption of prior literature on model collapse is that models are updated by training on a mix of fresh and re-used data. The "replace" update scenario Shumailoy al. (2024) assumes model trainers train the next generation using only generated data outputted by the prior version of the model, an interesting but impractical setting Schaeffer et al. (2025). The "accumulate" update scenario (Gerstgrasser al., 2024) assumes model trainers augment their original data at each update step with additional data that may contain Al-generated content. Finally, the "aceumulate and subsample" update scenario (Kazdan et al., 2025) subsamples a fixed-size subset of original and accumulated data for each update, acknowledging real-world compute limits.
We believe that the "accumulate-ane cawpuuulaatt-an--sutut and-subsamplee" update paradigm best reflects reality and so we leverage it in our work. We support this opinion with evidence from three well-documented model families: Llama, GPT, and Phi. Table records the training data used in publicly disclosed generations of these models. As the table shows, trainers re-use some prior training data for model updates, supplementing this with additional web content. Whitepapers for models published after 2023 generally omit training data information but suggest collection of new online data for updates.
3

---

Published as a conference paper ICLR 2026
Initial training data
Updase new scraped content
DEs
D.
D
%
Da
D12
Internet
D.
D
D
17
D1
1- B
of proprattary dets
Proportion ofreww in Lit data in updlet dattaec
Figure 1: Our dataset update scheme, parameterized by and 3. This paradigm best aligns with evidence from the literature given in $3 and strongly indicates that interactions between models, facilitated by training on others' generated dota, are an important consideration for empirical and theoretical work on model evolution.
Despite these realities, most prior work on model collapse still overlooks a fundamental reality in model updates: future models trained on internet-souree content will be trained on oulputs from other generative Al models, not merely their own. Already, the internet is filled with generated content from various models (Sun et al., 2025). As model trainers collect new data to facilitate model updates, internet-sourced data will inevitably contain content from other generative models, so:
At each update step, models may be trained on their own and on other models' generated outputs.
FORMALIZING AN ITERATIVE, INTERACTIVE MODEL TRAINING PIPELINE
Section provides empirical evidence for two realities overlooked by prior studies: internet-scraped training datasets used for initial training may have substantial overlap, and models may be updated using each others' generated outputs. To study the effect of these two factors on model evolution, we propose a general workflow in which multiple entities regularly update their models using mix of private, public, and generated data. Based on $3, we consider three types of training/update data:
D.: Public data used during initial training/updates by multiple entities (real data only).
D&: Private data used only by entity l for initial training/updates (real data only).
D, (Dr1, Di2,... Da Public data used for updates time / by multiple entities (synthetic data). Dik is data generated by the hthh entity based on model 0
Mapping these realistic scenarios, D. could be a public dataset like Common Crawl; Dy could be a private dataset of math problems curated by entity k; and D, could be an internet scrape from after initial model training. We weight the relative impact of these data types by the ratios o, 3.
3, 0 < 3 S 1, is relative size of the initial public data set D,, compared to the initial private data DR. This fraction remains constant if/when initial data is reused for updates.
a, 0 S
is the fraction of new data introduced at generation i, relative to the amount of initial data reused (following the "accumulate and subsample" paradigm of Kazdan et al. (2025)).
Interactive training workflow. We consider K entities, each seeking to train or update its own generative AI model. In the initial phase of training, denoted by time f. = 0, each entity k trains its model based on a combination of a publicly available dataset D, as well its own private dataset Dr. The trained model is represented generally by parameter be,.. i.c., 0. = & (D., Dz). k 1,.... K. where each o represents generic training algorithm. For model updates stages t t> 0, model parameters are updated via:
1. New public data D, is generated uniformly at random using the most recent version of the models. Specifically. the data are sampled i.i.d. according to the mixture * Σ PR.
where Pk.0 denotes the generative model used by k-th entity.
2. This data is placed online and collected by entities as training data for the next model update.
3. Each entity composes its training data for the next update, using mix of the initial dataset
(D., D&) and newly collected data D.. Contributions from each dataset are weighted by a, B.

---

Published as a conference paper at ICLR 2026
4. Each entity ho = 1,... K updates it model parameters via 8r, - -0,,((0-11.. D., Dn. D.). Here, & a training algorithm that depends on the previous model parameter 0x, as well as the data. As before, training may employ subsampling, weighting, and randomization.
In this workflow, entities interact through the release of publicly available synthetic data produced by prior generations of other entities' models. Thus, even though initial private training data are never shared, it could end up positively impacting other entities' models. This potential benefit of synthetic data sharing appears only in this interaction paradigm and has not been recognized in prior work.
THEORY
We theoretically analyze the behavior of the interactive workflow. Similar to prior work (Gerstgrasser et al., 2024; Kazdan al., 2025; Dey & Donoho, 2024; Dohmatob et al., 2025; Barzilai & Shamir, 2025), we focus on the linear regression models where each data point consists of a feature-response pair (z: y) R x R. By the universality results of Dey & Donoho (2024), the analysis of this settings also applies to generalized linear models satisfying appropriate asymptotic normality assumptions.
Notation. For a X q matrix A, we use A to denote the Moore-Penrowr pseudoinverse and vec(.A) denote the pq vector obtained by stacking the columns. @ denotes the Kronecker product. For a E [0, 1] we set ā 1 a and f
B.
Training Workflow. We follow the training pipeline outlined in Section 4 in which K different models are trained on a mixture of private, public, and generated data. At initialization, each entity fh, E [K] combines its private data Dr = (#xi, M= with public data D. (Iess yos) to produce an estimate 0Ro by minimizing the empirical loss
E(an)e BoL(x, y. 0) E(..)) Lo 0)
where L(x, y, = (y x placed on the private data. Training then proceeds for generation stages t = 1.2.3.... as follows:
x 0)² is the squared error loss and 0 S 8o controls the relative weight
1. Each entity & uses its most recent parameter estimate 0 to generate data Di: (Iikss Mt according the Gaussian model y N(x 0,-1, 0²). The entire collection of generated samples is combined into single public data set D, = 1
2. Each entity l produces a new estimate dy by minimizing the empirical loss
El a, v. + E(EyieD. (1, + +E (xx,
with weights 0 < az Bt I 1.
We note that our framework could easily be extended to accommodate new, human- gemaangenerated data at each time step. but we omit this in our formulation for analytic simplicity. Throughout our analysis we assume that all features are deterministic. We represent dataset D₂ with ñn d matrix Xx = [ăkı...., "eeng. 1 and ñe x 1 vector OA [ikI.... yesee" and use the same convention for the public data (X.,94 y=) and the generated data (X ).x). Data across different entities are then combined into "lifted" representations, which are denoted using boldface:
91
Xa
Vez
X
yo
X,
31
BiKk
We note that information about which entity produced which sample is required for the analysis, but is not used during the training, where all data from the same generation are treated interchangeably.
Bias- Variance Decomposition. We derive exact formulas for the mean and variance of the estimators at each stage of the workflow. Given the features (X, X., X,) and learning weights (ax: 3,) define
Š diag( S Sa)= ==)= xT
S. = - X.
S. diag( (e1, ..., S.h) = x X
G; = 5, + ā B, @ Sz)) + ax @ S,)
S. = k Σk Su
P. = G. 13.5 B1x1x @ S.)
Q: = G ITS,
5

---

Published as a conference paper at ICLR 2026
where II = k (1aXK & Ia) is orthogonal projection matrix and cgg 0.
Theorem Conditional on the initial data Do (D ................................ DR, D+), the estimates 0, vec(O.1..... Ock) are Gaussian with mean and variance
X [
EiO, Do] - M.
Cov(O, Do) = Ct.
where the matrices M, and C₁ are defined recursively with Mo Po and Co OKdxKa and C==Q( C.-I)Q 121.
M P. Q.M--:
Theorem shows that the conditional mean of each estimate 0.. is a linear combination of the individual ordinary least squares (OLS) estimates X .......... X x UK and X," y, for the private data and public data, respectively. For each generation i, similarity across entities can be assessed by comparing the of the K' (K 1) block partitioning of M, At initialization, the off-diagonal blocks for the private data are zeroed out, but in later stages, these blocks become nonzero thereby allowing private data be shared across entities. Homogenization (i.e., shrinkage towards global consensus) occurs when row blocks are identical, and thus each entity has the same mean.
For our next result we mimic the experimental setup in Section and assume that the initial data are generated from Gaussian model with common ground truth parameter and the heterogeneity across datasets arises from the differences in the features, i.e., the matrices SI....,k.
Theorem 2. Suppose that the initial data are generated independently according the model x N(x 0, (g)) where 0 E Rdd is fixed parameter. If J....... G2 are full rank then
E = -I- Q: ....--QII----. Q G.G) & @8), Cov(O,) = Mi S
0
M +C, St
To help interpret this result, observe that Go is full rank. then each initial estimate is unbiased, and unbiasedness persists throughout every stage of training. Conversely, if Go is rank deficient, then at least one (and possibly all) of the initial estimates is biased. Remarkably, Theorem shows that may still be possible for all entities have vanishing bias, provided that Q ..... Q, converges zero. Specific conditions under which this occurs are considered in the next section.
Asymptotic Variance. To provide a finer analysis the training dynamics we now suppose that the weights and features satisfy a; a, 3g B, and S, S for t 2 1. Setting P = P: and Q Q1. the matrices M, and C2 defined in Theorem can be expressed explicitly as
M - = (E C. C= E
Classical results matrix analysis (Higham, 2002) imply that if the spectral radius of Q strictly less than one, then these matrices converge to well-defined limits M and C' satisfying
M = (T- vec(C) o²
Q vec ves Q)
The following result provides sufficient condition for convergence in terms of the triple ($, S). In particular, if S is proportional S then the condition is satisfied for all 0 S < 1 and 0 < 3 1. Note that the boundary case a = corresponds to the recursive training setting of Shumailov et al. (2024) where the variance increases linearly across generations, and thus convergence does occur.
Lemma 1. Suppose that S X$ + (1 () @ S.) for some 0 X 1. Then, the spectral radius of Q is strictly less than one for all 0 < < and < 3 X.
We summarize findings with the following characterization of the asymptotic variance:

---

Published as a conference paper at ICLR 2026
= 0.1
B == 0.5
B 0.9
Relative Efficiency
Relative Efficiency
Emficeeccy
3.-
Model
Model
04-
0.4
Model
02-
0.-
0.3
0.0
0.0
0.4
3.5
Figure 2: Predicted relative efficiency across a, values for a K = model system (dimension 15 and rank 5). Curves show rafio of MSE of the minimum variance unbiased estimator to asymptotic MSE for given
a, 8 setting obtained from Theorem 3 optimal). Setting B 0.5 produces best results across models.
Theorem 3. Consider the setting of Theorem 2 and suppose that Ot, a, Br 33 and S. S for 1. IfG G2 has full rank and Q has spectral radius strictly less than one, then
[ - 1x 00,
Cov(O.) t o² o2M
S+
0 S
M +C
where M and C are given by (2).
MSE and relative efficiency. The expression for the mean and variance in Theorems and 3 provide explicit formulas for the mean squared error (MSE) Bi] 01]3] of entity & at each generation and the mean squared prediction error (MSPE) X[]]Xm (Oex- [?] for entity m's private feature matrix.
We can use this compute the optimal a, f values for the interactive training setting. Figure compares the asymptotic MSE for training workflow with given 3 (from Theorem 3) with the MSE an idealized setting where each entity has access to the entire collection of real data (both private and public). Each curve represents the relative efficiency, i.e., the ratio of optimal MSE to entity-specific workflow MSE, with values close to one indicating near optimality. These results demonstrate that setting with 3 0.5 and 0.5 achieves the best global performance for all models. When 33 is much larger (0.9), relatively small a values also improve model performance.
6 EXPERIMENTAL EVALUATION
To understand how theoretical predictions bear out in practice, generation models. In each, we train interacting models (per our framework in Figure 1) for several generations and evaluate how their performance changes
experiments text-
their and other models' tasks. Here, we present the setup and results for a K = 2 and K = 3 interacting at 3 = 0.5 model system. See Appendix for full K results various beta values.
Table 2: Change in loss behavior for K 2 interacting models at 0.5. We show results as initial + final prediction loss values for models on their own and the other models' tasks on their own and the other models' tasks over T generations. For clarity, we colorize loss increase decrease and constancy
(A < 0.1). (a) OPT models (T=15)
Liama 3.2 (IB) models (T=15)
(c) Llama 3.2 (3B) models (T=8)
Medel Tank
33-+33
33-43
B3
or Tak
Model
28-29
28
88- 50
Model or Tark
24 25
24
Madel
18-7777 2
18 1.7
or Task2
Mide1
122
1:2
11:
Model or Task
1.004
10001:000
Misel
losk
31
33
220 or Task2
Midel
23
19.0
Midel or Task
Taok
Experiment setup. Training large language models from scratch computationally infeasible for us, so we simulate the initial setup of language models trained on dataset (D., (D,,Dx) at time t by fine-tuning K instances
given pre-trained model architecture on carefully chosen D., Dx- (Zhang al., 2022), Llama
We experiment with two language model urchitecture--PPIII 3.2 IB (Dubey et al., 2024), and Llama 3.2 3B (Dubey et al., 2024) assess how interactive training scales. Public sources (Touvron et al., 2023a; Zhang al., 2022) state that both models were pretrained on BookCorpus (Zhu et al., 2015), CC-Stories (Anderson, 2022), the English portion of CommonCrawl, and public Reddit data. We approximate D. with BookCorpus due to practical constraints. Each model is given its own initial task-specific dataset Dr SSiiQ (Johannes Welbi,
7

---

Published as a conference paper at ICLR 2026
all a, generation
a 0.O, generation 15
a- 0.5, generation 15
e- 1.0, generation 15
Dataset
Pr
D+
Figure 3: PCA of embeddings of outputs produced by models 0₁1 and 0.2 on datasets Di, D2, and D.. Results for K = 2 Llama models trained with 8 = 0.5 and varying a. Leftmost plot shows embeddings at generation = 0, which are identical for all a; while right plots show embeddingg at = 15 with different a.
Table 3: Cosine similarity of embedded output representations for K = Llama models at 3 0.5. We show initial (t=0) final (t=15) for Dj D2, and D... We colorize increase and decrease
a
a 0.5
a
D
0.73 0.70
0.73
0.88
0.73 0.86
D2
0.88 - 0.79
0.88
0.93
0.88 + 0.94
D.
0.50 + 0.59
0.50
0.64
0.50 + 0.58
2017) (science questions) for the k = 0 model; and OpenAI's GSM8K (Cobbe et al., 2021) (grade school level math problems) for the k model. When simulating K 3, we assign the third model the A12 ARC dataset (reasoning questions) (Clark et al., 2018).
Interactive training proceeds outlined in Figure 1, with fixed o, values for each experiment. After training a new model generation Oz,, we use O.p produce synthetic data D++1,8 & that becomes part of the next generation's 's training data (if a > 0). We produce D++1,8 by randomly sampling prompts from Dr for each of the K models and prompting 6.. to complete the text.
Training and evaluation. We run experiments on 2 model systems with (0,0.5, 1) and 3 {0, 0.5, 1}. each for 7" = 15 generations of training. At each training generation, models are fine-tuned on datasets of fixed size n 12, 500 drawn i.i.d. from the datasets Dr. D., D; with weights a,, a;3, and a/ K, respectively. This mimics the accumulate and subsample setup of Kazdan et al. (2025) with the additional wrinkle of data-mediated model interactions.
We train each model for 100 steps per generation on single NVIDIA H200 GPU using mixed- precision, the AdamW optimizer with a learning rate of 8 warmup ratio of 0.025, and gradient accumulation over 2 steps. A table with training hyperparameters is the Appendix. After training record token-wise average cross-entropy loss by feeding each model prompts
each generation, from each test set of private data Dr and evaluating semantic "distance" between predicted and correct answer. We also compute embedded representations of models' completions of the first 200 elements each of Di and D., to see how various a, B affect models' representational spaces. Embeddings computed via the outputs with similar embeddings (measured via cosine similarity), their feature spaces are more aligned. These two metrics allow us to evaluate how data-mediated interactions affect (1) models' performance on their own and other models' tasks and (2) model homogeneity.
rmers python library. If models produce
As predicted in Figure 2, an 0.5, B 0.5 setting produces optimal results in terms of model performance across tasks. Table 2 reports the change in models' loss values between the first and last training generation for the 3 = 0.5 setting. When used for each training update, resulting in stable or slightly worse performance on different tasks, as models are forced into a local minimum. When outputs. They degrade on their original task due to the lack of real data but improve slightly on the other model's 's task. However, at other model's task. Table 4 shows similar results when we scale up = - K 3, introducing more diversity. Results for other B and K 3 are in Appendix and echo findings here.
Results.
0, the same (human-generated) data
1.0, models are only trained on generated
0.5, models perform well on their own tasks and improve the
While some amount of mixing improves model performance on previously unseen tasks, homoge- nization occurs for D. at all a and for D when a > 0. These are the settings under which models share information, either via common dataset D. or interactive training when 0. Figure 3 shows
8

---

Published as a conference paper at ICLR 2026
the benefit of interactive training when 3 are reasonable, visualizing PCA-reduced embeddings from D1, D2. and D_ for 3 = 0.5 and varying as training progresses For 0.5, the PCA shows clearly separated task clusters, indicating that both models are better tailored to the individual tasks. In contrast, the lack of well-defined clusters when * 0.5 suggests reduced sensitivity task type, e.g. generic answers. Yet, in Table 3, we see that when a = O, models homogenize slightly on the shared task D. but diverge on model-specific tasks. This makes sense since models the models do not train on each other's tasks. Once a > 0, homogenization increases for all datasets/tasks.
Table 4: Change in loss behavior for K 3 interacting models at B 0.5. We show results initial + final prediction loss values for models their and the other models' lasks over generations. For
clarity, we colorize loss increase decrease and constancy (A < 0.1).
(a) OPT models (T=15)
(b) Laana 3.2 (1B) models (T=8)
==0
0.5
1.0
0
0.5
1.0
Model
Model
on Task
3.3 3.3
3.3
3.3 3.3
-5
on Task
2.6 2.7
2.6 - 2.7
2.6 - 2.8
Model 2 on Task 2
Model Tisk 2
18
1.1 12
Model 3 3
2.8 2.9
2.8 2.9
3.1
Model 3
2.3
2.3 2.6
Task
Model on Task 2
Model
2.2
on Task 2
2.0
Model 2
Model
3.9
Model 3 Task
Model 3
3.7
3.7 3.4
>.3.6
Task
3.1
3.1 3.0
Model
Model
on Task
3.3 3.0
on Task
2.9
Model 2
Model Task
3.8
Model
Model
Task 2
2.9
2.9
1.9
2.9
2.2
Task 2
2.0
2.5
2.0
1.3
2.0 1.4
7 DISCUSSION
Limitations. Our work has several limitations. First, our theory considers only linear models, which, while common in the model collapse literature, may not capture nuances present in larger models. Second, we run experiments on LLMs in controlled settings, the dynamics of which may differ from real-world LLMs. Third, arguments against the increasing presence of generated outputs in training datasets (e.g. (Drayson et al., 2025)) are in Appendix B. Also, our theoretical framework assumes that new data in model updates is purely synthetic. In reality, if internet scrapes are used to create model update datasets, they will contain both synthetic and real data. Finally, we assume that model trainers use new scraped data for each model update but only reuse data from initial training. This assumption may limit the range of outcomes. Although our framework can accommodate an arbitrary number of interacting entities, exploring configurations with large and T was computationally infeasible, as each setting requires repeated fine-tuning and evaluation of multiple large models. Therefore, restricted our experiments to smaller practically achievable scales that preserve interpretability.
Broader Impacts. If data-mediated interactions homogenize generative models, causing them to coalesce on certain viewpoints, this could lead to pervasive bias in Al-generated content. Peterson (2025) discusses this possibility, while Wenger & Kenett (2025) showed homogeneity across creative outputs from many LLMs, suggesting these homogenization effects may already be felt. Much future study is needed to evaluate the extent to which mediatedd interactions fuel homogeneity.
Conclusions and Future Work. We provide first look at possible outcomes of genAl models trained on each others' data and find mixed effects. Training on other models' data exposes models to concepts possibly missed in their work could consider additional nuances of interactions between models, explore how these interactions evolve in other modalities like image generation, and investigate whether fixed points (e.g. like the universal T² 7/6 pathway of (Dey & Donoho, 2024)) exist under this paradigm.
training data, but can homogenize model behaviors. Future

---

Published as a conference paper at ICLR 2026
REFERENCES
How Generative Al Changing Creative Work. Harvard Business Review, 2022. https://htt /hbr.
org/2022/11/bmrr htwp//owwwwwetteeeeeooeee/ee generative aeee-ai-iss changing-crretiieeee work.
Command and command plus model card, 2024. https: http:///oooccccoee.eeeeeeoe
responsible-us usse..
Marah Abdin, Jyoti Aneja, Sebastien Bubec, Caio César, Teodoro Mendes, Phi-2: The surprising power of small language mod-
Weizhu Chen, al.
https: httpp::/www.nnne http///wwww..ooottto///////////t cwm/en-uuueeeeoccooooooomoo research/blog/ httpr//ooo../ooo////o//////e/raa phi-2-the-surpriatt ppl.errteeeeapppiieeeeeeeerrraar
els. 2023.
Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404. /4219, 2024.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303..m.... 08774, 2023.
Meta AI. The Future of AI: Built with LLama, 2024a. https: //ai:://ai.. www.. wtw.:/amta....m//opooae/aao Future-of-ai-bui wtw---awwiiwwwwwh--------------- alama/
Meta AL Llama 4 model card, 2025. https://github. md.
1lama-model loddelssbllbbl ht///mainnn /ttm/maannmmmmm/////////////// s //tss/lammmm/ 11ama4 /MODEL_CARD 1
Open AI
source of what we see putpss:/openarl com/ index/ understanding- the source-or waff-whtt-ae we see-and hear
Understanding
the
and hear online, 2024b.
Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel Leleune, Ali Siahkoohi, and Richard G Baraniuk. Self-consuming generative models go mad. Proc. of ICLR, 2024.
Mayssa Ahmad Ali Elfa and Mina Eshaq Tawfilis Dawood. Using artificial intelligence for enhancing human creativity. Journal of Art, Design and Music, 2(2):3, 2023.
Benjamin Anderson. Cc-stories, 2022.
https: stories Settorieesparuuet//ee pwwguet/commitss/eettteetteoeeee /main..
andersonbodefg/eee ct/odeetrceosssiessess
Anthropic. Dataset card for hh-rlhff https: http:s///hhginnggaa...eeeeeeeeee hh-rihf.
Anthropic. Model card and evaluations for claude models, 2023. https://
www- www-cdn.cr anthropic. com/ ott.:./.......eeec0ccaaaaaaaaaa. Model-CarddClaudee pd...
Daniel Barzilai and Ohad Shamir. When models don't collapse: On the consistency of iterative mle. arXiv preprint arXiv:2505. 2025.
Mare Brinner, Tarek Al Mustafa, and Sina ZarrieB. Enhancing domain-specific encoder models with llm-generated data: How to leverage ontologies, and how to do without them. arXiv preprint arXiv:2503.22000 2025.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, et al. Language models few-shot learners. Proc. of NeurlPS, 2020.
Aakanksha Chowdhery, Sharan Narang. Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 2023.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv: 1803. 05557v1. 2018
10

---

Published as a conference paper at ICLR 2026
Nick Clegg. Labeling Al-Generated Images
Facebook, Instagram and fb wtt.:/cc//wwwwwoemmmm oagereattae---eesee------oo----e
Threads. Meta Al Blog, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
Common Crawl. Common Crawl Open Repository of Web Crawl Data., 2025. https:// commoncrawl.cm org//
Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, et al. Scalable water- marking for identifying large language model outputs. Nature, 634(8035), 2024.
Apratim Dey and David Donoho. Universality of the x² pathway in avoiding model collapse. arXiv preprint 2rXiv:2410.22812, 2024.
Elvis Dohmatob, Yunzhen Feng, and Julia Kempe. Model collapse demystified: The case of regression. Proc. of NeurlPS, 2025.
George Drayson, Emine Yilmaz, and Vasileios Lampos. Machine Machine-generatee generated text detection prevents language model collapse. arXiv preprint arXiv:2502. 15654, 2025.
Nan Du, Yamping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, et al. Glam: Efficient scaling of language models with mixture-of-exppers, In Proc. of ICMI,, 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv preprini arXiv:2407.2 21177/17733 1783, 2024.
Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, and Julia Kempe. Beyond model col- lapse: Scaling up with synthesized data requires reinforcement. In ICML Workshop on Theoretical Foundations of Foundation Models, 2024.
US GAO. Science and tech spotlight generative ai in health care, 2024. GAO-24-107634, https: gao.ggooooooooooo gov http///rouctsss gao- 100--4000000---
//www. www.wwaw....o......o.oo.........
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, et al. Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data. Proc. of COLM, 2024.
Kunal Handa, Alex Tamkin, Miles McCain, Saffron Huang, Esin Durmus, Sarah Heck, Jared Mueller, Jerry Hong, Stuart Ritchie, Tim Belonax, al. Which Economic Tasks Performed with AI? Evidence from Millions of Claude Conversations. arXiv preprint arXiv:2503.0470 2025.
Emily Harding. 2024 Priorities for the Intelligence Community. gic and International Studies, 2024. https://www.co. www.w.c.s.....o http://wwwccissogg//aaaaaaaaaeee org.s/als/anaass/////oooa 2024-pri 1924-prioritiss www.ennneeennniee community--0
Center for Strate-
Ryuichiro Hataya, Han Bao, and Hiromi Arai. Will large-scale generative models corrupt future datasets? In Proc. of ICCV, 2023.
Nicholas Higham. Accuracy and stability of numerical algorithms. SIAM, 2002.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. Proc. of NeurtPS, 2022.
Adit Jain and Vikram Krishnamurthy. Interacting large language model agents. bayesian social learning based interpretable models. JEEE Access, 2025.
AQ Jiang, A Sablayrolles, A Mensch, C Bamford. DS Chaplot, D de las Casas, F Bressand, G Lengyel, G Lample, I. Saulnier, al. Mistral 7b (2023). arXiv preprint arXiv:23 10.06825, 2023..
11

---

Published as a conference paper at ICLR 2026
Matt Gardner Johannes Welbi, Nelson F, Liu. Crowdsourcing multiple choice science questions. 2017.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08360 2020.
Joshua Kazdan, Rylan Schaeffer, Apratim Dey, Matthias Gerstgrasser, Rafael Rafailov, David L Donoho, and Sanmi Koyejo. Collapse or thrive? perils and promises of synthetic data self-generating world. Proc. of ICML, 2025.
Yo-whan Kim, Samarth Mishra, Sou Young Jin, Rameswar Panda, Hilde Kuehne, Leonid Karlinsky, Venkatesh Saligrama, Kate Saenko, Aude Oliva, and Rogerio Feris. How transferable are video representations based on synthetic data? Proc. of NeurIPS, 2022.
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need ii: phi-1.5 technical report. arXiv preprint #rXiv:2309005663, 2023.
Shayne Longpre, Robert Mahari, Naana Obeng-Marnu, William Brannon, Tobin South, Jad Kabbara, and Sandy Pentland. Data authenticity, consent, and provenance for ai are all broken: What will take to fix them? 2024.
Matteo Marchi. Stefano Soatto, Pratik Chaudhari, and Paulo Tabuada. Heat death of generative models in closed-loop learning. Proc. of IEEE 63rd Conference on Decision and Control (CDC), 2024.
Gonzalo Martínez, Lauren Watson, Pedro Reviriego, José Alberto Hernández, Marc Juarez, and Rik Sarkar. Towards understanding the interplay of generative artificial intelligence and the internet. In International Workshop on Epistemic Uncertainty in Artificial Intelligence. Springer, 2023.
Hana Matatov, Marianne Aubin Le Quéré, Ofra Amir, and Mor Naaman. Examining the prevalence and dynamies of ai-generated media in art subreddits. arXiv preprint arXiv:2410.07300, 2024.
Andrew J Peterson. Ai and the problem of knowledge collapse. Al & SOCIETY, pp. 1-21, 2025.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training 2018.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Sandeep Reddy. Generative ai in healthcare: an implementation science informed translational path on application, integration and governance. Implementation Science, 19(1):27, 2024.
Reuters. OpenAl says ChatGPT's weekly users have grown to 200 million, 2024.
https://www.t http///wwwwtt..see////.//eeeeest www http:///ww........ee.ooooooaaaaa wewp wwwoi-rooow------- 200p:mm-0000mm11mm 200-:onn------200 08-29/.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi.
Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156, 2023.
Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, and Sanmi Koyejo. Position: Model collapse does not mean what you think. arXiv preprint arXivv2503.03150. 2025.
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. Ai models collapse when trained on recursively generated data. Nature, 631(8)22)7555555, 2024.
Zhen Sun, Zongmin Zhang, Xinyue Shen, Ziyi Zhang, Yule Liu, Michael Backes, Yang Zhang, and Xinlei He. Are We in the Al-Gienerated Text World Already? Quantifying and Monitoring AIGT on Social Media. Proc. of ACI., 2025.
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang. et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint 2rXiv:2403.05530 2024a.
12

---

Published as a conference paper at ICLR 2026
Jamba Team, Barak Lenz, Alan Arazi, Amir Bergman, Avshalom Manevich, Barak Peleg, Ben Aviram, Chen Almagor, Clara Fridman, Dan Padnos, et al. Jamba-1.5: Hybrid transformer-mablee models at scale. arXiv preprint 22t2:2200..2250000 2024b.
Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze -ze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint 1rXiv:2201.08233 2022.
Xinyu Tian and Xiaotong Shen. Generative distribution prediction: A unified approach to multimodal learning, arXiv preprint arXiv:2502.07090, 2025.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 230.1 2023a,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307. 09288, 2023b.
Ze Wang, Zekun Wu, Jeremy Zhang, Navya Jain, Xin Guan, and Adriano Koshiyama. Bias am- plification: Language models as increasingly biased media. arXiv preprint arXiv:2410.15234, 2024.
Emily Wenger and Yoed Kenett. We're different, we're the same: Creative homogeneity across Ilms. arXiv preprint 19X1, 2025. https: http:///wwxvvv..o/a//////// 1361.
XAI. Grok Beta- The Age of Reasoning Agents, 2025.
Hanlin Zhang, Benjamin Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. arXiv preprint arXiv:2311.04377 2023.
Jinghui Zhang, Dandan Qiao, Mochen Yang, and Qiang Wei. Regurgitative training: The value of real data in training large language models. arXiv preprint orXiv:2407.128833 2024.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li. Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022.
Yukun Zhu, Ryan Kiros, Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference Computer Vision (ICCV), pp. 19-27, Santiago, Chile, 2015.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43-76. 2020.
Edward Zitron. There is No Al Revolution, 2025. wheres- the monee/.
13

---

Published as a conference paper at ICLR 2026
Appendix
A EXACT TRAINING DATA STATEMENTS FROM LLM PAPERS
Table 5 lists statements made about model training and fine-tuning data for large-scale generative Al models that do not explicitly list training data sources.
Model
Pre-training data
Fine-tuning data
models are on proprictary of publicly available information from the Internet. datasets that license from third party besinesses, data that
Clande (Anthropic 2023)
Publicly released HuggingFace (Amihropic).
afliruttively
that crowd workers provide.
GPT4+ (Achiam 2023)
No information provided
No information provided
Grok (xAI, 2025)
No information provided
information provided
Jamba (Team 2024b)
Our pre training dataset is a mixture of
publicly available web documents, codk, books and scientilic articles.
When performing supervised fine-tuning, we make heavy synthctic data.
We create our datuset for language model pre training containing knowledge
We produce the aligned Llarta 3 models applying several the model with human feedhack
(Dubey et 2024)
variety data
post training, aligning
until the end 2023, Much the data we utilize obtained from the web
publicly available, licensed data and information from Meta's products and services. This includes publicly shared posts from Instagram and Facebook and people's 's interactions with Meta AL.
Lama 4 (AI, 2025)
No information provided.
Our training data of consists heavily litscred publicly available web data from various open internet
(Supervised fine tuning] loverages highly curated high-quality data across diverse domains, c.g. math, coding, reasoning. conversation, model identity safety.
Phi 3 (Abdin et 2024)
as syntheric I.I. M-geerrerd
Table 5: Exact wording of training and fine-tuning data discussion from whitepapers in which data sources are not explicitly listed.
Table 6: Examples of training data listed prominent language models. use explicitly stated; significant overlap expected (e.g. GLaM (Du et al., 2022) and PaLM (Chowdhery et al., 2023) are trained on Microsoft's 's internal re- re-creation of WebText (Brown et al., 2020)). We only include models for which training data sources explicitly stated, See Table in Appendix for information other prominent models.
Model
CommonCrawl
WehText
Github
Wikipedia
Books
ArXiv
StackExchange
News
Chinchilla (Hoffmaan al., 2022)
GLaM (Du al., 2022)
GPT (Radford et 2018)
GPT-2 (Radford et al., 2019)
GPT-3 (Brown et al., 2020)
LaMDA (Thoppilan et al., 2022)
Llama (Touvron al., 2023a)
PaLM (Chowdhery et al., 2023)
(Abdin al., 2023)
B COUNTERARGUMENTS
We argue that Al-generated content from variety of sources will be increasingly prevalent online, resulting in future genAl models being regularly trained on each other's outputs. We believe this vision of future data-mediated interactions between models is reasonable, based on evidence from academic literature and corporate reports. However, others may disagree with our argument. Here, we present possible counterarguments our view to catalyze future work and discussion.
Can't we use watermarks to filter Al-generated content from future internet-scraped datasets? Several companies have publicly stated that they watermark Al-generated content (Dathathri et al., 2024; AI, 2024b; Clegg, 2024), making this argument plausible. Furthermore, (Drayson et al., 2025) show that using watermark detection techniques can help avoid model collapse under certain circumstances. However, reliance on watermarking has two major issues. First, watermarks are difficult reliably detect and/or easily removed from generated content (Zhang et al., 2023; Longpre
14

---

Published as a conference paper at ICLR 2026
Table 7: Change in loss behavior for interacting LLama models various a, show results as initial + final prediction loss values for models on their own and the other models' tasks, a initial T = and final= 15 generations. For clarity, we colorize loss increase decrease and constancy 0.1).
B 0
7- 0.5
8
1.0
Model I
Task
2.8 1.5
2.8 3.2
2.8 3.1
2.8 2.9
2.8 2.9
2.8 3.0
2.8 2.8
2.8 2.8
2.8 + 3.2
Model 2
Task
1.2 2.0
1.2 1.9
1.2
1.2 13
1.2 19
1.2
1.2 1.2
1.2 18
Model on Task
2.9
2.0
2.0 1.7
2.0
2.0
2.0
2.0 2.6
2.0
2.0
Model
on Tusk
5.0
3.7 3.1
3.7 3.7
3.7
3.0
3.7 3.0
et al., 2024; Sadasivan et al., 2023). Second, detection via watermark requires sharing of watermark detection information, which is essentially theerthc problem that relies other companies' willingness to cooperate. Both issues make watermarks an unreliable mechanism to rid datasets of Al-generated content.
What if there's 's less Al-generated online content than we think? Some work suggests there may be less Al- Al-generated content online than previously postulated (Matatov et al., 2024). However, other works consistently point to an uptick in Al-generated content online (Sun et al., 2025). We believe the widespread adoption and use of generative AI models across industries (AI, 2024a; Handa et al., 2025; Reuters, 2024), particularly for use in content creation (gen, 2022), provides strong evidence that Al Al-generated content will become a regular part of online life. Further empirical work is needed to vet both claims.
What if one model provider dominates online content? Although numerous generative AI models available online, some evidence suggests that landscape. A market research firm estimated that in January 2025, Open Al's ChatGPT had 340 million monthly active users, Microsoft Copilot had 11 million, Google Gemini had 80 million, and Anthropic's Claude had 2 million (Zitron, 2025). If only one model/company dominates, our paradigm training on other models' data will no longer be relevant and collapses back to the single model setting of prior work, e.g. (Shumailov et al., 2024; Kazdan et al., 2025; Dey & Donoho, 2024). Currently, though, this market research suggests that there are several models used by millions of users, making our assumptions somewhat reasonable. Future research could analyze market trends in model use and adoption to determine realistic assumptions.
companies may dominate the AT
C ADDITIONAL RESULTS
Tables 7 and show full results for all tested 8, a settings for the K = 2 setting across both the Llama and OPT model architectures. As stated in the main paper body, the 3 0.5 and a 0.5 settings perform best. Figures 4, 5, and 6 show loss values at each generation for (1) predicted theoretical results, (2) OPT = 2 models, and (3) Llama K models. Finally, Figure shows results for K system of interacting OPT models.
D CODE FOR EXPERIMENTS
Code to generate the theory and experimental figures shown in the main paper body can be found at: https: / www..tt...hcooooooooo com.ub.ccomaaaa www./arruuaaeeco.cooooooossseaee www.uuusaaaauuuaaeaaaaa moakee//mtttttt. ltt.emmiieeeeeettiiiiooooo ion.
E PROOFS FOR RESULTS IN SECTION 5
Before diving into the proofs, we recall the workflow setup and provide some preliminary results. The initial data are represented by matrix-vector pairs for the private data (X -.)...,; ... (X Kx ūx)
15

---

Published as a conference paper at ICLR 2026
Table 8: Change loss behavior for K interacting OPT models at various a, We show results as initial + final prediction loss values for models on their own and the other models' tasks, a initial = and final 15 generations. clarity, we colorize loss increase decrease and constancy 0.1).
B
0.5
8
1.0
Model I
Task
3.3 1.8
3.3 3.6
3.3 3.5
3.3 3.3
3.3 3.
3.3 355
3.3
3.3 3.2
3.3 3.5
Model 2
Task
1.8 2.6
1.8 2.1
1.8
1.8
1.8 22
1.8
1.8 1.8
1.8 2.2
Model
on Task
3.7
3.1
2.2
3.1
3.1
Model on Tusk
5.6
5.1 3.6
5.1
3.5
5.1
3.5
Theoretical prediction error across generations for different a, B values, K=2
Mean Squared
Model 1ask
Model 2 on Task 1
Model on Task
Model Task
H-0.3
Prediction
7.5
15
Mean Squared
Predict Error
0.50
0.25
5
10
15
15
15
Moan Squared Prediction Error
Model Task
Model 2 on Task
Model 2 on Task 2
Model on Task
8-0.9
alphswo5
Model Riting Generation
Model Fitting Goneration
Model Fitting Generation
Model Goneration
Figure 4: Predicted behavior time for K 2 model system with varying o, We equations for MSE from theorem and run simulations K 2, dimension 50, rank 15.
and for the public data (X.,V.). At each generation I, the data (Xes, produced by entity k is given by
X + Wekk
Vik
where d1-1, Rdd is the most recent parameter estimate and wik o is Gaussian noise that is independent across entities and generations. To represent the notation compactly, we use the embedding
51
Xa
ye1
X
yo
X,
=
yt
w:
Xx
9K
Xxx
MiK
Linear dynamical system
According to the workflow, each parameter estimate obtained as the minimizer in Rd of the empirical loss:
K Myij
a. x20112 + - x-0
at
X1
K
where ap 0 and only the first two terms are present at initialization. The minimum norm solution given closed form by
k at
02 k
du = (ar + B.S.. +
Σ Sik)
a. v. +
K
K
j=1
j=1
16

---

Published as a conference paper at ICLR 2026
Evaluation loss across generations for different a and values (K=2) OPT- 350m
Model en Task
Model 2 on lask 1
Model on Task
Model 2 on task
B-0.0
F-035
0-110
Figure 5: Actual behavior time for interactions between OPT models (K = 2) with varying a,
Evaluation loss across generations for different a and values (K=2) Llama-3.2-18
Task
Task
Model Task
Model 2 on
p-0.5
5-1.0
10
20
Model Fiting Generatione
Model Conerations
Model Generations
Model Generatinn
Figure 6: Actual behavior over time for interactions between Llama models (K = 2) with varying 9.
where S. = X TTTTx. S. = X, Si XTx, and (-)+ denotes the Moore-Penrose pseu- doinverse. Stacking the estimates into a vector vec ,,0x) and using the identity A = A AAt, we can express all estimate updates simultaneously
0, G. (a.3,5 a,3,(1k @ S.)xtv++
(1K X Σ Sty
where 1x denotes the K x 1 vector of ones and G; is the block diagonal matrix given by
= 5, + S.) aI & S.),
S.
Sk
Defining the orthogonal projection matrix II = K 1 1 K & La we can write
1
Szy x = ISS.X y:- Σ
( &
K
j=1
Introducing the matrices P. = G. BB,
(1 8 S.)] and Q. = a, GSS, can express (3)
as
x ÿ X, y.
0, = P
+ Q.X,t
Using y: X,0,-1 w. and Q. X,
Qth we obtain
X
i = P
(4)
X y
17

---

Published as a conference paper at ICLR 2026
Evaluaton generations differett values
Ncc8
Padl Pickl Tac
Figure 7: Actual behavior over time for interactions between OPT models (K' = 3) with varying 0,
Parameter
Value
bfl6
True
data_seed
generation
dataloader num ____n_uuu_orrrrss
4
dataset_text_feet
input_ids
eval_on_start
True
eval_strategy
steps
eval_steps
100
gradient accumulation steps
2
mondient_checkpoingg
False
deglude_nun_inputtookeeee
True
learning_rate
8 x 10-6
logging_stee steps
5
Ir trscheedule__yye
constant_with_watt
max_seq_length
512
max_steps
100
num Len__tain__eeee train_eppohhs
1
optim
adamw_tod ttrch
per sor__dvice__dde _lle__ea__________eeeeeeeeeee_ batch size
16
per per_device_trail _triiee___ri___e sasti__atch_sirtee
2 (8 for T=8 experiments)
remove_ Lanove_unused_columm
False
seed
generation
warmup_ratio
0.025 no
save_ save_strategy
Table 9: Fine-tuning hyperparameterrs used.
This expression shows that the estimates evolve according to discrete-time linear dynamical system (also known as a Kalman filter model) with state variable θ,.
E. PROOF OF THEOREM
We now derive the distribution of Ô, conditional on the initial data Do given by (X, ū) and (X.: y.). Specifically, we show that the estimates are Gaussian with mean and variance
X ū x y.
E Do] M,
Cov(ê, Do) = C.
where the matrices M, and C, are defined recursively with Mo = Po and Co = Oa and
M. P. Q.M.
C. C St + Cr-1)Qr,
12 1.
The proof is by mathematical induction. For the base case = 0 invoke (4) along with Qo 0 to that O is deterministic function of the initial data with Mo Po and Co OKdx
18

---

Published as a conference paper at ICLR 2026
For the inductive case, assume that the stated distribution holds up generation t 1. From the definition of the workflow, (4) holds with tt N(0,,²1) independent of everything else. Thus 0 is Gaussian with mean
X X y/+ + +Q Dol = = Q.M,
E Dol P
X y++
M.
and covariance
Cov(@; Do) = Cov(Q, Do) Cov(Q,X) = Q.C,- + oo SQQSTQ
Ci
This concludes the proof of Theorem 1.
E.2 PROOF OF THEOREM 2
Under the assumptions of the theorem, we have that
N ([ {[
0
(Ix+1 0),
5
(5)
v
S.St
st
The goal for this proof is verify that if GI....., G. are full rank then
S
0
E[0,] (I - Q...- (I--:--Q-II-GG))))
@ 0), Cov(o,) = M2
M, + +c.
0
S
proceed by mathematical induction. Consider the case | = 0. By (4) along with Qo 0,
mean is
SS 0
[|Oo]] P.
(1x+1 2 GoG (1x (1x &0). S.S.
Likewise, recalling that Mo = Po. the variance is
(
x ) Mo = oM
St 0 St
Cov(Oo) = Mo Cov
M
Next, suppose that G G. are full rank and the stated distribution holds up to time - By Theorem we know that O, is Gaussian and so all that remains is to verify the given expressions for the mean and covariance. By the linearity of expectation and (4), the mean satisfies
E 0 = P
X
ý v.]
+ Q.E
E X,
(. 2 0) Qi(Ix & -.) ((I--GGTt(L. 2 0),
where the last follows from the inductive assumption applied to E(0,-1]. Moreover, from the definitions of P, and Qt, have
Pl @ 0) + Q @ 80)= = G (..3,5 5 + a, @ S.)) + a,ISS, (Ix 80)
= G @ 0)
=Ix 3
where the second line follows from the identity ITS, (1 Ta) (Ix ()) & Ia) and the last line holds because Ge is full rank. Combining the above displays gives the desired expression for the mean. The expression for the variance follows directly from (5) and Theorem
E.3 PROOF THEOREM?
If the spectral radius of Q is strictly less than one, then Q' + 0 as t + oc, and the Neumann series in (1) converge the well-defined limits (2). These limits can also be seen the (necessarily unique) solutions to the fixed point equations
M = P MQ.
C= + +C(C+o2st Q
where the expression for the covariance is known as the discrete time Lyapunov equation. Combining these convergence results with Theorem completes the proof.
19

---

Published as a conference paper at ICLR 2026
E.4 PROOF OF LEMMA
If a - 0 or if S = 0 then Q = 0 and so the stated result holds. Henceforth, we assume 0 < a < 1
and S is nonzero. Suppose that y = XS + (1 X) @ S.) for some 0 < 3 s A A and y > Then,
G a3S + aB(Ix & S.) a(Ix @ S)
aB (75 - (1 (1-x)(Ix 8 S.)) + a3(1x & S.) + a(Ix @ @
aBy 8 a ( A ー-8
(Ix & S.) aIxx S).
A
A
Hence,
aB7 ax
A A +
Q (ōS + IK @ A) IIS,,
A
ad
To proceed, observe that each diagonal block of S diag(S 1 -i...., Sk) lies in the span of K ΣR Sk, and thus S lies in the span of IK @ A. Accordingly, we can write
S6S + IK & A) (Ix & A A R(RR (Ix @ A+/2).
where (.)1/2 denote the symmetric positive semidefinite square root of a positive semidefinite and 4/2)). To bound the spectral radius, denoted by p(-), that fact R == (Ix @ ((Tx &
that the eigenvalues AB and BA are the same for any square matrices A and B along with that fact that k & A commutes with IT to write
p(Q) p (oS + 1x 2 A) is))
p (S(6S + I A) II)
p(( 8 R RR +1) (I )
= =
||TTR +1) -TT|
where || denotes the operator norm and the last equality holds because 1RR(RR + 1) -! II is symmetric positive semidefinite. Letting c > denote the smallest nonzero singular value of R, we have
(1 + (Q) < |RRI||
(4)
[[[[Tx + + @ A+/2
[[x @ *+ & A+/*)|
[[ & + 1xxk SI @ A+
(8)
1
1x X A SA+/2
K
1 1x
|||
+/2
SA+
A 1A+/25A+22
where (a) is the definition of R:; (b) follows from the commutativity of IT and Tx & A+/2; (c) follows
from IT.IIT 1 @ S: (d) is the mixed-product property of the Kronneker product; (e) is the
basic identity ||A B|| |||||]]] B| for any matrices A and B; and (f) holds because | 1x 1.
Finally, using that < 5 A, we see that || +/2 SA +/2 s A/2 - 12 < This verifies that pl Q) is strictly less than one.
20


Reviews:
[{"review_id": "MQ3x28akj3", "reviewer": "Reviewer_rQht", "summary": "The paper studies how generative language models (LLMs) may interact recursively through training data that include other models’ outputs.\n\n1. It introduces a formal framework with two parameters — the synthetic data ratio ($\\alpha$) and the initial data weight ($\\beta$) — to describe cross-model data mixing.\n\n2. Theoretical analysis (Sec. 3) under a generalized linear model shows convergence and bias–variance behavior depending on $\\alpha$ and $\\beta$.\n\n3. Empirical results (Sec. 4; Fig. 3–5) using OPT-350M and LLaMA-1B demonstrate that moderate mixing ($\\alpha=\\beta=0.5$) improves both models’ performance but leads to representation homogenization.\n\nThe paper also concludes with a discussion of the implications for model diversity and long-term ecosystem dynamics (Sec. 5).", "questions": "You may refe to the content in the “weakness” section. If you can address the doubts raised there effectively, I will consider giving a higher score. \n\nI hope valuable work will not be overlooked.\n\n\nFor example, in addition to theoretical explanations based on existing assumptions, it would be great if you could highlight some unique insights proposed in this paper.\nPerhaps the paper already includes them, but I did not notice.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "1. Clear and meaningful problem setting：The paper addresses a timely and practically significant question — how recursive data interactions among generative models affect learning stability and diversity. The motivation and background are well-articulated (Sec. 1–2), making the research goal both relevant and understandable.\n\n2. Comprehensive and interpretable theoretical framework：The proposed formalism based on the parameters $\\alpha$ and $\\beta$ (Sec. 3) systematically captures cross-model data mixing. The accompanying bias–variance and convergence analysis provides solid conceptual grounding for the empirical findings (Fig. 2–3). Even without verifying every derivation, the overall reasoning is coherent and accessible.\n\n3. Exceptional clarity and readability：The writing is well-structured and accessible to readers beyond the immediate subfield. Explanations, figures, and notation are consistently clear, enabling a broad audience to grasp the motivation, methodology, and conclusions (Sec. 1–5).", "weaknesses": "1. Limited novelty in the modeling of cross-model interaction：The description of data-mediated interactions between models (Sec. 3) is clear and well-structured but largely descriptive. While it helps readers understand the setup, this section mainly formalizes an intuitive process rather than introducing a new mechanism or theoretical insight. As a result, the contribution of this part feels limited in terms of originality.\n\n2. Gap between theoretical modeling and practical relevance：Most of the paper focuses on theoretical modeling and proofs (Sec. 4–5). Although the derivations appear sound, the connection to real-world large-scale training scenarios remains weak. The introduction of parameters $\\alpha$ and $\\beta$ is conceptually useful, yet in practice, their exact values or ratios are difficult to estimate or control during continuous training. The conclusions drawn from the linear or generalized linear setting may not easily transfer to nonlinear or high-dimensional models.In essence, while the problem definition is good and $\\alpha$–$\\beta$ reasoning is meaningful, it is unclear how the theory can concretely guide actual large-model training.\n\n3. Experimental validation is narrow and idealized：The experiments (Sec. 4–5) mainly serve to verify the theory, but they do not provide further insights into realistic settings. Only two medium-sized models (OPT-350M and LLaMA-1B) and two datasets (SciQ, GSM8K) are used, with highly controlled data composition. The synthetic data are assumed to represent model outputs cleanly, without considering realistic mixtures of human and synthetic text (I know in limitation part). Scaling experiments or additional ablations (e.g., varying model size, task diversity, or realistic data proportions) would make the findings more convincing.\n\nOverall, the experimental content is rather insufficient. The question itself is meaningful, but it does not provide much insight in terms of conclusions. However, considering that this might be a theoretical paper, it is difficult to for me to  assess the practical value of such a theory. Therefore, I would lower the confidence to mitigate the possible impact of this uncertainty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GJ6EodNV0m", "reviewer": "Reviewer_Z5bc", "summary": "This paper investigates the phenomenon of data-mediated interactions among different generative AI models. Specifically, it studies what happens when different generative models are trained on each other’s outputs, a realistic scenario given the increasing prevalence of AI-generated content on the internet. The authors first review evidence that modern large language models (LLMs) are trained on overlapping, internet-scraped datasets that increasingly contain synthetic text from other models. Building on this, they formalize an iterative, interactive training framework where multiple entities train models using mixtures of public, private, and synthetically generated data. Then the authors give a theoretical analysis in a linear regression setting and derives closed-form dynamics for bias, variance, and convergence properties, showing that cross-model data sharing can promote homogenization while sometimes improving efficiency. Experiments using OPT-350M and LLaMA 3.2-1B fine-tuned on distinct tasks (SciQ and GSM8K) simulate multi-model interactions and confirm theoretical predictions: moderate mixing ($\\alpha = \\beta = 0.5$) yields the best balance.\nThe paper concludes that recursive cross-model training can both diversify and homogenize generative models.", "questions": "- Please kindly think of Weakness 1. The interaction of different models is a novel problem, while the analysis framework in this paper is a little weak. Could you improve the problem setting and make it closer to the reality?\n\n- There is a grammar mistake in the paper's title. Use \\textbf{each other's} instead of \\textit{each others'}.\n- I have a question on the proof of Lemma 1 (Appendix E.4). In line 1020, the derivation implicitly equates $(\\Pi S \\Pi)$ with $(I_K \\otimes \\underline{S})$. These two matrices are not equal. $(\\Pi S \\Pi)$ is a dense block matrix (specifically $\\frac{1}{K}(J \\otimes \\underline{S})$). $(I_K \\otimes \\underline{S})$ is a block-diagonal matrix. If the equation holds for the spectrum norm, I'd appreciate it if you can provide a detailed calculation.\n\n- Typos. Here I list several obvious mistakes and please proofread the whole paper to improve the quality of presentation. \n        \n- Line 276: its most recent parameter estimate $\\hat\\theta_{t−1,k}$ instead of $\\hat\\theta_{t−1,t}$.\n- Line 291. In the rightmost, $y_{t1}$ instaed of $y_{tk}$.\n- Different definition of $S_*$ in the main text Section 5 and appendix E.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- A novel problem. The paper's most significant contribution is the formalization of a new unstudied problem: \"data-mediated interaction\" within a multi-model systems. This shifts the research focus from the standard \"model collapse\" setting (a single model consuming its own outputs) to a more realistic and complex scenario where multiple heterogeneous models coexist and interact by training on a shared data pool containing each other's outputs.\n- Comprehensive Methodology: The paper supports its claims with a comprehensive methodological approach that provides both theoretical and empirical evidence. The authors develop a formal theoretical model (a linear regression setting) to make analytical predictions about the system's long-term dynamics and (2) validate these predictions with a set of well-designed experiments using large language models (OPT-350m and Llama 3.2 1B).", "weaknesses": "- Drawback of the whole setting. The theoretical and empirical framework assumes that fine-tuning data for each model is randomly sampled according to ($\\alpha-\\beta$) proportions (new vs. old, public vs. private). However, in practice, major foundation models rely heavily on highly curated, high-quality fine-tuning datasets that are explicitly designed to avoid noise or low-quality synthetic data. This mismatch between the model’s random-mixing assumption and real-world fine-tuning practices limits the external validity of the results — particularly the conclusions about homogenization and performance degradation under synthetic data reuse.\n- The theoretical analysis relies entirely on a linear regression model with Gaussian assumptions, which limits its generality to real-world large-scale nonlinear generative models. Although the authors cite “universality” results, the mapping from this toy model to practical LLM training remains speculative.\n- After examining the released code (sft-config), it appears that each fine-tuning round uses a very small effective data volume: the batch size is 2, with gradient accumulation over two steps and only 100 optimization steps per generation. This means that each “generation” sees at most a few thousand training tokens, which is extremely small compared to realistic fine-tuning scales for modern LLMs. Consequently, the observed trends in “cross-model interaction” may reflect under-trained or noisy optimization dynamics rather than genuine long-term convergence effects. Moreover, the experiments involve only K = 2 interacting models and explore just three discrete values for both $\\alpha$ and $\\beta$ ({0, 0.5, 1}), providing too coarse a sampling to fully characterize the theoretical phase behavior. These limitations substantially weaken the empirical support for the paper’s broader claims about multi-model ecosystems.\n- The paragraph leading with Step 3: Model Updates (lineno 180-188)' is misleading. The paper’s description of “model updates” incorrectly claims that successive generations of models such as GPT-1/2/3/4 and LLaMA-1/2/3 are typically trained by initializing with the previous generation’s weights. Only using the same datasets to train a family of models does not imply directly descend from one another'.\n- The proofs contain several typographical and notational issues that impede verification and, in a few places, likely invalidate steps. See more in Questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8QuTYZ7ySI", "reviewer": "Reviewer_1aha", "summary": "The authors consider what happens when a collection of models is iteratively trained on a combination of public data, private data, and data generated by all the other models. They begin by arguing that this setup reflects reality by surveying training datasets for a variety of generative AI models. Then, they present a simplified linear regression model for this setting. They derive the bias and variance of the collection of models after $t$ training iterations. They find that the models all converge most efficiently when about half of their initial data is public and about half of their data in future iterations consists of prior generation outputs. This prediction is validated in experiments training OPT and Llama, where each one has a private dataset (SciQ or GSM8k). Models are able to do transfer learning from each others' outputs.", "questions": "1. What exactly are the takeaways from the theoretical analysis? We see that under certain conditions, each model's estimate converges to the true parameter. Is Figure 2 the real takeaway from this section?\n2. In addition to loss, were there also the same patterns in model accuracy on SciQ and GSM8k?\n\n### Comments\n- It's a bit jarring for the paper to go from discussing generative AI for so long and then jump to a linear regression model. The rationale for this simplification makes sense, but should be mentioned/justified earlier in the abstract/intro (e.g., in the context of related work). \n- Section 3 has a lot of text, references, and tables for some well-known facts about model training. It could be summarized in a paragraph\n- the motivation in the intro for why it matters to study recursive training among multiple models vs just one single model is lacking. The intro just says it has \"received little attention,\" but doesn't explain why it matters that there are multiple models rather than just one. Are there new and different dynamics that occur? Otherwise we could just assume it's basically the same and that studying a single model recursively training is sufficient to understand what happens with multiple. \n- Some of the intro and related work was repetitive.\n- some in-text citations missing an author (gen, 2022), (app, 2024)\n- from the related work, it wasn't clear what it means for all models to have a \"bound in error $\\pi^2/6$\"\n- Line 100: \"long term effects\" is unclear; what is meant is iterative training dynamics, right?\n- Line 157: ... this doesn't seem overlooked. It seems like a widely understood fact that many models use CommonCrawl, arXiv, GitHub, Wikipedia, etc\n- Figure 1 is very nice.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "- The overall paper, presentation, and writing quality is high. This is a very well-executed research project.\n- The research topic of recursive training dynamics with multiple models is important, timely, and interesting\n- The model is well-designed (Figure 1 and Section 4 are great; I wish they had come two pages sooner)\n- The experimental setup is clever", "weaknesses": "- What we learn in the multi-model setting is limited and follows expectations: just as we've seen in single-model collapse with accumulation, but different private data can lead to some transfer learning in the population\n- The paper could do a better job of providing the intuitive takeaways from the theorems.\n- The paper is fairly verbose and repetitive; the core of the paper doesn't begin until page 5\n\n\n### Overall evaluation\nThis is a tricky paper to evaluate, as it's a very high-quality paper, but what we learn feels limited. Perhaps other reviewers will feel differently. This paper definitely deserves to be published and does contribute to the area of model collapse. The structure of the paper could be improved, getting to the contributions more quickly and providing clearer takeaways from the theory.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ysc4zKAbnp", "reviewer": "Reviewer_xD3M", "summary": "The paper describes work on investigating effects of recursive training of model generated data (+ combinations of human authored data) with LLMs. The authors draw motivation for the need for research in this direction by highlighting several realities and overlooked realities such as proprietary LLMs are trained mostly with internet-scraped data and how these datasets overlap substantially. The most important motivation that the authors emphasize is that future LLMs will evidently be trained will LLM-generated (their own) data. The authors back this with existing literature from learning theory on model collapse. In order to further shed light on the potential good or bad consequences of this model-data interaction across tasks, the authors formalize an interactive training pipeline controlled by alpha (fraction of new data per iteration) and beta (public data-private data partition). Results show from a small-scale experiment of two model providers (K=2) derived from Llama and OPT model architectures with t = 15 show that setting alpha and beta to 0.5 (equal partition) seemingly optimal results across tasks (science QA and math QA) compared to other values. Setting alpha to 0 denoting use of purely human-generated data results to degradation in task generalization while setting this to 1 denoting purely LLM-generated data results to equal degradation in the original task. \n\nOverall, the paper does present a simple and understandable method for potentially simulating model collapse and interactions upon training from LLM-generated data but my main issue is centered on grounding experiments with more rigor such as exploring K=10/20/50 or t=50. See feedback below for other concerns with the paper.", "questions": "“While some amount of mixing improves model performance on previously-unseen tasks, homogenization occurs for D∗ at all α and for  ̃Dk when α > 0—everywhere it can” - this sentence is confusing, can you please clarify and expand this?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper proposes a simple yet intuitive method for exploring training data dynamics of LLMs as shown in Figure 1. I found the paper to be fairly readable and the way the paper motivates the need for investigating recursive training from model-generated data to be useful in contextualizing the study and support for the experiments. I believe the model collapse research community may find this paper's results to be beneficial and interesting.", "weaknesses": "While I appreciate the simplicity and readability of the paper, there are some issues that I found that the authors can use to improve the quality/rigor of the study:\n\nFirst, the current experiment setup seems shortsighted to me with using only two model providers of K=2. Likewise, in terms of iteration, a realistic scenario would be repeatedly training on model-generated data by a longer margin, say t = 30 or 50 or even 100. The goal is to rigorously investigate how far can the performance converge or if there are possibilities of similar phenomenon like grokking. Model providers these days are extremely fast in releases (almost a new one every 3-4 months), hence I believe a more realistic setup is needed for the study. Likewise, a larger K such as 10 could also be explored to further investigate effects of training from a diverse collection of models.\n\nI believe the paper lacks equal discussion on the implications of training data dynamics that are grounded from the results. For example, the study shows that using a perfect split of 0.5 for alpha and beta seems to produce optimal results but this seems very idealistic and tied to the current experiment setup. More realistic setups might be more nuanced and highly dependent on factors such as training data quality, task diversity, etc. How can the study account for this? This part is underdeveloped in the paper.\n\nI suggest the authors to balance the structure of the paper by prioritizing experiment results. The current paper’s experiments and discussion are both pushed back to the last 2 pages of the paper and feels quite rushed/limited when you read it. The first four pages motivating the challenge could be condensed further to prioritize the results. I would also appreciate more expanded and clear discussion on task generalization as well as this seems underdeveloped as well.\n\nPlease improve your references and cite published articles. In the introduction alone, the main citations to support interdisciplinary use case adoption of generative AI are mostly blogs and websites, ignoring more qualified literature or previous works that have been peer-reviewed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)