Title: Escaping Collapse: The Strength of Weak Data for Large Language Model Training
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large-language models

synthetic data
Abstract: Synthetically-generated data plays an increasingly larger role in training large language models. However, while synthetic data has been found to be useful, studies have also shown that without proper curation it can cause LLM performance to plateau, or even "collapse", after many training iterations. In this paper, we formalize this question and develop a theoretical framework to investigate how much curation is needed in order to ensure that LLM performance continually improves. Our analysis is inspired by boosting, a classic machine learning technique that leverages a very weak learning algorithm to produce an arbitrarily good classifier. The approach we analyze subsumes many recently proposed methods for training LLMs on synthetic data, and thus our analysis sheds light on why they are successful, and also suggests opportunities for future improvement. We present experiments that validate our theory, and show that dynamically focusing labeling resources on the most challenging examples --- in much the same way that boosting focuses the efforts of the weak learner --- leads to improved performance.


Peer reviews:
comments: SUMMARY:
In this paper authors introduce a theoretical framework on understanding an important phenomenon - model collapse - as well as ways of escaping it. The authors connect the ideas of weak data and strong learner with a classic ML algorithm - boosting. Experiments validate theoretical results.

STRENGTHS AND WEAKNESSES:
### Strengths

- Well-written and solid theoretical grounds.
- Fully-accessible model and datasets which makes it fully reproducible.
- Experimental results are in line with the theoretical analysis.
- Two datasets - GSM8k (math) and MBPP (coding). 

### Weaknesses

- When T increases it seems from the experimental data (Appendix F) after T>5 improvements are diminishing. No explanation or analysis is given.
- Only one model (Gemma2 2B) and only one prompting method (when fine-tuning) - 3-shot prompting.

QUESTIONS:
For the filter-only setup, is the model trained on less samples (than w/ boosting) since we filter the bad ones out?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-24

comments: SUMMARY:
This paper addresses "model collapse" in LLMs trained on synthetic data by proposing a theoretical framework inspired by "boosting". It argues a "strong learner" (a powerful LLM) can continuously improve using only "weak data"—a minimal, external signal. The proposed algorithm iteratively identifies prompts where the model fails ("hard examples") and focuses a "weak labeler" on them. This feedback is mixed with high-quality synthetic data for retraining. The work proves this method prevents collapse and ensures convergence to an optimal model, outperforming baselines in experiments.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper addresses the critical and urgent problem of "model collapse" when training on synthetic data for the sustainable development of Large Language Models (LLMs).
2. The interesting theoretical framework offers a deep and insightful explanation for why the method is effective.
3. The experiments are well-structured to validate the theoretical claims.

Weakness:
+ The framework assumes a binary (good/bad), unambiguous, and efficiently evaluable quality function q. This is true for the tested tasks of math and coding, but for more subjective tasks like creative writing or summarization, quality is nuanced and hard to define, limiting the direct applicability of the theory.

QUESTIONS:
N/A

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-28

comments: SUMMARY:
The paper formalizes “training on synthetic data with minimal curation” as a boosting-style process. Experiments fine-tune Gemma-2 2B on GSM8K and MBPP. “Boosting” improves training accuracy and slightly increases test accuracy compared with (a) “do nothing” and (b) “filter-only” baselines, but the gains over the best supervised baseline are small

STRENGTHS AND WEAKNESSES:
Strengths:
* Unifying formalism that connects recent self-improvement heuristics (STaR / ReST) to classical boosting.
* Empirical section reproduces the model-collapse phenomenon and shows the proposed variant can mitigate it.

Weakness:
* The proof requires every generation to exactly fit the weighted dataset (Definition 1). This is unattainable for billion-parameter LLMs trained for a few hundred steps, undermining the practical relevance of Theorem 6.
* Experiments simulate the labeler with an already pretrained Gemma 2B, i.e. they assume access to an LLM stronger than the initial model. No evidence is given that a genuinely weak.
* No error bars, confidence intervals, or significance tests are reported.

QUESTIONS:
1. In practice the learner minimizes cross-entropy for limited epochs. How sensitive is convergence if the model only approximates the empirical distribution?
2. What is the wall-clock time and GPU budget per round for k = 8 on GSM8K? How would this scale to a 70B model on 1M prompts?
3. Does the algorithm still converge if the labeler’s errors are systematic (biased) rather than random?

LIMITATIONS:
See above.
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an effort to place the popular practice of LLM self-improvement—through rounds of synthetic data generation/pseudo-labelling, filtering, and fine-tuning—within a unifying theoretical framework. By inverting the traditional boosting framework from learning theory—shifting from weak learners with strong data to a setting with a strong learner and weak data, reflective of the current state of LLMs—they introduce a boosting-style algorithm that, upon convergence, offers probabilistic guarantees of producing correct responses to nearly all prompts. Experimental results display the effects of the combined algorithmic components on train and test error on mathematical and coding benchmarks, while further experiment in the appendix aims to empirically assess some relaxations of assumptions to the framework.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe paper comes up with an insightful theoretical framework for self-improvement and some theoretically-backed design principles for further developments in the area. The proposed boosting style algorithm is general enough to encompass commonly used frameworks and provides one of the first guarantees of convergence for self-improvement. Moreover, it allows diagnosing weaknesses of existing frameworks.\n\n**Weaknesses**\n\nThe work rests on some strong assumptions for the involved LLMs. Most notably the derivation assumes that the LLM can be considered as a strong learner powerful enough to interpolate a given conditional probability (which might be unrealistic in practice). Moreover, a perfect filter is also assumed. Thirdly, the verifier is considered sufficiently powerful to extract a (potentially tiny) fraction of high quality labels, which can be realized programmatically for some learning task categories, however again this might be facing limits in the wild. Empirical findings provide a mixed signal for the advantage that focusing on hard examples gives.\n\nQUESTIONS:\n- The impact of boosting towards the achievable test error remains unclear. Even though this is not set forth as a core concern of the paper it would be helpful for the reader that the authors seek a better explanation of the observed results. Moreover, can removing the focus on hard examples allow faster improvement over the first few rounds (as hinted in Fig.1 left), hence potentially be preferable under smaller computation budgets?\n\n- I think a valuable addition would be to explicitly compare the existing frameworks mentioned in the paper (STAR, REST, REST-EM) with the proposed framework, highlighting missing components or key variations. Moreover, including these frameworks as part of the experiments would provided useful baselines and expose which stages of the algorithm result in making a difference in practice.\n\n- A useful addition would be some further studies on the impact of the involved hyperparameters (e.g., the number of samples $k$, and fractions $\\alpha$ and $\\beta$).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}