Title: Data Mixing Can Induce Phase Transitions in Knowledge Acquisition
Venue: NeurIPS 2025
Accepted: yes
Keywords: knowledge acquisition

memorization

scaling laws

LLMs
Abstract: Large Language Models (LLMs) are typically trained on data mixtures: most data come from web scrapes, while a small portion is curated from high-quality sources with dense domain-specific knowledge.
In this paper, we show that when training LLMs on such data mixtures, knowledge acquisition from knowledge-dense datasets—unlike training exclusively on knowledge-dense data—does not always follow a smooth scaling law but can exhibit phase transitions with respect to the mixing ratio and model size. Through controlled experiments on a synthetic biography dataset mixed with web-scraped data, we demonstrate that: (1) as we increase the model size to a critical value, the model suddenly transitions from memorizing very few to most of the biographies; (2) below a critical mixing ratio, the model memorizes almost nothing even with extensive training, but beyond this threshold, it rapidly memorizes more biographies. We attribute these phase transitions to a capacity allocation phenomenon: a model with bounded capacity must act like a knapsack problem solver to minimize the overall test loss, and the optimal allocation across datasets can change discontinuously as the model size or mixing ratio varies. We formalize this intuition in an information-theoretic framework and reveal that these phase transitions are predictable, with the critical mixing ratio following a power-law relationship with the model size. Our findings highlight a concrete case where a good mixing recipe for large models may not be optimal for small models, and vice versa.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, authors investigate how mixing small amounts of high-quality data such as synthetic biographies with web-scraping data affects knowledge acquisition. Through controlled experiments. authors discover two types of phase transitions: along model size an along mixing ratio. The authors then attribute these sharp transitions to a capacity allocation phenomenon. They also formalize their intuition to derive a power-law relationship that accurately predicts the critical thresholds.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. Data mixing during pretraining is a critical and under-explored topic, and this paper addresses it using a highly controlled and well-designed experimental setup. By systematically varying the model size and the ratio of knowledge-dense to web data, the authors reveal two sharp phase transitions in knowledge acquisition. The clarity of these findings, and the effective presentation of results (e.g., Figures 1–3), make the empirical contribution compelling.\n\n\n2. In addition to analysis, authors also propose two actionable and well-motivated strategies to improve knowledge retention task. Overall, this paper has a complete analysis of theory, empirical findings and practical insights. \n\n\n**Weakness**\n\n1. Prior work (e.g., [1], [2]) has shown that phase transitions and emergent behaviors in LLMs can be artifacts of (a) metric thresholding and (b) sensitivity to random initialization. However, this paper does not adequately rule out those possibilities. For instance, all key evaluations use discrete accuracy measures (e.g., exact match of factual answers). It would strengthen the case considerably if the authors provided results using more continuous metrics (e.g., token-level F1, log-likelihood) and ran models with multiple random seeds to test robustness against training noise.\n\n\n2. Figures 1, 2, and 5 use a mix of linear and log scales on the y-axis—sometimes for the same metric (accuracy) across different plots. This inconsistency is not explained and raises the concern that log scaling may be exaggerating the sharpness of transitions. A clearer and more principled explanation of axis choices would help ensure the visualizations do not unintentionally bias interpretation.\n\n\n\n3. One of the motivations for data mixing is to enable multitask performance (e.g., reasoning, coding, factual recall). However, in this paper, the “web” data serves more as a placeholder than a concrete \"competing\" task (i.e., learning synthetic biography versus math). As a result, the paper does not deeply analyze trade-offs between capabilities caused by data mixes—for example, whether increasing knowledge-dense data harms performance on reasoning.  \n\n\n4. Although Section 6 introduces two mitigation strategies, both seem most applicable to factoid-style datasets like synthetic biographies, where each data point is short, modular, and isolated. It is less clear how these strategies would extend to richer forms of knowledge or reasoning, where facts are embedded in longer contexts or interconnected. This limits the generality of the proposed solutions.\n\n\n\n[1] Are Emergent Abilities of Large Language Models a Mirage?   http://arxiv.org/abs/2304.15004\n\n\n[2] Distributional scaling for emergent capabilities. https://arxiv.org/abs/2502.17356\n\nQUESTIONS:\n1. Authors claim they observe \"phase\" transitions across model scale and data ratio. But what qualifies as a phase transition versus a linear development? Without a frame of reference, how do we know the observe performance change is sharp and considers as a phase transition?\n\nLIMITATIONS:\nNo, authors adequately addressed the limitations and potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates how mixing knowledge-dense data with large-scale web data during LLM pre-training can cause *phase transitions* in knowledge acquisition. Using synthetic and real-world datasets, the authors show that knowledge retention does not always scale smoothly with model size or data mix ratio: instead, there are critical thresholds below which models memorize almost nothing, and above which they suddenly acquire much more knowledge. They develop an information-theoretic framework to explain this as a capacity allocation problem, and they validate it with empirical results and scaling laws. The paper also proposes practical strategies—like random subsampling and compact rephrasing—to improve knowledge retention under low mixing ratios.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n1. The paper demonstrates a substantial amount of work, including comprehensive experiments, thorough theoretical analysis, and corresponding application designs.\n2. The work presents many interesting findings, which have clear implications for data curation and training strategies for large language models at different scales.\n\n### Weaknesses\n\n1. The spacing between section titles and the main text is too narrow, which does not conform to the standard template. As a result, the overall content appears overly dense, negatively affecting readability.\n2. I believe the design of the Reasoning Tasks in Section 3.3 is questionable. Although the task appears to assess a slope-related problem involving mathematical reasoning, the evaluation instances frequently appear in the training data. As a result, the model can rely purely on memorization to answer the slope questions correctly, rather than genuinely computing the slope through intermediate reasoning steps. Therefore, in my opinion, this task design does not align with the fundamental definition of reasoning and remains within the scope of factual memorization.\n\nQUESTIONS:\n1. Based on Weakness 2, could you design a task that better aligns with the definition of reasoning to more effectively support your claims?\n2. Although your experimental setup follows prior work, I think this setting—consisting of only a single entity and its associated attributes—is insufficient to verify whether your conclusions hold in scenarios involving multiple entities and more complex relationships. Could you consider supplementing your study with some simple experiments under a more complex setting?\n3. The experiments in Section 6 appear to be conducted only on a 410M parameter model, which is not commonly used in current real-world scenarios. Could you reproduce this part of the study on a model with approximately 7B parameters to better demonstrate the generalizability of your applications?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the dynamics of knowledge acquisition in large language model. Specifically, this work examines how things change with different mixtures of data. It aims to address what is learned from small amounts of knowledge rich data and large amounts of less rich data. The authors propose a scaling law that predicts phase transitions in the learning along the axis of data scaling.\n\nSTRENGTHS AND WEAKNESSES:\nI like the framing and the angle of this paper. I think it addresses a timely and important question. I do have some concerns about clarity and experimental design.\n\nStrengths:\n- This work touches on an important topic and makes a reasonable contribution toward trying to deepen our understanding of knowledge acquisition in large language models.\n- To my knowledge, this work is novel and original. The controlled experiments on data mixture and the isolation of knowledge-dense data from more general data is interesting.\n- The writing is generally good, although I do have some clarity questions below.\n\nWeaknesses:\n- Clarity\n  - The words \"memorized\" and \"learned\" are used in confusing ways in this paper. Can the authors clarify how these things are measured and whether they mean the same thing or not?\n  - In the summary of Finding 1 on the second page, the authors write \"... the model suddenly memorizes most biographies. Can the authors give a little more description of what \"suddenly\" means and what an alternative would look like? \n- Formatting\n  - The margins over the figures on pages 2, 4, 5, 7, and 8 are smaller than the formatting rule allow. Please fix this.\n- Experimental Design\n  - In Section 2, the training setup describes multiple epochs of some data and only one epoch of the web data. Can the authors address how repeating data may be at play here? Is that a confounder? Is there a possibility that repeating the general data or not repeating the biography data would change the results?\n  - Synthetic data poses a potential problem here. This introduces some distillation potential if the synthetic data is generated with an LLM.\n  - The Compact Knowledge Mixing is not so clear to me. Can the authors make a bit more clear how the frequencies are changed? Is the data compressed to make it fewer tokens and thus more frequent in the training set? Adding data but keeping the ratio fixed is confusing to me.\n\nQUESTIONS:\nMy questions are listed with my weaknesses above.\n\nI'm excited to engage with the authors to clear up the aspects I don't fully understand and I'm optimistic that with some iteration this paper can be made stronger.\n\nLIMITATIONS:\nI'd like to see a bit more discussion on the limitations of experiments. The very controlled setting here allows for nice experimentation, but the authors should better describe and discuss how these findings may or may not translate to real settings where data isn't necessarily broken up so nicely into general data and specific data. Can the authors further explain which of their findings might relate to real world training regimes?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies the relationship between model training and dataset mixing, model size, revealing some interesting phenomenon, along with some theoretical analysis.

STRENGTHS AND WEAKNESSES:
Some interesting findings are revealed via the experiments in this paper. For instance, given a fixed data mixing, the model memorizes knowledge only when its model size is larger than threshold. Given a fixed model size, data mixing also has a threshold, below which leads to model memorizing nothing even with extensive training.

Some theoretical analysis are provided. The authors also discussed strategies to enhance knowledge acquisition under low mixing ratios.

QUESTIONS:
The paper has shown that the knowledge acquisition happens only when mixing ratio larger than a threshold for a fixed model size, what will happen if we use different mixing ratio at different stages of training, will the model forget what it already memorize?

Could the authors provide results (for example Figure 2) with mixing ratio from 0 to 1 rather than from 0 to 0.4?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03