Title: Superposition Yields Robust Neural Scaling
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM

superposition

neural scaling law
Abstract: The success of today's large language models (LLMs) depends on the observation that larger models perform better. However, the origin of this neural scaling law, that loss decreases as a power law with model size, remains unclear. We propose that representation superposition, meaning that LLMs represent more features than they have dimensions, can be a key contributor to loss and cause neural scaling. Based on Anthropic's toy model, we use weight decay to control the degree of superposition, allowing us to systematically study how loss scales with model size. When superposition is weak, the loss follows a power law only if data feature frequencies are power-law distributed. In contrast, under strong superposition, the loss generically scales inversely with model dimension across a broad class of frequency distributions, due to geometric overlaps between representation vectors. We confirmed that open-sourced LLMs operate in the strong superposition regime and have loss scaling inversely with model dimension, and that the Chinchilla scaling laws are also consistent with this behavior. Our results identify representation superposition as a central driver of neural scaling laws, providing insights into questions like when neural scaling laws can be improved and when they will break down.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores why larger language models (LLMs) perform better by examining the role of representation superposition in neural scaling laws. The authors construct a toy model to study how superposition and data structure influence the scaling of loss with model size. They find that under strong superposition, where all features are represented but overlap, the loss scales robustly as an inverse power law with model dimension, largely independent of feature frequency distributions. In contrast, weak superposition shows more fragile scaling behavior. The study concludes that representation superposition is a key mechanism underlying observed neural scaling laws and suggests that promoting superposition could lead to more efficient training strategies and model architectures for LLMs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n(1) The paper provides a comprehensive analysis of how representation superposition affects neural scaling laws, offering new insights into why larger language models perform better.\n\n(2) It systematically studies the relationship between superposition and data structure, using both theoretical analysis and extensive experiments to support its conclusions.\n\n(3) The findings have practical implications for developing more efficient training strategies and model architectures, potentially leading to better performance with less computation.\n\nWeakness:\n\n(1) The paper relies on a simplified toy model, which may not fully capture the complexities of real-world large language models.\n\n(2) The theoretical analysis is not fully rigorous, leaving some deeper questions unanswered and requiring further investigation.\n\n(3) The study focuses primarily on representation loss and does not explore other factors like dataset size or training steps in detail.\n\nQUESTIONS:\nIn my understanding, a larger dimension implies greater superposition, which also means more parameters. How can we be inspired to achieve better performance with fewer parameters?\n\nLIMITATIONS:\nThe study focuses primarily on representation loss and does not explore other factors like dataset size or training steps in detail.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work analyzes neural scaling laws in terms of the concept of superposition. The main body of the paper centers around a toy model for superposition first advanced by a team at Anthropic. In this model a small two-layer network is trained on a synthetic dataset where all features in the dataset can be directly controlled. The work identifies an optimization scheme such that strong superposition and weak supervision can be controlled via a weight decay parameter. Here, strong supervision is the state where the hidden activations of features fail to be fully orthogonal and the extent to which they are not orthogonal is determined by the frequency by which the feature appears. Weak superposition on the other hand is the situation where a subset of the features are represented by orthogonal vectors in the model and the rest are mapped to zero. The paper looks at how adjusting this hyperparameter, as well as the distribution of features impacts scaling constants in the model. They find two regimes. One, corresponding to strong superposition, where scaling closely follows a power law and one where it fails to follow a power law corresponding to weak superposition. The paper justifies this through an analysis of their toy model. The paper goes on to explore the statistics of large-scale LLMs and provides evidence that these fall into the strong superposition scheme.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- **Analyzing the relationship between scaling laws and superposition is a natural research direction:** Scaling laws and superposition are both central ideas that have arisen through the development and study of LLMs. It is natural to ask in what ways they may be related. The paper lays out a nice framework with which to think about this.\n- **The analysis of the toy set-up is comprehensive and yields a nice story:** Though the toy model used in this paper is not a contribution of the work, the paper puts it to good use and it is a very helpful aid when reasoning through the ideas presented and discussed by the paper. The notation and descriptions of the experiments are reasonably easy to follow and do not ask too much of the reader’s mathematical background.\n\t\nWeaknesses\n- **The structure of the paper could be improved:** The paper is primarily one long analysis of the toy model. This analysis could easily be partitioned into sections that would help organize the content. For instance, the analysis of the weak superposition regime could be a section (or subsection), the analysis of the strong superposition regime could be a section, and the analysis of non-toy models could be a section. This would enable the main takeaways of the paper to be more easily extracted and would help readers that do not want to read through the paper linearly. It would also be helpful to provide a conclusion or discussion section which summarizes the results and puts them into context.\n- **The section on real-world models is weak when compared to the analysis of the toy set-up:** As noted above, the reviewer felt that the toy dataset section was helpful and made the paper much more interesting since it enabled a more detailed analysis both from the perspective of being able to finely control the whole system for experiments (including the data distribution) but also being able to theoretically justify various observations. However, this toy set-up is very far from real machine learning. The paper includes a short section in the end where real LLMs are analyzed for some of the same properties that were observed in the toy model, but the reviewer found this far less convincing than the rest of the paper. It is unreasonable to ask too much in this larger setting, but a more detailed analysis would be helpful. Furthermore, there may be small scale experiments that can be run that still use real data and our easier to analyze than large-scale LLMs.\n\t\nNitpicks\n- The paper uses the terminology ‘weak representations’ and ‘weak superposition’, the reviewer would suggest changing the former as these are not the same thing and may be confused with the current naming convention.\n- Line 40: ‘depend sensitively on’ $\\mapsto$ ‘are sensitive to’.\n\nQUESTIONS:\n- The paper changes the hyperparameter settings so as to induce different kinds of superposition. The paper then observes different scaling properties. There is certainly a correlation here, but how do we know that superposition causes the scaling laws (it is possible that this falls out of the analysis and I am misremembering)?\n- It is this reviewer’s understanding that the paper does not consider the case where features are correlated. This is an essential feature of real-world data and it would be interesting to consider this. Have the authors thought about how this would change their model?\n- What does the statement ‘Conceptually, fitting…’ on Line 46 mean?\n\nLIMITATIONS:\nThe paper does not address limitations. Limitations might include the fact that most of the analysis is done on a very simple model and would need to be supported by evidence in larger, more real-world datasets/models.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an explanation for the power-law scaling behavior of LLMs' loss with model size, connecting it to superposition—the phenomenon where neural networks represent more features than their model dimensions by allowing some degree of interference (non-orthogonality) between feature representations. The authors build upon a previously introduced single-hidden-layer toy model, originally designed to demonstrate superposition, modifying it by incorporating a weight decay (or growth) term into the loss. This modification enables control over the degree of superposition, allowing its effects to be studied independently of data properties. The authors find that in a weak superposition regime, where only the most important (frequent) features are represented without interference, loss scales as a power law with model size only if the data distribution itself follows a power law. However, in a strong superposition regime, where all features are represented with interference and this interference dominates the loss, the loss scales as a power law with model size regardless of the data distribution. They provide a geometric explanation for this behavior in their toy model: in the strong superposition regime, the optimal representational strategy to minimize interference approximates an equal-angle tight frame. Under these conditions, the squared dot product between representations scales inversely with model size according to a power law with exponent 1. The authors then analyze the unembedding layers of several publicly available LLMs within a framework analogous to their toy model, finding evidence that these models operate in the strong superposition regime. This suggests that the mechanism identified in the toy model might also dominate the loss behavior of real LLMs, potentially explaining the empirically observed power-law scaling with model size. Furthermore, the empirically observed scaling exponents closely match the predicted value of 1. The authors also demonstrate alignment with the Chinchilla scaling laws by converting Chinchilla’s parameter-count-based term into a model-width-based term (consistent with their single-hidden-layer framework), again obtaining an exponent close to 1.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\n- A compelling and somewhat intuitive argument presenting a novel connection between two highly relevant and interesting neural network phenomena: superposition and scaling laws.\n- A useful and straightforward extension of the superposition toy model that allows interventions to induce or inhibit superposition, resulting in observations largely independent of data distribution or model size. This differentiates it from previous explanations of neural scaling laws that relied on data-distribution assumptions (e.g. uniform on a d-dimensional manifold [15]; or power-law distributed quanta [19]).\n- A rigorous, experiment-rich investigation of the toy model, revealing interesting behaviors across varying data distributions and degrees of superposition, consistent with a geometric explanation of power-law scaling under optimal arrangement conditions to minimize interference in superposition.\n- Insightful experiments with real LLMs that provide empirical support for the proposed theory.\n- The work goes beyond a purely descriptive model, potentially offering actionable insights for improving LLM architectures and training strategies.\n- The authors suggest many different directions for future work based on their model and gained insights, showcasing that it can be an important contribution to the community even beyond the results presented here.\n- Clearly written and overall very well presented.\n\n**Weaknesses**:\n\nAny potential limitation I identified has already been acknowledged and thoroughly discussed by the authors and does not, in my opinion, remotely warrant rejection. See the Questions section for additional points intended for discussion.\n\nQUESTIONS:\nThe points below are intended for discussion—feel free to consider them low priority and/or disregard some of them. The questions are intuition-driven, so please forgive any lack of rigor.\n\n- I would like to better understand your thinking behind the following phrasing:\n\n  > *\"Conceptually, fitting functions or manifolds and learning skills or grammars are primarily tasks of the transformer layers, while representation learning is more directly tied to the embedding matrix and the language model head.\"*\n\n  Defining representation learning strictly as learning token embeddings seems unusual to me. Aren’t intermediate activations also representations, capturing increasingly abstract and composed features (at least up to the middle layers)? Would it perhaps be beneficial to introduce a more specific term for what you call \"representation loss,\" clarifying that you specifically refer to non-contextual representations?\n\n- Given the compositional process across layers, which yields a combinatorially increasing number of features far exceeding the vocabulary size, should we expect loss due to superposition (geometric interference) to play an even more dominant role in what you've termed the _parsing loss_? Would this make weak superposition virtually impossible in LLMs?\n\n- If we do not assume the Linear Representation Hypothesis to hold, would this have any direct implications for your theory? In its current form I guess it would not, given that the non-contextual representations (token embeddings) you analyze are linear by definition, but I'm curious whether you foresee any implications arising from such a scenario—perhaps specifically for the parsing loss.\n\n- Since natural language data often follow power-law distributions, other theories relying on this assumption should also be considered plausible explanations for LLM scaling behavior. It would be beneficial to see a discussion regarding how compatible or complementary your theory of neural scaling with model size is relative to previously proposed theories.\n\n- What do you consider the most likely approach for falsifying your theory?\n\n- Considering that superposition has mostly been presented as a negative phenomenon and a significant interpretability challenge for neural networks, I suggest discussing the potentially adverse implications of your proposal to encourage superposition as a strategy for improving performance. Especially given that AI safety is becoming increasingly prominent, this concern merits acknowledgment and thoughtful discussion.\n\n- Typo in the caption of Figure 6: \"at stong superposition\" → \"at strong superposition\".\n\nLIMITATIONS:\nThe authors should discuss the potential negative implications for AI interpretability and safety arising from their suggestion to promote superposition in neural networks as a means of improving performance.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper argues that superposition is an important mechanism underlying observed neural scaling laws. To show this, the authors consider a toy setting adapted from the original toy models of superposition work and study how the level of superposition, controlled by a specifically defined weight decay term, influences scaling behavior. The key finding is that under weak superposition, power-law scaling occurs only if the features themselves are power-law distributed (model exponent is $\\alpha_m \\approx \\alpha - 1$, where $\\alpha$ is a data exponent), while under strong superposition, power-law scaling emerges largely independent of the feature distribution (model exponent, however, still depends on the data exponent, following $\\alpha_m \\approx \\max(1,\\ 2(\\alpha - 1))$). Beyond the toy setting, the paper also analyzes token representations in the last-layer weight matrix of practical LLMs and shows that they exhibit power-law scaling with a model exponent $\\alpha_m \\approx 1$, consistent with the toy example in the strong superposition regime with $\\alpha \\approx 1$. This connection is then further extended and tied back to the scaling behavior of the cross-entropy loss in LLMs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper addresses an important problem: understanding the underlying reasons behind neural network scaling laws is highly relevant in the current era of large models.\n2. The idea of superposition and its role in scaling laws is interesting and, as far as I know, novel. \n3. The paper focuses on a well-motivated toy example and makes a meaningful connection to practical models.\n4. The authors make a commendable effort to provide a thorough analysis, linking theoretical and intuitive ideas with empirical evidence.\n\nMain weaknesses:\n1. The writing of the paper is very dense. In particular, the monolithic 4.5-page results section was difficult to read and navigate, and I found it challenging to precisely assess the claims due to how scattered they are throughout the text. I believe the paper would benefit from a significant rewrite to improve clarity and structure. Additionally, dedicating more space in the main text to discuss how the proposed explanation of scaling laws connects to existing theories (it is very important for placing the work in context) and comment on how this explanation could inspire new training methods or architectures (this point is mentioned in the abstract but discussed only in the appendix) would benefit the paper.\n2. The choice of weight vector norms as indicators of features being represented should be more thoroughly explained, and their distributions along with the threshold values used should be commented on in more detail:\n    - Initially, it was not clear to me what \"feature being represented\" means and that it is different from a feature being correctly predicted. Because of this, I was confused during the first two sections about the idea of learning more features than dimensions, since learning more features should directly harm the quality of how well each of them is learned. The second section of the paper should explicitly discuss that representing a feature is different from correctly predicting it, and that superposition is a balance between representing more features and having destructive interference between them during prediction.\n    - The choice of the 0.5 threshold for defining whether a feature is represented or not is based on the empirical norm distribution shown in Figure 3a. However, as far as I understand, this distribution may vary significantly depending on the level of superposition (e.g., the norms are not close to one in the distribution shown in Figure 6b). Could you please clarify how the norm distribution changes across different superposition regimes, and whether the 0.5 threshold is meaningful in all cases, not only in the weak superposition setting?\n    - Could you please comment on why the vector norms are distributed bimodally in the case of strong superposition shown in Figure 6b? Given the form of the weight decay term, one might expect the raw norms to be close to 1. Why is the density low near 1? Intuitively, since the feature frequencies change smoothly and monotonically, one might expect feature representation behavior to also change smoothly. Why do features instead fall into two distinct modes? Is this an artifact of the particular weight decay formulation? And if so, why should we expect this behavior to reflect what happens in practical LLMs?\n3. I believe there is an important issue in the assumption made in equation (4). Specifically, $y_i$ are assumed to be equal to $0$ for features $i > m$. However, based on the original toy models of superposition work, for the unlearned features, $y_i = b_i$ where the learned bias term $b_i$ is the expected value of $x_i$. \n4. The assumption that strong superposition is characterized by ETF-like geometry is not convincingly validated. The main evidence I gathered from the paper comes from Appendix F.6 and Figure 15, specifically the observation that the variance of feature overlap is smaller for strongly represented features than that of $m^2 / 2$ random vectors. However, this alone is not particularly convincing. Do weakly represented features behave differently? Is the number of strongly represented features actually equal to $m^2 / 2$ in practice? Not according to Figure 6c. In general, for such an important claim, I would expect a more rigorous and convincing analysis.\n5. There is no clear explanation for the change in scaling observed with strong superposition as data exponents increase. The intuitive explanation that higher-frequency feature vectors become more separable (collapsing lower frequency vectors and increasing their errors due to feature overlap) is compelling, but seems to be inconsistent with Figures 6d and 16, where increasing data exponents actually increases feature overlap for strongly represented features and decreases feature overlap for weakly represented ones.\n\nAdditional concerns, comments, and questions:\n1. Line 62: The weak superposition regime is not defined at this point.\n2. Lines 88/95: It seems to me that “data” is usually used to refer to a dataset or data distribution, so it was a bit confusing that “input data” and “one data” referred to a single sample.\n3. Figure 3a: The small subplots are difficult to parse and should be commented on in more detail.\n4. Equation 2: Does this equation take into account the loss gradients or only the weight decay term?\n5. Figure 4b: A diverging color map centered at $\\alpha_m=1$ would benefit the figure. Currently, it is difficult to see that the values in the red box are close to 1.\n6. Figure 4b caption: What does $m = 10 \\sim 100$ mean here? Which model size is used?\n7. Line 184: The word “nonlinearity” appears twice.\n8. Figure 6c caption: “which is much larger than m (slowly increasing dashed line)” — I think it should be $m/n$.\n9. Figure 6e: The second increasing line is not defined. Which weight decay values are used in these experiments? Where are the points for high weight decay values close to 1?\n10. Why, in the analysis of token representations of LLMs, is the weight matrix of the last layer used rather than the embedding matrix? (Or are they tied in the considered models?)\n\nQUESTIONS:\nAll questions and concerns are detailed in the Strengths and Weaknesses section. The main weaknesses 2-5 will have the greatest impact on my final evaluation after the rebuttal.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-07"}