Title: From Linear to Nonlinear: Provable Weak-to-Strong Generalization through Feature Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: weak-to-strong generalization

theory

feature learning
Abstract: Weak-to-strong generalization refers to the phenomenon where a stronger model trained under supervision from a weaker one can outperform its teacher. While prior studies aim to explain this effect, most theoretical insights are limited to abstract frameworks or linear/random feature models. In this paper, we provide a formal analysis of weak-to-strong generalization from a linear CNN (weak) to a two-layer ReLU CNN (strong). We consider structured data composed of label-dependent signals of varying difficulty and label-independent noise, and analyze gradient descent dynamics when the strong model is trained on data labeled by the pretrained weak model. Our analysis identifies two regimes—data-scarce and data-abundant—based on the signal-to-noise characteristics of the dataset, and reveals distinct mechanisms of weak-to-strong generalization. In the data-scarce regime, generalization occurs via benign overfitting or fails via harmful overfitting, depending on the amount of data, and we characterize the transition boundary. In the data-abundant regime, generalization emerges in the early phase through label correction, but we observe that overtraining can subsequently degrade performance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a theoretical framework for studying weak-to-strong generalization via feature learning. It shows how a two-layer ReLU CNN learns from a linear CNN model, and characterize two regimes depending on the number of samples for training the stronger model. Empirical simulation match the derived theories.\n\nSTRENGTHS AND WEAKNESSES:\nIn terms of strengths, the paper studies weak-to-strong generalization with nonlinear models, in contrast to existing analysis with linear model or random feature model. Also, the theoretical analysis appears to be complete in terms of characterizing a transition from generalization to overfitting. For the weakness, refer to the question section.\n\nQUESTIONS:\n1. What is the role of weak model? It seems the weak model will learn to correctly classify both the easy-only and both-signal samples (with $o(1)$ error) while random guess the hard-only samples. Then the training of stronger model would be on a dataset with label noise almost only on hard-only samples. Is my understanding correct? If so, could the author comment on what is the difference to paper [2], which also considers label noise in their data model.\n\n2. How practical the data distribution is and how likely the theoretical results hold in practice? Could the authors provide some experiments done in real-world scenarios?\n\n3. Could the authors provide more discussions in relation to the existing theoretical analysis of weak-to-strong generalization? Currently the paper only mentions the difference in terms modelling assumption, without an in-depth discussion how the derived results in the paper show similar/different insights compared to existing works.\n\n4. It seems the settings and analysis techniques are based on [1,2]. It would be good if the authors could comment on the differences in terms of analysis and training dynamics, which could further help understand the insights as well as the technical contributions of the paper (if any).\n\n[1] Benign Overfitting in Two-layer Convolutional Neural Networks.\n\n[2] Benign Overfitting in Two-layer ReLU Convolutional Neural Networks.\n\nLIMITATIONS:\nThe discussions of limitation is insufficient. Authors are encouraged to add a separate paragraph outlining the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a theoretical analysis for weak-to-strong generalization in binary classification based on special structured data consisting of signal and noise patches (a variation of the multi-view dataset introduced in Allen-Zhu and Li (2020)). The authors take a linear CNN as the weak model and a two-layer ReLU CNN as the strong model. The main theoretical findings are the following:\n* A separation result on the generalization of the weak and strong models on the special binary classification task, showing that there exists some parametrization of the strong model that achieves zero test error, whereas the weak model cannot achieve zero test error.\n* A generalization bound for the supervised training of the weak model with gradient descent, providing a sample complexity of $n_{wk} \\gtrsim \\frac{\\sigma_p^2 d}{(2 p_e + p_b)^2 \\|\\mu\\|^4}$ for the weak model to achieve a close-to-optimal test error on the binary classification task, where $\\sigma_p^2$ is the variance of the noise patches, $p_e$ and $p_b$ are the probabilities of the easy-only and mixture of easy and hard patches, $d$ is the feature dimension, and $\\mu$ is the easy feature norm.\n* A phase transition analysis for weak-to-strong generalization with respect to the weak-to-strong training sample size $n_{st}$ in the \"data-scarce regime\", showing that when the strong model is trained with weak supervision, it can either generalize in the case of benign overfitting or fail to generalize in the case of harmful overfitting.\n* An analysis of the \"data-abundant regime\", showing that weak-to-strong generalization can arise via early stopping but vanishes when the strong model is overtrained with weak supervision.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n* While mainly leveraging existing problem setup and technical tools, this paper provides a fresh non-linear feature learning analysis of weak-to-strong generalization, which offers new angles to understand weak-to-strong generalization.\n* The setting and main takeaways from the theoretical analysis are clearly organized and presented. \n* The key theoretical insights outlined in Section 4 is helpful for understanding the intuitions behind the theoretical results and the implications of the results for weak-to-strong generalization.\n\n### Weaknesses\n* Some detailed settings and statements seem vague and could be better clarified. See the \"Questions\" section below for more details.\n* The empirical evidence of this paper seems limited, even given its theoretical nature. Section 5 only provides a single set of experiments on the synthetic binary classification task and does not investigate the scaling of weak-to-strong generalization with respect to $n_{st}$ that seems essential for demonstrating the phase transition in the \"data-scarce\" regime (see the second point in the \"Questions\" section below). That is to say, I don't think strong experiments on real data and models are necessary for this work, given its theoretical nature. But given the (possibly inevitable) limitations of some theoretical statements in the paper (e.g., Theorem 3.4, see the second point in the \"Questions\" section below), it would be helpful to at least provide thorough simulations on the synthetic binary classification task.\n* A potential limitation inherited in the problem setup of this work is that the capacity gap between the weak and strong model completely comes from the architecture of the weak and strong models with respect to the special structured data. This is fairly different from weak-to-strong generalization, or the superalignment problem that it aims to address, in practice. The capacity gaps between large foundation models are often due to many other factors beyond the architecture (e.g., pretraining data and methods) and shared across various tasks, more like differences in the qualities of their representations. While I think the learning dynamics analysis of weak-to-strong generalization from linear to non-linear models on special structured data is theoretically interesting and valuable, I subjectively believe that it has a considerable gap from the practical weak-to-strong generalization problem. I think such a gap should be carefully discussed, maybe as a part of the conclusion/limitations at the end.\n\nQUESTIONS:\n* It is unclear from the setup in Section 2.1 how the \"easy\" and \"hard\" signals are defined and distinguished for various statements. \n    - From the proof of Proposition 2.2, the mixture of signs where the hard signals are sampled uniformly seems to be the only key difference between the easy and hard signals that leads to the separation of the algorithm-agnostic capacity gap between the weak and strong models (which I think should be highlighted explicitly). \n    - However, additional assumptions on the sampling probabilities $p_e, p_h, p_b$ and signal strength $\\|\\mu\\|,\\|\\nu\\|$ are needed (e.g., Condition (C5)) for weak-to-strong generalization when analyzing the training dynamics of GD. The current organization of these assumptions is somehow confusing. It may be helpful to partition the assumptions in Condition 3.1 into several parts. For example, C5 and C6 may be separated as a stand-alone additional assumption on the easy v.s. hard signals for the training dynamics analysis.\n* The phase transition in the \"data-scarce\" regime with respect to $n_{st}$ is not very clear from the statement of Theorem 3.4. In particular, Theorem 3.4 shows that $n_{st} \\ge C_2 \\frac{\\sigma_p^4 d}{p_b^2 \\|\\nu\\|^4}$ leads to benign overfitting, while $n_{st} \\le C_4 \\frac{\\sigma_p^4 d}{p_b^2 \\|\\nu\\|^4}$ leads to harmful overfitting. Here, the upper and lower bounds are asymptotically the same, and there are no discussions on conditions like $C_2 \\approx C_4$, showing that this indeed leads to a phase transition. The simulation in Section 5 shows one synthetic example with three seemingly arbitrary $n_{st}$, without investigating the scaling of weak-to-strong generalization with respect to $n_{st}$ that could demonstrate the phase transition.\n\nLIMITATIONS:\nMajor limitations are discussed in the weaknesses section. Here are some minor points:\n* The explanations for some assumptions (specifically, C1-C4) in Condition 3.1 is vague. \n* Section 4 may be partitioned into subsections of \"data-scarce\" and \"data-abundant\" regimes to be more organized.\n* The meaning of \"filters\" in line 312 is not clear. The explanation in lines 310-312 is not very clear either.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a theoretical and empirical analysis of weak-to-strong generalization, where a stronger model trained under supervision from a weaker model can outperform the teacher. This work fills a gap by analyzing a special setting: weak-to-strong training from a linear CNN to a two-layer ReLU CNN on patch data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper provides a detailed and mathematically sound characterization of weak-to-strong generalization in both data-scarce and data-abundant regimes, including tight bounds and transition conditions between benign and harmful overfitting.\n\nWeaknesses:\n1. The data distribution is highly engineered, with orthogonal easy/hard signal vectors and shuffled patch structures. While analytically convenient, it lacks realism and limits the applicability of the results to real-world tasks.\n2. Much of the analytical machinery (e.g., signal-noise decomposition, patch-wise CNN analysis, benign overfitting dynamics) closely follows prior work. The contribution lies more in adapting known tools to a new but contrived setup, rather than developing fundamentally new techniques.\n3. All experiments are conducted on synthetic data tailored to the theoretical setup. Even simple real-data tests (e.g., CIFAR with weak supervision) would help assess whether the observed phenomena persist beyond the toy model.\n\nQUESTIONS:\n1. Why does the data distribution assign only the positive direction $\\mu_y$ for easy signals, while it allows both $\\pm\\nu_y$ directions for hard signals? In Definition 1, the easy signals are always set to $\\mu_y$, whereas the hard signals are drawn uniformly from all sign combinations $\\pm\\nu_y$. This introduces a structural asymmetry. I understand that this may be designed to ensure the weak model can reliably learn easy signals while remaining ineffective on hard signals, but I would appreciate clarification on whether this choice is essential to the theoretical guarantees, or whether the same qualitative conclusions would hold under a more symmetric signal setup (e.g., also randomizing the sign of $\\mu_y$).\n2. How sensitive is the theory to the assumption that the weak model labels are not adversarial, especially in the data-abundant regime? Can small adversarial label flips cause failure of weak-to-strong generalization?\n\nLIMITATIONS:\nWhile the paper presents a rigorous analysis, its setting is highly stylized and somewhat artificial. The data distribution is carefully engineered to contain label-aligned \"easy\" and \"hard\" signals with hand-picked orthogonality, which may not reflect the structure or complexity of real-world data. As a result, it is unclear how broadly the insights or guarantees would transfer to practical scenarios, such as vision or NLP tasks. Moreover, much of the technical analysis builds directly on tools and mechanisms developed in prior work on benign overfitting and feature learning (e.g., Cao et al. 2022; Kou et al. 2023), such as signal-noise decompositions and patch-wise convolutional structure. While the application to weak-to-strong generalization is interesting, the paper feels more like a thoughtful reuse of existing techniques in a contrived setting tailored to fit them, rather than a technically novel contribution in its own right. A more ambitious advance would involve analyzing weak-to-strong generalization in less restrictive or more realistic settings.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies weak-to-strong generalization, a phenomenon where a strong model trained with the supervision from a weaker model outperforms the weaker one. This work focuses on the theoretical analysis to interpret the model behavior in weak-to-strong generalization with a linear CNN as a weak model and a two-layer ReLU CNN as a strong model. Depending on the amount of data, this work provides rigorous explanations for the performance in weak-to-strong generalization. Specifically, when the data is scarce, the generalization can occur via benign overfitting, or fail due to harmful overfitting; when the data is abundant, the generalization occurs at the early stage of model training and diminishes as the training progresses.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. This paper is well-written and clearly organized. The writing is concise and effective in conveying the claims. Overall, it's of high quality in terms of clarity.\n\n2. This work provides a theoretical and detailed analysis of the weak-to-strong generalization, specifically from the aspects of gradient descent with linear/nonlinear models, which has the potential to inspire such modeling or further research in the community.\n\nWeaknesses:\n\n1. This paper mainly focuses on the theoretical analysis, and the experiments are limited to synthetic data modeled similarly in the setup. It would be better to include more empirical insights or experiments with simple real data.\n\n2. It would be better for this paper if more discussions on the empirical insights could be added, such as identifying key aspects of data that we can adjust or change in real data to make weak-to-strong generalization occur.\n\nQUESTIONS:\n1. Some empirical insights might be helpful, such as identifying key aspects of real data that trigger benign or harmful overfitting in the data-scarce regime. For example, does a smaller dimension for data $d$, a larger proportion of mixed data $p_b$, or a larger number of samples for strong models $n_{st}$, tend to trigger benign overfitting more easily?\n\n2. Is it possible to include experiments with simple real data? What would be the challenges or limitations in validating the theoretical results in real data?\n\n3. As the weak-to-strong generalization refers to the stronger model outperforming the weaker model, is it needed to compare the performance of weaker and stronger models in the theoretical analysis in Theorem 3.4? It doesn't seem to be an obvious comparison in terms of the test errors.\n\n4. How are the conditions 3.1 obtained, and how do they ensure the training properties as discussed in lines 187-192?\n\nLIMITATIONS:\nThe authors didn't mention the limitations or potential negative societal impact of their work in the main paper. Discussing the possible scope of applying the modeling or constraint to real data might be helpful.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper attempts to find a simple setting in which weak to strong generalization provably occurs.  Their weak learner is a linear network with 3 copied weights (what they call a linear CNN).  Their strong learner is a 1-hidden layer ReLU CNN in which the second layer weights are set to one and only the first layer weights learn.  The ReLU CNN has three types of hidden units in correspondence with three patches in the data. Their model of data is a very contrived data distribution consisting of 3 different patches (matched to both the architecture of the linear and one-hidden layer CNN) with a set of easy, and hard signals in each patch, or noise.  They ask if the weak learner learns on actual data first, then the strong learner learns on data generated by the weak learner, when will the strong learner outperform the weak learner? \n\nThe types of results they can prove relevant to this are:\n\n1) In a data scarce regime (not much data but above a threshold), the weak learner performs as well as expected, and the strong learner can perform better than the weak learner due to benign overfitting, in which it does not overfit to the mistakes of the weak learner. \n\n2) In a data abundant regime, the strong learner provably exhibits weak to strong generalization, if the strong learner's learning is stopped before an early stopping time.  If the strong learner learns for too long it could potentially overfit to the mistakes of the weak learner (though no theoretical guarantees are derived at the convergence point of strong learning.  But early stopping of the strong learner provably prevents this.\n\n3) They have simulations in which they train the strong learner for longer than the early stopping time and find performance degrades then plateaus. No theory is provided for this.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1) The paper provides a simple example in which weak to strong generalization is provable.\n2) Some simulations bear out the predictions of the theory, and also reveal other phenomena at late times in learning that are beyond the reach of this theory.\n\nWeakness\n\n1) A major weakness is the highly contrived nature of the data and its match to the architecture.  The data comes in 3 patches, and the architecture comes with hidden units (or linear units) with receptive fields matched exactly to one of the three patches.  What goes into each patch has a strange combinatorial nature out of a finite set of objects (fixed orthogonal vectors).  Only first layer weights learn; second layer weights are fixed.  This strong match between data and architecture is unrealistic.  It seems that the analysis is tied to this match.  It is less clear what one can conclude about how the structure of realistic data impacts weak to strong generalization, and what is going on when there is not such contrived match, or such a toy dataset.  It is unclear how their derivations could be generalized. \n2) Less of a concern but an area for improvement in exposition: the authors need 6 conditions C1-C6 in section 3.  Some of them are well explained intuitively whereas others are not.  More discussion of these conditions and their roles would help.\n\nQUESTIONS:\nWhat are the prospects of generalizing your theory methods to more general structures in data and lack of match to architectures?\n\nLIMITATIONS:\nYes. But further discussion on the limitations of their data assumptions would be helpful.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}