Title: Weak-to-Strong Generalization under Distribution Shifts
Venue: NeurIPS 2025
Accepted: yes
Keywords: Weak-to-strong generalization

Distribution shift
Abstract: As future superhuman models become increasingly complex, accurately supervising their behavior may exceed human capabilities. 
Recent works have demonstrated that in such scenarios, weak models can effectively supervise strong models, a phenomenon known as weak-to-strong generalization. However, we find that naive weak-to-strong generalization fails under distribution shifts, often leading to worse performance of the strong model than its weak supervisors. To address this, we propose RAVEN, a robust weak-to-strong generalization framework that dynamically learns the optimal combinations of weak models in addition to parameters of the strong model. We demonstrate the effectiveness of RAVEN on image classification, text classification, and preference alignment tasks. RAVEN outperforms alternative baselines by over 30% on out-of-distribution tasks while matching or surpassing existing methods on in-distribution tasks. Moreover, our results show that RAVEN assigns higher weights to more accurate weak models, demonstrating its ability to automatically identify trustworthy supervision.


Peer reviews:
comments: SUMMARY:
The paper introduces RAVEN, a weak-to-strong generalization framework that adaptively weights weak models to reliably supervise a strong model under distribution shifts, achieving significant performance gains across multiple tasks.

STRENGTHS AND WEAKNESSES:
Pros:

1. The paper proposes RAVEN, a framework that dynamically learns to combine weak models for supervising a stronger model.

2. RAVEN introduces adaptive weighting and easy-sample guided initialization.

3. Extensive experiments on image classification, text classification, and preference alignment show that RAVEN outperforms baselines.

Cons:

1. While the core ideas are sound, the adaptation mechanism (especially how adaptive weights are computed and updated) is difficult to follow. I suggest adding a clear flowchart or algorithmic table that outlines the entire pipeline step-by-step. This would make the framework more accessible to readers unfamiliar with meta-learning or ensemble supervision.

2. The ablation study section could be further strengthened. One useful baseline would be a variant where the weak teachers are used for static weight computation, without adaptive updates. This could isolate the contribution of adaptive weighting more effectively.

3. Table 5 is mentioned in the results but lacks table 5 contents in the paper. This disrupts the flow of the paper and leaves the reader uncertain about the insights it is intended to convey.

QUESTIONS:
see above

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-14

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces RAVEN, a robust weak-to-strong (W2S) generalization framework, to address the performance degradation that occurs when a distribution shift exists between the training data of weak models and the fine-tuning data of a strong model. RAVEN employs an ensemble of weak models and dynamically learns an adaptive weighting scheme to combine their outputs for supervision. The effectiveness of RAVEN is demonstrated through extensive experiments across image classification, text classification, and preference alignment tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n    1. This work introduces and formalizes a critical problem in W2S generalization.\n    2. RAVEN’s proposed adaptive weighting ensemble is an intuitive and effective solution.\n    3. The experiments and analysis are also comprehensive.\n\n\nWeaknesses:\n\n    1. This paper introduces a novel scenario, named weak-to-strong generalization under distribution shifts. However,  the choice of weak and the strong models is not very representative. For instance, the paper utilizes AlexNet and DINO for image classification, why not evaluate more advanced models like vision-language models?\n\n    2.  The proposed Robust AdaptiVe wEightiNg (RAVEN) approach incorporates adaptive weighting and easy-sample guided initialization. However, both of these techniques are inspired by prior works, so the methodological contribution cannot be considered highly novel. Moreover, the paper does not further explore some aspects, such as whether linear or nonlinear combinations would be more effective in adaptive weighting.\n\n    3. The experimental designs for text classification and preference alignment are not fully convincing, as the performance gap between the weak and strong models is marginal. In the text classification experiments, the weak model underperforms the strong model by only about 4% in accuracy. While in preference alignment, the difference shrinks to less than 2%. A more rigorous experimental setup should employ model pairs with clearly distinguishable capabilities to properly validate the proposed weak-to-strong generalization framework.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the possibility of training strong models with supervision from weak models where the latter are trained with data from a distribution that is different from the one that the former needs to be trained on. The authors propose to adaptively combine weak models via learnable weights to guide the training of a strong model for this purpose. The experimental results on a number of standard vision and language datasets show strong performance improvements over current SOTA. Experimental results on a number of standard image and text classification, as well as preference alignment tasks show strong performance improvements over current SOTA.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths:\n\n1. The idea of adaptively combining weak models to guide the training of a strong model for weak-to-strong generalization under distribution shifts is a simple and intuitive one.\n\n2. The authors take care to handle practical challenges such as collapse / shortcut learning prevalent in optimization under such scenarios by warmup with easy samples, the predictions on which all weak models agree on, a technique known to be effective in the literature on learning with noisy labels. This drawing of connection between the task of weak-to-strong generalization and learning with noisy labels is an interesting one, and can help further cross-over of ideas between the two areas.\n\n3. The experimental results on a number of standard image and text classification, as well as preference alignment tasks show strong performance improvements over current SOTA.\n\n## Weaknesses:\n\n1. The feasibility of faithfully identifying the correct weak model under distribution shift is not clear. Under arbitrary distribution shifts, weak models may have arbitrarily uncertain predictions [1]. Without specific characterizations of the nature of the distribution shift wrt the weak model, general claims about such identifiability of correct weak models under distribution shift are unreliable.\n\n2. Weak models consistently providing the same predictions may not necessarily be a good indicator of reliability, especially under distribution shift. Since all the weak models in the ensemble are trained on the same source distribution, they are more likely to exhibit the same pattern of mistakes when faced with the same of kind of distribution shift.\n\n3. Remark 1 states that with no additional guidance, the strong model converges to putting the highest amount of weight to the best-performing weak model by the end of training. While this is an interesting observation, it is important to reflect further on this. Why does the strong model necessarily need to converge to the best-performing weak model, in the absence of any information about which weak model is actually good? What characteristic does the best-performing weak model have that drives the optimization of the strong model in its direction?\n\nReferences:\n\n[1] Ovadia et al., \"Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift\", NeurIPS 2019.\n\nQUESTIONS:\nPlease see Weaknesses in the Strengths and Weaknesses section.\n\nLIMITATIONS:\nPlease see Weaknesses in the Strengths and Weaknesses section.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces RAVEN, a robust framework for Weak-to-Strong (W2S) Generalization under distribution shifts. W2S generalization is the paradigm where a strong model is trained using supervision from weaker models. The authors observe that naive W2S methods fail under distribution shifts, where the weak model’s training and fine-tuning data distributions differ. RAVEN addresses this by dynamically learning optimal combinations of multiple weak models through adaptive weighting and easy-sample guided initialization. The method is evaluated across image classification, text classification, and preference alignment tasks, showing significant improvements over state-of-the-art baselines, even in out-of-distribution (OOD) settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nFirst, the paper addresses a pressing challenge in scalable model supervision: how to train models when high-quality supervision is unavailable or misaligned. By explicitly modeling distribution shifts—a common but under-explored failure mode in W2S settings—it moves the field toward more realistic and robust solutions. Second, the proposed RAVEN framework is methodologically sound, incorporating adaptive weighting and a principled training schedule that includes easy-sample guided initialization. These innovations allow the model to automatically prioritize reliable weak supervision, leading to statistically significant improvements across multiple tasks. Extensive experiments and ablations strongly support the claims, showing consistent gains over diverse and strong baselines.\n\nWeaknesses\nOne limitation is that the diversity among weak models in RAVEN is achieved only via different random seeds. This simplification might not reflect real-world annotator variance, especially in domains like healthcare or law where supervision quality is tied to domain-specific expertise. Moreover, while the adaptive weighting strategy is powerful, the paper acknowledges challenges in optimizing finer-grained model-sample-specific weights, leaving room for improvement in dynamic supervision. Lastly, the preference alignment experiment, though promising, is narrowly scoped and may not generalize to other types of alignment tasks beyond \"helpfulness\" and \"harmlessness\" datasets.\n\nQUESTIONS:\nSuggestions:\n\n1. The current setup varies only random seeds, which doesn’t capture real-world annotator diversity. Using weak models with different architectures, data sources, or noise types would better simulate practical supervision scenarios and strengthen the method’s robustness.\n\n2. The paper lacks theoretical analysis explaining RAVEN’s behavior under distribution shifts. Adding convergence guarantees or formal justification for adaptive weighting would make the approach more rigorous and broadly credible.\n\n3. The current tasks focus on standard classification and narrow preference alignment. Evaluating on more complex domains like medical diagnosis or reasoning tasks would better demonstrate RAVEN’s generalizability.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}