W eak2 W ise: An Automated, Lightweight Framework for Weak- LLM -Friendly Reasoning Synthesis (2025)

Abstract: Recent advances in large language model (LLM) fine‐tuning have shown that training data augmented with high-quality reasoning traces can remarkably improve downstream performance. However, existing approaches usually rely on expensive manual annotations or auxiliary models, and fail to address the unique constraints of smaller “weak” LLMs. To bridge these gaps, we introduce Weak2Wise, a fully automated, lightweight framework for synthesizing high‐quality, weak-LLM-friendly reasoning traces. Starting from a QA dataset, Weak2Wise filters out the samples that can already be correctly answered by the weak LLM, gathers diverse candidate reasoning traces from multiple strong LLMs, and leverages our Step‐Mask scoring to rank and truncate the most guidance‐effective traces. These reasoning traces are then used for fine‐tuning, yielding substantial improvements in the weak LLM’s reasoning abilities. The name Weak2Wise has two meanings: using a “weak” LLM to select the “wisest” reasoning traces generated by stronger LLMs, and fine‐tuning the same weak LLM on these reasoning traces to become “wiser”. We further use Weak2Wise to build GR-1K, a 1,000‐sample math and science QA‐reasoning dataset optimized for weak LLMs, and fine‐tune Qwen2.5‐7B on it to create GR‐7B, which achieves superior performance on AIME2024, MATH‐500, and GPQA Diamond benchmarks. Our codes are publicly released to facilitate further research.

## introduction the quality of training data plays a critical role in the fine-tuning of large language models (llms). during the fine-tuning stage, the inclusion of reasoning traces in the training data leads to notable improvements in the reasoning capabilities of llms @xcite ; given a question and its answer, different strong llms generate various candidate reasoning traces. the key issue is to identify the reasoning trace that is truly suitable for fine-tuning a weak llm. note that the mathematical question in the figure is merely used to illustrate the pipeline, while actual questions we used are much more difficult. @xcite . a high-quality dataset can significantly enhance the performance of an llm in a specific domain, even with a limited number of examples @xcite . to obtain high-quality reasoning traces for finetuning, researchers have explored automated methods beyond costly manual annotations. some methods @xcite @xcite automate the majority of the pipeline, yet still rely on manual intervention at critical steps .other methods @xcite @xcite achieve full end-to-end automation but require training additional models, which in turn compromises the lightweight design. however, weak llms 2 often follow different reasoning patterns from strong llms' 3 , making them unable to interpret certain traces @xcite @xcite . only few methods @xcite note this gap and adjust their synthesis processes to fit reasoning patterns of weak llms. but they merely replicate these patterns rather than truly integrate weak llms into the synthesis pipeline. this raises a natural and important question: how can we synthesize high-quality reasoning traces with an automated and lightweight method, while ensuring that these reasoning traces are truly friendly to weak llms? to address this challenge, we propose weak2wise, an automated and lightweight framework for weak-llm-friendly reasoning synthesis. the key issue is to identify the reasoning trace that is truly suitable for fine-tuning weak llms, as illustrated in figure 1 . unlike prior methods that rely on external reward models or multi-agent coordination, weak2wise leverages the weak llm's own performance to evaluate reasoning traces. the core idea is to evaluate each candidate reasoning trace's guidance effectiveness using a novel step-mask scoring: incrementally masking partial reasoning steps and querying the weak llm itself reveals how well each trace aids the weak llm in reaching the correct answer. weak2wise offers several distinct advantages: i) full automation: the entire pipeline is fully automated, requiring no human intervention; ii) lightweight design: no additional reward models or agent-based frameworks are required, keeping the entire process simple and scalable; and iii) weak-llm-friendly: by leveraging the weak llm's own behavior during evaluation, synthetic reasoning traces are adapted to reasoning patterns of weak llms. we evaluate weak2wise on some of the most challenging mathematical and scientific reasoning benchmarks, including aime2024 (maxwell-jia, 2024), math-500 @xcite , and gpqa diamond @xcite . applied to the weak llm qwen2.5-7b @xcite , our framework achieves consistent and substantial improvements across all datasets. ablation studies further verify the effectiveness of the step-mask scoring and truncation strategies. in summary, we make the following major con3 we define strong llms as large-parameter models with demonstrated high reasoning performance. to the best of our knowledge, this is the first framework to incorporate the weak llm itself into the reasoning synthesis process. ## conclusion in this paper, we introduce weak2wise, a fully automated, lightweight framework for synthesizing high-quality, weak-llm-friendly reasoning traces. our framework innovates by leveraging the weak llm's own performance to evaluate reasoning traces with a novel step-mask scoring mechanism. through experiments on challenging benchmarks, we demonstrated that weak2wise effectively synthesize reasoning traces for fine-tuning weak llms. weak2wise offers an efficient solution for improving reasoning in llms, making high-quality reasoning more accessible for practical applications.