Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis
  from Scratch
Authors: name: Yuyang Ding; name: Xinyu Shi; name: Xiaobo Liang; name: Juntao Li; name: Qiaoming Zhu; name: Min Zhang
Published: 2024-10-24 12:42:04

Abstract: The availability of high-quality data is one of the most important factors in
improving the reasoning capability of LLMs. Existing works have demonstrated
the effectiveness of creating more instruction data from seed questions or
knowledge bases. Recent research indicates that continually scaling up data
synthesis from strong models (e.g., GPT-4) can further elicit reasoning
performance. Though promising, the open-sourced community still lacks
high-quality data at scale and scalable data synthesis methods with affordable
costs. To address this, we introduce ScaleQuest, a scalable and novel data
synthesis method that utilizes "small-size" (e.g., 7B) open-source models to
generate questions from scratch without the need for seed data with complex
augmentation constraints. With the efficient ScaleQuest, we automatically
constructed a mathematical reasoning dataset consisting of 1 million
problem-solution pairs, which are more effective than existing open-sourced
datasets. It can universally increase the performance of mainstream open-source
models (i.e., Mistral, Llama3, DeepSeekMath, and Qwen2-Math) by achieving 29.2%
to 46.4% gains on MATH. Notably, simply fine-tuning the Qwen2-Math-7B-Base
model with our dataset can even surpass Qwen2-Math-7B-Instruct, a strong and
well-aligned model on closed-source data, and proprietary models such as
GPT-4-Turbo and Claude-3.5 Sonnet.