FLAMES : Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline (2025)

Abstract: Recent works improving LLM math reasoning with synthetic data have used unique setups, making comparison of data synthesis strategies impractical. This leaves many unanswered questions about the roles of different factors in the synthetic data pipeline, such as the impact of filtering low-quality problems. To address this gap, we introduce FLAMES, a Framework for LLM Assessment of Math rEasoning Data Synthesis, and perform a systematic study of 10 existing data synthesis strategies and multiple other factors impacting the performance of synthetic math reasoning data. Our FLAMES experiments provide several valuable insights about the optimal balance of difficulty and diversity of synthetic data. First, data agents designed to increase problem complexity lead to best improvements on most math metrics. Second, with a fixed data generation budget, keeping higher problem coverage is more important than keeping only problems with reliable solutions. Third, GSM8K- and MATH-based synthetic data can lead to improvements on competition-level benchmarks, showcasing easy-to-hard generalization. Leveraging insights from our FLAMES experiments, we design two novel data synthesis strategies for improving out-of-domain generalization and robustness. Further, we develop the FLAMES dataset, an effective blend of our novel and existing data synthesis strategies, outperforming public datasets on OlympiadBench (+15.7), CollegeMath (+4.5), GSMPlus (+6.5), and MATH (+3.1). Fine-tuning Qwen2.5-Math-7B on the FLAMES dataset achieves 81.4% on MATH, surpassing larger Llama3 405B, GPT-4o and Claude 3.5 Sonnet.

## introduction solving math problems is a key measure of evaluating the reasoning ability of large language models (llms) @xcite @xcite * work done while interning at amazon agi foundations. figure 1 : math benchmark scores for popular llms. qwen2.5-math-7b + x (flames or orcamath) denotes results obtained by finetuning qwen2.5-math-7b model with x dataset. comparison of flames data with other public math datasets is shown in tables 3 , 4 . @xcite . due to the challenges of creating a large-scale human-crafted dataset, llmbased generation of synthetic data has been explored and proven effective in improving llm math reasoning capabilities @xcite @xcite . while early works on math reasoning data synthesis focused on synthesis of solutions (reasoning traces) for existing problems, e.g. rejection sampling @xcite , recent works have shifted focus towards increasing problem difficulty and diversity by synthesizing new problems. recent works such as orca-math @xcite , metamath @xcite , and openmath-instruct-2 (toshniwal et al., 2024a) have proposed one or more strategies foot_0 for generating new math problems. however, these works often use different setups (see figure 2 ), making the comparison of data synthesis strategies across studies infeasible. this lack of standardization deprives researchers and practitioners of practical insights, leaving them uninformed about the real factors driving performance improvements detailed version is shown in appendix figure 5 via math reasoning data synthesis. figure 2 summarizes setups of four popular math reasoning data synthesis works in terms of four experimental factors, highlighting the lack of standardization. as different problem synthesis models and student models have been used in the orcamath and scalequest papers, for example, it is infeasible to conclude whether suggester-editor @xcite is a better data synthesis strategy or question fine-tuning (qft) @xcite . this lack of standardization poses several important research questions about math reasoning data synthesis, which are currently under-studied. rq1: how do different strategies of data quality control affect the performance of synthetic data, and what strategy works the best? rq2: which data synthesis strategy performs optimally for improving student model math reasoning? rq3: with a fixed compute budget for synthetic problem generation, should we mix data from multiple strategies or just scale the best strategy? rq4: how does the choice of problem and solution generation models impact performance with math synthetic data? to study these open research questions, we propose the framework for llm assessment of math reasoning with data synthesis (flames). figure 3 shows a flow diagram of the flames framework. in this framework, we create a comprehensive list of the factors that may impact performance within the synthetic data pipeline, and we choose fixed values for them based on existing literature and our experimental findings. the flames framework enables controlled experiments where we vary only one factor, keeping other factors fixed. we perform a controlled study of ten existing data synthesis strategies, six data quality control strate-gies, two problem generation models, and two solution generation models. these experiments provide many valuable insights into data synthesis for llm math reasoning. first, we find that data agents designed to increase problem complexity lead to best improvements on most math metrics. second, with a fixed data generation budget, keeping higher coverage of synthetic problems (even with some inaccuracy) is more important than keeping only problems with reliable solutions. third, synthetic data generated based on gsm8k and math can lead to improvements on competition-level benchmarks, showcasing easy-to-hard generalization capability. fourth, choice of solution generation model impacts student model performance more than choice of problem generation model. leveraging the findings from our study of existing data synthesis strategies ( further, we study the impact of mixing data from different agents, and design an effective blend of agent data. we show empirically that this blend (flames small) leads to balanced performance across five evaluation datasets, with performance surpassing all ten existing agents on ood and competition-level benchmarks. we scale this blend to construct the flames large (1m) and flames xl (1.5m) datasets. our experiments across five student models ( in summary, our paper makes four contributions. 1. we propose the flames framework for controlled study of multiple factors involved in the math synthetic data pipeline. this enables systematic comparison of existing and future math data agents. ## the flames framework figure 3 shows a flow diagram of the flames framework. there are multiple factors that play key roles in final student model performance after finetuning with synthetic data. as part of the flames framework, we list these factors and provide their standard values based on either controlled experiments or existing literature. we highlight the factors for which we perform controlled experiments in figure 3 , and discuss those briefly in this sec-tion. we refer readers to appendix a for a detailed discussion of these factors. problem synthesis agents: math data synthesis agents interact with one or more seed problems and a problem generation model to synthesize novel math problems. these data agents play a crucial role in the synthetic math data pipeline, and are the focus of many recent works @xcite @xcite . in our work, we use the train splits from the popular gsm8k @xcite and math @xcite data for seed problems. we discuss details of 10 existing and 2 novel data agents in section 3. quality control: all problems generated in the flames framework are first deduplicated using exact match. we remove any synthetic problem if there is a high amount of token overlap between the synthetic problem and a test set problem. we study additional data quality control techniques in section 4. student model: choice of student model plays a crucial role for studying llm math reasoning improvements with synthetic data. deepseek-math-7b (shao et al., 2024) is a relatively-performant non-instruction-tuned base model, and is commonly used for measuring llm math reasoning with synthetic data @xcite . we use this model as the underlying student model for the flames framework. problem synthesis model: to generate problems, data agents interact with qwen2.5-32b-instruct @xcite , which is chosen based on a controlled experiment (see section 5.5). ## data synthesis agents we select 10 existing problem generation agents based on recent works in math data synthesis. based on the primary objective of these agents, we categorize them as belonging to one of four categories: 1) in-domain practice, 2) in-domain complexity enhancing, 3) robustness enhancing, and 4) out-of-domain enhancing. additionally, based on experimental findings detailed in section 5.2, we propose two novel agents under the categories of robustness enhancing and out-of-domain enhancing. we briefly describe these 12 agents here, and refer readers to appendix c for details. ## data quality control a challenge for llm-based math problem synthesis is in controlling the quality of synthetic data, i.e. the removal of unsolvable or ill-formatted problems and incorrect solutions. existing works for math data synthesis primarily rely on proprietary models like gpt-4 for problem and solution generation and assume that generated data is of good quality. consequently, the impact of data quality control for math synthetic data has been an understudied problem. our work is the first to study this factor in a principled way and provide insights on what works best for math synthetic data. recent work @xcite has introduced the use of an llm-based solvability filter to remove ill-formatted or unsolvable problems and a reward model (internlm2-7b-reward) to select the preferred solution. self-consistency @xcite is another viable alternative for ensuring problem and answer quality, where multiple (three in our case) solutions are generated for a problem with temperature sampling and only those problems are kept where at least a pre-determined number of the solutions lead to the same answer. we use these to design six data filtering strategies for our experiments. in strict self-consistency, only problems with a matching answer in all 3 solutions are kept and one solution is randomly selected. majority self-consistency (or majority) keeps problems where at least 2 solutions contain matching answers, and one of those solutions is randomly selected. solvability + rm first filters problems which are deemed "unsolvable foot_6 " by the qwen2.5-math-7b-instruct model, then selecting the solution which receives the highest reward according to the internlm2-7b-reward reward model (rm) @xcite . majority + first is similar to majority self-consistency, but uses the first solution in case no majority is found. this method doesn't remove any problems as part of the data quality control. solvability + first filters unsolvable problems, then includes the first solution for each problem considered solvable. first keeps the first generated solution for each synthetic problem, keeping all the problems. we note that these different techniques are designed to remove noisy synthetic data, but may also end up removing some legitimate problems and solutions, especially harder problems. we choose these six strategies to evaluate a diverse array of coverage, difficulty and accuracy levels in filtered synthetic datasets. we discuss findings of our systematic study of these strategies in section 5.1. ## experimental results here we explore quality control methods for synthetic math reasoning data (section 5.1). we utilize the flames framework to compare math reasoning data synthesis strategies (section 5.2). we introduce and show the effectiveness of the flames datasets compared with existing math datasets (section 5.3 and section 5.4). last, we evaluate the relative impact of problem and solution generation teacher models (section 5.5). ## conclusion we introduced the flames framework, which enables fine-grained analysis of the math data synthesis pipeline. we performed a controlled study of 12 data synthesis strategies (including 2 novel strategies), 6 data quality control strategies, 2 problem generation models, and 2 solution generation models, providing valuable insights for improving llm math reasoning with synthetic data. we studied the impact of mixing data from different agents and designed an effective blend (the flames datasets) without use of any proprietary models. our rigorous evaluations establish drastic improvements with our proposed flames datasets over public math datasets.