M uggle M ath: Assessing the Impact of Query and Response Augmentation on Math Reasoning (2024)

Abstract: In math reasoning with large language models (LLMs), fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective, profoundly narrowing the gap between open-sourced LLMs and cutting-edge proprietary LLMs. In this paper, we conduct an investigation for such data augmentation in math reasoning and are intended to answer: (1) What strategies of data augmentation are more effective; (2) What is the scaling relationship between the amount of augmented data and model performance; and (3) Can data augmentation incentivize generalization to out-of-domain mathematical reasoning tasks?To this end, we create two new dataset AugGSM8K and AugMATH, by complicating and diversifying the queries and sampling multiple reasoning paths from GSM8K and MATH.We obtained a series of LLMs called MuggleMath by fine-tuning LLaMA models on AugGSM8K and AugMATH. MuggleMath substantially achieves new state-of-the-art on GSM8K and MATH.A log-linear relationship and a segmented log-linear are presented between MuggleMath’s performance and the amount of augmented data on GSM8K and MATH, respectively.We also find that it is weak in out-of-domain math reasoning generalization from AugGSM8K to MATH and from AugMATH to GSM8K, which suggests that augmenting queries that cover a broader range of subjects is more beneficial for generalization.

## introduction the emergence of large language models (llms) @xcite @xcite has profoundly revolutionized the field of natural language processing, exhibiting versatile performance in various tasks like code generation @xcite @xcite , instruction following @xcite @xcite , long context question answering @xcite , and math reasoning @xcite @xcite . math reasoning as a representative reasoning task is widely studied to access the reasoning abilities in llms @xcite . proprietary llms, such as gpt-3.5, and gpt4 (openai, 2023) have shown exceptional mathematical reasoning abilities, while there remains a substantial gap between open-source llms, such as gpt-j @xcite and llama @xcite ) and the cutting-edge proprietary models. to enable better mathematical reasoning abilities in open-sourced llms, they generally undergo a fine-tuning stage on supervised reasoning datasets. a series of efforts are committed to enhancing the mathematical reasoning capabilities of open-source llms, where a mainstream approach involves first augmenting new mathematical problems and answers, followed by supervised fine-tuning on the augmented dataset @xcite @xcite . this type of approach has achieved good results, and in this paper, we would like to explore what are the key factors affecting the effectiveness of data augmentation for mathematical reasoning tasks and the scaling relationship between the amount of data augmentation and model performance. specifically, with the help of proprietary models (gpt-3.5 and gpt-4), we applied five types of mathematical problem augmentation methods based on human experience in creating variations of mathematical problems similar to @xcite . we further generated multiple reasoning paths for each augmented problem since distinct reasoning paths can also enhance chain-of-thought reasoning @xcite @xcite . we obtained two new datasets called auggsm8k and augmath after data augmentation on two widely used mathematical reasoning datasets gsm8k @xcite and @xcite . by supervised fine-tuning on the open-source llama @xcite and llama-2 @xcite llms on auggsm8k and augmath, we obtained a series of models dubbed mugglemath. we find that with sufficient amounts of data, mug-glemath achieves a new state-of-the-art on gsm8k and math. in addition to this, we find a loglinear relationship between the performance of mugglemath and the amount of data augmentation over a range of data volumes on gsm8k and a segmented log-linear relationship on math. although mugglemath achieves strong performance on the gsm8k and math test set, the rationales for performance improvement by data augmentation remain unclear. we are therefore interested in the specific reason behind the performance improvement and whether it brings enhancement in llms' mathematical reasoning capabilities generally. to validate the generalization of mugglemath, we conduct multi-task learning and analyze the transferability with auggsm8k and augmath. we found that llms trained with supervised learning after data augmentation on gsm8k only bring marginal improvements to performance on math and the similar conclusion is fit for augmath and gsm8k. by visualizing the data distribution in the embedding space of llama-2-7b, we observe that the embedding distribution of problems in auggsm8k is very close to that of gsm8k, but significantly different from the problem distribution in the math dataset. the reason behind can be attributed to the fact that gsm8k and math have different reasoning difficulty, response style and require different mathematical knowledge. the main contributions of our work can be summarized as follows: data augmentation for llm data augmentation is a common technique to improve downstream task performance in nlp @xcite . in the era of large language models, data augmentation is usually used for generating instruction following sft datasets @xcite @xcite . queries @xcite and responses @xcite of sft datasets can both be augmented by prompting state-of-the-art proprietary llms. compared with their work, we are concentrated on augmenting math sft dataset and we are more interested in scaling relationships on in-domain and out-ofdomain generalizations. ## experiments we first introduce our experimental setup in section 3.1 and dataset augmentation method in section 3.2. then we conduct analyses spanning several aspects of data augmentation to answer the three research questions in abstract. for space saving, we mainly analyze the augmentation on gsm8k and detailed discussion of augmentation on math are list in appendix a. ## conclusion in this paper, we investigate the scaling property of query and response augmentation with respect to math reasoning in-domain and out-of-domain performance. we find that query and response augmentation can improve in-domain performance very effectively which has a similar improvement to human-written query-response pairs augmentation. although we can obtain state-of-the-art indomain performance by such augmentation, we find that the math reasoning ability improved by the in-domain augmented data is hard to generalized to out-of-domain datasets. therefore, when using synthetic data, increasing the diversity of the queries to improve the overall mathematical reasoning ability of the llms is crucial. in the future, how to use synthetic data to enhance the overall mathematical reasoning ability of the model is a very important research topic.