M u M ath-Code: Combining Tool-Use Large Language Models with Multi-perspective Data Augmentation for Mathematical Reasoning (2024)

Abstract: The tool-use Large Language Models (LLMs) that integrate with external Python interpreters have significantly enhanced mathematical reasoning capabilities for open-source LLMs, while tool-free methods chose another track: augmenting math reasoning data. However, a great method to integrate the above two research paths and combine their advantages remains to be explored. In this work, we firstly include new math questions via **mu**lti-perspective data augmenting methods and then synthesize **code**-nested solutions to them. The open LLMs (e.g., Llama-2) are finetuned on the augmented dataset to get the resulting models, **MuMath-Code** ( \mu -Math-Code). During the inference phase, our MuMath-Code generates code and interacts with the external python interpreter to get the execution results. Therefore, MuMath-Code leverages the advantages of both the external tool and data augmentation. To fully leverage the advantages of our augmented data, we propose a two-stage training strategy: In Stage-1, we finetune Llama-2 on pure CoT data to get an intermediate model, which then is trained on the code-nested data in Stage-2 to get the resulting MuMath-Code.Our MuMath-Code-7B achieves 83.8% on GSM8K and 52.4% on MATH, while MuMath-Code-70B model achieves new state-of-the-art performance among open methods—achieving 90.7% on GSM8K and 55.1% on MATH. Extensive experiments validate the combination of tool use and data augmentation, as well as our two-stage training strategy.We release the proposed dataset along with the associated code for public use: https://github.com/youweihao-tal/MuMath-Code.

## introduction in natural language processing (nlp), large language models (llms) (radford et al., 2019; † equal contribution. § work done while the author was interning at tal. * corresponding author. mumath-code exhibits a substantial improvement in performance on both gsm8k @xcite and math @xcite , relative to the previous approaches. @xcite especially the proprietary ones such as gpt-4 (ope-nai, 2023a) and claude-3 (anthropic, 2024) have demonstrated superiority in a variety of tasks, e.g., text classification @xcite @xcite , automated coding @xcite , instructions following @xcite , and math problem solving @xcite @xcite . among these tasks, the capability to han-dle math problems stands as a typical and critical criterion for the evaluation of different llms. however, a significant performance disparity is observed between open-source llms, for instance, llama @xcite , and their proprietary counterparts, when it comes to mathematical reasoning ability. in recent years, many scholarly publications have been directed towards improving the mathematical proficiency of llms, which can be categorized into two distinct research trajectories: those that purely rely on natural language reasoning and those that incorporate external tools. the former methods are tool-free, mainly depends on data augmentation to enhance the models' mathematical reasoning capability, while the second trajectory (namely tool-use llms) are often coupled with external python interpreters. from the perspective of knowledge distillation @xcite @xcite @xcite , both mainstream approaches transfer math reasoning abilities from the powerful teacher models (for instance, gpt-4) to the inferior open foundation models. the tool-free methods synthesize a large number of new math problems and corresponding solutions, taking the original training math qa pairs as the initial data seeds. scaling law theoretically provides the basis for the ongoing improvement of llms' performance by constantly incorporating new training data. representative approaches are rft @xcite , metamath @xcite , wizard-math @xcite , mugglemath @xcite , mumath @xcite , etc. as for the second trajectory, code executors substantially enhance llms in particularly challenging computational and logical tasks, thereby alleviating the problem-solving burden on them. this tooluse category is exemplified by pal @xcite , @xcite , mammoth @xcite , tora @xcite and math-coder @xcite . although the aforementioned research paths have been individually successful, to date, few methods have been developed that amalgamate their respective advantages. in this paper, we propose a novel method that integrates tool usage with data augmentation to synthesize a large amount of multi-perspective mathematical questions and solutions (we employ the augmenting methods introduced in a previous work mumath @xcite ). specifically, we utilize proprietary llms (like gpt-4) to generate python code while synthesizing new solutions to math problems, and then fine-tune the open-source models (e.g., llama) on the augmented dataset. the resulting model, mumath-code, is thus equipped with the ability to write code for math problem solving. during the inference phase, our mumath-code can generates both cot @xcite reasoning texts and python code blocks. these code blocks are then extracted and executed by an external python interpreter, and the execution results are returned to mumath-code for subsequent rounds of cot reasoning or code generation until the final result is obtained or the maximum number of execution rounds is reached. the multi-perspective mathematical question set comprises questions augmented via rephrasing @xcite , alteration @xcite , fobar @xcite , bf-trans @xcite , besides those from the original training sets. regarding the solutions nested with python code, we leverage a general pattern like the ones used in tora @xcite and mathcoder @xcite : cot-pot interleaving. however, we propose prefix cot, code debugging and pseudo-answer guidance filtering to improve the consistency and quality of our augmented solutions. the prefix cot is a thoughtful analysis in pure natural language before code generation, making the llms consider this analysis while generating all the subsequent content, which thus are helpful for the models to learn the whole solution. besides, we prompt gpt-4 to debug and correct the inexecutable code when requesting the solutions, and we keep the faulty code since this process of verification and correction can help boost the models' coding proficiency. furthermore, for those synthesized questions via alteration, which lack ground truth answers as filtering guidance, we choose the majority-voting answers as the pseudo-answers. this process can increase the correctness of the generated solutions and thus improve the data quality generally. we name the proposed dataset as mumath-code-data and denote it as d µ-code . moreover, previous tool-use llms for math are derived by directly finetuning on code-nested data, which thus fail to fully harness the intrinsic natural language reasoning capability of the llms themselves. different from the other tool-use methods, we design a two-stage training strategy to better combine the advantages of data augmenta-tion and external code execution. the first stage is to enhance the models' pure language mathematical reasoning, where the largest (751k) dataset proposed in mumath (here called mumath-data and denoted as d µ ) is utilized to finetune llama, and get an intermediate model, mumath. in the second stage, we continue finetuning mumath on mumath-code-data to equip the model with the ability to write code for solving math problems. the resulting model, mumath-code, is thus can be prompted to leverage the python interpreter to execute its generated code for securing the desirable outputs at inference time. our contributions are summarized as follows: ## methodology we employ the augmented questions from mu-math @xcite , detailed in appendix a, and synthesize code-nested solutions to them. to help the models better learn such solutions with multi-turn code generation, code execution and pure natural language reasoning, we propose prefix cot, code debugging, and pseudo-answer guidance filtering to augment the quality of the synthetic data, as well as a two-stage training strategy. figure 2 delineates the overall pipeline. ## conclusion in this paper, we propose a multi-perspective and code integrated math reasoning dataset called mumath-code-data, where each solution contains multi-turn code generation, code execution and pure natural language analysis (cot). through a two-stage training strategy, our mumath-code models outperforms the state-of-the-art open methods and even some powerful proprietary ones across different scales on the in-domain reasoning datasets (i.e., gsm8k and math) as well as those out-of-domain ones. additionally, ablation studies demonstrates the effectiveness of our three novel methods for the data synthesis: prefix cot, code debugging and pseudo-answer guidance filtering. our work represents a new attempt at integrating mathematical question augmentation (tool-free) with code generation and execution (tool-use) to enhance the mathematical reasoning capabilities of llms, and we hope it can inspire subsequent research endeavors.