M ath G enie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLM s (2024)

Abstract: Large language models (LLMs) have exhibited great potential in mathematical reasoning. However, there remains a performance gap in this area between existing open-source models and closed-source models such as GPT-4. In this paper, we introduce MathGenie, a novel method for generating diverse and reliable math problems by leveraging the ground-truth solutions of the seed data. We augment these ground-truth solutions and use a specially finetuned model to translate these augmented solutions back into new questions. Subsequently, we generate code-integrated solutions for these questions. To ensure the correctness of the code-integrated solutions, we employ rationale-based verification for filtering. Then, we finetune various pretrained models, ranging from 7B to 70B, on the newly curated data, resulting in a family of models known as MathGenie. These models consistently outperform previous open-source models across five representative mathematical reasoning datasets, achieving state-of-the-art performance. In particular, MathGenie-InternLM2 achieves an accuracy of 87.7% on GSM8K and 55.7% on MATH, securing the best overall score.

## introduction large language models (llms), such as @xcite , and others @xcite @xcite , have demonstrated outstanding capabilities in mathematical reasoning. existing methods explored three main types of solution formats for tackling the math-solving problem: text-only chain-of-thought (cot) solutions @xcite , codeonly program-of-thought (pot) solutions (chen * equal contribution † corresponding author @xcite , and code-integrated solutions @xcite . among these, code-integrated solutions demonstrate superior performance over both cot and pot solutions @xcite @xcite , indicating their effectiveness in enhancing problem-solving abilities. in this paper, we focus on creating diverse and reliable augmented questions and ensuring the reliability of the generated code-integrated solutions, thereby better finetuning pretrained base models. existing works, such as tora @xcite and mathcoder @xcite , build codeintegrated solutions and augment math questions using off-the-shelf gpt-4. however, scaling up the training data with gpt-4 becomes prohibitively expensive. consequently, developing a free open-source model to generate large-scale synthetic data presents a promising alternative, offering scalability and cost-effectiveness @xcite . to effectively scale-up math problem-solving data, we focus on handling two critical challenges: (1) how to generate high-quality and diverse math problems to aid in generalization, and (2) how to generate accurate and reliable solutions for the augmented problems without human-annotated ground truth, preferably in a code-integrated format. a unified framework, mathgenie, is proposed, which consists of three components as shown in fig. 1 to tackle the abovementioned challenges: iterative solution augmentation, question back-translation and verification-based solution filtering. iterative solution augmentation and question back-translation aims to generate diverse and reliable math problems. unlike direct question augmentation @xcite , the proposed math problem back-translation leverages the constraints and logical relationships inherent in mathematical solutions to create a diverse and high-quality set of new math problems. specifically, we iteratively augment the human-annotated solutions from the relatively small training sets of math @xcite and gsm8k @xcite this method draws inspiration from instruction backtranslation @xcite , which back-translates instructions from texts in a web corpus. however, the key difference is that our source solutions for back-translation are augmented from existing ones to ensure the reliability and solvability of the augmented questions. these newly generated math problems lack reliable ground-truth solutions, which necessitates the proposed verification-based solution filtering. we first build a model, m code , capable of generating code-integrated solutions and verifying these solutions. then, code-integrated solutions of the new questions are generated with this model. to enhance the reliability of these code-integrated solutions, we use m code to verify the model-generated solutions by generating verification rationales for them, as demonstrated in step 3 of fig. 1 . the verification rationales use interleaved natural language and code to verify the correctness of the solutions, as shown in tab. 13 and tab. 14 in appendix g. only the solutions verified to be correct are retained, thus improving the accuracy and quality of the generated data. based on the proposed mathgenie framework, we obtain a large-scale model-generated math problem-solution dataset, mathgeniedata, featuring diverse augmented questions and reliable codeintegrated solutions. to evaluate the effectiveness of the questionsolution augmentation framework mathgenie, we finetune various state-of-the-art pretrained models, ranging from 7b to 70b. this results in math-genielm, a new family of math models with excellent performance. our models demonstrate high accuracy across five diverse and representative math datasets: math, gsm8k, svamp, simuleq, and mathematics. in particular, mathgenielm-internlm2 achieves an accuracy of 87.7% on gsm8k and 55.7% on math, achieving the best overall score. when enhanced by majority voting, mathgenielm-llama-2-70b attains a 10-path accuracy rate of 91.7% on gsm8k and 63.3% on math. the main contributions of this paper are summarized as follows: (1) we propose the math-genie pipeline, which is designed to enhance the scale, diversity, and quality of synthetic math questions, as well as to improve the accuracy of the augmented question: ann's favorite restaurant was having a winter sale. for @xmath015 each and 2 desserts worth $20 each. she also bought 4 appetizers, all at the same price. how much did each appetizer cost? figure 2 : comparison between direct question augmentation (left) and iterative solution augmentation and question back-translation (right). direct question augmentation damages the hidden constraints between the conditions (cost of part of the things bought must not be more than the total cost), thus producing a question with no answer. question back-translation considers the solution, and correctly augments the question. ## mathgenie in this section, we introduce mathgenie, a pipeline for creating diverse and reliable math problems through back-translation, and curating high-quality code-integrated solutions through verification. we begin by introducing the seed data and solution generator model. next, we present the proposed math-genie pipeline, which consists of three key steps, as shown in fig. 1 : iterative solution augmentation, question back-translation, and verification-based solution filtering. seed data. the seed data consists of two parts: (1) the first part is used for data augmentation, consisting of 15k math problems and human-annotated solutions from the training sets of gsm8k and math. we denote it as d @xmath1, s i )} n @xmath2, where q i is the i-th question, s i is its natural-language solution, and n is the total number of cases. (2) the second part is used for training our candidate solution generator model, which serves to generate candidate solutions for the augmented questions. we denote this part of the seed data as d @xmath3, s i code , v i code )} n @xmath4, where q i is the question, s i code is its code-integrated solution, and v i code is the code-integrated verification rationales for the question-solution pair. it contains 80k samples of code-integrated solutions for problems in gsm8k and math, as well as codeintegrated verification rationales for these solutions. multiple different solutions are collected for each question. we acquire these solutions and verification rationales using the gpt-4 code interpreter, which consist of interleaved natural language and code. ## related works large language models for mathematical reasoning. llms have demonstrated remarkable performance in mathematical reasoning tasks. cot @xcite enhances llms' capability for multistep reasoning. self-consistency @xcite selects the final answer through majority voting. csv @xcite introduces codebased self-verification. other research efforts focus on pretraining or finetuning llms, thereby producing math-specific llms, such as llemma @xcite , wizardmath @xcite , mammoth @xcite , tora @xcite , and mathcoder @xcite . it is noted that code-integrated models @xcite have shown superior capabilities over cot-style models. this paper develops synthetic math problems and solutions using free models to enhance mathematical reasoning. instruction-following datasets for llms. recent studies @xcite @xcite have begun to utilize synthetic instructions generated by llms, such as gpt-4 or gpt-3.5, to distill into smaller models. wizardlm @xcite proposes complex instructions to enrich the seed data for general chat models. this paper, however, focuses on math problem augmentation, particularly for code-integrated math-specific models. ## conclusion in this paper, we propose a coordinated pipeline consisting of iterative solution augmentation and question back-translation to produce large-scale synthetic math questions, and verification-based solution filtering to filter the generated codeintegrated solutions. combined, these three components effectively create new questions and ensure the reliability of the corresponding code-integrated solutions. experiments show that mathgenielm achieves superior performance across five math problem-solving benchmarks and on six different pretrained base models, offering insights into the development of math problem-solving models and providing hope for extension to other reasoning tasks.