Math for AI: On the Generalization of Learning Mathematical Problem Solving

Venue: NeurIPS_cc_2024_Workshop_MATH-AI
Published: 2024-09-26 10:33:23

Abstract: There has been a growing interest in enhancing the mathematical problem-solving (MPS) capabilities of LLMs. While some researchers focus on developing specialized math models to advance AI for math, others study mathematical reasoning with a ''math for AI'' perspective, positing that integrating mathematical reasoning data could enable LLMs to perform complex reasoning more broadly. This hypothesis draws from neuroscience studies which show that solving mathematical problems aids in the development of general reasoning skills in humans. The concept of ''math for AI'' has gained particular relevance as the research community increasingly focuses on complex reasoning -- Given the scarcity of complex and lengthy chain-of-thought data, MPS emerges as a prime candidate for collecting or synthesizing substantial volumes of intricate thought processes, thus serving as a potential key resource for enhancing general complex reasoning. However, it remains unclear whether skills acquired through learning MPS can extend to other reasoning tasks or merely improve MPS-specific benchmark scores. 
In this paper, we present a comprehensive empirical analysis to address this question.
Specifically, we explore three prevalent methods for improving MPS: (1) continual pretraining on mathematical text; (2) instruction pretraining on large-scale QA pairs synthesized from raw text; and (3) instruction tuning on MPS datasets. 
Through controlled experiments and evaluations across seven distinct reasoning domains, we find that extensive continual pretraining on mathematical texts and instruction pretraining on diverse QA pairs can improve performance on more non-MPS reasoning tasks generally. 
However, instruction tuning on benchmark-oriented datasets to enhance MPS performance fails to yield significant gains in broad reasoning tasks. 
These findings indicate that most readily available data sources do not support the ''math for AI'' objective in enhancing non-MPS tasks. Identifying which data sources best contribute to the acquisition of complex reasoning skills remains a crucial question for future research.

Main paper: 
 introduction: 
 Cognitive neuroscience research has consistently demonstrated that learning to solve mathematical problems enhances general reasoning abilities in humans, as engaging in mathematical problemsolving promotes logical thinking, abstract reasoning, and transferable problem-solving strategies across various domains (Dehaene et al., 2004; Hawes & Ansari, 2020) . This notion -that learning math fosters the development of general reasoning skills -points toward a "math for AI" vision, where incorporating mathematical reasoning data into AI training could help large language models (LLMs) develop more complex and versatile reasoning abilities. The "math for AI" goal is particularly relevant to recent attentions to complex reasoning abilities of LLMs (OpenAI, 2024) , as mathematical problem-solving (MPS) is one of the few domains where large volumes of long and intricate CoT data can be generated or synthesized (Tang et al., 2024; Lu et al., 2024) , making it a valuable data source to potentially learn complex reasoning. However, while numerous models have been developed to tackle mathematical problem-solving (Cobbe et al., 2021b; Yu et al., 2023; Luo et al., 2023a) , their evaluations focus narrowly on benchmarks like GSM8K (Cobbe et al., 2021a) and MATH (Hendrycks et al., 2021b) , and it is unclear whether these approaches and the accompanied datasets can really help learn other types of reasoning. Therefore, these works, whether intentional or not, fall within the "AI for math" scope and fail to demonstrate their impact for the "math for AI" objective. Thus, a key question remains: Does learning mathematical problem-solving contribute to the development of a model's general reasoning abilities, or does it merely enhance performance on MPS benchmarks?In this study, we conduct empirical analysis focusing on this central question. Specifically, we explore whether training LLMs on mathematical problem-solving tasks can help broader reasoning tasks beyond mathematics. We first identify three common training strategies to enhance LLMs' capabilities in solving mathematical problems: (1) Continual pretraining on mathematical text involves extending the pretraining of LLMs on large-scale mathematical text to enhance their adaptability to the mathematical domain, such as RhO-Math (Lin et al., 2024) and Deepseek-Math (Shao et al., 2024) . ( 2) Instruction pretraining on diverse QA pairs is a method focused on training models using diverse question-answer pairs from raw texts, typically encompassing various formats and types of math problems (Yue et al., 2024; Cheng et al., 2024) . ( 3) Instruction tuning on MPS datasets involves fine-tuning models on MPS datasets. This is the most common method adopted to learn mathematical problem-solving and lead to state-of-the-art performance (Yu et al., 2023; Gou et al., 2023; LI et al., 2024; Tong et al., 2024) .We perform control experiments and evaluate a series of model created by the three training strategies above, where the models are either from open-source checkpoints or our own training. We assess these models across multiple benchmarks involving MPS benchmarks and six types of non-MPS reasoning: mathematical reasoning (excluding problem-solving) , STEM reasoning, logical reasoning, commonsense reasoning, symbolic reasoning, and agent reasoning. When trained exclusively on mathematical texts, we observed that models tend to lose their ability to follow general instructions and become limited to performing only math-related tasks. To mitigate this effect, we also incorporated general chat-based data into the training process. This approach simulates a realistic development scenario where math-related training is integrated as part of broader model training, rather than isolating it to create a model solely capable of MPS tasks.Our experimental results reveal that continual pretraining on raw mathematical texts enhances performance across a broader range of reasoning tasks. However, as we transition from continual pretraining to instruction pretraining and instruction tuning, the diversity of data drops, leading to decreased improvements. Particularly, MPS-oriented training negatively impacts performance on non-mathematical tasks. These findings also suggest that most open-source datasets in the math domain, which specifically target mathematical problem-solving, are unable to facilitate broader types of reasoning tasks to fulfill the "math for AI" goal. We encourage future research to reconsider the objectives when studying mathematical reasoning. If the goal is to enhance general reasoning capabilities rather than "AI for math", it may be worthwhile to explore which data sources, whether math-related or otherwise, can effectively contribute to the acquisition of more diverse reasoning skills.In the final part of this work, we perform a pilot study, trying to identify potential data sources that could enhance reasoning skills. To this end, we experiment with three popular non-MPS SFT datasets that cover various thought reasoning processes, including coding-related tasks, a broad array of reasoning-intensive tasks and state-of-the-art conversational datasets. Unfortunately, none of these datasets demonstrated significant improvements across a wide spectrum of reasoning tasks. This points to a pessimistic conclusion that, in comparison to the extensive data used in pretraining, the relatively modest volume of SFT data is insufficient to substantially improve the model's general reasoning capabilities, even when the data originates from diverse domains. 
training paradigms for mathematical problem-solving: 
 The improvement of mathematical problem-solving abilities in LLMs has been explored through various training approaches, each with its own strengths and focus. Starting from a pretrained base model, in this study, we explore three prominent training strategies as followed. Due to the expensive cost of running some of the training paradigms, we obtain the required model from either the opensource checkpoints or our own training as we also detail next.Continual Pretraining on Mathematical Text. In mathematics, where texts often involve multistep reasoning and formal expressions, this approach helps models better grasp the reasoning pat- terns (Lewkowycz et al., 2022) . Due to the expensive cost of running continual pretraining, in this study, we experiment with two open-weight LLMs continually pretrained on mathematical-related text: RhO-Math (Lin et al., 2024) and DeepSeekMath (Shao et al., 2024) . DeepSeekMath-Base is continual pretrained based on the DeepSeek-Coder-Base model using a large mathematical corpus called DeepSeekMath Corpus. It achieves 64.2% on GSM8K and 36.2% on the competition-level MATH dataset. Rho-Math-7B is continual pretraining with Selective Language Modeling method through OpenWebMath corpus on Mistral-7B, achieving 66.9% on GSM8K and 31.0% on MATH dataset. Distinct from normal continual pretraining, Rho-Math utilizes another reference model to select tokens and only optimize losses on the selected tokens. However, the reference model is created by training on task-specific SFT datasets. While Rho-Math demonstrated superior performance on mathematical problem-solving, in §3.3 we will show that this training scheme may potentially overfit on benchmark tasks as well, and fail to achieve significant gains on non-MPS tasks.Instruction Pretraining on Diverse QA Pairs. Instruction pretraining using diverse questionanswer (QA) pairs improves a model's generalization across diverse tasks while enhancing its instruction-following capabilities (Yue et al., 2024; Chung et al., 2024; Cheng et al., 2024) . This approach involves with large QA datasets, often synthesized from raw text, encompassing various formats, complexities, and problem types. Typically, powerful LLMs like GPT-4 are used to filter raw text and generate relevant QA pairs. In our study, we leverage the open-weight MammoTH2 model (Yue et al., 2024) to evaluate it on broader tasks. MammoTH2 was trained on approximately 10 million QA pairs synthesized through open-source LLMs from a wide range of mathematical, science and engineering texts.Instruction Tuning on MPS Datasets. Unlike continual pretraining or instruction pretraining on diverse QA pairs, this approach focuses on smaller, domain-specific datasets typically aligned with benchmark tasks. This is the most commonly used approach to boost MPS scores due to its efficiency. To assess whether models finetuned on MPS datasets can generalize beyond their source tasks, we use two different MPS-oriented datasets to train two models on our own : Math-COT SFT and Math-POT SFT. Math-COT SFT was trained on the MetaMath dataset (Yu et al., 2023) , which draws primarily from the GSM8K and MATH benchmarks, all structured in a chain-of-thought (CoT) format. Math-POT SFT, on the other hand, was trained on the NuminaMath-TIR dataset (LI et al., 2024) , which includes problems from GSM8K and MATH, as well as other benchmarks, with tasks presented in natural language and solutions in code snippets. The NuminaMath-TIR dataset directly leads to the NuminaMath model that wins a recent AI for Math competition.1 The training strategies described in §2.1, if exclusively used, could lead to the development of models specialized solely in mathematical reasoning tasks. However, this work focuses on studying "math for AI", the impact of math-related training and data on general model development. And it is a common practice to mix different sources of datasets to perform training (Xu et al., 2023;  Under review as a conference paper at ICLR 2025 Meta, 2024). Given this context, it is crucial for developers to understand: how would incorporating additional math-related training impact the original general training performance? To investigate this, we design our experiments to mimic the realistic setting, focusing on a simple yet prevalent training pipeline: a pretrained base model followed by the original SFT training (e.g., on general conversational data). We then conduct controlled experiments to introduce additional math-related data into this training pipeline, aiming to evaluate its influence on the model's performance across various tasks. we explore two different ways of integrating math-related training: two-stage training and mix-data training, as we detail below. The process is illustrated in Figure 1 .Since continual pretraining and instruction pretraining typically serve as an intermediate stage to obtain an enhanced base model followed by SFT training (Shao et al., 2024; Yue et al., 2024) , we examine a two-stage training approach that injects math-related data in a mid-training stage. Specifically, in the first stage, one of the three methods outlined in §2.1 is applied, designed to strengthen the model's foundational mathematical reasoning abilities. In the second stage, we fine-tune these first-stage models using general conversation data to broaden their applicability to a variety of reasoning tasks, we choose UltraChat (Ding et al., 2023) as the general SFT dataset in this work, which is commonly used to create chat models (Tunstall et al., 2023) . This process helps the models adapt to instruction-following tasks, thereby improving their versatility across different domains.Considering that the two-stage training method may weaken a model's generalization ability due to catastrophic forgetting, we explore another commonly adopted training strategy for incorporating additional SFT datasets, which mixes various SFT data sources together. We only experiment this method for instruction tuning on MPS datasets, since the other two are designed to be conducted in a separate, intermediate training stage. In this mix-data training approach, the training data is a mixture of either Math-COT SFT or Math-POT SFT data combined with Ul-traChat data. Unlike two-stage training, where the model undergoes independent two sequential fine-tuning stages, the mix-data approach consolidates the training process into a single stage. 
experiments: 
 We consider seven particular models from three training strategies which aimed at enhancing the math reasoning capabilities. And we assess the generalization capabilities across multiple types of reasoning benchmarks of these models, encompassing both MPS and non-MPS tasks.We compare several models across the three studied training strategies to evaluate their performance on reasoning tasks. The models used in the first stage of training come from approaches in §2.1, which are outlined as follows:(1) For continual pretraining on mathematical text, we leveraged two existing checkpoints: deepseek-math-7b-base and rho-math-7b-v0.1. Their corresponding base models, are Deepseek-Coder-Base and Mistral-7B, respectively. (2) For instruction pretraining on diverse QA pairs , we used the checkpoint MAmmoTH2-7B, and Mistral-7B serves as its base model. (3) For instruction tuning on MPS datasets, we fine-tuned the base model mistral-7b-v0.1 ourselves using the MetaMath (Yu et al., 2023) and NuminaMath-TIR (LI et al., 2024) datasets to get the Math-COT SFT model and the Math-POT SFT model. These models serve as the first-stage models for further tuning. After obtaining these first-stage models from each of three approaches, we performed a second-stage finetuning on both the math-specialized models and their corresponding base models. In this stage, we fine-tuned the models using the filtered UltraChat (Ding et al., 2023) data, which consists of general conversational content with approximately 200K samples.Mix-data training setup Additionally, we conducted mix-data training through these SFT datasets. The UltraChat data was combined with either MetaMath or NuminaMath-TIR data, randomly shuffled and mixed together. Then we fine-tuned the checkpoint mistral-7b-v0.1 on these two mixture data. All the training methods that we study are summarized in Table 1 . DeepSeek-Coder-Base → DeepSeekMath Corpus → DeepSeekMath-Base → UltraChat → DeepSeekMath (2-stage) Mistral-7B-Base → OpenWebMath Corpus → Rho-Math-7B → UltraChat → Rho-Math-7B (2-stage) Mistral-7B-Base → WebInstruct → MAmmoTH2-7B → UltraChat → MAmmoTH2-7B (2-stage) Mistral-7B-Base → MetaMath → Math-COT SFT → UltraChat → Math-COT SFT (2-stage) Mistral-7B-Base → NuminaMath-TIR → Math-POT SFT → UltraChat → Math-POT SFT (2-stage) Mix-data Training Process Mistral-7B-Base → MetaMath + UltraChat → Math-COT SFT (mixed) Mistral-7B-Base → NuminaMath-TIR + UltraChat → Math-POT SFT (mixed)Math Reasoning (problem-solving) GSM8K, GSM8K MQA, MATH, MMLU-math Math Reasoning (excluding problem-solving) MR-BEN-math, DocMath (Zhao et al., 2024) Logical Reasoning ZebraLogic (Bill Yuchen Lin, 2024), ProofWriter (Tafjord et al., 2020) , LogiQA (Liu et al., 2020) STEM Reasoning GPQA (Rein et al., 2023) , MMLU-stem Commonsense Reasoning NQ (Lee et al., 2019) , SWAG (Zellers et al., 2018) , WinoGrande (Sakaguchi et al., 2021) , ARC-challenge (Clark et al., 2018) Symbolic Reasoning BBH (Suzgun et al., 2022) Agent Reasoning MiniWoB++ (Liu et al., 2018) We use the sanitized version of Ultrachat provided by HuggingFace2 , To balance the exposure of the math and general conversation data, we randomly selected 200K data samples from MetaMath for SFT. For NumniaMath-TIR only has 72K items, so we keep all the samples for SFT. More training hyperparameters are showed in Appendix C.1.To evaluate models' multi-dimensional reasoning capabilities, we choose seven reasoning tasks: math reasoning (problem-solving) (MPS), math reasoning (exculding problem-solving), logical reasoning, STEM reasoning, commonsense reasoning, symbolic reasoning and agent reasoning. The corresponding benchmarks are shown in Table 2 . The GSM8K MQA dataset is derived from the original GSM8K format, repurposed into a multiple-choice question format. The MMLU-math and MMLU-stem are the math and stem sub-categories of MMLU (Hendrycks et al., 2021a) . The MR-BEN-math is only the math subject of MR-BEN (Zeng et al., 2024) . See more introduction of benchmarks in Appendix C.3 Table 3 presents the performance of models of three kinds of training strategies on seven kinds of reasoning tasks. The results are calculated as the average value across each reasoning domain.Results for each benchmark are showed in Appendix A.1.Learning mathematical problem-solving helps mathematical reasoning in general We could observe that all models demonstrate improvements on math problem-solving (MPS) tasks, but the gains for Math-COT and Math-POT models with two-stage training are relatively smaller compared to other methods. Mixed training for Math-COT and Math-POT lead to much higher performance on MPS tasks. On other types of math reasoning tasks which are not problem-solving, all models demonstrate gains despite smaller magnitudes compared to problem-solving tasks. This suggests that learning mathematical problem-solving is able to generalize and help other types of mathematical reasoning as well.Continual pretraining generally improves non-mathematical reasoning while selective continual pretraining falls short The improvements on mathematical reasoning tasks are actually expected, yet we note that this work emphasizes more the effect on other non-mathematical reasoning tasks. We first observe that continual pretraining of DeepSeekMath enhances performance in 3 out of 5 non-mathematical tasks, achieving a notable increase of 4.6 points in STEM reasoning and 3.8 points in symbolic reasoning. DeepSeekMath is also the only one among these models that can achieve an average of over 2-point gain on some non-mathematical reasoning domains. Conversely, Rho-Math, another variant of continual pretraining, only showed improvements in 2 out of 5 non-mathematical reasoning domains with limited gains under 2 points. In more detail, as shown in Figure 2 , the Rho-Math perform worse than DeepSeekMath on more datasets. As introduced in §2.1, Rho-Math employs a selective language modeling loss that leverages a reference model to help select tokens for optimization -this reference model, trained on task-specific SFT datasets, may introduce biases that compromise the generalization capacity. Previously, the extent of this compromise was unknown as only mathematical problem-solving tasks were assessed. Therefore, we urge the research community to conduct to more comprehensive evaluations of a model's reasoning capabilities, to gain a more complete understanding of different training algorithms. Otherwise, in the case of Rho-Math, although it achieves similar gains on MPS benchmarks as DeepSeekMath while being trained on far fewer tokens, the trade-offs compared to standard continual pretraining were not initially clear, as we now demonstrate.Instruction pretraining sometimes help non-mathematical reasoning, while instruction tuning generally impairs We observe that instruction pretraining with the MAmmoTH2 model improves 3 out of 5 non-mathematical reasoning tasks, despite small gains around 1 point. However, instruction tuning on MPS datasets, the most commonly adopted method to learn mathematical problem solving, undermines the original training pipeline on most non-mathematical reasoning tasks, except for the agent reasoning task. This points to a pessimistic reality: most previous efforts that develop new MPS datasets and advance state-of-the-art for mathematical reasoning may not generalize to facilitate learning in other types of reasoning. In fact, the created data resources may even negatively impact other reasoning abilities, a phenomenon that contradicts intuitive expectations based on human learning studies. Agent task specicfic tuning As the models exhibit significant variation in performance on the agent reasoning task, which is likely due to the fixed-format code required as input for agent tasks. The performance comparison becomes highly dependent on the models' ability to generate accurate code. To reduce this disparity, we replaced the second-stage UltraChat data with task-specific data related to the benchmark. Specifically, we used data from Mini-Wob++, generated by Claude-2, as the second-stage training data. The results of this adjustment are shown in Figure 3 . We observe that Rho-Math, MAmmoTH2 and DeepSeekMath all demonstrate improvement over the base model, while Math-COT SFT and Math-POT SFT continue to underperform, reinforcing the notion that models trained via SFT have limited generalization capabilities. 
what other data sources contribute to reasoning -a pilot study: 
 So far, we have explored the effect of various math-related data sources on general reasoning learning, and we have concluded that only continual training with raw math text has a significantly positive effect on general reasoning learning. However, continual pretraining is typically large-scale and computationally expensive. In this section, we perform a pilot study to search for efficient SFT datasets from non-mathematical tasks, to examine whether they can help learn reasoning. Specifically, we identify the following three non-MPS SFT datasets as our targets to study, based on their diverse task coverage as showed in Table 4 :Under review as a conference paper at ICLR 2025 • Magicoder-Evol-Instruct3 (Wei et al., 2023) is used primarily to enhance code generation capabilities in LLMs. The dataset was decontaminated and repurposed from an earlier open-source instruction dataset, Evol-CodeAlpaca4 , which has augmented questions and answers by GPT-4. The dataset helping improve the performance of LLMs on code generation and program algorithm tasks, particularly in diverse programming contexts.• Magpie-Reasoning5 is a specialized SFT dataset designed to improve the reasoning capabilities of LLMs. It is generated by Qwen2-72B-Instruct (Yang et al., 2024) and Llama-3-70B Instruct (Meta, 2024) using Magpie (Xu et al., 2024b) . It consists of 150K samples of conversations, covering a mix of tasks including mathematical reasoning, code-based reasoning, and general logic-based problem-solving.• OpenOrca6 is a large, open-domain dataset that spans diverse fields, including math, science, general knowledge, and other multi-domain tasks, with the distributions outlined in Orca (Mukherjee et al., 2023) . This dataset is augmented from FLAN collection data (Longpre et al., 2023) with GPT-4. Given resource limitations, we performed SFT on 200K samples.Setup Based on the results in §3.3, we found that, when using the same mathematical SFT datasets, mix-data training model generally outperforms two-stage training model in terms of performance. Therefore, we employ mix-data training process for the three non-MPS datasets in this section. Specifically, each non-MPS SFT dataset is randomly mixed with filtered UltraChat data, following the same approach used in §3.1. The initial model remains mistral-7b-v0.1.As shown in Table 5 , despite incorporating diverse datasets such as Magicoder-Evol-Instruct, Magpie-Reasoning, and OpenOrca, the generalization capability of the models across different reasoning tasks remains limited. See detail results on all benchmarks in Appendix A.1. Additionally, the performance among each model still remains some distinction. The Magicoder-Evol-Instruct SFT model shows improvements in fewer areas compared to the other models. This may be attributed to the narrower scope of this dataset, which primarily focuses on code-related tasks. In contrast, the Magpie-Reasoning SFT model demonstrates performance improvements in aUnder review as a conference paper at ICLR 2025 broader range of tasks. This is likely due to its more balanced dataset, which covers both code algorithms and reasoning tasks. Interestingly, the OpenOrca SFT model, despite its broader coverage of reasoning, coding, and general knowledge, shows relatively fewer performance gains compared to Magpie. This could be due to the complexity and diversity of the OpenOrca dataset, which might introduce competing learning objectives, causing the model to struggle in balancing between different types of tasks. While there are some localized improvements in certain domains, such as agent reasoning, where the models exhibit noticeable gains, the overall trend indicates that SFT method, even with diverse and extensive datasets, struggles to generalize effectively across a wide range of reasoning challenges. How to find efficient datasets to enhance general reasoning abilities of LLMs still remain as a critical challenge for future researches to study. 
related works: 
 While LLMs exhibit remarkable performance out of the box, especially in tasks that require pattern recognition and language understanding (Zhao et al., 2022; Brown, 2020; Wei et al., 2022; Creswell et al., 2022) , their ability to perform complex reasoning often requires additional refinement through targeted training methods.Supervised Fine-Tuning A key method for enhancing LLM performance is Supervised Fine-Tuning (SFT). SFT not only improves a model's ability to follow instructions but also enhances its performance on intricate tasks requiring specialized knowledge by training on well-curated datasets (Xu et al., 2023; Zhou et al., 2023; Wu et al., 2023b; Yuan et al., 2023b; Chen et al., 2023b) . As LLMs continue to evolve, researchers also employ SFT as a crucial step in tailoring the models for more complex reasoning scenarios or tasks (Huang & Chang, 2022; Wang et al., 2023b) .In the context of mathematical reasoning, SFT has demonstrated substantial improvements in model performance (Cobbe et al., 2021b; Nye et al., 2021; Yuan et al., 2023a; Yue et al., 2023; Wang et al., 2023a; Li et al., 2023; Liu et al., 2023; Chen et al., 2024) . For instance, the MetaMath model, fine-tuned on an augmented GSM8K and MATH dataset, demonstrated notable improvements on mathematical problem-solving benchmarks (Yu et al., 2023) . In addition to mathematical reasoning, SFT has also been utilized to achieve better results on other types of reasoning tasks. It has been applied to domains like commonsense reasoning (Huang et al., 2022; Bian et al., 2024) and logical reasoning (Luo et al., 2023b; Chen et al., 2023c; Li et al., 2024) , Moreover, researchers also reveal that SFT also helps LLMs handle more dynamic and context-rich tasks like agent-based reasoning (Gou et al., 2023; Chen et al., 2023a) , where understanding interactions and goals in simulated environments is essential.Continual Pretrain Continual pretraining is another widely adopted approach to enhance the performance of LLMs in specific domains (Aharoni & Goldberg, 2020) . Unlike SFT, which relies on task-specific datasets, continual pretraining exposes models to large-scale, domain-relevant corpora Paster et al. (2023) ; Wang et al. (2023c) . The large-scale corpora expands the model's knowledge base and helps the model generalize better within specialized areas (Jin et al., 2021; Gupta et al., 2023; Ke et al., 2023; Wu et al., 2023a; Bian et al., 2024) . In the realm of mathematical problem solving, continual pretraining also has been instrumental in improving models' abilities to tackle complex reasoning tasks (Lewkowycz et al., 2022; Lin et al., 2024; Shao et al., 2024) . 
conclusion: 
 In this paper, we explored the generalization potential of three different training strategies to learn mathematical problem-solving. Our experiments evaluated models trained using (1) continual pretraining on mathematical text, (2) instruction tuning on diverse QA pairs, and (3) instruction tuning on MPS datasets. The results indicate that only continual pretraining on raw mathematical text can lead to significant gains on most domains. In contrast, models fine-tuned on MPS SFT datasets struggled to generalize beyond math-specific tasks and even impaired other reasoning abilities. These observations imply that previous researches on mathematical reasoning may put too much focus on mathematical problem-solving task, which stay far away from the "math for AI" goal. Future research could explore how both math-related or non-math datasets can be leveraged to better develop models capable of handling a wider variety of reasoning tasks. 
 
 Appendix: A.1 BENCHMARK RESULTSIn this section, we present the detailed results of each trained model. 8 shows the commonsense benchmarks results of 2-stage models and mix-data training models. We could observe that models not outperform in this reasoning domain, particular both continual pretrained and instruction pretrained models. This may suggest that when models incorporate math-related data in training process, it potentially shifts the focus away from the general commonsense reasoning patterns.Table 9 provides detailed results for each benchmark of math (problem-solving) and logical reasoning tasks of models SFT through mix-data training process on non-MPS datasets. We could observe that all models indicate a drop on the benchmarks expect for GSM8K. This indicate this non-MPS data do not enhance the models capbility on math problem solving. Table 10 presents detailed results for each benchmark of math (excluding problem-solving) and logical reasoning of models SFT through mix-data training process on non-MPS datasets. These models demonstrate similar proficiency across these benchmarks. Table 11 presents detailed results for each benchmark of commonsense reasoning of models SFT through mix-data training process on non-MPS datasets. Same to the math-related models, these models show a reduction among commonsense reasoning task. This may indicate that these non-MPS data also not help to develop the necessary capabilities of models for solving general commonsense reasoning problems. Under review as a conference paper at ICLR 2025 Here are the brief introduction to each benchmark. For some complex benchmarks, we also present the corresponding prompt for evaluation.GSM8K GSM8K (Cobbe et al., 2021a ) is a dataset specifically designed for evaluating LLMs in the domain of multi-step mathematical reasoning. The problem in this dataset are high quality linguistically diverse grade school math word problems created by human problem writers.https://www.kaggle.com/competitions/ai-mathematical-olympiad-prize/leaderboardhttps://huggingface.co/datasets/HuggingFaceH4/ultrachat 200khttps://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110Khttps://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1https://huggingface.co/datasets/Magpie-Align/Magpie-Reasoning-150Khttps://huggingface.co/datasets/Open-Orca/OpenOrca 



Reviews: review: title: title: Recommendation to accept
main_review: main_review: The work makes a concrete effort by determining when training LLM models on mathematical problem-solving (MPS) tasks extends their reasoning skills beyond strictly defined domains. The sufficiency of the paper in the testing of various types of reasoning algebra in logical, commonsense, and STEM reasoning indeed uses experimental evidence to enhance one’s comprehension of the effect of variances in mathematical training on general reasoning. This paper contributes to the understanding of the "math for AI" hypothesis since it shows that reasoning objectives are readily achieved after the continual pretraining of LLMS on mathematical texts. It assists in elucidating the use of mathematical problem solving data in training large language models and provides a definite path towards the enhancement of reasoning capacities of LLMs thereby making it an excellent candidate for acceptance.
title: title: The paper examines whether mathematical problem-solving training for large language models enhances general reasoning skills, using three training strategies and rigorous methodologies, but could improve with more discussion on the mathematical concepts and dataset specifics
main_review: main_review: The authors investigates whether training large language models (LLMs) in mathematical problem-solving (MPS) can enhance general reasoning abilities beyond MPS benchmarks. The authors differentiate between “AI for math” and “math for AI,” highlighting the broader reasoning potential of the latter. They employ three training strategies—continual pretraining on mathematical texts, instruction pretraining on synthesized QA pairs, and instruction tuning on benchmark datasets—evaluating them across seven reasoning domains. The methodologies are rigorous and detailed. However, the paper would benefit from an overview discussion of the underlying mathematical concepts, more detailed information about the datasets used, and increased literature review for better contextualization.
title: title: Promising Insights with Room for Growth: Evaluating Mathematical Problem-Solving for AI Reasoning
main_review: main_review: This paper explores the use of mathematical problem-solving (MPS) data to enhance reasoning abilities in large language models (LLMs). The authors evaluated three training strategies: continual pretraining on mathematical text, instruction pretraining on synthetic QA pairs, and instruction tuning on benchmark datasets. The findings show that only continual pretraining on mathematical text reliably improves broader reasoning skills, while other methods fall short. Future work will focus on identifying data sources that can more effectively develop general reasoning abilities in AI models.

# Pros
## 1. Strong Writing:
The authors pose a clear research question and provide a thorough analysis of whether mathematical problem-solving enhances general reasoning abilities beyond just improving MPS benchmarks.

## 2. Valuable Findings:
They find that most open-source math datasets fail to support broader reasoning tasks, suggesting the need to shift research focus from “AI for math” to enhancing general reasoning skills.

## 3. Solid Experimentation:
The study effectively addresses overfitting by incorporating general instruction-following data into math-related training and adopting hybrid training.

# Cons
## 1. Lack of Solutions:
The paper identifies challenges but does not propose concrete solutions to address the limitations found.