Learning Multi-Step Reasoning by Solving Arithmetic Tasks (2023)

Abstract: Mathematical reasoning is regarded as a necessary ability for Language Models (LMs). Recent works demonstrate large LMs’ impressive performance in solving math problems. The success is attributed to their Chain-of-Thought (CoT) reasoning abilities, i.e., the ability to decompose complex questions into step-by-step reasoning chains, but such ability seems only to emerge from models with abundant parameters. This work investigates how to incorporate relatively small LMs with the capabilities of multi-step reasoning. We propose to inject such abilities by continually pre-training LMs on a synthetic dataset MsAT which is composed of Multi-step Arithmetic Tasks. Our experiments on four math word problem datasets show the effectiveness of the proposed method in enhancing LMs’ math reasoning abilities.

## introduction making language models (lms) perform mathematical reasoning is a valuable, yet challenging research objective @xcite . recently, we have witnessed largescale lms' impressive performance on a series of reasoning tasks via chain-of-thought prompting @xcite . this method elicits large lm's ability to decompose a complex problem into several intermediate steps. however, it is believed that such ability only emerges from sufficiently large models (empirically more than 100b parameters) @xcite . in this paper, we examine how to incorporate moderate-sized lms, e.g., roberta @xcite , with such multi-step reasoning ability via continual pre-training to improve the performance on math problems. correctly understanding numbers is a prerequisite of mathematical reasoning abilities. but @xcite shows that medium-sized figure 1 : a math word problem example with different kinds of answers. in question, <num0>, <num1>, @xmath0. lms have a deficiency in numerical comprehension. to overcome this issue, previous works inject numerical reasoning skills into lms following two approaches. the first is masking numbers with special tokens, and generating symbolic expressions with a structured neural decoder @xcite . an example of such expression is provided in figure 1 . the second strategy continually pre-trains lms on synthetic numerical tasks, which requires models to learn how to perform computation involving numbers @xcite . however, both approaches suffer from critical limitations. for symbolic methods, they neglect the information carried by the numbers, which could provide crucial hints for solving math problems @xcite . as for continual pre-training methods, lms' arithmetic skills are not reliable. previous works indicate that such skills are highly influenced by the training data @xcite and hard for extrapolation @xcite . motivated by these shortcomings, we propose to first pre-train moderate-sized lms on a synthetic dataset called msat (multi-step arithmetic tasks) before downstream task fine-tuning. to make sure lms capture the information carried by the numbers, we keep the numbers in the questions instead of masking them during both pre-training and finetuning. instead of making lms conduct computation internally, msat encourages lms to generate a series of intermediate steps leading to the answer. experiments on four math word problem datasets with two backbone models demonstrate the effectiveness of our method in enhancing lms' math reasoning performance. ## method our method essentially appends a continual pretraining stage before fine-tuning lms on downstream tasks. the continual pre-training serves two purposes: first, we tokenize numbers digit-bydigit to improve lms' numerical comprehension; second, we make lms learn multi-step reasoning skills from the proposed synthetic task. ## experiments now we investigate whether our pre-training method facilitates models on math word problem (mwp) solving tasks. all results are averaged over three different runs. ## pre-training analysis in this section, we provide a careful analysis of our pre-training method from various perspectives to understand why it works. ## related work in this work, we focus on improving moderatesized lm's mwp performance by injecting multistep reasoning ability. hence, our work closely relates to both reasoning ability injection @xcite and mwp solving @xcite @xcite . ## conclusion we propose a novel synthetic pre-training task, msat, to incorporate lms with multi-step reasoning skills that improve performance on mwp tasks. this pre-training task encourages lms to generate intermediate reasoning steps instead of predicting final numerical answers directly. our experiments show that the proposed method is effective in improving the moderate-sized lm's performance on mwp solving tasks.