Learning by Analogy: Diverse Questions Generation in Math Word Problem (2023)

Abstract: Solving math word problem (MWP) with AI techniques has recently made great progress with the success of deep neural networks (DNN), but it is far from being solved. We argue that the ability of learning by analogy is essential for an MWP solver to better understand same problems which may typically be formulated in diverse ways. However most existing works exploit the shortcut learning to train MWP solvers simply based on samples with a single question. In lack of diverse questions, these methods merely learn shallow heuristics. In this paper, we make a first attempt to solve MWPs by generating diverse yet consistent questions/equations. Given a typical MWP including the scenario description, question, and equation (i.e., answer), we first generate multiple consistent equations via a group of heuristic rules. We then feed them to a question generator together with the scenario to obtain the corresponding diverse questions, forming a new MWP with a variety of questions and equations. Finally we engage a data filter to remove those unreasonable MWPs, keeping the high-quality augmented ones. To evaluate the ability of learning by analogy for an MWP solver, we generate a new MWP dataset (called DiverseMath23K) with diverse questions by extending the current benchmark Math23K. Extensive experimental results demonstrate that our proposed method can generate high-quality diverse questions with corresponding equations, further leading to performance improvement on Diverse-Math23K. The code and dataset is available at: https://github.com/zhouzihao501/DiverseMWP .

## introduction solving math word problem (mwp) aims to infer a mathematical equation and final answer from the natural language description of a math problem. to alleviate this limitation, recent focus has been put on how to augment high-quality data for mwps. along this line, there have been some proposals @xcite @xcite @xcite . though demonstrating encouraging results, these current practices only consider word-level or sentence-level alternative expressions of the original problem, owing to the rigorous requirement in logic and numerical quantity. as illustrated in table @xcite , the back translation augmentation method @xcite generates less diverse data sharing very limited semantic differences from the original counterpart. on the other hand, @xcite publish a diverse mwp dataset (called unbiasedmwp), which was collected by manual annotation with huge cost but the size is limited. in this paper, we make a first attempt to solve mwps by automatically generating multiple diverse yet consistent questions (together with their corresponding equations), as illustrated in motivated by these findings, we propose a diverse questions generation framework (dqgf) to generate high-quality and diverse questions with their corresponding equations for a given mwp. our dqgf consists of three components as shown in figure 1 . (1) diverse equations generator: it generates diverse and meaningful equations from the original mwp based on two generation strategies. specifically, we propose a subequation based strategy that extracts sub-equations from the original equation, and a unit based strategy that generates equations according the units (e.g., "dollars" in extensive experiments on the existing dataset unbiasedmwp @xcite show that our proposed dqgf could generate high-quality diverse questions with corresponding equations, thus increasing the accuracy of the mwp solver. to further verify the effectiveness of the dqgf, we produce a new dataset (called diversemath23k) with diverse questions from the current benchmark dataset math23k @xcite . we also propose a new group-accuracy metric on all questions of a problem. experimental results show that dqgf can effectively improve the overall performance of the solver on diversemath23k, demonstrating its ability of learning by analogy. in summary, our contributions are as follows: kumar et al. (2022) further propose back translation, synonym replacement, and named-entity replacement to augment data. li et al. (2022a) and liu et al. (2021a) transform the declarative sentence into the question sentence and reverse the operation of expression to generate mwps. these methods effectively improve the performance of mwp solvers. but most of them are rule-based and augment data with limited semantic differences from the original data. ## methodology figure 1 shows the overview of the proposed diverse questions generation framework (dqgf). we firstly put the original mwp into the diverse equations generator to generate diverse equations, then the generated equation and scenario description of the original mwp are fed into the trained equation-aware question generator to produce corresponding questions. in this way, we will obtain diverse questions with their equations, forming new candidate mwps. finally, these candidate mwps are further filtered by the data filter. in what follows, we will introduce diverse equations generator, equation-aware question generator, and data filter respectively in section 3.1, section 3.2, and section 3.3. ## conclusion and future work in this paper, we explore the ability of learning by analogy for mwp solvers. to do this, we propose a diverse questions generation framework (dqgf) to automatically generate diverse questions with their corresponding equations for a give mwp, which consists of diverse equations generator, equationaware question generator and data filter. based on the trained dqgf, we further produce a new mwp dataset (diversemath23k) with diverse questions. experimental results demonstrate that dqgf could generate high-quality diverse questions and improve effectively the overall performance of the mwp solver. in the future, we will focus on optimizing the model in the solver to improve its ability of learning by analogy and increase the group accuracy on the mwps with diverse questions.