L ogic S olver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning (2022)

Abstract: Recently, deep learning models have made great progress in MWP solving on answer accuracy. However, they are uninterpretable since they mainly rely on shallow heuristics to achieve high performance without understanding and reasoning the grounded math logic. To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11,495 MWPs and annotates interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation. Different from existing MWP datasets, our InterMWP benchmark asks for a solver to not only output the solution expressions but also predict the corresponding logical formulas. We further propose a novel approach with logical prompt and interpretation generation, called LogicSolver. For each MWP, our LogicSolver first retrieves some highly-correlated algebraic knowledge and then passes them to the backbone model as prompts to improve the semantic representations of MWPs. With these improved semantic representations, our LogicSolver generates corresponding solution expressions and interpretable knowledge formulas in accord with the generated solution expressions, simultaneously. Experimental results show that our LogicSolver has stronger logical formula-based interpretability than baselines while achieving higher answer accuracy with the help of logical prompts, simultaneously. The source code and dataset will be available at https://github.com/yangzhch6/InterMWP.

## introduction automatically math word problem (mwp) solving is a challenging task in natural language processing since it aims to transform a concise narrative rich in mathematical relationships into a solution equation, compared with the common mwp datasets, intermwp requires a solver to predict expression tree and the corresponding linguistic logic formulas simultaneously for improving the interpretability of a solver. as illustrated in figure 1 @xcite . recently, the task of mwp solving automatically has attracted a lot of research attention. several deep-learning-based approaches @xcite @xcite @xcite have been proposed and made great progress for mwp solving. however, as shown in figure 1 (a), current models treat mwp as a seq2seq task, ignoring the interpretability. the grounded logic in the problem consists of two algebraic knowledge formulas: @xmath0, as shown in figure 1(b) . without logical reasoning, it is difficult for an mwp solver to explain why such an equation should be generated as the solution. there are two main reasons for the current dilemma: 1) the lack of relevant and easily exploitable interpretable mwp datasets. 2) current models mainly rely on shallow heuristics to achieve high performance and lack grounded math logic reasoning, as shown in @xcite . to overcome this dilemma and make a step towards interpretable mwp solving, we propose a novel high-quality interpretable mwp dataset 1 called intermwp consisting of 11,495 annotated samples and 210 different logic formulas based on algebraic knowledge. in our intermwp dataset, each solution equation is annotated with interpretable logic formulas in a tree structure as the grounded logic of each solution equation. as shown in figure 1 (b), each inner node is annotated with an interpretable algebraic knowledge formula which represents the grounded logic for the subtree with the current node as the root ancestor. with these logic annotations, our intermwp asks for a solver to not only output the solution equation but also output the logic formulas simultaneously when the current predicted node is an inner-node (operator) during expression reasoning. therefore, an mwp solver developed on intermwp can output a solution equation while generating a reasonable formula-based interpretation. we use answer accuracy from prior works @xcite @xcite , together with formula accuracy and logic accuracy we proposed in section 5.1 to evaluate the model's solving ability and interpretability. to leverage mathematical logic knowledge and empower an mwp solver with interpretability, we further present a novel framework named logic-solver which extracts mathematical logic knowledge as logical prompts to improve the semantic representations of mwps and enhance the ability of explanation generation. in our logicsolver, we design a logic formula retriever to first extract logic prompts consisting of logic formulas highly-correlated with current mwp. then, the logic prompts will be concatenated with the problem text as the input and drive the mwp model to produce the solution equation. finally, to obtain the logic formulas-based explanation, we propose a logic generator to predict logic formulas for each inner-node of the solution expression tree. experimental results show that our logicsolver has stronger logical formula-based interpretability than baselines while achieving higher answer accuracy with the help of logical prompts, simultaneously. in this work, our contributions can be summarized in the following three folds: