Solving Math Word Problems with Multi-Encoders and Multi-Decoders (2020)

Abstract: Math word problems solving remains a challenging task where potential semantic and mathematical logic need to be mined from natural language. Although previous researches employ the Seq2Seq technique to transform text descriptions into equation expressions, most of them achieve inferior performance due to insufficient consideration in the design of encoder and decoder. Specifically, these models only consider input/output objects as sequences, ignoring the important structural information contained in text descriptions and equation expressions. To overcome those defects, a model with multi-encoders and multi-decoders is proposed in this paper, which combines sequence-based encoder and graph-based encoder to enhance the representation of text descriptions, and generates different equation expressions via sequence-based decoder and tree-based decoder. Experimental results on the dataset Math23K show that our model outperforms existing state-of-the-art methods.

## introduction math word problems (mwps) solving, a task that transforms text descriptions into solvable equation expressions, is considered a crucial step towards general ai @xcite . since semantic understanding and mathematical logic reasoning both contribute to correct answers, mwps solving remains a challenging topic in nlp. problem: a slow car drives 58(n 1 ) km/h, and a fast car drives 85(n 2 ) km/h. the two cars drive at the same time in inverse direction, and they meet after 5(n 3 ) hours. how many kilometers does the fast car drive more than the slow car when they meet? researches on mwps solving has a long history. early researches focused on rule-based methods @xcite @xcite and statistical machine learning methods @xcite @xcite ) that map problems into predefined templates. the main drawbacks of these methods lie in their heavy dependency on manual features and incapacity to generate new templates for new problems. consequently, they can only achieve satisfactory results on small-scale datasets @xcite . recently, more researchers have been introducing seq2seq models, which are capable of generating new equation expressions that do not exist in the training set @xcite @xcite . however, these models may generate invalid expressions since the sequencebased decoder cannot control the generation process. based on the fact that each equation expression could be transformed into an abstract syntax tree (ast), some studies @xcite changed the pattern of sequence generation from left to right and followed the top-down decoding process. such tree-based decoders match the prefix order of ast. although these models considered the structural information of equation expressions, they ignored that text descriptions also contain rich structural information, such as dependency parse tree and numerical comparison information. the dependency parse tree represents various grammatical relationships between pairs of text words, for example, nouns are usually matched with verbs, and numerals are usually matched with quantifiers. in now back to the design of decoder, existing methods only adopt one decoder, which limits the generation ability of the model. @xcite ) provided an ensemble model that selects the result according to various models' generation probability. however, there is still one decoder for a single model. @xcite integrated two decoders in one model, but both are sequence-based decoders. the same type of decoder cannot significantly improve generalization performance. with the aim of solving aforementioned challenges, we propose a novel model with multi-encoders and multi-decoders, which combines sequence-based encoder and graph-based encoder to enhance the representation of text descriptions, and obtains different equation expressions via sequence-based decoder and tree-based decoder. specifically, we leverage a sequence-based encoder to get the context representation of text descriptions, and integrate the dependency parse tree and numerical comparison information via a graph-based encoder. in the decoding stage, a sequence-based decoder is used to generate the suffix order of ast, and a tree-based decoder is used to generate the prefix order. the final result is selected according to the generation probability of different decoders. the main contributions of this paper are summarized as follows: ## related work mwps solving may date back to the 1960s and continues attracting current nlp researchers. here we will introduce recent studies based on the seq2seq framework. the work presented in @xcite reviews more early approaches. @xcite made the first attempt to directly generate equation expressions by using the seq2seq model and published a high-quality chinese dataset math23k. @xcite found that using the suffix order of ast can eliminate brackets in the original expressions, and proposed an equation normalization method to reduce the number of duplicated equations. @xcite proposed a twostage model that first used a seq2seq model to generate expressions without operators, and then used a recursive neural network to predict the operator between numbers. (chiang and chen, 2019) adopted a stack to track the semantic meanings of numbers. @xcite added different functional multihead attentions to the seq2seq framework. @xcite applied double sequencebased decoders in one model. however, these seq2seq models only consider input/output objects as sequences, ignoring the important structural information of equation expressions. consequently, they cannot guarantee the generation of valid equation expressions. the idea of the tree-based decoder was proposed in @xcite . they changed the pattern of sequence generation from left to right and followed the top-down decoding process. however, these methods ignored rich structural information contained in text descriptions. @xcite proposed the graph-based encoder. @xcite ) integrated the dependency parse tree and constituency tree of text descriptions. @xcite constructed the quantity cell graph and quantity comparison graph. since these methods considered the structure information of text descriptions, they have been current state-of-the-art models. the encoders and decoders designed by these seq2seq models are summarized in ## methodology the framework of our model is shown in figure 1 , which consists of four components: the sequencebased encoder obtains the context representation of text descriptions; the graph-based encoder integrates the dependency parse tree and numerical comparison information; the sequence-based decoder generates the suffix order of ast, and the tree-based decoder generates the prefix order. the final generation result is selected according to the generation probability of different decoders. ## experiments in this section, we evaluate our model on a large-scale dataset math23k. we compare our model with several state-of-the-art methods and demonstrate the effectiveness of our model via a series of controlled experiments. our code can be downloaded at https://github.com/yibinshen/multimath . ## conclusion and future work inspired by the fact that text descriptions and equation expressions have structural information, a model with multi-encoders and multi-decoders is proposed in this paper. to be specific, we use the sequencebased encoder to obtain the context representation, and the graph-based encoder is used to integrate the structure information of text descriptions. two types of decoders generate different expressions, which strengthens the generation ability of the model. experimental results on math23k proves the advantages of our model over existing state-of-the-art methods. the experiment analysis shows that the effectiveness of mathematical logic mining from the problem. in future work, we will explore the question generation technique to increase samples of the dataset and solve the problems with complex expressions.