An Edge-Enhanced Hierarchical Graph-to-Tree Network for Math Word Problem Solving (2021)

Abstract: Math word problem solving has attracted considerable research interest in recent years. Previous works have shown the effectiveness of utilizing graph neural networks to capture the relationships in the problem. However, these works did not carefully take the edge label information and the long-range word relationship across sentences into consideration. In addition, during generation, they focus on the most relevant areas of the currently generated word, while neglecting the rest of the problem. In this paper, we propose a novel Edge-Enhanced Hierarchical Graph-to-Tree model (EEH-G2T), in which the math word problems are represented as edge-labeled graphs. Specifically, an edge-enhanced hierarchical graph encoder is used to incorporate edge label information. This encoder updates the graph nodes hierarchically in two steps: sentence-level aggregation and problem-level aggregation. Furthermore, a tree-structured decoder with a split attention mechanism is applied to guide the model to pay attention to different parts of the input problem. Experimental results on the MAWPS and Math23K dataset showed that our EEH-G2T can effectively improve performance compared with state-of-the-art methods.

## introduction math word problem solving is an important natural language processing (nlp) task that has recently been attracting increasing research interests. math word problems are narrative text that describe a scene with several math variables and ask a question about an unknown quantity. a simple example is illustrated in figure 1 . based on the given problem, the target is to infer the difference between the number of boxes of apples and pears. @xcite and eeh-g2t. previous works @xcite @xcite used sequenceto-sequence (seq2seq) methods with an attention mechanism @xcite to generate math expression sequences from math word problems. to capture the structural information of math expressions, many works @xcite @xcite treat math expressions as binary trees and propose several sequence-to-tree (seq2tree) frameworks. these tasks are designed to obtain the pre-order sequence of the expression tree, and they generate the current node based on its parent node and sibling node at each time step. some works that represent problems as graphs also show better performance. graph2tree @xcite connects each number in the problem with its nearby nouns to enrich the quantity representations. ka-s2t (wu et al., 2020) connects words with its category in the external knowledge base to capture common sense information. although these methods report promising results, several challenges still remain. 1) long-range word relationships across sentences should be taken into consideration. as shown in figure 1 , the word "pear" in the second sentence should be associated to the word "pear" in the last sentence. without long-range relationships, it is difficult for the model to connect these two words that are 15 steps apart. 2) previous methods did not carefully take the edge label information into consideration. in figure 1 , the label on the edge between "kilograms" and "pear" is nmod (noun compound modifier), while the label "category" on the edge between "apples" and "pears" means they belong to the same category in the external knowledge base. such edge labels can also provide rich syntactic and semantic information. 3) when generating expressions, previous methods tend to focus on the areas in the problem that are most relevant to the currently generated words, and ignore the semantic clues provided by the rest of the problem. as shown in figure 2 , to generate "360" instead of "240" at time step 3, the model needs to pay attention to the entire problem to obtain important clues that the current sub-expression "/360 24" is the number of apple boxes and 360 is the weight of the apples. however, previous methods focused on the problem areas that are most relevant to the currently generated word (i.e., the number 360 itself), without noticing the rest of the problem. to tackle these challenges, we propose a novel edge-enhanced hierarchical graph-to-tree framework (eeh-g2t) for math word problem solving. eeh-g2t represents each math word problem as a graph in which the nodes are connected by labeled edges. to obtain the edge-aware problem representations, we propose an edgeenhanced hierarchical graph encoder that explicitly incorporates edge label information. in addition, the hierarchical encoder updates the nodes in two steps: sentence-level aggregation and problemlevel aggregation. this hierarchical structure can first capture the local relations between words within the sentence and then capture the long-range dependencies between words across sentences. further, we use a split attention mechanism to guide the decoder to pay attention to different parts of the entire input problem, not just the most relevant part of the currently generated word. the main contributions of this paper can be summarized as follows: ## related work math word problem solving: solving math word problems has long been a very popular task and various methods have been proposed in the past few years @xcite @xcite . previous methods usually treated the math word problem as a sequence, and use the same linear encoder to encode math word problems @xcite . recently, many works that treat math word problems as graphs have shown better performance. @xcite connects each number in the problem with nearby nouns to enrich the problem representations. @xcite connects words that belong to the same category in the external knowledge base to capture common sense information. @xcite construct an input graph from both the math problem and its corresponding dependency tree to incorporate structural information. however, these methods only capture the local neighbor information of nodes as additional features to enrich the problem representations and ignore the long-range relations across sentences. in this paper, we propose an edge-enhanced hierarchical graph encoder that captures both the local relations between words within a sentence and the long-range relations between words across sentences. to further guide the decoder to pay attention to different parts of the entire input problem, we propose a split attention mechanism. graph neural networks: many works on graph neural networks (gnns) have been applied to a variety of tasks in recent years, such as node classification @xcite , relation extraction @xcite , and code summarization @xcite . @xcite proposed a labeled edge graph convolutional neural network model on a document-level graph for inter-sentence relation extraction. @xcite simultaneously exploits syntactic structure and typed dependency labels to improve neural event detection. inspired by such works, we also leverage edge label information to enrich the problem representations. ## conclusion in this study, we proposed a novel edge-enhanced hierarchical graph-to-tree model called eeh-g2t for the math word problem solving task. we used an edge-enhanced hierarchical graph encoder that updates the graph nodes in two steps, namely sentence-level aggregation and problem-level aggregation. additionally, edge label information was incorporated into the model to enrich the problem representations. we proposed a split attention mechanism to guide the decoder to pay attention to different parts of the entire input problem during generation. experimental results confirmed that the proposed model, eeh-g2t, outperformed other state-of-the-art models.