DRTS Parsing with Structure-Aware Encoding and Decoding (2020)

Abstract: Discourse representation tree structure (DRTS) parsing is a novel semantic parsing task which has been concerned most recently. State-of-the-art performance can be achieved by a neural sequence-to-sequence model, treating the tree construction as an incremental sequence generation problem. Structural information such as input syntax and the intermediate skeleton of the partial output has been ignored in the model, which could be potentially useful for the DRTS parsing. In this work, we propose a structural-aware model at both the encoder and decoder phase to integrate the structural information, where graph attention network (GAT) is exploited for effectively modeling. Experimental results on a benchmark dataset show that our proposed model is effective and can obtain the best performance in the literature.

## introduction discourse representation tree structure (drts) is a form of discourse structure based on discourse representation theory of @xcite , a popular theory of meaning representation @xcite @xcite . it is designed to account for a variety of linguistic phenomena, including the interpretation of pronouns and temporal expressions within and across sentences. correspondingly, as one type of discourse parsing, drts parsing @xcite can be helpful for paragraph or document-level text understanding by converting drs to tree-style drts. @xcite . figure 1 shows an example of drts, where the leaf nodes are discourse representation units (drus), upon which a discourse tree structure built. in particular, a dru consists of several individual tuples, where each tuple denotes a relation inside the dru. for example, there is a relationship figure 1 : left: our proposed model with two structureaware module. right: the drts for a clause in a document: "the letter x4 warns jewish women x16 that they will suffer if they date arab men. there have been only a few existing studies related to drts parsing @xcite . in particular, the end-to-end encoderdecoder model of @xcite gives the stateof-the-art performance, which converts the task into a sequence-to-sequence problem. the input sequence consists of words in paragraphs, encoded by a bilstm structure, and the output sequence is top-to-bottom depth-first traversal of the output drts tree, which is decoded incrementally with an attention-based lstm feature representation module. during decoding, @xcite separate the skeleton generation and the dru producing, as illustrated by figure 1 . although highly effective, the above model ig-nores some useful structure information in both the encoder and the decoder, which can be potentially useful for our task. specifically, for encoding, syntax-based tree structure information has been demonstrated effective for a number of nlp tasks @xcite , including several other types of discourse parsing @xcite . for decoding, the skeleton structure of drts can be also beneficial for our task. as a two-phase decoding strategy is exploited, the skeleton tree from the first phase could be helpful for dru parsing of the second phase. we propose to improve drts parsing by making use of the above structure information, modeling dependency-based syntax of the input sentences as well as the skeleton structure to enhance the baseline model of @xcite using graph attention network (gat) @xcite , which has been demonstrated effective for tree/graph encoding @xcite . in particular, we first derive dependency tree structures for each sentence in a paragraph from the stanford parser, and then encode them directly via one gat module, which are fed as inputs for decoding. second, after the first-state skeleton parsing is finished, we encode the skeleton structures by another gat module, feeding the outputs for dru parsing. following @xcite , we conduct experiments on the groningen meaning bank (gmb) dataset. results show that structural information is highly useful for our task, bring a significantly better performance over the baseline. in particular, dependency syntax gives an improvement of 2.84% based on the standard evaluation metrics and the skeleton structure information gives a further improvement of 1.41%. finally, our model achieves 71.65% f1-score for the task, 4.25% better than the baseline model. additionally, our model is also effective for sentence-level drts parsing, leading to an increase of 1.72% by the f1score by our final model. we release our code and best models at http://github.com/seanblank/ drtsparsing for facilitating future research. ## discourse representation tree (drt) formally, a drt structure consists of two components according to the function: (1) the leaf nodes and (2) the tree skeleton (non-terminal nodes), respectively. similar to other types of discourse representation methods, we have minimum semantic units named by dru, and then a discourse tree is built by the discourse relationships between these minimum units. figure 2 shows the full tree version of figure 1 in the introduction. ## baseline we take the multi-step encoder-decoder method of @xcite as the baseline model for drts parsing. first, an encoder is used to convert one input paragraph into neural vectors by using word embeddings as well as bilstms, and then a multistep decoder is exploited to generate a full tree structure in a sequential manner incrementally. ## structure-aware seq2seq to represent the structure features, we use a gat module on top of encoder and skeleton decoder stage to enhance the baseline model. the graph module is designed to learn non-local and nonsequential information from structural inputs. in this section, we first describe the gat in detail and then illustrate its application on our task. ## related work discourse parsing is one important topic in the nlp. there are several main types of discourse parsing tasks in the literature, including rhetorical structure theory (rst; @xcite based parsing, centering theory (ct; @xcite based parsing and drt based parsing in this study. discourse representation theory (drt) based parsing is a relatively classic, yet not fully researched semantic analysis task because of its complexity. @xcite present the first work of a data-driven drt parser, using a graphbased representation of drt structures. @xcite apply the idea of neural machine translation for graph-based drt parsing, achieving impressing performance. these studies only focus on sentence-level drt representations, as the complexity would increase much at the paragraph level. in contrast, we investigate the paragraph level drt parsing. drts parsing simplifies graphs into trees. there are two existing papers in this line. @xcite are the first to work on drts parsing, who propose an end-to-end sequence-to-sequence model for the task. further, @xcite improve the model by suggesting several effective strategies including supervised attention, copying from alignments, and constraint-based inference. in this work, we improve drts parsing instead of @xcite with two types of structure information. syntax information has been widely exploited for nlp tasks. seminal work exploits discrete features designed by experts @xcite . recently, a range of neural modules have been proposed to encode syntax, such as tree-lstm @xcite @xcite , tree-cnn @xcite and the recently proposed implicit approaches @xcite . syntax has been demonstrated effective for rst based discourse parsing as well @xcite . our work is to build a syntax tree-aware model and we are the first to use syntax for drt based discourse parsing. gnn has received increasing interests for its strong capability of encoding structural information @xcite @xcite @xcite . gat is one representative model, which demonstrates success in a number of nlp tasks @xcite . in this work, we exploit gat to represent treestructural information for drts parsing. ## conclusion we investigated the representation of structural information for discourse representation tree structure parsing, showing that a graph neural network can bring significant improvements. in particular, we use gat for representing syntax in encoding, and representing a structural backbone for decoding. experiments on the standard gmb dataset show that our method is high effective, achieving the best results in the literature.