A Top-down Neural Architecture towards Text-level Parsing of Discourse Rhetorical Structure (2020)

Abstract: Due to its great importance in deep natural language understanding and various down-stream applications, text-level parsing of discourse rhetorical structure (DRS) has been drawing more and more attention in recent years. However, all the previous studies on text-level discourse parsing adopt bottom-up approaches, which much limit the DRS determination on local information and fail to well benefit from global information of the overall discourse. In this paper, we justify from both computational and perceptive points-of-view that the top-down architecture is more suitable for text-level DRS parsing. On the basis, we propose a top-down neural architecture toward text-level DRS parsing. In particular, we cast discourse parsing as a recursive split point ranking task, where a split point is classified to different levels according to its rank and the elementary discourse units (EDUs) associated with it are arranged accordingly. In this way, we can determine the complete DRS as a hierarchical tree structure via an encoder-decoder with an internal stack. Experimentation on both the English RST-DT corpus and the Chinese CDTB corpus shows the great effectiveness of our proposed top-down approach towards text-level DRS parsing.

## introduction text-level parsing of discourse rhetorical structure (drs) aims to identify the overall discourse structure and the rhetorical relations between discourse units in a text. as a fundamental research topic in natural language processing, text-level drs parsing plays an important role in text understanding and can benefit various down-stream applications, such as document summarization @xcite , sentiment analysis @xcite ), text categorization @xcite , pronoun ⇤ corresponding author commentary (sn) overall-branch (sn) coordinating (nn) e7 e6 coordinating (ns) e5 e4 purpose (sn) coordinating (nn) according to rhetorical structure theory (rst) @xcite , a text can be presented by a hierarchical tree structure known as a discourse tree(dt). figure 1 illustrates an excerpt with its gold standard drs from article chtb 0005 in the chinese cdtb (connectivedriven discourse treebank) corpus @xcite . we can find that, in the dt, each leaf node corresponds to an elementary discourse unit (edu), and various edus are recursively combined into high level larger discourse units in a bottom-up fashion. in this example, 7 edus are connected by 6 rhetorical relations, while in each non-terminal node, the rhetorical relation and the nuclearity type are labeled. correspondingly, text-level drs parsing consists of three components, i.e., bare drs generation (hierarchical span determination), rhetorical nuclearity determination and rhetorical relation classification. during the past decade, text-level drs parsing has been drawing more and more attention and achieved certain success @xcite @xcite @xcite @xcite . however, all the previous studies on text-level drs parsing adopt bottom-up approaches. that is, adjacent edus are recursively combined into high-level larger text spans by rhetorical relations to form a final discourse tree in a bottom-up way. in this paper, we justify that compared with a bottom-up approach, a top-down approach may be more suitable for textlevel drs parsing from two points-of-view, for text-level drs parsing, a top-down approach can better grasp the overall drs of a text and conform to the human perception process. additionally, just noted as @xcite , they employed a top-down strategy in the chinese cdtb annotation practice. that is, a top-down approach is consistent with the annotation practice of a drs corpus. in this paper, we propose a topdown neural architecture to text-level drs parsing. in particular, we cast top-down text-level drs parsing as a recursive split point ranking task, where various edus associated with split points are arranged in different levels according to the rank of the split point. in this way, we can determine the complete drs as a hierarchical tree structure via an encoder-decoder with an internal stack. it is worthwhile to mention that, at each time step, we use the biaffine attention mechanism @xcite to compute the attention vector and determine the next split point, along with the corresponding nuclearity and relation jointly. ## related work in the literature, previous studies on text-level discourse parsing can be classified into two categories, probabilistic cky-like approaches @xcite @xcite @xcite and transition-based approaches @xcite @xcite @xcite . probabilistic cky-like approaches normally exploit various kinds of lexical, syntactic and semantic features to compute the probability of the relation between the edus, and select the two edus with the highest relational probability to merge into one text span. in this way, the final discourse tree is generated. recently, various deep learning models are employed to capture hidden information to compute the relational probability, e.g. recursive deep models @xcite , and attention-based hierarchical neural network models @xcite . as an alternative, transition-based approaches employ the dependency structure to directly represent the relations between edus. @xcite first build a discourse dependency treebank by converting the rst-dt corpus and then apply graph based dependency parsing techniques to discourse parsing. @xcite propose a shift-reduce discourse parser using a representation learning approach to achieve the state-of-the-art performance. @xcite propose a pipelined two-stage parsing approach. first, a transition-based model is employed to parse a bare discourse tree. then, an independent relation labeller is adopted to determine discourse relations. @xcite present two variants of transition-based discourse parsing using a feedforward neural network model. @xcite build a transition based rst parser with implicit syntactic features. in particular, the information of sentence boundaries and paragraph boundaries is embedded as additional features. it is worthwhile to emphasize that, all the above studies on text-level discourse parsing employ the bottom-up approaches. so far, only @xcite and @xcite make the preliminary explorations on constructing sentence-level dts in a top-down fashion. @xcite proposed a framework for both the edu segmenter and the sentence-level discourse parser uniformly. following the work of @xcite al. (2019), liu et al. (2019) proposed hierarchical pointer network for better dependency and sentence-level discourse parsing. however, both studies consider merely sentencelevel discourse parsing. while it is simple but effective to encode entire sentence sequentially, entire text-level discourse larger than sentence, such as paragraph and document, is obviously much more complicated. statistics on the rst-dt corpus show each sentence only contains 2.5 edus on average while each document contains 55.6 edus on average. the representation for large text span can impact the parsing performance very much. in this paper, we present a top-down neural architecture to text-level discourse rhetorical structure parsing. different from @xcite and @xcite , we propose a hierarchical discourse encoder to better present the text span using both edus and split points. benefiting from effective representation for large text spans, our text-level discourse parser achieves competitive or even better results than those best reported discourse parsers either neural or non-neural with hand-engineered features. ## top-down neural architecture our top-down neural architecture consists of three parts, i.e., edu encoder, split point encoder and attention-based encoder-decoder. among them, the edu encoder and the split point encoder are responsible for representing the edus and the split points, respectively. different from @xcite and @xcite , we combine the representation of both edus and split points hierarchically to better represent the text span rather than only using the representation of the last edu as the representation of the text span. in this way, the global information can be exploited for our text-level discourse parsing. in the following, we take figure 1 as the example to illustrate the architecture. ## experimentation in this section, we systematically evaluate our topdown text-level discourse parser. ## conclusion in this paper, we propose a top-down neural architecture to text-level discourse parsing. in particular, we cast the discourse parsing task as a edu split point ranking task, where a split point is classified to different levels according to its rank, and the edus associated with the split point are arranged accordingly. in this way, we can determine the complete discourse rhetorical structure as a hierarchical tree structure. specifically, after encoding the edus and edu split points, a encoder-decoder with an internal stack is employed to generate discourse tree recursively. experimentation on the english rst-dt corpus and the chinese cdtb corpus shows the great effectiveness of our proposed approach. in the future work, we will focus on more effective discourse parsing with additional carefully designed features and joint learning with edu segmentation.