RST Parsing from Scratch (2021)

Abstract: We introduce a novel top-down end-to-end formulation of document level discourse parsing in the Rhetorical Structure Theory (RST) framework. In this formulation, we consider discourse parsing as a sequence of splitting decisions at token boundaries and use a seq2seq network to model the splitting decisions. Our framework facilitates discourse parsing from scratch without requiring discourse segmentation as a prerequisite; rather, it yields segmentation as part of the parsing process. Our unified parsing model adopts a beam search to decode the best tree structure by searching through a space of high scoring trees. With extensive experiments on the standard RST discourse treebank, we demonstrate that our parser outperforms existing methods by a good margin in both end-to-end parsing and parsing with gold segmentation. More importantly, it does so without using any handcrafted features, making it faster and easily adaptable to new languages and domains.

## introduction in a document, the clauses, sentences and paragraphs are logically connected together to form a coherent discourse. the goal of discourse parsing is to uncover this underlying coherence structure, which has been shown to benefit numerous nlp applications including text classification @xcite , summarization @xcite , sentiment analysis @xcite , machine translation evaluation @xcite and conversational machine reading @xcite . rhetorical structure theory or rst @xcite , one of the most influential theories of discourse, postulates a hierarchical discourse structure called discourse tree @xcite . the leaves of a dt are clause-like units, known as elementary discourse units (edus). adjacent edus and higher-order spans are connected hierarchically through coherence relations (e.g., contrast, expla-nation). spans connected through a relation are categorized based on their relative importancenucleus being the main part, with satellite being the subordinate one. fig. 1 exemplifies a dt spanning over two sentences and six edus. finding discourse structure generally requires breaking the text into edus (discourse segmentation) and linking the edus into a dt (discourse parsing). discourse parsers can be singled out by whether they apply a bottom-up or top-down procedure. bottom-up parsers include transition-based models @xcite @xcite or globally optimized chart parsing models @xcite @xcite . the former constructs a dt by a sequence of shift and reduce decisions, and can parse a text in asymptotic running time that is linear in number of edus. however, the transition-based parsers make greedy local decisions at each decoding step, which could propagate errors into future steps. in contrast, chart parsers learn scoring functions for sub-trees and adopt a cky-like algorithm to search for the highest scoring tree. these methods normally have higher accuracy but suffer from a slow parsing speed with a complexity of o(n 3 ) for n edus. the top-down parsers are relatively new in discourse @xcite @xcite . these methods focus on finding splitting points in each iteration to build a dt. however, the local decisions could still affect the performance as most of the methods are still greedy. like most other fields in nlp, language parsing has also undergone a major paradigm shift from traditional feature-based statistical parsing to end-toend neural parsing. being able to parse a document end-to-end from scratch is appealing for several key reasons. first, it makes the overall development procedure easily adaptable to new languages, domains and tasks by surpassing the expensive feature engineering step that often requires more time and domain/language expertise. second, the lack of an explicit feature extraction phase makes the training and testing (decoding) faster. because of the task complexity, it is only recently that neural approaches have started to outperform traditional feature-rich methods. however, successful document level neural parsers still rely heavily on handcrafted features @xcite @xcite . therefore, even though these methods adopt a neural framework, they are not "end-to-end" and do not enjoy the above mentioned benefits of an end-to-end neural parser. moreover, in existing methods (both traditional and neural), discourse segmentation is detached from parsing and treated as a prerequisite step. therefore, the errors in segmentation affect the overall parsing performance @xcite . in view of the limitations of existing approaches, in this work we propose an end-to-end top-down document level parsing model that: we make our code available at https://ntunlpsg.github.io/project/rst-parser ## model assuming that a document has already been segmented into edus, following the traditional approach, the corresponding discourse tree (dt) can be represented as a set of labeled constituents. traditionally, in rst parsing, discourse segmentation is performed first to obtain the sequence of edus, which is followed by the parsing process to assemble the edus into a labeled tree. in other words, traditionally discourse segmentation and parsing have been considered as two distinct tasks that are solved by two different models. on the contrary, in this work we take a radically different approach that directly starts with parsing the (unsegmented) document in a top-down manner and treats discourse segmentation as a special case of parsing that we get as a by-product. importantly, this novel formulation of the problem allows us to solve the two problems in a single neural model. our parsing model is generic and also works in the same way when it is fed with an edu-segmented text. before presenting the model architecture, we first formulate the problem as a splitting decision problem at the token level. ## experiments we conduct our experiments on discourse parsing with and without gold segmentation. we use the standard english rst discourse treebank or rst-dt @xcite for training and evaluation. it consists of 385 annotated wall street journal news articles: 347 for training and 38 for testing. we randomly select 10% of the training set as our development set for hyper-parameter tuning. following prior work, we adopted the same 18 courser relations defined in @xcite . for evaluation, we report the standard metrics span, nuclearity, relation and full f1 scores, computed using the standard parseval @xcite and rst-parseval @xcite metrics. ## related work discourse analysis has been a long-established problem in nlp. prior to the neural tsunami in nlp, discourse parsing methods commonly em-ployed statistical models with handcrafted features @xcite @xcite . even within the neural paradigm, most previous studies still rely on external features to achieve their best performances @xcite @xcite @xcite . these parsers adopt a bottom-up approach, either transition-based or chart-based parsing. recently, top-down parsing has attracted more attention due to its ability to maintain an overall view of the input text. inspired by the stack-pointer network @xcite for dependency parsing, @xcite first propose a seq2seq model for sentence-level parsing. zhang et al. ( 2020 ) extend this to the document level. @xcite adopt a greedy splitting mechanism for discourse parsing inspired by @xcite 's work in constituency parsing. by using pretrained models/embeddings and extra features (e.g., syntactic, text organizational features), these models achieve competitive results. however, their decoder infers a tree greedily. our approach differs from previous work in that it can perform end-to-end discourse parsing in a single neural framework without needing segmentation as a prerequisite. our model can parse a document from scratch without relying on any external features. moreover, it can apply efficient beam search decoding to search for the best tree. ## conclusion we have presented a novel top-down end-to-end method for discourse parsing based on a seq2seq model. our model casts discourse parsing as a series of splitting decisions at token boundaries, which can solve discourse parsing and segmentation in a single model. in both end-to-end parsing and parsing with gold segmentation, our parser achieves state-of-the-art, surpassing existing methods by a good margin, without relying on handcrafted features. our parser is not only more effective but also more efficient than the existing ones. this work leads us to several future directions. our short-term goal is to improve the model with better architecture and training mechanisms. for example, joint training on discourse and syntactic parsing tasks could be a good future direction since both tasks are related and can be modeled within our unified conditional splitting framework. we also plan to extend our parser to other languages.