Multilingual Neural RST Discourse Parsing (2020)

Abstract: Text discourse parsing plays an important role in understanding information flow and argumentative structure in natural language. Previous research under the Rhetorical Structure Theory (RST) has mostly focused on inducing and evaluating models from the English treebank. However, the parsing tasks for other languages such as German, Dutch, and Portuguese are still challenging due to the shortage of annotated data. In this work, we investigate two approaches to establish a neural, cross-lingual discourse parser via: (1) utilizing multilingual vector representations; and (2) adopting segment-level translation of the source content. Experiment results show that both methods are effective even with limited training data, and achieve state-of-the-art performance on cross-lingual, document-level discourse parsing on all sub-tasks.

## introduction rhetorical structure theory (rst) @xcite is one of the most influential theories of discourse analysis, under which a document is represented by a hierarchical discourse tree. as shown in figure 1a , the leaf nodes of an rst tree are text spans named elementary discourse units (edus), and the edus are connected by rhetorical relations (e.g., cause, contrast) to form larger text spans until the entire document is included. the rhetorical relations are further categorized to nucleus (core part) and satellite (subordinate part) based on their relative importance. thus, document-level discourse parsing consists of three sub-tasks: tree construction, nuclearity determination and relation classification. moreover, downstream natural language processing tasks can benefit from rst-based structure-aware document analysis, such as summarization @xcite and machine comprehension @xcite . by utilizing various linguistic characteristics (e.g., n -gram and lexical features, syntactic and organizational features), statistical approaches have obtained substantial improvement on the english rst-dt benchmark @xcite @xcite @xcite . recently, neural networks have been making inroads into discourse analysis frameworks, such as attention-based hierarchical encoding @xcite and integrating neural-based syntactic features into a transition-based parser @xcite . @xcite and their follow-up work @xcite successfully explored encoder-decoder neural architectures on sentence-level discourse analysis, with a top-down parsing procedure. although discourse parsing has received much research attention and progress, the models are mainly optimized and evaluated in english. the main challenge is the shortage of annotated data, since manual annotation under the rst framework is labor-intensive and requires specialized linguistic knowledge. for instance, the most popular benchmark english rst-dt corpus @xcite only contains 385 samples, which is much smaller than those of other natural language processing tasks. the treebank size of other languages such as german @xcite , dutch @xcite and basque @xcite are even more limited. such limitations make it difficult to achieve acceptable performance on these languages required to fully support downstream tasks, and also lead to poor generalization ability of the computational approaches. since the treebanks of different languages share the same underlying linguistic theory, data-driven approaches can benefit from joint learning on multilingual rst resources @xcite . therefore, in this paper, we investigate two methods to build a cross-lingual neural discourse parser: (1) from the embedding perspective: with the cross-lingual contextualized language models, we can train a parser on the shared semantic space from multilingual sources without employing a language indicator; (2) from the text perspective: since each edu is a semantically-cohesive unit, we can unify the target language space by edu-level translation, while preserving the original edu segmentation and the discourse tree structures (see figure 1c ). to this end, we adapted and enhanced an end-to-end neural discourse parser, and investigated the two proposed approaches on 6 different languages. while the rst data for training is still in a small scale, we achieved the state-of-the-art performance on all fronts, significantly surpassing previous models, and even approaching the upper bound of human performance. moreover, we conducted a topic modeling analysis on the collected multilingual treebanks to evaluate the model generality across various domains. ## experimental results and analyses in this section, we describe data collection and present the experimental setting, results and analyses of the proposed methods. ## conclusion in this paper, we investigated two approaches for cross-lingual neural discourse parsing. experimental results show that both utilizing cross-lingual representation and adopting segment-level translation contribute to obtaining state-of-the-art performance on various treebanks. moreover, monolingual models can also benefit from cross-lingual training by introducing data from more domains. for future work, we consider conducting domain adaption via few-shot learning to make our approach more generalizable.