DMRST : A Joint Framework for Document-Level Multilingual RST Discourse Segmentation and Parsing (2021)

Abstract: Text discourse parsing weighs importantly in understanding information flow and argumentative structure in natural language, making it beneficial for downstream tasks. While previous work significantly improves the performance of RST discourse parsing, they are not readily applicable to practical use cases: (1) EDU segmentation is not integrated into most existing tree parsing frameworks, thus it is not straightforward to apply such models on newly-coming data. (2) Most parsers cannot be used in multilingual scenarios, because they are developed only in English. (3) Parsers trained from single-domain treebanks do not generalize well on out-of-domain inputs. In this work, we propose a document-level multilingual RST discourse parsing framework, which conducts EDU segmentation and discourse tree parsing jointly. Moreover, we propose a cross-translation augmentation strategy to enable the framework to support multilingual parsing and improve its domain generality. Experimental results show that our model achieves state-of-the-art performance on document-level multilingual RST parsing in all sub-tasks.

## introduction rhetorical structure theory (rst) @xcite is one of the predominant theories for discourse analysis, where a document is represented by a constituency tree with discourserelated annotation. as illustrated in figure 1 , the paragraph is split to segments named elementary discourse units (edus), as the leaf nodes of the tree, and they are further connected by rhetorical relations (e.g., elaboration, attribution) to form larger text spans until the entire document is included. the spans are further categorized to nucleus (the core part) or satellite (the subordinate part) based on their relative importance since discourse parsing provides structural information of the narrative flow, downstream natural language processing applications, such as reading comprehension @xcite , sentiment analysis @xcite , and text summarization @xcite , can benefit from incorporating semantic-related information. rst discourse parsing has been an active research area, especially since neural approaches and large-scale pre-trained language models were introduced. on the test set of the english rst benchmark @xcite , the performance of automatic parsing is approaching that of human annotators. however, compared with other offthe-shelf text processing applications like machine translation, rst parsers are still not readily applicable to massive and diverse samples due to the following challenges: (1) most parsers take edu segmentation as a pre-requisite data preparation step, and only conduct evaluations on samples with gold edu segmentation. thus it is not straightforward to utilize them to parse raw documents. in this paper, to tackle the aforementioned challenges, we propose a joint framework for documentlevel multilingual rst discourse analysis. to achieve parsing from scratch, we enhance a topdown discourse parsing model with joint learning of edu segmentation. since the well-annotated rst treebanks in different languages share the same underlying linguistic theory, data-driven approaches can benefit from joint learning on multilingual rst resources @xcite . inspired by the success of mixed multilingual training @xcite , we further propose a cross-translation data augmentation strategy to improve rst parsing in both language and domain coverage. we conduct extensive experiments on rst treebanks from six languages: english, spanish, basque, german, dutch, and portuguese. experimental results show that our framework achieves state-of-the-art performance in different languages and on all sub-tasks. we further investigate the model's zero-shot generalization capability, by assessing its performance via language-level cross validation. additionally, the proposed framework can be readily extended to other languages with existing treebanks. the pre-trained model is built as an off-the-shelf application, and can be applied in an end-to-end manner. ## methodology in this section, we elaborate on the proposed joint multilingual rst discourse parsing framework. we first integrate edu segmentation into a topdown transformer-based neural parser, and show how to leverage dynamic loss weights to control the balance of each sub-task. we then propose cross-translation augmentation to improve the multilingual and domain generalization capability. ## experimental results in this section, we elaborate on experiment settings of the multilingual rst segmentation and parsing task, compare our proposed framework with previous models, and conduct result analysis. ## analysis on zero-shot generalization incorporating discourse information is beneficial to various downstream nlp tasks, but only a small number of languages possess rst treebanks. such treebanks have limited annotated samples, and it is difficult to extend their sample size due to annotation complexity. to examine if our proposed multilingual framework can be adopted to languages without any monolingual annotated sample (e.g., italian, polish), we conducted a zero-shot analysis via language-level cross validation. in each round, we select one language as the target language, and rst treebanks from the remaining 5 languages are used to train the multilingual parser. we then evaluate it on the test set from the target language. for example, we assume that a small set of portuguese articles is to be parsed, and we only have training samples from the other 5 languages (i.e., en, es, de, nl, and eu). then zero-shot inference is conducted on portuguese. as shown in ## conclusions in this work, we proposed a joint framework for document-level multilingual rst discourse parsing, which supports edu segmentation as well as discourse tree parsing. experimental results showed that the proposed framework achieves stateof-the-art performance on document-level multilingual discourse parsing on six languages in all aspects. we also demonstrated its inference capability when limited training data is available, and it can be readily extended to other languages.