GCDT : A C hinese RST Treebank for Multigenre and Multilingual Discourse Parsing (2022)

Abstract: A lack of large-scale human-annotated data has hampered the hierarchical discourse parsing of Chinese. In this paper, we present GCDT, the largest hierarchical discourse treebank for Mandarin Chinese in the framework of Rhetorical Structure Theory (RST). GCDT covers over 60K tokens across five genres of freely available text, using the same relation inventory as contemporary RST treebanks for English. We also report on this dataset’s parsing experiments, including state-of-the-art (SOTA) scores for Chinese RST parsing and RST parsing on the English GUM dataset, using cross-lingual training in Chinese and English with multilingual embeddings.

## introduction hierarchical discourse parsing has shown its importance in document-level natural language understanding (nlu) tasks, such as text summarization @xcite @xcite @xcite and sentiment analysis @xcite @xcite . among discourse frameworks, rhetorical structure theory (rst, @xcite is a document-level discourse analysis formalism that assumes a single-rooted, labeled constituent tree for each document. unlike the penn discourse treebank (pdtb, @xcite ), which primarily focuses on local discourse relations and for which more data exists in chinese, rst builds a document tree using nested relations within a sentence, across sentences, and across paragraphs. rst is thus particularly significant at the macrolevel, which is more challenging for understanding discourse organization than at the micro-level @xcite @xcite . despite the complexity of rst and the human labor required, many new datasets have come out in the past decades @xcite , including english @xcite , basque @xcite , bangla @xcite , brazilian portuguese @xcite , dutch @xcite , german @xcite , persian @xcite , russian @xcite , spanish @xcite , and the spanish-chinese parallel corpus @xcite . however, a substantial gap remains in the availability of document-level hierarchical discourse datasets for non-european languages, particularly chinese, of sufficient magnitude for training contemporary neural parsers. aside from the small parallel spanish-chinese dataset by in this paper, we present the georgetown chinese discourse treebank (gcdt) corpus, 1 a new, freely available, multi-genre rst corpus of 50 medium to long documents for mandarin chinese, as the sample subtree shown in figure 1 . the corpus covers over 60k tokens and 9k elementary discourse units (edus). in addition to presenting the sota parsing results in monolingual settings for the dataset, we jointly train a model with both english and chinese datasets, testing finetuning and automatic translation-based approaches to improve performance in both chinese and english as the target language. experimenting with different monolingual and multilingual embeddings, we find that joint training and translation improve performance on the smaller chinese and larger english datasets. however, finetuning only helps with the smaller chinese data. finally, we show that monolingual roberta embeddings outperform multilingual embeddings in applicable settings. still, the best overall performance is achieved using chinese and english data in a multilingual training regime. the spanish-chinese parallel corpus @xcite is a small chinese rst corpus (15k tokens) constructed for translation studies. to support this goal, its edus are adjusted to align between spanish and chinese rather than staying faithful to the syntax of the individual languages. its relation inventory is also distinct from inventories used for english corpora, as are the segmentation criteria used in the corpus, limiting its compatibility with other datasets. another older chinese rst corpus was reported in @xcite with 97 news commentaries annotated. however, to our knowledge, the dataset is no longer accessible or used in rst pars-ing or other tasks @xcite . ## gcdt: georgetown chinese discourse treebank gcdt is an open-source multi-genre rst dataset in mandarin chinese. following the design of gum @xcite , gcdt contains 50 documents, 10 from each of 5 genres which also appear in gum: academic articles, biographies (bio), interview conversations, news, and how-to guides (whow), as shown in relation annotation gcdt builds up constituent discourse trees based on gold edus using rstweb @xcite . we use the enhanced two-level relation labels from gum v8.0.0 with 15 coarse and 32 fine-grained relations (see appendix a for relation distributions in gcdt and gum). ## experiments we present benchmark results on gcdt using the sota multilingual parser, dmrst @xcite . results are shown in two experimental settings: monolingual training using only one dataset (either chinese gcdt or english gum v8.0.0) and multilingual training using data from both corpora (gcdt+gum). besides directly combining corpora from the two languages, we also experiment with finetuning and automatic edu-wise translation. we use the same set of hyperparameters as reported in @xcite . similarly, we also report monolingual and multilingual parsing performance on gum v8.0.0. datasets cross-genre adaptability remains a bottleneck in rst parsing @xcite . to isolate cross-lingual versus cross-genre influences, we conduct monolingual and multilingual experiments using the following data compositions: 1) gcdt: 50 chinese documents from 5 genres; 2) gum-12: 193 english documents from 12 genres; 3) gum-5: 99 gum documents from the same 5 genres in gcdt. ## results we present monolingual and multilingual results on gcdt and gum in tables 2 and 3 , as well as genre-wise performance on gcdt in firstly, joint training outperformed monolingual results in all three test scenarios: gcdt, gum-5, and gum-12. for example, training on gcdt+gum-12 using xlm roberta achieved an f_rel of 52.61 on gcdt, higher than the 50.45 trained with only gcdt, and the same embedding. secondly, more genres from gum (gcdt+gum-12) achieved better performance than training only using the same genres (gcdt+gum-5) when tested on gcdt. thirdly, pretraining on the gcdt+gumcombined training sets and training on the training set of the target corpus improves performance on chinese gcdt but deteriorates on the english gum. we hypothesize that with more english training data available, there is less headroom for improvement. in contrast, finetuning for the smaller chinese dataset added to the comparatively little information available to the parser. lastly, results show that augmenting with automatic translation and using monolingual embeddings achieved the best performance on three of the four test scenarios, while the best result on gcdt was achieved by training together with gum-12 and finetuning on gcdt. genre-wise analysis we further select three models trained in the monolingual gcdt and translation-augmented scenarios, gcdt+gum-5 and gcdt+gum-12, using the chinese roberta embedding @xcite . ## conclusion this paper presents gcdt, the largest rst dataset for mandarin chinese, which closely follows established rst guidelines and is highly comparable to existing english rst corpora. besides evaluating annotation quality and establishing sota results on this dataset in monolingual settings, we also jointly train on gcdt and a similar english corpus-gum-and demonstrate that multilingual training and automatic edu translation boost parser performance. however, finetuning is only helpful when targeting the smaller chinese dataset. we further conduct per-genre analyses and show that parsing performance varies widely between some genres but less between others. we hope that this dataset can alleviate the lack of training resources for hierarchical discourse parsing in chinese and facilitate multilingual and multi-genre rst parsing, as well as other downstream nlp tasks.