C hinese Paragraph-level Discourse Parsing with Global Backward and Local Reverse Reading (2020)

Abstract: Discourse structure tree construction is the fundamental task of discourse parsing and most previous work focused on English. Due to the cultural and linguistic differences, existing successful methods on English discourse parsing cannot be transformed into Chinese directly, especially in paragraph level suffering from longer discourse units and fewer explicit connectives. To alleviate the above issues, we propose two reading modes, i.e., the global backward reading and the local reverse reading, to construct Chinese paragraph level discourse trees. The former processes discourse units from the end to the beginning in a document to utilize the left-branching bias of discourse structure in Chinese, while the latter reverses the position of paragraphs in a discourse unit to enhance the differentiation of coherence between adjacent discourse units. The experimental results on Chinese MCDTB demonstrate that our model outperforms all strong baselines.

## introduction discourse parsing aims to study the internal structure of texts and understand the semantic relationship between various kinds of text units, such as clauses, sentences, sentence groups, paragraphs, and chapters. since it is fundamental to understanding the overall text semantics, discourse parsing has been widely applied to various natural language processing (nlp) applications, such as sentiment analysis @xcite , question answering @xcite , and summarization @xcite . as one of the most influential theories in discourse parsing, rhetorical structure theory (rst) @xcite represents text as a hierarchical structure known as a discourse tree. in the literature, various rst-style corpora have been built, such as rst discourse treebank (rst-dt) @xcite and macro chinese discourse treebank (mcdtb) @xcite . in rst-style discourse parsing, the parser first identifies whether there is a rhetorical relationship between discourse units to construct a naked tree and then recognizes the nuclearity and relation labels for each relationship, as shown in figure 1 . according to the granularity of the leaf nodes, the discourse tree is divided into three levels: clause level, sentence level and paragraph level @xcite . this paper focuses on constructing paragraph-level chinese discourse trees where the leaf node is a paragraph. we call a leaf node that contains only one paragraph as pdu (paragraph-level discourse unit) to distinguish the elementary discourse unit (edu) at the clause level. it is more important for downstream tasks as the upper part of a complete discourse tree. in english, there is a series of successes @xcite @xcite in rst-dt @xcite , especially @xcite and @xcite got excellent performance in clause-level discourse parsing. however, owing to linguistic and cultural differences, discourse structure is different between chinese and english: english is linear while chinese is spiral @xcite . therefore, the state-of-the-art model in english cannot be directly transformed into chinese well. besides, fewer studies @xcite on paragraphlevel discourse parsing may suffer from data sparsity and its performance is much lower than that of sentence-level or clause-level. it is more challenging for the paragraph-level discourse parsing because of its longer text length (over 4.5 times), fewer samples (less than 20%) and fewer connectives than clause-level, especially in chinese, as shown in in this paper, we analyzed the differences in paragraph-level discourse structure between english and chinese and proposed a global backward reading mode that processes discourse units from the end to the beginning in a document to utilize the bias of discourse structure in chinese. moreover, we proposed the triple semantic matching model based on bert (tm-bert) that views three adjacent discourse units as a triangle to capture the relationship across discourse units better. to deal with longer paragraph-level discourse units, we used only 1-2 pdus to represent a discourse unit and proposed a local reverse reading mode to reverse the order of pdus. the experimental results on chinese mcdtb demonstrate that our proposed model with global backward and local reverse reading outperforms all strong baselines. ## related work in english, rst-dt @xcite is one of the popular discourse corpora @xcite @xcite , which annotates the discourse structure, nuclearity, and relationship of a document. most previous studies have focused on complete discourse parsing and can be mainly categorized into the shift-reduce algorithm @xcite @xcite , the probabilistic cky-like algorithm @xcite @xcite , and the bottom-up algorithm @xcite @xcite . recently, the generative algorithm @xcite and the top-down algorithm @xcite tried out discourse parsing. at the high level (paragraph-level), @xcite built paragraph-level discourse trees by bottom-up algorithm after pruning and revising the original discourse trees in the rst-dt. in the literature, there are three kinds of strategies for representing larger discourse units. most traditional machine learning methods @xcite @xcite extracted semantic features from the first and last edu of a large discourse unit. neural network methods @xcite tended to choose all edus of the larger discourse unit as its semantic representation. other works @xcite selected only one edu (e.g. the first one) as the representation according to nuclearity. in chinese, a few discourse corpora @xcite were annotated at the clause-level. to the best of our knowledge, the mcdtb @xcite is the only available paragraph-level chinese corpus. the bottom-up algorithm @xcite was the earliest applied to construct paragraph-level discourse structure trees in mcdtb. recently, @xcite built discourse structure trees by the shift-reduce algorithm in mcdtb. all of them used all pdus for the semantic representation of a larger discourse unit. ## motivation and methods in this section, we propose a global backward reading mode to construct a paragraph-level discourse tree by the shift-reduce algorithm to utilize the bias of chinese discourse structure. in particular, we propose a triple semantic matching model based on bert as a local model in the shift-reduce algorithm to capture the across discourse relationship better. in addition, we propose a local reverse reading mode to enhance the differentiation of discourse units coherence and obtain eight kinds of representation strategies to deal with the longer discourse units at the paragraph-level. ## experimentation 4.1 experimental settings and baselines in chinese, we evaluate our model on mcdtb with 720 documents annotated paragraph-level discourse tree. following the previous work @xcite , we transform the non-binary trees of the original data into right-binary trees in mcdtb. in particular, to balance the training set and the test set, we divide the documents containing different numbers of paragraphs into the training set (576 documents) and the test set (144 documents) according to the proportion. we randomly select 10% of the training set as the validation set. following @xcite and @xcite , we use micro-f 1 as the evaluation metric, which evaluates how likely discourse units are correctly merged. this evaluation method is fairer though it will get lower scores than what was reported by the existing systems. to verify the effectiveness of our model, we use five strong baselines. rule (left/right) refers to producing a complete left-branching/right-branching tree by always merging the leftmost/rightmost two discourse units. jpc18 @xcite is the first model to identify the paragraph-level discourse structure but has not built a discourse tree for a document yet. we modified it to build a paragraphlevel discourse tree from bottom to top. zcp19 @xcite is the state-of-the-art model for constructing paragraph-level discourse trees by the shift-reduce algorithm. since the above two systems did not recognize the nuclearity and classify the relation of a span, we did that through stgsn @xcite after generating new spans. due to the overwhelming impact of bert in many nlp applications, bert is selected as the local model for building discourse trees with the shift-reduce algorithm, to be another baseline. ls19 @xcite is the state-of-the-art clause-level discourse parser by the shift-reduce algorithm in rst-dt, which is transformed to process chinese. ## conclusions in this paper, we propose a global backward reading mode in the shift-reduce algorithm, which processes discourse units from the end to the beginning of a document to utilize the left-branching bias of discourse structure better in chinese. besides, we propose a tm-bert as the local model and eight representation strategies with the local reverse reading mode to enhance the differentiation of the coherence between discourse units. the experiment results on mcdtb show that our method achieved the state-of-the-art in discourse tree construction. in the future, we will explore how to construct an end-to-end paragraph-level discourse parser that well integrates discourse structure learning with nuclearity recognition and relation classification in chinese.