Adversarial Learning for Discourse Rhetorical Structure Parsing (2021)

Abstract: Text-level discourse rhetorical structure (DRS) parsing is known to be challenging due to the notorious lack of training data. Although recent top-down DRS parsers can better leverage global document context and have achieved certain success, the performance is still far from perfect. To our knowledge, all previous DRS parsers make local decisions for either bottom-up node composition or top-down split point ranking at each time step, and largely ignore DRS parsing from the global view point. Obviously, it is not sufficient to build an entire DRS tree only through these local decisions. In this work, we present our insight on evaluating the pros and cons of the entire DRS tree for global optimization. Specifically, based on recent well-performing top-down frameworks, we introduce a novel method to transform both gold standard and predicted constituency trees into tree diagrams with two color channels. After that, we learn an adversarial bot between gold and fake tree diagrams to estimate the generated DRS trees from a global perspective. We perform experiments on both RST-DT and CDTB corpora and use the original Parseval for performance evaluation. The experimental results show that our parser can substantially improve the performance when compared with previous state-of-the-art parsers.

## introduction as the main linguistic theory on discourse rhetorical structure (drs), rhetorical structure theory (rst) @xcite describes an article as a discourse tree (dt). as illustrated in figure 1 , each leaf node of the tree corresponds to an elementary discourse unit (edu), and relevant leaf nodes are connected by relation and nuclearity (nucleus (n) or satellite (s)) tags to form high-layer discourse units (dus), where the nucleus is considered more important than the satellite. since the rst structure can well describe the organization of an article, it has been playing a central role in various down-stream tasks like summarization @xcite ), text categorization @xcite , and so on. with the release of various discourse corpora, text-level dsr parsing has been drawing more and more attention in the last decade. however, since the corpus annotation is usually time-consuming, existing drs corpora are much limited in size. for example, the english rst-dt @xcite corpus only contains 385 wsj articles, and the chinese cdtb @xcite corpus only contains 500 newswire articles. in this situation, previous studies usually rely on multifarious handengineered features @xcite @xcite @xcite . and all these systems perform drs parsing in a bottom-up fashion. until recently, some researchers turn to top-down drs parsing @xcite @xcite to explore the potential capabilities of data-driven models. nevertheless, text-level drs parsing is still challenging and worthy of in-depth exploration. theoretically, in supervised learning, annotated data corpora can provide neural models with specific learning objectives, and the corpus size limitation will weaken the learning of these goals. to mitigate this problem, we researchers need (i) an efficient model to better learn from the limited data and (ii) more high-quality training objectives to enhance the model learning. existing studies on text-level drs parsing show that considering the above-mentioned status quo, in this work, we study a global optimization method based on the well-performing top-down parsers. for model structure, we take the top-down parser of @xcite as our baseline system and make some improvements to it. for global optimization, we first utilize a novel strategy to transform both gold standard and predicted drs trees into tree diagrams with two color channels. after that, an lsgan-based adversarial bot is structured between gold and fake tree diagrams as an examiner for global estimation and optimization. experimental results on the rst-dt and cdtb corpora show that our approaches are effective. ## related work in the literature, previous studies on rst-style drs parsing mainly consist of two categories, i.e., bottom-up and top-down frameworks. for the first category, early studies on drs parsing heavily relied on hand-crafted features and linguistic characteristics @xcite @xcite . during the past decade, more and more researchers turned to data-driven approaches, and some effective strategies were proposed to adapt to the small-scale data corpora. among these studies, @xcite @xcite used some trivial features as auxiliaries in their data-driven systems; @xcite harnessed task supervision from related tasks, alternative views on discourse structures, and crosslingual data to alleviate the data insufficiency problem; @xcite introduced a two-stage parser to first parse a naked tree structure and then determine rhetorical relations for different discourse levels to mitigate data sparsity; @xcite employed both syntax information and discourse boundaries in their transition-based system and achieved good performance. for the second category, some researchers @xcite @xcite turned to top-down frameworks to tap the potential capabilities of data-driven models. among them, @xcite have achieved certain success in sentencelevel drs parsing. nevertheless, due to the longdistance dependency over the discourse, text-level drs parsing remains challenging. to alleviate this problem, @xcite proposed a top-down architecture tailored for text-level drs parsing. @xcite used contextualized word representation and proposed to parse a document in three granularity levels for good performance. in the past decade, gans have achieved great progress in nlp @xcite @xcite . however, to our knowledge, there is still no research on adversarial learning in drs parsing so far. in this work, we explore to adversarially train a discriminator to estimate the quality of the entire drs tree for global optimization. notably, we propose to transform each drs tree into a continuous tree diagram, and thus our adversarial method does not suffer from the "discrete data" problem. ## baseline top-down architecture in this section, we give a brief introduction to our baseline system, the top-down parser of @xcite , and make some improvements to it. the parsing process is illustrated in figure 3 . hierarchical split point encoding. for split point representation 1 , zhang et al. (2020) introduced a hierarchical rnn-cnn architecture in their paper. firstly, they use an attention-based gru encoder to encode each edu, obtaining e i . then, the obtained edu vectors are fed into another bigru for context modeling, as shown in figure 3 . next, a cnn net with a window size of 2 and a stride size of 1 is built for each window of edus in the discourse for split point encoding. to our knowledge, zhang et al. (2020) produced dummy split points at both ends of a discourse. since the dummy split points do not participate in the split point selection process, they could be redundant. here, we try to simplify the parsing procedure with the dummy split points discarded, as shown in figure 3 . following previous work @xcite , we also splice the sentence-and paragraph-level boundary feature vectors to the representation of split points to enhance the encoder model. top-down split point ranking. after achieving split point representations, an encoder-decoder is used to rank the split points, as shown in figure 3 . during encoding, the previously obtained split point vectors are taken as input to the bigru encoder, obtaining h 0 , . . . , h n-2 . during decoding, a uni-directional gru with an internal stack is used to control the split point ranking process. initially, the stack contains only one element, i.e., indexes of the boundary split points in the discourse. notably, since we do not add dummy split points in this parser, we allow patterns like (τ, τ ) to appear in the stack. at the j-th step, the tuple (b, e) is popped from the stack and we enter the concatenated c @xmath0; h e ) into the decoder for d j . after that, a biaffine function @xcite is built between the encoder and decoder outputs for split point ranking. different from @xcite , all split points in the interval @xcite are selectable in this work. at the step j, we calculate the attention score between h i and d j as: ## adversarial learning for drs parsing this section introduces the proposed adversarial learning method which consists of two parts: graphical representation of gold and fake drs trees and the adversarial model learning process. ## conclusion in this research, we explored a global optimization method based on recent top-down frameworks. particularly, we proposed a novel strategy to transform both gold standard and predicted drs trees into tree diagrams with two color channels. on this basis, we produced an lsgan-based adversarial bot between gold and fake trees for global optimization. experimental results on two popular corpora showed that our proposed adversarial approach is effective in drs parsing and has established new state-of-the-art results for both corpora.