Not Just Classification: Recognizing Implicit Discourse Relation on Joint Modeling of Classification and Generation (2021)

Abstract: Implicit discourse relation recognition (IDRR) is a critical task in discourse analysis. Previous studies only regard it as a classification task and lack an in-depth understanding of the semantics of different relations. Therefore, we first view IDRR as a generation task and further propose a method joint modeling of the classification and generation. Specifically, we propose a joint model, CG-T5, to recognize the relation label and generate the target sentence containing the meaning of relations simultaneously. Furthermore, we design three target sentence forms, including the question form, for the generation model to incorporate prior knowledge. To address the issue that large discourse units are hardly embedded into the target sentence, we also propose a target sentence construction mechanism that automatically extracts core sentences from those large discourse units. Experimental results both on Chinese MCDTB and English PDTB datasets show that our model CG-T5 achieves the best performance against several state-of-the-art systems.

## introduction discourse relation describes the logical connection between two discourse units (e.g., clauses, sentences, or paragraphs). as an essential discourse analysis task, discourse relation recognition is to recover what rhetorical relation exists between discourse units (dus). due to the absence of explicit connectives, implicit discourse relation recognition (idrr) is still a challenging task and research hotspot. moreover, idrr is beneficial to many downstream natural language processing (nlp) applications, such as machine translation @xcite ), text generation @xcite , and text summarization @xcite . with the success of representation learning in discourse analysis, most existing methods of idrr * corresponding author focus on three aspects: enhancing discourse units representation @xcite @xcite , enhancing semantic interaction @xcite @xcite , and joint learning with other tasks @xcite @xcite . they all regard idrr as a classification task and lack a deeper understanding of the relation semantics; even recent work @xcite with labeling embedding cannot directly introduce the prior knowledge of the discourse relation semantics into their models. ningbo free trade zone ... has achieved du 1 fruitful results after three years of construction. ... the development level is among the best... ... the ningbo free trade zone had completed du 2 a total of us$812 million in import and ... at the same time, the bonded zone has ... relation elaboration target sentence du 2 is a detailed description of du 1 . in the stage of implicit discourse relation annotation, annotators usually not only give the relation type but also provide a description or basis for the relation. therefore, we hope the model, like a human, gives a target sentence instead of a simple label index for understanding the relation deeper. the target sentence should describe the core information of two dus and their relation through natural language. as an example in the question-answering (qa) method can incorporate prior knowledge into the model using the generation instead of the classification. it has achieved success in a few fine-grained tasks, such as named entity recognition @xcite and coreference resolution @xcite . however, it is a challenge to directly apply the traditional question-answering method to idrr due to the following two issues. first, unlike the above finegrained tasks where the answers or the clues exist in the input context, the discourse relation in idrr is implicit between two dus and does not appear explicitly in the context. it makes the idrr model unable to extract the answer from the input directly. second, since du usually is large and contains several sentences, the target sentence which contains two dus as shown in besides, the classification model and the generation model have their complementary advantages. the former usually has better performance on major classes due to searching in a limited space, while the latter can introduce prior knowledge to capture the semantics of minor classes better, and its result is a natural language expression with stronger interpretability. therefore, how to combine the advantages of the classification model and the generation model is another challenge. different from previous work, we first regard idrr as a text generation task and design three forms of the target sentence to represent prior knowledge. in particular, inspired by the annotation work @xcite , we use questions instead of answers to describe the discourse relation between two dus as the target sentence. therefore, our model can understand discourse relation deeper by generating a target sentence that describes the relation meaning instead of an index of the relation type. moreover, we design a method to automatically extract the core information from these large dus by semantic role labeling and then compress the du into a short form. to address the second challenge, we propose a cg-t5 model that combines the classification and generation model to leverage their complementary advantages. specifically, inspired by pre-trained models (e.g. bert @xcite and gpt-2 @xcite )), we first extract hidden states of the encoder in t5 @xcite and feed them to the span representation layer for the classification task and then use the t5 decoder for the generation task. finally, we combine these two models with a jointly learning mechanism. our cg-t5 can integrate the advantage of two different models: the classification model constrains the generation model, while the generation model explains and supports the classification model. in summary, the main contributions of this paper are fourfold: ## related work we first briefly introduce relative discourse corpora, then summarize the existing methods of idrr, and finally introduce the success of the questionanswering method in fine-grained tasks. ## idrr as text generation to deeper understand the semantics of discourse relation, we regard idrr as a text generation task, as shown in figure 1 . unlike previous work, we use a generation model instead of the classification model to generate the target sentence representing the semantics of discourse relation. then, we obtain the relation type by mapping it into the corresponding discourse relation. in this section, we mainly describe our solution for the first challenge of obtaining the target sentence: its different forms and its corresponding construction method. ## cg-t5 model to integrate the advantages of the classification model and the generation model in idrr, we propose the classification and generation t5 (cg-t5) model that recognizes relation class and generates the target sentence simultaneously, as shown in figure 3 . thanks to the excellent performance of t5 @xcite on many nlp tasks, we choose it as the backbone of cg-t5. better than bert @xcite , an encoder architecture for classification, and gpt-2 @xcite , a decoder architecture for generation, t5 has an encoder-decoder architecture that allows us to naturally joint learning classification and generation foot_2 . cg-t5 comprises three parts: the classification module based on the encoder, the generation module based on the decoder, and the joint learning module. therefore, when given two dus as the input, cg-t5 has two outputs: the class label from the classification module and the target sentence from the generation module. ## experimentation in this section, we first introduce the datasets and experimental settings. then, we evaluate our model cg-t5 on mcdtb and pdtb. ## analysis to further demonstrate the effectiveness of our cg-t5, we choose the 1st-fold data set of mcdtb as an example to further analyze the following two parts: the ablation experiments of joint modeling and the text generation quality assessment.