A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in C hinese Conversational Speech (2021)

Abstract: In this paper, we present a neural model for joint dropped pronoun recovery (DPR) and conversational discourse parsing (CDP) in Chinese conversational speech. We show that DPR and CDP are closely related, and a joint model benefits both tasks. We refer to our model as DiscProReco, and it first encodes the tokens in each utterance in a conversation with a directed Graph Convolutional Network (GCN). The token states for an utterance are then aggregated to produce a single state for each utterance. The utterance states are then fed into a biaffine classifier to construct a conversational discourse graph. A second (multi-relational) GCN is then applied to the utterance states to produce a discourse relation-augmented representation for the utterances, which are then fused together with token states in each utterance as input to a dropped pronoun recovery layer. The joint model is trained and evaluated on a new Structure Parsing-enhanced Dropped Pronoun Recovery (SPDPR) data set that we annotated with both two types of information. Experimental results on the SPDPR dataset and other benchmarks show that DiscProReco significantly outperforms the state-of-the-art baselines of both tasks.

## introduction pronouns are often dropped in chinese conversations as the identity of the pronoun can be inferred from the context @xcite without causing the sentence to be incomprehensible. the task of dropped pronoun recovery (dpr) aims to locate the position of the dropped pronoun and identify its type. conversational discourse parsing (cdp) is another important task that aims to analyze the discourse relations among utterances ## problem formulation we first introduce the problem formulation of these two tasks. following the practices in @xcite @xcite , we formulate dpr as a sequence labeling problem. dpr aims to recover the dropped pronouns in an utterance by assigning one of 17 labels to each token that indicates the type of pronoun that is dropped before the token @xcite . cdp is the task of constructing the conversational discourse structure by predicting the discourse relation @xcite among utterances. the discourse relations may characterize one utterance as agreeing with, responding to, or indicate understanding of another utterance in the conversational context. let us denote an input pro-drop utterance of n tokens as @xmath0, w 2 , • • • , w n ), and its contextual utterances as @xmath1, x 2 , • • • , x m ) where the ith contextual utterance x i is a sequence of l i tokens: x @xmath2,1 , • • • , w i,l i ). our task aims to (1) model the distribution p(x j |x i , c) to predict the relation between each pair of utterances (i.e., (x i , x j )) for cdp, and (2) model ŷ = arg max y p(y|x, c) to predict the recovered pronoun sequence ŷ for the input utterance x. each element of ŷ is chosen from one of the t possible labels from @xmath3, • • • , y t-1 }∪{none} to indicate whether a pronoun is dropped before the corresponding token in utterance x and the type of the dropped pronoun. the label "none" means no pronoun is dropped before this token. ## the spdpr dataset to verify the effectiveness of discproreco, we need a conversational corpus containing the annotation of both dropped pronouns and discourse relations. to our knowledge, there is no such a public available corpus. therefore, we constructed the first structure parsing-enhanced dropped pronoun recovery (spdpr) dataset by annotating the discourse structure information on a popular dropped pronoun recovery dataset (i.e., chinese sms). the chinese sms/chat dataset consists of 684 multi-party chat files and is a popular benchmark for dropped pronoun recovery @xcite . in this study, we set the size of the context snippet to be 8 utterances which include the current pro-drop utterance plus 5 utterances before and 2 utterances after. when performing discourse relation annotation we ask three linguistic experts to independently choose a head utterance for the current utterance from its context and annotate the discourse relation between them according to a set of 8 pre-defined relations (see appendix a). the inter-annotator agreement for discourse relation annotation is 0.8362, as measured by fleiss's kappa. the resulting spdpr dataset consists of 292,455 tokens and 40,280 utterances, averaging 4,949 utterance pairs per relation, with a minimum of 540 pairs for the least frequent relation and a maximum of 12,252 for the most frequent relation. the spdpr dataset also annotates 31,591 dropped pronouns (except the "none" category). ## related work dropped pronoun recovery is a critical technique that can benefit many downstream applications @xcite @xcite . @xcite for the first time proposed this task, and utilized a maximum entropy classifier to recover the dropped pronouns in text messages. @xcite further employed a linear-chain crf to jointly predict the position and type of the dropped pronouns in a single utterance using hand-crafted features. due to the powerful semantic modeling capability of deep learning, zhang et al. (2019); yang et al. (2019) introduced neural network methods to recover the dropped pronoun by modeling its semantics from the context. all these methods represent the utterances without considering the relationship between utterances, which is important to identify the referents. zero pronoun resolution is also a closely related line of research to dpr @xcite @xcite . the main difference between dpr and zero pronoun resolution task is that dpr considers both anaphoric and non-anaphoric pronouns, and doesn't attempt to resolve it to a referent. existing discourse parsing methods first predicted the probability of discourse relation, and then applied a decoding algorithm to construct the discourse structure @xcite @xcite . a deep sequential model @xcite was further presented to predict the discourse dependencies utilizing both local information of two utterances and the global information of existing constructed discourse structure. all these methods consider how to do relation prediction independently. however, in this work, we explore the connection between the cdp and dpr, and attempt to make these two tasks mutually enhance each other. ## conclusion this paper presents that dropped pronoun recovery and conversational discourse parsing are two strongly related tasks. to make them benefit from each other, we devise a novel framework called discproreco to tackle these two tasks simultaneously. the framework is trained in a joint learning paradigm, and the parameters for the two tasks are jointly optimized. to facilitate the study of the problem, we created a large-scale dataset called spdpr which contains the annotations of both dropped pronouns and discourse relations. experimental results demonstrated that discproreco outperformed all baselines on both tasks. relation description different participant agreement a participant provides a response to a previous request or suggestion understanding a participant indicates understanding of a previous utterance directive a participant asks another one to do something question a general request for another participant answer a participant provides the information requested by another participant feedback a participant responds to another speaker's utterance same participant expansion a participant provides an elaboration of a previous utterance contingency a participant continues to say something else