Task-Aware Self-Supervised Framework for Dialogue Discourse Parsing (2023)

Abstract: Dialogue discourse parsing is a fundamental natural language processing task. It can benefit a series of conversation-related downstream tasks including dialogue summarization and emotion recognition in conversations. However, existing parsing approaches are constrained by predefined relation types, which can impede the adaptability of the parser for downstream tasks. To this end, we propose to introduce a task-aware paradigm to improve the versatility of the parser in this paper. Moreover, to alleviate error propagation and learning bias, we design a graph-based discourse parsing model termed DialogDP. Building upon the symmetrical property of matrix-embedded parsing graphs, we have developed an innovative self-supervised mechanism that leverages both bottom-up and top-down parsing strategies. This approach allows the parsing graphs to mutually regularize and enhance each other. Empirical studies on dialogue discourse parsing datasets and a downstream task demonstrate the effectiveness and flexibility of our framework.

## introduction dialogue discourse parsing (ddp) plays an essential role in the field of natural language processing (nlp), serving as a foundational task and receiving increasing attention from the research community @xcite @xcite . the formulation of the ddp task is rooted in the segmented discourse relation theory @xcite , distinguishing it from the rhetorical structure theory @xcite and the penn discourse relation theory @xcite that primarily underpins text-level discourse parsing @xcite . the primary objective of ddp is to recognize the links and relations between utterances in dialogues. @xcite conversation snippet. here a and b are two speakers with subscripts indicating the turn of the dialogue. "qap", "cq", "exp" and "comment" stand for dependency relations. the parsed dependencies between utterances should form a directed acyclic graph (dag). for example, dependent utterance b 3 in fig. 1 depends on two head utterances b 1 and a 2 in "explanation" and "qap" relations, respectively. the early neural-based paradigm @xcite for the ddp task involved sequentially scanning the utterances in dialogues and subsequently predicting dependency links and corresponding relation types, which is prone to the severe error propagation issue @xcite , as the learned representations relied on historical predictions. this observation inspired us to formalize ddp from a graph perspective, where the prediction of links and relations for each pair of utterances is independent of the others. existing methods @xcite resorted to the bottom-up strategy for ddp, where for each dependent utterance, the parser retrieved only one head utterance. these models @xcite may suffer from measurement bias @xcite as the training labels are distorted with the one-head retrieval for samples with multi-head labels. furthermore, these multi-head labels serve as long-tailed data, and overlooking the long-tailed data may result in a biased distribution estimation during training @xcite . meanwhile, as a foundational nlp task, ddp has proved beneficial for downstream tasks including dialogue summarization @xcite and emotion recognition in conversations (erc) @xcite . nevertheless, existing parsers are constrained by predefined relation types, posing a potential obstacle to the parser's adaptability for downstream tasks. for example, existing relations like narration and background do not/ or rarely exist in the erc datasets. in addition, a well-designed ddp fine-tuned with task-aware dependency labels can capture emotion shifts, which benefits the downstream erc task. overall, the previous methods have three limitations, i.e., error propagation, learning bias of distorted training labels and a single strategy, and incompatibility of predefined relations with downstream tasks. to this end, we propose a task-aware self-supervised framework for ddp task. concretely, a graph-based model dialogdp, utilizing biaffine mechanism @xcite , is designed for ddp, avoiding sequential error propagation. the model consists of symmetric parsers, simultaneously performing bottom-up and top-down parsing. we investigate parsed links and relation graphs of the two strategies and design a bidirectional selfsupervision mechanism encouraging the two strategies to acquire similar graphs through mutual learning. moreover, we propose a soft-window triangular (swt) mask incorporating a soft constraint, as opposed to a hard constraint @xcite , to guide the parsers. swt mask encourages parsers to prioritize candidate links within a flexible window for each utterance. to enhance the adaptability of dialogdp for downstream tasks, we propose a novel paradigm involving fine-tuning with task-specific re-annotated relations. we validate the effectiveness of our taskaware paradigm on downstream erc. the contributions of this paper are: ## 14163 in summary, despite the good performance, the existing methods still have problems in modeling and application. modeling-wise, the ddp is currently limited to either a top-down or bottom-up manner, leading to a gap in achieving bidirectionality. application-wise, the issue arises due to the constraints imposed by predefined relation types, thereby limiting the benefits to only tasks directly associated with these pre-defined relation types. consequently, there is a need to establish connections between the parsed dependency relations and downstream tasks. in this paper, we propose a new framework to address these important issues. ## methodology dialogdp is tailored for integral graph-driven discourse parsing, which is more computationefficient and avoids error propagation in the training process. specifically, the bidirectional selfsupervision allows the parser to parse the dialogue with both top-down and bottom-up strategies. the employed strategies in both directions can mutually reinforce and guide each other. moreover, the swt mask guides the two strategies and imposes soft constraints on the integral learned graph, prompting the parser to prioritize the candidate links of each utterance within a flexible window. ## 14164 context-speaker encoder we calculate the arc scores (s a @xmath0, s a i , ..., s a n ] bu ) and relation label scores (s l @xmath1, s l i , ..., s l n ] bu ) with bottom-up strategy through: here, h @xmath2, while h d′ @xmath3. s a @xmath4. by observing the arc (relation) scores of both the top-down strategy and the bottom-up strategy, we can readily infer their high relevance. in other words, s a bu ∝ s a td ′ and s l bu ∝ s l td ′ . hence, we designed a bidirectional self-supervision mechanism for regularizing the bidirectional strategies by assuming s a @xmath5. ## 14165 this limitation hampers the generalization capability in real-world scenarios.to this end, we devise a soft-window mechanism that can be implemented by a carefully designed mask. each element m ω i,j combining the triangular mask with the softwindow mask, swt masks for bottom-up and topdown strategies, m bu and m td are obtained by the scores of arcs and relations after the softwindow triangular masking are o @xmath6, respectively. we denote o a i,j := [o a ] i,j and o l i,j,k := [o l ] i,j,k . ## 14166 algorithm 1: task-aware annotation data: downstream dialogue u d with labels @xmath7, predicted links y result: task-aware relations t d 1 define state transition mapping g(li, lj), (li, @xmath8; 2 while yi,j in y and yi,j ̸ = 0 do 3 taking the downstream erc task as an example, task-aware ddp leverages the state transition to represent emotion shift which can enhance erc performance @xcite and contribute to the research in the psychology domain @xcite . the state transition mapping g (eq. 13) from head u d i to u d j is obtained based on their target labels l i and l j , where l i , l @xmath9, 1, 2} corresponding to {negative, neutral, positive} sentiment polarities. the numbers from 0 to 5 in g are the relation types for erc task-aware ddp, referring to "same_polarity", "sentiment_to_neutral", "neg-ative_to_positive", "neutral_to_negative", "neu-tral_to_positive", and "positive_to_negative", respectively. in this case, the task-aware dialogdp fine-tuned with re-annotated relations is capable of capturing sentiment shifting between utterances. ## conclusion in this paper, we proposed a bidiretional selfsupervised dialogue discourse parser dialogdp and 'ldorjxh7xuq $ffxudf\ 'hhs6htxhqwldo :dqjhwdo 2xu'ldorj'3 a task-aware paradigm combining ddp and downstream task erc. first, we designed two graphbased parsers leveraging both bottom-up and topdown parsing strategies and eliminating sequential error propagation. then a bidirectional selfsupervision mechanism is designed to reduce the learning bias by exploiting the structural symmetry of two strategies, and thus avoid the reliance on external knowledge. furthermore, a soft-window triangular mask, tailored with statistical information, is utilized to effectively handle long-term dependencies. second, we presented a task-aware paradigm bridging the gap between the foundational ddp with downstream tasks through finetuning dialogdp with task-aware dependency relation annotations. empirical studies on ddp and downstream erc show the superiority and adaptability of our dialogdp and task-aware paradigm.