Simple Conversational Data Augmentation for Semi-supervised Abstractive Dialogue Summarization (2021)

Abstract: Abstractive conversation summarization has received growing attention while most current state-of-the-art summarization models heavily rely on human-annotated summaries. To reduce the dependence on labeled summaries, in this work, we present a simple yet effective set of Conversational Data Augmentation (CODA) methods for semi-supervised abstractive conversation summarization, such as random swapping/deletion to perturb the discourse relations inside conversations, dialogue-acts-guided insertion to interrupt the development of conversations, and conditional-generation-based substitution to substitute utterances with their paraphrases generated based on the conversation context. To further utilize unlabeled conversations, we combine CODA with two-stage noisy self-training where we first pre-train the summarization model on unlabeled conversations with pseudo summaries and then fine-tune it on labeled conversations. Experiments conducted on the recent conversation summarization datasets demonstrate the effectiveness of our methods over several state-of-the-art data augmentation baselines.

## introduction abstractive conversation summarization, which targets at processing, organizing and distilling human interaction activities into short, concise and natural text @xcite , is one of the most challenging and interesting problems in text summarization. recently, neural abstractive conversation summarization has received growing attention and achieved remarkable performances by adapting document summarization pre-trained models and @xcite and incorporating structural information @xcite @xcite @xcite . however, most of these models usually require abundant human-annotated summaries to yield the state-of-the-art performances @xcite , making them hard to be applied into realworld applications (e.g. summarizing counseling sessions) that lack labeled summaries. data augmentation, which perturbs input data to create additional augmented data, has been utilized to alleviate the need of labeled data in various nlp tasks, and can be categorized into three major classes: (1) manipulating words and phrases at the token-level like designed word replacement @xcite , word deletion/swapping/insertion @xcite , token/span cutoff @xcite ; (2) paraphrasing the entire input text at the sentence-level through round-trip translation @xcite @xcite or syntactic manipulation @xcite ; and (3) adding adversarial perturbations to the original data which dramatically influences the model's predictions @xcite @xcite . despite the huge success, the former two mainly perturbs sentences locally while ignoring the diverse structures and context information in dialogues to create high-quality augmented conversations for summarization. the third one might utilize context through additional backward passes, but often require significant amount of computational and memory overhead @xcite , especially for summarization tasks with long input. to this end, we introduce simple and novel set of conversational data augmentation (coda) techniques for conversation summarization guided by conversation structures and context, including: (1) random swapping/deletion randomly swap or delete utterances in conversations to perturb the discourse relations, (2) dialogue-acts-guided insertion randomly insert utterances based on the dialogue acts like self-talk, repeating utterance and back-channel @xcite to interrupt the conversations, and (3) conditional-generation-based substitution randomly substitute utterances in conversations based on pre-trained utterance generation models conditioned on the conversation context. examples for operations in coda are shown in figure 1 . to further enhance the performance when labeled summaries are limited, we extend coda to semisupervised settings, semi-coda, where we combine coda with two-stage noisy self-training @xcite to utilize conversations without annotated summaries. specifically, we repeat the process where we first generate pseudo summaries for unlabeled conversations with the base summarization model, then we pre-train a new model on pseudo data points and fine-tune the model on labeled conversations to form the updated summarization model. to sum up, our contributions are: ## conclusion in this work, we introduced a simple yet effective set of conversational data augmentation methods coda, for improving conversation summarization in low-resourced settings. to further utilize unlabeled conversations, we proposed semi-coda that utilizes a two-stage noisy self-training framework. experiments on both in-domain and out-ofdomain evaluations demonstrated that our coda augmented conversations better compared to previous state-of-the-art augmentation methods. in the future, we plan to examine diverse conversation structures for conversation augmentation and work on zero-shot conversation summarization tasks.