Curriculum Prompt Learning with Self-Training for Abstractive Dialogue Summarization (2022)

Abstract: Succinctly summarizing dialogue is a task of growing interest, but inherent challenges, such as insufficient training data and low information density impede our ability to train abstractive models. In this work, we propose a novel curriculum-based prompt learning method with self-training to address these problems. Specifically, prompts are learned using a curriculum learning strategy that gradually increases the degree of prompt perturbation, thereby improving the dialogue understanding and modeling capabilities of our model. Unlabeled dialogue is incorporated by means of self-training so as to reduce the dependency on labeled data. We further investigate topic-aware prompts to better plan for the generation of summaries. Experiments confirm that our model substantially outperforms strong baselines and achieves new state-of-the-art results on the AMI and ICSI datasets. Human evaluations also show the superiority of our model with regard to the summary generation quality.

## introduction as billions of people engage in instant messaging and other forms of interaction, there is notable interest in techniques to process and distill recorded dialogues into concise and natural summaries @xcite . the inherent challenges of the dialogue make abstractive dialogue summarization particularly difficult. first, the available labeled data is substantially smaller than news summarization data, e.g., 137 meetings in ami @xcite vs. 312k articles in cnn/daily mail @xcite . second, everyday dialogue involves a dynamic information exchange flow @xcite ) such that salient information is often scattered across multiple utterances by different interlocutors @xcite . ## task formulation given a dialogue x consisting of d utterances from multiple speakers, abstractive dialogue summarization aims to compress the input dialogue x into a concise summary y , typically maximizing the conditional probability p (y |x; θ, ϕ) over n instances, where θ here represents prompt-related parameters and ϕ denotes the remaining parameters of the backbone model, such as bart. ## approach to address the task of abstractive dialogue summarization, we propose an encoder-decoder architecture with heterogeneous prompts, which gradually increases the degree of prompt perturbation via a curriculum learning strategy, and conducts the optimization using a self-training technique. ## quantitative analysis 6.1 ablation study to better quantify the contributions of different components in our model, we conduct ablation studies with three types of simplified architectures as follows. the first type removes the all curriculum prompts, and the second drops topic-aware prompts. the third variant is to use our model without self-training.