Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data (2020)

Abstract: Recent advances in open-domain dialogue systems rely on the success of neural models that are trained on large-scale data. However, collecting large-scale dialogue data is usually time-consuming and labor-intensive. To address this data dilemma, we propose a novel data augmentation method for training open-domain dialogue models by utilizing unpaired data. Specifically, a data-level distillation process is first proposed to construct augmented dialogues where both post and response are retrieved from the unpaired data. A ranking module is employed to filter out low-quality dialogues. Further, a model-level distillation process is employed to distill a teacher model trained on high-quality paired data to augmented dialogue pairs, thereby preventing dialogue models from being affected by the noise in the augmented data. Automatic and manual evaluation indicates that our method can produce high-quality dialogue pairs with diverse contents, and the proposed data-level and model-level dialogue distillation can improve the performance of competitive baselines.

## introduction open-domain dialogue systems have attracted much research attention @xcite , thanks to the success of neural generation models trained with large-scale data. existing research has been endeavored to address various aspects in dialogue systems, such as modeling persona @xcite @xcite , expressing emotion @xcite , or generating knowledge-grounded dialogues @xcite @xcite . in general, training neural open-domain dialogue models requires a large amount of high-quality paired data, e.g., post-response pairs, which are usually labor-intensive and time consuming to collect. a feasible solution to this data dilemma is to use data augmentation techniques, which are popular in various research areas such as computer vision @xcite or machine translation @xcite . nevertheless, this technique is rarely investigated in the study of opendomain dialogues, and few existing approaches are specifically designed for either the generationbased dialogue models @xcite or the retrieval-based dialogue models @xcite . moreover, existing data augmentation approaches only take a set of paired data as input without considering to utilize unpaired data. as a matter of fact, high-quality unpaired data, i.e., non-conversational texts, are generally easier to collect compared to high-quality dialogue pairs. specifically, these unpaired data provide us a rich bank of alternative expressions for different contents. it is thus feasible to augment the training dialogue pairs utilizing sentences extracted from the unpaired data. as shown in figure 1 , we can extract various sentences from the unpaired data that are similar to a given post-response pair (i.e., anchor pair). augmented pairs that carry richer expressions can be then constructed by combining these extracted sentences. to the best of our knowledge, there are no previous studies for opendomain dialogues that try to construct augmented dialogue pairs by utilizing retrieved unpaired data. in this paper, we propose a novel data augmentation method "dialogue distillation" to improve the performance of open-domain dialogue models by utilizing unpaired data. our method involves two phases of distillation. the first phase is at the data level as it constructs (i.e., distills) postresponse pairs by matching sentences retrieved from a set of unpaired data. specifically, given a set of training pairs { x i , y i }, a randomly selected sentence s is firstly used as a query to retrieve the most similar x i , and then the corresponding y i are used as queries to retrieve similar s i from the unpaired data. augmented pairs s, s i are then constructed and filtered using a ranking module. note that different from previous approaches, the post and response sentences that constitute an augmented pair are both from the unpaired data, which are human written and thereby fluent and contentrich. the second phase is at the model-level as it distills a teacher model using the augmented data. specifically, we borrow the idea of knowledge distillation @xcite to first train a teacher model on a set of high-quality dialogue pairs, and then distill the dialogue model by mimicking the distribution produced by the teacher model on the augmented data to prevent the final dialogue models from being affected by the noise in the augmented data. automatic and manual evaluation results indicate that our data-level distillation process can produce high-quality post-response pairs that are content-rich, and our model-level distillation process can better utilize these augmented data to improve the performance of both retrieval-based and generation-based open-domain dialogue models. our contributions are summarized as follows: ## related work there are two major categories of open-domain dialogue models: 1) retrieval-based models, which retrieve the best matching response from the precollected dialogues @xcite ; and 2) generation-based models, which decode responses from a learned distribution @xcite . recent advances in these two categories all focus on dnn-based data-driven methods @xcite . data augmentation is an effective approach to boost the performance of neural models. it has been explored in various nlp tasks, such as text classification @xcite , machine reading comprehension @xcite and machine translation @xcite . although proved to be effective, this technique is rarely investigated in open-domain dialogue models. few existing approaches are restricted to only take the dialogue pairs as their inputs @xcite @xcite , whereas unpaired texts, i.e., sentences without replies, are not utilized. note that the pre-training based methods @xcite @xcite share a similar motivation with our study, i.e., to boost the performance of neural nlp models utilizing unlabeled (i.e., unpaired) texts. nevertheless, the data augmentation method proposed in our study can be regarded as a supplement to these pre-training approaches. experiments demonstrate that our method can be used to improve the performance of dialogue models even if these models are initialized with strong pretrained models. our study is also related to the knowledge distillation method @xcite , which also employs a teacher model and tries to minimize unpaired data 𝒟 paired data 𝒟 ranking module 𝑆 , 𝑆 , … , 𝑆 𝑆 , 𝑆 , … , 𝑆 𝑆 , 𝑆 , … , 𝑆 … 𝑆 retrieve in 𝒟 retrieve in 𝒟 ## data-level distillation the data-level distillation in our method aims at constructing a set of new post-response pairs d a by matching non-parallel sentences retrieved from unpaired data d u . specifically, d p consists of n post-response pairs: note that m is usually much larger than n because non-parallel sentences are generally easier to collect. further, the output of our data-level distillation process is a set of augmented post-response pairs: the data-level distillation involves two major processes: 1) constructing candidate pairs and 2) filtering low-quality candidates. the whole framework is shown in figure 2 and detailed below. ## model-level distillation a straightforward way to improve a dialogue model with the augmented dialogue data is to directly merge the original paired data d p with d a . however, this naive approach may lead to sub-optimal performance since the augmented pairs in d a might algorithm 1 data-level distillation process sample a sentence s ∼ du. 5: retrieve n posts {xi} n @xmath0. 6: get the responses {yi} n @xmath1. 7: retrieve m sentences {sij} m @xmath2. ## conclusion this paper presents a novel dialogue distillation method that consists of two processes, i.e., 1) a data augmentation process to construct new postresponse pairs from unpaired data and 2) a model distillation process that distills a teacher model trained on the original data to the augmented data. automatic and manual evaluation shows that our method can produce high-quality post-response pairs that are both coherent and content-rich, which can be further used to improve the performance of competitive baselines. our method may inspire other research in low-resource nlp tasks.