Paraphrase Augmented Task-Oriented Dialog Generation (2020)

Abstract: Neural generative models have achieved promising performance on dialog generation tasks if given a huge data set. However, the lack of high-quality dialog data and the expensive data annotation process greatly limit their application in real world settings. We propose a paraphrase augmented response generation (PARG) framework that jointly trains a paraphrase model and a response generation model to improve the dialog generation performance. We also design a method to automatically construct paraphrase training data set based on dialog state and dialog act labels. PARG is applicable to various dialog generation models, such as TSCP (Lei et al., 2018) and DAMD (Zhang et al., 2019). Experimental results show that the proposed framework improves these state-of-the-art dialog models further on CamRest676 and MultiWOZ. PARG also outperforms other data augmentation methods significantly in dialog generation tasks, especially under low resource settings.

## introduction task-oriented dialog systems that are applied to restaurant reservation and ticket booking have attracted extensive attention recently @xcite @xcite . specifically, with the progress on sequence-to-sequence (seq2seq) learning @xcite , neural generative models have achieved promising performance on dialog response generation @xcite @xcite . however, training such models requires a large amount of high-quality dialog data. since each dialog is collected through a human-human or humanmachine interaction, it is extremely expensive and time-consuming to create large dialog dataset covering various domains @xcite . after dialogs are collected, we also need to annotate dialog states and dialog acts, which are then used to train language understanding models and learn dialog policy. hiring crowd-sourcing workers to perform these annotations is very costly. therefore, we propose automated data augmentation methods to expand existing well-annotated dialog datasets, and thereby train better dialog systems. we propose to augment existing dialog data sets through paraphrase. paraphrase-based dataaugmentation methods have been proved to be useful in various tasks, such as machine translation @xcite ), text classification @xcite , question answering @xcite and semantic parsing @xcite . all these approaches first find a set of semantically similar sentences. however, finding isolated similar sentences are not enough to construct a dialog utterances' paraphrase. because an utterance's paraphrase must fit the dialog history as well. for example, when the system says "do you prefer a cheap or expensive restaurant?", the user may state his intent of asking for a cheap restaurant by "cheap please." or "could you find me a cheap restaurant?" . however, the latter is obviously an improper response which is not coherent with the system question. in other words, a paraphrased dialog utterance needs to serve the same function as the original utterance under the same dialog context. therefore, we propose to construct dialog paraphrases that consider dialog context in order to improve dialog generation quality. we also propose the paraphrase augmented response generation (parg), an effective learning framework that jointly optimizes dialog paraphrase and dialog response generation. to obtain dialog paraphrases, we first find all the user utterances that serve the same function in different dialogs, such as different ways of asking for italian food. then we select the utterances that have the same semantic content but different surface form, to construct a high-quality dialog paraphrase corpus. the corpus is then used to train a paraphrase generation model to generate additional user utterances. finally, the augmented dialog data is used to train a response generation model. we leverage the multistage seq2seq structure @xcite for both paraphrase and response generation. moreover, these two models are connected through an additional global attention @xcite between their decoders, so they can be optimized jointly during training. in our experiments, we apply our framework on two state-of-the-art models, tscp @xcite and damd @xcite on two datasets camrest676 @xcite and multiwoz @xcite , respectively. after applying our framework, the response generation models can generate more informative responses that significantly improves the task completion rate. in particular, our framework is extremely useful under low-resource settings. our paraphrase augmented model only needs 50% of data to obtain similar performance of a model without paraphrase augmentation. our proposed method also outperforms other data augmentation methods, and its comparative advantage increases in settings where only a small amount of training data is available. however, those approaches are not applicable for natural language processing since language is not spatially invariant like images. the word order in a sentence impacts its semantic meaning @xcite . therefore, human language augmentation methods aim to generate samples with the same semantic meaning but in different surface forms. such an idea led to recent augmentation work on the language understanding task @xcite @xcite . however, there is no data augmentation work on task-oriented dialog generation. paraphrase is the technique that generates alternative expressions. most of the existing work on paraphrase aims to improve the quality of generated sentences. for example, phrase dictionary @xcite and semantic annotations @xcite are used to assist the paraphrase model to improve the language quality. to make a controllable paraphrase model, syntactic information @xcite is also adopted. and, recently, different levels of granularity @xcite are considered to make paraphrase decomposable and interpretable. in this paper, we utilize a language environment to assist paraphrase, and use paraphrase as a tool to augment the training data of dialog systems. ## proposed framework in this section, we first introduce how to construct a paraphrase dataset to train paraphrase generation models. then we describe the work flow of the proposed parg model. ## results and analysis the experimental results on camrest676 and mul-tiwoz are shown in we observe some common conclusions supported by the experimental results on both datasets. first, our proposed data augmentation framework significantly improves the system's task completion ability (emr, succ.f1, info and succ) consistently without harming the language fluency. this indicates that incorporating additional dialog paraphrases is beneficial for learning informative responses, since more user expressions are seen by the model during training. secondly, our framework outperforms other data augmentation methods in terms of dialog task relevant metrics under all circumstances. in particular, paraphrase based methods are more likely to produce more fluent and informative responses than local substitution methods (wordsub and textsub), because neural generative models consider dialog history to generate more coherent utterances. the improvement of parg over uttersub suggests that our paraphrase generation model provides a more robust way of utilizing the additional information contained in paraphrases. our paraphrase generation model outperforms other paraphrase based methods (naepara and srpara) since the decoding process of previous system action and the gradient back-propagation through the belief span decoder provide strong dialog context information for paraphrase generation. thirdly, the less data is available, the more improvement can be achieved through our data augmentation. it is worth noting that after applying parg, the model trained on only 50% data obtain comparable results to the model trained on the full dataset without data augmentation, in terms of task relevant metrics. the similar results are also observed by comparing the models trained on 20% data with augmentation and 50% data without augmentation. this indicates that our method is of great significance under low resource settings. parg sometimes gets a slightly lower bleu score compared to other methods. this is potentially because that although seq2seq models can learn responses which corresponding to a correct action, the surface language can still vary among training and testing utterances due to the natural variety of human languages. therefore, the bleu score, which measures the likeness of surface language, may drop despite the system generate good functional responses. we also observe some diverse results on cam-rest676 and multiwoz. under the full data setting, the improvement gained by our data augmentation method on camrest676 is lower than on multi-woz, since the single domain task in camrest676 is easy and the data is enough for model training without conducting augmentation. while for mul-tiwoz, due to large language variations and the complex ontology, the utterance space is not wellexplored, thus the response generation process can benefit more through incorporating additional dialog data. ## ablation study we observe that removing the utterance filter brings the biggest drop in response quality in terms of combined score (-0.039). this suggests the importance of using only high-quality paraphrases to train the response generation model, because the ill generation utterances will introduce errors to the downstream model. the model also suffers from a performance drop (-0.028) after removing the previous system action decoder, which indicates that the supervision from previous system action labels is beneficial for generating better paraphrases. finally, we train the paraphrase generation model and response generation model separately and observe a slight drop of combined score (-0.010). this is because through the attention connection between the paraphrase decoder and belief span decoder, the loss computed for response generation can also guide the paraphrase generation model to generate paraphrases that directly benefit to the response generation process. although the improvement is relatively marginal, joint training has additional advantages in simplifying the training process. specifically, we only need to conduct a single run of training and optimize a single set of hyperparameters. ## case study and error analysis we conduct several case studies to illustrate the response generation quality, paraphrase generation quality, as well as errors made by our model. when we reduce the training data to half, tscp generates wrong dialog state slots, and therefore recommends an expensive restaurant. but parg does not suffer from this problem and generates a correct response. this example suggests that parg can effectively improve the quality of dialog generation in low resource settings. although our paraphrase augmented data augmentation framework shows a notable superiority on the dialog generation quality, it still has some limitations. in the example, the word "place" in the paraphrase refers to another site irrelevant to the hotel in the previous system response, which might be an attraction or a restaurant. the domain of the previous turn should also be considered in the dialog function to provide more domain information, which is regarded as a potential solution for this issue. we also compare the utterances generated by different data augmentation methods to show the superiority of parg in terms of paraphrase generation quality. we select textsub and srpara for comparison, since they are the best replacement-based and paraphrase-based methods achieving the highest combined scores on multiwoz respectively. ## human evaluation we conduct human evaluation to further illustrate parg's superiority in terms of paraphrase generation. we use one-to-one comparison to evaluate the relative quality of paraphrases generated by parg versus strong baselines (naepara and srpara). in our experiments, we advise the judges to evaluate the quality of a paraphrase according to its similarity of user intent with the original utterance. we sample one hundred dialog turns. and in each turn, the paraphrase generated by parg is given one-to-one comparisons with each baseline's paraphrase by five judges. specifically, we ask the judges to choose whether the paraphrase generated by parg is of better, equal or worse quality than the paraphrase generated by naepara or srpara, given the original utterance. the results are shown in ## conclusion in this paper, we propose to use dialog paraphrase as data augmentation to improve the response generation quality of task-oriented dialog systems. we give out the definition of the paraphrase for a dialog utterance and design an approach to construct paraphrase dataset from a dialog corpus. we propose a paraphrase augmented response generation (parg) framework which consists of a para-phrase generation model, an utterance filter and a response generation model, where the models are trained jointly to take fully advantage of the paraphrase data for better response generation performance. our framework achieves significant improvements when it is applied to state-of-the-art response generation models on two datasets. it also beats other data augmentation methods, especially under the low-resource settings.