PRAL : A Tailored Pre-Training Model for Task-Oriented Dialog Generation (2021)

Abstract: Large pre-trained language generation models such as GPT-2 have demonstrated their effectiveness as language priors by reaching state-of-the-art results in various language generation tasks. However, the performance of pre-trained models on task-oriented dialog tasks is still under-explored. We propose a Pre-trainedRole Alternating Language model (PRAL), explicitly designed for task-oriented conversational systems. We design several techniques: start position randomization, knowledge distillation, and history discount to improve pre-training performance. In addition, we introduce a high-quality large-scale task-oriented dialog pre-training dataset by post-prossessing13 dialog datasets. We effectively adapt PRALon three downstream tasks. The results show that PRAL outperforms or is on par with state-of-the-art models.

## introduction and related work current approaches to building task-oriented dialog systems still require a substantial amount of annotations and therefore are labor-intensive. on the other hand, large-scale pre-trained language models such as bert @xcite and gpt @xcite have achieved great success on various nlp tasks. there have been several attempts to apply these language models to dialog systems directly. for example, transfer-transfo @xcite fine-tuned gpt on the persona-chat dataset @xcite and achieved the state-of-the-art performance on chitchat dialog generation. dialogpt @xcite utilizes a large reddit corpus to further pre-train gpt-2 @xcite . all of these studies pointed to a promising direction towards building dialog systems with large-scale language models and less annotation. ## methods in this section, we will first briefly introduce ardm, our base dialog model, and then describe a set of techniques proposed in pral. figure 1 shows the main structure of pral. ## experiments we pre-train pral on pretraindial. we use gpt-2 large as the teacher model. we use adamw optimizer with warm-up steps as 10 percent of the training step. the learning rate is set to be 1×10 -4 . for the calculation of loss, we set α 0 to be 0.1 and set λ to be 0.9999. the discount factor γ is set to be 0.95. to show the generalizability, we finetune pral on three downstream dialog generation tasks, camrest676, multiwoz and persuasionfor-good, as is shown in table . 2. refer to appendix b for more experiment setting. camrest676 @xcite is a dialog dataset for restaurant recommendation containing 680 dialogs. we use bleu-4 metrics to measure the quality of generated sentences, and success f1 to evaluate the responses on specific slots, such as address, phone, postcode. sequicity is the state-of-the-art method in task-oriented dialog tasks that requires annotations. pral beat all other models, including a concurrent work soloist @xcite on both bleu-4 and success f1. it is worth noting that pral does not need any annotation. soloist and dialogpt have a close performance with our model. however, soloist uses around 1 million dialogues, di-alogpt uses around 147 million dialogues, meanwhile we only use around 142k dialogues, which model bleu-4 success f1 sequicity 21.4 0.852 sequicity (w/o rl) 22.9 0.821 gpt-2-finetune 21.8 0.851 dialogpt 25.2 0.861 soloist 25.5 0.871 ardm 26.2 0.864 pral 27.2 0.874 -w/ teacher gpt(small) 26.9 0.869 -w/o teacher gpt 25.0 0.865 -w/o loss discount 27.0 0.867 -w/o spr 26.6 0.869 (a) results on camrest676 dataset. model supervision bleu-4 inform success dialog state dialog act human ---0.989 0.965 baseline × 18.9 0.825 0.729 hdsa 23.6 0.877 0.734 larl × 12.8 0.828 0.792 soloist × 18.0 0.896 0.793 ardm × × 20.6 0.874 0.728 pral × × 21.2 0.899 0.798 (b) results on multiwoz dataset perplexity ↓ bleu-1 ↑ bleu-2 ↑ fluency ↑ logic ↑ coherence ↑ diversity ↑ overall ↑ avg.donation ↑ ardm 10.1 16.5 6.44 0.39 0.41 0.37 0.27 0.18 0.62 pral 10.3 17.3 10.9 0.61 0.59 0.63 0.73 0.82 0.99 (c) persuasionforgood. automatic evaluation and human evaluation results ablation studies on camrest676 shows that the teacher gpt plays the most important role. the fact that pral with teacher gpt (small) in multiwoz @xcite ) contains around 10k dialogues covering various domains. we evaluate the models with on bleu-4, inform rate, and success rate which measures if the system provides the requested information. pral outperforms the attention seq2seq model which is used as the baseline in multiwoz @xcite in all metrics. without using any annotation, pral also outperforms or achieve comparable results with hdsa @xcite , larl @xcite and soloist. except for hdsa which requires both dialog state and dialog act, pral achieves a better bleu score than all other models. pral outperforms ardm in all metrics, which further validates the effectiveness of the pre-training process. persuasionforgood we also evaluate our method on persuasionforood @xcite , where a persuader tries to persuade users to donate money to children. there are a total of 1,017 dialogues. although not a traditional task-oriented dialog bench-mark, it is a good benchmark for human evaluation. automatic metrics evaluation is efficient but could fail to capture the text quality on a deeper and complicated level. we choose this task also because it benefits children. unlike camrest676 and multiwoz, the language in persuasionforgood dataset is so diverse that bleu-4 scores of all of the models are too low to be scientific metrics. therefore, we use bleu-1 and bleu-2 instead. our model achieves a significantly higher score on blue metrics, especially on bleu-2 (63% up). in human evaluation, we ask evaluators that how much they are willing to donate after the conversation and acquire their ratings in terms of fluency, logic, coherence, and diversity. the result suggests that pral outperforms ardm on all the metrics. for human evaluation details, please refer to appendices c. case studies show some linguistic problems in ardm, such as repetition and unnaturalness. meanwhile, with pre-training, pral is more natural and persuasive. please refer to appendices d for an example of pral and ardm. ## conclusion we propose pral, a large pre-trained dialog system for task-oriented generation. we incorporated methods that are designed for large dialog system into pral with good performances on three downstream tasks. the model generates more fluent, coherent, diverse, and logical dialogs according to human evaluation results. we also release a highquality dialog dataset for the pre-training process.