AuGPT : Auxiliary Tasks and Data Augmentation for End-To-End Dialogue with Pre-Trained Language Models (2021)

Abstract: Attention-based pre-trained language models such as GPT-2 brought considerable progress to end-to-end dialogue modelling. However, they also present considerable risks for task-oriented dialogue, such as lack of knowledge grounding or diversity. To address these issues, we introduce modified training objectives for language model finetuning, and we employ massive data augmentation via back-translation to increase the diversity of the training data. We further examine the possibilities of combining data from multiples sources to improve performance on the target dataset. We carefully evaluate our contributions with both human and automatic methods. Our model substantially outperforms the baseline on the MultiWOZ data and shows competitive performance with state of the art in both automatic and human evaluation.

## introduction unlike traditional task-oriented systems based on modularized pipelines @xcite , end-to-end dialogue systems integrate nearly all functionality required to hold a dialogue into a single neural network @xcite @xcite , reducing error-propagation and data annotation requirements. while these systems are not yet ready for production use, they made considerable progress in recent years, especially with the advent of pretrained neural language models (lms) @xcite @xcite . systems such as gpt-2 finetuned by @xcite show that with an lm pre-trained on a large number of generaldomain dialogues without annotation, only small amounts of data are required to perform well in a given task-oriented domain. on the other hand, the pre-trained lms run enormous risks. first, solely training for response generation may result in a lack of grounding for the responses, where the lm hallucinates words without any relation to the database. this has been addressed by multi-task training and auxiliary training objectives @xcite to an extent. second, finetuning on small datasets may reduce response diversity and fluency due to neural networks' known propensity for catastrophic forgetting @xcite -the model overfits the finetuning dataset too tightly, "forgetting" the pre-trained language modeling capabilities. this paper presents an end-to-end model for multi-domain task-oriented response generation on the multiwoz data @xcite , 1 where we address the above problems with pretrained lms. augpt is based on the gpt-2 lm and @xcite 's basic approach. our contributions can be summarized as follows: ## related work while the first attempts to build generative endto-end task-oriented systems mimicked the traditional dialogue system components @xcite , the task was soon recast as a sequence prediction problem in a two-stage setup. a sequenceto-sequence (seq2seq) model first generates the belief state based on dialogue context, then generates the system response based on the context and the belief state (sequicity; @xcite . recently, large-scale multi-domain task-oriented datasets were proposed @xcite @xcite . to address multiple domains, @xcite introduce the labes-s2s model that -in addition to a two-stage seq2seq approach -models belief states as discrete latent variables. @xcite present damd, a three-stage seq2seq architecture which explicitly decodes the system action. they optimize for multiple good actions given a single belief state. @xcite investigate sharing of domain knowledge and performance on unseen domains. @xcite 's lava model employs reinforcement learning over latent system actions initialized using a variational autoencoder. the line of research closest to our work makes use of large pre-trained lms based on the transformer architecture @xcite such as gpt-2 @xcite or bert @xcite . for example, @xcite propose finetuning bert @xcite for taskoriented dialogue, @xcite extended the gpt-2 lm to model open-domain chit-chat. we follow research initiated by @xcite , who use gpt-2 to model multidomain task-oriented dialogues. recently, three similar modifications to their model were proposed, namely soloist @xcite , simpletod @xcite , and the approach by @xcite . our work extends these models and proposes a novel training approach and data augmentation strategies based on back-translation @xcite . earlier works used a single pivot language @xcite , whereas our work applies 10 languages to increase variability. ## method the task-oriented setting requires the dialogue system to respond adequately to the user's input and fulfill its goal, e.g., booking a train or request-ing restaurant details. the system must process the user's input, keep track of the belief state (user preferences regarding individual slots, i.e., indomain attributes) and generate a relevant response in natural language. it must also interact with a database to incorporate external information into its responses (see figure 1 for an example). following @xcite , we choose the gpt-2 lm as our backbone and use the lm to model both the belief state and the response. ## results we first discuss quantitative results for both corpusbased and crowdsourced human evaluation, then include a qualitative analysis of the model behavior. ## ablation study we tested many variants of our method with different combinations of components to evaluate their contributions. the results are presented in we can see that all proposed contributions which are a part of our final system, except for the unlikelihood training, have a positive effect on the variants are denoted with their respective modifications compared to the default: "w/o. unlikelihood" = unlikelihood loss was not used; "w/o. clean" uses all training samples as opposed to using only the ones consistent with the database; "w/o. pre-training" = taskmaster-1 and schema-guided datasets were not used for training (this variant was selected for human evaluation); "all auxiliary" = using two additional auxiliary tasks (see the method section); "w/o. consistency" = dialogue consistency task is not used; "old consistency" refers to the consistency task by @xcite (see the section 3.2). training on all data as opposed to using only "clean" samples clearly reduces performance. on the other hand, unlikelihood training improves performance only in convlab while causing a performance drop in corpus-based metrics. this can be caused by the fact that the unlikelihood training promotes diversity and reduces repetitions on the token level, and thus does not play well with corpus-based evaluation. we did not notice any increase in performance when the user intent prediction and system action prediction auxiliary tasks were used (cf. section 3.2). the reason for this behavior could be that the model learns to represent the actions well enough implicitly, without the need for these additional objectives. therefore, these tasks are not a part of our final model. ## conclusions & future work we present a dialogue modeling pipeline based on the pre-trained gpt-2 language model. augpt uses modified training objectives and employs data augmentation to increase the diversity of generated utterances. our experiments show that the proposed approach outperforms baselines and is competitive with state of the art on the multiwoz dataset. we also run a series of ablation experiments to assess the individual contributions of the modifications. according to our detailed ablation study, training data augmentation using backtranslation via multiple languages and a modified auxiliary training objective for dialogue consistency detection are the features that contribute most to our system's performance. additionally, we perform a qualitative analysis of the outputs to give a better insight into our model behavior. in the future, we plan to construct a latent representation of the belief state and optimize it jointly with the language model. we will replace the deterministic lexicalization with a trainable alternative, and possibly even integrate the database module into the model. to improve the transfer to new domains, we will learn a domain embedding and optimize it jointly with the model, unifying all datasets.