M in TL : Minimalist Transfer Learning for Task-Oriented Dialogue Systems (2020)

Abstract: In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to “carryover” the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20% training data, and 3) Lev greatly improves the inference efficiency.

## introduction building robust task-oriented dialogue systems is challenging due to complex system design and limited availability of human-annotated data @xcite . a dialogue agent is expected to learn dialogue reasoning, decision making, and language generation, which require a large amount of training data. however, collecting and annotating data for training a dialogue system is time-intensive and not transferable among domains @xcite . one possible workaround is to leverage the pre-trained language model to reduce human supervision @xcite . recent progress in pre-training language models has been shown to be promising in alleviating the data scarcity problem @xcite . such models are typically pre-trained on large-scale plain text with self-supervised objectives, e.g., language modeling @xcite and language denoising @xcite . fine tuning pre-trained language models improves a wide range of natural language processing applications @xcite , notably machine translation @xcite , and personalized dialogue response generation @xcite . however, adapting pre-trained language models to task-oriented dialogue systems is not trivial. current state-of-the-art (sota) approaches in task-oriented dialogue rely on several tasks-specific modules, such as state operation predictor @xcite for dialogue state tracking, and copy-net @xcite for end-to-end dialogue task completion @xcite . such modules are usually absent in the pre-training stage. therefore, tasks-specific architecture modifications are required in order to adapt pre-trained language models to different dialogue tasks. in this work, we aim to simplify the process of transferring the prior knowledge of pre-trained language models for improving task-oriented dialogue systems. we propose minimalist transfer learning (mintl), a simple yet effective transfer learning framework that allows to plug-and-play pre-trained sequence-to-sequence (seq2seq) models and jointly learn dialogue state tracking (dst) and dialogue response generation. unlike previous approaches @xcite , which use a copy mechanism to "carryover" the previous dialogue states and generate new dialogue states, we introduce levenshtein belief spans @xcite which models the difference between old states and new states. in practice, mintl first decodes the lev for updating the previous dialogue state; then, the updated state is used to search the external knowledge base; and finally, a response decoder decodes response by conditioning on the dialogue context and knowledge base match result. mintl is easy to set up by using different pretrained seq2seq backbones. we conduct extensive experiments on both dst and end-to-end dialogue response generation tasks with two pre-trained seq2seq models, such as t5 @xcite and bart @xcite . the experimental result on a large-scale task-oriented dialogue benchmark multiwoz @xcite suggests that our proposed method significantly improves sota performance in both the full data and simulated low resource setting. our contributions are summarized as follows: ## related work pre-trained language models. language model (lm) pre-training @xcite @xcite , has been shown to be beneficial in nlp downstream tasks. generative pre-trained unidirectional lms (e.g., gpt2) are effective in language generation tasks @xcite @xcite . several works have applied a generative pre-training approach in open domain chitchat tasks @xcite , and achieved promising results. on the other hand, bidirectional pre-trained lms @xcite significantly improve the performance of natural language understanding tasks. these models are usually evaluated on classification tasks such as the glue benchmark @xcite , extractive question answering tasks @xcite , and dialogue context understanding @xcite . however, their bidirectionality nature makes them difficult to be applied to natural language generation tasks @xcite . recent works @xcite @xcite unified unidirectional lm and bidirectional lm pre-training approaches, and proposed a seq2seq lm, which are pre-trained with language denoising objectives. a systematic study conducted by @xcite suggests that the combination of an encoder-decoder architecture and language denoising pre-training objectives yields the best result in both language understanding and generation tasks. notably, the two latest pre-trained chatbots, meena @xcite and bst @xcite , are also built on an encoder-decoder architecture. in this work, we transfer the prior knowledge of seq2seq lms to task-oriented dialogues, and successfully improve the sota @xcite result with less human annotation. task-oriented dialogue. task-oriented dialogue systems are designed to accomplish a goal described by a user in natural language. such systems are usually built with a pipeline approach. the pipeline often requires natural language understanding (nlu) for belief state tracking, dialogue management (dm) for deciding which actions to take, and natural language generation (nlg) for generating responses @xcite . to simplify the system design and reduce human supervision, several end-to-end trainable systems have been proposed @xcite @xcite @xcite @xcite . these methods have been shown to achieve promising results in single-domain tasks. however, the recently proposed multi-domain taskoriented dialogue datasets @xcite bring new challenges for multi-domain dialogue state tracking and response generation. several follow up works @xcite ; budzianowski and vulić, 2019; mehri et al., 2019; madotto et al., 2020b) improved on the initial baselines with various methodologies. zhang et al. (2019b) proposed the domain aware multi-decoder network and augmented the system act labels by leveraging the user act annotation, achieving the sota results in multiwoz. however, the aforementioned works rely on taskspecific design and extensive human annotations. to reduce the human effort and simplify the system design, we propose a simple transfer learning framework that can be easily set up with pre-trained seq2seq models and obtain decent performance with a small fraction of the training data. ## methodology in this section, we first provide the notations that are used throughout the paper, then we introduce the lev for efficient dst, and finally, describe the mintl framework and two backbone models. notations. let us define a dialogue @xmath0, r 1 , . . . , u t , r t } as an alternating set of utterances from two speakers, where u and r represent the user utterance and the system response, respectively. at turn t, we denote a dialogue context as c @xmath1, r t-w , . . . , r t-1 , u t } and system response as r t , where w is the context window size. @xmath2, . . . , b t } is the dialogue states for each turn. we define b t , the dialogue state at turn t, as a dictionary that maps (domain: d i , slot: s j ) a pair into values v, where @xmath3, . . . , d n } are the domains, and @xmath4, . . . , s m } are slots to track. thoughtout the paper, we denote the value of a pair ## conclusion in this paper, we proposed mintl, a simple and general transfer learning framework that effectively leverages pre-trained language models to jointly learn dst and dialogue response generation. the lev is proposed for reducing the dst complex-ity and improving inference efficiency. in addition, two pre-trained seq2seq language models: t5 @xcite and bart @xcite are incorporated in our framework. experimental results on multiwoz shows that, by using mintl, our systems not only achieve new sota result on both dialogue state tracking and end-to-end response generation but also improves the inference efficiency. in future work, we plan to explore taskoriented dialogues domain-adaptive pre-training methods @xcite to enhance our language model backbones, and extend the framework for mixed chit-chat and taskoriented dialogue agents @xcite .