Soloist: Building Task Bots at Scale with Transfer Learning and Machine Teaching (2021)

Abstract: We present a new method, Soloist,1 that uses transfer learning and machine teaching to build task bots at scale. We parameterize classical modular task-oriented dialog systems using a Transformer-based auto-regressive language model, which subsumes different dialog modules into a single neural model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model, which can generate dialog responses grounded in user goals and real-world knowledge for task completion. The pre-trained model can be efficiently adapted to accomplish new tasks with a handful of task-specific dialogs via machine teaching, where training samples are generated by human teachers interacting with the system. Experiments show that (i)Soloist creates new state-of-the-art on well-studied task-oriented dialog benchmarks, including CamRest676 and MultiWOZ; (ii) in the few-shot fine-tuning settings, Soloist significantly outperforms existing methods; and (iii) the use of machine teaching substantially reduces the labeling cost of fine-tuning. The pre-trained models and codes are available at https://aka.ms/soloist .

## introduction the increasing use of personal assistants and messaging applications has spurred interest in building task-oriented dialog systems (or task bots) that can communicate with users through natural language to accomplish a wide range of tasks, such as restaurant booking, weather query, flight booking, it helpdesk (e.g., @xcite @xcite @xcite . the 1 task-oriented dialog with a single pre-trained model. in this paper, soloist refers to both the proposed bot building method and the dialog model or system developed using the method. a typical task-oriented dialog system uses a modular pipeline, which has four modules and executes sequentially @xcite , as shown in figure 1 @xcite . a natural language understanding (nlu) module identifies user intents and extracts associated information such as slots and their values from users' input. a dialog state tracker (dst) infers the belief state (or user goal) from dialog history. the belief state is often used to query a task-specific database (db) to obtain the db state, such as the number of entities that match the user goal. the dialog state and db state are then passed to a dialog policy (pol) to select the next system action. a natural language generation (nlg) module converts the action to a natural language response. most popular commercial tools for dialog development employ the modular systems, including google's dialog flow, foot_0 microsoft's power virtual agents (pva), 3 facebook's wit.ai, 4 amazon's lex, 5 and ibm's watson assistant. 6 they are designed mainly to help develop systems manually, namely, writing code, crafting rules and templates. unfortunately, even with these tools, building dialog systems remains a label-intensive, time-consuming task, requiring rich domain knowledge, reasonable coding skill, and expert experience. the cost of building dialog systems at scale (i.e., tens of thousands of bots for different tasks) can be prohibitively expensive. with the recent advances in neural approaches to conversational ai @xcite , researchers have been developing data-driven methods and neural models for either individual dialog modules or end-to-end systems. for example, recent attempts such as rasa @xcite , convlab @xcite , and conversation learner @xcite are made to allow the use of data-driven approaches based on machine learning and machine teaching to develop dialog modules. end-to-end trainable dialog systems have also been studied (e.g., @xcite @xcite @xcite @xcite . although these methods have achieved promising results, they require large amounts of task-specific labeled data for training, which are rarely available for new tasks in real-world applications. in this paper, we propose a novel method of building task bots at scale, soloist, which significantly eases the workflow of training and deploying dialog systems for new tasks, compared to existing tools and methods. our approach is inspired by the recent success of applying transfer learning to natural language processing (nlp) tasks: big language models pre-trained on large amounts of raw text (e.g., bert @xcite , roberta @xcite , and unilm @xcite ) can be effectively fine-tuned for a wide range of nlp tasks with few in-domain labels. recently, these pre-trained language models have also been employed to develop dialog modules such as nlu and dst @xcite @xcite . the proposed soloist uses a similar pretraining-and-fine-tuning framework for building end-to-end dialog systems. we parameterize a task bot using a transformer-based auto-regressive language model, which subsumes different dialog modules (i.e., nlu, dst, pol, and nlg) into a single neural model. task bot building proceeds in two stages: (i) in the pre-training stage, initialized using gpt-2 @xcite , we train a transformer-based, task-grounded, response generation model using large heterogeneous dialog corpora. the model learns the primary task completion skills such as dst and pol, and can generate dialog responses grounded in user goals and real-world knowledge for task completion. we show through a comprehensive empirical study that soloist is an effective method of building task bots at scale by successfully transferring two capabilities from the pre-trained model to a new task bot: (i) the capability of nlu and nlg learned on raw text, and (ii) the capability of grounding system responses in user goals and real-world knowledge for task completion, learned on the out-domain dialog corpora. soloist achieves state-of-the-art performance on two well-studied task-oriented dialog benchmarks, lifting the combined score by 10 points in automatic evaluation, and the success rate by 20 points in human evaluation. in the few-shot fine-tuning settings, soloist adapts to the new domain much more effectively than competing methods, achieving a reasonable success rate using less than 50 dialogs. the promising results demonstrate the potential of the new method for developing task bots at scale. instead of collecting, labeling data, and building one bot per task, we can pre-train a task-grounded response generation model, and adapt it to new tasks via transfer learning and machine teaching. ## experiments this section evaluates the proposed soloist to answer three questions: q1: how does soloist perform on standard benchmarks compared to sota methods? q2: does soloist meet the goal of effectively generalizing to new domains in the few-shot fine-tuning setting? q3: how effective machine teaching is for fine-tuning? note that we employ the conventional fine-tuning method without machine teaching for a fair comparison when studying q1 and q2. ## conclusion soloist is a method of building task bots at scale with transfer learning and machine teaching. unlike gpt-2, soloist is pre-trained in a task-grounded manner. so, it can generate responses grounded in user goals and real-world knowledge for task completion. experiments show that soloist creates new sota on two popular task-oriented dialog benchmarks, and that soloist outperforms existing methods by a large margin in the few-shot fine-tuning settings where only a limited number of task labels are available for fine-tuning. we hope that soloist can inspire dialog researchers and developers to comprehensively explore the new paradigm for building task bots based on task-grounded pre-training and finetuning via machine teaching, and improving the recipe we present in this paper, namely, formulating task-oriented dialog as a single auto-regressive language model, pre-training a task-grounded response generation model on heterogeneous dialog corpora, and adapting the pre-trained model to new tasks through fine-tuning using a handful task-specific examples via machine teaching.