Learning to Customize Model Structures for Few-shot Dialogue Generation Tasks (2020)

Abstract: Training the generative models with minimal corpus is one of the critical challenges for building open-domain dialogue systems. Existing methods tend to use the meta-learning framework which pre-trains the parameters on all non-target tasks then fine-tunes on the target task. However, fine-tuning distinguishes tasks from the parameter perspective but ignores the model-structure perspective, resulting in similar dialogue models for different tasks. In this paper, we propose an algorithm that can customize a unique dialogue model for each task in the few-shot setting. In our approach, each dialogue model consists of a shared module, a gating module, and a private module. The first two modules are shared among all the tasks, while the third one will differentiate into different network structures to better capture the characteristics of the corresponding task. The extensive experiments on two datasets show that our method outperforms all the baselines in terms of task consistency, response quality, and diversity.

## introduction generative dialogue models often require a large amount of dialogues for training, and it is challenging to build models that can adapt to new domains or tasks with limited data. with recent advances in large-scale pre-training @xcite @xcite , we can first pre-train a generative model on large-scale dialogues from the non-target domains and then fine-tune on the task-specific data corpus @xcite @xcite . while pre-training is beneficial, such models still require sufficient taskspecific data for fine-tuning. they cannot achieve satisfying performance when very few examples are given @xcite . unfortunately, this is often the case in many dialogue generation scenarios. for example, in personalized dialogue generation, we need to quickly adapt to the response style of a user's persona by just a few his or her dialogues @xcite ; in emotional dialogue generation, we need to generate a response catering to a new emoji using very few utterances containing this emoji @xcite . hence, this is the focus of our paper -few-shot dialogue generation, i.e. training a generative model that can be generalized to a new task (domain) within k-shots of its dialogues. a few works have been proposed to consider few-shot dialogue generation as a meta-learning problem @xcite @xcite . they all rely on the popular modelagnostic meta-learning (maml) method @xcite . take building personalized dialogue models as an example, previous work treats learning dialogues with different personas as different tasks @xcite . they employ maml to find an initialization of model parameters by maximizing the sensitivity of the loss function when applied to new tasks. for a target task, its dialogue model is obtained by finetuning the initial parameters from maml with its task-specific training samples. despite the apparent success in few-shot dialogue generation, maml still has limitations @xcite . the goal of generative dialogue models is to build a function mapping a user query to its response, where the function is determined by both the model structure and parameters @xcite . by fine-tuning with a fixed model structure, maml only searches the optimal parameter settings in the parameter optimization perspective but ignores the search of optimal network structures in the structure optimization perspective. more-over, language data are inherently discrete and dialogue models are less vulnerable to input changes than image-related models @xcite , which means gradients calculated from a few sentences may not be enough to change the output word from one to another. thus there is a need to develop an effective way to adjust maml for large model diversity in dialogue generation tasks. in this paper, we propose the customized model agnostic meta-learning algorithm (cmaml) that is able to customize dialogue models in both parameter and model structure perspective under the maml framework. the dialogue model of each task consists of three parts: a shared module to learn the general language generation ability and common characteristics among tasks, a private module to model the unique characteristic of this task, and a gate to absorb information from both shared and private modules then generate the final outputs. the network structure and parameters of the shared and gating modules are shared among all tasks, while the private module starts from the same network but differentiates into different structures to capture the task-specific characteristics. in summary, our contributions are as follows: the algorithm is general and well unified to adapt to various few-shot generation scenarios. an alternative promising solution to building few-shot dialogue models is the meta-learning methods, especially maml @xcite . @xcite propose to regard learning with the dialogue corpus of each user as a task and endow the personalized dialogue models by fine-tuning the initialized parameters on the taskspecific data. qian and yu (2019) and mi et al. (2019) treat the learning from each domain in multidomain task-oriented dialogue generation as a task, and apply maml in a similar way. all these methods do not change the original maml but directly apply it to their scenarios due to the model-agnostic property of maml. thus, task differentiation always counts on fine-tuning, which only searches the best model for each task at the parameter level but not the model structure level. ## dialogue model in this section, we firstly describe the network structure of the proposed dialogue model, and then briefly introduce its pre-training. ## customized model training after obtaining the pre-trained model θ from maml, we employ customized mode training with the following two updating steps: in the following, we will describe these two steps respectively as well as the gradient update of the whole dialogue model. ## conclusion in this paper, we address the problem of the fewshot dialogue generation. we propose cmaml, which is able to customize unique dialogue models for different tasks. cmaml introduces a private network for each task's dialogue model, whose structure will evolve during the training to better fit the characteristics of this task. the private module will only be trained on the corpora of the corresponding task and its similar tasks. the experiment results show that cmaml achieves the best performance in terms of response quality, diversity and task consistency. we also measure the model differences among tasks, and the results prove that cmaml produces diverse dialogue models for different tasks.