Hierarchical Inductive Transfer for Continual Dialogue Learning (2022)

Abstract: Pre-trained models have achieved excellent performance on the dialogue task. However, for the continual increase of online chit-chat scenarios, directly fine-tuning these models for each of the new tasks not only explodes the capacity of the dialogue system on the embedded devices but also causes knowledge forgetting on pre-trained models and knowledge interference among diverse dialogue tasks. In this work, we propose a hierarchical inductive transfer framework to learn and deploy the dialogue skills continually and efficiently. First, we introduce the adapter module into pre-trained models for learning new dialogue tasks. As the only trainable module, it is beneficial for the dialogue system on the embedded devices to acquire new dialogue skills with negligible additional parameters. Then, for alleviating knowledge interference between tasks yet benefiting the regularization between them, we further design hierarchical inductive transfer that enables new tasks to use general knowledge in the base adapter without being misled by diverse knowledge in task-specific adapters. Empirical evaluation and analysis indicate that our framework obtains comparable performance under deployment-friendly model capacity.

## introduction neural dialogue models @xcite @xcite have drawn increasing attention due to their high commercial value. previous work usually makes efforts to improve the diversity and coherence of the responses @xcite @xcite . however, the application of neural dialogue models also requires advanced conversation skills, and recently, a lot of work tries to enable models to express empathy @xcite , be knowledgeable @xcite , and demonstrate consistent personalities @xcite @xcite . specifically, the dialogue model is trained on a taskspecific dataset to learn the corresponding conversation skill. however, with the increasing number of online chit-chat scenarios, the dialogue system is further expected to continually specialize in new tasks without sacrificing the performance on old tasks. meanwhile, the dialogue system must keep its capacity as small as possible for the deployment on the computation resource-limited embedded devices. pre-trained models @xcite have successfully facilitated the learning of the downstream tasks in various fields. to address the challenge of continual dialogue learning, directly fine-tuning pre-trained models on each of the new dialogue tasks is a straightforward way to equip the dialogue system with new conversation skills continually. however, it explodes the capacity of the dialogue system because knowledge of new tasks need to be stored in new pre-trained models to avoid erasing knowledge of old tasks. a more advanced approach is to multi-task one pretrained model on all old tasks and then fine-tune it on new tasks, which can alleviate the capacity problem and use general knowledge between old tasks to improve the model performance on new tasks @xcite . nonetheless, these advantages come at the cost of performance decline on some old tasks due to knowledge interference between diverse tasks. to tackle these problems, we propose a hierarchical inductive transfer framework to construct and deploy the dialogue system with fewer computational resources. the framework is inspired by the fact that the conversational skills are multilayered, and while general skills, e.g., uttering fluent sentences, are necessary for all scenarios and the requisite for sophisticated skills, specialized skills, such as negotiating and debating, work for fewer occasions. in the hierarchy of conversational skills, the latter skills could be efficiently built upon the former skills if they are well-learned. however, considering it is difficult to determine the proper order of the skills and the skills needed for a dataset, we take the following practical approach. we first introduce the adapter module, consisting of a small sub-net, into the pre-trained model. each block of the pre-trained model is assigned two adapters inserted after the self-attention layer and the feed-forward layer. during training, adapters, as the only trainable parameters, learn knowledge of dialogue tasks, which avoids knowledge forgetting on pre-trained models and therefore keeps the capacity of the dialogue system almost constant as the number of dialogue tasks increases. then, we separate the adapter into the base adapter and the task-specific adapter to avoid the performance decline of models on old tasks caused by knowledge interference between diverse tasks. the former is multi-tasked with old tasks to obtain general knowledge by regularization between diverse tasks, which facilitates the learning of new tasks. the latter is further fine-tuned on any dialogue task to learn the corresponding task-specific knowledge, which maintains the model performance on old tasks. finally, the proposed framework significantly enhances the training efficiency due to the learning of dialogue tasks only being conducted via adapters. ## method in this section, we first describe the vanilla adapter and how to apply it to the dialogue tasks and then present the hierarchical inductive transfer to learn general knowledge and task-specific knowledge. ## conclusion in this work, we propose a hierarchical inductive transfer framework to efficiently train and deploy the pre-trained models for growing numbers of new dialogue tasks requiring diverse skills. considering the computation resource-limited embedded devices, we first adopt the adapter module, a small plug-in sub-net, as the only incremental and trainable parameters for learning each of the new dialogue tasks. to take advantage of knowledge in old tasks to facilitate the learning of new tasks, we fur-ther propose the hierarchical inductive transfer to alleviate knowledge interference between tasks and provide general knowledge for new tasks. extensive experiments and analysis demonstrate that the proposed framework achieves high computational efficiency with competitive performance. tiancheng zhao, ran zhao, and maxine eskénazi. 2017. learning discourse-level diversity for neural dialog models using conditional variational autoencoders. in proceedings of the 55th annual meeting of the association for computational linguistics, acl 2017, vancouver, canada, july 30 -august 4, volume 1: long papers, pages 654-664. association for computational linguistics. hao zhou, minlie huang, tianyang zhang, xiaoyan zhu, and bing liu. 2018. emotional chatting machine: emotional conversation generation with internal and external memory. in proceedings of the thirty-second aaai conference on artificial intelligence, (aaai-18), the 30th innovative applications of artificial intelligence (iaai-18), and the 8th aaai symposium on educational advances in artificial intelligence (eaai-18), new orleans, louisiana, usa, february 2-7, 2018, pages 730-739. aaai press.