Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking (2021)

Abstract: This ability to learn consecutive tasks without forgetting how to perform previously trained problems is essential for developing an online dialogue system. This paper proposes an effective continual learning method for the task-oriented dialogue system with iterative network pruning, expanding, and masking (TPEM), which preserves performance on previously encountered tasks while accelerating learning progress on subsequent tasks. Specifically, TPEM (i) leverages network pruning to keep the knowledge for old tasks, (ii) adopts network expanding to create free weights for new tasks, and (iii) introduces task-specific network masking to alleviate the negative impact of fixed weights of old tasks on new tasks. We conduct extensive experiments on seven different tasks from three benchmark datasets and show empirically that TPEM leads to significantly improved results over the strong competitors.

## introduction building a human-like task-oriented dialogue system is a long-term goal of ai. great endeavors have been made in designing end-to-end taskoriented dialogue systems (tdss) with sequenceto-sequence (seq2seq) models @xcite @xcite @xcite @xcite , which have taken the state-of-the-art of tdss to a new level. generally, seq2seq models leverage an encoder to create a vector representation of dialogue history and kb information, and then pass this representation into a decoder so as to output a response word by word. for example, glmp @xcite is a representative end-to-end tds, which incorporates kb information into seq2seq model by using a global memory pointer to filter irrelevant kb knowledge and a local memory pointer to instantiate entity slots. despite the remarkable progress of previous works, the current dominant paradigm for tds is to learn a seq2seq model on a given dataset specifically for a particular purpose, which is referred to as isolated learning. such learning paradigm is theoretically of limited success in accumulating the knowledge it has learned before. when a stream of domains or functionalities are joined to be trained sequentially, isolated learning faces catastrophic forgetting @xcite @xcite . in contrast, humans retain and accumulate knowledge throughout their lives so that they become more efficient and versatile facing new tasks in future learning @xcite . if one desires to create a human-like dialogue system, imitating such a lifelong learning skill is quite necessary. this paper is motivated by the fact that a cognitive ai has continual learning ability by nature to develop a task-oriented dialogue agent that can accumulate knowledge learned in the past and use it seamlessly in new domains or functionalities. continual learning @xcite @xcite is hardly a new idea for machine learning, but remains as a non-trivial step for building empirically successful ai systems. it is essentially the case for creating a high-quality tds. on the one hand, a dialogue system is expected to reuse previously acquired knowledge, but focusing too much on stability may hinder a tds from quickly adapting to a new task. on the other hand, when a tds pays too much attention to plasticity, it may quickly forget previously-acquired abilities @xcite . in this paper, we propose a continual learning method for task-oriented dialogue system with iterative network pruning, expanding and masking (tpem), which preserves performance on previously encountered tasks while accelerating learning progress on the future tasks. concretely, tpem adopts the global-to-local memory pointer networks (glmp) @xcite as the base model due to its powerful performance in literature and easiness for implementation. we leverage iterative pruning to keep old tasks weights and thereby avoid forgetting. meanwhile, a network expanding strategy is devised to gradually create free weights for new tasks. finally, we introduce a task-specific binary matrix to mask some old task weights that may hinder the learning of new tasks. it is noteworthy that tpem is model-agnostic since the pruning, expanding and binary masking mechanisms merely work on weight parameters (weight matrices) of glmp. we conduct extensive experiments on seven different domains from three benchmark tds datasets. experimental results demonstrate that our tpem method significantly outperforms strong baselines for task-oriented dialogue generation in continual learning scenario. ## experimental setup datasets since there is no authoritative dataset for tds in continual learning scenario, we evaluate tpem on 7 tasks from three benchmark tds datasets: (1) in-car assistant @xcite ) that contains 2425/302/304 dialogues for training/validation/testing, belonging to calendar scheduling, weather query, and poi navigation domains, (2) multi-woz 2.1 @xcite that contains 1,839/117/141 dialogues for training/validation/testing, belonging to restaurant, attraction, and hotel domains, and (3) cam-rest @xcite that contains 406/135/135 dialogues from the restaurant reservation domain for training/validation/testing. implementation details following @xcite , the word embeddings are randomly initialized from normal distribution n (0, 0.1) with size of 128. we set the size of encoder and decoder as 128. we conduct one-shot pruning with ratio @xmath0.5. the hyperparameters α and β are set to 32 and 50, respectively. we use adam optimizer to train the model, with an initial learning rate of 1e -3 . the batch size is set to 32 and the number of memory hop k is set to 3. we set the maximum re-training epochs to 5. that is, we adopt the same re-training epochs for different tasks. we run our model three times and report the average results. baseline methods first, we compare tpem with three widely used tdss: ptr-unk @xcite , mem2seq @xcite , and glmp @xcite . in addition, we also compare tpem with ucl (ahn et al., 2019) which is a popular continual learning method. furthermore, we report results obtained by the base model when its parameters are optionally re-initialized after a task has been visited (denoted as re-init). we also report the results of re-init with network expansion (denoted as re-init-expand). different from glmp that keeps learning a tds by utilizing parameters learned from past tasks as initialization for the new task, both re-init and re-initexpand save a separate model for each task in inference without considering the continual learning scenario. ## conclusion in this paper, we propose a continual learning method for task-oriented dialogue systems with iterative network pruning, expanding and masking. our dialogue system preserves performance on previously encountered tasks while accelerating learning progress on subsequent tasks. extensive experiments on 7 different tasks show that our tpem method performs significantly better than compared methods. in the future, we plan to automatically choose the pruning ratio and the number of re-training epochs in the network pruning process for each task adaptively.