Continual Learning in Task-Oriented Dialogue Systems (2021)

Abstract: Continual learning in task-oriented dialogue systems allows the system to add new domains and functionalities overtime after deployment, without incurring the high cost of retraining the whole system each time. In this paper, we propose a first-ever continual learning benchmark for task-oriented dialogue systems with 37 domains to be learned continuously in both modularized and end-to-end learning settings. In addition, we implement and compare multiple existing continual learning baselines, and we propose a simple yet effective architectural method based on residual adapters. We also suggest that the upper bound performance of continual learning should be equivalent to multitask learning when data from all domain is available at once. Our experiments demonstrate that the proposed architectural method and a simple replay-based strategy perform better, by a large margin, compared to other continuous learning techniques, and only slightly worse than the multitask learning upper bound while being 20X faster in learning new domains. We also report several trade-offs in terms of parameter usage, memory size and training time, which are important in the design of a task-oriented dialogue system. The proposed benchmark is released to promote more research in this direction.

## introduction task-oriented dialogue systems (tods) are the core technology of the current state-of-the-art smart assistants (e.g. alexa, siri, portal, etc.). these systems are either modularized as a pipeline of multiple components, namely, natural language understanding (nlu), dialogue state tracking (dst), dialogue policy (dp) and natural language generation (nlg), or end-to-end, where a single model implicitly learns how to issue apis and system responses. these systems are continuously updated with new features based on the user's needs, e.g., adding new slots and intents, or even completely new domains. however, existing dialogue models are trained with the assumption of having a fixed dataset at the beginning of the training, and they are not designed to add new domains and functionalities through time without incurring the high cost of retraining the whole system. the ability to acquire new knowledge continuously, a.k.a. continual learning (cl) @xcite , represents a common challenge to many production and ondevice dialogue systems where there is a continual growth of 1st and 3rd-party-developer domains that are added after deployment. therefore, it is crucial to design dialogue systems with cl ability. figure 1 shows an high-level intuition of cl in tods. in the cl setting the main challenge is catastrophic forgetting @xcite . this phenomena happens because there is a distributional shift between the tasks in the curriculum which leads to catastrophic forgetting of the previously acquired knowledge. to overcome this challenge three kinds of methods are usually deployed: loss regularization, for avoiding interference with the previously learned tasks, rehearsal, which use episodic memory to recall previously learned tasks, and architectural, which add task-specific parameters for each learned task. however, architectural methods are usually not considered as a baseline, especially in sequence-to-sequence (seq2seq) generation tasks @xcite , because they usually require an additional step during testing to select which parameter to use for the given task. to the best our knowledge, continual learning in task-oriented dialogue systems @xcite is mostly unexplored or has been studied only in limited settings (e.g., nlg @xcite ) using only a few tasks learned continuously. given the importance of the task in the dialogue setting, we believe that a more comprehensive investigation is required, especially by comparing multiple settings and baselines. therefore, in this paper, we make the following contributions: ## adaptercl motivated by the lack of architectural baselines for cl in seq2seq modelling, we propose a novel architectural method called adaptercl. our proposed method parameterizes each task using residual adapters @xcite and uses an entropy-based classifier to select which adapter to use at testing time. this method is designed for large pre-trained language models, e.g., gpt-2, since only the task-specific parameters are trained, while the original weights are left frozen. residual adapters are trainable parameters added on top of each transformer layer, which steer the output distribution of a pre-trained model without modifying its original weights. an adapter block consists of layer normalization @xcite , followed by two linear layers @xcite with a residual connection. given the hidden representation at layer l, denoted as @xmath0, of a transformer @xcite , where d is the hidden size and p is the sequence length, the residual adapter computes denotes the layer normalization. the bottleneck dimension b is a tunable hyper-parameter that allows to adjust the capacity of the adapter according to the complexity of the target task. we define the set of as the set of parameters for the adapter i for a model with l layers. to continuously learn new tasks, we first spawn a new adapter, parameterized by µ, and then we train its parameters as in equation 3. for instance, given the dataset d t and the model with its corresponding adapter µ t , the loss is defined as: ## experimental settings in this section we describe 1) the datasets used for creating the learning curriculum, 2) the evaluation metric used to evaluate the different settings, and 3) the experimental setups. ## results & analysis the main results in the e2e setting are summarized in ## related work continual learning methods are usually developed and benchmarked on computer visions tasks. interested readers may refer to @xcite @xcite for an overview of the existing approaches, and to section 2.2 for more details on the three main cl approaches studied in this paper. continual learning has also been studied in the long life learning (lll) scenario, where a learner continuously accumulates knowledge and makes use of it in the future @xcite . in this paper, we study the setting in which a series of tasks is learned continuously. cl in nlp has been explored for both classification (d' @xcite @xcite and generation @xcite a fine-tuning step) during inference. finally, continual learning has been used for sentence encoding @xcite , composition language learning @xcite and relation learning @xcite . however, these methods are specific to particular applications not generalizable to tods. ## conclusion in this paper, we proposed a benchmark for continual learning in task-oriented dialogue systems, with 37 tasks to be learned sequentially on four settings: intent recognition, dialogue state tracking, natural language generation, and end-to-end. then, we implemented three continual learning methodologies, namely regularization, rehearsal and architectural. for the latter, we proposed a simple yet effective method based on residual adapters and a perplexitybased classifier to select which adapter to use at inference time. finally, we analyzed the trade-off between the performance, the number of parameters, training time and the episodic memory sizes of the evaluated baselines.