Continual Prompt Tuning for Dialog State Tracking (2022)

Abstract: A desirable dialog system should be able to continually learn new skills without forgetting old ones, and thereby adapt to new domains or tasks in its life cycle. However, continually training a model often leads to a well-known catastrophic forgetting issue. In this paper, we present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks. To avoid forgetting, we only learn and store a few prompt tokens’ embeddings for each task while freezing the backbone pre-trained model. To achieve bi-directional knowledge transfer among tasks, we propose several techniques (continual prompt initialization, query fusion, and memory replay) to transfer knowledge from preceding tasks and a memory-guided technique to transfer knowledge from subsequent tasks. Extensive experiments demonstrate the effectiveness and efficiency of our proposed method on continual learning for dialog state tracking, compared with state-of-the-art baselines.

## introduction recently, most studies have focused on developing dialog systems for specific domains in an offline manner, assuming the data distribution stays the same. however, this is far from realistic because a deployed dialog system is often required to support new domains and provide more services constantly over time. therefore, it is crucial for a dialog system to continually learn new tasks without forgetting old ones with high efficiency. previous studies on continual learning @xcite ) mainly focused on solving the catastrophic forgetting (cf) problem @xcite : when a neural model is trained on a sequence of tasks, new tasks may interfere catastrophically with old tasks. simply storing a model version for each task to mitigate forgetting is prohibitive as the number of tasks grows, especially when the model size is large. to mitigate catastrophic forgetting with low computation and storage overhead, recent methods freeze the backbone model and propose to train a weight/feature mask @xcite or an adapter @xcite for each task independently. however, the techniques above are still not efficient enough, and they largely ignore knowledge transfer among tasks. in this paper, we develop prompt tuning @xcite for continual learning. we freeze the backbone pre-trained model and train a few prompt tokens' embeddings for each task, which is highly parameter-efficient to avoid forgetting. as illustrated by yellow components in figure 1 , we concatenate the input with a few tunable task-specific prompt tokens before feeding it to a frozen pretrained model. since these prompt tokens have only a small number of parameters (0.1% of the pretrained model's parameters in our experiments), we can efficiently train and store the prompt for each task. during inference, the same pre-trained model can handle different tasks by inputting different prompts, which is friendly for deployment. unlike the vanilla approach of training each task's prompt from scratch and fixing it afterward, we propose continual prompt tuning, a framework that enables knowledge transfer between tasks. we consider transferring knowledge from both preceding tasks (forward) and subsequent tasks (backward). to realize forward transfer, we propose several techniques, including continual prompt initialization, query fusion, and memory replay (green solid arrows in figure 1 ). to achieve positive backward transfer, we propose a memory-guided technique that uses subsequent tasks' data to update the previous tasks' prompts selectively (red dashed arrows in figure 1 ). we conduct experiments on dialog state tracking (dst), a core component of a dialog system, using the schema-guided dialog dataset @xcite . the model continually learns new services that have multiple slots to fill. we concatenate all slots' descriptions with the input and insert a sentinel token after each description, formulating dst as a masked spans recovering task, which is similar to the pre-training objective of t5 @xcite . we empirically show that our proposed framework effectively outperforms stateof-the-art baselines on continual learning for dst, and is extremely efficient in terms of computation and storage. 1 to summarize, our main contributions are: 1. for the first time, we develop prompt tuning for continual learning, which avoids forgetting efficiently and is friendly for deployment. ## experimental setup recently, @xcite proposed a continual learning benchmark for task-oriented dialog systems and compared several classic cl methods. we adapt their data processing steps and baselines in our experiments. ## experiments and analysis the experiments are organized as follows. we compare our method with baselines in sec. 5.1, and present a comprehensive ablation study in sec. 5.2. we investigate the effect of prompt initialization in sec. 5.3, and the effect of model size and prompt length in sec. 5.4. ## conclusion in this paper, we develop prompt tuning for continual learning for the first time. we propose continual prompt tuning, a highly parameter-efficient framework that avoids forgetting and enables forward/backward knowledge transfer among tasks. for forward transfer, we explore continual prompt initialization, query fusion, and memory replay techniques. for backward transfer, we devise a memory-guided technique. extensive experiments on continual learning for dst demonstrate the effectiveness and efficiency of our proposed method compared with state-of-the-art baselines. our method and findings will foster more future studies towards building more scalable, adaptable task-oriented dialog systems.