Prompt Conditioned VAE : Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022)

Abstract: Lifelong learning (LL) is vital for advanced task-oriented dialogue (ToD) systems. To address the catastrophic forgetting issue of LL, generative replay methods are widely employed to consolidate past knowledge with generated pseudo samples. However, most existing generative replay methods use only a single task-specific token to control their models. This scheme is usually not strong enough to constrain the generative model due to insufficient information involved. In this paper, we propose a novel method, prompt conditioned VAE for lifelong learning (PCLL), to enhance generative replay by incorporating tasks’ statistics. PCLL captures task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. Moreover, it leverages a distillation process to further consolidate past knowledge by alleviating the noise in pseudo samples. Experiments on natural language understanding tasks of ToD systems demonstrate that PCLL significantly outperforms competitive baselines in building lifelong learning models.

## introduction task-oriented dialogue (tod) systems are of great importance in advanced ai applications @xcite @xcite . however, most existing tod systems are developed under the assumption that the data distribution remains unchanged @xcite . unless the entire system is retrained, this setup may not be realistic when the tod system deployed in practice needs to support new features and provides more services over time based on user demands. without incurring the high cost of retraining, lifelong learning (ll) is able to acquire new knowledge continuously while preserving previously learned knowledge @xcite . hence, it's crucial to equip natural language understanding (nlu) modules, the vital components of tod systems, with the lifelong learning ability. the main issue for lifelong learning is catastrophic forgetting @xcite , which refers to the phenomenon that a model forgets previously learned tasks when learning new tasks. various approaches have been proposed to alleviate this issue @xcite @xcite . the replay-based methods are among the most effective and widely used ones @xcite @xcite . the main idea of replay-based methods is to retrain samples or representations from already seen tasks when learning new tasks @xcite . some methods explicitly store previously seen real samples for replaying (experience replay) @xcite . however, this setting will be infeasible when data from previous tasks is unavailable due to data security concerns. other methods try to generate pseudo samples using a generative model (generative replay). this variant relieves the burden of storing previously seen data and has been widely adopted in previous studies @xcite @xcite . the key to generative replay is to produce pseudo samples to approximate the real data distribution of previous tasks. intuitively, higher quality pseudo samples can better preserve learned tasks and lead to less forgetting in ll. however, the generation of pseudo samples for each seen task in previous studies @xcite is usually controlled by a single task-specific token. it has been observed that this scheme is usually insufficient to constrain the plm @xcite , due to limited information involved. consequently, the generated pseudo samples suffer from problems such as not being fluent or not corresponding well to the designated task. moreover, those special tokens are only introduced in the fine-tuning stage of the plm. this enlarges the gap between pre-training and fine-tuning of the plm @xcite and harms the quality of the generated pseudo samples. in addition, generated noisy pseudo samples may degenerate the ll performance. to address the above issues, we propose a novel method, prompt conditioned vae for lifelong learning (pcll), to enhance generative replay on nlu tasks of tod systems. to impose strong control over the pseudo-sample generation, pcll explicitly models latent task-specific distributions using a conditional variational autoencoder (cvae) @xcite . then it incorporates the corresponding task statistics to guide the generation of pseudo samples. to reduce the gap between pretraining and finetuning, we construct natural language prompts to unify different nlu tasks while being specific to each task. these prompts not only contain meaningful semantics compared to special tokens, but also serve as conditions to assist cvae in capturing task distributions. moreover, pcll employs a knowledge distillation scheme to alleviate the impact of noisy pseudo samples during the replay process. leveraging the above strategies, pcll can generate highquality pseudo samples that better approximate the real distributions of previous tasks while tackling the aforementioned issues. we validate our method on nlu tasks of tod systems including both intent detection and slot filling. the results indicate that our approach generates high-quality pseudo samples and significantly outperforms competitive baselines. our main contributions are as follows, ## conclusion in this paper, we propose pcll to enhance generative replay for addressing catastrophic forgetting of lifelong learning in building nlu modules of tod systems. to construct high-quality pseudo samples, pcll captures task-specific distributions with a prompt conditioned vae to guide the generation of pseudo samples. empirical results on two nlu tasks and extensive analyses demonstrate the superior performance of pcll and the high quality of its generated pseudo samples. currently, we do not consider lifelong learning in the low-resource setting where only limited labeled data are available. in the future, we will extend our framework to lifelong few-shot learning.