Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022)

Abstract: Identifying all possible user intents for a dialog system at design time is challenging even for skilled domain experts. For practical applications, novel intents may have to be inferred incrementally on the fly. This typically entails repeated retraining of the intent detector on both the existing and novel intents which can be expensive and would require storage of all past data corresponding to prior intents. In this paper, the objective is to continually train an intent detector on new intents while maintaining performance on prior intents without mandating access to prior intent data. Several data replay-based approaches have been introduced to avoid catastrophic forgetting during continual learning, including exemplar and generative replay. Current generative replay approaches struggle to generate representative samples because the generation is conditioned solely on the class/task label. Motivated by the recent work around prompt-based generation via pre-trained language models (PLMs), we employ generative replay using PLMs for incremental intent detection. Unlike exemplar replay, we only store the relevant contexts per intent in memory and use these stored contexts (with the class label) as prompts for generating intent-specific utterances. We use a common model for both generation and classification to promote optimal sharing of knowledge across both tasks. To further improve generation, we employ supervised contrastive fine-tuning of the PLM. Our proposed approach achieves state-of-the-art (SOTA) for lifelong intent detection on four public datasets and even outperforms exemplar replay-based approaches. The technique also achieves SOTA on a lifelong relation extraction task, suggesting that the approach is extendable to other continual learning tasks beyond intent detection.

## introduction modern dialog systems are reliant on user intent detection wherein the system is required to classify a user utterance into one of multiple pre-defined intents. a static intent detection model is often insufficient for real-world applications because (i) user intents evolve over time (ii) additional functionality may be added to the system and (iii) some intents may become obsolete. consider a conversational information technology (it) helpdesk in an enterprise, where a ticket raised by an employee is classified into one of the multiple pre-defined intents (corresponding to different it complaints) and accordingly routed by the system to the appropriate it support staff. each ticket comprises a textual it problem description manually entered by an employee in natural language. in a typical it enterprise, the helpdesk team provides support for every it application from installation to maintenance. as applications get upgraded, and newer applications are introduced, the set of intents can change and the intent detection model must be updated. in the above example, the intent detection model is required to classify both old and newly introduced intents after updation. however, privacy concerns may limit access to prior intent data, and even if prior data is available, retraining the intent detection model from scratch becomes computationally expensive due to the ever-increasing data volume. an additional problem is that the data is skewed towards prior intents. recently, there has been a lot of emphasis on continual learning to address these issues for computer vision @xcite and natural language processing @xcite , wherein the objective is to train a model on a sequence of novel tasks while maintaining performance on prior learned tasks, i.e., to prevent catastrophic forgetting @xcite with little or no data from old tasks. @xcite ; @xcite ; @xcite formulate multiclass-text classification as a task-incremental learning (task-il) problem where the objective is to train an "incremental learner" on a sequence of tasks. as shown in fig. 1 (a), they treat each classification task from a novel domain as a separate task introduced at each incremental step. they further assume that each task definition is complete, i.e., for each classification task they have access to all possible class labels and the set of class labels for that task remains immutable. however, in real-world applications, a task itself may evolve over time as shown in fig. 1 (b), where, for an intent detection task in the banking domain a new set of intents are added at each incremental step based on new user needs and new functionality introduced by the service provider. also in task-il, the task identity is provided at inference time, i.e., "which task has to be performed for a given utterance?". this restricts the job of the "task incremental learner" to choosing one of the task-specific classes as a response. this is in contrast to lifelong intent detection, where the "incremental learner" has to consider the complete set of intents that have been added so far, independent of the number of incremental steps. to overcome catastrophic forgetting in task-il, lamol @xcite uses a plm as the task incremental learner and uses the same learner for generative replay, i.e., for generating samples of old tasks in task-il. @xcite propose a residual adapter @xcite based architecture for lifelong intent detection in taskoriented dialogue systems. unlike existing task-il approaches, the authors also predict task-id, i.e., which adapter to use at test time by using an entropy-based classifier. similar to lamol, they also model lifelong intent detection as task-il where dialogues from new domains are added to the system over a period of time. however, the task-specific adapters are trained in isolation which often results in erroneous task-id predictions due to an overlap between the intents introduced over dif-ferent incremental steps. @xcite propose a novel strategy based on exemplar-replay (msr) which tries to perform knowledge distillation at prediction level and feature level and uses inter-class margin loss to minimize distance between new and old class embeddings. in the current work, we model "lifelong intent detection" as a class-incremental learning (class-il) @xcite problem where a new set of intents/classes are added over a period of time, i.e., at each incremental step. to address the issue of catastrophic forgetting during "lifelong intent detection", we extend lamol to the class-il scenario and propose the "prompt augmented generative replay" (pager) model where, similar to existing approaches (de masson d' @xcite @xcite @xcite @xcite we do not store real samples in replay memory but instead store concept words for each intent. in successive incremental steps, we generate samples for each prior intent based on the natural language intent definition and the stored intent specific concept words via pager. to further improve the quality of generative replay and overall classification performance, we perform supervised contrastive fine-tuning of pager and distill knowledge from the previous version of the "lifelong intent detector" while adapting it to the new set of intents. our key contributions are as follows: (i) we propose a novel incremental learning approach using prompt based classification and generative replay via a common pre-trained language model. (ii) our generative replay approach uses prompt based generation guided by intent specific concept words and intent definition, yielding high quality samples. (iii) to boost the quality of generative replay, we propose an approach for supervised contrastive finetuning of pre-trained language models (plms). (iv) our approach, pager, outperforms exemplar replay based approaches for lifelong intent detection on three public intent classification datasets, one text classification dataset and two public multidomain dialog intent detection datasets. (v) we further demonstrate that pager also outperforms the state-of-the-art on a lifelong relation extraction task by a wide margin suggesting generalizability of the proposed technique across tasks. ## preliminaries recent work has demonstrated how downstream tasks can be performed with plms using prompts @xcite in a zero shot setting. typically, a prompting function x ′ = f prompt (x) is applied to an input x to obtain the corresponding prompt x ′ containing x, an (intermediate) answer z and task specific discrete or continuous tokens as task descriptors. for instance, for the sentiment analysis task, ⟨x ′ = [x] the movie is [z]⟩ is an example of a prompt where [x]="i like this movie." and "the movie is" are task specific discrete tokens. given a prompt, an answer z can be generated using a plm model p(z | [x] the movie is; φ). prompts with a random or true value of z are referred to as filled prompts and answered prompts respectively. @xcite propose prompt-based fine-tuning of plms to minimize the performance gap between supervised and zero-shot prompt approaches on downstream tasks. our approach is inspired by lamol @xcite which fine-tunes a gpt-2 plm over a sequence of tasks. in lamol, each task is represented as a prompt generated via a prompt function f prompt (context) = context, question, an s, answer, eos)), where context corresponds to the input, question acts as a task descriptor, answer refers to the output and an s, eos refers to special tokens. lamol was initially used for training a single multi-task nlp model without any task-specific parameters @xcite . for the sentiment analy-sis task (context: "i like this restaurant.", question: "is this sentence positive or negative?", answer: "positive"), context represents the input sentence/paragraph, question includes all possible class labels and answer represents the true label. similar to generative pre-training @xcite at inference time, lamol generates a class label given a context and task-identifier, i.e., p (answer | context, question, an s). in lamol, generative replay is conditioned solely on task-specific tokens. similar to lamol, we use gpt-2 as plm for both lifelong intent detection and generative replay. ## problem description in this paper, we model the "lifelong intent detection" problem as a class-incremental learning problem where a set of new intents are introduced at each incremental step. at the i th incremental step the whole labelled data corresponding to previous intents, i.e., d i @xmath0... ∪ d 1 train is not available for training but a fixed size memory m ≪ |d old | is accessible by the incremental learner. we represent the incremental learner trained at the i th incremental step by il i . and the il i is evaluated on d i test-f @xmath1..., d 1 test which includes utterances corresponding to all the intents seen so far in addition to new intents. i @xmath2, n 2 , ..., n t } represents the num-ber of intents introduced over each incremental step where n i represents number of new intents introduced at i th incremental step and t represents the total number of incremental steps for a dataset @xmath3, ..., ∪d t . thus at the i + 1 th incremental step the objective is to train a learner il i+1 given d train i+1 , m and evaluate it on d i+1 test-f ull . ## proposed approach in pager, we model "lifelong intent detection" as a text generation problem, as shown in fig. 2 (a ), where the objective is to generate the correct label y i+1 j for intents given a user utterance x i+1 jk , while also utilizing generative replay to maintain performance on old intents. at the i + 1 th incremental step, we jointly fine-tune il i+1 on the "lifelong intent detection" lid and "labelled utterance generation" (lug) task given d i+1 train , s i which consists of labelled (d i+1 train ) and pseudo-labelled (s i ) utterances respectively. pseudo-labelled utterances corresponding to intents present in d i old are generated from il i based on prompts stored in a fixed size memory, m as shown in fig. 2 (b ). to minimize the likelihood of incorrect (x i+1 jk ,y i+1 j+1 ) pairs, we do supervised contrastive fine-tuning of il i+1 , as shown in fig. 2 (c ). to alleviate catastrophic forgetting, we use knowledge distillation @xcite by fine-tuning il i+1 on s i with soft targets on lid and lug whereas we use hard targets while fine-tuning il i+1 on novel intents in d i+1 train . for the remainder of this section, we drop the superscript i + 1 for simplicity which indicates the i + 1 th incremental step. we use (x,y) for (x i+1 jk ,y i+1 j ) and refer to the i + 1 th step as the current incremental step. ## experimental setup in this section, we describe the various datasets, baseline approaches, evaluation metrics and training details. we subsequently present the results and discuss our key observations. for further training details, please refer to a.2 in appendix. ## results and discussion for lifelong intent detection task, we compare pager with sota approaches for class-il from the nlp and vision community on three imbalanced and one balanced intent detection dataset. the comparison was across three different sizes for m. for a given m, we run all baselines, including pager-base and pager over four different intent permutations and report acc t avg and f t avg . as shown in ## ablation study we conduct ablation experiments on the b77 dataset across all memory sizes to demonstrate the efficacy of pager's sub-components as shown in ## conclusion the current paper addresses the problem of lifelong intent detection and proposes prompt augmented generative replay (pager) to alleviate catastrophic forgetting. we model lifelong intent detection as a class-il problem and use a common plm for intent detection and pseudo-labelled data generation which is jointly trained for both tasks yielding sota performance. future work would involve demonstrating the versatility of pager on a variety of different nlp tasks, and to apply it to real world class-il scenarios wherein the initial number of classes is large and relatively few intents are added or removed in each iteration with significant class imbalance.