SlowGuess/NAACL_2022_OCR huggingface dataset:
Vaibhav Varshney, Mayur Patidar, Rajat Kumar, Lovekesh Vig, Gautam Shroff TCS Research, New Delhi, India
{varshney.v, patidar.mayur, k.rajat2, lovekesh.vig, gautam.shroff}@tcs.com
Identifying all possible user intents for a dialog system at design time is challenging even for skilled domain experts. For practical applications, novel intents may have to be inferred incrementally on the fly. This typically entails repeated retraining of the intent detector on both the existing and novel intents which can be expensive and would require storage of all past data corresponding to prior intents. In this paper, the objective is to continually train an intent detector on new intents while maintaining performance on prior intents without mandating access to prior intent data. Several data replay-based approaches have been introduced to avoid catastrophic forgetting during continual learning, including exemplar and generative replay. Current generative replay approaches struggle to generate representative samples because the generation is conditioned solely on the class/task label. Motivated by the recent work around prompt-based generation via pretrained language models (PLMs), we employ generative replay using PLMs for incremental intent detection. Unlike exemplar replay, we only store the relevant contexts per intent in memory and use these stored contexts (with the class label) as prompts for generating intent-specific utterances. We use a common model for both generation and classification to promote optimal sharing of knowledge across both tasks. To further improve generation, we employ supervised contrastive fine-tuning of the PLM. Our proposed approach achieves state-of-the-art (SOTA) for lifelong intent detection on four public datasets and even outperforms exemplar replay-based approaches. The technique also achieves SOTA on a lifelong relation extraction task, suggesting that the approach is extendable to other continual learning tasks beyond intent detection.
Modern dialog systems are reliant on user intent detection wherein the system is required to classify
a user utterance into one of multiple pre-defined intents. A static intent detection model is often insufficient for real-world applications because (i) User intents evolve over time (ii) Additional functionality may be added to the system and (iii) Some intents may become obsolete.
Consider a conversational Information Technology (IT) helpdesk in an enterprise, where a ticket raised by an employee is classified into one of the multiple pre-defined intents (corresponding to different IT complaints) and accordingly routed by the system to the appropriate IT support staff. Each ticket comprises a textual IT problem description manually entered by an employee in natural language. In a typical IT enterprise, the helpdesk team provides support for every IT application from installation to maintenance. As applications get upgraded, and newer applications are introduced, the set of intents can change and the intent detection model must be updated.
(A) Task-Incremental Lifelong Learning
(B) Class-Incremental Lifelong Learning
Figure 1: Task-Incremental (Task-IL) Vs Class-Incremental Lifelong Learning (Class-IL).
In the above example, the intent detection model is required to classify both old and newly introduced intents after updation. However, privacy concerns may limit access to prior intent data, and even if prior data is available, retraining the intent detection model from scratch becomes computationally expensive due to the ever-increasing data volume. An additional problem is that the data is skewed towards prior intents. Recently, there has
Findings of the Association for Computational Linguistics: NAACL 2022, pages 1113 - 1127
been a lot of emphasis on continual learning to address these issues for computer vision (Parisi et al., 2018) and natural language processing (Biesialska et al., 2020), wherein the objective is to train a model on a sequence of novel tasks while maintaining performance on prior learned tasks, i.e., to prevent catastrophic forgetting (McCloskey and Cohen, 1989) with little or no data from old tasks.
Sun et al. (2020a); Wang et al. (2021); Madotto et al. (2021) formulate multiclass-text classification as a task-incremental learning (Task-IL) problem where the objective is to train an "incremental learner" on a sequence of tasks. As shown in Fig. 1 (A), they treat each classification task from a novel domain as a separate task introduced at each incremental step. They further assume that each task definition is complete, i.e., for each classification task they have access to all possible class labels and the set of class labels for that task remains immutable. However, in real-world applications, a task itself may evolve over time as shown in Fig. 1 (B), where, for an intent detection task in the banking domain a new set of intents are added at each incremental step based on new user needs and new functionality introduced by the service provider. Also in Task-IL, the task identity is provided at inference time, i.e., "Which task has to be performed for a given utterance?". This restricts the job of the "task incremental learner" to choosing one of the task-specific classes as a response. This is in contrast to lifelong intent detection, where the "incremental learner" has to consider the complete set of intents that have been added so far, independent of the number of incremental steps.
To overcome catastrophic forgetting in Task-IL, LAMOL (Sun et al., 2020a) uses a PLM as the task incremental learner and uses the same learner for generative replay, i.e., for generating samples of old tasks in Task-IL. Madotto et al. (2021) propose a residual adapter (Houlsby et al., 2019) based architecture for lifelong intent detection in task-oriented dialogue systems. Unlike existing Task-IL approaches, the authors also predict task-id, i.e., which adapter to use at test time by using an entropy-based classifier. Similar to LAMOL, they also model lifelong intent detection as Task-IL where dialogues from new domains are added to the system over a period of time. However, the task-specific adapters are trained in isolation which often results in erroneous task-id predictions due to an overlap between the intents introduced over dif
ferent incremental steps. Liu et al. (2021b) propose a novel strategy based on exemplar-replay (MSR) which tries to perform knowledge distillation at prediction level and feature level and uses inter-class margin loss to minimize distance between new and old class embeddings.
In the current work, we model "lifelong intent detection" as a class-incremental learning (Class-IL) (Rebuffi et al., 2017) problem where a new set of intents/classes are added over a period of time, i.e., at each incremental step. To address the issue of catastrophic forgetting during "lifelong intent detection", we extend LAMOL to the Class-IL scenario and propose the "Prompt Augmented Generative Replay" (PAGeR) model where, similar to existing approaches (de Masson d'Autume et al., 2019; Wang et al., 2020; Qian et al., 2021; Xia et al., 2021; Wang et al., 2019; Obamuyide and Vlachos, 2019; Han et al., 2020) we do not store real samples in replay memory but instead store concept words for each intent. In successive incremental steps, we generate samples for each prior intent based on the natural language intent definition and the stored intent specific concept words via PAGeR. To further improve the quality of generative replay and overall classification performance, we perform supervised contrastive fine-tuning of  $\text{PAGeR}$  and distill knowledge from the previous version of the "lifelong intent detector" while adapting it to the new set of intents.
(i) We propose a novel incremental learning approach using prompt based classification and generative replay via a common pre-trained language model.
(ii) Our generative replay approach uses prompt based generation guided by intent specific concept words and intent definition, yielding high quality samples.
(iii) To boost the quality of generative replay, we propose an approach for supervised contrastive finetuning of pre-trained Language Models (PLMs).
(iv) Our approach,  $PAGeR$ , outperforms exemplar replay based approaches for lifelong intent detection on three public intent classification datasets, one text classification dataset and two public multidomain dialog intent detection datasets.
(v) We further demonstrate that  $PAGeR$  also outperforms the state-of-the-art on a lifelong relation extraction task by a wide margin suggesting generalizability of the proposed technique across tasks.
2.1 Lifelong Learning
van de Ven and Tolias (2018) proposed three problem scenarios of increasing difficulty for evaluating continual learning algorithms based on whether the task identity is known or not and if not known whether it has to be predicted at inference time or not. The three scenarios correspond to (i) Task Incremental Learning (Task-IL) (ii) Domain-incremental learning (Domain-IL) and (iii) Class-incremental learning (Class-IL) respectively. Among these three scenarios, Class-IL is the most challenging whereas Task-IL is comparatively the easiest. Different approaches have been proposed to alleviate catastrophic forgetting for continual learning including Rehearsal-based methods where a subset of labelled training data from prior tasks is stored in a memory buffer and later replayed while training the model on the current task. Chaudhry et al. (2019b) proposed Experience Replay (ER) where the data from prior tasks is interleaved with data from the current task to retain performance on old tasks. GEM (Lopez-Paz and Ranzato, 2017) ensures that at every training step the loss of each previous task, approximated by the samples in episodic memory, does not increase. whereas A-GEM (Chaudhry et al., 2019a) attempt to ensure that at every training step the average episodic memory loss over previous tasks does not increase. EWC (Kirkpatrick et al., 2017) is an example of a Regularization based approach which remembers old tasks by selectively slowing down learning on the weights important for old tasks. iCaRL (Rebuffi et al., 2017) uses prototype rehearsal along with Knowledge distillation to alleviate catastrophic forgetting in a Class-IL scenario along with a herding (Welling, 2009) based approach for prioritized exemplar selection. DER (Buzzega et al., 2020) combines rehearsal with knowledge distillation and regularization to alleviate catastrophic forgetting in all three scenarios by matching the network's logits sampled throughout the optimization trajectory. In architecture based approaches the model is expanded for each new task with task-specific components (Rusu et al., 2016) or the same model is used for different tasks by identifying appropriate subnetworks (Wortsman et al., 2020). Shin et al. (2017) provide a Generative Replay based approach where pseudo-labelled data corresponding to the prior task is generated and interleaved with data from the current task to
train a solver whereas van de Ven et al. (2020) use the same model as both a solver and generator and instead of replaying actual samples, replay latent representations.
2.2 Reversal-based Methods
Monaikul et al. (2021); Qian et al. (2021); Cao et al. (2021) use different variants of knowledge distillation and representative sample selection methods to alleviate catastrophic forgetting for lifelong learning of NER (Monaikul et al., 2021), NMT (Qian et al., 2021) and hate speech classification on social media (Cao et al., 2021). For lifelong language learning on stream of text examples, de Masson d'Autume et al. (2019) use sparse experience replay, local adaptation during training, and inference respectively. Wang et al. (2020) extend the work of de Masson d'Autume et al. (2019) and propose a sparse experience replay augmented meta-learning based approach to learn better initialization for local adaptation. EA-EMR (Wang et al., 2019), EMAR (Han et al., 2020) are different methods for the lifelong relation extraction task described in section 6. Xia et al. (2021) propose a novel few-shot class incremental text classification task and model text classification as textual entailment.
2.3 Generative Replay based Methods
Sun et al. (2020a) propose an approach for TASK-IL where they generate pseudo-labelled samples corresponding to old tasks for replay and use the same model for generating a label given a sample and task identifier. Sun et al. (2020b); Chuang et al. (2020) use a common classifier-generator model LAMOL with distillation (Sun et al., 2020a) for TASK-IL.
2.4 Prompting PLMs
(Radford et al., 2019) demonstrate the possibility of solving multiple NLP tasks in the zero-shot setting by expressing each of them via a task specific natural language prompt where the corresponding output is generated in an autoregressive fashion. For a given task, there was still a significant performance gap between zero-shot and supervised learning. To reduce this gap, Schick and Schütze (2021) propose an approach where they express text classification and natural language inference in a cloze-style format and fine-tune the PLM in a few-shot setting. The same approach has since been attempted for various NLP tasks (Liu et al., 2021a).
In the context of Task-IL, LAMOL expresses different NLP tasks in the format proposed by (McCann et al., 2018) and fine-tune GPT-2 on the entire task-specific data.
In  $PAGeR$ , we propose three prompts, two for expressing intent detection and one for labelled data generation for Class-IL. Pseudo-labelled utterance generation is guided by intent labels along with concepts words which are stored in memory instead of actual samples. We jointly fine-tune GPT-2 on an intent detection and labelled utterance generation task. To minimize the likelihood of incorrect (Utterance, label) pairs, unlike (Chen et al., 2020; Wu et al., 2020; Khosla et al., 2020; Gunel et al., 2021), we fine-tune GPT-2 on a supervised contrastive learning objective. To further alleviate catastrophic forgetting, we also use Knowledge Distillation (Hinton et al., 2015).
Recent work has demonstrated how downstream tasks can be performed with PLMs using prompts (Liu et al., 2021a) in a zero-shot setting. Typically, a prompting function  $x' = f_{prompt}(x)$  is applied to an input  $x$  to obtain the corresponding prompt  $x'$  containing  $x$ , an (intermediate) answer  $z$  and task specific discrete or continuous tokens as task descriptors. For instance, for the sentiment analysis task,  $\langle x' = [x]$  The movie is  $[z] \rangle$  is an example of a prompt where  $[x] =$  "I like this movie." and "The movie is" are task specific discrete tokens. Given a prompt, an answer  $z$  can be generated using a PLM model  $p(z \mid [x])$  The movie is;  $\varphi$ . Prompts with a random or true value of  $z$  are referred to as Filled Prompts and Answered Prompts respectively. Schick and Schütze (2021) propose prompt-based fine-tuning of PLMs to minimize the performance gap between supervised and zero-shot prompt approaches on downstream tasks.
Our approach is inspired by LAMOL (Sun et al., 2020a) which fine-tunes a GPT-2 PLM over a sequence of tasks. In LAMOL, each task is represented as a prompt generated via a prompt function  $f_{prompt}(Context) = Context, Question, ANS, Answer, EOS)$ , where Context corresponds to the input, Question acts as a task descriptor, Answer refers to the output and ANS, EOS refers to special tokens. LAMOL was initially used for training a single multi-task NLP model without any task-specific parameters (McCann et al., 2018). For the Sentiment Analy
sis task (Context: "I like this restaurant.", Question: "Is this sentence positive or negative?", Answer: "positive"), Context represents the input sentence/paragraph, Question includes all possible class labels and Answer represents the true label. Similar to generative pre-training (Radford and Narasimhan, 2018) of a transformer based decoder only model, LAMOL performs prompt based fine-tuning of GPT-2 by maximizing  $P(Answer \mid Context, Question, ANS)$  for answer generation and  $P(Context, Question, ANS, Answer, EOS \mid Task)$  for generative replay. At inference time, LAMOL generates a class label given a Context and task-identifier, i.e.,  $P(Answer \mid Context, Question, ANS)$ . In LAMOL, generative replay is conditioned solely on task-specific tokens. Similar to LAMOL, we use GPT-2 as PLM for both lifelong intent detection and generative replay.
4 Problem Description
In this paper, we model the "lifelong intent detection" problem as a class-incremental learning problem where a set of new intents are introduced at each incremental step. Consider  $\mathcal{D}^i = \{\mathcal{I}_1^i,\mathcal{I}_2^i,\dots,\mathcal{I}_m^i\}$  and  $\mathcal{I}_j^i = \{\bigcup_{k = 1}^r (x_{jk}^i,y_j^i)\}$  where  $\mathcal{D}^i = \mathcal{D}_{train}^i\cup \mathcal{D}_{val}^i\cup \mathcal{D}_{test}^i$  represents the set of new intents and  $\mathcal{I}_j^i$  represents labelled data corresponding to the  $j^{th}$  intent introduced at the  $i^{th}$  incremental step respectively.  $x_{jk}^{i}$  represents the  $k^{th}$  natural language utterance corresponding to the  $j^{th}$  intent introduced in the  $i^{th}$  incremental step and the corresponding intent label is represented by  $y_j^i$ . Unique intent labels introduced at the  $i^{th}$  incremental step are represented by  $Y^{i} = \{y_{1}^{i},y_{2}^{i},\ldots ,y_{m}^{i}\}$  and there will not be any overlap between the intents introduced at different incremental steps, i.e.,  $Y^{i}\cap Y^{t} = \emptyset$  where  $t\in \{1,2,\dots,i - 1\}$  and  $Y_{all}^{i}$  represent unique intent labels introduced so far where  $Y_{all}^{i} = Y^{i}\cup Y^{i - 1}\cup \dots \cup Y^{1}$ . At the  $i^{th}$  incremental step the whole labelled data corresponding to previous intents, i.e.,  $\mathcal{D}_{old}^i = \mathcal{D}_{train}^{i - 1}\cup \mathcal{D}_{train}^{i - 2}\cup \dots \cup \mathcal{D}_{train}^1$  is not available for training but a fixed size memory  $\mathcal{M}\ll |\mathcal{D}_{old}|$  is accessible by the incremental learner. We represent the incremental learner trained at the  $i^{th}$  incremental step by  $\mathcal{IL}_i$ . And the  $\mathcal{IL}_i$  is evaluated on  $\mathcal{D}_{test-full}^i = \mathcal{D}_{test}^i\cup \mathcal{D}_{test}^{i - 1}\cup \dots ,\mathcal{D}_{test}^1$  which includes utterances corresponding to all the intents seen so far in addition to new intents.
$\mathcal{I}_{dist} = \{\mathcal{N}^1,\mathcal{N}^2,\dots ,\mathcal{N}^\mathcal{T}\}$  represents the num
ber of intents introduced over each incremental step where  $\mathcal{N}^i$  represents number of new intents introduced at  $i^{th}$  incremental step and  $\mathcal{T}$  represents the total number of incremental steps for a dataset  $\mathcal{D} = \mathcal{D}^1\cup \mathcal{D}^2\cup ,\ldots ,\cup \mathcal{D}^\mathcal{T}$ . Thus at the  $i + 1^{th}$  incremental step the objective is to train a learner  $\mathcal{L}_{i + 1}$  given  $\mathcal{D}_{i + 1}^{train}$ ,  $\mathcal{M}$  and evaluate it on  $\mathcal{D}_{test - full}^{i + 1}$ .
In  $PAGeR$ , we model "Lifelong Intent Detection" as a text generation problem, as shown in Fig. 2 (A), where the objective is to generate the correct label  $y_{j}^{i + 1}$  for intents given a user utterance  $x_{jk}^{i + 1}$ , while also utilizing generative replay to maintain performance on old intents. At the  $i + 1^{th}$  incremental step, we jointly fine-tune  $\mathcal{IL}_{i + 1}$  on the "Lifelong Intent Detection"  $LID$  and "Labelled Utterance Generation" ( $LUG$ ) task given  $\mathcal{D}_{train}^{i + 1}$ ,  $\mathcal{S}^i$  which consists of labelled  $(\mathcal{D}_{train}^{i + 1})$  and pseudo-labelled  $(S_i)$  utterances respectively. Pseudo-labelled utterances corresponding to intents present in  $\mathcal{D}_{old}^i$  are generated from  $\mathcal{IL}_i$  based on prompts stored in a fixed size memory,  $\mathcal{M}$  as shown in Fig. 2 (B). To minimize the likelihood of incorrect  $(x_{jk}^{i + 1}, y_{j + 1}^{i + 1})$  pairs, we do supervised contrastive fine-tuning of  $\mathcal{IL}_{i + 1}$ , as shown in Fig. 2 (C). To alleviate catastrophic forgetting, we use knowledge distillation (Hinton et al., 2015) by fine-tuning  $\mathcal{IL}_{i + 1}$  on  $\mathcal{S}^i$  with soft targets on  $LID$  and  $LUG$  whereas we use hard targets while fine-tuning  $\mathcal{IL}_{i + 1}$  on novel intents in  $\mathcal{D}_{train}^{i + 1}$ .
For the remainder of this section, we drop the superscript  $i + 1$  for simplicity which indicates the  $i + 1^{th}$  incremental step. We use  $(x,y)$  for  $(x_{jk}^{i + 1},y_j^{i + 1})$  and refer to the  $i + 1^{th}$  step as the current incremental step.
5.1 Lifelong Intent Detection (LID)
In Task-IL (van de Ven and Tolias, 2018), a task-identifier is provided at inference time, and the model performs the corresponding task. In Class-IL, no class/task information is provided apriori which makes the problem more challenging and not amenable to existing models such as LAMOL. We propose two ways of creating a prompt so that we can extend LAMOL to Class-IL.
(i) Prompt without question (PWQ): In PWQ, we convert each user utterance  $x$  in a prompt by applying a pre-defined template as mentioned in Eq 1. An Answered Prompt corresponding to PWQ is shown in Fig. 3 (A).
(ii) Prompt with incremental question (PWIQ): In PWIQ, we transform each user utterance  $x$  into a prompt by application of a pre-defined template as mentioned in Eq 2. where IncQ at the current incremental step includes all the intent labels  $Y_{all}^{i + 1}$  seen so far. The Answered Prompt corresponding to PWIQ is shown in Fig. 3 (B).
$$
f _ {p r o m p t} ^ {P W Q} (x) = (x, A N S, y, E O S) \tag {1}
$$
$$
f _ {\text {p r o m p t}} ^ {P W I Q} (x) = \left(x, \operatorname {I n c} Q, A N S, y, E O S\right) \tag {2}
$$
ANS,  $EOS$  refers to special tokens used during prompt creation. We perform  $PWQ$  based fin-tuning of  $\mathcal{I}\mathcal{L}_i$  on  $\mathcal{D}_{train}^{i + 1}\cup \mathcal{S}^i$  to arrive at the  $\mathcal{I}\mathcal{L}_{i + 1}$  by maximizing  $\log (p(y,EOS\mid x,ANS;\varphi))$  or by minimizing the loss function as mentioned in Eq. 3.  $(w_{1},\dots,w_{n})$ $(y_{1},\dots,y_{N})$  are the set of tokens present in  $x$  and  $y$  respectively.  $n$  and  $N$  represent the number of tokens in the utterance and intent label respectively.
$$
L _ {I D} = - \log (p (y, E O S \mid x, A N S; \varphi)) \tag {3}
$$
5.2 Labelled Utterance Generation (LUG)
To generate high quality pseudo-labelled utterances for generative replay in subsequent incremental steps, we condition our intent specific generative model on an "intent label" and the corresponding intent Concepts. For each intent  $I_{j}$ , we obtain the top-K tfidf words  $tf -idf_{j}$  from the utterances corresponding to intent  $I_{j}$  to create a prompt. We use a maximum of  $K = 50$  tfidf words per intent. Words which are part of both  $x$  and  $tf -idf_{j}$  are referred to as intent Concepts and  $C_{j} = \{c_{j1}, c_{j2}, \dots, c_{jr}\}$  represents a set of Concept sequences one per labelled utterance present in  $I_{j}$ . We use the template outlined in Eq. 4 to create a prompt for LUG and the corresponding Answered Prompt as shown in Fig. 3 (C). We train  $\mathcal{TL}_{i+1}$  for prompt based generation by minimizing the loss function in Eq. 5.
$$
f _ {\text {p r o m p t}} ^ {L U G} (y, c) = (y, c, S E P, x, A N S, y, E O S) \tag {4}
$$
We use  $\mathcal{L}_{i}$  along with  $\mathcal{M}$  to generate the pseudo-labelled data  $S^i$  corresponding to intents present in  $\mathcal{D}_{old}^{i}$  for training of  $\mathcal{L}_{i + 1}$ . Existing replay based approaches either store a few labelled utterances per intent in  $\mathcal{M}$  or generate an equal number of pseudo-labelled utterances. PAGeR, instead uses the same memory to store tfidf concept