SlowGuess/NAACL_2022_OCR huggingface dataset:
Learning for NLP (RepLANLP-2019), pages 224-229, Florence, Italy. Association for Computational Linguistics.
German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71.
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2018. Continual lifelong learning with neural networks: A review. CoRR, abs/1802.07569.
Jing Qian, Hong Wang, Mai ElSherief, and Xifeng Yan. 2021. Lifelong learning of hate speech classification on social media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2304-2314, Online. Association for Computational Linguistics.
Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pretraining.
Alec Radford, Jeff Wu, R. Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. 34:8689-8696.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, J. Kirkpatrick, K. Kavukcuoglu, Razvan Pascanu, and R. Hadsell. 2016. Progressive neural networks. *ArXiv*, abs/1606.04671.
Timo Schick and Hinrich Schütze. 2021. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Computational Linguistics.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2020a. {LAMAL}: {LA}nguage modeling is all you need for lifelong language learning. In International Conference on Learning Representations.
Jingyuan Sun, Shaonan Wang, Jiajun Zhang, and Chengqing Zong. 2020b. Distill and replay for continual language learning. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3569-3579, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Gido M van de Ven, Hava T Siegelmann, and Andreas S Tolias. 2020. Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11:4069.
Gido M van de Ven and Andreas S Tolias. 2018. Generative replay with feedback connections as a general strategy for continual learning. arXiv preprint arXiv:1809.10635.
Chengyu Wang, Haojie Pan, Yuan Liu, Kehan Chen, Minghui Qiu, Wei Zhou, Jun Huang, Haiqing Chen, Wei Lin, and Deng Cai. 2021. MeLL: Large-Scale Extensible User Intent Classification for Dialogue Systems with Meta Lifelong Learning, page 3649-3659. Association for Computing Machinery, New York, NY, USA.
Hong Wang, Wenhan Xiong, Mo Yu, Xiaoxiao Guo, Shiyu Chang, and William Yang Wang. 2019. Sentence embedding alignment for lifelong relation extraction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 796-806, Minneapolis, Minnesota. Association for Computational Linguistics.
Zirui Wang, Sanket Vaibhav Mehta, Barnabas Poczos, and Jaime Carbonell. 2020. Efficient meta lifelong-learning with limited memory. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 535-548, Online. Association for Computational Linguistics.
Max Welling. 2009. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, page 1121-1128, New York, NY, USA. Association for Computing Machinery.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online. Association for Computational Linguistics.
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. 2020. Supermasks in superposition. In Advances in Neural Information Processing Systems, volume 33, pages 15173-15184. Curran Associates, Inc.
Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020. Clear: Contrastive learning for sentence representation. *ArXiv*, abs/2012.15466.
Congying Xia, Wenpeng Yin, Yihao Feng, and Philip Yu. 2021. Incremental few-shot text classification with multi-round new classes: Formulation, dataset and system. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1351-1360, Online. Association for Computational Linguistics.
Jiaming Xu, Bo Xu, P. Wang, Suncong Zheng, Guanhua Tian, and Jun Zhao. 2017. Self-taught convolutional neural networks for short text clustering. Neural networks : the official journal of the International Neural Network Society, 88:22-31.
Lifelong Intent Detection: CLINC150 (Larson et al., 2019) is a crowdsourced multi-domain (10 domains such as utility, travel etc.) intent detection dataset comprising of 23,700 queries with 22,500 in-scope queries labelled with 150 intents and 1,200 out-of-scope queries. We only use in-scope queries, with both the balanced and imbalanced versions of the provided dataset. HWU64 (Liu et al., 2019) is a crowdsourced multi-domain (21 domains such as alarm, cooking etc.) intent detection dataset comprising of 25,716 queries labelled with 64 intents and is an imbalanced dataset. BANKING77 (Casanueva et al., 2020) is an imbalanced, fine-grained intent detection dataset from the banking domain comprising of 13,083 customer queries labelled with 77 intents. SGD (Rastogi et al., 2020) and MWOZ (Budzianowski et al., 2018) are multi-domain dialogue datasets which consist of dialogues from 19 and 8 domains respectively. Total number of intents in SGD and MWOZ are 39 and 5 respectively.
Text Classification: Stackoverflow (S20) (Xu et al., 2017) is a multi-class classification dataset comprising of 20,000 question titles from Stackoverflow each tagged with one of 20 different tags.
Lifelong Relation Extraction: FewRel (Han et al.,
2018): is a relation extraction dataset where the objective is to predict a relation type between a pair of entities in a given sentence/paragraph. It consists of 56,000 examples tagged with 80 relations introduced incrementally at each step. For all datasets, we use standard train-test split provided by respective authors (shown in table 5).
Table 5 contains details about different intent detection datasets  $\mathcal{D}$  such as CLINC150 with balanced  $(C150^{B})$  and imbalanced version  $(C150^{I})$ , BANKING77, HWU64 (H64), and one text classification dataset, i.e., Stackoverflow (S20). It also contains details about SGD, MWOZ two multi-domain dialogue datasets which is used for intent detection task only. In these dialog datasets, the intent of an utterance can be determined on the basis of that utterance along with  $K$  previous history utterances from the dialog where  $K$  is hyperparameter for preprocessing during dataset creation. In our experiments, we have taken  $K = 0$  and  $K = 1$  to create the dataset for intent detection from SGD and MWOZ datasets.
Table 6 contains details about total number of incremental steps  $(\mathcal{T})$ , total number of intents  $(Y_{all}^{\mathcal{T}})$  in an intent detection dataset  $(\mathcal{D})$  and how these intents have been introduced over incremental steps, i.e.,  $\mathcal{I}_{dist}$ . And all of these datasets are available in english language and released under creative Commons licences.
\( {\mathcal{D}}_{train} \)	\( {\mathcal{D}}_{val} \)	\( {\mathcal{D}}_{test} \)
\( {\mathrm{{CLINC150}}}^{\left( {\mathrm{C}{150}}^{B}\right) } \)	15000	3000	4500
\( {\mathrm{{CLINC150}}}^{\left( {\mathrm{C}{150}}^{I}\right) } \)	10525	3000	4500
\( BANKING77(B77) \)	8463	1540	3080
\( HWU64 \)	8954	1076	1076
Stackoverflow (S20)	16000	-	4000
SGD	8843	1291	2421
MWOZ	2494	208	227
Table 5: Dataset Details
Dataset	Intents	T	I dist
C150B	150	10	{15, 15, 15, 15, 15, 15, 15, 15, 15}
C150I	150	10	{15, 15, 15, 15, 15, 15, 15, 15, 15}
B77	77	7	{20, 10, 10, 10, 10, 10, 7}
H64	64	7	{10, 10, 10, 10, 10, 10, 4}
S20	20	5	{4, 4, 4, 4, 4}
SGD	39	19	{1,2,2,2,4,1,2,2,3,1,2,3,3,2,2,4,2,1,1}
MWOZ	5	5	{1,1,2,1,1}
Table 6: Dataset Details, where "Intents"  $(Y_{all}^{\mathcal{T}})$  represents total number of intents introduced after  $\mathcal{T}$  incremental steps.
Different shuffled versions of a dataset Same set of intents are introduced over different incremental steps to create different shuffled versions of a dataset  $\mathcal{D}$ , as shown in Fig. 6. We run all
basielines,  $PAGeR$  and it's variant on four different shuffled versions of  $\mathcal{I}_{dist}$  for a given  $\mathcal{D}$ .
Figure 6: Different shuffled versions of BANKING77 and FewRel dataset.
For all the experiments reported in this paper we employ the GPT-2 PLM. To ensure a fair comparison across baselines we use GPT-2 as a common backbone for all baseline models. Training for all experiments is run for a fixed number of eight epochs. During training, the batch size was constrained to lie within the range  $[8 - 32]$  and the maximum utterance length was fixed to 200 words. We used the Adam (Kingma and Ba, 2014) optimizer (with default values) for all the models and the learning rate was set to 5e-5. In all variants of PAGeR, we use top-k sampling (Holtzman et al., 2020) with  $k = 20$  and  $\lambda_1 = 1$ ,  $\lambda_2 = 0.25$ ,  $\lambda_3 = 0.25$ . While synthesizing samples for contrastive learning, four negative samples were created for each positive sample. The distillation loss was optimized with a temperature value of 2. Baselines EWC, A-GEM, ER, DER are derived from the DER code repository. Code for PAGeR experiments is derived from LAMOL and "Hugging Face" (Wolf et al., 2020). All the experiments were run on Titan P100, V100 and A100 GPUs. For exemplar replay based approaches,  $\mathcal{M}$  is defined as the percentage of total training samples that can be stored for replay.  $\mathcal{M}$  is uniformly distributed across existing intents. Upon arrival of new intents,  $\mathcal{M}$  is uniformly redistributed across the total intents, both
$^{1}$  https://github.com/aimagelab/mammoth
old and new. In  $PAGeR$ , instead of real samples, we only store the corresponding concept words. For a fair comparison, we calculate the number of tokens that can be stored in the same amount of memory  $\mathcal{M}$  and we generate one pseudo-labelled utterance corresponding to each entry in  $\mathcal{M}$ . We run every approach four times with shuffled versions of  $\mathcal{D}$ , i.e., set of intents introduced at  $\mathcal{T}^{th}$  incremental step can be introduced at  $1^{st}$  incremental step in another run. Please refer to A.3 for further details about memory management over incremental steps for different datasets.
A.3 Memory  $(\mathcal{M})$
To alleviate catastrophic forgetting,  $\mathcal{M}$  contains labelled utterances corresponding to old intents  $Y_{all}^{i}$  while training the model at  $i + 1^{th}$  incremental step. The size of  $\mathcal{M}$  is defined in terms of the percentage of whole labelled training utterances which can be stored in it. For  $PAGeR$ , we define the memory in terms of number of tokens where the number of tokens are obtained by tokenizing the labelled utterances present in the memory at a given incremental step. Instead of storing labelled utterances in  $PAGeR$ , we only store corresponding Concepts which take less memory in terms of tokens. So in  $PAGeR$  we can store more number of Concepts in the same amount of memory as shown in Table 7.
And after  $i + 1^{th}$  incremental step,  $\mathcal{M}$  is equally divided among the intents seen so far, i.e.,  $Y_{all}^{i+1}$  and excess existing labelled utterances corresponding to old intents  $Y_{all}^{i}$  are removed randomly.
Intent Detection
TC
M	Approach	C150B	C150I	B77	H64	S20
10%	Baselines	1500	1040	1000	900	1600
PAGeR	6574	4025	3673	2480	4042
5%	Baselines	750	520	500	450	800
PAGeR	2964	1751	1892	1238	2041
1%	Baselines	150	104	100	90	160
PAGeR	512	516	232	121	401
A.4.1 MSR Vs PAGeR
Liu et al. (2021b) (MSR) shows the experiments in lifelong intent detection with a fixed memory of 200 only. We performed same experiment with  $PAGeR$  and compiled the average accuracy at the last incremental step in table 8.
Table 7: No. of Labelled Utterances/Concepts stored in  $\mathcal{M}$
Dataset	MSR	PAGeR
CLINC150 (C150B)	78.00	92.16
HWU64	52.14	82.38
SNIPS	93.57	94.27
Table 8: PAGeR experiments with fixed memory size (200)
A.4.2 Catastrophic forgetting over incremental steps
We evaluate all baselines, PAGeR-Base and  $PAGeR$  on four shuffled versions of a dataset and report Average Accuracy along with the Standard Deviation as shown in Table 9. We observed that  $PAGeR$  gets similar results on different shuffled versions of a dataset.
It's also evident from Fig. 7, forgetting is less in case of  $PAGeR$  as compared to other baselines as number of intents increases over incremental steps.
Intent Detection	TC
M	Approach	C150B	C150I	B77	HWU64	S20
UB	95.7	94.6	91.9	90.3	90.8
LB	36.9±3.9	30.1±1.4	23.3±2.2	24.4±2.9	21.6±2.9
EWC	36.9±4.9	31.6±4.0	21.1±2.0	23.8±2.4	20.8±1.2
AdapterCL	88.1±0.06	79.9±0.32	77.7±0.97	77.6±0.15	82.5±0.55
10%	A-GEM	30.8±10.7	13.6±0.8	17.3±3.5	18.8±2.9	67.6±5.1
ER	88.9±0.8	81.7±0.4	84.7±1.6	78.3±1.4	85.2±0.6
DER	90.8±0.9	83.9±0.7	80.7±3.7	81.3±2.1	85.6±1.4
PAGeR-Base (LAMOL)	92.7±0.29	89.7±2.14	82.9±2.9	84.0±1.5	72.5±3.5
PAGeR	94.9±0.22	93.4±0.37	90.2±0.26	89.9±0.67	88.7±0.3
5%	A-GEM	30.8±9.1	13.4±1.8	19.2±2.9	18.4±3.1	68.2±4.4
ER	85.2±1.7	71.1±2.1	76.5±1.8	72.0±1.2	82.6±0.6
DER	88.1±0.8	78.1±3.4	73.9±2.7	76.5±1.2	83.9±0.9
PAGeR-Base (LAMOL)	92.0±0.27	87.5±1.45	81.1±3.8	82.2±2.3	71.3±2.2
PAGeR	94.4±0.35	92.9±0.29	89.2±0.48	88.0±0.21	88.0±0.6
1%	A-GEM	28.7±10.5	13.9±1.0	25.3±15.0	17.9±1.4	62.4±6.4
ER	64.8±3.2	41.3±1.8	51.6±3.7	46.2±0.8	76.3±0.8
DER	73.0±1.8	45.9±2.7	53.0±2.7	49.4±1.5	77.4±1.3
PAGeR-Base (LAMOL)	86.9±0.34	63.6±2.37	68.1±1.4	70.5±2.8	68.3±2.1
PAGeR	92.4±0.71	76.4±1.53	80.1±1.46	79.2±0.84	84.3±1.0
Table 9: Average Accuracy (↑) with standard deviation on different datasets.
Figure 7: Average accuracy over incremental steps for different datasets. Here, average accuracy is shown on one shuffled version of each dataset.