SlowGuess/NAACL_2021_OCR huggingface dataset:
where  $b$  is the batch index and  $B$  is the total number of batches in an epoch.
Illustration. In Figure 2(B), after learning the first task (Task 0), we obtain its useful neurons marked in orange with a 1 in each neuron, which serves as a mask in learning future tasks. In learning task 1, those useful neurons for task 0 are masked (with 0 in those orange neurons or cells on the left). The process also learns the useful neurons for task 1 marked in green with 1's. When task 2 arrives, all important neurons for tasks 0 and 1 are masked, i.e., its mask entries are set to 0 (orange and green before training). After training task 2, we see that task 2 and task 1 have a shared neuron that is important to both of them. The shared neuron is marked in both red and green.
We now evaluate B-CL by comparing it with both non-continual learning and continual learning baselines. We follow the standard CL evaluation method in (Lange et al., 2019). We first present B-CL a sequence of aspect sentiment classification (ASC) tasks for it to learn. Once a task is learned, its training data is discarded. After all tasks are learned, we test all task models using their respective test data. In training each task, we use its validation set to decide when to stop training.
5.1 Experiment Datasets
Since B-CL works in the CL setting, we employ a set of 19 ASC datasets (reviews of 19 products) to produce sequences of tasks. Each dataset represents a task. The datasets are from 4 sources: (1) HL5Domains (Hu and Liu, 2004) with reviews of 5 products; (2) Liu3Domains (Liu et al., 2015) with reviews of 3 products; (3) Ding9Domains (Ding et al., 2008) with reviews of 9 products; and (4) SemEval14 with reviews of 2 products - SemEval 2014 Task 4 for laptop and restaurant. For (1), (2) and (3), we split about  $10\%$  of the original data as the validation data, another about  $10\%$  of the original data as the testing data. For (4), we use 150 examples from the training set for validation. To be consistent with existing research
Data source	Task/domain	Train	Validation	Test
Liu3domain	Speaker	352	44	44
Router	245	31	31
Computer	283	35	36
HL5domain	Nokia6610	271	34	34
Nikon4300	162	20	21
Creative	677	85	85
CanonG3	228	29	29
ApexAD	343	43	43
Ding9domain	CanonD500	118	15	15
Canon100	175	22	22
Diaper	191	24	24
Hitachi	212	26	27
Ipod	153	19	20
Linksys	176	22	23
MicroMP3	484	61	61
Nokia6600	362	45	46
Norton	194	24	25
SemEval14	Rest.	3452	150	1120
Laptop	2163	150	638
Table 2: Number of examples in each task or dataset. More detailed data statistics are given in the Appendix.
(Tang et al., 2016), examples belonging to the conflict polarity (both positive and negative sentiments are expressed about an aspect term) are not used. Statistics of the 19 datasets are given in Table 2.
5.2 Compared Baselines
We use 18 baselines, including both non-continual learning and continual learning methods.
Non-continual Learning (NL) Baselines: NL setting builds a model for each task independently using a separate network. It clearly has no knowledge transfer or forgetting. We have 3 baselines under NL, (1) BERT, (2) Adapter-BERT and (3) W2V (word2vec embeddings). For BERT, we use trainable BERT to perform ASC (see Sec. 3); Adapter-BERT adapts the BERT as in (Houlsby et al., 2019), where only the adapter blocks are trainable; W2V uses embeddings trained on the Amazon review data in (Xu et al., 2018) using FastText (Grave et al., 2018). We adopt the ASC classification network in (Xue and Li, 2018), which takes both aspect term and review sentence as input.
Continual Learning (CL) Baselines. CL setting includes 3 baselines without dealing with forgetting (WDF) and 12 baselines from 6 state-of-the-art task incremental learning (TIL) methods dealing with forgetting. WDF baselines greedily learn a sequence of tasks incrementally without explicitly tackling forgetting or knowledge transfer. The 3 baselines under WDF are also (4) BERT, (5) Adapter-BERT and (6) W2V.
The 6 state-of-the-art CL systems are: KAN, SRK, HAT, UCL, EWC and OWM. KAN (Ke et al.,
2020b) and SRK (Lv et al., 2019) are TIL methods for document sentiment classification. HAT, UCL, EWC and OWM were originally designed for image classification. We replace their original MLP or CNN image classification network with CNN for text classification (Kim, 2014). HAT (Serra et al., 2018) is one of the best TIL methods with almost no forgetting. UCL (Ahn et al., 2019) is a latest TIL method. EWC (Kirkpatrick et al., 2016) is a popular regularization-based class incremental learning (CIL) method, which was adapted for TIL by only training on the corresponding head of the specific task ID during training and only considering the corresponding head's prediction during testing. OWM (Zeng et al., 2019) is a state-of-the-art CIL method, which we also adapt to TIL.
From the 6 systems, we created 6 baselines using W2V embeddings with the aspect term added before the sentence so that the CL methods can take both aspect and the review sentence, and 6 baselines using BERT (Frozen) (which replaces W2V embeddings). Following the BERT formulation in Sec. 3, it can naturally take both aspect and review sentence. Adapter-BERT is not applicable to them as their architecture cannot use an adapter.
Unless otherwise stated, for the task sharing module, we employ 2 layers of fully connected network with dimensions 768 in TCL. We also employ 3 knowledge sharing capsules. The dynamic routing is repeated for 3 iterations. For the task-specific module, We employ the embedding with 2000 dimensions as the final and hidden layer of the TSM. The task ID embeddings have 2000 dimensions. A fully connected layer with softmax output is used as the classification heads in the last layer of the BERT, together with the categorical cross-entropy loss. We use 140 for  $s_{\mathrm{max}}$  in Eq. 11, dropout of 0.5 between fully connected layers. The training of BERT, Adapter-BERT and B-CL follow that of (Xu et al., 2019). We adopt  $\mathrm{BERT}_{\mathrm{BASE}}$  (uncased). The maximum length of the sum of sentence and aspect is set to 128. We use Adam optimizer and set the learning rate to 3e-5. For the SemEval datasets, 10 epochs are used and for all other datasets, 30 epochs are used based on results from validation data. All runs use the batch size 32. For the CL baselines, we train all models with the learning rate of 0.05. We early-stop training when there is no improvement in the validation loss for 5 epochs. The
Scenario	Category	Model	Acc.	MF1
Non-continual Learning	BERT	NL	0.8584	0.7635
Adapter-BERT	NL	0.8596	0.7807
W2V	NL	0.7701	0.5189
Continual Learning	BERT	WDF	0.4960	0.4308
Adapter-BERT	WDF	0.5403	0.4481
W2V	WDF	0.8269	0.7356
KAN	0.8549	0.7738
SRK	0.8476	0.7852
BERT (Frozen)	EWC	0.8637	0.7452
UCL	0.8389	0.7482
OWM	0.8702	0.7931
HAT	0.8674	0.7816
W2V	KAN	0.7206	0.4001
SRK	0.7101	0.3963
EWC	0.8416	0.7229
UCL	0.8441	0.7599
OWM	0.8270	0.7118
HAT	0.8083	0.6363
B-CL (forward)	0.8809	0.7993
B-CL	0.8829	0.8140
batch size is set to 64. For all the CL baselines, we use the code provided by their authors and adopt their original parameters (for EWC, we adopt its TIL variant implemented by (Serrà et al., 2018)).
Since the order of the 19 tasks may have an impact on the final results, we randomly choose and run 5 task sequences and average their results. We compute both accuracy and Macro-F1 over 3 classes of polarities, where Macro-F1 is the major metric as the imbalanced classes introduce biases on accuracy. Table 3 gives the average results of 19 tasks (or datasets) over the 5 random task sequences.
Overall Performance. Table 3 shows that B-CL outperforms all baselines markedly. We discuss the detailed observations below:
(1) For non-continual learning (NL) baselines, BERT and Adapter-BERT perform similarly. W2V is poorer, which is understandable.
(2) Comparing NL (non-continual learning) and WDF (continual learning without dealing with forgetting), we see WDF is much better than NL for W2V. This indicates ASC tasks are similar and have shared knowledge. Catastrophic forgetting (CF) is not a major issue for W2V.
However, WDF is much worse than NL for BERT (with fine-tuning) and Adapter-BERT (with adapter-tuning). This is because BERT with fine-tuning learns highly task specific knowledge (Merchant et al., 2020). While this is desirable for NL,
Table 3: Accuracy (Acc.) and Macro-F1 (MF1) averaged over 5 random sequences of 19 tasks.
Model	Acc.	MF1
B-CL (-KSM; -TSM)	0.5403	0.4481
B-CL (-KSM)	0.8614	0.7852
B-CL (-TSM)	0.8312	0.7107
B-CL	0.8829	0.8140
Table 4: Ablation experiment results.
it is bad for WDF because task specific knowledge is hard to share across tasks or transfer. Then WDF causes serious forgetting (CF) for CL.
(3) Unlike BERT and Adapter-BERT, our B-CL can do very well in both forgetting avoidance and knowledge transfer (outperforming all baselines). For state-of-the-art CL baselines, EWC, UCL, OWM and HAT, although they perform better than WDF, they are all significantly poorer than B-CL as they don't have methods to encourage knowledge transfer. KAN and SRK do knowledge transfer but they are for document-level sentiment classification. They are weak, even weaker than other CL methods.
Effectiveness of Knowledge Transfer. We now look at knowledge transfer of B-CL. For forward transfer (B-CL(forward)) in Table 3, we use the test accuracy and MF1 of each task when it was first learned. For backward transfer (B-CL in Table 3), we use the final result after all tasks are learned. By comparing the results of NL with the results of forward transfer, we can see whether forward transfer is effective. By comparing the forward transfer result with the backward transfer result, we can see whether the backward transfer can improve further. The average results of B-CL forward (B-CL(forward)) and backward (B-CL) are given in Table 3. It shows that forward transfer of B-CL is highly effective (forward results for other CL baselines are given in the Appendix and we see B-CL's forward result outperforms all baselines' forward results). For backward transfer, B-CL slightly improves the performance.
Ablation Experiments. The results of ablation experiments are in Table 4. “-KSM; -TSM” means without knowledge sharing and task specific modules, simply deploying an Adapter-BERT. “-KSM” means without the knowledge sharing module. “-TSM” means without the task specific module. Table 4 clearly shows that the full B-CL system always gives the best overall results, indicating every component contributes to the model.
This paper studies continual learning (CL) of a sequence of ASC tasks. It proposed a novel technique called B-CL that can be applied to pretrained BERT for CL. B-CL uses continual learning adapters and capsule networks to effectively encourage knowledge transfer among tasks and also to protect task-specific knowledge. Experiments show that B-CL markedly improves the ASC performance on both the new task and the old tasks via forward and backward knowledge transfer.
This work was supported in part by two grants from National Science Foundation: IIS-1910424 and IIS-1838770, a DARPA Contract HR001120C0023, and a research gift from Northrop Grumman.
Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. 2019. Uncertainty-based continual learning with adaptive regularization. In NIPS.
Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2019. Efficient lifelong learning with A-GEM. In ICLR.
Zhiyuan Chen and Bing Liu. 2018. Lifelong machine learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12(3):1-207.
Zhiyuan Chen, Nianzu Ma, and Bing Liu. 2015. Life-long learning for sentiment classification. In ACL.
Zhuang Chen and Tieyun Qian. 2019. Transfer capsule network for aspect level sentiment classification. In ACL.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT.
Xiaowen Ding, Bing Liu, and Philip S Yu. 2008. A holistic lexicon-based approach to opinion mining. In Proceedings of the 2008 international conference on web search and data mining.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. 2017. Pathnet: Evolution channels gradient descent in super neural networks. CoRR.
Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018. Learning word vectors for 157 languages. In LREC.
Xu He and Herbert Jaeger. 2018. Overcoming catastrophic interference using conceptor-aided backpropagation. In ICLR.
Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. 2011. Transforming auto-encoders. In International conference on artificial neural networks.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In ICML.
Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of ACM SIGKDD.
Nitin Kamra, Umang Gupta, and Yan Liu. 2017. Deep generative dual memory network for continual learning. CoRR.
Zixuan Ke, Bing Liu, and Xingchang Huang. 2020a. Continual learning of a mixed sequence of similar and dissimilar tasks. In NeurIPS.
Zixuan Ke, Bing Liu, Hao Wang, and Lei Shu. 2020b. Continual learning with knowledge transfer for sentiment classification. In ECML-PKDD.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In EMNLP.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. 2016. Overcoming catastrophic forgetting in neural networks. CoRR.
Matthias De Lange, Rahaf Aljundi, Marc Masana, and Tinne Tuytelaars. 2019. Continual learning: A comparative study on how to defy forgetting in classification tasks. CoRR.
Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In NIPS.
Bing Liu. 2015. Sentiment analysis: Mining opinions, sentiments, and emotions. Cambridge University Press.
Qian Liu, Zhiqiang Gao, Bing Liu, and Yuanlin Zhang. 2015. Automated rule selection for aspect extraction in opinion mining. In *IJCAI*.
David Lopez-Paz and Marc'Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In NIPS.
Guangyi Lv, Shuai Wang, Bing Liu, Enhong Chen, and Kun Zhang. 2019. Sentiment classification by leveraging the shared knowledge from a sequence of domains. In DASFAA.
Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In CVPR.
Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation.
Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, and Ian Tenney. 2020. What happens to BERT embeddings during fine-tuning? CoRR.
German Ignacio Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. 2019. Continual lifelong learning with neural networks: A review. Neural Networks.
Qi Qin, Wenpeng Hu, and Bing Liu. 2020. Using the past knowledge to improve sentiment classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 1124-1133.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. 2017. icarl: Incremental classifier and representation learning. In CVPR.
Mohammad Rostami, Soheil Kolouri, and Praveen K. Pilly. 2019. Complementary learning for overcoming catastrophic forgetting using experience replay. In *IJCAI*.
Paul Ruvolo and Eric Eaton. 2013. ELLA: an efficient lifelong learning algorithm. In ICML, pages 507-515.
Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. In NIPS.
Ari Seff, Alex Beatson, Daniel Suo, and Han Liu. 2017. Continual learning in generative adversarial nets. CoRR, abs/1705.08395.
Joan Serrà, Didac Suris, Marius Miron, and Alexandros Karatzoglou. 2018. Overcoming catastrophic forgetting with hard attention to the task. In ICML.
Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In NIPS.
Lei Shu, Hu Xu, and Bing Liu. 2017. Lifelong learning CRF for supervised aspect extraction. In ACL.
Daniel L. Silver, Qiang Yang, and Lianghao Li. 2013. Lifelong machine learning systems: Beyond learning algorithms. In Lifelong Machine Learning, Papers from the 2013 AAAI Spring Symposium, Palo Alto, California, USA, March 25-27, 2013.
Chi Sun, Luyao Huang, and Xipeng Qiu. 2019. Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In NAACL.
Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. In EMNLP.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS.
Hao Wang, Bing Liu, Shuai Wang, Nianzu Ma, and Yan Yang. 2019. Forward and backward knowledge transfer for sentiment classification. In ACML.
Shuai Wang, Guangyi Lv, Sahisnu Mazumder, Geli Fei, and Bing Liu. 2018. Lifelong learning memory networks for aspect sentiment classification. In IEEE International Conference on Big Data.
Rui Xia, Jie Jiang, and Huihui He. 2017. Distantly supervised lifelong learning for large-scale social media sentiment analysis. IEEE Trans. Affective Computing, 8(4):480-491.
Hu Xu, Bing Liu, Lei Shu, and Philip S. Yu. 2019. BERT post-training for review reading comprehension and aspect-based sentiment analysis. In NAACL-HLT.
Hu Xu, Sihong Xie, Lei Shu, and Philip S. Yu. 2018. Dual attention network for product compatibility and function satisfiability analysis. In AAAI.
Wei Xue and Tao Li. 2018. Aspect based sentiment analysis with gated convolutional networks. In ACL.
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. 2019. Continuous learning of context-dependent processing in neural networks. Nature Machine Intelligence.
Wei Zhao, Haiyun Peng, Steffen Eger, Erik Cambria, and Min Yang. 2019. Towards scalable and reliable capsule networks for challenging NLP applications. In ACL.