Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks (2021)

Abstract: This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks. Although some CL techniques have been proposed for document sentiment classification, we are not aware of any CL work on ASC. A CL system that incrementally learns a sequence of ASC tasks should address the following two issues: (1) transfer knowledge learned from previous tasks to the new task to help it learn a better model, and (2) maintain the performance of the models for previous tasks so that they are not forgotten. This paper proposes a novel capsule network based model called B-CL to address these issues. B-CL markedly improves the ASC performance on both the new task and the old tasks via forward and backward knowledge transfer. The effectiveness of B-CL is demonstrated through extensive experiments.

## introduction continual learning (cl) aims to incrementally learn a sequence of tasks. once a task is learned, its training data is often discarded @xcite . this is in contrast to multi-task learning, which assumes the training data of all tasks are available simultaneously. the cl setting is important in many practical scenarios. for example, a sentiment analysis company typically has many clients and each client often wants to have their private data deleted after use. in the personal assistant or chatbot context, the user does not want his/her chat data, which often contains sentiments or emotions, uploaded to a central server. in such applications, if we want to improve sentiment analysis accuracy for each user/client without breaching confidentiality, cl is a suitable solution. there are two main types of continual learning: (1) task incremental learning (til) and (2) class incremental learning (cil). this work focuses task id domain/task one training example(in that domain/task) 1 vacuum cleaner [cf] this vacuum cleaner sucks !!! 2 desktop [kt] the keyboard is clicky . 3 tablet @xcite the soft keyboard is hard to use. 4 (new task) laptop the new keyboard sucks and is hard to click! on til, where each task is a separate aspect sentiment classification (asc) task. an asc task is defined as follows @xcite : given an aspect (e.g., picture quality in a camera review) and a sentence containing the aspect in a particular domain (e.g., camera), classify if the sentence expresses a positive, negative, or neutral (no opinion) about the aspect. til builds a model for each task and all models are in one neural network. in testing, the system knows which task each test instance belongs to and uses only the model for the task to classify the instance. in cil, each task contains one or more classes to be learned. only one model is built for all classes. in testing, a test case from any class may be presented to the model to classify without giving it any task information. this setting is not applicable to asc. our goal of this paper is to achieve the following two objectives: (1) transfer the knowledge learned from previous tasks to the new task to help learn a better model for the new task without accessing the training data from previous tasks (in contrast to multi-task learning), and (2) maintain (or even improve) the performance of the old models for previous tasks so that they are not forgotten. the focus of the existing cl (til or cil) research has been on solving (2), catastrophic forgetting (cf) @xcite . cf means that when a network learns a sequence of tasks, the learning of each new task is likely to change the net-work parameters learned for previous tasks, which degrades the model performance for the previous tasks @xcite . in our case, (1) is also important as asc tasks are similar, i.e., words and phrases used to express sentiments for different products/tasks are similar. to achieve the objectives, the system needs to identify the shared knowledge that can be transferred to the new task to help it learn better and the task specific knowledge that needs to be protected to avoid forgetting of previous models. fine-tuned bert @xcite is one of the most effective methods for asc @xcite . however, our experiments show that it works very poorly for til. the main reason is that the fine-tuned bert on a task/domain captures highly task specific information which is difficult to transfer to a new task. in this paper, we propose a novel model called b-cl (bert-based continual learning) for asc continual learning. the key novelty is a building block, called continual learning adapter (cla) inspired by the adapter-bert in @xcite . cla leverages capsules and dynamic routing @xcite to identify previous tasks that are similar to the new task and exploit their shared knowledge to help the new task learning and uses task masks to protect task-specific knowledge to avoid forgetting (cf). we conduct extensive experiments over a wide range of baselines to demonstrate the effectiveness of b-cl. in summary, this paper makes two key contributions. (1) it proposes the problem of task incremental learning for asc. (2) it proposes a new model b-cl with a novel adapter cla incorporated in a pre-trained bert to enable asc continual learning. cla employs capsules and dynamic routing to explore and transfer relevant knowledge from old tasks to the new task and uses task masks to isolate task-specific knowledge to avoid cf. to our knowledge, none of these has been done before. ## related work continual learning (cl) has been studied extensively @xcite . to our knowledge, no existing work has been done on cl for a sequence of asc tasks, although cl of a sequence of document sentiment classification tasks has been done. ## preliminary this section introduces bert, adapter-bert and capsule network as they are used in our model. bert for asc. due to its superior performance, this work uses bert @xcite and its transformer @xcite architecture as the base. we also adopt the asc formulation in @xcite , where the aspect term and review sentence are concatenated via @xcite . the sentiment polarity is predicted on top of the [cls] token. although bert can achieve impressive performance on a single asc task, its architecture and fine-tuning paradigm are not suitable for cl (see sec. 1). experiments show that it performs very poorly for cl (sec. 5.4). we found that adapter-bert @xcite is a better fit for cl. adapter-bert. adapter-bert basically inserts a 2-layer fully-connected network (adapter) in each transformer layer of bert (see figure 1(a) ). during training for the end-task, only the adapters and normalization layers are trained, no change to any other bert parameters, which is good for cl because fine-tuning bert itself causes serious forgetting. adapter-bert achieves similar performances to fine-tuned bert @xcite . we propose to exploit the adapter idea and the capsule network to achieve effective cl for asc tasks. capsule network. capsule network (capsnet) is a relatively new classification architecture @xcite . unlike cnn, capsnet replaces the scalar feature detectors with vector capsules that can preserve additional information such as position and thickness in images. a typical capsnet has two capsule layers. the primary layer stores low-level feature maps and the class layer produces the probability for classification with each capsule corresponding to one class. it uses a dynamic routing algorithm to enable each lower level capsule to send its output to the similar (or "agreed", computed by dot product) higher level capsule. this is the key property that we exploit to identify and group similar tasks and their shared features or knowledge. note that the proposed b-cl does not adopt the whole capsule network as we are only interested in the capsule layers and dynamic routing instead of @xcite and its adapters in a transformer @xcite layer. an adapter is a 2-layer fully connected network with a skip-connection. it is added twice to each transformer layer. only the adapters (yellow boxes) and layer norm (green boxes) layers are trainable. the other modules (grey boxes) are frozen. (b). proposed b-cl, which replaces the adapter with cla. cla has two sub-modules: knowledge sharing module (ksm) and task specific module (tsm). each of these modules has a skip-connection. ## continual learning adapter (cla) recall the proposed b-cl aims to achieve (1) knowledge transfer between related old tasks and the new task through knowledge sharing and (2) forgetting avoidance through preventing task specific knowledge of previous tasks from being overwritten by the new task learning. inspired by adapter-bert, we propose the continual learning adapters (cla) to replace the adapters in adapter-bert to enable cl as in figure 1(b ) to achieve bert based continual learning for asc. the architecture of cla is shown in figure 2(a ). it contains two modules: (1) knowledge sharing module (ksm) for identifying and exploiting shareable knowledge from the similar previous tasks and the new task, and (2) task specific module (tsm) for learning task specific neurons and protecting them from being updated by the new task. cla takes two inputs: (1) hidden states h (t) from the feed-forward layer inside a transformer layer and (2) task id t. the outputs are hidden states with features good for the t-th task. ksm leverages capsule layers (see below) and dynamic routing to group similar tasks and the shareable knowledge, whereas tsm takes advantage of task mask (tm) to protect neurons for a particular task and leave other neurons free. those free neurons are later used by tsm for a new task. since tms are differentiable, the whole system b-cl can be trained end-to-end. we detail each module below. ## experiments we now evaluate b-cl by comparing it with both non-continual learning and continual learning baselines. we follow the standard cl evaluation method in @xcite . we first present b-cl a sequence of aspect sentiment classification (asc) tasks for it to learn. once a task is learned, its training data is discarded. after all tasks are learned, we test all task models using their respective test data. in training each task, we use its validation set to decide when to stop training. ## conclusion this paper studies continual learning (cl) of a sequence of asc tasks. it proposed a novel technique called b-cl that can be applied to pretrained bert for cl. b-cl uses continual learning adapters and capsule networks to effectively encourage knowledge transfer among tasks and also to protect task-specific knowledge. experiments show that b-cl markedly improves the asc performance on both the new task and the old tasks via forward and backward knowledge transfer.