Using the Past Knowledge to Improve Sentiment Classification (2020)

Abstract: This paper studies sentiment classification in the lifelong learning setting that incrementally learns a sequence of sentiment classification tasks. It proposes a new lifelong learning model (called L2PG) that can retain and selectively transfer the knowledge learned in the past to help learn the new task. A key innovation of this proposed model is a novel parameter-gate (p-gate) mechanism that regulates the flow or transfer of the previously learned knowledge to the new task. Specifically, it can selectively use the network parameters (which represent the retained knowledge gained from the previous tasks) to assist the learning of the new task t. Knowledge distillation is also employed in the process to preserve the past knowledge by approximating the network output at the state when task t-1 was learned. Experimental results show that L2PG outperforms strong baselines, including even multiple task learning.

## introduction a typical sentiment analysis (sa) or social media company that provides sentiment analysis services has to work for a large number of clients @xcite . each client normally wants to study people's opinions about a particular category of products or services, which we also call a domain. if we regard each such study/project as a task, we can model a sa company's working on a large number of studies/projects for clients as performing a sequence of sa tasks. a natural question that one would ask is whether after analyzing opinions about a number of products or services (tasks), the sa system of the company can do better on a new task by retaining the knowledge learned from the past/previous tasks and selectively transfer the prior knowledge to the new task to help it learn better. the answer should be yes because words and phrases used to express opinions or sentiments in different domains are similar and thus can mostly be shared or transferred across domains, although different domains do have domain specific sentiment expressions. this is a lifelong learning setting @xcite @xcite . this paper focuses on lifelong sentiment classification @xcite . problem definition: we consider incrementally learning a sequence of supervised sentiment classification (sc) tasks, 1, ..., t, .... each task t has a training dataset d t @xmath0, y t i } nt @xmath1, where this problem is clearly related a continual learning (cl) @xcite @xcite @xcite @xcite , which also aims to learn a sequence of tasks incrementally. however, the main objective of the current cl techniques is to solve the catastrophic forgetting (cf) problem @xcite . that is, in learning each new task, the network parameters need to be modified in order to learn the new task. however, this modification can result in accuracy degradation for the previously learned tasks. in the problem defined above, our goal is to forward transfer the past knowledge to improve the new task learning. we don't need to ensure the classifiers or models learned for previous tasks still work well. 1 however, as we will see in the experiment section, the proposed method is able to outperform the current state-of-the-art cl algorithms. although there is some existing work on lifelong sentiment classification @xcite based on naive bayes. our deep learning model is based on an entirely different approach and it performs markedly better. to solve the proposed lifelong sentiment classification problem using a single neural network, two objectives have to be achieved. the first objective is to selectively transfer some pieces of knowledge learned in the past to assist the new task learning. knowledge selection is critical here because not every piece of the past knowledge is useful (some even harmful) to the new task. the second objective is to preserve the knowledge learned in the past during learning the new task because if many pieces of previous knowledge are corrupted due to updates made in learning a new task, future tasks will not be able to benefit from them. this paper proposes a novel model, called l2pg (lifelong learning with parameter-gates), to achieve the objectives. to achieve the first objective, we propose a novel mechanism called the parameter-gate (p-gate) to give suitable importance values to the network parameters representing the past knowledge according to how useful they are to the new task and transfer them to the new task to enable it to learn better. we split the parameters θ t of the proposed model f (•; θ t ) into three subsets: (1) the shared parameters θ s,t , (2) the task classification parameters θ c,t and (3) the p-gate parameters, where the shared parameters θ s,t and pgate parameters are continuously updated with the learning of each new task t. θ c,t remains unchanged for task t once the task is learned/trained. in learning a new task t, we only randomly initialize the task classification parameters θ c,t , and use an input p-gate to select parameters (or knowledge) from the shared parameters θ s,t-1 of the network state after learning task t -1 that are helpful to the new task t and use a block p-gate to block part of the previous training step parameters of θ s,t that are not useful (or harmful) to task t. to achieve the second objective, knowledge dis-tillation @xcite is used to ensure that the updated network can preserve the previous model's knowledge in learning the new task. this paper makes three main contributions: to our knowledge, no existing sentence sentiment classification corpus fits this need. experimental results show that l2pg outperforms state-of-the-art baselines including multitask learning, which optimizes all the tasks at the same time. ## related work our work is related to sentiment classification @xcite , lifelong learning and continual learning. for sentiment classification, recent deep learning models have been shown to outperform traditional methods @xcite @xcite @xcite . however, these models don't retain or transfer the knowledge to new tasks. lifelong learning: most relevant to our work is lifelong learning @xcite @xcite @xcite . for lifelong sentiment classification, @xcite used naive bayes to leverage word probabilities under different classes in old tasks/domains as priors to help optimize the new task learning. @xcite worked similarly but their method can improve the model of a previous task without retraining. @xcite proposed a voting method but their method works on the same data from different time periods. @xcite proposed a model using two networks, one for knowledge retention and one for feature learning. but it was shown to be weaker than @xcite . l2pg has a very different approach and performs markedly better. @xcite studied aspect level sentiment classification, which is not the goal of l2pg. however, to the best of our knowledge, none of these methods used gated mechanisms to regulate the transfer of knowledge in the lifelong learning process. continual learning: it is similar to lifelong learning, but its main goal is to overcome catastrophic forgetting to ensure learning of a new task will not forget the models learned for previous tasks @xcite . for example, lwf @xcite uses knowledge distillation loss to ensure that after learning a new task, it can still approximate the performance of the old tasks. ewc @xcite introduces constraints to control parameter changes when learning a new task. hat @xcite masks units that are important to previous tasks by a hard attention. pgma @xcite generates a subset of parameters. two reviews of continual learning can be found in @xcite . our lifelong learning setting focuses on transferring the past knowledge to the current task. we don't ensure that the models learned in the past still work well after learning a new task. although progressive networks @xcite also tries to help future learning through knowledge transfer, but it is not scalable as its network size scales quadratically in the number of tasks. knowledge distillation loss was proposed in @xcite for transferring knowledge in a large model to a smaller one. lwf uses knowledge distillation to help deal with forgetting. @xcite proposed an information preserving penalty, attention distillation loss, to preserve the information about existing classes. this setting is different from ours as it incrementally learns more classes. each of our tasks is an independent sentiment classification problem with multiple classes. ## the proposed l2pg model the working of the proposed model l2pg in learning the new task t is illustrated in figure 1 . our learner f (•; θ t ) consists of three modules and two loss functions. the first module is the shared knowledge module (sk), which consists of a cnn (i.e., convolutional neural network) with various filters. it contains the shared knowledge across tasks in its parameters θ s,t . the second module is the task classification module (tc) with parameters θ c,t , which is a fully connected layer for the classification of task t. there is one tc for each task and it is fixed once t is learned. the third module is the p-gate module (pg). in learning each new task t, a temporary copy of sk and of tc (in the yellow boxes of figure 1 ) are made from the state of the network after task t -1 was learned. for clarity, we use the superscript • to indicate a copy of something. for example, θ s,t-1,• and θ c,t-1,• denote the copies of θ s,t-1 and θ c,t-1 respectively. they are fixed and not updated during the learning of task t. sk (in the blue box) and pg (in the blue disk) are updated in learning task t, and are also used in testing. the goal of pg is to identify useful knowledge for task t from the parameters θ s,t-1,• of sk after task t -1 training and to block the unhelpful or harmful knowledge in sk (see sec. 3.3) for the current task. knowledge distillation is used to ensure that in learning task t, the knowledge gained from the previous tasks are not forgotten. updating the parameters of sk, tc and pg are done through back propagation. the two loss functions used are knowledge distillation loss and cross entropy loss. ## experiments we now evaluate l2pg and compare it with two main types of baselines, i.e., those under lifelong sentiment classification and those under continual learning for dealing with catastrophic forgetting. algorithm 1 l2pg -learning the new task t 1: input: training set d train t of task t, and shared parameters θ s,t-1,• and task classification parameters θ c,t-1,• from task t -1. 2: initialize: θ s,t 0 ← θ s,t-1 // 0 denotes training step θ c,t 0 ← random(|θ c,t |) 3: for each training step @xmath2, 1, ..., m do 4: sample one batch x t s from d train t ; 5: // compute outputs for loss l d 6: y t @xmath3; θ s,t-1,• , θ c,t-1,• ); 7: y t @xmath4; θ s,t s , θ c,t-1,• ) ; 8: // compute output for loss l c 9: y @xmath5; θ s,t s , θ c,t s ); 10: update parameters: 11: parameters θ s,t s , θ c,t s , w z and w b are updated by minimize eq. 12; 12: // use the trained p-gate parameters to // select the knowledge for the next step 13: we carried out experiments on two datasets. the first dataset is for document level sentiment classification with two classes, positive and negative. it consists of reviews of 16 diverse kinds of products (domains) commonly used in multi-task text classification @xcite . the reviews of the first 14 products are from amazon.com. the remaining two are about movie reviews (imdb and mr). the number of training and testing samples for each product (or task) is about 1,400 and 400, respectively. we call this dataset mix-16, which gives us 16 tasks, one per product category/domain. the second dataset is for sentence-level sentiment classification and is created by us. it consists of review sentences of 10 types of products/domains crawled from amazon.com, which gives us 10 tasks. each sentence is labeled with positive, negative or neutral. the sentences with conflict opinions (e.g., both positive and negative) are not used. sentence sentiment classification of each domain forms a task. the review sentences of each product are annotated by two annotators independently. we trained all the annotators and provided them with an annotation instruction document. after training, each of them was asked to perform annotation of 50 sentences to assess their annotation quality. they started their annotation only after we were satisfied with their annotations. after they completed their annotations, sentences with disagreements were identified and discussed by the annotators to come to an agreement. the kappa score for annotator agreement was 0.7947. note that we are aware that there are some existing sentence sentiment classification data, but each of them is only from reviews of a single product. we are unable to create many different domain tasks from them to suit lifelong learning. furthermore, they mostly have only two classes, positive and negative, which do not reflect all review sentences because many review sentences express no sentiment (neutral), e.g., "i bought this camera yesterday." that is why we created the new dataset with 10 different categories of products, which give us 10 tasks for lifelong learning. 2 we denote this dataset as amazon-10. ## conclusion this paper proposed an effective model l2pg for lifelong sentiment classification. l2pg not only can retain what it has learned, but also selectively transfer the past knowledge to learn the new task better. the key component is the proposed parameter gate (p-gate) mechanism that is able to select the right previously learned knowledge or parameters to transfer to the new task. knowledge distillation is also employed to maintain the knowledge or models learned for the previous tasks. empirical evaluation showed l2pg outperforms strong baselines in lifelong learning, continual learning, and even multi-task learning.