A Multi-Task Incremental Learning Framework with Category Name Embedding for Aspect-Category Sentiment Analysis (2020)

Abstract: (T)ACSA tasks, including aspect-category sentiment analysis (ACSA) and targeted aspect-category sentiment analysis (TACSA), aims at identifying sentiment polarity on predefined categories. Incremental learning on new categories is necessary for (T)ACSA real applications. Though current multi-task learning models achieve good performance in (T)ACSA tasks, they suffer from catastrophic forgetting problems in (T)ACSA incremental learning tasks. In this paper, to make multi-task learning feasible for incremental learning, we proposed Category Name Embedding network (CNE-net). We set both encoder and decoder shared among all categories to weaken the catastrophic forgetting problem. Besides the origin input sentence, we applied another input feature, i.e., category name, for task discrimination. Our model achieved state-of-the-art on two (T)ACSA benchmark datasets. Furthermore, we proposed a dataset for (T)ACSA incremental learning and achieved the best performance compared with other strong baselines.

## introduction sentiment analysis has become an increasingly popular natural language processing (nlp) task in academia and industry. it provides real-time feedback on consumer experience and their needs, which helps producers to offer better services. to deal with the presence of multiple categories in one document, (t)acsa tasks, including aspectcategory sentiment analysis (acsa) and targeted aspect-category sentiment analysis (tacsa), were introduced. the main purpose for acsa task is to identify sentiment polarity (i.e. positive, neutral, negative and none) of an input sentence upon specific predefined categories @xcite . for example, as shown in in order to deal with acsa with both multiple categories and multiple targets, tacsa task was introduced @xcite to analyze sentiment polarity on a set of predefined targetcategory pairs. an example is shown in multi-task learning, with shared encoders but individual decoders for each category, is an approach to analyze all the categories in one sample simultaneously for (t)acsa @xcite . compared with single-task ways @xcite , multi-task approaches utilize category-specific knowledge in training signals from each task and get better performance. however, current multi-task models still suffer from a task sentence labels acsa food is always fresh and hot-ready to eat, but it is too expensive (food,positive), (service, none), (price, negative), (ambience, none) (anecdotes/miscellaneous, none) tacsa i like restaurant-1 because it's cheap, but restaurant-2 is too expansive. unfortunately, in the current multi-task learning (t)acsa models, the encoder is shared but the decoders for each category are individual. this parameter sharing mechanism results in only the shared encoder and target-category-related decoders are finetuned during the finetuning process, while the decoder of source categories remains unchanged. the finetuned encoder and original decoder of source categories may cause catastrophic forgetting problem in the origin categories. for real applications, high accuracy is excepted in source categories and target categories. based on the previous researches that decoders between different tasks are usually modeled by mean regularization @xcite , an idea comes up to further make the decoders the same by sharing the decoders in all categories to decrease the catastrophic forgetting problem. but here raises another question, how to identify each category in the encoder and decoder shared network? in our approach, we solve the category discrimination problem by the input category name feature. in this paper, we proposed a multi-task category name embedding network (cne-net). the multi-task learning framework makes full use of training signals from all categories. to make it feasible for incremental learning, both encoder and decoders for each category are shared. the category names were applied as another input feature for task discrimination. we also present a new task for (t)acsa incremental learning. in particular, our contribution is three-folded: ## datasets this section describes the benchmark datasets we used to evaluate our model, the incremental learning task definition, the methodology to prepare the incremental learning dataset, and the evaluation metric. ## approach in this section, we describe the architecture of cne-net for (t)acsa task. in bert classification tasks, the typical approach is feeding sentence "[cls]tokens in sentence[sep]" into the model, while the token "[cls]" is used as a feature for classification. in order to encode category names into bert model, as well as analyze sentiment polarity of all the categories simultaneously, we made two significant differences from the original bert, one on the encoder module and another on the decoder module. ## conclusion in this paper, in order to make multi-task learning feasible for incremental learning, we proposed cne-net with different attention mechanisms. the category name features and the multi-task learning structure help the model achieve state-of-the-art on acsa and tacsa tasks. furthermore, the shared encoder and decoder layers weaken catastrophic forgetting in the incremental learning task. we proposed a task for (t)acsa incremental learning and achieved the best performance with cne-net compared with other strong baselines. further research may be concerned with zero-shot learning on new categories.