An Iterative Multi-Knowledge Transfer Network for Aspect-Based Sentiment Analysis (2021)

Abstract: Aspect-based sentiment analysis (ABSA) mainly involves three subtasks: aspect term extraction, opinion term extraction, and aspect-level sentiment classification, which are typically handled in a separate or joint manner. However, previous approaches do not well exploit the interactive relations among three subtasks and do not pertinently leverage the easily available document-level labeled domain/sentiment knowledge, which restricts their performances. To address these issues, we propose a novel Iterative Multi-Knowledge Transfer Network (IMKTN) for end-to-end ABSA. For one thing, through the interactive correlations between the ABSA subtasks, our IMKTN transfers the task-specific knowledge from any two of the three subtasks to another one at the token level by utilizing a well-designed routing algorithm, that is, any two of the three subtasks will help the third one. For another, our IMKTN pertinently transfers the document-level knowledge, i.e., domain-specific and sentiment-related knowledge, to the aspect-level subtasks to further enhance the corresponding performance. Experimental results on three benchmark datasets demonstrate the effectiveness and superiority of our approach.

## introduction aspect-based sentiment analysis (absa) has drawn increasing attention in the community, which includes three subtasks: aspect term extraction (ae), opinion term extraction (oe) and aspect-level sentiment classification (sc). the first two subtasks aim to extract the aspect term and the opinion term appearing in one sentence, respectively. the goal of the sc subtask is to detect the sentiment polarity towards the extracted aspect term. most existing studies generally handle each task separately @xcite @xcite or take oe as auxiliary task for ae or sc @xcite @xcite , where these separate approaches need to be pipelined or integrated together for practical use. recently, some researches point out that joint methods can achieve promising performance than separate ones, where only two subtasks are coupled, such as ae, oe @xcite or ae, sc @xcite @xcite . more recently, @xcite focus on modeling the interactive relations, i.e., bidirectional ae↔oe, unidirectional @xmath0. to further enhance these subtasks, several researchers seek to the external accessible document-level corpora (containing domainspecific/sentiment-related knowledge foot_0 ) due to the limited aspect-level data @xcite @xcite . as a better case, @xcite merge the documentlevel domain-specific and sentiment-related knowledge together to enhance the ae and sc subtasks, where the two kinds of knowledge are indiscriminate. 2 despite their effectiveness, we argue that the above methods are insufficient to yield satisfactory results for end-to-end absa task due to 1) they merely couple two subtasks or not modeling all bidirectional interactive relations among three subtasks (ae↔oe, ae↔sc and oe↔sc), and 2) the document-level domain-specific/sentiment-related knowledge is coarsely used, which is insufficient to exert their advantages. first, the interactive relations among three aspect-level subtasks are mutually collaborative. for instance, in the sentence "the fish is very delicious.", the opinion term "delicious" indicates that the sentiment polarity of the aspect term "fish" is positive, suggesting the strong interactive correlation among them. conversely, given the aspect term "fish" and its sentiment polarity positive, the word "delicious" rather than other words (e.g., "very") in the sentence will be easily extracted as an opinion term. therefore, the bidirectional relations between three aspect-level subtasks are closely related and they can incrementally promote one another, as shown in the left part of fig. 1 . second, the document-level corpora, which contain domain-specific and sentiment-related knowledge, should be pertinently utilized for enhancing the three aspect-level subtasks of absa. in fact, most aspect and opinion terms own distinct domain-specific properties @xcite while sentiment polarities (i.e., positive, negative, and neutral) are typically domain-invariant. for instance, the aspect term "fish" and the opinion term "delicious" reflect distinct domain-specific characteristics, indicating that they belong to restaurant domain rather than laptop domain. conversely, the domain-specific properties can help distinguish these aspect and opinion terms from other domains or background words (e.g., "very"). therefore, the domain-specific knowledge should be pertinently leveraged to help identify the aspect term and the opinion term rather than on judging sentiment polarity. meanwhile, the sentiment-related knowledge should be targeted at benefiting the sc subtask rather than the ae and oe subtasks, as shown in the right part of fig. 1 . therefore, we propose an iterative multi-knowledge transfer network (imktn) to fully exploit the interactive relations via transferring knowledge at both the token level and the document level for the absa task. partially inspired by the superiority of capsule network in distinguishing different features by feature clustering @xcite , we design a novel routing algorithm, which can mutually transfer task-specific knowledge among the three aspect-level subtasks, as illustrated in the left part of fig. 1 . furthermore, imktn employs a more fine-grained way to pertinently transfer document-level knowledge to aspect-level subtasks, as shown in the right part of fig. 1 , where the knowledge from domain classification subtask only serves for the ae and oe subtasks while the knowledge from document-level sentiment classification subtask only helps the sc subtask. all multi-knowledge transfer processes are iteratively conducted for fully exploiting the knowledge in all tasks to enhance the absa task. in summary, our contributions are three-fold: ## task definition in this section, we formulate the aspect-level tasks and document-level tasks, where the documentlevel tasks are taken as auxiliary tasks for improving the aspect-level tasks. aspect-level tasks. following (chen and qian, 2020), the absa task is formulated as three sequence labeling subtasks. given an input sentence @xmath1: 1) for the ae subtask, we aim to inference a tag sequence y @xmath2, where y ae @xmath3, ia, o} denotes the beginning and the inside of an aspect term, and other words. 2) for the oe subtask, we aim to inference a tag sequence y @xmath4, where y oe @xmath5, ip, o} denotes the beginning and the inside of an opinion term, and other words. 3) for the sc subtask, we aim to inference a tag sequence y @xmath6, where y sc @xmath7, neg, neu} denotes positive, negative and neutral sentiment polarities. document-level tasks. this work contains two document-level subtasks: domain classification (ddc) and sentiment classification (dsc). for an input document @xmath8, s 2 , . . . , s m } with m sentences, the ddc and dsc aim to predict a domain label y @xmath9, restaurant} and a sentiment label y @xmath10, respectively. ## model as shown in fig. 2 , the imktn consists of four parts: 1) shared encoder, for extracting n-gram features; 2) task-specific layers, for capturing sentence representations; 3) aspect-level knowledge transfer, including three routing blocks, for fully transferring knowledge among the aspect-level subtasks for mutual reinforcing; and 4) document-level knowledge transfer, for pertinently transfer-ring document-level knowledge to corresponding aspect-level tasks. finally, multi-source information is aggregated for the next iteration. ## related work aspect-based sentiment analysis. existing models typically handle the absa task independently or jointly. apparently, separately treating each subtask cannot exploit the inter-task correlations, leading to restricted performances, such as ae @xcite @xcite @xcite @xcite @xcite and sc @xcite @xcite @xcite @xcite @xcite @xcite . by contrast, the integrated or joint methods @xcite @xcite @xcite @xcite @xcite ) can model the interactive correlations and thus achieve promising results. different from above studies, we focus on exploiting the inter-task correlations among the three aspect-level subtasks and thus incrementally boost one another. besides, we observe the task characteristics and then use the document-level corpora to pertinently help the corresponding aspect-level subtasks. capsule network. capsule network @xcite has been widely applied in many natural language processing tasks. in absa, wang et al. (2019) focus on building multiple capsules for aspect category sentiment analysis, which do not employ the routing procedure. chen and qian (2019) construct a transfer capsule network for transferring semantic knowledge from dsc to sc via sharing the encoder, which utilizes the vanilla capsule network only for the sc subtask. du et al. (2019) combine capsule network with interactive attention to model the interactive relationship between the given aspect term and context for the sc subtask. jiang et al. (2019) release a new large-scale multi-aspect multi-sentiment dataset and use capsule network building a strong baseline. unlike these methods, we pay attention to the end-to-end absa task rather than the individual subtask, and propose a dynamic-length to dynamiclength routing algorithm, which can efficiently perform the multi-knowledge transfer. ## conclusion in this paper, we propose an iterative multiknowledge transfer network for the absa task, which can fully exploit the inter-task correlations among the three aspect-level subtasks with the proposed routing algorithm. moreover, we design a more fine-grained method enabling our model to incorporate the document-level knowledge for pertinently enhancing the corresponding aspect-level tasks. experimental results on three benchmark datasets demonstrate the effectiveness of our proposed approach, which yields state-of-the-art performance on most metrics. we following (chen and qian, 2020) fix the domain-specific and general-specific word embeddings in all models, where the domain-specific embedding vectors are 100 dimensions. the trainable weight matrices in the cnn are initialized by following the glorot uniform strategy @xcite . besides, all biases are initialized as zero. we tune the number of cnn layer on the validation set of each dataset. finally, the cnn layer number in the shared encoder is set to 2, and is fixed as 2, 2, 1 for the ate subtask, the ote subtask, and the asc subtask in task-specific layers, respectively. the cnn layer in the shared encoder has 150 filters with kernel size @xmath11. the cnn layers in each taskspecific encoder have 300 filters with kernel size @xmath12. the activation function is relu for each cnn layer. dropout is employed after the embedding layer and each cnn layer, which is empirically set to 0.5. the discount coefficients λ 1 , λ 2 , λ 3 , λ 4 and λ 5 in loss functions are not fine-tuned and empirically set to 1.0. since the extracted aspect term may consist of several tokens and the predicted polarity of each token may be inconsistent, we thus following (chen and qian, 2020) only take the sentiment polarity of the first token of the current aspect term as the sentiment label for measuring the performance. we also note that only aspect terms have sentiment annotations and thus following (chen and qian, 2020) only consider asc predictions on these aspect term-related tokens for computing the asc loss and ignore the sentiments predicted on other tokens. for training, we first train the model with document-level tasks for five epochs, and then alternately train our model on aspect-level tasks with 2 epochs and document-level tasks with 1 epoch. finally, we train the model for a fixed number of epochs, and obtain the best results at the epoch with the best f1-absa score on the validation set for producing the testing results, as did in @xcite . in our experiments, following (chen and qian, 2020), we also use bert large @xcite as the backbone to further investigate our model performance. the neural model is implemented in keras and all computations are done on an nvidia tesla v100 gpu, where each experiment runs about 1∼3 hours. hyperparameter configurations for best-performing models have explained above. the method of choosing hyperparameter values is manual tuning on the validation and the criterion used to select is f1-absa. the downloadable version of used data can be found in: https:// github.com/ruidan/imn-e2e-absa , provided by imn @xcite , where we use this data without any pre-processing.