METN et: A Mutual Enhanced Transformation Network for Aspect-based Sentiment Analysis (2020)

Abstract: Aspect-based sentiment analysis (ABSA) aims to determine the sentiment polarity of each specific aspect in a given sentence. Existing researches have realized the importance of the aspect for the ABSA task and have derived many interactive learning methods that model context based on specific aspect. However, current interaction mechanisms are ill-equipped to learn complex sentences with multiple aspects, and these methods underestimate the representation learning of the aspect. In order to solve the two problems, we propose a mutual enhanced transformation network (METNet) for the ABSA task. First, the aspect enhancement module in METNet improves the representation learning of the aspect with contextual semantic features, which gives the aspect more abundant information. Second, METNet designs and implements a hierarchical structure, which enhances the representations of aspect and context iteratively. Experimental results on SemEval 2014 Datasets demonstrate the effectiveness of METNet, and we further prove that METNet is outstanding in multi-aspect scenarios.

## introduction aspect-based sentiment analysis (absa) aims to determine the orientation of sentiment expressed on each aspect in a sentence @xcite . for instance, in the sentence "although the service is not that great, i still like the food", the user mentions two aspects "service" and "food" and expresses negative sentiment and positive sentiment respectively. the absa task consists of two subtasks including aspect extraction and aspect sentiment classification @xcite . in this paper, we assume that the aspects are given and only focus on the aspect sentiment classification task. in absa task, multiple aspects could appear in one sentence. when predicting the sentiment of current aspect, words related to other aspects with different sentiment tendencies maybe become noises. therefore, how to effectively model the semantic relationship between given aspect and context words is an important challenge. the traditional methods mainly rely on manually designed features, which is labor-intensive, and this representation method almost reaches its performance bottleneck @xcite . boosted by the recent development of deep learning techniques, some studies utilize attention mechanism to solve the above problems well, and many neural attention models have been proposed @xcite @xcite . in these works, the model generally obtains the aspect representation first and then applies attention mechanism to extract the context features related to the given aspect for sentiment prediction. however, the attention mechanism has some drawbacks. when a sentence contains multiple aspects with different sentiment tendencies, opinion modifiers of other aspects are noise information for the current aspect. but, it is hard for the attention mechanism to differentiate opinion modifiers of multiple aspects, which directly affects the final prediction result. for example, in the sentence "i like coming back to mac os but this laptop is lacking in speaker quality compared to my $400 old hp laptop", the attention mechanism should pay more attention to the opinion word "like" with positive sentiment tendency for aspect "mac os". however, the attention mechanism tends to involve irrelevant opinion words, such as "lacking" with negative sentiment tendency, which interferes with the sentiment prediction of aspect "mac os". to this end, some researchers put forward many works @xcite to overcome the shortcomings inherent in attention mechanism for the absa task. however, most of these works are devoted to designing complex neural networks to improve the representation learning of the context. few works focus on how to improve the representation learning of the aspect. in recent work, @xcite learns the effective representations of aspect and context alternately with an iteration mechanism and finally obtains more accurate prediction results. in order to make the aspect play a better role in the absa task, we propose the mutual enhanced transformation network (metnet). metnet improves the representation learning of the aspect and extracts more effective context features based on enhanced aspect representation. inspired by tnet @xcite and coattention-memnet @xcite , metnet utilizes the hierarchical structure to learn the representations of aspect and context iteratively, and we name each computational layer as bidirectional enhancement transformation (bet) layer. each bet layer has three parts, including a bidirectional lstm, an aspect enhancement module, and the aspect-specific transformation (ast) units. specifically, bet learns the context dependency of sentence by a bidirectional lstm first. then, the aspect enhancement module utilizes the extracted context features to improve the aspect representation. after that, the ast units fuse the aspect information into each context word to obtain a sentence-level context representation. contexts and aspects passing through multiple bet layers are eventually fed into a sentiment feature extractor. since gcae @xcite adds aspect information when extracting sentiment features, which further strengthens the connection between aspect and context. we replace the feature extractor from cnn to gcae. moreover, in order to help gcae extract sentiment features more accurately, we utilize relative position information to scale the input of gcae. the main contributions of this work are summarized as follows: (1) we propose an aspect enhancement module which utilizes the extracted context features to enhance aspect representation. the enhanced aspect representation is utilized to obtain the more effective context representation. (2) based on the aspect enhancement module, we propose the mutual enhanced transformation network (metnet) for aspect-based sentiment analysis which learns the representations of aspect and context alternately and iteratively. experimental results confirm the effectiveness of metnet, and metnet is outstanding in multi-aspect scenarios. ## related works the traditional methods @xcite for the absa task mainly utilize manually designed features such as sentiment lexicon, n-grams, and dependency information, which is labor-intensive. also, the quality of the features directly affects the classification accuracy of methods. with the development of deep learning on natural language processing tasks, many neural network models @xcite @xcite @xcite have been proposed. these methods automatically learn sentiment features of sentence and achieve good results. @xcite adopted two lstms to model the two clauses of given sentence and then integrated the coding features from the two lstms for prediction. recently, because attention mechanism can clearly capture the semantic relationship between given aspect and words in sentence, attention-based neural models have attracted growing interest. @xcite proposed an attention-based lstm network. @xcite adopted an end-to-end memory network architecture where each computational layer was based on the attention mechanism. @xcite utilized the attention mechanism to generate different aspect representations based on individual context words and then extracted context features based on tailor-made aspect representations. existing methods pay little attention to how to improve the representation learning of the aspect. inspired by tnet @xcite and coattention-memnet @xcite , we propose the mutual enhanced transformation network (metnet) for aspect-based sentiment analysis. the main differences between our method and existing methods are as follows: (1) metnet improves the representation learning of the aspect. (2) we propose the bidirectional enhancement transformation (bet) component which can be repeated by the hierarchical structure to obtain more effective representations of aspect and context. ## model overview in this section, we describe our proposed metnet model which is shown in figure 1 . the metnet model can be roughly divided into three parts, namely the bert layer, the bidirectional enhancement transformation (bet), and the convolutional feature extractor. ## conclusion in this paper, we present an end-to-end solution, a mutual enhanced transformation network (metnet), to solve the issue of insufficient aspect representation learning in absa task. first, we propose bidirectional enhancement transformation (bet) component to improve the representation learning of aspect, which meanwhile achieves alternative learning of aspect and context. secondly, metnet uses a hierarchical structure to iteratively learn aspect and context to obtain better representations. experimental results demonstrate the effectiveness of metnet for aspect-based sentiment analysis. especially, met-net performs well in both single-aspect scenarios and multi-aspect scenarios. as future work, we will consider how to use sentiment lexicon and attention mechanism to further improve aspect representation learning.