Graph Attention Network with Memory Fusion for Aspect-level Sentiment Analysis (2020)

Abstract: Aspect-level sentiment analysis(ASC) predicts each specific aspect term’s sentiment polarity in a given text or review. Recent studies used attention-based methods that can effectively improve the performance of aspect-level sentiment analysis. These methods ignored the syntactic relationship between the aspect and its corresponding context words, leading the model to focus on syntactically unrelated words mistakenly. One proposed solution, the graph convolutional network (GCN), cannot completely avoid the problem. While it does incorporate useful information about syntax, it assigns equal weight to all the edges between connected words. It may still incorrectly associate unrelated words to the target aspect through the iterations of graph convolutional propagation. In this study, a graph attention network with memory fusion is proposed to extend GCN’s idea by assigning different weights to edges. Syntactic constraints can be imposed to block the graph convolutional propagation of unrelated words. A convolutional layer and a memory fusion were applied to learn and exploit multiword relations and draw different weights of words to improve performance further. Experimental results on five datasets show that the proposed method yields better performance than existing methods.

## introduction aspect-level sentiment classification is a finegrained subtask in sentiment analysis @xcite . given a sentence and an aspect that appears in the sentence, asc aims to determine the sentiment polarity of that aspect (e.g., negative, neutral, or positive). for example, a review of a restaurant "the price is reasonable although the service is poor." expresses a positive sentiment for the price aspect, but also conveys a negative sentiment for the service aspect, as shown in figure 1 . such a technique is widely used to analyze online posts reviews, mainly from amazon reviews or twitter, to help raise the ability to understand consumer needs or experiences with a product, guiding a manufacturer towards product improvement. aspect-level sentiment classification is much more complicated than sentence-level sentiment classification. asc task is necessary to identify the parts of the sentence that describe the correspondence between multiple aspects. traditional methods mostly use shallow machine learning models with hand-crafted features to build sentiment classifiers for the asc task @xcite .however, the process for manual feature engineering is time-consuming and labor-intensive as well as limited in classification performance recently, with the development of deep learning techniques, various attention-based neural models have achieved remarkable success in asc. @xcite @xcite @xcite . however, these methods ignored the syntactic dependence between context words and aspects in a sentence. as a result, the current attention model may inappropriately focus on syntactically unrelated context words. as shown in figure 1 , when predicting the emotional polarity of price, the attention mechanism may focus on the word poor, which is not related to its syntax. to address this issue, @xcite built a graph convolutional network (gcn) over a dependency tree to exploit syntactical information and word dependencies. however, the model assigns equal weight to the edges connected between words so that words may mistakenly associate syntactically unrelated words to the target aspect through iterations of graph convolutional propagation. as indicated in figure 1 , after three iterations, both reasonable (yellow lines) and poor (red lines) may be the price is resonable although the service is poor identified as descriptors of the aspect price, which is incorrect. as a result, the model will falsely classify the aspect price as a negative sentiment. in this paper, a graph attention model with memory fusion was proposed. this model extends the idea of graph convolutional networks in two aspects. first, the graph attention mechanism is applied to assign different weights to the edge, so the syntactical constraints can be imposed to block the propagation of syntactically unrelated words to the target aspect. second, a convolutional operation is applied to extract local information to exploit multiword relations, such as not good and far from perfect, which can further improve the performance. to integrate all features, a memory fusion layer, which is similar to a memory network, is applied to draw different weights for words according to their contribution to the final classification. experiments are conducted on five datasets demonstrate how the proposed model outperforms baselines for aspect-level sentiment analysis. the remainder of this paper is organized as follows. section 2 briefly reviews the existing works for aspect-level sentiment analysis. section 3 presents a detailed description of the proposed graph attention model with memory fusion. section 4 summarizes the implementation details and experimental results. the conclusions of this study are finally drawn in section 5. ## related works aspect-level sentiment classification is an important branch of sentiment classification, aiming to identify the sentiment polarity of an aspect target in a sentence. asc methods can be divided into traditional and deep learning methods. traditional methods usually used feature-based machine learning algorithms, such as a feature-based support vector machine (svm) @xcite . due to the inefficiency of manually constructed features, several neural network methods have been proposed for aspect-level sentiment analysis @xcite , which are mainly based on long shortterm memory (lstm) @xcite . @xcite indicated that the asc task's challenge is to identify better the semantic correlation between context words and aspect words so that several recent works widely applied an attention mechanism and achieved good performance. @xcite used an interactive attention network to obtain a two-way attention representation of context words and aspect words. @xcite proposed a joint model based on an attention mechanism to model aspects and sentences. @xcite proposed a selfsupervised attention model that can dynamically update attention weights. @xcite introduced the graph convolutional network into the sentiment classification task and achieved good performance. subsequently, @xcite proposed to use gcn on the dependency tree of a sentence to exploit the longrange syntactic information for the asc task. ## graph attention network with memory fusion the proposed graph attention network with memory fusion is mainly composed of the following four parts: a context encoder, a graph attention layer, a convolutional layer and a memory fusion layer, as shown in figure is trained on the dependency tree to mine explicit structural information between words. the convolutional layer was used to extract the local information around the sentiment word, which can dynamically deal with non-single word aspects such as not good and far from perfect, instead of only taking the average of its vectors. to merge all features, we adopt a memory fusion layer similar to a memory network @xcite , which can assign different weights to the context words according to their contribution to the final classification. the detailed description is presented as follows. ## experimental results this section conducts comparative experiments on five corpora against several previously proposed methods for aspect-level sentiment analysis. the experimental setting and empirical results are then presented in detail. ## conclusions in this study, a graph attention network with memory fusion is proposed for aspect-level sentiment analysis. a graph attention layer was implemented to capture a context word's syntactic relationship to the target aspect by learning different weights for edges to block the propagation from unrelated words. moreover, a convolutional layer and a memory fusion were used to learn the local information and draw different weights for context words. experimental results show that the g-att model yields better performance than the existing methods for aspect-based sentiment analysis. besides, ablation studies and case studies are provided to prove the effectiveness of the proposed model further. future works will improve the graph attention layer and dynamic to learn the attention score, so the proposed model can better integrate syntax-related context information.