Relational Graph Attention Network for Aspect-based Sentiment Analysis (2020)

Abstract: Aspect-based sentiment analysis aims to determine the sentiment polarity towards a specific aspect in online reviews. Most recent efforts adopt attention-based neural network models to implicitly connect aspects with opinion words. However, due to the complexity of language and the existence of multiple aspects in a single sentence, these models often confuse the connections. In this paper, we address this problem by means of effective encoding of syntax information. Firstly, we define a unified aspect-oriented dependency tree structure rooted at a target aspect by reshaping and pruning an ordinary dependency parse tree. Then, we propose a relational graph attention network (R-GAT) to encode the new tree structure for sentiment prediction. Extensive experiments are conducted on the SemEval 2014 and Twitter datasets, and the experimental results confirm that the connections between aspects and opinion words can be better established with our approach, and the performance of the graph attention network (GAT) is significantly improved as a consequence.

## introduction aspect-based sentiment analysis (absa) aims at fine-grained sentiment analysis of online affective texts such as product reviews. specifically, its objective is to determine the sentiment polarities towards one or more aspects appearing in a single sentence. an example of this task is, given a review great food but the service was dreadful, to determine the polarities towards the aspects food and service. since the two aspects express quite opposite sentiments, just assigning a sentence-level sentiment polarity is inappropriate. in this regard, absa can provide better insights into user reviews compared with sentence-level sentiment analysis. intuitively, connecting aspects with their respective opinion words lies at the heart of this task. most recent efforts @xcite @xcite resort to assorted attention mechanisms to achieve this goal and have reported appealing results. however, due to the complexity of language morphology and syntax, these mechanisms fail occasionally. we illustrate this problem with a real review so delicious was the noodles but terrible vegetables, in which the opinion word terrible is closer to the aspect noodles than delicious, and there could be terrible noodles appearing in some other reviews which makes these two words closely associated. therefore, the attention mechanisms could attend to terrible with a high weight when evaluating the aspect noodles. some other efforts explicitly leverage the syntactic structure of a sentence to establish the connections. among them, early attempts rely on handcrafted syntactic rules @xcite , though they are subject to the quantity and quality of the rules. dependency-based parse trees are then used to provide more comprehensive syntactic information. for this purpose, a whole dependency tree can be encoded from leaves to root by a recursive neural network (rnn) @xcite @xcite , or the internal node distance can be computed and used for attention weight decay @xcite . recently, graph neural networks (gnns) are explored to learn representations from the dependency trees @xcite @xcite . the shortcomings of these approaches should not be overlooked. first, the dependency relations, which may indicate the connections between aspects and opinion words, are ignored. second, empirically, only a small part of the parse tree is related to this task and it is unnecessary to encode the whole tree @xcite . finally, the encoding process is tree-dependent, making the batch operation inconvenient during optimization. in this paper, we re-examine the syntax information and claim that revealing task-related syntactic structures is the key to address the above issues. we propose a novel aspect-oriented dependency tree structure constructed in three steps. firstly, we obtain the dependency tree of a sentence using an ordinary parser. secondly, we reshape the dependency tree to root it at a target aspect in question. lastly, pruning of the tree is performed to retain only edges with direct dependency relations with the aspect. such a unified tree structure not only enables us to focus on the connections between aspects and potential opinion words but also facilitates both batch and parallel operations. then we propose a relational graph attention network (r-gat) model to encode the new dependency trees. r-gat generalizes graph attention network (gat) to encode graphs with labeled edges. extensive evaluations are conducted on the semeval 2014 and twitter datasets, and experimental results show that r-gat significantly improves the performance of gat. it also achieves superior performance to the baseline methods. the contributions of this work include: ## related work most recent research work on aspect-based sentiment analysis (absa) utilizes attention-based neural models to examine words surrounding a target aspect. they can be considered an implicit approach to exploiting sentence structure, since opinion words usually appear not far from aspects. such approaches have led to promising progress. among them, @xcite proposed to use an attention-based lstm to identify important sentiment information relating to a target aspect. some other efforts try to directly include the syntactic information in absa. since aspects are generally assumed to lie at the heart of this task, establishing the syntactic connections between each target aspect and the other words are crucial. @xcite manually defined some syntactic rules to identify the relations between aspects and potential opinion words. @xcite obtained partial alignment links with these syntactic rules and proposed a partially supervised word alignment model to extract opinion targets. afterward, neural network models were explored for this task. @xcite used a recursive neural network (rnn) to hierarchically encode word representations and to jointly extract aspects and sentiments. in another work, @xcite combined the recursive neural network with conditional random fields (crf). moreover, @xcite proposed an adaptive recursive neural network (adarnn) to adaptively propagate the sentiments of words to the target aspect via semantic composition over a dependency tree. @xcite further combined the dependency and constituent trees of a sentence with a phrase recursive neural network (phrasernn). in a simpler approach, @xcite used the relative distance in a dependency tree for attention weight decay. they also showed that selectively focusing on a small subset of context words can lead to satisfactory results. recently, graph neural networks combined with dependency trees have shown appealing effectiveness in absa. @xcite and @xcite proposed to use graph convolutional networks (gcn) to learn node representations from a dependency tree and used them together with the [falafel] neg was over cooked and dried but the [chicken] pos was fine. 0.00 0.00 0.00 0.01 0.00 0.00 0.27 0.71 0.01 0.00 0.00 0.00 other features for sentiment classification. for a similar purpose, @xcite used graph attention networks (gat) to explicitly establish the dependency relationships between words. however, these approaches generally ignore the dependency relations which might identify the connections between aspects and opinion words. ## aspect-oriented dependency tree in this section, we elaborate on the details of constructing an aspect-oriented dependency tree. ## relational graph attention network to encode the new dependency trees for sentiment analysis, we propose a relational graph attention network (r-gat) by extending the graph attention network (gat) @xcite to encode graphs with labeled edges. ## experiments in this section, we first introduce the datasets used for evaluation and the baseline methods employed for comparison. then, we report the experimental results conducted from different perspectives. finally, error analysis and discussion are conducted with a few representative examples. ## conclusion in this paper, we have proposed an effective approach to encoding comprehensive syntax information for aspect-based sentiment analysis. we first defined a novel aspect-oriented dependency tree structure by reshaping and pruning an ordinary dependency parse tree to root it at a target aspect. we then demonstrated how to encode the new dependency trees with our relational graph attention network (r-gat) for sentiment classification. experimental results on three public datasets showed that the connections between aspects and opinion words can be better established with r-gat, and the performance of gat and bert are significantly improved as a result. we also conducted an ablation study to validate the role of the new tree structure and the relational heads. finally, an error analysis was performed on incorrectly-predicted examples, leading to some insights into this task.