Syntax-Aware Graph Attention Network for Aspect-Level Sentiment Classification (2020)

Abstract: Aspect-level sentiment classification aims to distinguish the sentiment polarities over aspect terms in a sentence. Existing approaches mostly focus on modeling the relationship between the given aspect words and their contexts with attention, and ignore the use of more elaborate knowledge implicit in the context. In this paper, we exploit syntactic awareness to the model by the graph attention network on the dependency tree structure and external pre-training knowledge by BERT language model, which helps to model the interaction between the context and aspect words better. And the subwords of BERT are integrated into the dependency tree graphs, which can obtain more accurate representations of words by graph attention. Experiments demonstrate the effectiveness of our model.

## introduction aspect-level sentiment analysis (absa) @xcite , as a fundamental natural language processing task, has attracted lots of attentions recently. its typical application is to classify the target aspects of a product in a customer review into the sentiment polarities of positive, neutral, or negative according to the contexts @xcite . taking the sentence "the performance of this laptop is excellent, but the screen is terrible" as example, the correct absa results would be to judge the aspect word performance as positive, and screen as negative. the early approaches of aspect-level sentiment analysis mainly use handcrafted features to train a statistical classifier @xcite . with the rapid progress of deep neural network (dnn), lots of work based on dnn have been proposed @xcite @xcite . these studies mostly depend on recurrent neural networks (rnns) @xcite to model the semantic relationship between the aspect words and their contexts. however, their performances are affected by the rnn's limited ability of capturing long-distance dependencies @xcite ) and loss of syntax information @xcite . to overcome the shortages of rnn-based models mentioned above, there have been a lot of work using various methods like bert language model and graph neural networks recently. since bert language model @xcite based on transformer can alleviate long-distance dependence of rnn, some studies have designed bert-based models for the absa task and achieved outstanding results @xcite @xcite . to the best of our knowledge, most of these work just utilize bert simply as an embedding layer, which sequentially models the position embedddings of the context and cannot utilize the syntactic information. another improvement is the use of gnn in absa @xcite , since gnn has the capability to model linguistic structures like dependency trees other than only sequential data. with such a capability, the model can build the connection of some non-adjacent keywords (e.g., desk and like in figure 1 ) and improve their interactions. however, this kind of research does not go far in exploiting how to model the graph elaborately. for example, @xcite treat all the child nodes of a parent node equally. as they the desk ##s in their dorm ##itor ##ies like figure 1 : an example of the graph in our model. the complete sentence is they like the desks in their dormitories. the aspect word is desk (colored red), and the keyword which gives out sentiment information is like (colored yellow). the distance between the two words is 1 in the dependency tree shorter than 2 in sequence. and the meaning of desk is more important than they for the word like (colored yellow). through the survey above, we hope to take a further step in solving the long-distance dependence problem and introducing syntactic information for the absa task. in this paper, we propose a novel model named syntax-aware graph attention network (sagat) which fully exploits the potentials of bert and carefully models the syntactic relations between different words. specifically, sagat first obtain the representation of each subword in the sentence from the bert. unlike other bert-based models @xcite @xcite which merge the representation of subwords mostly by mean pooling to form word-level embeddings, we retain all the representations of subwords and use them in the subsequent dependency tree graph to obtain more accurate representations of words by graph attention performed later. following @xcite , we also build a graph for each sentence according to its dependency tree. in this kind of graph, we state above that a common parent word may treat its different child words differently as in figure 1 . to tackle this difference, we design the graph attention network to pay different attentions to different children. finally, we use the fusion and alignment layers to interact context and aspects, respectively. the fusion layer performs max-pooling on context and aspects to get the most significant features first, then the hidden states of each token in both context and aspects are updated by those captured features. the alignment layer utilizes attention mechanism to capture the most relevant features, and these features are also used for updating the hidden states of tokens. after all operations above, the representation of context and aspects concentrated to pass through a linear layer to get the final output. experiments demonstrate the effectiveness of our model. the main contributions of this paper are presented as follows: in this section, we will briefly review works on aspect-level sentiment analysis, graph neural network, and bert language model. ## methodology in this section, we will introduce the details of all the layers in our model separately. figure 2 shows the overall architecture of the proposed syntax-aware graph attention network (sagat), which consists of an encoding layer, graph attention layer, fusion layer, alignment layer, and output layer. ## experiments in this section, we describe our experimental setup and report our experimental results. ## conclusion in this paper, we proposed a model named sagat for aspect-level sentiment classification. to fully obtain both syntax and semantic information in the sequence, we utilize graph attention network and bert in our model. experiments on three datasets demonstrate the effectiveness of our model. we also perform some additional experiments to show the power of components in sagat.