Dual Graph Convolutional Networks for Aspect-based Sentiment Analysis (2021)

Abstract: Aspect-based sentiment analysis is a fine-grained sentiment classification task. Recently, graph neural networks over dependency trees have been explored to explicitly model connections between aspects and opinion words. However, the improvement is limited due to the inaccuracy of the dependency parsing results and the informal expressions and complexity of online reviews. To overcome these challenges, in this paper, we propose a dual graph convolutional networks (DualGCN) model that considers the complementarity of syntax structures and semantic correlations simultaneously. Particularly, to alleviate dependency parsing errors, we design a SynGCN module with rich syntactic knowledge. To capture semantic correlations, we design a SemGCN module with self-attention mechanism. Furthermore, we propose orthogonal and differential regularizers to capture semantic correlations between words precisely by constraining attention scores in the SemGCN module. The orthogonal regularizer encourages the SemGCN to learn semantically correlated words with less overlap for each word. The differential regularizer encourages the SemGCN to learn semantic features that the SynGCN fails to capture. Experimental results on three public datasets show that our DualGCN model outperforms state-of-the-art methods and verify the effectiveness of our model.

## introduction sentiment analysis has become a popular topic in natural language processing @xcite . aspect-based sentiment analysis (absa) talks an entity-level oriented fine-grained sentiment analysis task that aims to determine sentiment polarities of given aspects in a sentence. in figure 1 : an example sentence with its dependency tree from the restaurant reviews. this sentence contains two aspects but with opposite sentiment polarities. figure 1 , the comment is about a restaurant review. the sentiment polarity of the two aspects "price" and "service" are positive and negative, respectively. thus, absa can precisely identify user's attitudes towards a certain aspect, rather than simply assigning a sentiment polarity for a sentence. the key point in solving the absa task is to model the dependency relationship between an aspect and its corresponding opinion expressions. nevertheless, there probably exist multiple aspects and different opinion expressions in a sentence. to judge the sentiment of a particular aspect, previous studies @xcite @xcite @xcite @xcite have proposed various recurrent neural networks (rnns) with attention mechanisms to generate aspect-specific sentence representations and have achieved appealing results. however, an inherent defect makes the attention mechanism vulnerable to noise in the sentence. take figure 1 as an example; for the aspect "service", the opinion word "reasonable" may receive more attention than the opinion word "poor". however, the "reasonable" refers to another aspect, i.e., "price". more recent efforts @xcite @xcite @xcite @xcite have been de-voted to graph convolutional networks (gcns) and graph attention networks (gats) over dependency trees, which explicitly exploit the syntactic structure of a sentence. consider the dependency tree in figure 1 ; the syntactic dependency can establish connections between the words in a sentence. for example, a dependency relation exists between the aspect "price" and the opinion word "reasonable". however, two challenges arise when applying syntactic dependency knowledge to the absa task: 1) the inaccuracy of the dependency parsing results and 2) gcns over dependency trees do not work well as expected on datasets that are not sensitive to syntactic dependency due to the informal expression and complexity of online reviews. in this paper, we propose a novel architecture, the dual graph convolution network (dualgcn), as shown in figure 2 , to solve the aforementioned challenges. for the first challenge, we use the probability matrix of all dependency arcs from a dependency parser to build a syntax-based graph convolutional network (syngcn). the idea behind this approach is that the probability matrix representing dependencies between words contains rich syntactic information compared with the final discrete output of a dependency parser. for the second, we construct a semantic correlation-based graph convolutional network (semgcn) by utilizing a self-attention mechanism. the idea behind this approach is that the attention matrix shaped by self-attending, also viewed as an edge-weighted directed graph, can represent semantic correlations between words. moreover, motivated by the work of dgedt @xcite , we utilize a biaffine module to bridge relevant information between the syngcn and semgcn modules. furthermore, we design two regularizers to enhance our dualgcn model. we observe that the semantically related terms of each word should not overlap. therefore, we encourage the attention probability distributions over words to be orthogonal. to this end, we incorporate an orthogonal regularizer on the attention probability matrix for the semgcn module. moreover, the two representations learned from the syngcn and semgcn modules should contain significantly distinct information captured by the syntactic dependency and the semantic correlation. therefore, we expect that the semgcn module could learn semantic representations different from syntactic representations. thus, we propose a differential regularizer between the syngcn and semgcn modules. our contributions are highlighted as follows: ## related work traditional sentiment analysis tasks are sentencelevel or document-level oriented. in contrast, absa is an entity-level oriented and a more finegrained task for sentiment analysis. earlier methods @xcite @xcite are usually based on handcrafted features and fail to model the dependency between the given aspect and its context. recently, various attention-based neural networks have been proposed to implicitly model the semantic relation of an aspect and its context to capture the opinion expression component @xcite @xcite @xcite @xcite @xcite . for instance, @xcite proposed attentionbased lstms for aspect-level sentiment classification. @xcite and @xcite both introduced a hierarchical attention network to identify important sentiment information related to the given aspect. @xcite exploited a multi-grained attention mechanism to capture the word-level interaction between aspects and their context. @xcite designed a dual attention network to recognize conflicting opinions. in addition, the pre-trained language model bert @xcite has achieved remarkable performance in many nlp tasks, including absa. @xcite transformed absa task into a sentence pair classification task by constructing an auxiliary sentence. @xcite proposed a post-training approach on the bert to enhance the performance of fine-tuning stage for the absa task. another trend explicitly leverages syntactic knowledge. this type of knowledge helps to establish connections between the aspects and the other words in a sentence to learn syntax-aware feature representations of aspects. @xcite proposed a recursive neural network to adaptively propagate the sentiment of words to the aspect along the dependency tree. @xcite introduced an attention model that incorporated syntactic information to compute attention weights. (phan and ogunbona, 2020) utilized the syntactic relative distance to reduce the impact of irrelevant words. following this line, a few works extend the gcn and gat models by means of a syntactical dependency tree and develop several outstanding models @xcite @xcite @xcite . these works explicitly exploit the syntactic structure information to learn node representations from adjacent nodes. thus, the dependency tree shortens the distance between the aspects and opinion words of a sentence and alleviates the problem of long-range dependency. most recently, several works explore the idea of combining different types of graph for absa task. for instance, @xcite combined a dependency graph and a latent graph to generate the aspect representation. @xcite observed the characteristics of word co-occurrence in linguistics and designed hierarchical syntactic and lexical graphs. @xcite constructed aspect-focused and inter-aspect graphs to learn dependency feature of the key aspect words and sentiment relations between different aspects. in this paper, we propose a gcn based method combining syntactic and semantic features. we use a dependency probability matrix with richer syntactic information and elaborately design orthogonal and differential regularizers to enhance the ability to precisely capture the semantic associations. ## graph convolutional network (gcn) motivated by conventional convolutional neural networks (cnns) and graph embedding, a gcn is an efficient cnn variant that operates directly on graphs @xcite . for graph structured data, a gcn can apply the convolution operation on directly connected nodes to encode local information. through the message passing of multilayer gcns, each node in a graph can learn more global information. given a graph with n nodes, the graph can be represented as an adjacency matrix @xmath0. most previous work @xcite extend gcn models by encoding dependency trees and incorporating dependency paths between words. they build the adjacency matrix a over the syntactical dependency tree of a sentence. thus, an element a ij in a indicates whether the i-th node is connected to the j-th node. specifically, a @xmath1, and a @xmath2. in addition, the adjacency matrix a, composed of 0 and 1, can be deemed as the final discrete output of a dependency parser. for the i-th node at the l-th layer, formally, its hidden state representation, denoted as h l i , is updated by the following equation: ## proposed dualgcn figure 2 provides an overview of dualgcn. in the absa task, a sentence-aspect pair (s, a) is given, where @xmath3, a 2 , ..., a m } is an aspect. it is also a sub-sequence of the entire sentence @xmath4, w 2 , ..., w n }. then, we utilize bilstm or bert as sentence encoder to extract hidden contextual representations, respectively. for the bil-stm encoder, we first obtain the word embeddings @xmath5, x 2 , ..., x n } of the sentence s from an embedding lookup table @xmath6, where |v | is the size of vocabulary and d e denotes the dimensionality of word embeddings. next, the word embeddings of the sentence are fed into a bilstm to produce hidden state vectors @xmath7, h 2 , ..., h n }, where h @xmath8. the dimensionality of a hidden state vector d is output by a unidirectional lstm. for the bert encoder, we construct a sentenceaspect pair "[cls] sentence [sep] aspect [sep]" as input to obtain aspect-aware hidden representations of the sentence. moreover, in order to match the wordpiece-based representations of bert with the result of syntactic dependency based on word, we expand dependencies of a word into its all of subwords. then, the hidden representations of sentence are input into the syngcn and semgcn modules, respectively. a biaffine module is then adopted for effective information flow. finally, we aggregate all the aspect nodes' representations from the syngcn and semgcn modules via pooling and concatenation to form the final aspect representation. next, we elaborate on the details of our proposed dualgcn model. ## conclusion in this paper, we propose a dualgcn architecture to address the disadvantages of attention-based and dependency-based methods for absa tasks. our dualgcn model integrates syntactic knowledge and semantic information by means of the syngcn and semgcn modules. moreover, to effectively capture the semantic correlation between words, we propose orthogonal and differential regularizers in the semgcn module. these regularizers can attend to the semantically related items with less overlap of each word and capture feature representations that differ from the syntactic structure. extensive experiments on benchmark datasets show that our dualgcn model outperforms baselines.