Joint Aspect Extraction and Sentiment Analysis with Directional Graph Convolutional Networks (2020)

Abstract: End-to-end aspect-based sentiment analysis (EASA) consists of two sub-tasks: the first extracts the aspect terms in a sentence and the second predicts the sentiment polarities for such terms. For EASA, compared to pipeline and multi-task approaches, joint aspect extraction and sentiment analysis provides a one-step solution to predict both aspect terms and their sentiment polarities through a single decoding process, which avoid the mismatches in between the results of aspect terms and sentiment polarities, as well as error propagation. Previous studies, especially recent ones, for this task focus on using powerful encoders (e.g., Bi-LSTM and BERT) to model contextual information from the input, with limited efforts paid to using advanced neural architectures (such as attentions and graph convolutional networks) or leveraging extra knowledge (such as syntactic information). To extend such efforts, in this paper, we propose directional graph convolutional networks (D-GCN) to jointly perform aspect extraction and sentiment analysis with encoding syntactic information, where dependency among words are integrated in our model to enhance its ability of representing input sentences and help EASA accordingly. Experimental results on three benchmark datasets demonstrate the effectiveness of our approach, where D-GCN achieves state-of-the-art performance on all datasets.

## introduction end-to-end aspect-based sentiment analysis (easa) aims to extract aspect terms in the text and predict their sentiment polarities so as to understand targeted sentiment towards particular objects. for example, in the sentence "the ambiance is minimal but food is not phenomenal", the aspect terms are "ambiance" and "food" and the sentiment polarities towards them are positive and negative, respectively. in general, there are mainly three types of approaches for this task, i.e., pipeline, multi-task, and joint-label approaches. pipeline approaches @xcite @xcite perform aspect extraction and sentiment analysis in a sequence, which is not straightforward and suffers from error propagation among different steps; multi-task approaches @xcite @xcite @xcite apply an encoder to the input and use a separate decoding process to extract aspects and predict their sentiments, where there could be mismatches between the two decoding results. as a comparison, joint-label approaches @xcite @xcite @xcite extract aspect terms and predict their sentiments simultaneously through a unified labeling scheme, which not only provides an one-step solution to easa, but also avoids the aforementioned problems in the other two approaches. in most cases, previous studies demonstrate that a good modeling of contextual information is effective in improving easa performance. however, these studies mainly rely on powerful encoders (e.g., bi-lstm, cnn, bert) @xcite @xcite @xcite @xcite and pre-trained embedings (e.g., glove, word2vec, fasttext) @xcite to learn contextual information, with limited effort paid to leveraging advanced architectures and extra knowledge for this task. to extend such effort, graph convolutional networks (gcn) was proposed and shows its effectiveness in conventional sentiment analysis @xcite , as well as other tasks, e.g., text classification @xcite , neural machine translation @xcite , semantic role labeling @xcite , etc. moreover, consider that discriminatively modeling the contextual features of a given word according to their positional relations to the word is helpful in text representation learning figure 1 : the overall architecture of our approach, where the graph is built upon the dependency tree of an input sentence, with all edges in the graph illustrated in the adjacency matrix. the red, blue, and orange colors illustrate our modeling for contextual features in left, right, and self positional relations with a specific word, respectively. @xcite @xcite , any encoder for easa could also be beneficial from adding such treatment to model the input text. therefore, it is expected to enhance conventional gcn with directional information for different parts of input, so that one can distinguish them and appropriately model the contextual information for easa. in this paper, we propose directional graph convolutional networks (d-gcn) for easa, which performs the task following the sequence labeling paradigm and models dependency relations among words in the input with an appropriate architecture. specifically, for an input sentence, we firstly build the word relation graph upon its auto-processed dependency trees; then, we apply a direction mechanism in gcn, where for each word, we separately encode its associated contextual features (which are suggested by the graph) with respect to different positional relations (i.e., on the left, right or self). to further distinguish the importance of different contextual features, we also propose an attention mechanism, in which we assign different weights to such features that are computed according to the comparisons among them, so as to emphasize important ones for easa. to illustrate the effectiveness of our approach, experiments are performed on three benchmark datasets, where the results confirm that d-gcn is an appropriate model in leveraging dependency-based word relations for easa, with state-of-theart performance observed on all datasets. ## the approach the overall architecture of our approach is illustrated in figure 1 , which follows the sequence labeling paradigm for easa, where an input sentence for the d-gcn, there are l layers placed in between the context encoder (i.e., bert) and the output layer, where to feed them, an adjacency matrix (shown at the lower right side of figure 1 ) representing the graph is built on the dependency tree of the input sentence and an attention matrix (shown at the upper right side of figure 1 ) is applied to the edges in the graph to weight the contextual features associated with a specific word, i.e., "soup". in the following text, we firstly introduce normal gcn, then elaborate our proposed d-gcn, and finally illustrate easa labeling with d-gcn. ## conclusion in this paper, we propose a joint approach for easa with d-gcn, whose graph is built upon the dependency tree of the input sentence obtained from off-the-shelf toolkits. the novelty of this work lies in the direction modeling and attention applied in gcn, where in each d-gcn layer, for each word, we separately model its different contextual features by considering their direction to the word, and weight these features according to the comparisons among them. experimental results on three widely used benchmark datasets illustrate the effectiveness of our approach, with the state-of-the-art performance achieved on all datasets. further analysis confirms that both direction modeling and attention mechanism are helpful for the task.