Aspect-based Sentiment Analysis with Type-aware Graph Convolutional Networks and Layer Ensemble (2021)

Abstract: It is popular that neural graph-based models are applied in existing aspect-based sentiment analysis (ABSA) studies for utilizing word relations through dependency parses to facilitate the task with better semantic guidance for analyzing context and aspect words. However, most of these studies only leverage dependency relations without considering their dependency types, and are limited in lacking efficient mechanisms to distinguish the important relations as well as learn from different layers of graph based models. To address such limitations, in this paper, we propose an approach to explicitly utilize dependency types for ABSA with type-aware graph convolutional networks (T-GCN), where attention is used in T-GCN to distinguish different edges (relations) in the graph and attentive layer ensemble is proposed to comprehensively learn from different layers of T-GCN. The validity and effectiveness of our approach are demonstrated in the experimental results, where state-of-the-art performance is achieved on six English benchmark datasets. Further experiments are conducted to analyze the contributions of each component in our approach and illustrate how different layers in T-GCN help ABSA with quantitative and qualitative analysis.

## introduction aspect-based sentiment analysis (absa) processes fine-grained sentiment polarities towards specific aspects, where in many cases, it is required to identify different sentiments for multiple aspects in the same context. for example, in the sentence "the drink menu is limited but the wines are excellent.", the sentiment polarity towards "drink menu" is negative while that towards "wines" is positive; an figure 1 : an example sentence (including the aspect term "drink menu") with its dependency parsing result. absa system may predict wrong if it fails to capture the important contextual information for each aspects. therefore, to model such contextual information, neural models (e.g., bi-lstm and transformer @xcite ) have been widely used for absa and demonstrated to be useful for this task @xcite @xcite @xcite . as a further enhancement of encoding contextual information for absa, there are studies @xcite @xcite using graph convolutional networks (gcn) to learn from a graph that is often built over the dependency parsing results of the input texts. as a result, the gcn models are able to learn from distant wordword relations that are more helpful to absa. however, gcn models used in these studies are limited by omitting the information carried in dependency types and treating all word-word relations in the graph equally, therefore unimportant relations may not be distinguished and mislead absa accordingly. for example, figure 1 illustrates an example sentence with an aspect highlighted in red, where the aspect word "menu" is connected with three others words, i.e., "the", "drink", and "limited". the connection between "menu" and "limited" could be the most important one since its dependency type, i.e., "nsubj", suggests that "menu" is the nominal subject of "limited", which strongly guides sentiment analysis towards "menu". in this case, if the dependency type is not modeled, one may not be able to leverage such beneficial information. in addition, although previous gcn models learn such word-word relations by multiple gcn layers, they only use the output from the last layer for absa, where the encodings from intermediate layers are omitted and some essential information may be lost because different context information are modeled across layers. thus an appropriate approach is required to enhance current gcn models for absa. in this paper, we propose a type-aware graph convolutional networks (t-gcn) with multiple layers to enhance absa by incorporating both word relations and their dependency types to comprehensively learn from dependency parsing results. specifically, we firstly obtain the dependency parsing results of the input texts through off-the-shelf toolkits, then build the graph over the dependency tree with each edge labeled by the corresponding dependency type between the two connected words, later apply an attention mechanism to the graph to weight all edges according to their contributions to the task, and finally use attentive layer ensemble to weight and combine the contextual information learned from different gcn layers. in doing so, our proposed t-gcn model can not only model wordword relations and their dependency types, but also distinguish the important contextual information from such relations to enhance absa. experiments on six english benchmark datasets are conducted to evaluate the proposed model, where the results illustrate its effectiveness and state-of-the-art performance is observed over previous studies on all datasets. we also perform further analysis to in-vestigate the contribution of each component (i.e., type-aware graph, attention for edges, and attentive layer ensemble) in our approach, and illustrate how different layers in t-gcn helps absa with quantitative and qualitative studies. ## the approach given an input sentence @xmath0, x 2 , • • • , x n and the aspect terms a ⇢ x (a is usually a sub-string of x ), the conventional absa approaches often take the sentence-aspect pair as the input and predicts a's sentiment polarity b y @xcite @xcite @xcite @xcite @xcite . we follow this paradigm and the overview of our approach is illustrated in figure 2 , with a contextual encoder (i.e., bert), the proposed t-gcn and the attentive layer ensemble (ale). the overall conceptual formalism of our approach can be written as ## analyses 5.1 ablation study to explore the effectiveness of different components in our model, i.e., type-aware graph (tg), here are some observations. first, it is clearly indicated in results that, the model performance drops on all datasets if any component is excluded from the full model. this observation indicates that all three components play important roles in our approach to enhance absa; each one has its unique contribution to the full model. second, for each single components, compared with the results from gcn baseline (id: 8), the results from models with a particular module (id: 5-7) demonstrate that the attention mechanism is the most important one to improve model performance, where on all datasets, the model (id: 6) with attention outperforms the others. this observation complies with our intuition because the attention directly guides the model to distinguish the contextual information to the aspect words, so that informative words are highlighted so as to improve absa accordingly. ## related work absa is in the line of research on sentiment analysis in a fine-grained level focusing on categoriz-ing sentiment polarities for a specific aspect (e.g., "chicken") or category (e.g., "food") in a sentence. conventionally, this task is formulated as to classify a sentence-aspect pair and most of studies try to explore the contextual information between aspect and the entire sentence to facilitate the analysis of sentiment @xcite @xcite @xcite @xcite @xcite . to further enhancing the modeling of contextual information, dependency parses were leveraged by many studies, where adaptive recursive neural networks @xcite , attention mechanism @xcite , and key-value memory networks @xcite 2020 ) leveraged graph neural models (e.g., gcn) for absa with their graph built upon the dependency tree obtained from off-the-self dependency parsers, and demonstrated promising results. the models in their studies normally focus on building the graph with the dependency structure without considering dependency types, meanwhile treating the edges in the graph equally. in addition, they usually use the output of the last layer to predict sentiment labels although their models consist multiple layers. thus, our approach differs from previous graph-based ones on several aspects, including the integration of depdendency type information, applying attention to edges, and ensemble of multiple layers to comprehensively learn from the graph model. ## conclusion in this paper, we propose a neural approach for absa with t-gcn, where the input graph is built on the dependency tree of the input sentence. specifically, the edges in the graph are constructed on top of both dependency relations and types for the input sentence; for each word, we use attention to weight all such type-aware edges associated to it in the t-gcn; we also apply attentive layer ensemble to comprehensively learn contextual information from different t-gcn layers. experimental results on six widely used english benchmark datasets demonstrate the effectiveness of our approach, where state-of-the-art performance are achieved on all datasets. further analyses illustrate the validity of incorporating type information into our model as well as applying attentive ensemble to learning from its multiple layers.