Convolution over Hierarchical Syntactic and Lexical Graphs for Aspect Level Sentiment Analysis (2020)

Abstract: The state-of-the-art methods in aspect-level sentiment classification have leveraged the graph based models to incorporate the syntactic structure of a sentence. While being effective, these methods ignore the corpus level word co-occurrence information, which reflect the collocations in linguistics like “nothing special”. Moreover, they do not distinguish the different types of syntactic dependency, e.g., a nominal subject relation “food-was” is treated equally as an adjectival complement relation “was-okay” in “food was okay”. To tackle the above two limitations, we propose a novel architecture which convolutes over hierarchical syntactic and lexical graphs. Specifically, we employ a global lexical graph to encode the corpus level word co-occurrence information. Moreover, we build a concept hierarchy on both the syntactic and lexical graphs for differentiating various types of dependency relations or lexical word pairs. Finally, we design a bi-level interactive graph convolution network to fully exploit these two graphs. Extensive experiments on five bench- mark datasets show that our method outperforms the state-of-the-art baselines.

## introduction aspect-level sentiment classification (asc) @xcite aims to determine the sentiment polarity (i.e., positive, negative, neutral) of the aspect(s) in a sentence. take the review "great food but the service was dreadful" as an example. given two aspect terms "food" and "service", the goal is to infer the sentiment polarities for the aspect terms: positive for food and negative for service. asc can provide fine-grained analysis of the users' opinion towards the specific aspect and is fundamental to *corresponding author. many natural language processing tasks. consequently, it has aroused much research attention in recent years. early studies on asc @xcite mostly use machine learning algorithms to build sentiment classifier. later, various neural network models @xcite @xcite are proposed for this task, including long short-term memory (lstm) based @xcite , convolutional neural networks (cnn) based @xcite , and memory based @xcite or hybrid methods @xcite . these models represent the sentence as a word sequence and neglect the syntactic relations between words, and thus it is hard for them to find the opinion words which are far away from the aspect term. to solve this problem, several recent researches @xcite @xcite leverage the graph based models to incorporate the syntactic structure of a sentence, and have shown better performance than those without considering syntactic relations. despite of their effectiveness, the seminal syntax based methods ignore the corpus level word co-occurrence information. moreover, they do not distinguish the different types of syntactic dependency. we argue that both will incur information loss. (1) the frequently co-occurred words represent the collocations in linguistics. for example, in the sentence "food was okay, nothing special", the word pair "nothing special" occurs five times in the semeval training set, denoting a negative polarity. without such global information to counteract the positive polarity of "okay", syntax based methods will make wrong prediction on "food". (2) each type of syntactic dependency denotes a specific relation. for example, in "i like hamburgers", "i-like" is a nsubj relation, and "like-hamburgers" is a dobj relation. if the nsubj relation and dobjs relation are treated equally, we are unable to differentiate the subject and the object of the action "like". to tackle the above limitations, we propose a novel architecture which convolutes over hierarchical syntactic and lexical graphs. we first employ a global lexical graph to encode the corpus level word co-occurrence information, where nodes are words and the edge denotes the frequency between two word nodes in the training corpus. we then build a concept hierarchy on each of the syntactic and lexical graphs to distinguish different types of dependency relations or word co-occurrence relations. for example, the acomp relation "wasnothing" and the amod relation "nothing-special" are grouped into an adjective relation type, while the nsubj relation "food-was" will form another noun relation type. for illustration, we show in figure 1 a sample sentence with its dependency tree and the corresponding lexical and syntactic graphs in our and other works @xcite @xcite . in addition, we construct a lexical graph (fig. 1 (d) ) which also has a concept hierarchy to capture the various word co-occurrence relations. finally, in order to let the syntactic and lexical graphs cooperate with each other, we design a bi-level interactive graph convolution network to fully exploit these two graphs. we conduct extensive experiments on five se-meval datasets. results demonstrate that our model achieves the state-of-the-art performance. ## related work recent advances in aspect-level sentiment classification (asc) focus on developing various types of deep learning models. we briefly review the neural models without considering syntax, and then go to the syntax based ones. the neural models without considering syntax models can be mainly categorized into several types: lstm based @xcite @xcite , cnn based @xcite , memory based @xcite , and other hybrid methods @xcite . for example, @xcite use the gated neural network structures to model the interaction between the surrounding contexts and the target. @xcite employ a cnn instead of attention to extract important features from the transformed word representations. @xcite combine the cnn and gating structure to extract aspectspecific information from contexts. the syntactic information enables dependency information to be preserved in lengthy sentences, and helps shorten the distance between aspect and opinion words. there has long been research on incorporating syntactic information in documentlevel sentiment classification @xcite @xcite . later, @xcite ; nguyen and shirai (2015); @xcite ; @xcite also take the syntax structure of a sentence and or pos tags into account for aspect based sentiment analysis. nevertheless, the effect of syntactical structure has not been fully exploited without the proper utilization of the dependencies along the syntactic paths. more recently, several studies @xcite @xcite employ graph based models to integrate the syntactic structure. the basic idea is to transform the dependency tree into a graph, and then impose the graph convolutional networks (gcn) or graph attention networks (gat) to propagate information from syntax neighbourhood opinion words to aspect words. there are also attempts @xcite at exploiting the word co-occurrence information for sentiment analysis. unlike all the aforementioned methods, our model exploits both the syntactic and lexical graphs for capturing the dependency relations in a sentence and the word co-occurrence relations in the train-ing corpus. moreover, we construct the concept hierarchy for each graph, which can group relations with similar uses or meanings together and reduce noises. as we will show in our experiments, the introduction of hierarchy greatly boosts the performance. ## preliminary problem definition (asc) given a review sentence @xmath0, ..., w a+1 , ..., w a+m , ..., w n ] consisting of n words and a corresponding m-length aspect starting from the (a+1) th position, the aspect-level sentiment classification task asc aims at identifying the sentiment polarity of the given aspect(s) in a sentence. hierarchical syntactic graph construction a syntactic graph (sg) has a node set v s and an edge set e s . each node v in v s is a word in the sentence and each edge e in e s between two words denotes that they are syntactically related. existing syntax integrated methods for asc @xcite @xcite do not utilize various types of dependency relations and an edge in their syntactic graph as shown in fig. 1 (b ) simply denotes there exists a dependency relation between two words. as we pointed out in the introduction, each dependency relation represents a specific grammatical function that a word plays in a sentence and should be used in its own manner. however, since there are a good number of relations in the parsed tree, directly using one dependency relation as a type of edge in the graph may incur noises like a parsing error. to solve this problem, we add a syntactic concept hierarchy r s over the dependency relations. specifically, we group 36 dependency relations into 5 relation types, including "noun", "verb", "adverb", "adjective", and "others", denoted as s 1 .. s 5 in r s , respectively. in particular, since most aspect and opinion words are noun and adjective, respectively, they become two main types. verb expresses an action, an event, or a state, and adverb modifies verbs and adjectives, thus they also become two types. all the remaining constitutes the "others" type. we then construct a hierarchical syntactic graph hsg based on the syntactic concept hierarchy. specifically, hsg is denoted as {v s , e s , r s }, where v s , e s , and r s is a node set, an edge set, and a syntactic relation type set, respectively. note that each edge in e s is now attached with a label denoting the dependency relation type in r s . ## proposed model in this section, we present our proposed bigcn model. we first show its architecture in figure 2 . as can be seen from fig. 2 (a) , bigcn takes the global lexical graph and the word sequence as the input to get the initial sentence representation. it then introduces a hiaragg module where the local lexical graph and syntactic graph interact with each other to refine the sentence representation. finally, bigcn obtains the aspect-oriented representation via the mask and gating mechanism for better predicting the sentiment polarity towards a specific aspect in the sentence. ## conclusions in this paper, we propose a novel framework bigcn to leverage the graph based methods for aspect level sentiment classification tasks. besides the ordinary syntactic graph, we employ a lexical graph to capture the global word co-occurrence information in the training corpus. furthermore, we build a concept hierarchy on each of the lexical and syntactic graphs, such that the functionally different types of relations in the graph can be treated separately. finally, we design a hieragg module to let the lexical and syntactic graphs work in a cooperative way. we conduct a set of experiments on five real world datasets. the results prove that our model achieves the state-of-the-art performance.