Aspect and Opinion Term Extraction Using Graph Attention Network (2023)

Abstract: In this work we investigate the capability of Graph Attention Network for extracting aspect and opinion terms. Aspect and opinion term extraction is posed as a token-level classification task akin to named entity recognition. We use the dependency tree of the input query as additional feature in a Graph Attention Network along with the token and part-of-speech features. We show that the dependency structure is a powerful feature that in the presence of a CRF layer substantially improves the performance and generates the best result on the commonly used datasets from SemEval 2014, 2015 and 2016. We experiment with additional layers like BiLSTM and Transformer in addition to the CRF layer. We also show that our approach works well in the presence of multiple aspects or sentiments in the same query and it is not necessary to modify the dependency tree based on a single aspect as was the original application for sentiment classification.

## introduction extracting information from customer feedback is a key capability required for identifying current drawbacks and scope for further improvement. online shoppers routinely provide feedback on their experience with the purchased product that are not just important for the other potential customers but also a critical feedback to the product manufacturers for the next cycle of iteration. similar feedbacks are available in various other domains ranging from manufacturing to healthcare where granular opinions (sentiments) about various dimensions (aspects) of the used product (or service) are available in textual form but need to be understood. due to the presence of multiple aspects (and the corresponding sentiments) extraction of these aspect-sentiment pairs is a challenging task and since its introduction in 2014 @xcite aspect based sentiment analysis (absa) attracted various different approaches and is still under active consideration. absa demands semantic understanding of the sentence where it is necessary to identify the aspect terms (defining "what") and the opinion terms (defining "why") and the connection between each related pairs (resulting in a positive, neutral or negative sentiment, i.e., "how"). a relatively simple example would be "the price is reasonable although the service is poor" where "price" and "service" are the aspects with the corresponding opinion terms "reasonable" and "poor", respectively. here, the relative locations of the aspect and opinion word can help each other to identify these terms. however, other examples like "prices are in line" (neutral sentiment) and "for the price, you can not eat this well in manhattan" (positive sentiment) it is not obvious which opinion terms are driving the sentiments and their linkage with the aspect terms. as a result, while it is possible to capture the syntactical structure and dependency between words it is necessary to capture a deeper meaning of each token (word) that should not be relied upon the limited number of examples (that are hallmark of all absa datasets) but rich representations provided by large language models like bert and roberta. in addition to large model based initial representations, additional information from parts-ofspeech and dependency structure play a critical role as can be seen in some of the previous absa work where the objective is to predict only the sentiment associated with a particular aspect. while it is generally understood that a confluence of deep encoders and graph-based representation of the input sentence will drive better performance it is not clear what is the optimal graph representation of a sentence, especially for absa kind of task. in case of polarity detection, it can be argued that only the part of the dependency tree associated with a particular aspect is significant and the rest of the tree can be ignored. however, when we convert @xcite the task into aspect and opinion term extraction it is not clear whether any part of the sentence has higher significance compared to the rest. in this work we address the problem of aspect and opinion term extraction from input sentences. as an example, "the weather was gloomy, but the food was tasty" has two sets of aspect-opinionsentiment tuples, i.e., (weather, gloomy, negative) and (food, tasty, positive). we use absa datasets with token level labels -one set for the aspects and one set for the opinions. thus, the aspect and opinion term extraction becomes a ner task with different types of entity classes. the tags for the aspects include the sentiment, (for opinion it is only the bieos tag), however, what is missing is the connection between the aspects and opinion terms. thus, our aspect models predict (weather, negative) and (food, positive), opinion models predict (gloomy) and (tasty), without making the subsequent contractions. we encode the graph associated with the dependency parsing of the input sentence and create token (node) level representations that take care of both the neighborhood (connected edges) and dependency type (edge type) of each token. the organization of the paper is as follows. in the next section we provide a detailed literature survey on the techniques employed for aspect and opinion extraction task of absa. next, we present the details of the proposed model. subsequently, the model predictions and comparisons with other baseline methods are discussed. finally, conclusions are drawn and scope for future works is outlined. ## related work most of the absa work concern with the sentiment polarity detection associated with a particular aspect and the approaches varied from initial svm classifier with handcrafted features to deep learning classifiers based on rnn @xcite @xcite , transformer @xcite @xcite and memory network @xcite @xcite . there are few graph based approaches as well, e.g., graph convolution network (gcn) based model of @xcite and @xcite , graph attention network (gat) based model of @xcite and @xcite where the latter modified the original dependency tree to create an aspect-oriented dependency tree that was used further in a relational gat (r-gat) where different relations contributed differently in the computation of nodal representations. however, the approach assumes the presence of only one aspect at a time in the given input. on the other hand, an end-to-end absa tries to extract all the aspect terms in a given query simultaneously along with the corresponding sentiments. the first approach towards e2e-absa was provided by @xcite where a unified tagging scheme (combining the position and sentiment) was used, i.e., the token labels were one of b-{pos,neg,neu}, i-{pos,neg,neu}, e-{pos,neg,neu}, s-{pos,neg,neu} or o, denoting the beginning, inside or end of an aspect, singleword aspect, with positive, negative or neutral sentiment respectively, and finally the outside of that aspect. in this work, bert @xcite was used to embed the tokens and other layers like, gru, transformer @xcite or crf was used on the bert output. li et al. (2019c) ignored the opinion term extraction and that was addressed by @xcite who proposed a two-stage pipeline framework. in the first stage, aspect-sentiment pairs were extracted using the above mentioned unified tagging scheme. in addition, opinion spans were extracted using bieos tagging scheme. the aspectsentiment and opinion pairs thus extracted were matched against each other in the second stage where an mlp-based classifier was used to find the compatibility of these pairs. zhang et al. (2020) proposed a multi-task framework to jointly detect aspects, opinions, and sentiment dependen-cies. however, instead of using the unified tagging scheme they used two sets of bios tags to identify the aspects and sentiments before connecting them with the corresponding sentiment. while all these approaches extract aspects and opinion pairs in isolation a recent approach based on pointer network is proposed by @xcite where an encoder-decoder architecture is used to generate all the aspect-opinion-sentiment tuples. ## methodology we have modified the r-gat approach @xcite which was originally targeted for polarity prediction. the approach utilizes the dependency structure of the input sentence which captures the grammatical structure by connecting the words with the corresponding dependency type. the limitations of common approaches that do not pay attention to the parts-of-speech and the dependency structure are discussed by @xcite and the importance of the syntactic relations are emphasized, especially in the context of the aspect word. to bring out the relations of different words with the aspect word(s) an aspect oriented dependency tree (aodt) was proposed by @xcite where the root of the original dependency tree was shifted to the target aspect word followed by pruning some of the unnecessary relations. however, this approach is not exactly applicable for the current study since we do not know the aspect words (or the opinion words for that matter), a priori. however, knowing that aspect words are nouns and opinion words are generally adjectives, we have randomly chosen one of the nouns present in the sentence as a surrogate aspect word (or adjectives for opinions) and modified the dependency tree based on this word. ## conclusion in this work we have applied relational graph attention network which was previously used for classifying sentiment polarity associated with a specific aspect. however, here we show that the dependency structure of the input query when encoded by a graph attention network is powerful enough to improve the performance and it is not necessary to tie to a particular aspect. we have used a surrogate aspect/opinion term by selecting a noun/adjective token if available. the strength of the approach is evident in the superior results that we have obtained for both aspect and opinion term extraction on four commonly used datasets. in addition, we have compared bilstm and transformer as additional layers along with a crf layer and found that bilstm layer consistently performs better than a transformer layer.