Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis (2020)

Abstract: The aspect-based sentiment analysis (ABSA) consists of two conceptual tasks, namely an aspect extraction and an aspect sentiment classification. Rather than considering the tasks separately, we build an end-to-end ABSA solution. Previous works in ABSA tasks did not fully leverage the importance of syntactical information. Hence, the aspect extraction model often failed to detect the boundaries of multi-word aspect terms. On the other hand, the aspect sentiment classifier was unable to account for the syntactical correlation between aspect terms and the context words. This paper explores the grammatical aspect of the sentence and employs the self-attention mechanism for syntactical learning. We combine part-of-speech embeddings, dependency-based embeddings and contextualized embeddings (e.g. BERT, RoBERTa) to enhance the performance of the aspect extractor. We also propose the syntactic relative distance to de-emphasize the adverse effects of unrelated words, having weak syntactic connection with the aspect terms. This increases the accuracy of the aspect sentiment classifier. Our solutions outperform the state-of-the-art models on SemEval-2014 dataset in both two subtasks.

## introduction the process of understanding the sentiments expressed by consumers in a product review (opinionated text) is referred to as sentiment analysis. deep insights into the opinionated text are gained through a fine-grained entity-or aspect-based sentiment labeling of the product being reviewed. such insights can be invaluable for business decision making. aspect-based sentiment analysis (absa) consists of two sub-tasks, namely an aspect extraction (ae) and an aspect sentiment classification (asc). however, the majority of reported works focused on one of the two sub-tasks alone. representative works include @xcite @xcite for aspect extraction and @xcite @xcite for aspect sentiment classification. recent approaches @xcite @xcite attempted to develop an integrated solution to solve both tasks simultaneously by formulating both sub-tasks as a single sequence labelling with a unified tagging scheme. adding unified tokens introduces overhead and complexity in the original absa tasks. thus, multi-task models often have poorer performance compared with single-task models which are trained independently. recent advances in the nlu introduced contextualized language models, namely openai gpt @xcite , bert @xcite and roberta @xcite . these models can capture the characteristics of word uses and account for different textual context in which words appear. upon investigating the latest bert/robertabased architectures used in aspect extraction, it became apparent that they were unable to determine the boundaries of multi-word aspects. for instance, the extractors broke the multi-word expression,"quality of food" into "quality of" and "food". we hypothesize that this shortcoming is caused by the inability of the contextualized embeddings to encode rich syntactical information. in this paper, we integrate syntactical information into contextualized embeddings and propose an absa solution consisting of an aspect extractor and an aspect sentiment classifier as illustrated by fig. 1 . the proposed ae architecture, named contextualized syntax-based aspect extraction (csae), consists of pos embeddings, dependency-based embeddings @xcite and selfattention in addition to roberta layer. our asc solution is closely related to the work of @xcite in which the local context focus (lcf) mechanism is exploited to down-weight the contribution of words that are far away from local context. however, this approach simply regarded the word counts between two words as their semantic relative distance and neglected the mutual syntactical relationship. our approach employs the shortest path between two words in dependency parsing tree as a syntactic relative distance (srd). we name this model local context focus on syntax -asc (lcfs-asc). comparative experiments are conducted on two semeval-2014 datasets @xcite to demonstrate the importance of syntactical features in improving both ae and asc models. the main contributions of this paper can be highlighted as: ( 1 ## related work this section details the evolution of absa solutions from word-embedding-based models to contextualized-embedding-based models and highlights their strengths and weaknesses. ## proposed method given a contextual sentence s consisting of n tokens, @xmath0, n]}, an end-toend absa tasks aims to extract the set a of m aspect terms being mentioned where @xmath1, m]}; and determine the polarity y @xmath2, n egative, n eutral} associated with each extracted aspect. ## conclusion and future work we proposed an end-to-end absa solution which pipelined an aspect extractor and an aspect sentiment classifier. the results indicate that exploitation of syntactical structures of sentences empowers the contextualized models to improve on current works in both asc and ae tasks. our proposed aspect sentiment classifier outperformed post-training asc model and enabled the creation of a domain-independent solution. the proposed srd allows the aspect sentiment classifier to focus on critical sentiment words which modify the target aspect term through dependency-based structure. the substantial improvements highlight the under-performance of recent contextualized embedding models in "understanding" syntactical features and suggests future directions in developing more syntax-learning contextualized embeddings. one can try to adapt our proposed csae architecture for an integrated approach by applying the unified tagging scheme; thereby, aspect extraction and sentiment classification can be achieved simultaneously.