Enhancing Aspect-level Sentiment Analysis with Word Dependencies (2021)

Abstract: Aspect-level sentiment analysis (ASA) has received much attention in recent years. Most existing approaches tried to leverage syntactic information, such as the dependency parsing results of the input text, to improve sentiment analysis on different aspects. Although these approaches achieved satisfying results, their main focus is to leverage the dependency arcs among words where the dependency type information is omitted; and they model different dependencies equally where the noisy dependency results may hurt model performance. In this paper, we propose an approach to enhance aspect-level sentiment analysis with word dependencies, where the type information is modeled by key-value memory networks and different dependency results are selectively leveraged. Experimental results on five benchmark datasets demonstrate the effectiveness of our approach, where it outperforms baseline models on all datasets and achieves state-of-the-art performance on three of them.

## introduction aspect-level sentiment analysis (asa) determines the sentiment polarity of a given input text on the fine-grained level, where the sentiment towards a particular aspect in the text is predicted instead of the entire input. e.g., the sentiment of an aspect "bar service" in the sentence "total environment is fantastic although bar service is poor." is negative, although the text as a whole conveys a positive sentiment polarity. due to its high practical value in many scenarios, e.g., product review analysis, social media tracking, etc., asa attracts much attention in the natural language processing (nlp) community for years @xcite @xcite @xcite . in recent studies, neural networks, especially recurrent models with attention mechanism, are widely applied in this task, where many of them @xcite @xcite @xcite @xcite ) model semantic relatedness between context and aspect words to facilitate sentiment analysis on aspects. there are other approaches using additional inputs such as word position @xcite , document information @xcite , commonsense knowledge @xcite . among all such inputs, dependency results of the input text are proved to be a kind of useful information @xcite @xcite @xcite , because they can help the model locate important content that modifies the aspect words and thus further suggests the sentiment towards the aspect words. previous approaches with attention mechanism @xcite , graph neural networks (gnn) @xcite @xcite and transformer @xcite are applied in leveraging such information. however, most of them mainly focus on using the dependencies among words and omit to leverage other information such as relation types, which could provide useful cues to predict the sentiment. also, they model all dependency information instances equally without weighting them according to their contribution to the task, where noisy information from the auto-generated dependency tree may hurt model performance. therefore, improved methods are expected to comprehensively and efficiently learn dependencies among words to enhance asa. to address the aforementioned limitations, in this paper we propose an effective and efficient neural approach to asa with incorporating word dependencies, which is acquired from off-the-shelf toolkits and modeled by key-value memory net-cop ## the approach the task of asa aims to analyze the sentiment of a text towards a specific aspect, which is formalized as a classification task performing on sentence-aspect pairs @xcite @xcite @xcite @xcite @xcite . in detail, each input sentence and the aspect terms in it are denoted by @xmath0, x 2 , • • • , x n and @xmath1, a 2 • • • , a m , respectively, where a is the sub-string of x (a ⇢ x ), n and m refer to the word-based length of x and a. following this paradigm, we design the architecture of our approach in figure 1 , with a bert-based @xcite encoder illustrated on the left to compute the sentence-aspect pair representation r, and enhanced by the word dependency information obtained from the kvmn module on the right, then the result is fed into a softmax decoder to predict the text sentiment towards the aspect. therefore, asa through our approach can be formalized as ## related work different from sentiment analysis for large granular texts, such as document and sentences, asa focuses on processing sentiment polarities for a specific aspect (e.g., "pizza") or category (e.g., "food") in a piece of text. to address this task, early approaches @xcite followed the sentence classification paradigm and recent studies enhanced it as a mission of sentenceaspect pair classification with applying neural approaches @xcite @xcite @xcite @xcite such as recurrent models (e.g., bi-lstm) and pretrained encoders (e.g., bert) for effectively capturing contextual information. in addition to improving the input form, advanced models such as memory networks @xcite @xcite @xcite , attention mechanism @xcite @xcite , capsule networks @xcite @xcite , gnn @xcite @xcite , and transformer @xcite ) are applied to this task, with other studies leveraging external resources, including position information @xcite , document information @xcite , commonsense knowledge @xcite , etc. among all resources, syntactic information was proved to be the most effective one and successfully adopted in recent studies with gnn @xcite @xcite . compared with previous studies, our approach offers an alternative way to use kvmn and syntactic information for asa. consider those studies using memory networks where their memories are represented by contextual features of the aspect terms, dependency information was not leveraged in their work. in addition, compared with those approaches leveraging word dependencies (i.e., us-ing attention mechanism or gnn), where they not only omitted useful dependency information such as relation types, but also demanded a complicated model structure in doing so, our approach ensures comprehensively encoding from both word-word relations and their dependency types, and models them in an efficient way by kvmn. ## conclusion in this paper, we propose an effective neural approach to improve asa with word dependencies by kvmn, where for each aspect term, we firstly extract the words associated to it according to the dependency parse of the input sentence and their corresponding dependency relation types, then use kvmn to encode and weight such information to enhance asa accordingly. in our approach, not only word-word relations but also their dependency types are leveraged in a kvmn, which to our best knowledge are the first attempts in all related syntax-driven studies for asa. experimental results on five widely used benchmark datasets demonstrate the effectiveness of our approach, and shows that second-order word dependency is the best choice for asa, where the new state-of-the-art results are achieved on three datasets. moreover, further analyses illustrate the validity of applying kvmn on both dependency relation and type information, especially the effectiveness of dependency types, which are often omitted in previous studies.