Aspect Is Not You Need: No-aspect Differential Sentiment Framework for Aspect-based Sentiment Analysis (2022)

Abstract: Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment classification task. Most recent efforts adopt pre-trained model to classify the sentences with aspects. However, the aspect sentiment bias from pre-trained model brings some noise to the ABSA task. Besides, traditional methods using cross-entropy loss are hard to find the potential associations between sentiment polarities. In this work, we analyze the ABSA task from a novel cognition perspective: humans can often judge the sentiment of an aspect even if they do not know what the aspect is. Moreover, it is easier to distinguish positive and negative sentiments than others for human beings because positive and negative are two opposite sentiments. To this end, we propose a no-aspect differential sentiment (NADS) framework for the ABSA task. We first design a no-aspect template by replacing the aspect with a special unbiased character to eliminate the sentiment bias and obtain a stronger representation. To better get the benefits from the template, we adopt contrastive learning between the no-aspect template and the original sentence. Then we propose a differential sentiment loss instead of the cross-entropy loss to better classify the sentiments by distinguishing the different distances between sentiments. Our proposed model is a general framework and can be combined with almost all traditional ABSA methods. Experiments on SemEval 2014 show that our framework is still able to predict the sentiment of the aspect even we don’t konw what the aspect is. Moreover, our NADS framework boosts three typical ABSA methods and achieves state-of-the-art performance.

## introduction aspect-based sentiment analysis (absa) @xcite aims to identify the sentiment polarity (i.e., negative, neutral, or positive) of each specific aspect term in a piece of text @xcite @xcite . for example, in "the food is great, but the service is terrible", the sentiment towards "food" is positive while the sentiment towards "service" is negative. we need to predict the sentiments of different aspect terms in a sentence. previous works usually employ pre-trained model to extract the embedding of the concatenation of the sentence and the aspect term. in this way, the attention mechanism in pre-trained model enhances the connection between aspect and its context @xcite . the experiment results verify its appealing performance. however, the pre-trained model on largescale raw corpora tends to internalize aspects' intrinsic attributes @xcite and brings some noise to the absa task. for example, for the sentence "desserts include flan and sopaipillas", a typical model bert-spc @xcite based on bert @xcite tends to classify the sentiment towards "desserts" as positive, while the label is neutral. this is because in pre-trained corpora, "desserts" often appears with words that contain positive sentiment, causing the word "desserts" to internalize positive sentiment as well. moreover, traditional text classification methods using the cross-entropy loss have some shortcomings. on the one hand, the cross-entropy loss suffers from lacking of robustness to noisy labels @xcite and the possibility of poor margins @xcite . on the other hand, the cross-entropy loss ignores the potential relationships between different sentiment polarities. meanwhile, the non-smooth anisotropic semantic space induced by pre-trained model @xcite also brings difficulty in distinguishing potential relationships between sentiments. to tackle these problems, we analyze the absa task from a human cognition perspective. people often pay attention to the learning strategy and feature representation in many nlp tasks, but ignore i had a @xmath0. inspired by the human cognition, we propose the no-aspect template differential sentiment (nads) framework. we first design a no-aspect template by replacing the aspect term in the sentence with the special sentiment unbiased character "< @xmath1. in this way, we can not only eliminate the sentiment bias in original sentence, but also learn a wider range of sentence patterns as shown in figure 1 to enhance the robustness of our framework. moreover, it helps our nads framework to judge the sentiment of the aspect without knowing the specific meaning of it, just like human beings. then, in order to reduce the semantic loss caused by the special character "< @xmath2, we utilize the masked aspect prediction to keep the original semantic information. morever, we design the differential sentiment loss to find the different distances between different sentiments and better distinguish different sentiments. our main contributions are: ## related work aspect-based sentiment analysis is a fine-grained sentiment classification task. recently, some works on absa have focused on leveraging syntactic knowledge through syntactic trees. @xcite reshaped the syntactic tree with aspect terms as the center and utilized a relational graph attention network to encode the new tree structure for sentiment prediction. @xcite combined the dependency relations from different parses before applying gnns over the resulting graph. another trend utilizes various attention mechanisms to find the semantic relation of an aspect and its context @xcite @xcite . attention mechanism helps to focus on the context related to aspect and shield the irrelevant context. besides, some works tried to integrate syntax tree and attention mechanism. most recent work @xcite utilized a mutual biaffine attention mechanism to fuse syntactic information and semantic information from syntactic tree. in parallel, the pre-trained language model bert @xcite has achieved remarkable performance in many nlp tasks. experiments show that using bert in absa can achieve better results @xcite than using static word embeddings such as word2vec @xcite and @xcite . however, @xcite showed that the sentiment bias of aspect caused by the pretrained model may perplex the absa task. they utilized external sentiment knowledge sentiword-net @xcite to extract prior three-categorical sentiment for aspect terms. then they proposed an adversarial network to eliminate the prior sentiment of aspect terms. however, it is not known whether the aspect sentiment polarity labeled from sentiwordnet is consistent with the sentiment bias in the pre-trained model. in addition, previous works using cross-entropy loss also ignored the potential associations between different sentiment polarities. in this paper, we propose a no-aspect template and utilize contrastive learning to eliminate sentiment bias and learn a wider range of sentence patterns to improve the robustness of the model. moreover, we design the differential sentiment loss to better distinguish the different distances between different sentiments and cluster the same sentiment. ## preliminaries in this section, we use causal inference @xcite to illustrate the theoretical basis of our framework. we illustrate the traditional absa methods and our nads framework by using a causal graph described in figure 2 . causal graph reflects the causal relationship between variables and we use "→" denotes the direct effect. for the absa task, the factors affecting sentiment prediction include the specific aspect term a that we need to predict and the context of the aspect c. both a and c are important to absa task because c contains the sentiment information and we need to know which aspect a to predict sentiment for. in traditional methods' causal graph as shown in figure 2 (a), the context and aspect capture the direct effect of sentiment via @xmath3. the fusing information captures the indirect effect of a and c on y via the m , i.e., a, @xmath4. the predicted result that y would obtain if a is set to a and c is set to c as: in our nads framework, we propose to exclude aspect sentiment bias effect on @xmath5. we utilize a special character "< @xmath6. we get the sentiment prediction y as: ## proposed nads in the absa task, given a sentence @xmath7, ω 2 , ..., ω τ , ..., ω τ +t , ..., ω n } and an aspect term @xmath8, ω τ +1 , ..., ω τ +t-1 }, the purpose is to predict the sentiment polarity of a in this s. as shown in figure 3 , our nads framework consists of three parts. we first propose no-aspect template and utilize contrastive learning between the noaspect template and original sentence to consider a wider range of sentence patterns and eliminate the sentiment bias in the aspect embedding. then, in order to make the sentence with the special characters "< @xmath9, we utilize the masked aspect prediction. finally we design the differential sentiment loss to learn the different distances between different sentiment polarities. we elaborate the details of our proposed nads. ## conclusion in this paper, we propose a nads framework which is more in line with human cognition for the absa task. our nads framework utilizes no-aspect contrastive learning to eliminate the sentiment bias of aspects and enhance the sentence representations. in addition, we construct a differential sentiment loss to better classify the sentiments through distinguishing the different distances between sentiment polarities. extensive experiments show that our nads framework boosts three typical absa methods and outperforms baselines. moreover, our nads framework can still perform well even we don't know what the aspect is. the test on the robustness dataset shows that our nads framework significantly improves the robustness of the model.