Venue: NAACL 2022

Aspect Is Not You Need: No-aspect Differential Sentiment Framework for Aspect-based Sentiment Analysis

Jiahao Cao, Rui Liu, Huailiang Peng1, Lei Jiang, Xu Bai

Institute of Information Engineering, Chinese Academy of Sciences

School of Cyber Security, University of Chinese Academy of Sciences

{caojiahao,liurui3221,penghuailing,jianglei,baixu}@iie.ac.cn

Footnote 1: Corresponding author.

###### Abstract

Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment classification task. Most recent efforts adopt pre-trained model to classify the sentences with aspects. However, the aspect sentiment bias from pre-trained model brings some noise to the ABSA task. Besides, traditional methods using cross-entropy loss are hard to find the potential associations between sentiment polarities. In this work, we analyze the ABSA task from a novel cognition perspective: humans can often judge the sentiment of an aspect even if they do not know what the aspect is. Moreover, it is easier to distinguish positive and negative sentiments than others for human beings because positive and negative are two opposite sentiments. To this end, we propose a no-aspect differential sentiment (NADS) framework for the ABSA task. We first design a no-aspect template by replacing the aspect with a special unbiased character to eliminate the sentiment bias and obtain a stronger representation. To better get the benefits from the template, we adopt contrastive learning between the no-aspect template and the original sentence. Then we propose a differential sentiment loss instead of the cross-entropy loss to better classify the sentiments by distinguishing the different distances between sentiments. Our proposed model is a general framework and can be combined with almost all traditional ABSA methods. Experiments on SemEval 2014 show that our framework is still able to predict the sentiment of the aspect even we don't know what the aspect is. Moreover, our NADS framework boosts three typical ABSA methods and achieves state-of-the-art performance.

+
Footnote †: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1599 - 1609

July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

Aspect-based sentiment analysis (ABSA) [7] aims to identify the sentiment polarity (i.e., negative, neutral, or positive) of each specific aspect term in a piece of text [8, 10, 11]. For example, in _"The food is great, but the service is terrible"_, the sentiment towards _"food"_ is positive while the sentiment towards _"service"_ is negative. We need to predict the sentiments of different aspect terms in a sentence.

Previous works usually employ pre-trained model to extract the embedding of the concatenation of the sentence and the aspect term. In this way, the attention mechanism in pre-trained model enhances the connection between aspect and its context [13, 20]. The experiment results verify its appealing performance. However, the pre-trained model on large-scale raw corpora tends to internalize aspects' intrinsic attributes [7] and brings some noise to the ABSA task. For example, for the sentence _"Desserts include flan and spoapillas"_, a typical model BERT-SPC [21] based on BERT [10] tends to classify the sentiment towards _"Desserts"_ as positive, while the label is neutral. This is because in pre-trained corpora, _"Desserts"_ often appears with words that contain positive sentiment, causing the word _"Desserts"_ to internalize positive sentiment as well. Moreover, traditional text classification methods using the cross-entropy loss have some shortcomings. On the one hand, the cross-entropy loss suffers from lacking of robustness to noisy labels [7] and the possibility of poor margins [1]. On the other hand, the cross-entropy loss ignores the potential relationships between different sentiment polarities. Meanwhile, the non-smooth anisotropic semantic space induced by pre-trained model [11] also brings difficulty in distinguishing potential relationships between sentiments.

To tackle these problems, we analyze the ABSA task from a human cognition perspective. People often pay attention to the learning strategy and feature representation in many NLP tasks, but ignore 

the organization of concepts between human and artificial intelligence. Intuitively, human can still perform well in the ABSA task without knowing the meaning of aspect. As shown in Figure 1, in _"I had a delicious shrimp."_, maybe we don't know what the _"shrimp"_ is (it is a kind of food), we can also easily classify the sentiment polarity of this word as positive. Because we can judge the sentiment of the aspect through its context. Moreover, in human perception, _"positive"_ and _"negative"_ are two completely opposite sentiments and _"neutral"_ sentiment is in between. The distance between _"positive"_ and _"neutral"_ is obviously closer than _"positive"_ and _"negative"_.

Inspired by the human cognition, we propose the no-aspect template differential sentiment (NADS) framework. We first design a no-aspect template by replacing the aspect term in the sentence with the special sentiment unbiased character "\(<aspect>\)" and utilize the contrastive learning between the no-aspect template and the original sentence. In this way, we can not only eliminate the sentiment bias in original sentence, but also learn a wider range of sentence patterns as shown in Figure 1 to enhance the robustness of our framework. Moreover, it helps our NADS framework to judge the sentiment of the aspect without knowing the specific meaning of it, just like human beings. Then, in order to reduce the semantic loss caused by the special character "\(<aspect>\)", we utilize the masked aspect prediction to keep the original semantic information. Morever, we design the differential sentiment loss to find the different distances between different sentiments and better distinguish different sentiments. Our main contributions are:

* We propose a no-aspect template and utilize the no-aspect contrastive learning to consider a wider range of sentence patterns and eliminate the sentiment bias in the aspect embedding. This also enables our model to predict the sentiment of the aspect without knowing what the aspect is, just like human beings.
* We design the differential sentiment loss to help us better distinguish different distances between different sentiments. Moreover, our differential sentiment loss can make samples with the same sentiment as close as possible, and samples with different sentiments as far away as possible.
* Experiments on SemEval 2014 show that our model enhances the performance of three typical ABSA methods and achieves new state-of-the-art. Additionally, the experiments on an aspect robustness test set ARTS show our NADS model can greatly improve the robustness of the model.

## 2 Related Work

Aspect-based sentiment analysis is a fine-grained sentiment classification task. Recently, some works on ABSA have focused on leveraging syntactic knowledge through syntactic trees. Wang et al. (2020) reshaped the syntactic tree with aspect terms as the center and utilized a relational graph attention network to encode the new tree structure for sentiment prediction. Hou et al. (2021) combined the dependency relations from different parses before applying GNNs over the resulting graph.

Another trend utilizes various attention mechanisms to find the semantic relation of an aspect and its context Tan et al. (2019); Li et al. (2018); Fan et al. (2018); Huang et al. (2018). Attention mechanism helps to focus on the context related to aspect and shield the irrelevant context. Besides, some works tried to integrate syntax tree and attention mechanism. Most recent work Li et al. (2021) utilized a mutual biaffine attention mechanism to fuse syntactic information and semantic information from syntactic tree.

In parallel, the pre-trained language model BERT Devlin et al. (2019) has achieved remarkable performance in many NLP tasks. Experiments show that using BERT in ABSA can achieve better results Li et al. (2021); Zhang et al. (2019) than using static word embeddings such as Word2vec Mikolov et al. (2013) and GloVe Pennington et al. (2014). However, Wang et al. (2021) showed that the sentiment bias of aspect caused by the pretrained model may perplex the ABSA task. They utilized external sentiment knowledge SentiWordNet Esuli and Sebastiani (2006) to extract prior three-categorical sentiment for aspect terms. Then they proposed an adversarial network to eliminate

Figure 1: Human performance on the ABSA task.

 

the prior sentiment of aspect terms. However, it is not known whether the aspect sentiment polarity labeled from SentiWordNet is consistent with the sentiment bias in the pre-trained model. In addition, previous works using cross-entropy loss also ignored the potential associations between different sentiment polarities.

In this paper, we propose a no-aspect template and utilize contrastive learning to eliminate sentiment bias and learn a wider range of sentence patterns to improve the robustness of the model. Moreover, we design the differential sentiment loss to better distinguish the different distances between different sentiments and cluster the same sentiment.

## 3 Preliminaries

In this section, we use causal inference Pearl et al. (2000); Robins (2003) to illustrate the theoretical basis of our framework. We illustrate the traditional ABSA methods and our NADS framework by using a causal graph described in Figure 2. Causal graph reflects the causal relationship between variables and we use "\(\rightarrow\)" denotes the direct effect. For the ABSA task, the factors affecting sentiment prediction include the specific aspect term \(A\) that we need to predict and the context of the aspect \(C\). Both \(A\) and \(C\) are important to ABSA task because \(C\) contains the sentiment information and we need to know which aspect \(A\) to predict sentiment for.

In traditional methods' causal graph as shown in Figure 2 (a), the context and aspect capture the direct effect of sentiment via \(C\to Y\) and \(A\to Y\). The fusing information captures the indirect effect of \(A\) and \(C\) on \(Y\) via the \(M\), \(i.e..,A,C\to M\to Y\). The predicted result that \(Y\) would obtain if \(A\) is set to \(a\) and \(C\) is set to \(c\) as:

\[Y_{a,c}=Y(A=a,C=c,M=m) \tag{1}\]

where \(m=M_{a,c}\) denotes the information about the fusion of aspect \(a\) and context \(c\). According to this formula, traditional methods well consider the role of the aspect and its context in the ABSA task. However, in human cognition, the specific meaning of aspect does not affect people's judgment of its sentiment. Traditional methods ignored the aspect sentiment bias which makes aspect have a direct impact on the prediction results \(Y\) via \(A\to Y\). It may cause ABSA models to suffer from the spurious correlation between aspect and sentiment, and thus fail to conduct effective reasoning.

In our NADS framework, we propose to exclude aspect sentiment bias effect on \(A\to Y\) in ABSA as shown in Figure 2 (b). We utilize a special character "\(<aspect>\)" that without sentiment bias to replace the original aspect in the sentence and employ the masked aspect prediction to keep the original semantic information of the sentence. We get the sentiment prediction \(Y\) as:

\[Y_{a^{*},c}=Y(A=a^{*},C=c,M=m^{*}) \tag{2}\]

where \(a^{*}=\)"\(<aspect>\)" and \(m^{*}=M_{a^{*},c}\). In this way, we eliminate the direct impact of aspect's sentiment bias on the prediction results and keep the original semantic information.

## 4 Proposed NADS

In the ABSA task, given a sentence \(S=\{\omega_{1},\omega_{2},...,\omega_{7},...,\omega_{r+t},...,\omega_{n}\}\) and an aspect term \(A=\{\omega_{7},\omega_{r+1},...,\omega_{r+t-1}\}\), the purpose is to predict the sentiment polarity of \(A\) in this \(S\). As shown in Figure 3, our NADS framework consists of three parts. We first propose no-aspect template and utilize contrastive learning between the no-aspect template and original sentence to consider a wider range of sentence patterns and eliminate the sentiment bias in the aspect embedding. Then, in order to make the sentence with the special characters "\(<aspect>\)" keep the original semantic information, we utilize the masked aspect prediction. Finally we design the differential sentiment loss to learn the different distances between different sentiment polarities. We elaborate the details of our proposed NADS.

### No-aspect Contrastive Learning

For each \(\{S,A\}\) pair, we utilize a special character "\(<aspect>\)" without sentiment bias to replace the whole aspect term \(A\) in the sentence. We denote the no-aspect template as \(T\):

\[T=Replace(\{S|A=a\},<aspect>) \tag{3}\]

Figure 2: (a) and (b) are the causal graphs for traditional ABSA methods and our NADS. \(A\): aspect terms. \(C\): context. \(M\): the fusing information of aspect and context. \(Y\): sentiment of the aspect. \(a^{*}\): the special character without sentiment bias.

 

To better use the information from no-aspect template and regularize pre-trained anisotropic embedding space, we utilize contrastive learning between the original sentence and no-aspect template. Specifically, for each sentence-aspect pair \((s_{i},a_{i})\), we denote the positive sentence as:

\[s^{+}_{i}=T_{i} \tag{4}\]

where \(T_{i}\) is the no-aspect template of \(s_{i}\). Thus we get a positive instance \((s^{+}_{i},<aspect>)\) for \((s_{i},a_{i})\). We obtain feature representation for each sentence-aspect pair and positive instance through the encoder \(f_{\theta}(\cdot)\):

\[h_{i}=f_{\theta}(s_{i},a_{i}) \tag{5}\]

\[h^{+}_{i}=f_{\theta}(s^{+}_{i},<aspect>) \tag{6}\]

where \(h_{i}\) and \(h^{+}_{i}\) denote the feature representation of original sentence-aspect pair and positive instance. In our NADS framework, we utilize BERT to get the embedding of each sentence-aspect pair by inputting the concatenation of the aspect term and the sentence. For other models we use their methods as the encoder to get the embedding for each pair \((s_{i},a_{i})\). We denote all other sentences in the mini-batch as negative instances, so the contrastive loss is:

\[\mathcal{L}_{con}=-log\frac{e^{sim(h_{i}h^{+}_{i})/\tau}}{\Sigma^{N}_{j=1}e^{ sim(h_{i}h^{+}_{j})/\tau}} \tag{7}\]

where \(\tau\) is the temperature hyperparameter and \(sim(\cdot)\) is the cosine similarity. \(N\) is the batch size.

By comparing the original sentence with the no-aspect template, we eliminate the sentiment bias caused by the aspect terms in the original sentence and learn not only the information of a sentence, but also the information of a group of sentence patterns. This helps us to improve the robustness of the model. Moreover, contrastive learning helps us regularize pre-trained anisotropic embedding space to prepare for differential sentiment loss.

### Masked Aspect Prediction

In section 4.1, we utilize the "\(<aspect>\)" to construct the no-aspect template. However, we think that directly using a special character "\(<aspect>\)" that does not exist in the pre-trained model may cause trouble to remain the semantic information. Therefore, we utilize masked aspect prediction for the special characters "\(<aspect>\)" to keep the original semantics. Specifically, we mask the aspects by using "\(<aspect>\)" and predict the original aspect terms in the position of "\(<aspect>\)" in our ABSA training dataset. According to Hong et al. (2021), our purpose is to train the embedding of "\(<aspect>\)" to keep the complete semantic information. For our NADS framework, we denote the embedding of "\(<aspect>\)" position as \(h_{[<asp>]}\). We feed \(h_{[<asp>]}\) into a softmax layer to predict the original aspect:

\[\widehat{Y}^{a}=softmax(W_{1}h_{[<asp>]}+b_{1}) \tag{8}\]

where the \(W_{1}\) and \(b_{1}\) are trainable parameters, \(\widehat{Y}^{a}\) indicates the predict probability of the aspect word at its position. We get the masked aspect prediction loss through the cumulative of log-likelihood on

Figure 3: An overview of proposed no-aspect differential sentiment (NADS) framework.

 

predictions of each "\(<aspect>\)" position:

\[{\cal L}_{asp}=-\Sigma logp(\widehat{Y}^{a}) \tag{9}\]

Specially, we only predict the position of "\(<aspect>\)" in sentences. The masked aspect prediciton helps us keep the original semantic information of the sentence after replacing the aspect.

### Differential Sentiment Loss

After regularizing pre-trained anisotropic embedding space by using the contrastive learning between original and no-aspect template, we design our differential sentiment loss to better distinguish different sentiments. We first embed our labels into the same size of \(h_{i}\). We convert positive, neutral and negative sentiment labels into label embeddings \(L=\{l_{pos},l_{neu},l_{negg}\}\). The distance between the sentence-aspect pair embedding \(h_{i}\) and label embedding \(l_{i}\) is:

\[d(h_{i},l_{i})=1-\frac{h_{i}^{\top}l_{i}}{\|h_{i}\|\cdot\|l_{i}\|} \tag{10}\]

For each sentence-aspect pair embedding \(h_{i}\), the distance between \(h_{i}\) and its label \(l_{i}\) should be closer than other label embeddings in \(L\). Thus, we utilize a triplet loss to make \(h_{i}\) closer to the right label embedding \(l_{i}\) and further to the other label embeddings. For each \(h_{i}\), the positive instance is the label embedding \(l_{i}\) and the negative instances are the other label embeddings in \(L\). Moreover, in human cognition, the distances between different sentiments are different. Thus, we set a specific margin for each negative instance to better distinguish the different distances between different sentiments. Our differential sentiment loss is given as:

\[\begin{split}{\cal L}_{ds}=&\Sigma_{l^{{}^{\prime}} _{i}\in L_{l^{{}^{\prime}}}\neq l_{i}}max(d(h_{i},l_{i})\\ &-d(h_{i},l^{{}^{\prime}}_{i})+m(l_{i},l^{{}^{\prime}}_{i}),0) \end{split} \tag{11}\]

where \(m(l_{i},l^{{}^{\prime}}_{i})\) is the specific margin for label \(l_{i}\) and \(l^{{}^{\prime}}_{i}\). According to human cognition, we denote that positive and negative sentiments should have the same distance to neutral sentiment and the distance between positive and negative is further. Thus, we set \(m(pos,neu)=m(neg,neu)\) and \(m(pos,neg)>m(pos,neu)\) in our model. Compared with the cross-entropy loss, our differential sentiment loss can better classify the sentiments through distinguishing the differences between sentiments. Moreover, our differential sentiment loss can jointly train the model and label embeddings and make our framework converge faster.

In order to judge the sentiment polarity of the sentence-aspect pair, we utilize cosine similarity to construct our scoring function:

\[S(h,l)=\frac{h^{\top}l}{\|h\|\cdot\|l\|} \tag{12}\]

where \(h\) is the embedding of the sentence-aspect pair, and \(l\) is the embedding of the label. We take the \(l\) with the largest score as our prediction result.

Our training goal is to minimize the following total objective function:

\[{\cal L}={\cal L}_{ds}+\lambda_{1}{\cal L}_{con}+\lambda_{2}{\cal L}_{asp} \tag{13}\]

where \(\lambda_{1}\) and \(\lambda_{2}\) are the weights of contrastive leanrning loss and masked aspect prediction loss.

## 5 Experiments

### Datasets

We evaluate our model on two ABSA task public datasets: Restaurant and Laptop reviews from SemEval 2014 Task 4 Pontiki et al. (2014). We remove several examples with "conflict" sentiment polarity labels in the reviews. Table 1 shows the statistics of these datasets.

### Baseline Methods

We compare our NADS with state-of-the-art baselines. The models are described as follows.

1) **BERT-SCP** Song et al. (2019) utilizes BERT to encode the sentence-aspect pair as : "[CLS] sentence [SEP] aspect [SEP]" and gets the embedding of "[CLS]". Our NADS framework utilizes BERT- SPC as the encoder.

2) **AEN+BERT** Song et al. (2019) utilizes BERT and several attention layers to encoder sentence-aspect pair. The embedding of the sentence and the embedding of the aspect are obtained respectively.

3) **CapsNet+BERT** Jiang et al. (2019) combines the BERT and capsule networks in ABSA.

\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Division** & **\#Pos** & **\#Neu** & **\#Neg** \\ \hline \multirow{2}{*}{Laptop} & Train & 976 & 455 & 851 \\  & Test & 337 & 167 & 128 \\ \cline{2-5}  & Train & 2164 & 637 & 807 \\ \cline{1-1} \cline{2-5}  & Test & 727 & 196 & 196 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistics on two datasets of ABSA.

 

4) **BERT-ADA**Rietzler et al., 2020) utilizes domain data to enhance the BERT and then uses task data to make supervised fine-tuning.

5) **SDGCN+BERT**Zhao et al., 2020) employs graph convolution network for sentences with multiple aspects.

6) **R-GAT+BERT**Wang et al., 2020) proposes an aspect-oriented tree and encodes new dependency trees with a relational GAT.

7) **DGEDT+BERT**Tang et al., 2020) proposes a mutual biaffine module to jointly consider the representations learned from Transformer and the GNN model over the dependency tree.

8) **DualGCN+BERT**Li et al., 2021) utilizes dependency tree to extract syntax information and self attention to extract semantic information.

Moreover, in order to prove the effectiveness of our NADS framework, we adopt our NADS to BERT-SPC, AEN+BERT and DualGCN+BERT.

### Implementation Details

We utilize the bert-base-uncased English version. Following the DualGCN Li et al. (2021), we use LAL-Parser Mrini et al. (2019) to get dependency tree for DualGCN+NADS. We randomly initialize the embedding of three sentiments and we set the \(\lambda_{1}=0.4\), \(\lambda_{2}=0.1\) during our training. The different margins \((mpos,neu),m(pos,neg))\) are set to (0.4, 0.6), (0.4, 0.6) for the laptop and restaurant datasets for our NADS framework. During training, we use AdamW as the optimizer and set the learning rate to \(2\times 10^{-5}\). We train the model up to 15 epochs with a batch size of 16.

### Comparison Results

We utilize the accuracy and macro-averaged F1-score to evaluate ABSA task. In order to better predict the correct sentiment in the test, we adopt three different ways to test our NADS framework.

1) **Original test**: utilizing the concatenation of the original aspect and the original sentence as the input and extract the embedding for prediction.

2) **Noasp test**: utilizing the concatenation of the "\(<aspect>\)" and the no-aspect template as the input of encoder. This test method can help us judge whether the model can correctly predict sentiment

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{3}{*}{Models} & Strategy & \multicolumn{2}{c}{14Rest} & \multicolumn{2}{c}{14Lap} \\ \cline{3-6}  & & Accuracy & Macro-F1 & Accuracy & Macro-F1 \\ \hline CapsNet+BERT Jiang et al. (2019) & Ori & 85.09 & 77.75 & 78.21 & 73.34 \\ BERT-ADA Rietzler et al. (2020) & Ori & 87.14 & 80.05 & 79.19 & 74.18 \\ SDGCN-BERT Zhao et al. (2020) & Ori & 83.57 & 76.47 & 81.35 & 78.34 \\ R-GAT+BERT Wang et al. (2020) & Ori & 86.60 & 81.35 & 78.21 & 74.07 \\ DGEDT+BERT Tang et al. (2020) & Ori & 86.30 & 80.00 & 79.80 & 75.60 \\ \hline \multirow{3}{*}{BERT-SPC Song et al. (2019)} & Ori & 84.46 & 76.98 & 78.99 & 75.03 \\  & Noasp & 81.77 & 70.81 & 75.47 & 69.65 \\  & Unite & 84.45 & 77.40 & 78.16 & 73.06 \\ \cline{2-6}  & Ori & 87.49 & **82.09** & **82.12** & **79.13** \\ NADS & Noisy & 87.04 & 81.77 & 81.01 & 77.69 \\  & Unite & **87.58** & 81.73 & 81.96 & 78.87 \\ \hline \multirow{3}{*}{AEN+BERT Song et al. (2019)} & Ori & 83.12 & 73.76 & 79.93 & 76.31 \\  & Noasp & 80.70 & 68.86 & 77.06 & 72.41 \\  & Unite & 80.97 & 71.65 & 78.16 & 74.39 \\ \cline{2-6}  & Ori & 84.00 & 75.88 & 81.33 & 77.78 \\ \cline{2-6}  & Noasp & 86.51 & **80.16** & 80.22 & 76.88 \\  & Unite & **86.68** & 79.69 & **81.48** & **78.07** \\ \hline \multirow{3}{*}{DualGCN+BERT Li et al. (2021)} & Ori & 87.13 & 81.16 & 81.80 & 78.10 \\  & Noasp & 81.95 & 72.42 & 77.53 & 73.49 \\ \cline{1-1}  & Unite & 84.90 & 77.24 & 78.64 & 74.43 \\ \cline{1-1} \cline{2-6}  & Ori & 87.49 & 82.07 & 82.75 & **79.95** \\ \cline{1-1}  & Noasp & 86.86 & 81.23 & 81.49 & 78.02 \\ \cline{1-1}  & Unite & **87.67** & **82.59** & **82.75** & 79.72 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of our NADS on three traditional methods to other baselines on two datasets.

 

without knowing the specific meaning of the aspect like human beings.

3) **Unite test**: using both Original test mode and Noasp test mode to get the scores of each label for each sentence-aspect pair and sum the scores of same label after normalization.

Table 2 shows our main experimental results. As we can see, our NADS framework outperforms all baselines on laptop and restaurant datasets, and the performance of the three traditional models: BERT-SPC, AEN+BERT and DualGCN+BERT has been improved after adding our NADS framework. Our NADS outperforms the BERT-SPC by 3.03%/3.13% on Restaurant/Laptop. The result demonstrates that our NADS framework effectively utilizes the way of human cognitive and plays a better role in the ABSA task. Compared with traditional methods, our no-aspect template eliminates the sentiment bias of aspect and learns more information of a group of sentence patterns, which can reduce the noise caused by aspect sentiment bias and enhance the robustness of our framework. Moreover, our differential sentiment loss can better classify the sentiment through distinguishing the difference in these three different sentiment polarities after contrastive learning. The experiments on three traditional methods also show that our framework well fits most of the existing models and boosts their performance.

In parallel, according to the experimental results of Noisy test, the performance of traditional methods drops significantly without knowing what aspect is. However, our NADS framework can still perform well without knowing the aspect just like human beings. In Noasp test mode, BERT-SPC drops 2.69%/3.52% on Restaurant/Laptop. By contrast, our NADS framework drops only 0.45%/1.11% on Restaurant/Laptop. We also find AEN+NADS increases 2.51% on Restaurant dataset, while the AEN+BERT drops 2.42%. This shows that our NADS can still perform well without knowing the specific meaning of the aspect. Comparing these three test modes, we can also find that the Unite test mode can achieve the most stable result in different models.

### Ablation Study

In order to further study the role of different modules in our framework, we conduct extensive ablation experiments. The results are shown in Table 3. NADS w/o NOASP denotes that we only utilize the original sentence and remove the contrastive learning. Without contrastive learning between no-aspect template and original sentence, the sentiment bias of aspect perplexes the prediction result and more importantly, differential sentiment loss will not work because of the anisotropic in BERT model. Therefore, its performance is degraded on both two datasets. NADS w/o MAP means that we remove the masked aspect prediction module so that we may lose the original semantic information of the sentence. NADS w/o DS indicates that we utilize the cross-entropy loss function instead of our differential sentiment loss. Without differential sentiment loss, the model cannot find the different distances between sentiments. Experiments show that every module is indispensable in our NADS framework.

### Selection of Margin

We experiment with different margins in the differential sentiment loss. In our framework, we only consider the \(m(pos,neu)\) and \(m(pos,neg)\). Figure 4 shows the accuracy of different \(m(pos,neg)\) when we set \(m(pos,neu)=0.4\) in our three methods based on our NADS framework on Laptop dataset. As we can see, the accuracy increases first and then decreases in the process of \(m(pos,neg)\) gradually increasing. The three models perform best when the \(m(pos,neg)\) is set to 0.6, 0.5 and 0.7. This experiment shows that the distance be

Figure 4: Effect of different \(m(pos,neg)\) while set \(m(pos,neu)=0.4\) in laptop dataset.

\begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{Models} & \multicolumn{2}{c}{14Rest} & \multicolumn{2}{c}{14Cap} \\ \cline{2-5}  & Acc & Macro-F1 & Acc & Macro-F1 \\ \hline NADS & **87.49** & **82.09** & **82.12** & **79.13** \\ NADS w/o NOASP & 85.22 & 78.88 & 79.43 & 75.30 \\ NADS w/o MAP & 87.04 & 81.73 & 81.18 & 78.51 \\ NADS w/o DS & 87.22 & 81.71 & 81.01 & 77.26 \\ \hline \end{tabular}
\end{table}
Table 3: Experimental results of ablation study.

 

tween positive and negative is indeed farther than that between positive and neutral. It proves the effectiveness of our differential sentiment loss.

### Sentiment Bias Elimination

In order to better understand the ability of our NADS framework to eliminate sentiment bias, we find several examples whose labels are neutral and show their prediction results in different models in Table 4, where \(P,N,O\) represent positive, negative, and neutral sentiments. We highlight the aspect words in red. We can see that our NADS framework outperforms all other models. For the aspect "_steak_" in the first sample, previous methods ignore the positive sentiment bias of "_steak_" and incorrectly predict the sentiment as positive. In contrast, our NADS eliminates the positive sentiment bias through no-aspect template and predicts the correct sentiment as neutral. Moreover, we also show the distribution of bad cases in Figure 5. The bad cases of neutral aspects terms in our NADS framework are significantly less than BERT-SVC. This proves the effectiveness of our NADS framework in eliminating sentiment bias. However, there are still some neutral aspect terms in our framework that are incorrectly predicted as shown in Table 4. A possible reason is that there may be other words in a sentence carrying the sentiment bias in addition to the current aspect.

### Robustness Study

In order to verify the robustness of our NADS, we test the robustness score of our framework on the Aspect Robustness Test Set (ARTS) [20]. The datasets enrich 14Lap and 14Rest according to three strategies: reverse the original sentiment of the target aspect (REVTGT), perturb the sentiments of the non-target aspects (REVON) and generate more non-target aspect terms that have opposite sentiment polarities to the target (ADDIDIFF). They take the original sentence and the three variants as an unit. Only if the original sentence and all variants are correct, the unit is correct. Calculate the accuracy of the units in the datasets as the final Aspect Robustness Score (ARS).

We compare the ARS of the three models betw

Figure 5: Distribution of bad cases of our NADS framework and BERT-SVC.

\begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{Best} & Lap \\ \cline{3-4} Models & Acc\(\rightarrow\)ARCS(Change) & Acc\(\rightarrow\)ARCS(Change) \\ \hline BERT-PT & 86.70\(\rightarrow\)95.29(Z+2.74) & 78.53\(\rightarrow\)53.29(Z+1.254) \\ RGAT & 84.41\(\rightarrow\)56.54(Z+2.87) & 78.08\(\rightarrow\)51.37(Z+2.67) \\ \hline BERT-SVC & 83.04\(\rightarrow\)54.54(Z+2.82) & 77.95\(\rightarrow\)50.94(Z+2.66) \\ NADS & 87.49\(\rightarrow\)64.55(**2.29**) & 82.12\(\rightarrow\)58.79(Z+1.23) \\ \hline AES+BERT & 81.32\(\rightarrow\)25.45(**4.57**) & 97.93\(\rightarrow\)30.09(**4.94**) \\ SNADS & 84.00\(\rightarrow\)66.16(**51.32**) & 81.33\(\rightarrow\)37.51(**1.44**) \\ \hline DualGCN+BERT & 87.61\(\rightarrow\)63.57(**23.56**) & 81.80\(\rightarrow\)57.99(Z+1.81) \\ DualGCN+NADS & 87.49\(\rightarrow\)66.16(**21.33**) & 82.75\(\rightarrow\)60.82(**21.93**) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Case study. Comparison of our NADS model to different baselines. Marker \(\checkmark\) indicates correct prediction while \(\times\) indicates incorrect prediction.

 

fore adding our NADS framework and after adding NADS. The results in Table 5 show that the ARS of the model has been significantly improved after adding our NADS framework. Our NADS framework adding DualGCN performs significantly better than other models with 21.33% and 21.93% decline on Restaurant and Laptop. This shows that our framework utilizing human cognition has better robustness than other models.

Moreover, we utilize the three test modes to test on the ARTS as shown in Table 6. As we can see, the AEN+NADS model with 57.39% and 44.18% decline on Restaurant and Laptop when using Original test mode. However, with 26.51% and 22.34% decline when utilizing Noasp test mode. In the overall scheme, the Noasp test mode and Unite test mode can get a more stable result than the Original test mode on the ARTS. Utilizing no-aspect template in test may be a more stable robustness test method.

## 6 Conclusion

In this paper, we propose a NADS framework which is more in line with human cognition for the ABSA task. Our NADS framework utilizes no-aspect contrastive learning to eliminate the sentiment bias of aspects and enhance the sentence representations. In addition, we construct a differential sentiment loss to better classify the sentiments through distinguishing the different distances between sentiment polarities. Extensive experiments show that our NADS framework boosts three typical ABSA methods and outperforms baselines. Moreover, our NADS framework can still perform well even we don't know what the aspect is. The test on the robustness dataset shows that our NADS framework significantly improves the robustness of the model.

## Acknowledgements

This paper is supported by Pilot Projects of Chinese Academy of Sciences (NO.XDC02030000).

## References

* Dai et al. (2021) Junqi Dai, Hang Yan, Tianxiang Sun, Pengfei Liu, and Xipeng Qiu. 2021. Does syntax matter? a strong baseline for aspect-based sentiment analysis with RoBERTa. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 1816-1829, Online. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Elsayed et al. (2018) Gamaleldin Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio. 2018. Large margin deep networks for classification. _Advances in Neural Information Processing Systems_, 31:842-852.
* Esuli and Sebastiani (2006) Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiwordnet: A publicly available lexical resource for opinion mining. In _Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)_.
* Fan et al. (2018) Feifan Fan, Yansong Feng, and Dongyan Zhao. 2018. Multi-grained attention network for aspect-level sentiment classification. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, pages 3433-3442, Brussels, Belgium. Association for Computational Linguistics.
* Hong et al. (2021) Jimin Hong, TaeHee Kim, Hyesu Lim, and Jaegul Choo. 2021. AVocaDo: Strategy for adapting vocabulary to downstream domain. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 4692-4700, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Hou et al. (2021) Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, and Bowen Zhou. 2021. Graph ensemble learning over multiple dependency trees for aspect-level sentiment classification. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 2884-2894, Online. Association for Computational Linguistics.
* Huang et al. (2018) Binxuan Huang, Yanglan Ou, and Kathleen M Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. In _International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation_, pages 197-206. Springer.
* Huang et al. (2020) Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Bae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020. Reducing sentiment bias in language models via counterfactual evaluation. In _Findings of the Association for Computational Linguistics: EMNLP 2020_, pages 65-83, Online. Association for Computational Linguistics.
* Jiang et al. (2011) Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In _Proceedings of the 49th Annual Meeting of the Association for Computational 

Linguistics: Human Language Technologies_, pages 151-160, Portland, Oregon, USA. Association for Computational Linguistics.
* Jiang et al. (2019) Qingnan Jiang, Lei Chen, Ruifeng Xu, Xiang Ao, and Min Yang. 2019. A challenge dataset and effective models for aspect-based sentiment analysis. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 6280-6285, Hong Kong, China. Association for Computational Linguistics.
* Li et al. (2020) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the sentence embeddings from pre-trained language models. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 9119-9130, Online. Association for Computational Linguistics.
* Li et al. (2018) Lishuang Li, Yang Liu, and AnQiao Zhou. 2018. Hierarchical attention based position-aware network for aspect-level sentiment analysis. In _Proceedings of the 22nd Conference on Computational Natural Language Learning_, pages 181-189, Brussels, Belgium. Association for Computational Linguistics.
* Li et al. (2021) Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma, Xiaojie Wang, and Eduard Hovy. 2021. Dual graph convolutional networks for aspect-based sentiment analysis. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 6319-6329, Online. Association for Computational Linguistics.
* Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. _arXiv preprint arXiv:1301.3781_.
* Mrini et al. (2019) Khalil Mrini, Franck Dernoncourt, Trung Bui, Walter Chang, and Ndapa Nakashole. 2019. Rethinking self-attention: An interpretable self-attentive encoder-decoder parser. _CoRR_, abs/1911.03875.
* Pearl et al. (2000) Judea Pearl et al. 2000. Models, reasoning and inference. _Cambridge, UK: CambridgeUniversityPress_, 19.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
* Pontiki et al. (2014) Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)_, pages 27-35, Dublin, Ireland. Association for Computational Linguistics.
* Rietzler et al. (2020) Alexander Rietzler, Sebastian Stabinger, Paul Opitz, and Stefan Engl. 2020. Adapt or get left behind: Domain adaptation through BERT language model finetuning for aspect-target sentiment classification. In _Proceedings of the 12th Language Resources and Evaluation Conference_, pages 4933-4941, Marseille, France. European Language Resources Association.
* Robins (2003) James M Robins. 2003. Semantics of causal dag models and the identification of direct and indirect effects. _Oxford Statistical Science Series_, pages 70-82.
* Song et al. (2019) Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. _arXiv preprint arXiv:1902.09314_.
* Tan et al. (2019) Xingwei Tan, Yi Cai, and Changxi Zhu. 2019. Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 3426-3431, Hong Kong, China. Association for Computational Linguistics.
* Tang et al. (2016) Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016. Effective lstms for target-dependent sentiment classification. In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_, pages 3298-3307.
* Tang et al. (2020) Hao Tang, Donghong Ji, Chenliang Li, and Qiji Zhou. 2020. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 6578-6588, Online. Association for Computational Linguistics.
* Wang et al. (2021) Bo Wang, Tao Shen, Guodong Long, Tianyi Zhou, and Yi Chang. 2021. Eliminating sentiment bias for aspect-level sentiment classification with unsupervised opinion extraction. In _Findings of the Association for Computational Linguistics: EMNLP 2021_, pages 3002-3012, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Wang et al. (2020) Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020. Relational graph attention network for aspect-based sentiment analysis. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 3229-3238, Online. Association for Computational Linguistics.
* Xing et al. (2020) Xiaoyu Xing, Zhijing Jin, Di Jin, Bingning Wang, Qi Zhang, and Xuanjing Huang. 2020. Tasty burgers, soggy fries: Probing aspect robustness in aspect-based sentiment analysis. In _Proceedings of the 2020 

Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 3594-3605. External Links: Link Cited by: SS1, SS2.
* C. Zhang, Q. Li, and D. Song (2019)Aspect-based sentiment classification with aspect-specific graph convolutional networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, pp. 4568-4578. External Links: Link, Document Cited by: SS1, SS2.
* Z. Zhang and M. R. Sabuncu (2018)Generalized cross entropy loss for training deep neural networks with noisy labels. In 32nd Conference on Neural Information Processing Systems (NeurIPS), External Links: Link Cited by: SS1, SS2.
* P. Zhao, L. Hou, and O. Wu (2020)Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. Knowledge-Based Systems193, pp. 105443. External Links: Link, Document Cited by: SS1, SS2.