SlowGuess/NAACL_2022_OCR huggingface dataset:
Table 2 shows our main experimental results. As we can see, our NADS framework outperforms all baselines on laptop and restaurant datasets, and the performance of the three traditional models: BERT-SPC, AEN+BERT and DualGCN+BERT has been improved after adding our NADS framework. Our NADS outperforms the BERT-SPC by  $3.03\% / 3.13\%$  on Restaurant/Laptop. The result demonstrates that our NADS framework effectively utilizes the way of human cognitive and plays a better role in the ABSA task. Compared with traditional methods, our no-aspect template eliminates the sentiment bias of aspect and learns more information of a group of sentence patterns, which can reduce the noise caused by aspect sentiment bias and enhance the robustness of our framework. Moreover, our differential sentiment loss can better classify the sentiment through distinguishing the difference in these three different sentiment polarities after contrastive learning. The experiments on three traditional methods also show that our framework well fits most of the existing models and boosts their performance.
In parallel, according to the experimental results of Noasp test, the performance of traditional methods drops significantly without knowing what aspect is. However, our NADS framework can still perform well without knowing the aspect just like human beings. In Noasp test mode, BERT-SPC drops  $2.69\% / 3.52\%$  on Restaurant/Laptop. By contrast, our NADS framework drops only  $0.45\% / 1.11\%$  on Restaurant/Laptop. We also find AEN+NADS increases  $2.51\%$  on Restaurant dataset, while the AEN+BERT drops  $2.42\%$ . This shows that our NADS can still perform well without knowing the specific meaning of the aspect. Comparing these three test modes, we can also find that the Unite test mode can achieve the most stable result in different models.
In order to further study the role of different modules in our framework, we conduct extensive ablation experiments. The results are shown in Table 3. NADS w/o NOASP denotes that we only utilize
Models	14Rest	14Lap
Acc	Macro-F1	Acc	Macro-F1
NADS	87.49	82.09	82.12	79.13
NADS w/o NOASP	85.22	78.88	79.43	75.30
NADS w/o MAP	87.04	81.73	81.18	78.51
NADS w/o DS	87.22	81.71	81.01	77.26
Table 3: Experimental results of ablation study.
Figure 4: Effect of different  $m(pos, neg)$  while set  $m(pos, neu) = 0.4$  in laptop dataset.
the original sentence and remove the contrastive learning. Without contrastive learning between no-aspect template and original sentence, the sentiment bias of aspect perplexes the prediction result and more importantly, differential sentiment loss will not work because of the anisotropic in BERT model. Therefore, its performance is degraded on both two datasets. NADS w/o MAP means that we remove the masked aspect prediction module so that we may lose the original semantic information of the sentence. NADS w/o DS indicates that we utilize the cross-entropy loss function instead of our differential sentiment loss. Without differential sentiment loss, the model cannot find the different distances between sentiments. Experiments show that every module is indispensable in our NADS framework.
5.6 Selection of Margin
We experiment with different margins in the differential sentiment loss. In our framework, we only consider the  $m(pos, neu)$  and  $m(pos, neg)$ . Figure 4 shows the accuracy of different  $m(pos, neg)$  when we set  $m(pos, neu) = 0.4$  in our three methods based on our NADS framework on Laptop dataset. As we can see, the accuracy increases first and then decreases in the process of  $m(pos, neg)$  gradually increasing. The three models perform best when the  $m(pos, neg)$  is set to 0.6, 0.5 and 0.7. This experiment shows that the distance be
#	Case	BERT-SPC	AEN+BERT	DualGCN+BERT	NADS
1	I asked for a simple medium rare steak.	(P×)	(P×)	(P×)	(O✓)
2	Desserts include flan and sobaipillas.	(P×,P×,P×)	(P×,P×,P×)	(P×,P×,P×)	(O✓,P×,P×)
We started with the scallops and asparagus
3	and also had the soft shell crab as well as the cheese plate.	(O✓,P×,P×,P×)	(P×,P×,P×,P×)	(O✓,O✓,P×,P×)	(O✓,O✓,O✓,P×)
4	Try the rose roll ( not on menu).	(N×)	(N×)	(O✓)	(O✓)
5	There was only one waiter for the whole restaurant upstairs.	(N×)	(N×)	(O✓)	(O✓)
Figure 5: Distribution of bad cases of our NADS framework and BERT-SPC.
tween positive and negative is indeed farther than that between positive and neutral. It proves the effectiveness of our differential sentiment loss.
5.7 Sentiment Bias Elimination
In order to better understand the ability of our NADS framework to eliminate sentiment bias, we find several examples whose labels are neutral and show their prediction results in different models in Table 4, where  $P, N, O$  represent positive, negative, and neutral sentiments. We highlight the aspect words in red. We can see that our NADS framework outperforms all other models. For the aspect "steak" in the first sample, previous methods ignore the positive sentiment bias of "steak" and incorrectly predict the sentiment as positive. In contrast, our NADS eliminates the positive sentiment bias through no-aspect template and predicts the correct sentiment as neutral. Moreover, we also show the distribution of bad cases in Figure 5. The bad cases of neutral aspects terms in our NADS framework are significantly less than BERT-SPC. This proves the effectiveness of our NADS framework in eliminating sentiment bias. However, there are still some neutral aspect terms in our framework that are incorrectly predicted as shown in Table 4.
Table 4: Case study. Comparison of our NADS model to different baselines. Marker  $\checkmark$  indicates correct prediction while  $\times$  indicates incorrect prediction.
Models	Rest	Lap
Acc→ARS(Change)	Acc→ARS(Change)
BERT-PT	86.70→59.29(↓27.41)	78.53→53.29(↓25.24)
RGAT	84.41→56.54(↓27.87)	78.08→51.37(↓26.72)
BERT-SPC	83.04→54.82(↓28.22)	77.59→50.94(↓26.65)
NADS	87.49→64.55(↓22.94)	82.12→58.77(↓23.35)
AEN+BERT	83.12→25.45(↓57.67)	79.93→30.09(↓49.84)
AEN+NADS	84.00→26.61(↓57.39)	81.33→37.15(↓44.18)
DualGCN+BERT	87.13→63.57(↓23.56)	81.80→57.99(↓23.81)
DualGCN+NADS	87.49→66.16(↓21.33)	82.75→60.82(↓21.93)
Table 5: Our NADS performance on aspect robustness test set. We compare the accuracy on original and the new test sets, and calculate the change of accuracy.
Models	Strategy	Rest	Lap
Acc→ARS(Change)	Acc→ARS(Change)
NADS	Ori	87.49→64.55(↓22.94)	82.12→58.77(↓23.35)
Noasp	87.04→64.38(↓22.66)	81.01→59.56(↓21.35)
Unite	87.58→64.91(↓22.67)	81.96→60.19(↓21.77)
AEN+NADS	Ori	84.00→26.61(↓57.39)	81.33→37.15(↓44.18)
Noasp	86.51→60.00(↓26.51)	80.22→57.88(↓22.34)
Unite	86.68→56.34(↓30.34)	81.48→50.78(↓30.70)
DualGCN+NADS	Ori	87.49→66.16(↓21.33)	82.75→60.82(↓21.93)
Noasp	86.86→64.46(↓22.40)	81.49→60.03(↓21.46)
Unite	87.67→65.36(↓22.31)	82.75→60.66(↓22.09)
Table 6: Comparison of three test modes on aspect robustness test set.
A possible reason is that there may be other words in a sentence carrying the sentiment bias in addition to the current aspect.
5.8 Robustness Study
In order to verify the robustness of our NADS, we test the robustness score of our framework on the Aspect Robustness Test Set (ARTS) (Xing et al., 2020). The datasets enrich 14Lap and 14Rest according to three strategies: reverse the original sentiment of the target aspect (REVTGT), perturb the sentiments of the non-target aspects (REVNON) and generate more non-target aspect terms that have opposite sentiment polarities to the target (ADDDIFF). They take the original sentence and the three variants as an unit. Only if the original sentence and all variants are correct, the unit is correct. Calculate the accuracy of the units in the datasets as the final Aspect Robustness Score (ARS).
We compare the ARS of the three models be
fore adding our NADS framework and after adding NADS. The results in Table 5 show that the ARS of the model has been significantly improved after adding our NADS framework. Our NADS framework adding DualGCN performs significantly better than other models with  $21.33\%$  and  $21.93\%$  decline on Restaurant and Laptop. This shows that our framework utilizing human cognition has better robustness than other models.
Moreover, we utilize the three test modes to test on the ARTS as shown in Table 6. As we can see, the AEN+NADS model with  $57.39\%$  and  $44.18\%$  decline on Restaurant and Laptop when using Original test mode. However, with  $26.51\%$  and  $22.34\%$  decline when utilizing Noasp test mode. In the overall scheme, the Noasp test mode and Unite test mode can get a more stable result than the Original test mode on the ARTS. Utilizing no-aspect template in test may be a more stable robustness test method.
In this paper, we propose a NADS framework which is more in line with human cognition for the ABSA task. Our NADS framework utilizes no-aspect contrastive learning to eliminate the sentiment bias of aspects and enhance the sentence representations. In addition, we construct a differential sentiment loss to better classify the sentiments through distinguishing the different distances between sentiment polarities. Extensive experiments show that our NADS framework boosts three typical ABSA methods and outperforms baselines. Moreover, our NADS framework can still perform well even we don't know what the aspect is. The test on the robustness dataset shows that our NADS framework significantly improves the robustness of the model.
This paper is supported by Pilot Projects of Chinese Academy of Sciences (NO.XDC02030000).
Junqi Dai, Hang Yan, Tianxiang Sun, Pengfei Liu, and Xipeng Qiu. 2021. Does syntax matter? a strong baseline for aspect-based sentiment analysis with RoBERTa. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1816-1829, Online. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Gamaleldin Elsayed, Dilip Krishnan, Hossein Mobahi, Kevin Regan, and Samy Bengio. 2018. Large margin deep networks for classification. Advances in Neural Information Processing Systems, 31:842-852.
Andrea Esuli and Fabrizio Sebastiani. 2006. Sentiword-net: A publicly available lexical resource for opinion mining. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06).
Feifan Fan, Yansong Feng, and Dongyan Zhao. 2018. Multi-grained attention network for aspect-level sentiment classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3433-3442, Brussels, Belgium. Association for Computational Linguistics.
Jimin Hong, TaeHee Kim, Hyesu Lim, and Jaegul Choo. 2021. AVocaDo: Strategy for adapting vocabulary to downstream domain. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4692-4700, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Xiaochen Hou, Peng Qi, Guangtao Wang, Rex Ying, Jing Huang, Xiaodong He, and Bowen Zhou. 2021. Graph ensemble learning over multiple dependency trees for aspect-level sentiment classification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2884-2894, Online. Association for Computational Linguistics.
Binxuan Huang, Yanglan Ou, and Kathleen M Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, pages 197-206. Springer.
Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2020. Reducing sentiment bias in language models via counterfactual evaluation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 65-83, Online. Association for Computational Linguistics.
Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent Twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational
Linguistics: Human Language Technologies, pages 151-160, Portland, Oregon, USA. Association for Computational Linguistics.
Qingnan Jiang, Lei Chen, Ruifeng Xu, Xiang Ao, and Min Yang. 2019. A challenge dataset and effective models for aspect-based sentiment analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6280-6285, Hong Kong, China. Association for Computational Linguistics.
Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119-9130, Online. Association for Computational Linguistics.
Lishuang Li, Yang Liu, and AnQiao Zhou. 2018. Hierarchical attention based position-aware network for aspect-level sentiment analysis. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 181-189, Brussels, Belgium. Association for Computational Linguistics.
Ruifan Li, Hao Chen, Fangxiang Feng, Zhanyu Ma, Xiaojie Wang, and Eduard Hovy. 2021. Dual graph convolutional networks for aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6319-6329, Online. Association for Computational Linguistics.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Khalil Mrini, Franck Dernoncourt, Trung Bui, Walter Chang, and Ndapa Nakashole. 2019. Rethinking self-attention: An interpretable self-attentive encoder-decoder parser. CoRR, abs/1911.03875.
Judea Pearl et al. 2000. Models, reasoning and inference. Cambridge, UK: Cambridge University Press, 19.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th
International Workshop on Semantic Evaluation (Se-mEval 2014), pages 27-35, Dublin, Ireland. Association for Computational Linguistics.
Alexander Rietzler, Sebastian Stabinger, Paul Opitz, and Stefan Engl. 2020. Adapt or get left behind: Domain adaptation through BERT language model finetuning for aspect-target sentiment classification. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4933-4941, Marseille, France. European Language Resources Association.
James M Robins. 2003. Semantics of causal dag models and the identification of direct and indirect effects. Oxford Statistical Science Series, pages 70-82.
Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314.
Xingwei Tan, Yi Cai, and Changxi Zhu. 2019. Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3426-3431, Hong Kong, China. Association for Computational Linguistics.
Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016. Effective lstms for target-dependent sentiment classification. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3298-3307.
Hao Tang, Donghong Ji, Chenliang Li, and Qiji Zhou. 2020. Dependency graph enhanced dual-transformer structure for aspect-based sentiment classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6578-6588, Online. Association for Computational Linguistics.
Bo Wang, Tao Shen, Guodong Long, Tianyi Zhou, and Yi Chang. 2021. Eliminating sentiment bias for aspect-level sentiment classification with unsupervised opinion extraction. In *Findings of the Association for Computational Linguistics: EMNLP* 2021, pages 3002-3012, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Kai Wang, Weizhou Shen, Yunyi Yang, Xiaojun Quan, and Rui Wang. 2020. Relational graph attention network for aspect-based sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3229-3238, Online. Association for Computational Linguistics.
Xiaoyu Xing, Zhijing Jin, Di Jin, Bingning Wang, Qi Zhang, and Xuanjing Huang. 2020. Tasty burgers, soggy fries: Probing aspect robustness in aspect-based sentiment analysis. In Proceedings of the 2020
Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3594-3605, Online. Association for Computational Linguistics.
Chen Zhang, Qiuchi Li, and Dawei Song. 2019. Aspect-based sentiment classification with aspect-specific graph convolutional networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4568-4578, Hong Kong, China. Association for Computational Linguistics.
Zhilu Zhang and Mert R Sabuncu. 2018. Generalized cross entropy loss for training deep neural networks with noisy labels. In 32nd Conference on Neural Information Processing Systems (NeurIPS).
Pinlong Zhao, Linlin Hou, and Ou Wu. 2020. Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. Knowledge-Based Systems, 193:105443.