Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training (2021)

Abstract: Aspect-based sentiment analysis aims to identify the sentiment polarity of a specific aspect in product reviews. We notice that about 30% of reviews do not contain obvious opinion words, but still convey clear human-aware sentiment orientation, which is known as implicit sentiment. However, recent neural network-based approaches paid little attention to implicit sentiment entailed in the reviews. To overcome this issue, we adopt Supervised Contrastive Pre-training on large-scale sentiment-annotated corpora retrieved from in-domain language resources. By aligning the representation of implicit sentiment expressions to those with the same sentiment label, the pre-training process leads to better capture of both implicit and explicit sentiment orientation towards aspects in reviews. Experimental results show that our method achieves state-of-the-art performance on SemEval2014 benchmarks, and comprehensive analysis validates its effectiveness on learning implicit sentiment.

## introduction aspect-level sentiment analysis (absa) is a finegrained variant aiming to identify the sentiment polarity of one or more mentioned aspects in product reviews. recent studies tackle the task by either employing attention mechanisms @xcite or incorporating syntax-aware graph structures @xcite @xcite @xcite . both methodologies aim to capture the corresponding sentiment expression towards a particular aspect, which is usually an opinion word that explicitly expresses sentiment polarity. for instance, given the review on a restaurant "great food but the service is dreadful", current models attempt to find "great" for aspect "food" to determine the positive sentiment polarity towards it. ## implicit sentiment as sentiment that can only be inferred within the context of reviews, many researches address the presence of implicit sentiment in sentiment analysis. toprak et al. (2010); russo et al. (2015) proposed similar terminologies (as implicit polarity or polar facts), and provided corpora containing implicit sentiment. deng and wiebe (2014) detected implicit sentiment via inference over explicit sentiment expressions and so-called 1 https://github.com/tribleave/scapt-absa goodfor/badfor events. choi and wiebe (2014) used +/-effectwordnet lexicon to identify implicit sentiment, by assuming sentiment expressions are often related to states and events which have positive/negative/null effects on entities. to investigate the ubiquitous of implicit sentiment in absa, we split semeval-2014 restaurant and laptop benchmarks into explicit sentiment expression (ese) slice and implicit sentiment expression (ise) slice, based on the presence of opinion words. fan et al. ( 2019 ) have annotated opinion words for target aspects on semeval benchmarks. we notice that the provided datasets do not keep the original order and have some differences in texts. thus, we first match the annotations to the original datasets, and then manually pick the reviews including opinion words towards the aspect from the remaining part. as results shown in ## methodology in this section, we introduce the pre-training and fine-tuning scheme of our models. in pre-training, we introduce supervised contrastive pre-training (scapt) for absa, which learns the polarity of sentiment expressions by leveraging retrieved review corpus. in fine-tuning, aspect-aware finetuning is adopted to enhance the ability of models on aspect-based sentiment identification. ## results and analysis this section mainly demonstrates the experiment results. our model achieves state-of-the-art on three absa benchmarks, and we illustrate the representation alignment effect of supervised contrastive learning and the effectiveness of other parts from several perspectives. moreover, we reveal that our model is capable to identify implicit sentiment, and attributes its effectiveness to supervised contrastive learning in scapt. ## related work neural network methods for absa the early neural network methods @xcite in absa employed various of attention mechanisms to identify aspect-related context. memory network @xcite @xcite was further proposed to identify corresponding sentiment expression for aspects. recent efforts @xcite ) used syntax information from dependency trees to enhance attention-based models. a lot of works @xcite @xcite ) make use of graph neural networks to incorporate tree-structured syntactic information and capture aspect-related information in text. another line in absa concentrated on utilizing external corpus and pre-trained knowledge to enhance semantic awareness of models @xcite @xcite . contrastive representation learning our work adopts contrastive method in representation learning to acquire discriminating instance representations. recent work on contrastive representation learning of instances usually based on estimating representation similarities on similar and dissimilar pairs, which are usually composed in a self-supervised manner @xcite . specially, @xcite illustrated a supervised contrastive method to build positive pairs between instances with same class label, and put their representations together. in this work, our models learn to capture implicit sentiment from informative but noisy language resources in supervised contrastive pre-training. ## conclusion in this paper, we introduce supervised contrastive pre-training (scapt) for absa. by noticing that implicit sentiment is not well-handled by current neural network based absa models, we argue that more sentiment knowledge is required to solve this issue. we therefore retrieve large-scale indomain annotated corpora, and propose scapt to learn sentiment knowledge from the corpora. experimental results show that our proposed models with scapt achieve sota performance. moreover, scapt is proven to be effective in implicit sentiment learning. we hope to inspire future researches on learning and modeling implicit sentiment with knowledge-enhanced methods.