Eliminating Sentiment Bias for Aspect-Level Sentiment Classification with Unsupervised Opinion Extraction (2021)

Abstract: Aspect-level sentiment classification (ALSC) aims at identifying the sentiment polarity of a specified aspect in a sentence. ALSC is a practical setting in aspect-based sentiment analysis due to no opinion term labeling needed, but it fails to interpret why a sentiment polarity is derived for the aspect. To address this problem, recent works fine-tune pre-trained Transformer encoders for ALSC to extract an aspect-centric dependency tree that can locate the opinion words. However, the induced opinion words only provide an intuitive cue far below human-level interpretability. Besides, the pre-trained encoder tends to internalize an aspect’s intrinsic sentiment, causing sentiment bias and thus affecting model performance. In this paper, we propose a span-based anti-bias aspect representation learning framework. It first eliminates the sentiment bias in the aspect embedding by adversarial learning against aspects’ prior sentiment. Then, it aligns the distilled opinion candidates with the aspect by span-based dependency modeling to highlight the interpretable opinion terms. Our method achieves new state-of-the-art performance on five benchmarks, with the capability of unsupervised opinion extraction.

## introduction aspect-based sentiment analysis (absa) @xcite aims to determine sentiment polarity w.r.t. a specified aspect term in a piece of text. for example, in "the food is tasty, but the service is terrible", the sentiment towards aspect term (at) "food" is positive according to the opinion term (ot) "tasty", while the sentiment towards "service" is negative according to "terrible". the most comprehensive setting of absa is aspect sentiment triplet extraction (aste) @xcite consisting of a series of subtasks, i.e., aspect extraction, aspect-level sentiment classification and opinion extraction. thereby, given a piece of text, aste can produce a set of triples, i.e., (aspect term, sentiment, opinion term), to describe sentiment with details of what, how and why, so it enjoys full interpretability. continuing the above example, aste can generate triples like (food, positive, tasty). however, the human annotation on opinion terms is much more label-intensive than traditional sentiment analysis task. therefore, by following many recent works, we target the practical subtask of aste, called aspectlevel sentiment classification (alsc). it predicts a three-categorical sentiment (i.e., positive, neutral or negative) of a given aspect term in a sentence. most recent works capture the modification relation between aspect and opinion terms in an implicit manner, which is usually achieved by integrating graph neural networks (gnns) over dependency parsing tree into text representation learning @xcite @xcite . further, the performance can be significantly boosted when incorporating pre-trained transformer encoder, e.g., bert @xcite in a fine-tuning paradigm @xcite @xcite . although these works achieve excellent results even close to humans, they cannot derive the interpretability to explain why an aspect is associated with the polarity prediction. luckily, pre-trained transformer encoder can also be used to explain linguistic knowledge underlying the given text via dependency probing @xcite . this has been exploited by @xcite to reveal that, after fine-tuning the pre-trained encoder on alsc, an aspect-centric dependency tree can be induced to highlight the modifier of an aspect. intuitively, the highlighted modifier is viewed as an opinion word of the corresponding aspect, which thus, to some extent, brings the interpretability back. nevertheless, compared with the span-formatted opinion terms in aste, the opinion word illustrated by the induced dependency tree can only pro-vide an intuitive, noisy, word-level sentiment cue but is far from the human-level interpretability as in opinion extraction. what's worse, as verified by @xcite , the transformer pre-trained on large-scale raw corpora tends to internalize terms' intrinsic attributes, so it causes the problem of sentiment bias when generating text given a particular term prompt. in alsc scenario, we found that sentiment bias also exists and affects a model to determine the sentiment of an aspect term regardless of its contextual information (e.g., opinion terms). the bias is especially obvious for the aspects that can imply strong sentiment themselves. for example, for "there's candlelight and music", a model based on pre-trained transformer is likely to misclassify the sentiment towards "music" as positive, whereas the oracle label is neutral. in this work, based on the pre-trained transformer encoder to ensure state-of-the-art alsc performance, we aim to eliminate sentiment bias in the alsc scenario while equip the model with human-level aspect-opinion interpretability. to this end, we propose a span-based anti-bias aspect representation learning (sarl) framework for alsc with unsupervised opinion extraction. first, instead of widely feeding a concatenation of sentence and aspect into a pre-trained transformer, we adopt a span-level paradigm @xcite which focuses on deriving span representation of an aspect term. then, we propose an anti-bias aspect encoding module to eliminate the sentiment bias existing in aspect representations, which is achieved by an adversarial learning against the aspect's prior sentiment indicated in sentiwordnet @xcite . next, built upon sentiment-agnostic aspect representation from the above encoder, we propose an aspect-opinion dependency alignment module to capture explicit modifications from opinion term candidates to the targeted aspect, and integrate the modifications into aspect representation via gating. the integrated representation is lastly passed into a neural classifier for sentiment prediction. for pairwise aspect-opinion alignment via a span-based model, our work share a high-level inspiration with spanmlt @xcite but differs in that, spanmlt targets fully-supervised aspect and opinion terms extraction whereas ours leverages the alignment to empower aspect-opinion interpretability even without opinion supervisions. our main contributions are as follows: ## methodology this section begins with a task definition of alsc. then, we present our span-based anti-bias aspect representation learning (sarl) framework (as in figure 1 ) consisting of an adversarial anti-bias aspect encoder ( §2.1) and a distilled aspect-opinion dependency aligner ( §2.2). lastly, we detail the training and inference of the proposed model ( §2.3), including unsupervised opinion extraction. task definition. given a sentence with n words, @xmath0, . . . , x n ], alsc aims to predict a threecategorical sentiment from {p ositive, n eutral, n egative} of an aspect term, a, where a is a span of x, from s a and to e a , i.e., @xmath1:e a . ## conclusion in this work, we propose sarl framework for alsc. specifically, we first present an adversarial anti-bias aspect encoder to eliminate sentiment bias in aspects and then propose an aspect-opinion dependency aligner to unsupervisedly extract opinions. the experiments on 5 benchmarks can greatly support our motivations and empirical results show state-of-the-art performance with interpretability.