Is “hot pizza” Positive or Negative? Mining Target-aware Sentiment Lexicons (2021)

Abstract: Modelling a word’s polarity in different contexts is a key task in sentiment analysis. Previous works mainly focus on domain dependencies, and assume words’ sentiments are invariant within a specific domain. In this paper, we relax this assumption by binding a word’s sentiment to its collocation words instead of domain labels. This finer view of sentiment contexts is particularly useful for identifying commonsense sentiments expressed in neural words such as “big” and “long”. Given a target (e.g., an aspect), we propose an effective “perturb-and-see” method to extract sentiment words modifying it from large-scale datasets. The reliability of the obtained target-aware sentiment lexicons is extensively evaluated both manually and automatically. We also show that a simple application of the lexicon is able to achieve highly competitive performances on the unsupervised opinion relation extraction task.

## introduction sentiments of words can be subtle. we are used to using the same word to express different emotions in different contexts. "hot", for example, suggests a negative sentiment when commenting a computer hardware and a positive sentiment when commenting a pizza, even itself alone is identified without any general orientation. in these situations, it is the composition of a word, contexts, and commonsense carries an opinion. automatically detecting such context dependent sentiments would strengthen both our understanding of implicit opinions in languages and improve existing sentiment analyses models, which is the main topic of this work. to handle shifts of word sentiment, prior works studied how to adapt existing sentiment lexicons to new domains @xcite 2019). by modeling differences and similarities of text topics, they can detect new sentiments of words as the domain changes. the basic assumption of those domain-level sentiment lexicons is that a word keeps a consistent sentiment within a domain. this assumption, however, might be strong for fine-granularity analyses of text sentiments: words (especially, neural words such as "long", "fast") could exhibit different orientations even in the same domain (figure 1 ). to collect more detailed information of a sentiment, another branch of works (aspect-based sentiment analysis @xcite , opinion relation extraction @xcite ) attempt find answers of "who express what opinion on which target" for opinion bearing texts. existing solutions heavily rely on manual annotations and linguistic rules, which are either hard to scale-up or hard to be complete. in this work, we study the task of extracting target-aware sentiment lexicons. an entry of such lexicon is a pair of a sentiment word and a target word, and their collocation expresses a sentiment. it improves existing domain-dependent lexicons by being more concrete and accurate on describing opinions. departing from approaches adopted in existing aspect-based analyses, we aim to build context-aware lexicons by minimizing the requirement of annotations (e.g., only document-level sentiment labels) and errors from handcrafted patterns. our method starts from a target word (e.g., an aspect in product reviews), and extract sentiment words from its local context. the main strategy is to perturb context words and see how the sentiment of the target word changes: words with high influence on the target's sentiment hold high probability of forming a collocation with the target. we accomplish this by observing the behaviour of a well-trained document-level sentiment classifier when we change the contexts of the target word. two types of perturbations are examined, discrete perturbation which only requires a black-box classifier, and continuous perturbation which asks for network gradients. we collect evidences of each candidate pair on large datesets to ensure the reliability of the final lexicon. finally, the polarities of a lexicon entry can also be obtained by querying the sentiment classifier. on two online product review domains (electronic and restaurant), we evaluate the extracted target-aware lexicon both manually and automatically. quantitative and qualitative results show that the lexicons are reasonable to reflect common sentiment usage in each domain. as an application, we apply the lexicons to the task of unsupervised opinion relation extraction. the model performs significantly better than the baseline extractor, and even competitive with a recent supervised model on restaurant reviews. we summarize main contributions as follows, ## definitions and the task let d be a document with sentences s 1 , s 2 , ..., s |d| and @xmath0. foot_0 given a corpus @xmath1, y i )} |d| @xmath2.g., screen, pizza), foot_2 our task is to extract targetaware opinion words of t only using documentlevel sentiment labels. precisely, we aim to output a set of triples (t, o, p), where o is an opinion word commonly used to comment target t and @xmath3. we develop the lexicon extractor in three steps. first, we build an approximate target-level sentiment classifier (section 3) using document-level sentiment labels. second, for each sentence s containing target t, we calculate how important a word @xmath4.1 and 4.2). we aggregate scores of w over all its occurrences to get its confidence of being an opinion word of t. finally, we derive the polarity of w by querying the classifier with template sentences (section 5). ## approximating target-oriented opinion to identify target-aware opinion words, our key approach is to inspect how the opinion of a target changes when its context words change. hence, it is crucial to know the polarity of a target in documents. however, annotations in d are documentlevel: for a document, its sentiment label expresses overall sentiments for all targets in the document, rather than a specific one. for example, the restaurant review in figure 2 talks about 5 targets, each of them is commented by different opinion words with different polarities. in one of our datasets, 93% of documents contain multiple sentences (6 in average), and more than 82% contain multiple targets (7 in average). therefore, directly using document-level sentiment labels could be inappropriate for target-level analyses. on the other hand, it is quite expensive to annotate target-level sentiments, and existing datasets are far from enough for a robust commonsense opinion extractor. to deal with this problem, we borrow the idea of distant supervision @xcite : if a document is labelled as positive, at least one sentence (target) in it is positive. by seeing a large amount of positive documents, a classifier may be able to generalize patterns of their positive sentences, thus may help finding sentence-level (target-level) opinions. here we simply build a document-level sentiment classifier, and apply it on sentences to get pseudo target-level sentiment labels (for simplicity, we assume one sentence contains one target). advanced distant supervision models could also be applied, but we find this simple method preforms quite well in our experiments. to build the sentiment classifier, we fine-tune bert @xcite on d to encode domain specific semantics and augment it with a sentiment prediction task to encode sentiment information. for a document d, we feed its word sequence into bert and obtain a vector representation @xmath5, then we apply a softmax operator on d to get the probability of its sentiment p (y|d), for each sentence s containing t, we apply above classifier to predict pseudo sentiment label y p of s. in the following sections, we will rely on the set s @xmath6, y p )|@xmath7. ## importance scores we propose two score functions for measuring a context word w's influence on the target-oriented sentiment: one is discrete perturbation which only requires outputs of the sentiment classifier, another is continuous perturbation which needs network gradients. they are also called model-free and the cpu is very hot. ## polarity inference given the importance scores of words with respect to t, we can rank them accordingly and take the top-k words as t's opinion lexicon. as the final step, we are left to determine the polarity of an opinion word o. we accomplish this by building template sentences which try to carry the semantic like "what opinion on which target". we call these sentences template which will be use to probe the sentiment classifier's knowledge on (t, o)'s polarity. formally, define t to be a set of templates, each template τ ∈ t takes an opinion word and a target as input, outputs a natural language sentence τ (t, o). here, we use the following two templates, by feeding τ (t, o) into the sentiment classifier, we obtain p (y p |τ (t, o)), and the polarity distribution p of (t, o) is averaged over all templates, ## experimental results and analyses we wish to evaluate the merit of our target-aware sentiment lexicon in this section. we first introduce the experimental setup in section 6.1. then, we design detail experiments to answer the following key questions. ## conclusion in this paper, we propose a framework for automatic target-aware sentiment mining from texts without manual annotations or linguistic rules. we evaluate the proposed framework on two largescale online review domains: restaurant and electronic with both manual checking and automatic downstream tasks. we also achieve significant improvements by applying the opinion lexicon to the task of unsupervised opinion relation extraction. to investigate the insight of commonsense sentiment mined from the texts, we visualize several real-world examples and analyze them in-depth. the extensive experimental results demonstrate the excellent performance in building a target-aware sentiment lexicon.