Domain-Specific Sentiment Lexicons Induced from Labeled Documents (2020)

Abstract: Sentiment analysis is an area of substantial relevance both in industry and in academia, including for instance in social studies. Although supervised learning algorithms have advanced considerably in recent years, in many settings it remains more practical to apply an unsupervised technique. The latter are oftentimes based on sentiment lexicons. However, existing sentiment lexicons reflect an abstract notion of polarity and do not do justice to the substantial differences of word polarities between different domains. In this work, we draw on a collection of domain-specific data to induce a set of 24 domain-specific sentiment lexicons. We rely on initial linear models to induce initial word intensity scores, and then train new deep models based on word vector representations to overcome the scarcity of the original seed data. Our analysis shows substantial differences between domains, which make domain-specific sentiment lexicons a promising form of lexical resource in downstream tasks, and the predicted lexicons indeed perform effectively on tasks such as review classification and cross-lingual word sentiment prediction.

## introduction sentiment analysis is among the most prominent forms of natural language processing, with applications such as social media analytics @xcite @xcite , marketing and customer support @xcite , as well as recommendation @xcite . apart from machine learning-driven systems @xcite @xcite , which require supervision using labeled training data, there are also lexical resource-driven systems that exploit sentiment lexicons and can be run out-of-the-box without the need for any labeled training data. well-known sentiment lexicons include the @xcite opinion lexicon, senti-wordnet @xcite , liwc @xcite , and vader @xcite . there are numerous techniques for lexicon-driven sentiment analysis @xcite , sen-tistrength @xcite being an example of a more modern lexicon-driven sentiment analysis system. sentiment lexicons can also be used to bootstrap domain-specific supervised sentiment analysis models @xcite . a sentiment lexicon is a resource that, for a given word (form) w, provides an annotation label l w describing its overall sentiment polarity. some lexicons merely provide labels in {positive, negative} or {positive, neutral, negative}. others offer more informative intensity scores to account for the fact that some words are more negative or positive than others. for example, an emphatic word such as spectacular is generally considered stronger than a simple good (de @xcite . such scores could be in the range @xcite , with -1 denoting the most negative sentiment polarity, whereas +1 is the most positive score. in this paper, we consider two perennial problems with sentiment lexicons: to mitigate these shortcomings, we induce domain-specific sentiment lexicons using an automated data-driven approach. in our experiments, we consider a corpus of reviews from 24 different domains and first induce seed lexicons using linear predictors. subsequently, we extend their coverage based on large-scale word vector representations with a deep neural regression model. while our lexicons do not resolve the issues of context and polysemy -these are perhaps best addressed within a full-fledged machine learning architecture -many differences in sentiment polarity for a word stem from divergent uses across different domains. our experiments confirm that there are substantial differences between domains and that the predicted lexicons prove useful in review classification and cross-lingual word-level sentiment prediction. ## method our approach proceeds in two steps. first, we rely on labeled documents for a set of different domains to induce seed data for each of the domains using simple linear predictors. this seed data already accounts for the differences between domains. however, after the first step, the coverage of the resulting seed data is limited to words occurring in the labeled corpora, which may be small. hence, in a second step, we rely on deep neural models, exploiting vector representations of words to learn sentiment intensity scores for a much larger vocabulary. ## results in the following, we report on a series of experimental results to assess the merits of our proposal. in section 3.1, we induce seed data based on a large-scale review data set. in section 3.2, we then proceed with our domain-specific neural expansion approach. we first evaluate it on human-labeled data, and subsequently apply it to the complete vocabulary to induce large-scale domain-specific lexicons with high coverage. finally, in section 3.3, we evaluate the effectiveness of these induced domain-specific lexicons on review classification and cross-lingual word-level sentiment prediction. ## related work the traditional way of obtaining sentiment lexicons has been to build them manually, relying either on experts or invoking crowd-sourcing. a prominent example is the @xcite opinion lexicon. there are numerous algorithms that aim to increase the coverage of an individual sentiment lexicon. often, these start from seeds and then rely on graph-based algorithms to gather additional data, as for instance explored by @xcite and in the approach used to induce sentiwordnet @xcite . the extension can also be based on vector representations of words, as proposed in the densifier approach @xcite . such work has shown that dense word vectors trained on large amounts of data harbour signals that are useful for sentiment analysis. instead of a regular supervised setup, @xcite used distant supervision based on emoticons to obtain sentiment labels for entire sentences. they then trained a sentiment model on sentence vector representations sharing a common representation space with word vectors, which allowed them to apply the trained model to predict word-level scores. however, techniques such as the above mostly have not targeted domain-specific sentiment lexicons. the socialsent project @xcite induced reddit community-specific sentiment lexicons without labeled corpora. their sentprop approach constructs a graph of words and then considers random walks emanating from a small set of seed words with known sentiment polarity. the polarity scores are based on the frequency of random walk visits and the polarity of the seed word from which those random walks started. while reddit communities provide substantial diversity, the language used in reddit posts differs quite substantially from the kinds of language one encounters in reviews. @xcite adopted sentprop to customize an existing general-purpose sentiment lexicon for use in one specific domain. we instead focus on inducing a number of domain-specific lexicons to obtain a lexical resource that is more suitable for typical sentiment analysis use cases. the approach by @xcite also starts from labeled data for consumer products. it infers word polarity scores directly based on posterior probabilities and inverse document frequencies. however, such scores are limited to words that occur in the labeled training data. instead, in our work, we draw on word vectors to greatly enhance the coverage of the lexicons beyond the words present in the category-specific labeled data. our initial seed data approach is based on linear models optimized for maximum margin discrimination between the positive and negative classes, in line with the observations by @xcite , who found that linear models outperformed more sophisticated semi-supervised or transductive learning algorithms in their experiments. cross-lingual propagation of sentiment lexicons has been studied in a number of previous approaches. for example, dong and de melo (2018a) and dong and de melo (2018b) induced sentiment embeddings using translation graphs. in our experiments, we considered cross-lingual word embeddings for crosslingual transfer. ## conclusion in this paper, we present new domain-specific sentiment lexicons for a number of domains. we bootstrap this data from a large-scale review corpus covering 24 domains and then rely on a neural model to substantially extend its coverage. our analysis shows that there are substantial differences between domains, which make domain-specific sentiment lexicons an important form of lexical resource in downstream tasks. further experiments show that the predicted lexicons outperform domain-independent lexicons on unsupervised review classification and can also be used for cross-lingual word-level sentiment prediction. our data is freely available under an open source license from http://sentimentanalysis.org .