Context-sensitive Sentiment Propagation in W ord N et (2018)

Abstract: In this paper we present a comprehensive overview of recent methods of the sentiment propagation in a wordnet. Next, we propose a fully automated method called Classifier-based Polarity Propagation, which utilises a very rich set of features, where most of them are based on wordnet relation types, multi-level bag-of-synsets and bag-of-polarities. We have evaluated our solution using manually annotated part of plWordNet 3.1 emo, which contains more than 83k manual sentiment annotations, covering more than 41k synsets. We have demonstrated that in comparison to existing rule-based methods using a specific narrow set of semantic relations our method has achieved statistically significant and better results starting with the same seed synsets.

## introduction princeton wordnet @xcite has been expanded with sentiment annotation in several projects. however in all these approaches only a very limited part of princeton wordnet was manually annotated, and the annotation for the remaining part was automatically extended by propagation algorithms, e.g. wordnet-affect @xcite or sentiwordnet @xcite , see also sec. 3. manual emotive annotation was done for plwordnet @xcite (a wordnet of polish) on several times larger scale. in the most contemporary version more than 54 000 lexical units (i.e. word senses) are described by sentiment polarity, basic emotions and fundamental human values, cf. @xcite . only nouns and adjectives are annotated, but the manual annotation coverage of these two part-of-speech categories is almost 24%. having this large amount of metadata we started to look at methods of automated expansion of such information in a wordnet. most of the existing solutions are based on a set of handcrafted rules for transferring the polarity along different types of wordnet relations. the proposed method does not require manually designed rules as they are discovered automatically. ## background lexicons are an important, inherent part of sentiment analysis and opinion mining systems. there are three general approaches to compile sentiment lexicon i.e. corpus-based approach: dictionarybased and manual @xcite . manual approaches are laborious and time-consuming, so there is a great need for fast, automated methods of the construction of sentiment lexicons especially for low-resourced languages. the first built lexicons were limited only to simple word lists with positive and negative examples of words. however, the polarity of words often varies across their senses due to the semantic ambiguity. we assume that a sense-based sentiment lexicon may enable more accurate estimation of the sentiment polarity of complex phrases or sentences. one of the possible ways to construct a sense-aware sentiment lexicon is to use a wordnet (i.e. a dictionary-based approach). approaches of this kind of generally aim at extending a small set of seed words with known polarity using lexical relations of a wordnet, e.g. hypernymy, synonymy, antonymy, etc. most of the existing solutions rely on a simple polarity propagation from annotated synsets (seeds) to their not annotated neighbours, and mostly utilise only specific subset of relations like hypernymy, hyponymy, similarity and antonymy @xcite . these approaches do not take into account the full structure of wordnet or even wider contexts of synsets (e.g. n-th level relations). a common approach to construct a non-english sentiment lexicon is a simple translation of sentiwordnet @xcite polarity annotations to another language. simple rule-based propagation prepared for one language does not necessarily perform well for other languages, because wordnets for different languages may differ strongly, e.g. in the number of relation instances and a different semantic structure. on the other hand, corpus-based solutions require a high quality systems for word sense disambiguation. a good method for sentiment propagation should be adaptable to the structure of any wordnet with the least human effort. ## related works there is a vast amount of methods to construct sentiment lexicons, but most of them were evaluated only for english, on princeton wordnet @xcite . one of the major sentiment lexicons for english -sentiwordnet -was introduced in @xcite , and in @xcite its extended version was described. the main objective was to construct a large lexical resource with sentiment polarity of lexical meanings rather than words. one of the approaches based on a non-english wordnet was evaluated in @xcite . the authors compared three methods: the first method resulted in a general performance decrease in comparison to sentiwordnetfrom 62% to 58% of overall precision, recall and f-score. the second method was based on iterative label propagation with rules using lexical relations from wordnet. factors such as seed set size, its composition and number of iterations had a great impact on propagation performance. when high-quality pre-selected seed synsets are used, the obtained performance is significantly higher. one of the drawbacks of their approach is the simplicity of seed selection criteria. the best results were achieved using a mixed dataset derived from a large sentiment lexicon -the general inquirer lexicon @xcite . the performance reached 75% of f-score, precision and recall. the authors concluded that the size of a seed set is the most important factor, but the quality of the seeds also matters. almost the same performance was achieved by combining transfer method with propagation (74%). the results may also suggest that simple transfer methods are not perfect, but combining multiple approaches with transfer methods may bring us a promising result. extended research on the polarity propagation for non-english wordnets was presented in @xcite . authors applied the same propagation algorithm to five wordnets for different languages. the propagation method was similar to the methods used in their previous works. words and their polarity extracted from the wellknown general inquirer lexicon were translated with a machine translation service and manually mapped to the corresponding synsets in particular wordnets. the seed set consisting of synsets with known polarity was expanded using wordnet relations to cover the entire networks. the resulting lexicons varied significantly in their size and precision score. the conclusion was that the way the wordnets are built seems to affect propagation performance. @xcite is a first attempt to build an arabic sentiment lexicon on a basis of arabic wordnet. propagation procedure involves an expansion step which is expanding the sentiment lexicon by iteratively reaching concepts of the wordnet and scoring step evaluating the sentiment score of reached concepts according to their distance from the seeds. a task-based evaluation was applied. the acquired polarity scores were incorporated into features for sentiment classification task evaluated on arabic corpora. there were several attempts to construct a large sentiment lexicon for polish in an automated way e.g. @xcite . @xcite attempted to build a polarity lexicon from web documents. they utilized plwordnet (yet without sentiment annotation) as a general resource to develop domainaware polarity lexicons. a large semantic lexicon with over 70,000 concepts from web reviews was built where each term in this lexicon was described by a vector of sentiment values, representing the polarity of this term in various domains. plword-net was utilised to identify semantic relations between acquired terms. to determine their polarity a supervised learning with naive bayes and svm was applied. this approach was extended in @xcite where the semantic lexicon was expanded to 140,000 terms. to enlarge the lexicon the authors used a simple rule-based propagation with an adaptation of random walk algorithm. sentiwordnet construction in its recent stages was generally based on the glosses from princeton wordnet. @xcite proposed a lexicon construction method for wordnets, for which a simple transfer method could not be easily applied or external sources of knowledge such as tagged and disambiguated glosses are not available. this approach was based on relational propagation scheme with local, collective classification method, namely iterative classification algorithm (ica) for determining polarity of synsets. the training features for the classifier were obtained using only a neighbourhood of annotated synsets, consisting of nodes with known polarity. they manually annotated specific synsets in the wordnet and used them as seeds for the propagation process. however, the details of the feature extraction were not specified and there was no evaluation for their approach. in @xcite the propagation was performed by using an adaptation of loopy belief propagation (lpa) on princeton word-net 3.0. three different variants of the lpa have been tested and evaluated. the evaluation was carried out in two ways. firstly, the authors compared their results with polarity scores from sentiword-net (mean square error), but skipping the objective class. secondly, evaluation was done by comparison with polarity of words existing in the general inquirer lexicon. the resultant performance was ambiguous, and the main conclusion was that semantic relations within wordnet may not be a well correlated with the sentiment relations. ## classifier-based polarity propagation we propose a fully automated method called classifier-based polarity propagation (henceforth cpp) with a very rich set of features. in sec-tion 5.1 we compare the results obtained by cpp with rule-based and relation-based method called seed propagation and its best configuration presented by @xcite . ## conclusions the results prove that the proposed method performs better in almost all cases comparing to simple rule-based methods which transfer known polarity from seeds to other parts of wordnet. surprisingly for us, the solution with sorting synsets in each iteration in descending order by the number of neighbours with known polarity did not provide any increase of propagation quality. we think that the further work should be concentrated on training the classifier after each iteration and in this scenario sorting before classifying should be beneficial.