Propagation of emotions, arousal and polarity in W ord N et using Heterogeneous Structured Synset Embeddings (2019)

Abstract: In this paper we present a novel method for emotive propagation in a wordnet based on a large emotive seed. We introduce a sense-level emotive lexicon annotated with polarity, arousal and emotions. The data were annotated as a part of a large study involving over 20,000 participants. A total of 30,000 lexical units in Polish WordNet were described with metadata, each unit received about 50 annotations concerning polarity, arousal and 8 basic emotions, marked on a multilevel scale. We present a preliminary approach to propagating emotive metadata to unlabeled lexical units based on the distribution of manual annotations using logistic regression and description of mixed synset embeddings based on our Heterogeneous Structured Synset Embeddings.

## introduction rapid growth of interest in sentiment analysis comes from its vast potential in automatic detection of subjectivity (whether the text expresses a subjective opinion rather than an objective fact) and polarity (whether the expressed opinion is positive, negative, or neutral) in large amount of textual data. for instance, sentiment analysis systems proved useful in automatic analysis of many different kinds of textual data, such as emails, tweets, blogs, reviews, newspaper headlines or novels @xcite . whereas introduction of advanced computational methods (e.g. machine learning) to natural language processing resulted in the development of sentiment analysis methodologies, the scarcity of high quality and large scale data sources greatly constrains their usage. numerous attempts were made to annotate words in terms of emotions for various languages @xcite . however, such datasets are typically limited in size and consist of several thousands of words, while natural lexicons are known to be much bigger. since annotations are provided manually by either qualified experts (usually 2-3 independent annotators) or a group of naive participants, data collection is typically very expensive in terms of time and money. therefore, most of the available resources describe word meanings in terms of polarity, without further distinguishing various emotion categories attributed to them. in emotion research, words are usually characterized according to two dominant theoretical accounts on the nature of emotion: dimensional account and categorical account. according to the first one proposed in @xcite , each emotional state can be represented by its location in a multidimensional space, where valence or polarity (ranging from negativity to positivity) and arousal (from low to high) explain most of the observed variance. a competing account distinguishes several basic categories of emotional states, with more complex, subtle emotion states emerging as their combination. there have been various interpretations of the basic emotions concept, and different numbers of emotion categories were proposed by different theories, with @xcite and @xcite gaining most recognition in the scientific community. in this work, we used a large dataset described in @xcite , containing metadata for a total of over 30000 word meanings from polish wordnet @xcite , annotated in terms of polarity, arousal, as well as 8 basic emotion categories (i.e. joy, sadness, trust, disgust, fear, anger, surprise, anticipation). here, we present a novel propagation approach to automatically extend the original dataset by deriving emotion metadata for lexical units and synsets that are not present in this dataset. if effective, our approach could alleviate the problem of data scarcity and facilitate the widespread use of sentiment analysis in various applications including but not restricted to artificial intelligence, computational linguistics, psychology or business. ## dataset description in the sentimenti database @xcite , a total of over 20,000 unique respondents (with approximately equal number of male and female participants) was sampled from polish population. multiple demographical characteristics such as: sex, age, native language, place of residence, education level, marital status, employment status, political beliefs and income were controlled. the annotation schema was based on the procedures most widely used in previous studies, aiming to create the first standardized datasets of polish words characterized in terms of emotion @xcite ; nawl be, @xcite ; plwordnet-emo @xcite ). thus, we collected annotations of valence (polarity), arousal, as well as eight emotion categories: joy, sadness, trust, disgust, fear, anger, surprise and anticipation. by combining simple annotation schema with crowd annotation, we were able to effectively acquire large amount of data and preserve its high quality at the same time. the total number of over 30,000 word meanings was annotated, with each meaning ranked at least 50 times on each scale. moreover, in a followup study a total number of over 7,000 texts (short phrases or paragraphs of text) were annotated in the same way, with each text assessed at least 25 times on each scale. before attempting the assessment task, subjects were instructed to rank word meanings rather than words, as well as encouraged to indicate their immediate, spontaneous reactions. participants had unlimited time to complete the task and they were able to quit the assessment session at any time and resume their work later on. the source of texts were reviews from two domains: medicine foot_0 (2000 reviews) and hotels foot_1 (2000 reviews). due to difficulties in observing neutral reviews in the selected sources, we have chosen them from websites describing medical information foot_2 (500 paragraphs) and the hotel industry foot_3 (500 paragraphs). we also selected phrases using lexico-semantic-syntactic patterns (lss) manually created by linguists to capture one of the four effects affecting sentiment: increase, decrease, transition, drift. most of these phrases belong to the previously mentioned subject areas. the source for the remaining phrases were polish wordnet glosses @xcite . ## emotive propagation in this study, we decided to follow the idea presented in @xcite . the idea is to apply a semi-supervised learning on a large seed of emotively annotated synsets. the seed is used to train a classifier and then to predict the polarity categories for unlabeled synsets in a close vicinity of labeled ones. starting from the labeled synsets we visit their neighbors and annotate them iteratively by applying our classifier. the propagation process ends when all synsets become annotated. in @xcite the authors proposed a rich set of wordnet-based features to describe the synsets representing an initial seed for emotive propagation. for every synset existing in a seed they extracted the features capturing the structure of its neighborhood by taking into account the neighboring synsets (with their relative location) and assigned polarities. the features were generated on a basis of a template being defined in terms of 4 feature variables. a single feature is generated by initializing the variables in the template with a specific combination of possible values. the variables in the template are defined as follows: extracted features were converted to a bag-ofwords model, where the elements of a bag were representing the synsets or their polarities. then the authors used these features as a signal for a classifier to decide whether a given synset should be positive, negative, neutral or ambiguous. such an approach generated vectors of very large dimensions, thus it increases the overall propagation time. the classification procedure was time-consuming because the process of feature generation produced high-dimensional data. ## results and discussion comparing the f1 scores of the model in a task of valence, arousal and emotion propagation, we can observe that the valence propagation performs better than the propagation of most emotive dimensions ( ## conclusions the results of the propagation task suggest that the novel method presented here can be useful for both valence and emotions datasets. heterogeneous structured synset embeddings allow for effective scaling up of annotated datasets. the paper also describes a large dataset: "sentimenti" which covers more than 30,000 wordmeanings in polish annotated with 8 basic emotions as well as polarity and arousal. the data gathered in the psycholinguistic study were used to enhance affective annotation of polish wordnet @xcite . tthe emotive metadata were propagated to unlabeled lexical units, enabling emotive categorization of the whole wordnet. categorization results proved that the ml methods trained on the data enriched with detailed description of synset features and relations between lexical units and synsets resulted in effective models for emotional metadata propagation. such metadata propagation methods are successful only when based on large data sets and multilevel annotations of a given wordnet. in the current study, we showed that their effectiveness can be high also for very subjective emotional features, especially valence propagation. emotional metadata propagation for polish proved to have a high accuracy for most of the emotion values. however, a question remains why some of the emotion values were attributed less effectively -whether it was the lack of input data or it is a property of those emotions expressed verbally to be less evident and more dispersed.