Identifying Emotional and Polar Concepts via Synset Translation (2024)

Abstract: Emotion identification and polarity classification seek to determine the sentiment expressed by a writer. Sentiment lexicons that provide classifications at the word level fail to distinguish between different senses of polysemous words. To address this problem, we propose a translation-based method for labeling each individual lexical concept and word sense. Specifically, we translate synsets into 20 different languages and verify the sentiment of these translations in multilingual sentiment lexicons. By applying our method to all WordNet synsets, we produce SentiSynset, a synset-level sentiment resource containing 12,429 emotional synsets and 15,567 polar synsets, which is significantly larger than previous resources. Experimental evaluation shows that our method outperforms prior automated methods that classify word senses, in addition to outperforming ChatGPT. We make the resulting resource publicly available on GitHub.

## introduction emotion identification is the semantic task of analyzing a piece of text to identify a set of underlying emotions from a predefined inventory @xcite . polarity classification is the closely related task of determining the polarity of a text, which can be positive, negative, or neutral @xcite . these two tasks are variations on sentiment analysis, the extraction of sentiment that a writer expresses toward some object @xcite . following kakkonen and galić @xcite , we refer to a text, a word token, or a lexical concept as sentimental if it is associated with any emotion or non-neutral polarity. the scope of sentiment analysis can be a single word @xcite @xcite , a sentence @xcite , or longer texts such as twitter posts and customer reviews @xcite @xcite . in this paper, we focus on sense-level sentiment; knowing the sentiment of the individual words in a text can help determine its overall sentiment. emotion identification is more informative than polarity classification, but it is also more subjective in the sense that we would expect more disagreement among annotators. for example, determining that the word murder has a negative polarity is more objective than deciding which combination of emotions, such as anger, disgust, fear, and sadness, best relate to the word. this subjectivity is only heightened by the lack of consensus on the set of basic human emotions. researchers have proposed inventories of six @xcite , eight @xcite , or more fundamental emotions. therefore, while we explore both tasks, we place greater emphasis on polarity classification. since many emotion-bearing words are polysemous, we focus our attention on word senses and lexical concepts. senses are associated with one specific meaning of a word, so classifying sentiments at the level of senses avoids the ambiguity that arises from words having multiple meanings. in wordnet @xcite , sets of words that express the same concept are grouped together in synsets, each uniquely corresponding to a single concept. for example, the synset that contains the words sadness, sorrow, and sorrowfulness corresponds to the concept which is defined as "the state of being sad". synsets in wordnet are connected via various relations. a word can convey different sentiments depending on its sense in a given context; we assume that the sentiment associated with a specific sense/synset/concept is fixed. while it is true that the sentiment of a sense can too change depending on the context in which it is used, this ambiguity is much less prevalent among senses than it is among words. thus, by labeling a synset, we provide a single emotional label for all word senses in the synset. furthermore, we hypothesize that the sentiment of a given concept is likely to be the same in other languages. for example, the concept mentioned above is also expressed by the spanish word tristeza and the yoruba word ibanuje. we test this hypothesis by developing methods that classify english word senses by leveraging multilingual translations. conversely, we leverage english sentiment labels for other languages. in this paper, we outline the development of an automatic method that leverages multilinguality to identify sentimental concepts. unlike existing resources that were constructed by expanding a core of manually-annotated synsets, we propose a fully automatic method that can provide labels for a significantly larger number of synsets. our method achieves a precision of 96.0% and 92.0% on identifying emotional and polar synsets, respectively. of those, a correct emotional label is assigned with an accuracy of 84.3%, and a correct polarity label is assigned with an accuracy of 95.8%. the resulting resource, which we call sentisynset, contains 12,429 emotional and 15,567 polar synset labels. when used in conjunction with word sense disambiguation techniques, the resource could be useful for the downstream application of sentiment analysis at the level of sentences and documents. sen-tisynset is publicly available on github, together with our code. foot_0 ## related work in this section, we provide an overview of the related work on emotion identification and polarity classification at the synset level. our focus is on the resources based on the princeton wordnet @xcite , which consists of 117,659 synsets, each corresponding to a specific concept defined by its gloss. emotion identification wordnet-affect @xcite and @xcite associate a subset of wordnet synsets with emotional classifications. wordnet-affect contains 2,874 synsets, each associated with one or more of 32 emotions. it was constructed by first manually annotating a relatively small "core" of emotional synsets, which was later expanded by leveraging inter-synset relations in wordnet. sentisense encompasses 2,190 synsets labeled with one of 14 emotional categories. while its development is similar to that of wordnet-affect, they differ in their specific sets of manually annotated synsets and the wordnet relations chosen for extension. wordnet-affect and sentisense are built upon emotional inventories that are not only mutually incompatible but also rooted in separate psychological theories of emotion. this misalignment complicates data integration, consistency maintenance, and interpretation. meanwhile, reconciling the two resources by mapping their distinct emotion inventories remains problematic. for example, senses of the words abashed and upset are both identified with anxiety in wordnet-affect, but are respectively labeled with disgust and anger in sentisense; however, senses of the words embarrassment and nausea are both identified with disgust in sentisense, but are respectively labeled with shame and general-dislike in wordnet-affect. these discrepancies highlight the inherent subjectivity in emotion identification, thus motivating our prioritization of the more objective task of polarity classification. additionally, both resources provide limited coverage of wordnet of less than 3,000 synsets each; this limited coverage arises from their semi-automatic construction. we aim to address this problem by developing a scalable automatic method that can classify a much larger proportion of wordnet synsets. polarity classification sentiwordnet @xcite stands as a prominent resource for polarity classification. it assigns each synset a positive, negative, and objective score, with values ranging from 0.0 to 1.0, summing up to 1 across the three categories. these scores are produced by a committee of classifiers which leverage synset glosses. since the method is entirely automated, polarity scores are assigned to every wordnet synset. contrariwise, our method, while automated, is focused on precision, rather than coverage; we do not seek to label every synset, but rather aim to label as many synsets as possible with high confidence. ml-senticon @xcite attains polarity labels for synsets using a variation of the method used to create sentiwordnet. as such, the resource has the same drawbacks as sentiwordnet. in addition to the synset labels, ml-senticon also contains lemma-level lexicons for english, spanish, catalan, basque, and galician that were developed by averaging the polarity values of all synsets belonging to a lemma. while these are useful lexicons, particularly because of the inclusion of low-resource languages, assigning labels to lemmas introduces issues with polysemy. multilinguality chen and skiena (2014) leverage multilingual information to develop word-level polarity lexicons for 136 major world languages. they create graphs connecting words from these languages, considering both cross-language links, such as translations and transliterations, and intralanguage links, such as synonyms and antonyms. they propagate english word-level polarity labels across the graphs to create lexicons for the non-english languages. while these automatically developed lexicons have high levels of agreement with human-annotated lexicons, they still retain the ambiguity that arises when sentiment labels are assigned to words rather than senses. applications of synset lexicons synset-level lexicons can be used for sentiment analysis at the broader levels of sentences and documents @xcite . these works find that using synset lexicons in conjunction with word sense disambiguation techniques for english texts results in more precise sentiment predictions than those achieved using word-level lexicons. similar improvements were observed using synset lexicons to classify non-english text as well @xcite . thus, the resource we develop can be used with these existing methods to perform downstream sentiment analysis tasks in multilingual settings. ## methodology our method to create a large set of sentimentlabeled synsets (sentisynset) consists of two main stages. the first stage is to identify a set of emotional or polar synsets that we refer to as the core. in the second stage, this core is extended via word-net relations that preserve sentiment. our approach differs from prior works in that we create our core automatically, rather than manually. while assigning labels, we follow the precedent established in previously mentioned related works to map a synset to only one sentiment label. ## experimental setup in this section, we provide details of our implementation and the datasets that we use. ## results in this section, we evaluate our method's performance in the desired tasks and discuss the newly created resource. ## error analysis in this section, we investigate incorrect labels produced by our method and discuss possible causes and solutions for such errors. ## conclusion we have presented a novel method that leverages multilingual translations to shift the sentimental classifications of word-level lexicons from words to synsets. the method is sufficiently general to be applied to the related yet independent tasks of emotion identification and polarity classification. the method outperforms existing methods used to automatically construct resources for the task of polarity classification. with our method, we constructed sentisynset, which is substantially larger than comparable english sentiment resources. the large number of labeled synsets, and the high precision of labeling demonstrate the method's usefulness. the new resource can be paired with word-sense disambiguation techniques for the downstream task of sentiment analysis at the level of sentences or documents. since our method is not dependent on emolex, it could also leverage information from multiple word-level lexicons, which could further improve the quality and size of sentisynset.