The L i L a H Emotion Lexicon of C roatian, D utch and S lovene (2020)

Abstract: In this paper, we present emotion lexicons of Croatian, Dutch and Slovene, based on manually corrected automatic translations of the English NRC Emotion lexicon. We evaluate the impact of the translation changes by measuring the change in supervised classification results of socially unacceptable utterances when lexicon information is used for feature construction. We further showcase the usage of the lexicons by calculating the difference in emotion distributions in texts containing and not containing socially unacceptable discourse, comparing them across four languages (English, Croatian, Dutch, Slovene) and two topics (migrants and LGBT). We show significant and consistent improvements in automatic classification across all languages and topics, as well as consistent (and expected) emotion distributions across all languages and topics, proving for the manually corrected lexicons to be a useful addition to the severely lacking area of emotion lexicons, the crucial resource for emotive analysis of text.

## introduction emotion lexicons are rather scarce resources for most languages @xcite , although they are a very important ingredient for robust emotion detection in text @xcite . they are mostly differentiated between depending on the way they encode emotions -either via continuous or discrete representations @xcite . in this work, we present emotion lexicons for three languages -croatian, dutch and slovene, developed by manually correcting automatic translations that are part of the nrc emotion lexicon @xcite . in that lexicon, sentiment (positive or negative) and a discrete model of emotion covering anger, anticipation, disgust, fear, joy, sadness, surprise and trust, are encoded via a binary variable for each emotion. the size of the lexicon is 14,182 entries, and automatic translations of that lexicon were performed by the authors in 2017 via google translate. the three languages in question are generally not covered well with emotional lexicons. the first lexical dataset encoding emotion for croatian was published recently @xcite , encoding valence and arousal for 3,022 words. for dutch two resources exist -the liwc (tausczik and pennebaker, 2010) translation into dutch @xcite covering around 4,500 entries, and the norms for 4,300 dutch words for dominance, valence and arousal @xcite . there are no available emotion lexicons for the slovene language except the automatic translation of the nrc lexicon. just recently, an automatic approach to building emotion lexicons for many languages beyond mere translation has emerged @xcite and its results cover all three languages in question, showing reasonable correlations with the available small psychological norms in various languages. ## the lexicon translation procedure we base our approach to building reliable emotion lexicons for the three languages in question on a very simple approach -we manually correct automatic translations of an existing english emotion lexicon. these automatic translations consist to the greatest part of single-word translations, only rare entries being translated with short phrases. we used different translators for each language as our aim was for the translations to be performed by native speakers. we used a uniform translation methodology across the three languages, ensuring comparability between the resulting lexicons. the guidelines presented to the translators were the following: while we decided to correct translations of all entries for croatian and slovene, for dutch we only corrected those entries that were marked for emotion, i.e., they had at least one positive value among the two sentiment and eight binary variables encoded in the lexicon. the reason for this decision was the limited availability of the dutch translator and the fact that the emotion-marked part of the lexicon is most useful for various applications. given this decision, the resulting lexicons of croatian and slovene have 14,182 entries, while the dutch version has only 6,468 entries. translating the lexicons resulted in a different level of modifications in each of the three languages. in ## analyses in this section, we present two types of analyses of the manually corrected lilah emotion lexicons -one based on machine learning that compares the original automatically translated nrc and the manually corrected lilah lexicon, and another based on descriptive statistics, exploiting only the lilah emotion lexicon. in the first analysis, we measure the impact of the corrections performed in the lexicon on the task of supervised classification of messages potentially containing socially unacceptable discourse (sud; hate speech being one type of sud). in the second analysis, we perform a descriptive analysis of emotion difference between socially unacceptable and socially acceptable discourse (sud and non-sud onwards), showcasing the potential of the new lexicon for data analysis. we perform both analyses on the same set of datasets with manual annotations of sud -the frenk croatian, english and slovene datasets and the lilah dutch dataset. all four datasets consist of comment threads of mainstream media facebook posts in croatia, great britain, slovenia and flanders. the comment threads in the datasets belong to two topics: migrants and lgbt. the size of the datasets varies, per language and topic, between 3836 and 6941 comments. each comment in a thread is annotated with an annotation schema encoding the type and target of sud @xcite . in this work, we only use the basic information whether a comment contains sud or not. the percentage of sud comments in the datasets lies between 32% (dutch lgbt) and 64% (croatian lgbt). ## conclusion in this paper, we presented the lilah emotion lexicons -manually corrected automatic translations of the nrc emotion lexicon into croatian, dutch and slovene. we have shown that the manually corrected lexicons are more potent in a supervised learning scenario where the lexicon information is used for data representation. we have additionally showcased a usage scenario of the lilah lexicons for emotion text analysis -we analysed emotional differences between texts containing socially acceptable and unacceptable discourse, showing a large amount of consistency across languages and topics, with some interesting peculiarities. the presented lexicons are available for download at http://hdl . handle.net/11356/1318 @xcite .