Converting Sentiment Annotated Data to Emotion Annotated Data (2019)

Abstract: Existing supervised solutions for emotion classification demand large amount of emotion annotated data. Such resources may not be available for many languages. However, it is common to have sentiment annotated data available in these languages. The sentiment information (+1 or -1) is useful to segregate between positive emotions or negative emotions. In this paper, we propose an unsupervised approach for emotion recognition by taking advantage of the sentiment information. Given a sentence and its sentiment information, recognize the best possible emotion for it. For every sentence, the semantic relatedness between the words from sentence and a set of emotion-specific words is calculated using cosine similarity. An emotion vector representing the emotion score for each emotion category of Ekman’s model, is created. It is further improved with the dependency relations and the best possible emotion is predicted. The results show the significant improvement in f-score values for text with sentiment information as input over our baseline as text without sentiment information. We report the weighted f-score on three different datasets with the Ekman’s emotion model. This supports that by leveraging the sentiment value, better emotion annotated data can be created.

## introduction an emotion or a feeling represents a state of mind for any person. various researchers have put-forward classification of emotions into various categories such as plutchick emotion model with 8 basic emotions @xcite , the ekman's model with six basic emotions -anger, disgust, fear, happy, surprise, sadness @xcite and so on. users easily share their experiences, opinions, and emotions on various topics, product reviews on social platforms such as twitter, facebook, whatsapp. understanding the emotions expressed in such short posts can facilitate many important downstream applications such as an emotion-aware chatbots, analysis of user reviews, personalized recommendations, a help to psychologically ill patients, and so on. therefore, it is important to develop the effective emotion recognition models to automatically identify emotions from such text or messages. the task of emotion detection is typically modelled as supervised multi-class classification or multi-labelled classification task. supervised models need very large annotated data. such datasets may not be readily available and are costly to obtain @xcite . in case of unavailability of annotated data, unsupervised learning approaches @xcite can be an ideal solution for the emotion recognition. however, we assume that text annotated with sentiment information (positive, negative or neutral) is easily available. sentiment classification predicts positive or negative sentiment polarity of a sentence whereas emotion classification labels the sentence at fine-grain level with one of the emotions, such as happy, surprise, anger, fear, disgust, sadness etc. happy and surprise emotion are termed as the positive sentiment emotions and anger, disgust, fear, sadness as the negative sentiment emotions. for example in the sentence, passed an exam by two points . . . (+1) the sentiment information provided with sentence in above example, helps to confirm that the sentence is made with positive emotion such as happiness, surprise etc rather than the negative emotions. a close friend of mine have not contacted me long time. . . . (-1) sentiment value of -1 shows exclusion of positive emotions by reducing chances of emotion recognition system being confused them with positive emotions. the sentiment information helps to narrow down choices to one of the negative emotions for the sentence and it must be sadness, fear, anger, or disgust. therefore, we aim to use this sentiment information along with the sentence to create the emotion labelled dataset by recognizing emotion in unsupervised way. to be precise to create the emotion labelled resources with help of available sentiment labelled resources, as a further fine grained emotion analysis task. in this paper, we propose an unsupervised approach based on a agrawal (2012), with our modifications as discussed later in detail. we use the sentiment labelled data that is the sentence and the respective sentiment information as input and recognize the best possible emotion for the same. this approach uses word-embeddings to represent the words in the sentences as well as emotionspecific words. the cosine similarity measure is used to calculate the semantic relatedness between a sentence and the emotion-specific words. the vector representing score for every emotions category is calculated for each sentence and then emotion-score is modified using dependency relations of open-class words from that sentence. the rest of the paper is organized as follows. the section 2 describes related supervised, unsupervised and hybrid approaches previously proposed in the literature. the section 3 discusses the methodology of proposed system. section 4 briefs on the experimental set up, datasets used as well as modifications done to the dataset as required for experimentation. the results are discussed in section 5. ## related work the state of the art approaches for emotion recognition task is supervised approach. the labeled training data is a crucial resource required for building such systems. due to the lack of a large human annotated datasets, many emotion classification tasks have been performed on text data gathered from social media such as twitter, and the hash-tags, emojis or emoticons are used as the emotional labels for the same. as the unsupervised approaches do not need the annotated dataset, different unsupervised approaches are also performed by researchers. a @xcite found open-class words which they named as nava words that is noun, adjective, verb, adverb words. pointwise mutual information (pmi) based model with syntactic dependencies is used to perform emotion recognition. shoushan li and zhou (2015) created a dependence factor graph (dfg) as learning model based on label dependence and context dependence. the hybrid approaches use the unsupervised approaches for feature creation, pattern extraction which are later used by supervised classification models for emotion classification. carlos argueta (2015) had proposed an unsupervised graph-based approach for boot-strapping the twitter-specific emotion-bearing patterns and then used them for classification task. @xcite used predefined linguistic patterns to extract emotion causes and considered them as features for classification using svm. jianfei yu (2018) have used transfer learning approach for sentiment classification task and then emotion classification task. also few researchers have contributed towards creation of emotionaware embedding. distant supervision and recurrent neural network (rnn)-based approach is proposed for learning emotion-enriched representations.(ameeta agrawal, 2018) ## proposed system the emotion recognition framework for unsupervised approach is as shown in figure 1 . the input sentence is pre-processed to get the open-class words from that sentence. the second step is to compute the semantic relatedness using cosine similarity between word embedding of words in sentence and emotion-specific words. the module three modifies the emotion score for every emotion from vector computed at module 2. later, module 4 averages over emotion vectors of all words of a sentence to find resulting emotion present in that sentence. let s be the sentence, @xmath0, w 2 , • • •, w @xmath1. let e be the set of possible emotions from selected emotion model such as @xmath2, e 2 , • • •, e m } . to every emotion category, we have assigned few affect bearing words which represent that emotion. table-1 shows few affect-bearing words used for each emotion category. the aim is to predict the best possible emotion es belongs to e for the sentence s. eventually, one of the emotions from the ekman's emotion model-anger, disgust, fear, happy, sadness and surprise will be predicted for every sentence. ## experimental setup the datasets used for testing and recognizing emotions are isear dataset (ise), twitter emotion corpus @xcite and semeval-2018 affect in tweets english test dataset (saif m. mohammad, 2018). isear dataset (ise) the "international survey on emotion antecedents and reactions" dataset published by scherer and wallbott is built by collecting questionnaires answered by people with different cultural backgrounds @xcite . a total of 7,665 sentences labeled with single emotions. the labels are joy, fear, anger, sadness, disgust, shame, and guilt. twitter emotion corpus @xcite is prepared with emotion-word hashtags as emotion labels. these are termed as noisy labels as labelled by users. this corpus contains 21050 sentences labelled with one of the emotions from ekman's emotion model. semeval-2018 affect in tweets english test dataset (saif m. mohammad, 2018) is gold standard multi-labelled dataset with 3259 tweets annotated with multiple emotions. the emotion labels are anger, anticipation, disgust, fear, joy, love, optimism, pessimism, sadness, surprise, trust. every emotion is labelled as 0 or 1 to show presence of that emotion. if all are 0 then tweet is considered to be neutral. ## results and discussions as shown in table -3, the experiments are performed in two different ways, first by considering only text/sentences as input and secondly by considering text and its sentiment value as input. the experiments conducted with only sentences as input, serves here as baseline against experiments using sentences with sentiment information. the sentence with respective sentiment information as input shows significant improvement in weighted f-score value. the results are shown in the table-3. it is observed that google word2vec word vectors performs better than other word embedding. the semeval-2018 task-1 dataset (saif m. mohammad, 2018) is multi-labelled dataset. the annotated emotions are assigned independently. this task is multi-class emotion recognition so we consider prediction 'correct' even if one of the assigned emotions is predicted by our system. it is visible in table-4 that sentences with sentiment value as input, the f-score of every individual emotion category has been improved drastically. this shows better prospects for such emotion recognition and conversion process for new resource creation with fine-grained labeling from the sentiment to the emotion. recall values in case of the method with sentiment information has increased by approximately 50% than method without sentiment values. this shows significant improvement in correctly predicting emotion on use of sentiment information. the table-6 illustrates the confusion matrices for results of emotion recognition on isear dataset. it can be seen that positive emotions and negative emotions are rarely confused with each other by using sentiment information. the recall and precision is also increased for every emotion. yet emotions belonging to the same sentiment value need to be achieved with better accuracy. the emotion 'surprise' is not part of emotion labels for isear so 0s in row for 'surprise'.