EMO - KNOW : A Large Scale Dataset on Emotion-Cause (2023)

Abstract: Emotion-Cause analysis has attracted the attention of researchers in recent years. However, most existing datasets are limited in size and number of emotion categories. They often focus on extracting parts of the document that contain the emotion cause and fail to provide more abstractive, generalizable root cause. To bridge this gap, we introduce a large-scale dataset of emotion causes, derived from 9.8 million cleaned tweets over 15 years. We describe our curation process, which includes a comprehensive pipeline for data gathering, cleaning, labeling, and validation, ensuring the dataset’s reliability and richness. We extract emotion labels and provide abstractive summarization of the events causing emotions. The final dataset comprises over 700,000 tweets with corresponding emotion-cause pairs spanning 48 emotion classes, validated by human evaluators. The novelty of our dataset stems from its broad spectrum of emotion classes and the abstractive emotion cause that facilitates the development of an emotion-cause knowledge graph for nuanced reasoning. Our dataset will enable the design of emotion-aware systems that account for the diverse emotional responses of different people for the same event.

## introduction emotion-cause analysis is gaining interest due to its potential impact on applications such as empathetic dialog systems and mental health support chats. it is more challenging than traditional emotion recognition in text because it requires a higher level of semantic understanding @xcite @xcite . most existing works @xcite @xcite model emotion cause analysis as an extraction problem, resulting in datasets that provide only specific, descriptive information on emotion causes and lack deeper, abstract causes. for example, in a tweet, a user expresses their emotion by stating, "i am often on social because i feel very lonely in real life. my friends suck, my relatives suck even more and i have nobody in my life. even my dog was taken away, so if you are wondering about how i am feeling? yes, i am lonely." the second sentence is typically annotated as the emotion's cause in existing datasets, but this fails to highlight the more generalized root cause. this way of annotation, however, fails to highlight the more generalized root cause, which in this case is the lack of fulfilling relationships in the person's life. this level of abstraction is essential for comprehensive emotioncause understanding and reasoning. a further limitation of existing datasets lies in the coarse granularity of their emotion categories. modern theories of emotion suggest that emotions are not simply reducible to a basic set of 6-8 emotion categories @xcite . therefore, labeling user's emotions with a limited set of emotions will either miss out on many other emotions or forcefully label their emotions as something else. for this reason, a finer granularity of emotion categories would better reflect the true diversity and nuance of human emotions. for instance, 'pride' and 'gratitude' both fall under the category of 'joy,' but each carries distinct nuances and potentially different causes. while previous work by zhan et al. ( 2022 ) has made progress in summarizing events that trigger emotions, is limited in scale and covers only seven emotions. to enable ai models to reason more diversely and discerningly about emotion-cause, a comprehensive knowledge base is required. however, manually annotating a large-scale dataset is expensive. in this work, we present emo-know, a validated dataset on emotion causes curated from 9.8 million tweets over 15 years. our dataset contains tweets where users describe their emotions and triggering events, spanning 48 emotion types. unlike other datasets that rely on outside labeling, ours uses the users' own words, making it more genuine. with this data, combined with the emotion-cause details, we can create a map that shows how emotions are linked to causes, helping us understand the connections better. @xcite . our contributions are a method to create a large-scale, high ecological validity emotion dataset, validation with automatic metrics and human evaluators, and open-sourcing the dataset foot_0 (700k datums) for researchers. ## related work annotated emotion cause datasets. the field of emotion analysis has seen a rise in interest recently, leading to the creation of several datasets that provide annotations for both emotion and its causes. @xcite for example, @xcite introduced a dataset of 5,964 entries with word-level annotations of emotion and emotion causes, covering 6 emotion classes. @xcite annotated 2,105 news articles with clauses that contain 6 emotion classes and emotion causes. for english corpora, bostan et al. (2020) annotated a dataset of emotion, emotion cause and affect intensity, comprised of 5000 news headlines spanning 15 emotion classes. @xcite introduced a dataset of 1720 triplets @xmath0, emotion experiencer, emotion cause>. diverging from the extractive approach, zhan et al. (2022) developed a new dataset focusing on emotion and its triggers by summarizing the events that lead to particular emotions. this dataset has 1883 reddit posts, annotated with emotions (7 classes) and abstractive summaries of their triggers. as manual annotation often involves the cost of human labor, most annotated datasets are limited in size, and limited in the number of emotion classes they contain. automatic data labeling: in an effort to circumvent the costs of manual annotation, researchers have devised ways to generate large-scale datasets using pre-existing models. for instance, welivita et al. (2021) produced edos by fine-tuning roberta on a small label set created through crowdsourcing, and then used it to label 1 million conversations in the opensubtitles dataset @xcite . similarly, welivita and pu (2022) created heal -a knowledge graph consisting of 1 million distress narratives, annotated for emotion, post summarization, and node clustering using a variety of pre-existing models. a prevalent shortcoming of emotion labeling with current models is their limited accuracy; the emotion classifiers utilized in both heal and edos only achieved accuracies ranging from 65% to 65.88%. we address this limitation by harnessing users' self-reported emotion labels and rigorously assessing the performance of our fine-tuned emotion-cause summarization model with human evaluators. ## conclusion we created a novel, comprehensive dataset of 700k datum points, which aims to simultaneously identify emotions in text and determine the underlying causes behind each emotion. we detailed the data curation, label generation and evaluation pipeline so that researchers can use to create their own datasets that cater their needs. finally, we published the dataset so that other researchers can make use of the dataset for their tasks.