RESEMO : A Benchmark C hinese Dataset for Studying Responsive Emotion from Social Media Content (2024)

Abstract: On social media platforms, users’ emotions are triggered when they encounter particular content from other users,where such emotions are different from those that spontaneously emerged, owing to the “responsive” nature. Analyzing the aforementioned responsive emotions from user interactions is a task of significant importance for understanding human cognition, the mechanisms of emotion generation, and behavior on the Internet, etc. Performing the task with artificial intelligence generally requires human-annotated data to help train a well-performing system, while existing data resources do not cover this specific area, with none of them focusing on responsive emotion analysis. In this paper, we propose a Chinese dataset named ResEmo for responsive emotion analysis, including 3813 posts with 68,781 comments collected from Weibo, the largest social media platform in China. ResEmo contains three types of human annotations with respect to responsive emotions, namely, responsive relationship, responsive emotion cause, and responsive emotion category. Moreover, to test this dataset, we build large language model (LLM) baseline methods for responsive relation extraction, responsive emotion cause extraction, and responsive emotion detection, which show the potential of the proposed ResEmo being a benchmark for future studies on responsive emotions.

## introduction with the rise of social media and increasing user activity on online platforms, many researchers focus on social media text processing @xcite @xcite . on social media, people tend to express their sentiments or emotions in their generated content and thus attract much attention from existing studies (demszky et al., 2020; chen * corresponding author @xcite @xcite @xcite . different from emotions delivered in normal scenarios mostly driven by various causal events, interactions on social media play the dominant role in pushing individuals to develop emotions with respect to the content they engage with @xcite , where such emotions are responsive to the previous user posts and comments. figure 1 provides an example of a blog post with its comments, where the comment id:1-3 is semantically responsive to both id:1-1 and id:1-2, and expresses an emotion of "cynicism", with corresponding word-level emotion cause marked in red. apparently, to analyze such responsive emotions and emotion causes, it is beneficial to mine the responsive relationships conveyed in the context of social interactions. however, responsive relationships are often implicit and intricate on social media. users typically browse a blog post and its accompanying comments before composing their own comments. throughout this process, they form opinions based on the overall impressions gathered from the blog and comments they have browsed. as a result, they may respond to multiple comments or the blog post itself, creating implicit one-to-many responsive relationships. additionally, there can be a time delay between a user's comment and the post or comments they are responding to due to the asynchronous nature of social media interactions. these unique characteristics present challenges in mining response relationships. considering that currently there are no existing models or systems capable of addressing these challenges, one should prepare such resources in order to perform research related to this topic. therefore, collecting data with interactive responsive emotions from social platforms and constructing a dataset with explicit annotations for responsive relationships, as well as emotion categories and causes has become a vital step for later studies on responsive emotion analysis. figure 1 : an example of a blog post with its comments. in weibo, the blog post and accompanying comments are organized in a 3-level tree structure, including the blog post as the root node, and two levels of comments, where the comments directly respond to the blog post from the first-level comments. these first-level comments can also be further replied to, forming the second-level comments. note that in this social platform, all subsequent interactions and replies within second-level comments are organized on the same level. in the resemo dataset, comments with the same parent node are assigned with indices in chronological order from oldest to newest. the dataset includes three types of annotations, i.e. responsive relation, responsive emotion cause and responsive emotion category. for example, given the annotation target comment id:1-3, it is semantically responsive to id:1-1 and id:1-2, thus its responsive relationship is id:1-1 and id:1-2, while the responsive emotion cause is marked in red, which acts as the reason behind the conveyed emotion of id:1-3. the responsive emotion category of comment id:1-3 is labeled as cynicism. the dataset only contains chinese and the english translation is only given for reference. despite the existence of datasets @xcite @xcite that provide emotion and emotion cause annotations from conversation corpus or social media posts, they do not fully consider the responsive relationship within the context. as a result, they do not adequately capture the implicit one-to-many and temporal delay characteristics of responsive relationships mentioned earlier. for instance, datasets like @xcite gather dialog corpus from tv series, where each utterance mainly serves as a direct response to the previous one, representing a one-to-one and instant responsive relationship. additionally, @xcite collect data from an english social platform, but their annotation only focuses on emotion categories, disregarding the responsive relationships. in this paper, we introduce a novel chinese dataset called resemo that aims to facilitate an in-depth analysis of responsive emotions in social media text. the raw data for resemo is collected from weibo, the largest social media platform in china. as shown in figure 1 , for a given blog post with accompanying comments, we annotate each comment with three types of labels: responsive relationship, responsive emotion cause, and responsive emotion category. the responsive relationship label indicates which preceding comments or the blog post itself the current comment is responding to, allowing for the possibility of a one-tomany relationship. the responsive emotion cause label identifies a specific word-level text span that acts as the reason behind the conveyed emotion in the current comment. lastly, the responsive emotion category label classifies the specific type of emotion expressed in the current comment. using the annotated data, several baseline methods based on large language model (llm) are employed for tasks such as responsive relationship extraction, responsive emotion cause extraction, and responsive emotion detection. the experimental results demonstrate the potential benefits of mining the responsive relationship, as it proves advantageous for both emotion cause extraction and responsive emotion detection. furthermore, these results validate the suitability of resemo as a benchmark dataset for future studies on responsive emotions. ## the resemo dataset in this section, we first introduce how we collect our data, then we introduce the annotation guidelines, and annotation process and finally discuss properties of our dataset. ## experiments the resemo dataset serves as a foundational resource for analyzing emotions in social media. to assess the effectiveness of this dataset, we propose llm-based baseline models for tasks, such as responsive emotion category detection, responsive emotion cause extraction, and responsive relationship extraction. ## related work as shown in the meld dataset @xcite includes 13,000 utterances from 1,433 dialogue segments of the tv series "friends". in addition to dialogue text, the dataset also annotates various modal information, including speech and facial expressions. the iemocap dataset @xcite provides detailed motion capture information about the head, face, hand movements, etc., to showcase emotional expressions and postures during interpersonal interactions. the cped dataset @xcite consists of over 12,000 dialogues from 392 speakers in 40 tv programs. it is a large-scale chinese personalized and emotion-driven dialogue dataset, aiming to perform both personality and emotion analysis. besides, the goemotions dataset @xcite includes 58,000 english reddit comments that have been labeled with 27 different emotions or neutral. mastodon explores the relationship between conversational behavior and emotion recognition, suggesting that this correlation can be leveraged for transfer learning between two tasks @xcite . the emotyda dataset investigates the role of multimodality and emotion recognition in dialog behavior classification @xcite . the memor dataset is proposed to perform emotion category detection in situations with missing modalities @xcite . the peld dataset collects data from tv show dialogues, creating an emotional dialogue dataset with personality traits @xcite . the empatheticdialogues dataset aims to advance the development of conversational agents capable of better understanding and responding to human emotions and concerns @xcite . ecf establishes a dataset comprising 9,272 multimodal emotion-cause pairs @xcite . ## conclusion in this paper, we propose a chinese dataset named resemo for responsive emotion analysis in the social media domain. the dataset consists of 3,813 posts with 68,781 comments collected from weibo, including three types of annotation, namely, responsive relationship, responsive emotion cause and responsive emotion categories. we test several llm-based baseline methods on resemo for responsive relationship extraction, responsive emotion detection, and responsive emo-tion cause extraction. the experimental results validate our proposed resemo dataset in terms of consistency and effectiveness. besides, the unique characteristic of responsive relationships also shows its advantages in responsive emotion analysis.