Stance and Hate Event Detection in Tweets Related to Climate Activism - Shared Task at CASE 2024 (2024)

Abstract: Social media plays a pivotal role in global discussions, including on climate change. The variety of opinions expressed range from supportive to oppositional, with some instances of hate speech. Recognizing the importance of understanding these varied perspectives, the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) at EACL 2024 hosted a shared task focused on detecting stances and hate speech in climate activism-related tweets. This task was divided into three subtasks: subtasks A and B concentrated on identifying hate speech and its targets, while subtask C focused on stance detection. Participants’ performance was evaluated using the macro F1-score. With over 100 teams participating, the highest F1 scores achieved were 91.44% in subtask C, 78.58% in subtask B, and 74.83% in subtask A. This paper details the methodologies of 24 teams that submitted their results to the competition’s leaderboard.

## introduction in an era dominated by digital communication, social media platforms serve as dynamic arenas where global conversations unfold in real-time. twitter (now x) foot_0 , in particular, with its diverse community, has emerged as a vital space for discussions on pressing global issues. among these, the discourse surrounding climate change stands out as a critical topic that captivates the attention of users worldwide, with the masses expressing myriad opinions towards climate change @xcite . as public awareness of climate change grows, and global movements like friday for future (fff) @xcite ) that aim to draw policymakers' attention towards climate change house various social media platforms, the need for a nuanced understanding of the discourse around climate change in the digital realm becomes essential. the escalating concern regarding climate change, coupled with the diverse range of discourse observed on twitter, presents a distinctive amalgamation that encapsulates the intricate spectrum of emotions expressed by individuals toward this global issue. within this spectrum lie various layers, including stance, which reflects individuals' inclinations toward specific viewpoints. as opinions are freely voiced, the prevalence of hate speech also emerges @xcite . moreover, using humor in language is both an engaging and intricate mechanism for conveying ideas on pressing matters @xcite . in order to unravel these complexities and enhance our understanding of online discussions concerning climate change, @xcite introduced a comprehensive multi-aspect dataset consisting of tweets related to climate change. this dataset includes five key aspects: the relevance of tweets to climate change, the stance conveyed in tweets, the presence of hate speech, the targets of such hate speech, and the presence of humor. expanding upon this, we launched a shared task at the case 2024 workshop, held alongside eacl 2024, by utilizing this dataset. this shared task is subdivided into three subtasks: subtask a focuses on hate speech detection, subtask b revolves around identifying targets within hate speech, and subtask c delves into stance detection in tweets. through this shared task, our objective is to foster active participation and cooperation in tackling the critical challenge of discerning stances on complex issues and identifying and curtailing hate speech within the digital sphere. the subsequent sections of this paper are structured as follows: section 2 presents an overview of the dataset used in our shared task. section 3 out-lines the specific subtasks of the shared task. furthermore, section 4 explains about methodologies used by the teams submitting system description papers. section 5 discusses a brief analysis of these system descriptions, while section 6 serves as the concluding segment of the paper. ## dataset in our shared task, we utilized the climaconvo dataset introduced by @xcite . this dataset includes a total of 15,309 tweets centered around the climate crisis issue. the dataset has 6 major tasks, viz. relevance, stance, hate speech, hate direction, hate targets, and humor. only 10,407 of the tweets in this data were relevant, while the remaining 4,902 were non-relevant. we only used three tasks in our shared task: hate speech detection, hate targets detection, and stance detection. a total of 10,407 tweets were used for both subtask a and subtask c, while 999 tweets were used for subtask b in the shared task. for each subtask, we divided the dataset into stages for training, evaluating, and testing in a stratified way, keeping a proportionate split ratio of approximately 70-15-15. ## shared task description hate speech refers to any form of communication that explicitly attacks an individual or a group based on their inherent characteristics, such as gender, religion, or race @xcite . stance describes the attitude or perspective expressed in a text towards a particular claim or topic @xcite . stance and hate detection can be used to analyze the structure of user interactions in conversational threads, providing valuable insights into the dynamics of online discussions. ## discussion the results and methodologies presented by the teams participating in this shared task offer valuable insights into the current state-of-the-art in hate speech detection, target identification, and stance detection. these tasks are essential in understanding the dynamics of online discourse, particularly on social media platforms. a notable trend across all subtasks is the heavy reliance on transformer-based models, particularly bert and its variants. these models have shown exceptional capability in understanding the intricacies of natural language, especially in informal and idiosyncratic texts commonly found on social media. their success underlines the importance of advanced models in handling the complexities of language in these contexts. ensemble and hybrid approaches have also been prevalent, adopted by teams like aast-nlp @xcite and cuet_binary_hackers @xcite . another critical aspect highlighted by several teams is handling class imbalance in datasets. the use of external datasets to enrich training data, as seen in the approaches of hamison-mtl @xcite and hamison-generative (fraile-hernandez and peñas, 2024), indicates a growing recognition of the value of diverse and expansive data sources. this approach can lead to better generalization and robustness of the models. preprocessing and feature engineering also play a crucial role, as demonstrated by teams like masonperplexity @xcite , and bryndza @xcite . the way data is prepared and presented to models can significantly impact their effectiveness, highlighting the importance of meticulous data handling. incorporating the latest advancements in llms further enriches the discussion of shared tasks' outcomes and future directions. the use of llms, as demonstrated by teams like bryndza @xcite and masonperplexity @xcite , marks a significant shift in the approach to understanding and processing natural language on social media platforms. despite these advances, several challenges and potential future directions emerge. ensuring that models perform well across different contexts remains a significant challenge, given the variability in expressions of hate speech and stances. additionally, the subtlety and ambiguity in language use, especially in these domains, continue to pose significant hurdles. ## conclusion the shared task at the case 2024 workshop has made significant strides in advancing our understanding of hate speech detection, target identifica-tion, and stance detection in social media contexts, focusing on twitter conversations about climate change. the diversity of approaches employed by the participants, predominantly centered around sophisticated transformer-based models like bert and its variants, demonstrates the complexity of analyzing online discourse. however, this field of study still faces significant challenges, including ensuring the adaptability of models across various contexts, refining language processing to capture subtle nuances, and navigating the ethical implications of automated content analysis. this task has provided a comprehensive benchmark for current methodologies and set the stage for future research in the rapidly evolving domain of nlp, emphasizing the need for continued innovation in understanding the complexities of digital communication.