CUET _ B inary_ H ackers at C limate A ctivism 2024: A Comprehensive Evaluation and Superior Performance of Transformer-Based Models in Hate Speech Event Detection and Stance Classification for Climate Activism (2024)

Abstract: The escalating impact of climate change on our environment and lives has spurred a global surge in climate change activism. However, the misuse of social media platforms like Twitter has opened the door to the spread of hatred against activism, targeting individuals, organizations, or entire communities. Also, the identification of the stance in a tweet holds paramount significance, especially in the context of understanding the success of activism. So, to address the challenge of detecting such hate tweets, identifying their targets, and classifying stances from tweets, this shared task introduced three sub-tasks, each aiming to address exactly one mentioned issue. We participated in all three sub-tasks and in this paper, we showed a comparative analysis between the different machine learning (ML), deep learning (DL), hybrid, and transformer models. Our approach involved proper hyper-parameter tuning of models and effectively handling class imbalance datasets through data oversampling. Notably, our fine-tuned m-BERT achieved a macro-average $f1$ score of 0.91 in sub-task A (Hate Speech Detection) and 0.74 in sub-task B (Target Identification). On the other hand, Climate-BERT achieved a $f1$ score of 0.67 in sub-task C. These scores positioned us at the forefront, securing 1st, 6th, and 15th ranks in the respective sub-tasks. The detailed implementation information for the tasks is available in the GitHub.

## introduction over the decades, climate change has evolved into a pressing issue for nature and all earth's species, with alarming consequences. reports from the intergovernmental panel on climate change (ipcc) confirm that climate change is resulting in more frequent and severe weather events, including heatwaves, droughts, and floods foot_1 . these events can lead to crop failures, food shortages, displacement of people, melting of glaciers and ice caps, rising sea levels, and increased coastal flooding. preserving a harmonious climate is vital for ensuring balanced ecosystems, optimal temperature conditions, and biodiversity @xcite . this urgent issue has spurred people worldwide to voice their concerns and participate in a growing number of climate change activism events on a global scale @xcite . these events aim to raise awareness about the impact of climate change and the urgent need for action. one such prominent movement is 'fridayforfuture' (fff), initiated by greta thunberg, a swedish schoolgirl, in august 2018, to exert pressure on policymakers to take necessary actions against climate change @xcite . other notable climate activism movements, including 'extinction rebellion', 'earth strike', and 'climate justice now', have further fueled the global movement against climate change @xcite @xcite . however, contemporary activism extends beyond street protests to online platforms, with social media users expressing their thoughts on climate movements through tweets and comments. but some people share hateful, aggressive, and humorous tweets targeting activism @xcite . hate speech not only undermines the objectives of activism but also poses a threat to the well-being of individuals, organizations, and communities involved in the movement @xcite . whereas stance detection in text is also a vital component in assessing the dynamics of protests and activism. it helps understand whether activist movements and protests are being supported or opposed @xcite . despite numerous studies conducted in recent years on identifying hate speech and its targets in social media text, this context in climate activism remains an under-explored domain @xcite @xcite . as these events serve as crucial platforms for promoting environmental awareness and policy changes, there is a need for a comprehensive understanding of the stance and mitigation strategy for hateful tweets. as contributors to this endeavor, our principal contributions are delineated below: the later part of the paper is organized as follows: section 3 provides the task and dataset description, section 4 outlines the methodology, section 5 presents the result analysis, and section 6 delves into error analysis for each task. lastly, section 7 encapsulates the conclusion. ## task and dataset description the shared task encompasses three distinct subtasks: sub-task a, focusing on hate speech detection; sub-task b, centered around target detection; and sub-task c, concentrating on stance detection @xcite . the organizers introduced a dataset called climaconvo @xcite , comprising 15,309 tweets related to various climate movements. sub-tasks a, b, and c utilized subsets of this dataset. ## methodology in this section, we delineate our methodology step by step. figure 1 depicts a visual representation of the methodology. ## results and analysis in this section, we delve into a comprehensive comparative analysis of our proposed models across all three sub-tasks. ## error analysis the study investigated the performance of m-bert (sub-task a and b) and climate-bert (sub-task c) models using quantitative and qualitative methods. text samples were randomly chosen for all subtasks to facilitate quantitative analysis. ## conclusion in this paper, we present a fine-tuned approach utilizing various models, specifically proposing finetuned m-bert, distil-bert, climate-bert, and cnn+bigru. the results indicate that m-bert achieved a higher f 1 score for both sub-tasks a and b. the highest f 1 score that we achieved for sub-task a is 0.91, for sub-task b it is 0.74, and for sub-task c it is 0.67. several models like climate-bert, bigru, lr, and svm performed equally well with the same f 1 score for sub-task c. our paper includes a detailed comparison among several models, both before and after addressing the class imbalance in the datasets. notably, in most cases, the performance showed significant improvement. this paper also delved into effective preprocessing of data and data oversampling. these findings will create new opportunities for upcoming research work, drawing inspiration from this paper.