ARC - NLP at C limate A ctivism 2024: Stance and Hate Speech Detection by Generative and Encoder Models Optimized with Tweet-Specific Elements (2024)

Abstract: Social media users often express hate speech towards specific targets and may either support or refuse activist movements. The automated detection of hate speech, which involves identifying both targets and stances, plays a critical role in event identification to mitigate its negative effects. In this paper, we present our methods for three subtasks of the Climate Activism Stance and Hate Event Detection Shared Task at CASE 2024. For each subtask (i) hate speech identification (ii) targets of hate speech identification (iii) stance detection, we experiment with optimized Transformer-based architectures that focus on tweet-specific features such as hashtags, URLs, and emojis. Furthermore, we investigate generative large language models, such as Llama2, using specific prompts for the first two subtasks. Our experiments demonstrate better performance of our models compared to baseline models in each subtask. Our solutions also achieve third, fourth, and first places respectively in the subtasks.

## introduction there is a growing challenge of detecting hate speech within the context of digital communication, particularly in climate change activism, by means of natural language processing @xcite . the shared task on hate speech and stance detection during climate activism organized in the workshop on challenges and applications of automated extraction of socio-political events from text (case) @xcite aims to provide an opportunity to study important components in identifying events during climate change activism. the task includes three subtasks for detecting (a) hate speech, (b) its target, and (c) stance being supported or opposed. our proposed approach in the shared task is to employ large encoder models, such as bertweet @xcite , enhanced with optuna @xcite to improve model performance by optimizing deep learning hyperparameters and also tweet-specific elements, such as hashtags, urls, and emojis. additionally, we leverage the capabilities of generative large language models, such as llama2 @xcite . lastly, we propose hybrid solutions that benefit from both encoder and generative models. generative models serve as a decision support mechanism, particularly in instances where the encoder model's predictions are ambiguous or uncertain. the performances of our models are measured on the climaconvo dataset @xcite . in this study, we report the details of our solutions, which obtain 3rd place in subtask a, 4th place in subtask b, and 1st place in subtask c. ## main approach our approach includes three solutions. first, we employ encoder models for text classification with a specific focus on tweet-specific elements such as hashtags, urls, and emojis. second, we employ generative large language models. lastly, we provide hybrid solutions that benefit from both encoder and generative models. we use pytorch @xcite and hugging face @xcite for model implementations. ## leaderboard results in this section, we report the results of all submitted models on the test data. the optimized parameters of our submitted models are reported in in in in ## conclusion we conclude that the optimized bertweet model outperforms other encoder models in all subtasks, indicating the importance of tweet-specific elements (hashtag, url, and emoji) in hate event detection. overall, generative models perform poorly in this task. more investigation is needed to understand their capabilities for hate speech detection. a possible reason for poor performance could be our prompts or generation config. nevertheless, the support of llama2 increases the performance in subtask a. in future work, state-of-the-art generative mod-els like gpt3.5 foot_4 or gpt4 foot_5 can be employed in addition to llama2 and mistral. moreover, prompt tuning can improve the performance of generative models and extend the work for generalizing model understanding capacity.