Decoding Hate: Exploring Language Models’ Reactions to Hate Speech (2025)

Abstract: Hate speech is a harmful form of online expression, often manifesting as derogatory posts. It is a significant risk in digital environments. With the rise of Large Language Models (LLMs), there is concern about their potential to replicate hate speech patterns, given their training on vast amounts of unmoderated internet data. Understanding how LLMs respond to hate speech is crucial for their responsible deployment. However, the behaviour of LLMs towards hate speech has been limited compared. This paper investigates the reactions of seven state-of-the-art LLMs (LLaMA 2, Vicuna, LLaMA 3, Mistral, GPT-3.5, GPT-4, and Gemini Pro) to hate speech. Through qualitative analysis, we aim to reveal the spectrum of responses these models produce, highlighting their capacity to handle hate speech inputs. We also discuss strategies to mitigate hate speech generation by LLMs, particularly through fine-tuning and guideline guardrailing. Finally, we explore the models’ responses to hate speech framed in politically correct language.

## introduction social media and internet platforms have significantly enhanced global connectivity and communication. however, this increased interconnectivity has also highlighted the growing issue of hate speech, affecting individuals worldwide @xcite . studies indicate that around 30% of young people encounter cyberbullying @xcite , and 46% of black/african american adults reported experiencing racial harassment online @xcite , underscoring the urgent need to address hate speech. the rise of llms introduces new complexities regarding hate speech. llms are trained on vast amounts of online data, including social media, enabling them to generate text autonomously. this raises concerns about their potential to produce harmful or offensive content @xcite , especially targeting minority groups and vulnerable individuals. the presence of hate speech in their training datasets necessitates an in-depth examination of the likelihood that llms could replicate and disseminate hate speech. as llms become increasingly integrated into various platforms, such as chatbots @xcite @xcite , multi-party chats @xcite , or automated bots on social media @xcite , it becomes critical to develop strategies that minimise these risks and promote their ethical use. we aim to understand how different llms respond when prompted with hate speech, defined as language characterised by offensive, derogatory, humiliating, or insulting discourse @xcite that promotes violence, discrimination, or hostility towards individuals or groups @xcite based on attributes such as race, religion, ethnicity, or gender @xcite . we analyse the reactions of seven state-of-the-art llms to more than 26 000 english hate speech sentences, simulating potential user interactions when not required to perform any specific task; they simply generate content in a vanilla mode. we then examine the content they produce and determine how to prevent them from generating hate speech if it occurs. we aim to address: chatbot directed abuse: recent research on chatbot-directed abuse focuses on detecting, understanding, and mitigating abusive interactions aimed at conversational agents @xcite . other works examined how conversational agents should respond to abuse @xcite , or how to protect chatbots from toxic content @xcite . moreover, there are efforts to build data collections to help detect this issue @xcite . some efforts focused on developing english-hindi datasets to detect offensive speech in conversational settings @xcite . large language models: llms have been used in numerous tasks, including translation, sentiment analysis, and chatting applications. their role, especially in promoting or mitigating hate speech in the latter, is critical. it is imperative to study the potential misuse of llms and the harm they may cause @xcite . while llms have shown promise in tasks such as hate speech detection @xcite @xcite and generating counterspeech @xcite @xcite , little attention has been paid by independent researchers to investigating whether these models might inadvertently perpetuate or exacerbate hate speech. llms safety and hate speech generation: recent research on hate speech safety in llms focuses on strategies for controlling language generation content. other works study llms' tendency to generate harmful narratives @xcite . moreover, there is a compilation of datasets for evaluating and improving the safety of llms @xcite . other studies have explored how llms react to hate speech in both direct and indirect manners. on the one hand, some research focuses on how llms can be trained to avoid generating or perpetuating hate speech @xcite . techniques such as fine-tuning models on curated datasets that exclude hate speech or incorporating explicit constraints during the training process have shown promise @xcite . implicit hate: recently, there have been efforts to define implicit hate and propose ways to detect it @xcite . moreover, works like @xcite suggest methods to reduce the hate intensity (i.e., convert explicit hate to implicit or polite hate). ## methodology here we present the pipeline (figure 1 ), datasets, and models employed in this work. our main study involves giving the llms hate speech sentences to see how they respond in conversation. ## conclusions cutting-edge llms continue to produce hate speech in unsupervised scenarios. open-source models tend to generate more hate speech compared to privatised models like gpt and gemini, which have some blocking mechanisms. however, when given clear instructions, these models reduce hate speech generation. this highlights the need for caution when using these models, as they can harm minority groups with hateful messages. our findings indicate that while commercial models generally show restraint in producing hate speech, further research is needed to enhance the moderation mechanisms of open-source models. addressing mismatches in moderation styles tailored to subtle forms of hate speech is a critical area for future research. additionally, future research could compare different sizes of the same model to see if there are changes in hate speech generation.