Hyderabadi Pearls at Multilingual Counterspeech Generation : HALT : Hate Speech Alleviation using Large Language Models and Transformers (2025)

Abstract: This paper explores the potential of using fine- tuned Large Language Models (LLMs) for generating counter-narratives (CNs) to combat hate speech (HS). We focus on English and Basque, leveraging the ML_MTCONAN_KN dataset, which provides hate speech and counter-narrative pairs in multiple languages. Our study compares the performance of Mis- tral, Llama, and a Llama-based LLM fine- tuned on a Basque language dataset for CN generation. The generated CNs are evalu- ated using JudgeLM (a LLM to evaluate other LLMs in open-ended scenarios) along with traditional metrics such as ROUGE-L, BLEU, BERTScore, and other traditional metrics. The results demonstrate that fine-tuned LLMs can produce high-quality contextually relevant CNs for low-resource languages that are comparable to human-generated responses, offering a sig- nificant contribution to combating online hate speech across diverse linguistic settings.

## introduction the unchecked proliferation of hate speech online has become a significant societal concern, prompting the need for effective countermeasures. conventional content moderation strategies, such as removing hateful content and suspending user accounts, have been criticized for potentially limiting freedom of expression and not addressing the underlying causes of hate speech @xcite . counter-narratives (cns), defined as non-aggressive responses that challenge hateful messages using evidence-based arguments, promoting empathy and understanding, offer a more promising approach. research has shown that cns can be effective in mitigating hate speech online. they can help to de-escalate heated online discussions, offer alternative perspectives to bystanders, and potentially even encourage individuals who engage in hate speech to reconsider their views @xcite . however, manually creating cns poses challenges in scalability due to the huge amount of hate speech online @xcite . recent advances in natural language processing (nlp), particularly in large language models (llms), provide a potential solution. llms, trained on extensive text datasets, have shown remarkable capabilities in various nlp tasks, including text generation @xcite . by finetuning, these models can be adapted for specific tasks such as cn generation, potentially enabling the automatic creation of high-quality, contextually relevant cns at scale @xcite . this paper examines the application of fine-tuned llms for generating cns against hate speech in english and basque using the ml_mtconan_kn dataset, which is derived from the conan @xcite @xcite and @xcite datasets. includes hate speech-counter-narrative pairs enriched with relevant knowledge. selecting english and basque allows the exploration of cn generation in both high-resource and low-resource language settings. additionally, the fine-tuned models were evaluated on italian and spanish datasets to assess the cross-lingual applicability of fine-tuned models in one language to others. this paper, by analysing outputs from a pre-trained llm and comparing different fine-tuning and post-processing techniques, aims to: this research ultimately aims to contribute to the development of robust, scalable, and effective tools to combat online hate speech, fostering a more inclusive and respectful online environment. ## approach this paper involves a series of systematic experiments, including those conducted as part of the official submission and additional explorations performed post-submission. the primary objective is to fine-tune existing large language models (llms) to enhance their ability to generate effective counter-narratives. model selection is guided by two criteria: the models' capacity for fine-tuning and their performance on established benchmarks. for this task, the counter-narratives were generated solely using the existing knowledge provided in the original dataset. no external or additional knowledge sources were incorporated in the generation process. ## methodology our experiments are conducted on the ml_mtconan_kn dataset for english and basque, which is derived from the conan and mt-conan datasets. our choice of large language models (llms) for fine-tuning reflects a strategic approach to counternarrative generation: the selected models were chosen for their strong performance on various nlp tasks and their strategic size, ranging from 5 to 10 billion parameters, which makes them well-suited for fine-tuning. this size range strikes a balance between capability and the practicality of using widely available hardware. fine-tuning these models on standard gpus, such as those accessible through google colab or kaggle notebooks, often requires additional optimization techniques. to address this, methods such as qlora (quantized low-rank adaptation; @xcite ) were employed, allowing efficient finetuning of llms on limited computational resources. ## results as outlined in 3, we conducted a series of experiments to fine-tune the models, aiming to optimize their performance for counter-narrative generation. the exact prompts used in the experiments are detailed in a. if a prompt name is not mentioned for a specific experiment, the default prompt is used. ## discussion the experiments provided valuable insights into the strengths and limitations of fine-tuning large language models (llms) for counter-narrative generation across different languages. several key themes emerged from the results: ## conclusion this study investigated the fine-tuning of large language models (llms) for counter-narrative (cn) generation across english, basque, italian, and spanish. by examining both high-resource and low-resource settings, we identified key strengths, limitations, and challenges in leveraging llms for this socially impactful task. the findings underscore the critical role of language-specific fine-tuning in improving performance, particularly for low-resource languages like basque, where general-purpose models struggle due to limited data. in contrast, high-resource languages such as english, italian, and spanish showcased robust results, with fine-tuned models often outperforming general-purpose models like gpt-4o in alignment and relevance metrics. however, gpt-4o performed better in the judgelm scores. this paper underscores the importance of: with further research and incorporating the learnings from this paper, llms can become more scalable, reliable, and inclusive, enabling their effective deployment in combating hate speech and fostering constructive dialogue across diverse linguistic and cultural contexts.