Large Language Models as Detectors or Instigators of Hate Speech in Low-resource E thiopian Languages (2025)

Abstract: We introduce a multilingual benchmark for evaluating large language models (LLMs) on hate speech detection and generation in low-resource Ethiopian languages: Afaan Oromo, Amharic and Tigrigna, and English (both monolingual and code-mixed). Using a balanced and expert-annotated dataset, we assess five state-of-the-art LLM families across both tasks. Our results show that while LLMs perform well on English detection, their performance on low-resource languages is significantly weaker, revealing that increasing model size alone does not ensure multilingual robustness. More critically, we find that all models, including closed and open-source variants, can be prompted to generate profiled hate speech with minimal resistance. These findings underscore the dual risk of exclusion and exploitation: LLMs fail to protect low-resource communities while enabling scalable harm against them. We make our evaluation framework available to facilitate future research on multilingual model safety and ethical robustness.

## introduction and related work hate speech is a growing problem online, particularly in linguistically diverse and politically fragile contexts like ethiopia, where social media has become a vehicle for disinformation, incitement, and inter-ethnic hostility. platforms such as x (formerly twitter), instagram, facebook, and youtube have increasingly reduced reliance on human moderators, instead turning to automated moderation systems powered by large language models (llms) @xcite . while llms have proven effective at content moderation in high-resource languages such as english, their performance in low-resource settings remains underexplored and potentially unreliable. recent advances in pretrained language models @xcite @xcite have revolutionised natural language processing (nlp), including applications in toxicity detection and content moderation. however, this progress has been overwhelmingly focused on english @xcite @xcite , with only limited attention given to low-resource languages, including ethiopian languages @xcite . moreover, very few studies systematically assess the risks of llms being used not just to detect, but also to generate hateful content @xcite . in ethiopia, the risks posed by the dual role of llms are delicate. while local languages such as afaan oromo, amharic and tigrigna are among the most widely spoken in the country and increasingly used online, they are largely unsupported by state-of-the-art language models. this gap creates a dangerous asymmetry: automated systems may fail to detect hate speech in these languages, while the same models, especially open-source or publicly accessible ones, can be used to produce hateful, targeted content at scale. our work addresses this dual challenge by introducing a benchmark to evaluate llms as both detectors and generators of hate speech in low-resource languages. we focus on five language categories: monolingual english (m-english), code-mixed english (c-english), afaan oromo, amharic and tigrigna. we build on an existing annotation scheme @xcite that defines hate speech along three dimensions: target (e.g., ethnicity, religion, gender), type (e.g., insult, threat, incitement), and nature (e.g., slur, stereotype, irony). this framework supports fine-grained annotation and multilingual evaluation. in summary, our contributions are as follows: ## discussion our findings challenge the common assumption that larger models consistently perform better @xcite . while this holds for m-english, it does not extend to lowresource languages like afaan oromo, amharic, and tigrigna, even in c-english. in these cases, increasing model size often leads to worse performance (see appendix f), indicating that scale alone does not guarantee multilingual robustness. we further observed that quantisation, a weight compression approach, significantly depletes performance on these low-resource languages, even when the same models retain strong results in m-english. equally concerning, all models, regardless of size, were easily prompted to generate profiled hate speech. as these systems are deployed globally, their current limitations in safety must be addressed to prevent scalable and targeted harm. ## conclusion and future work this paper introduced a multilingual benchmark to evaluate llms on detecting and generating hate speech in afaan oromo, amharic, tigrigna, monolingual english and code-mixed english. we found that while llms struggle to detect hate in lowresource languages, they remain permissive in generating targeted hate when prompted, posing serious risks for online spaces. future work will explore prompts written in low-resource languages to assess models' direct linguistic understanding and safety alignment. we recommend stronger investment in fine-tuning and safety evaluation for low-resource settings, especially for downstream tasks like moderation and harm prevention.