Who Speaks Matters: Analysing the Influence of the Speaker’s Linguistic Identity on Hate Classification (2025)

Abstract: Large Language Models (LLMs) offer a lucrative promise for scalable content moderation, including hate speech detection. However, they are also known to be brittle and biased against marginalised communities and dialects. This requires their applications to high-stakes tasks like hate speech detection to be critically scrutinized. In this work, we investigate the robustness of hate speech classification using LLMs particularly when explicit and implicit markers of the speaker’s ethnicity are injected into the input. For explicit markers, we inject a phrase that mentions the speaker’s linguistic identity. For the implicit markers, we inject dialectal features. By analysing how frequently model outputs flip in the presence of these markers, we reveal varying degrees of brittleness across 3 LLMs and 1 LM and 5 linguistic identities. We find that the presence of implicit dialect markers in inputs causes model outputs to flip more than the presence of explicit markers. Further, the percentage of flips varies across ethnicities. Finally, we find that larger models are more robust. Our findings indicate the need for exercising caution in deploying LLMs for high-stakes tasks like hate speech detection.

## introduction language technologies are increasingly being used in content moderation tasks, including hate speech detection, because of their ability to handle large volumes of data @xcite . however, the use of llms in a highstakes task like hate speech detection requires caution, because llms are known to be brittle, biased and non-deterministic, especially when additional information that is not relevant to the task itself is present @xcite . there is extensive documentation of biases against marginalized communities and dialects that leads to disparate treat- if i tell him who looks like a monkey i'll probably get banned jamaican singaporean explicit implicit (dialect) wah gwaan, if me tell him dat yuh look like a baboon, me probably get block, ya hear? me no want no trouble, but dem police acome find me, ya get mi? the jamaican person said, ' if i tell him who looks like a monkey i'll probably get banned ' explicit implicit (dialect) ah, lor, if i tell him dat guy looks like a langur, i'll probably get banned from facebook, or maybe even get a police report lodged against me, wah piang! the singaporean person said, ' if i tell him who looks like a monkey i'll probably get banned ' if i tell him who looks like a monkey i'll probably get banned not hateful not hateful not hateful llm existing identityagnostic analysis proposed identityfocuse analysis not hateful c la s s if ie s c la s s if ie s classifies c la s s if ie s c la s s if ie s hateful speaker identity ment and representational harms in downstream tasks, including hate speech detection @xcite @xcite @xcite @xcite @xcite @xcite . as llms are adopted globally, they need to be inclusive of people of all nationalities. however, prior work has shown a preference in these models toward american english @xcite , while despite it being a global language, different dialects of english are used in different geographical locations @xcite . previous studies @xcite @xcite have investigated the effect of assigning a culture to the model, but haven't been able to capture the impact of this identity of the user. in this work, we analyse the robustness of language models (3 llms and 1 lm) in hate speech detection of english sentences spoken by people of varying linguistic identities, as highlighted in figure 1 illustrates our setup with an example. our contributions can be summarized as follows. ## experimental setup in this work, we use 2 datasets: mpbhsd @xcite and hatexplain @xcite . the mpbhsd dataset is derived from twitter, 4chan, and reddit, and comprises 600 hateful (h) and 2400 non-hateful (nh) sentences. the hatexplain dataset is derived from twitter and gab, and we sample 3000 sentences, comprising 2094 hateful (h) and 906 non-hateful (nh) sentences. llms. we do our evaluation on three llms: llama-3-8b, llama-3-70b (ai@meta, 2024), @xcite , and 1 fine-tuned bert-based model trained on hatexplain dataset @xcite . for mphsd dataset, we leverage the pre-annotated gpt-4-turbo instead of prompting gpt-4o. we also consider two types of prompting strategies: zero-shot classification and in-context learning (icl) as both their uses have been studied before @xcite . we keep the temperature 0 for all models to ensure deterministic outputs. for zero-shot, we use a classification prompt designed in @xcite . for icl, we randomly sample 4 prompts (srivastava, 2025) for hate and non-hate examples to be appended to the system prompt above. 4 do the models flip when inputs are marked with speaker identity? linguistic identity. we consider the following 5 nationalities as our linguistic identity: indian, singaporean, british, jamaican, and african-american. these nationalities are chosen for the distinct english by these nations. we also choose the african-american dialect to represent its distinctness from the standard american english @xcite . while these nationalities represent geographic diversities, they also serve as an umbrella dialect to micro-dialects and communities present within the region. ## conclusion in this work, we evaluate the robustness (or lack of thereof) of llms in hate speech classification. specifically, we injected explicit and implicit dialectal markers of speaker's ethnicity in the input. we evaluated 4 lms by measuring the percentage of flips of the model outputs from the unmarked prompt. we find that the % of flips is governed by nature of the model, speaker's identity, the type of marker injected and the target of the speech. this depicts the unreliability of llms in real-world applications and presses the need for more caution while deploying these systems.