Large Language Models Discriminate Against Speakers of German Dialects
Authors: name: Minh Duc Bui, name: Carolin Holtermann, name: Valentin Hofmann, name: Anne Lauscher, name: Katharina von der Wense

Dialects represent a significant component of human culture and are found
across all regions of the world. In Germany, more than 40% of the population
speaks a regional dialect (Adler and Hansen, 2022). However, despite cultural
importance, individuals speaking dialects often face negative societal
stereotypes. We examine whether such stereotypes are mirrored by large language
models (LLMs). We draw on the sociolinguistic literature on dialect perception
to analyze traits commonly associated with dialect speakers. Based on these
traits, we assess the dialect naming bias and dialect usage bias expressed by
LLMs in two tasks: an association task and a decision task. To assess a model's
dialect usage bias, we construct a novel evaluation corpus that pairs sentences
from seven regional German dialects (e.g., Alemannic and Bavarian) with their
standard German counterparts. We find that: (1) in the association task, all
evaluated LLMs exhibit significant dialect naming and dialect usage bias
against German dialect speakers, reflected in negative adjective associations;
(2) all models reproduce these dialect naming and dialect usage biases in their
decision making; and (3) contrary to prior work showing minimal bias with
explicit demographic mentions, we find that explicitly labeling linguistic
demographics--German dialect speakers--amplifies bias more than implicit cues
like dialect usage.