Evaluation of A frican A merican Language Bias in Natural Language Generation (2023)

Abstract: While biases disadvantaging African American Language (AAL) have been uncovered in models for tasks such as speech recognition and toxicity detection, there has been little investigation of these biases for language generation models like ChatGPT. We evaluate how well LLMs understand AAL in comparison to White Mainstream English (WME), the encouraged “standard” form of English taught in American classrooms. We measure large language model performance on two tasks: a counterpart generation task, where a model generates AAL given WME and vice versa, and a masked span prediction (MSP) task, where models predict a phrase hidden from their input. Using a novel dataset of AAL texts from a variety of regions and contexts, we present evidence of dialectal bias for six pre-trained LLMs through performance gaps on these tasks.

## introduction task-specific models proposed for speech recognition, toxicity detection, and language identification have previously been documented to present biases for certain language varieties, particularly for african american language (aal) @xcite @xcite . there has been little investigation, however, of the possible language variety biases in large language models (llms) @xcite @xcite , which have unified multiple tasks through language generation. while there are largely beneficial and socially relevant applications of llms, such as in alleviating barriers to mental health counseling foot_0 and medical healthcare @xcite access, there is also potential for biased models to exacerbate existing societal inequalities @xcite @xcite . past algorithms used in psychiatry and medicine have been shown to be racially biased, in some cases leading to, for example, underestimating patient risk and denial of care @xcite . furthermore, llms capable of understanding aal and other language varieties also raise important ethical implications, such as enabling increased police surveillance of minority groups (see @xcite and section 8 for further discussion). therefore, it is necessary to investigate the potential language variety biases of language generation models to both increase accessibility of applications with high social impact and also anticipate possible harms when deployed. moreover, prior work @xcite has shown that african american speakers talking about racerelated issues use language in ways which may draw on morphosyntactic features of aal in order to subtly foreground the race aspect of the discussion topic without explicit mention. most training corpora include little representation of aal (see further discussion in section 3), and even those that do can still fail to capture its significant regional and contextual variation (see @xcite for examples). without the ability to interpret these subtler meanings of aal, llms will undoubtedly exacerbate the misunderstandings which already take place between aal speakers and other communities. ## background: bias in measuring aal understanding, we identify evidence of bias through performance gaps and analysis of model behavior with each language variety. following blodgett et al. (2020), findings of bias could result in both allocational harms and representational harms posed by the evaluated models foot_1 . while llms are becoming more available and valuable resources, the models' lack of understanding of aal limits their use by aal speakers, and this disparity will only grow as the use of these models increases across social spheres. our evaluation attempts to quantify these error disparities @xcite by measuring models' understanding of aal and wme texts. when llms do not perform equally well on different language varieties, the llm itself as a resource becomes unfairly allocated, and speakers of minoritized language varieties like aal are less able to leverage the benefits of llms. aal speakers would be particularly unfairly impacted with applications in areas of health, including mental health. additionally, our evaluation includes a qualitative analysis of how aal is currently understood and produced by llms. prior sociolinguistic works discuss and study how attitudes toward african american speakers have formed linguistic prejudices against aal @xcite , as well as how stereotyped uses of aal by non-aal speakers can perpetuate racial divides @xcite . stereotypical or offensive uses of aal by llms thus reflect a representational harm to aal speakers that can further promote these views. we advocate for approaches which carefully consider sociolinguistic variation in order to avoid generation of inappropriate speech across different settings. ## data biases in pre-trained language models can often be attributed to common training datasets. training corpora for llms are typically drawn from internet sources, such as large book corpora, wikipedia, outbound links from reddit, and a filtered version of common crawl foot_2 in the case of @xcite , which can severely under-represent the language of african americans @xcite . though few estimates of the presence of aal in datasets exist, one study estimates that in the colossal cleaned crawl corpus (c4), only 0.07% of documents reflect aal @xcite . beyond c4, african americans are significantly underrepresented in data sources such as wikipedia foot_3 (0.05%) and news articles (6%; pew research center 2023), falling well below the national average. additionally, as models learn from gold standard outputs provided by annotators, they learn to reflect the culture and values of the annotators as well. ## methods we evaluate multiple language generation models using two tasks. in the counterpart generation task, we evaluate models on producing near semantically-equivalent wme text given aal text and vice versa to target llms' ability to interpret and understand aal. a second task, masked span prediction, requires models to predict tokens to replace words and phrases hidden or masked from the input. this task resembles that of @xcite , but spans vary in length and position. much like bart pre-training @xcite , span lengths are drawn from a poisson distribution (λ = 2) and span locations are sampled uniformly across words in the original text. we independently mask noun phrases, verb phrases, and random spans from the text for more fine-grained analysis. while our focus is on measuring model capabilities in interpreting aal foot_6 , these generation tasks allow us to test whether the model understands the language well enough to produce it. it is not our goal to produce a llm that can generate aal within the context of downstream tasks (see section 8 for further discussion). ## related work while few have specifically focused on bias against aal in language generation, related work has ex-tensively investigated societal biases in language tasks. one large-scale study @xcite investigates performance on the standard glue benchmark @xcite using a synthetically constructed aal dataset of glue for fine-tuning. they show performance drops on a small humanwritten aal test set unless the roberta model is fine-tuned. racial bias in generation. mitigating and evaluating social biases in language generation models is a challenging problem due to the apparent tradeoffs between task performance and bias mitigation, the many possible sources of bias, and the variety of biases and perspectives to examine @xcite . a number of studies have proposed bias evaluation measures, often using prompts crafted to reveal biased associations of, for example, occupation and gender (i.e., "the [man/woman] worked as a ...") @xcite @xcite @xcite and in other cases, graph representations to detect subjective bias in summarization @xcite and personas for dialogue generation @xcite . however, the bias measurements in many of these approaches are not directly applicable to language in a natural setting, where the real-life harmful impacts of bias in language generation would be more prevalent. ## conclusion we demonstrate through investigation of two tasks, counterpart generation and masked span prediction, that current llms have difficulty both generating and interpreting aal. our results show that llms do better matching the wording of gold standard references when generating wme than when generating aal, as measured by rouge and bertscore. human evaluation shows that llm output is more likely to be judged as human-like and to match the input dialect when generating wme than aal. notably, however, llms show difficulty in generating wme that matches the meaning and tone of the gold standard, indicating difficulty in interpreting aal. our results suggest that more work is needed in order to develop llms that can appropriately interact with and understand aal speakers, a capability that is important as llms are increasingly deployed in socially impactful contexts (e.g., medical, crisis).