Evaluation of African American Language Bias in Natural Language Generation
Authors: Nicholas Deas
Jessica A Grieser
Shana Kleiner
Desmond U. Patton
Elsbeth Turcan
Kathleen McKeown
Venue: EMNLP 2023 Main (2023)
Keywords: benchmarking large language models
african american language
bias and fairness
language generation

Abstract: While biases disadvantaging African American Language (AAL) have been uncovered in models for tasks such as speech recognition and toxicity detection, there has been little investigation of these biases for language generation models like ChatGPT. We evaluate how well LLMs understand AAL in comparison to White Mainstream English (WME), the encouraged "standard" form of English taught in American classrooms. We measure large language model performance on two tasks: a counterpart generation task, where a model generates AAL given WME and vice versa, and a masked span prediction (MSP) task, where models predict a phrase hidden from their input. Using a novel dataset of AAL texts from a variety of regions and contexts, we present evidence of dialectal bias for six pre-trained LLMs through performance gaps on these tasks.

Evaluation of African American Language Bias in Natural Language
Generation
Nicholas Deas Columbia University Department of Computer Science ndeas@cs. columbia. edu
Jessi Grieser
Shana Kleiner
University of Michigan
University of Pennsylvania
Department of Linguistics
School of Social Policy and
jgrieser@umich, edu
Practice, Annenberg School for
Communications skleine skleiner@upenn edu
Desmond Patton
Elsbeth Turcan Columbia University Department of Computer Science eturcanêcs. cow umlia. edu
Kathleen McKeown
University of Pennsylvania School of Social Policy and Practice, Annenberg School for Communications dupatton@upenn. edu
Columbia University Department of Computer Science
kathyêcs cttchyeccscooumb......... edu
Abstract
While there are largely beneficial and socially relevant applications of LLMs, such in allevi- ating barriers to mental health counseling' and medical hcaltheare (Hsu and Yu, 2022) access. there is also potential for biased models to exacer- bate existing societal inequalities (Kordzadeh and Ghasemaghaei, 2022; Chang et al., 2019; Bender et al., 2021). Past algorithms used in psychiatry and medicine have been shown to be racially biased, in some cases leading to, for example, underesti- mating patient risk and denial of care (Obermeyer et al., 2019; Straw and Callison-Burch, 2020). Fur- thermore, LLMs capable of understanding AAL and other language varieties also raise important ethical implications, such as enabling increased police surveillance of minority groups (see Patton et al. 2020 and section 8 for further discussion). Therefore, it is necessary investigate the poten- tial language variety biases of language generation models to both increase accessibility of applica- tions with high social impact and also anticipate possible harms when deployed.
Warning: This paper contains content and lan- guage that may be considered offensive some readers.
While biases disadvantaging African American Language (AAL.) have been uncovered mod- els for tasks such speech recognition and toxicity detection, there has been little investi- gation of these biases for language generation models like ChatGPT. We evaluate how well LLMs understand AAL in comparison to White Mainstream Finglish (WMF)), the encouraged "standard" form of English taught in American classrooms. We measure large language model performance two tasks; counterpart gener- ation task, where a model generates AAL given WME and vice versa, as well as a masked span prediction (MSP) task, where models predict phrase hidden from their input. Using a novel dataset of AAL texts from variety of regions and contexts, we present evidence of dialectal bias for six pre-trained LLMs through perfor- mance gaps these tasks.
1 Introduction
Moreover, prior work (Grieser, 2022) has shown that African American speakers talking about race- related issues use language in ways which may draw on morphosyntactic features of AAL in order to subtly foreground the race aspect of the discus- sion topic without explicit mention. Most training corpora include little representation of AAL (see further discussion in section 3), and those that do can still fail capture its significant regional and contextual variation (see Farrington al. 2021 for examples). Without the ability interpret these
Task-specifes models proposed for speech recog- nition, toxicity detection, and language identifica- tion have previously been documented to present biases for certain language varieties, particularly for African American Language (AAL) (Sap et al., 2022; Koenecke et al., 2020; Meyer et al., 2020; Blodgett and O' D'Cono,,, 2017). There has been lit- tle investigation, however, of the possible language variety biases in Large Language Models (LLMs) (Dong et al., 2019; Brown et al., 2020; Raffel et al., 2020), which have unified multiple tasks through language generation.
htt /tops://wwww.. wwai com/

---

subtler meanings of AAL, LLMS will undoubtedly exacerbate the misunderstandings which already take place between AAL speakers and other com- munities.
large language models on two language generation tasks: counterpart generation between language varieties and masked span prediction: (2) we use a novel dataset of AAL text from multiple con- texts (social media, hip-hop lyries, focus groups, and linguistic interviews) with human-annotatee annotated counterparts in WME; and (3) we document per- formance gaps showing that LLMs have more dif- ficulty both interpreting and producing AAL com- pared to WME; our error analysis reveals patterns of AAL features that models have difficulty inter- preting in addition to those that they can under- stand.
AAL
WME
Since RLD IILAD gone & thee ONLY WRK
Since Red head gose, working.
Source Test
only fling
WME
CharGiT Counterpart
Sinus ain't moumd my
Since
my hendl
head ain't right, and that's the
gonz and thatss the only thing
only thing kengin' me gring
that works
GPT-4
Since Red gume, my heed
Since Rel left hrad
Counterpart
thatss only
woekin'
hass't
and that's only thing warks.
Table I: Examples of ChatGPT and GPT-4 counterpart predictions. Given text in either WME or AAL, models attempt a semanticall--equivan language variety.
rewriting in the other
2 Background: Bias
In measuring AAL understanding, we identify evi- dence of bias through performance gaps and analy- sis of model behavior with each language variety. Following Blodgett et al. (2020), findings of bias could result in both allocational harms and repre- sentational harms posed by the evaluated models?. While L.MSs are becoming more available and valuable resources, the models' lack of understand- ing of AAL limits their use by AAL speakers, and this disparity will only grow as the use of these models increases across social spheres. Our eval- uation attempts to quantify these error disparities (Shah et al., 2020) by measuring models' under- standing of AAL and WME texts. When LLMs do not perform equally well on different language varieties, the LLM itself as resource becomes unfairly allocated, and speakers of minoritized lan- guage varieties like AAL are less able to leverage the benefits of LLMs. AAL speakers would be particularly unfairly impacted with applications in areas of health, including mental health.
Given the lack of African American representa- tion in LLMs and the possible harms to the AAL- speaking community, we focus on LLMs' under- standing of AAL to investigate biases. AAL is a language variety which follows consistent morpho- logical, syntactic, and lexical patterns distinct from WME, such as the dropped copula (c.g.. "she at work") and aspect markers (e.g., the habitual be: "he be running") (Lanehart, 2001; Green, 2009). We use Grieser (2022)'s 's definition of AAL as the grammatically patterned variety of English used by many, but not all and not exclusively, African Americans in the United States. Following Baker- Bell (2020) and Alim and Smitherman (2012), also use the definition of White Mainstream En- glish (WME) as the dialect of English reflecting the linguistic norms of white Americans. While previ- ous linguistic literature occasionally uses the terms "Standard American English" and "African Amer- ican Vernacular English." we employ AAL and WME instead to avoid the implication that AAL and other language varieties are "non-standard" and to more precisely identify the demographics of pro- totypical WME speakers, similarly Baker-Bell (2020) and Alim and Smitherman (2012). Exam- ples of AAL and WME are shown Table 1.
Additionally, our evaluation includes quali- tative analysis of how AAL is currently under- stood and produced by LLMs. Prior sociolinguis- tic works discuss and study how attitudes toward African American speakers have formed linguistic prejudices against AAL (Baker-Bell) Bell, 2020; Baugh, 2015), as well as how stereotyped uses of AAL by mon-AAL speakers can perpetuate racial divides (Ronkin and Karn, 1999). Stereotypical or offen- sive uses of AAL by LLMs thus reflect repre- sentational harm to AAL speakers that can further
We evaluate understanding of AAL by LLMs through production of language in each variety us- ing automatic metrics and human judgments for two tasks: a counterpart generation task akin to di- alect translation (Wan et al., 2020; Harrat et al., 2019) (see examples in Table 1) and a masked span prediction (MSP) task where models predict a phrase that was removed from their input, similar to Groenwold et al. (2020). We summarize our contri- butions as follows: (1) we evaluate six pre-trained,
? "Allocational harms are reflected in the unfair distribution of resources and opportunities among social groups, while representational harms are reflected in disparate or harmful representations of particular group (see Blodgett et al. (2020) for further discussion).

---

promote these views. We advocate for approaches which carefully consider sociolinguistic variation in order to avoid generation of inappropriate speech across different settings.
Black artists from Morgan (2001) and Billboard's 2022 Top Hip-Hop Artists. Finally, we use the tran- scripts of 10 focus groups concerning grief and loss in the Harlem African American community and conducted as part of ongoing work by the authors to better understand the impacts of police brutal- ity and other events on the grief experiences of African Americans. Following Bender and Fried- man (2018), data statement with further details is included in Appendix A.
3 Data
Biases in pre-trained language models can often be attributed to common training datasets. Training corpora for LLMs are typically drawn from internet sources, such as large book corpora, Wikipedia, out- bound links from Reddit, and a filtered version of Common Crawl in the case of GPT-3 (Brown et al., 2020), which can severely under-represent the lan- guage of African Americans (Pew Research Center, 2018; Dolcini et al., 2021). Though few estimates of the presence of AAL in datasets exist, one study estimates that in the Colossal Cleaned Crawl Cor- pus (C4), only 0.07% of documents reflect AAL (Dodge al., 2021). Beyond C4, African Amer- icans are significantly underrepresented in data sources such as Wikipedia (0.05%) and news articles (6%; Pew Research Center 2023), falling well below the national average. Additionally, as models learn from gold standard outputs provided by annotators, they learn to reflect the culture and values of the annotators well.
50 texts are sampled from cach dataset, result- ing in 300 candidate texts in total. We use a set of surface level and grammatical patterns to ap- proximately weight each sample by the density of AAL-like language within the text (patterns are listed in Appendix B). 12 additional texts are also sampled from each dataset for fine-tuning.
3.2 Data Annotations
Our interdisciplinary team includes computer sci- entists, linguists, and social work scientists and thus, we could recruit knowledgeable annotators to construct semantically-ecquiialee re-writings of
AAL texts into WME, referred to as counterparts.
3.1 Data Sources
The four human annotators included 2 linguistics students, 1 computer science student, and social work scientist, all of whom self-identify as AAL speakers and thus have knowledge of the linguis- tic and societal context of AAL and racial biases. These annotators were familiar with both AAL and WME, allowing them to provide accurate anno- tations and judgements of model generations both language varieties. Annotators were asked to rewrite the AAL text in WME, ensuring that the counterparts conserve the original meaning and tone closely as possible (see Appendix C.
There is significant variation in the use of features of AAL depending on, for example, the region or context of the speech or text (Washington et al., 1998; Hinton and Pollock, 2000). Accordingly, we collect a novel dataset of AAL from six different contexts. We draw texts from two existing datasets, the TwitterAAE corpus (Blodgett et al., 2016) and transcripts from the Corpus of Regional African American Language (CORAAL; Kendall and Far- rington 2021), well as four datasets collected specifically for this work: we collect all available posts and comments from r/BlackPeopleTTwiter belonging to "Country Club Threads", which des- ignates threads where only Black Redditors and other users of color may contribute 6. Given the influence of AAL on hip-hop music, we collect hip lyrics from 27 songs, from each of
To compute inter-annolater agreement, we asked each annotator to label the 72 additional texts, and they also shared a distinct 10% of the remainder of the dataset with each other annotator. We com- pute agreement using Krippendorff's alpha with Levenshtein distance (Braylan et al. 2022; see Ap- pendix D for more details) showing 80% agreement (a = 88000. After removing pairs from the dataset where annotators determined that no counterpart exists, the final dataset consists of 346 AAL-WME text pairs including the 72 additional texts. Dataset statistics are included in Table 2.
"nttp://commonerall. org/
"https://meta. wikintiiaiiiiiiiiiaa.aa.. www.eeiii..rgggwieeeeeeeee wttg//iki//oommnit..... ty- SeapghsssCommmiit___ssiieeee 2021 Report/ ThrivinggMovement
'nttps//reedit..om /ttpetcee////acceepleeeeeee
"To be verified as a person of color and allowed to con- tribute to Country Club Threads, users send in pictures of their forearm to reveal their skin tone.

---

Dalaset
+ Samples
Ang. Length (AAL)
Arg Length (WHSEE)
Ronge-1
Arg. (oV
AFg. Tox (WAL))
AIPP Commants
60
24:30
26.35
35.3
0.40
(1:39
xRPT
TwitteAAEE (Blolpet
CORAAT, (Kondall and Farrington, 2021)
94.1
0.09
Flocus Groups
29.20
26.56
71.4
0,06
0.07
Hip- Hop Lynis
57
9.39
10.58
67.5
0.47
0.40
AAL Total
346
16.23
16.60
77.4
0.33
0.25
Table 2: Characterization of the novel AAL dataset by text source including the number of text samples, length (in words) of aligned AAL and WME texts, Rouge-1 between aligned texts, and the average toxicity scores among dialects.
Translate following African American Vernacular English into Standard American English:
its chat-oriented successor, ChatGPT (GPT-3.5)*. GPT-4 (OpenAI, 2023), currently OpenAl's most advanced language model; T5 (Raffel et al., 2020); its instruction-tung variant, Flan-T5 (Chung al., 2022); and BART (Lewis al., 2020). Flan-T5, GPT-3, ChatGPT, and GPT-4 are evaluated the counterpart generation task, while GPT-3, BART, and T5 are evaluated on the MSP task. We note that the GPT models besides GPT-3 were not in- cluded in the MSP task because token probabilities are not provided by the OpenAl API for chat-based models. An example of the instruction provided to GPT models is provided in Figure 1. Notably, the instruction text provided to GPT models uses "African American Vernacular English" and "Stan- dard American English" because prompts with these terms were assigned lower perplexity than "African American Language" and "White Main- stream English" by all GPT models, and lower perplexity prompts have been shown to improve task performance (Gonen et al., 2022). Addition- ally, GPT models are simply asked to translate with no additional instructions in order to examine their natural tendency for tasks involving AAL text. We evaluate both Flan-T5 fine-tuned on the 72 ad- ditional texts, referred to as Flan-T5 (FT) in the results, and Flan-T5 without fine-tuning (with auto- matic metrics only). Additional modeling details and generation hyperparameters are included in Appendices E.1 and E.2.
and ain't sixteen years old, this shit has got to stop. => And he is not sixteen years old, this shit has got to stop.
Figure I: Example prompt provided to GPT models in the counterpart generation task including the instruction and AAL text for which the model generates WME counterpart (blue).
4 Methods
We evaluate multiple language generation mod- els using two tasks. In the counterpart genera- tion task, we evaluate models on producing near semantically-equivaaeee WME text given AAL text and vice versa to target LLMs' ability to interpret and understand AAL. second task, masked span prediction, requires models to predict tokens to re- place words and phrases hidden or masked from the input. This task resembles that of Groenwold al. (2020), but spans vary in length and posi- tion. Much like BART pre-training (Lewis al., 2020), span lengths are drawn from a Poisson dis- tribution (A = 2) and span locations are sampled uniformly across words in the original text. We in- dependently mask noun phrases, verb phrases, and random spans from the text for more fine-grained analysis.
While our focus on measuring model capabil- ities in interpreting AAL?, these generation tasks allow us to test whether the model understands the language well enough to produce it. It is not our goal to produce LLM that can generate AAL within the context of downstream tasks (see sec- tion 8 for further discussion).
4.2 Metries
We use both automatic and human evaluation met- rics for the counterpart generation task. As with most generation tasks, we first measure n-gram overlap of the model generations and gold stan- dard reference texts and in our experiments, we utilize the Rouge metric. In addition, to account for the weaknesses of word-overlap measures, we also measure coverage of gold standard references with BERTScore (Zhang* et al., 2020) using the
4.1 Models
We consider six different models for the two tasks where applicable: GPT-3 (Brown et al., 2020);
'In reference to model capabilities, "interpretation" and "understanding" refer to the ability of models to accurately enende the meaning and features of text in AAL and WMF as opposed to cognitive notions of these terms.

---

WME AAL
AAL WMB
50
101
100
Flun To (PT)
Plac
ChxtCPP
20
Monde
Figure 2: Automatic coverage metrics of moner-generatedd AAL and WME counterparts. "Human" (purple) scores represent coverage metrics between the original AAL text and human-annotated WME counterparts. Significant differences between scores in the WME - AAL direction and in the AAL - WME direction are denoted by (ps .05).
WME
Bomon Inlpond
Buna
Hon 15 (FT) GPT-3
Inligtan Biuxtoku
Figure 3: Human judgments for model- gener-eeeeeated AAL and WME counterparts. "Human" (purple) scores represent judgments of original AAL text and human annotated WME counterparts. Significant differences between scores in the WME - AAL direction and in the AAL - WME direction are denoted by (pS .05). Flan-T5 without fine-tuning was evaluated with automatic metrics after human judgements were collected.
microsof//dcbertttllrr----- checkpoint, because it is better correlated with human scores than other models". Specifically, original AAL is the gold standard for model-generated AAL and human an- notated WME counterparts are the gold standard for model-generated WME. some experiments, Rouge-1, Rouge-L, and BERTScore are presented as gaps, where scores for generating WME are sub- tracted from those for generating AAL Due to the tendency of models to avoid toxic outputs and neu- tralize text, also consider the percentage of toxic terms removed when transitioning from model in- puts one language variety to outputs in the other. Toxicity scores are derived as the number of words categorized as offensive in the word list of Zhou et al. (2021), and percent change between inputs and outputs calculated as Tozin
and annotated counterparts are randomly sampled from the dataset for human evaluation. We en- that annotators do not rate human model- generated counterparts for which they initially gen- erated the WME counterpart. All annotators are asked to rate cach assigned counterpart using 5- point Likert scales on the dimensions below.
Human-likeness (oaneeeesssmmarn (Human--ike)) measures whether the annotator believes that the text was generated by human or language model. Linguistic Match (Dialect) measures how well the language of the counterpart is consistent with the intended English variety (i.e., AAL WME). Meaning Preservation (Meaning) measures how accurately the counter- part conveys the meaning of the original text. And finally, Tone Preservation (Tone) measures how ac- curately the counterpart conveys the tone or other aspects beyond meaning of the original text. Ad- ditional details on the judgment instructions are included in Appendix C.2.
Human evaluation is also conducted on the gen- erated counterparts. The same linguisties student, computer science student, and social work scien- tists involved in creating the dataset of aligned counterparts were also asked to judge model gen- erations. As a baseline, human-generated counter- parts are included in the human evaluation. 100 WME and AAL texts along with their generated
In the masked span prediction task, span predic- tions are evaluated using automated metrics: model perplexity of the reference span, and the entropy of the model's top most probable spans. With the exception of GPT-3, experiments are repeated times, randomly sampling spans mask in each trial. Metries are reported as the percent change in perplexity between WME and AAL.
'https: //docs. www.g//ll.........eee//ooooooee c
201 t01tpp0ppwWun177x1
edit

---

5 Results
models consistently perform better when generat- ing WME, indicating that it harder for models reproduce similar content and wording as the gold standard when generating AAL. ChatGPT the worst model for producing WME from AAL., and ChatGPT and GPT-3 are nearly equally bad at pro- ducing AAL from WME. Flan-T5 (FT) does best for both language varieties, likely due to the fact that Flan-T5 (FT) was directly fine-tuned for the task. Flan-T5 without fine-tuning performs compar- atively with slightly lower coverage scores in both directions. We also compute the coverage scores between the original AAL text from the dataset and the human annotated counterparts in WME, labeled as "Human" in Figure 2. Models tend to generate counterparts with lower coverage scores than the text input to the model, which reflects the alternative language variety. This suggests that it is difficult for models to generate counterparts in either direction.
5.1 Are AAL and WME Metries Comparable?
Studies such Bugliarello al. (2020) might sug- gest that in translation- tasks, it is invalid to compare results from automatic metrics, such as BLEU, cross-lingually because: (1) different lan- guages may use different numbers of words to con- vey the same meaning, and (2) models for different languages utilize different tokenization schemes.
Though we emphasize that AAL and WME are language varieties of English rather than distinct languages, a similar argument may made that their Rouge scores are not directly comparable. However, the counterpart generation task setting does not suffer from either of the aforementioned weaknesses. To show this, we calculate differences in the number of words and 1-gram Type- Token Type-Token Ratio for AAL and WME text pairs in our dataset. As shown in Table 3, the total number of words in the AAL and WME texts are similar, and we find that the lengths of each pair of texts differ by less than 1/10th of word (0.095) on average. Bugliarello et al. (2020) also finds that among met- ries studied, translation difficulty is most correlated with the Type-Token Ratio (TTR) of the target lan- guage. Table shows that the difference in the 1-gram TTR between AAL and WME not sta- tistically significant. Finally, as the same models are applied to both AAL and WME texts, the tok- enization schemes are also identical. Therefore, the identified weaknesses of cross lons-llngual comparison do not upply to our results.
Figure 3 shows human judgments of model- generated WME and model-generated AAL. With the exception of Flan-T5 (FT)) we see that model- generated WME is judged as more human-like and closer to the intended language variety than mhodel-generaan AAL. These results confirm find- ings from automatic metrics, showing that models more easily generate WME than AAL. In contrast, for meaning and tone, the reverse is true, indicating that models generate WME that does not match the meaning of the original AAL.. The difference between scores on AAL and WME were signifi- cant on all metries for at least two of the models as determined by two-tailed t-test of the mcans (see in Figure for models with significant dif- ferences). We see also that the drop in meaning and tone scores from judgments on human WME is larger than the drop in human-like and dialect scores WME. These observations suggest that models have hard time interpreting AAL
Dialect AAL
# Words
TTR 0.274 (0.259, 0.290) 0.260 (0.245, 0.275)
5632
WME
5665
Table Comparison of Type-TToee Token Ratios between the AAL and WME texts in the dataset. 95% confidence intervals calculated using the Wilson Score Interval are shown in parenthesis.
Romood
11.
Fise-TB (PT)
Flae-Ts
GPT-
5.2 Counterpart Generation
ChaaGPT
0.2
GPTT-1
Figure 2 shows results using automatic coverage metrics on counterpart generations in AAL and WME. Rouge-1, Rouge-L and BERTScore (the coverage scores) for model output are computed over the generated AAL or WME in comparison to the corresponding gold standards. We note that the
AAL
AAl.
WMR
Dirosion
Figure 4: Percentage of toxicity removed for AAL and the respective aligned WME counterparts.

---

that gaps in coverage are consistently larger for the non-toxic subsets of the data, demonstrating that the use of profanity and toxic language are also not the primary cause of gaps in coverage metrics.
Jolqnuu Gip
IFT)
Flo
Charcer
GPE-I
-10
5.3 Masked Span Prediction
Thate
Not-
Julancst
AsL Pellersus Bella
Figure Gaps in Rouge-1 metric in non-toxic and toxic subsets of the AAL dataset. Negative Rouge gaps indicated greater WME performance than AAL.
10
BART
AAL Rakossss Wene
Saloven Gap
L00
Rander
Nostn
0m
8o0
ChaiGPT
Figure 7: Percent difference in perplexity and top-5 entropy between AAL and aligned WME texts. Nega- tive percentages indicate lower WME perplexity/entropy than AAL perplexity/entropy.
8001
Town
Mexman
Tim
Tosie
For MSP, we measure both perplexity and the en- tropy generating a masked span in either AAL or WME. A lower perplexity score indicates it is eas- ier for the model to determine the missing phrase, while lower entropy score indicates the model places high probability in its top predictions. Fig- 7 shows the differences in perplexity and en- tropy between masked span prediction for model- generated WME and for model -ener-egeeerteed AAL.
Bilosk
Figure Gaps human judgments of human and model-generated counterparts broken down into toxic and non-toxic subsets. Negative scores indicate better WME performance.
Toxicity scores (Figure 4) show that models tend remove toxic words when generating both AAL and 1ME To test whether removal of toxic words contributed the inability to preserve mean- ing, we computed both coverage scores and hu- man evaluation scores on two subsets of data: "toxic" texts (at least one term categorized as offen- sive) and "non-toxic" texts (no terms categorized as offensive). We display these results as the dif- ference in coverage scores and in human judgment scores between model generated AAL and WME shown in Figure and Figure 6. Positive scores indicate that AAL performs better. Here we see that human judgments on meaning and tone show that generated WME is worse than generated AAL for both toxic and non-toxic subsets. Thus, differences in use of toxic words between input and output can- not be the sole cause for lower scores meaning and tone. This confirms that models have difficulty interpreting features of AAL. We note furthermore
Negative percent changes in perplexity indicate that it is easier for models to predict spans in WME than AAL, while for entropy, indicate that models place higher probability in their top predictions for WME than for AAL sentences.
Discussion: How well do models interpret AAL?
We discussed earlier how Figure and Figure 6 demonstrate that models have difficulty interpreting AAL when generating WME. Figure 7 supports this finding well, models generally output higher perplexities for masked spans in AAL compared to aligned WME.
The largest gaps in perplexity between the two language varieties is assigned to masked verb phrases. One set of distinct features characteriz- ing AAL are verbal aspects which do not occur in WME such as the future gone (e.g., I'm gone do it later), so this result may suggest that models struggle with the use of AAL-specific aspects in particular over other AAL features. similar trend found in the entropy metric, suggesting that AAL
10 Models are developed to avoid generating toxic or offen- sive language, so the trend of neutralizing input texts in any dialect is expected. There are notable differences. however, in the extent to which this neutralization occurs. The results show that significantly higher proportion of toxic language removed when generating WME from AAI. than in the reverse direction.

---

GPS CUTFACHRE
ing "youth" instead of its more common meaning of "girlfrrendd. The models also struggle with fea- tures that appear the same as in WME, but have slightly different meanings in AAL. These include remote past been (example 2), which is incorrectly interpreted as past perfect (have been), and exis- tential if (example 3), which in WME is closest in meaning to "there" as in "there are ..." and is not correctly interpreted by any model.
Theee Brane
WSF.A
3kire
:IPT Cenoar
TettueAA diwidatttttttttt 2011
Joca Gruap Thaaedgs
We also include an example where GPT-4 mis- interprets the phrase "a nigga" as referencing an- other person, when in the provided context, the use most closely resembles referencing oneself. The word nigga, is one of the N-words, set of lexi- cal items well-documented by linguists as being misunderstood by non-native speaker in terms of their syntactic and semantic complexity (Rahman, 2012: Grieser, 2019; Smith, 2019). In particular, while the model removes the word in generating the WME counterpart, it does not correctly understand the use of the N-word as referencing the subject.
CJI Poos
Hipttpp Andta
Figure& 8: Breakdown of Rouge score gaps and per- cent changes in toxicity by data source in counterpart generation. Negative Rouge values indicate higher WME Rouge scores, and positive AT scores indicate the model-geneeated counterpart is more toxic than the gold standard.
Ex I: Source AAL
shili I'm tryno get my thingg guin" ton. Just bring my car back. shilt, faded.
WME
F'm trylng
Jost bring
Test
shil,
Hey, I'm trying to gui my thing going, ino Just bring my
Flan-T3 (FT) WME
car back. Hey, le faded.
Flan-T5 (FT) AAL
Shir, I'm myinng to do own thing, toc. Jus bring car back, shi,, he's high.
Without this understanding, it is probable that mod-
Ex E. Sonres AAI.
boun wilding since juvi, she goed girl.
Ansotated WME
good
els will both misinterpret the words as toxic and
ChaiGPT WME
detention, bui she was well belaved girl.
belussing rackinsly since my time jevemnle
use them in ways that are considered offensive the ApL-speaking community.
ChatGP AAL
I been wildia sunce was a shonty shee was straugh
good girt.
Ex 3: Source AAL
It used to be brokes contrellers.
In additional analysis of counterpart generations, we examined model performance gaps in each sub- set of the AAL dataset. Among subsets of the data, gaps between Rouge-1 metrics for AAL and WME counterparts vary significantly. GPT-4, for example, presents the largest performance gap for the TwitterAAE corpus (Blodgett and O'Connor, 2017), and the smallest gaps for the hip-hop and focus group subsets as shown in Figure 8. Manual inspection reveals that this aligns with the trends in AAL-use among the subsets as well: distinet features of AAL appear to be more frequent in the TwitterAAE dataset, while AAL features in the focus group transcripts appear to be more sparse. This pattern may be due to the makeup and con- text of the focus groups, as most participants were college-educated, working professionals and se- lected to specifically describe grief experiences, possibly affecting the use of AAL in the discus- sions. These results may suggest, as would be ex- pected, that higher density of AAL features leads to larger performance gaps.
Ansolated WME
There used to be broken controllers.
GPT4
used to be controlless would break.
they gonna really keep
Ex 4: Source AAL
nlagss ktt nigga know what fuck really going on.
Annotated WME
people let me know what the fack is rollyy going on.
And they are not really going to tell the truth when
GPT-4 WME
And they
be completely whitt las pepiini...
people
Table 4: Flan-T5 (FT), ChatGPT and GPT4 examples of model counterparts neutralizing the input text (red) and misinterpreting features of AAL or terms the input text (blue).
text also lowers model confidence in their own pre- dictions. These results for AAL support similar findings by Groenwold et al. (2020) for GPT-2 in completom setting.
Manual inspection revealed more fine-grained patterns of model behavior within aspectual verbs and other AAL features. Models seem to correctly interpret some specific features of AAL, namely: the use of ain't, 1, double negation, and habitual be. Examples of misinterpretation, however, are shown in Table 4 illustrating difficulty with several other aspects of AAL. Several mistakes involve lexical interpretation, such as in example 1, where the model is not able to interpret the meaning of "he faded" as "he's high" and example 2 where the model inserts shorty apparently intending the mean-
Related Work
While few have specifically focused on bias against AAL in language generation, related work has ex-

---

tensively investigated societal biases in language tasks. One large-scale study (Ziems al., 2022) investigates performance on the standard GLUE benchmark (Wang et al., 2018) using a synthetically constructed AAL dataset of GLUE for fine-tuning. They show performance drops on a small human- written AAL test set unless the Roberta model is fine-tuned.
stream, user-facing tasks. With the phonological differences between AAL and WME, automatic speech recognition (ASR) systems have shown large performance drops when transcribing speech from African American speakers (Koenecke et al., 2020; Martin and Tang, 2020; Mengesha et al., 2021). Toxicity detection and offensive language classification models have also been evaluated and have shown a higher probability of incorrectly la- beling AAL text as toxic or offensive when com- pared to WME text (Zhou et al., 2021; Rios, 2020; Sap et al., 2022). Most closely related to this work, one study evaluated bias against AAL in transformer generation models, showing that in a sentence auto-completion setting, GPT-2 generates AAL text with more negative sentiment than in aligned WME texts (Groenwold al., 2020). Fur- ther investigation of both a larger set of language generation models well as a broader set of gen- eration tasks would provide a clearer picture of model biases against AAL
Racial Bias in Generation. Mitigating and eval- uating social biases in language generation models is a challenging problem due to the apparent trade- offs between task performance and bias mitigation, the many possible sources of bias, and the vari- ety of biases and perspectives to examine (Sheng et al., 2021b; Akyürek et al., 2022). A number of studies have proposed bias evaluation measures, often using prompts crafted to reveal biased associ- ations of, for example, occupation and gender (i.e., "The mman/woman] worked as a a.."") (Sheng et al.,
2020, 2019; Kiritchenko and Mohammad, 2018; Dhamala et al., 2021; Shen et al., 2022) and in other cases, graph representations to detect subjec- tive bias in summarization (Li et al., 2021) and per- sonas for dialogue generation (Sheng al., 202 However, the bias measurements in many of these approaches are not directly applicable language in a natural setting, where the real-life harmful im- pacts of bias in language generation would be more prevalent.
8 Conclusion
We demonstrate through investigation of two tasks, counterpart generation and masked span prediction, that current LLMs have difficulty both generating and interpreting AAL. Our results show that LLMs do better matching the wording of gold standard references when generating WME than when gener- ating AAL, as measured by Rouge and BERTScore. Human evaluation shows that LLM output is more likely to be judged as human-like and to match the input dialect when generating WME than AAL. Notably, however, LLMs show difficulty in gener- ating WME that matches the mcaning and tone of the gold standard, indicating difficulty in interpret- ing AAL. Our results suggest that more work is needed in order to develop LLMs that can appropri- ately interact with and understand AAL speakers, a capability that is important as LLMs are increas- ingly deployed in socially impactful contexts (e.g.. medical, crisis).
AAL Feature Extraction. Past work makes progress in lowering performance gaps between AAL and WME by focusing on linguistic feature extraction tasks. Given that some features of AAL such as the aspectual verbs (i.c., habitual be, remote past been) do not have equivalent meanings and functions in WME (Green, 2009), standard part- of-speech (POS) taggers and dependency parsers cannot maintain performance for AAL text. Stud- ies have attempted to lessen this gap by creating POS tagger specifically for AAL through domain adaptation (Jørgensen al., 2016) and a depen- dency parser for AAL in Tweets (Blodgett et al., 2018). Beyond these tasks, considerable attention has been given to developing tools for features spe- cific to AAL and other language varicties, such as detecting dialect-specific constructions (Masis et al., 2022; Demszky et al., 2021; Santiago et al., 2022; Johnson et al., 2022) to aid in bias mitigation strategies.
Limitations
We acknowledge a few limitations accompanying the evaluation of biases in LLMs. While our analy- sis is primarily restricted to intrinsic evaluation of model biases, users primarily interact with LLMs in a chat-based interface such as with ChatGPT, or use the model for specific tasks such as question answering. This approach was chosen to analyze
AAL in Language Tasks. Bias has also been measured specifically with respect to AAL in down-

---

biases that would be present across all tasks involv- ing AAL. Performance gaps and biases analyzed in a task-spe setting, however, may yield different trends than presented in this paper, and we leave this investigation to future work.
practices and interactions (Delgado and Stefancic, 2023). Without these considerations, LLMs capa- ble of understanding AAL could inadvertently be harmful in contexts where African Americans con- tinue to be surveilled (e.g., social media analysis for policing).
Additionally, AAL exhibits significant variation by region, context, speaker characteristics, and many other variables. We attempt to more com- prehensively reflect real AAL use by drawing text from multiple sources and contexts, but are ulti- mately limited by the data available. For example, while CORAAL reflects natural AAL speech, it is limited to a select set of regions (e.g., New York, Georgia, North Carolina, Washington DC), and while the Twitter and Reddit AAL subsets may span many regions, they are also influenced by the linguistic features of social media. Similar biases may also exist in other underrepresented varieties of English such as Mexican American English, In- dian English, or Appalachian English. Due to the availability of data, we focus on AAL, but given texts in other varieties, this work could be extended to examine biases regarding these and other lan- guage varieties.
Despite this, including African American rep- resentation in language models could potentially benefit AAL speakers in socially impactful areas, such as mental health and healthcare (e.g.. patient notes that fully present the pain African American patients are experiencing in the emergency room, Booker et al. 2015). Considering both the poten- tial for misuse of the data as well as the potential for social good, we will make the Tweet IDs and other collected data available to those that have signed an MOU indicating that they will the data for research purposes only, limiting research to improved interpretation of AAL in the context of an application for social good. In the MOU, ap- plicants must include their intended use of the data and sign an ethics agreement.
A Acknowledgements
Finally, evaluation metrics relying on trained models or lexicons, such as BERTScore and toxi- city measures, may also inherently encode biases concerning AAL text. Rather than using model to measure toxicity, we instead use lexicon of offen- sive terms provided in Zhou et al. (2021) and used measure lexical biases toxicity models. Given that analyzing performance gaps relies on accu- rate and unbiased measures of model performance, future work may give attention to developing unbi- ased language generation metrics.
This work was supported in part by grant IIS- 2106666 from the National Science Foundation, National Science Foundation Graduate Research Fellowship DGE-2036197, the Columbia Uni- versity Provost Diversity Fellowship, and the Columbia School of Engineering and Applied Sci- ences Presidential Fellowship. Any opinion, find- ings, and conclusions or recommendations ex- pressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. We thank the anonymous re- viewers and the following people for providing feedback an earlier draft: Tuhin Chakrabarty, Esin Durmus, Fei-Tzin Lee, Smaranda Muresan, and Melanic Subbiah. We also thank Mayowa Fageyinbo. Gideon Kortenhoven, Kendall Lowe, and Tajh Martin for providing annotations.
Ethics Statement
We recognize that identifying potential bias against AAL in LLMs should also include a critically re- flexive analysis of the consequences if language models are better at understanding language vari- eties specific to marginalized communities such as AAL, and the extent to which that impacts those speakers. In prior research, Patton et al. (2020) have noted that decisions made by researchers engaged in qualitative analysis of data through language processing should understand the con- text of the data and how algorithmic systems will transform behavior for individual, commu- nity, and sestem-level audiences. Critical Race Theory posits that racism exists across language
References
Afra Feyza Akyürek, Muhammed Yusuf Kocyigit, Sejin Paik, and Derry Tanti Wijaya. 2022. Challenges in measuring bias via open-ended language generation. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 76-76, Seattle. Washington. Association for Compu- tational Linguistics.
H Samy Alim and Geneva Smitherman. 2012. Articu-

---

late while Black: Barack Obama, language, and race in the US. Oxford University Press.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Tiric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
April Baker-Bell. 2020.
Linguistic justice:: Black lan- guage, literacy, identity, and pedagogy. Routledge.
John Baugh. 2015. 755SWB (Speaking while Black); Linguistic Profiling and Discrimination Based on Speech as a Surrogate for Race against Speakers of African American Vernacular English. In The Oxford Handbook of African American Language. Oxford University Press.
Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos, Ryan Cotterell, and Naoaki Okazaki. 2020. It's casier to translate out of English than into it: Measuring neural translation difficulty by cross- mutual information. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics, pages 1640-1649, Online. Association for Computational Linguistics.
Emily M. Bender and Barya Friedman. 2018, Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587-604.
Emily M. Bender, Timnit Gebru, Angelina McMillan-
Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language mod- els be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Trans- parency, FAceT '21, page 610-623, New York, NY, USA. Association for Computing Machinery.
Kai-We Wei Chang, Vinodkumar Prabhakaran, and Vicente Ordonez. 2019. Bias and faimess in natural language processing. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Process- ing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)s Tutorial Abstracts, Hong Kong, China. Association for Computational Linguisties.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Pro- ceedings of the 58th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 5454- 5476, Online. Association for Computational Lin- guistics.
Hyung Won Chung, Le Hou, Shayne Longpre. Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang. Mostafa Dehghani, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery. Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai. Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Ja- cob Devlin, Adam Roberts, Denny Zhou, Quoe V. Lc, and Jason Wei. 2022. Scaling cearuuction-inetunee language models.
Su Lin Blodgett, Lisa Green, and Brendan O'Connor. 2016. Demographic dialectal variation in social English.
media: A case study of African- In Proceedings of the 2016 Conference on Empiri- cal Methods in Natural Language Processing, pages 1119-1130, Austin, Texas. Association for Computa- tional Linguistics.
Su Lin Blodgett and Brendan O'Connor. 2017. Racial disparity in natural language processing: A case study of social media african- english.
Richard Delgado and Jean Stefancic. 2023. Critical race theory: An introduction, volume 87. NyU press.
Su Lin Blodgett, Johnny Wei, and Brendan O' O'Connor. 2018 Twitter Universal Dependency parsing for African-America and mainstream American En- glish. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Vol- ume I: Long Papers), pages 1415-1425, Melbourne, Australia. Association for Computational Linguistics.
Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, and Jacob Eisenstein. 2021. Learning to recognize dialect features. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 2315-2338 Online. Association for Computa- tional Linguistics.
Staja "Star" Booker, Chris Pasero, and Keela A. Herr. 2015. Practice recommendations for pain assessment by self-report with african american older adults. Geriatric Nursing, 36(1):67-74.
Jwala Dhamala. Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. BOLD. In Proceedings of the 2021 ACM Conference on Fairness, Accountabiliy; and Transparency. ACM.
Alexander Braylan, Omar Alouso, and Matthew Lease, 2022. Measuring annotator agreement generally across complex structured, multi-object, and free- text annotation tasks. In Proceedings of the ACM Web Conference 2022. ACM.
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting

---

large webtext corpora: A case study on the colos- sal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1286-1305, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Alexander Johnson, Kevin Everson, Vijay Ravi, Anissa Gladney, Mari Ostendorf, and Abeer Alwan. 2022. Automatic Dialect Density Estimation for African American English. In Proc. Interspeech 2022, pages 1283-1287.
Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. Learning a POS tagger for AAVE-like language. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 1115-11220 San Diego, California. Associa- tion for Computational Linguistics.
M Margaret Dolcini, Jesse A Canchola, Joseph A Catania, Marissa M Song Mayeda, Erin I. Di- etz. Coral Cotto-Negrón, and Vasudha Narayanan. 2021. National-level disparities in internet access among low-income and black and hispanic youth: Current population survey. J Med Internet Res, 23 (I))):227733.
Tyler Kendall and Charlie Farrington. 2021. The corpus of regional african american language.
Li Dong, Nan Yang, Wenhui Wang. Furu Wei, Xi- aodong Liu, Yu Wang. Jianfeng Gao, Ming Zhou, and Hsiao- Hon. 2019. Unified language model pre-training for natural language understanding and generation. In Advances in Neural Information Pro- cessing Systems, volume 32. Curran Associates, Inc.
Svetlana Kiritchenko and Saif Mohammad. 2018. Ex- amining gender and race bias in two hundred senti- ment analysis systems. Proceedings of the Sev- enth Joint Conference on Lexical and Computational Semantics, pages 43-53, New Orleans, Louisiana. Association for Computational Linguisties.
Charlie Farrington, Sharese King, and Mary Kohn. 2021. Sources of variation in the speech of african amer- icans: Perspectives from sociophonetics. WIREs Cognitive Science, 12(3).
Allison Koenecke, Andrew Nam. Emily Lake, Joe Nudell, Minnie Quartey. Zion Mengesha, Con- nor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences of the United States of America, 117(14)7684-7689
Hila Gonen. Srini lyer, Terra Blevins, Noah A. Smith, and Luke Zeutlemoyrr 2022. Demystifying prompts language models via perplexity estimation.
Lisa Green. 2009. African American English: Linguistic Introduction. Cambridge Univ. Press.
Nima Kordzadeh and Maryam Ghasemaghaei. 2022. Algorithmic bias: review, synthesis. and future re- search directions. European Journal of Information Systems, 31(3):388-409
Jessica A Grieser. 2019. Toward understanding the n- words. American Speech: A Quarterly of Linguistic Usage, 94(4):409-41) 419.
Sonja L. Lanehart. 2001. Sociocultural and historical contexts of African American English. John Ben- jamins Publishing.
Jessica A Grieser. 2022. The Black side of the river: Race, language, and belonging in Washington, DC. Georgetown University Press.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising suuienceeto-sequenee pre- pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 787 7871-7880, Online. Association for Computa- tional Linguistics.
Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli. Sharon Levy, Diba Mirza, and William Yang Wang. 2020. Investigating African- American Vernacular English in transformer-based text generation. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP), pages 5877-5883, Online. As- sociation for Computational Linguistics.
Lei Li, Wei Liu, Marina Litvak, Natalia Vanetik, Ji- acheng Pei, Yinan Liu, and Siya Qi. 2021. Subjective bias in abstractive summarization.
Salima Harrat, Karima Meftouh, and Kamel Smaili. 2019. Machine translation for arabic dialects (sur- vey). Information Processing & Management, 56(2):262 22)22, Advance Arabic Natural Language Processing (ANLP) and its Applications.
Joshua L. Martin and Kevin Tang. 2020. Understanding Racial Disparities in Automatic Speech Recognition: The Case of Habitual "be". In Proc. Interspeech 2020, pages 626-630.
Linette N. Hinton and Karen E. Pollock. 2000. Re- gional variations in the phonological characteristies of african american vernacular english. World En- glishes, 1915779-71.
Tessa Masis, Anissa Neal, Lisa Green, and Brendan O'Connor. 2022. Corpus-guided contrast sets for morphosyntactic feature detection in low-resource English varieties. In Proceedings of the first work- shop on NLP applications to field linguistics, pages 11-25, Gyeongju, Republic of Korea. International Conference on Computational Linguistics.
I-Ching Hsu and Jiun-De Yu. 2022. A medical chat- bot using machine learning and natural language understanding. Multimedia Tools and Applications, 81(17)23777--3399.

---

Zion Mengesha, Courtney Heldreth, Michal Lahav, Ju- liana Sublewski. and Elyse Tuennerman. 2021. "i don't think these devices are very culturally sensi- tive." live.--ine iivpp-i of automated speech recognition er- rors on african americans. Frontiers in Artificial Intelligence, 4.
Harrison Santiago, Joshua Martin, Sarah Moeller, and Kevin Tang. 2022. Disambiguation of morpho- syntactic features of African American English - the case of habitual be. In Proceedings of the Second Workshop on Language Technology for Equality, Di- versity and Inclusion, pages 70-75, Dublin, Ireland. Association for Computational Linguisties.
Josh Meyer. Lindy Rauchenstein, Joshua D. Eisen- berg, and Nicholas Howell. 2020. Artie bias cor- pus: An open dataset for detecting demographic bias in speech applications. In Proceedings the Twelfth Language Rexources and Evaluation Confer- ence, pages 6462-6468, Marseille, France. European Language Resources Association.
Maarten Sap, Swabha Swayamdipta, Laura Vianna. Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. Annotators with attitudes: How annotator beliefs and identities bias toxic language detection. In Pro- ceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 5884-5906, Seattle, United States. Association for Computational Linguistics.
Marcyliena H. Morgan. 2001. 7. "nuthin' but a g thang":
Grammar and language ideology in hip hop identity. In Socioculuraal and Historical Contexts of African American English.
Deven Santosh Shah, H. Andrew Schwartz, and Dirk Hovy. 2020. Predictive biases in natural language processing models: A conceptual framework and overview. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 5248-5264 Online. Association for Computa- tional Linguistics.
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of popula- tions. Science, 346664644)444-4533
OpenAT. 2023. Gpt-4 technical report.
Tianshu Shen, Jiaru Li, Mohamed Reda Bouadjenek, Zheda Mai, and Scott Sanner. 2022. Unintended bias in language model-driven conversational recommen- dation.
Desmond U. Patton, William R. Frey, Kyle A. McGre- gor, Fei-Tzin Lee, Kathleen McKeown, and Emanuel Moss. 2020. Contextual analysis of social media: The promisc and challenge of eliciting context in so- cial media posts with natural language processing. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Sociery, AIES '20, page 337-342, New York, NY, USA. Association for Computing Machin- ery.
Emily Sheng, Josh Arnold, Zhou Yu, Kai-Wei Chang. and Nanyun Peng. 2021a. Revealing persona biases in dialogue systems.
Emily Sheng, Kai- Wei Chang, Prem Natarajan, and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages -254,, Online. Association for Computational Linguistics.
Pew Research Center. 2018. Newsroom employecs are less diverse than u.s. workers overall. Technical report, Washington, D.C.
Pew Research Center. 2023. U.s. journalists' beats vary widely by gender and other factors. Technical report, Washington, D.C.
Emily Sheng. Kai- Chang, Prem Natarajan, and Nanyun Peng. 2021b. Societal biases in language generation: Progress and challenges. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing ( (Volume I: Long Papers), pages 4275-4293, Online. Association for Computational Linguisties.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee. Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-5551
Jacquelyn Rahman. 2012. The n word: Its history and use in the african american community. Journal of English Linguisties- J ENGL LINGUIST, 40:137- 171.
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked babysitter: On biases in language generation. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNL) pages 3407- 3412, Hong Kong, China. Association for Computa- tional Linguistics.
Anthony Rios. 2020. Fuzze:
Fuzzy fairness evaluation of offensive language classifiers on african-american english. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01):881-889,
Maggie Ronkin and Helen E. Karn. 1999. Mock ebon- ies: Linguistic racism in parodies of ebonics on the internet. Journal of Sociolinguistics, 3(3):360-380
Hiram Smith. 2019. Has nigga been reappropriated term of endearment? (a qualitative and quantitative analysis). American Speech, 94:420-477

---

Isabel Straw and Chris Callison-Bonl Burch.. 2020. Artificial intelligence in mental health and the biases of lan- guage based models. PLOS ONE, (21220000
from social media, sociolinguistic interviews, focus groups, and hip-hop lyries and weight the probabil- ity of sampling a text using small set of known AAL morphosyntactic features. The datasets that were previously collected, CORAAL Kendall and Farrington 2021 and TwitterAAE (Blodgett et al., 2016), were originally created to study AAL and to study variation in AAL social media respec- tively. For all texts in the dataset, we also collect -aman-annotated counterparts WME to provide a baseline for model evaluations.
Yu Wan. Baosong Yang. Derek F. Wong, Lidia S. Chao, Haihua Du, and Ben C.H. Ao. 2020. Unsupervised neural dialect translation with commonality and diver- sity modeling. Proceedings of the AAA! Conference on Artificial Intelligence, 34(05):1 130-91377
Alex Wang. Amanpreet Singh. Julian Michael, Felix Hill. Omer Levy. and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for nat- ural language understanding, In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353-355, Brussels, Belgium. Association for Com- putational Linguistics.
A.2 Language Variety
All texts included in the dataset are in English (en- US) as spoken or written by African Americans in the United States with majority of texts reflect- ing linguistic features of AAL. Some texts notably contain no features of AAL and reflect WME.
Julie A. Washington, Holly K. Craig, and Amy J. Kush- maul. 1998. Variable use of african american en- glish across two language sampling contexts. Jour- nal of Speech, Language, and Hearing Research, 1155)1115-11124
A.3 Speaker Demographics
Tianyi Zhang*, Varsha Kishore Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
Most speakers included in the dataset are African American. The r/BPT texts were restricted to users who have been verified as African Ameri- can. CORAAL and focus group transcripts were originally interviews with African Americans, and hip-hop lyrics were restricted to African American artists. The TwitterAAE dataset is not guaranteed to be entirely African American speakers, but the texts are primarily aligned with AAL and have a high probability of being produced by AAL speak- ers. Other demographics such as age and gender are unknown.
Xuhui Zhou, Maarten Sap. Swabha Swayamdipta, Yejin Choi, and Noah Smith. 2021. Challenges in auto- mated debiasing for toxic language detection. In Proceedings of the 16th Conference of the European Chupter of the Association for Computational Lin- guistics: Main Volume, pages 3143-3155, Online. Association for Computational Linguistics.
Caleb Ziems, Jiaao Chen, Camille Harris, Jessica An- derson, and Diyi Yang. 2022. VALUE: Understand- ing dialect disparity in NLU. In Proceedings of the 60th Annual Meeting of the Association for Compu- lational Linguistics (Volume 1: Long Pupers), pages 3701-3720, Dublin, Ireland. Association for Compu- tational Linguistics.
A.4 Annotator Demographies
While all AAL texts in the dataset reflect natural us- age of AAL, the WME counterparts in the dataset are annotated. We recruited 4 human annotators to generate WME counterparts for each text. All annotators self-identify as African American, self identify as AAL speakers, and are native English speakers. Additionally, the 4 annotators are under- graduate and graduate students aged 20-28, 2 of whom were graduate students in sociolinguistics. All annotators were compensated at rate between $18 and S27 per hour depending the annotator's 's university and whether they were an undergraduate or graduate student.
A Data Statement
We provide details about our dataset in the follow- ing data statement. Much of the dataset is drawn from existing datasets that lack data statements, and in those cases, we include what information we can.
A.1 Curation Rationale
The dataset was collected in order to study the ro- bustness of LLMs features of AAL. The data is composed of AAL-usage in variety of regions and contexts to capture the variation in the use of and density of features. In order to better ensure the included texts reflect AAL, we sample texts
A.S Speech Situation
Speech situations vary among the 6 datasets we compose. The 1/BPT posts, r/BPT comments,

---

and TwitterAAE subsets are all originally type- written text, intended for a broad audience, and are drawn from asynchronous online interactions. The CORAAL and focus group transcript subsets are originally spoken and later transcribed, intended for others in their respective conversations, and are drawn from synchronous in-person interactions. Finally, the hip-hop lyrics subset are both spoken and written, intended for a broad audience of hip- hop listeners, and are likely repeatedly changed and edited before released. r/BPT comments and posts are sampled from the origin of the subreddit in October 2015, CORAAL transcripts are sampled from interviews between 1888 and 2005, hip-hop lyries are drawn from songs released in 2022, fo- cus groups were conducted between February and November 2022, and the time range of the Twitter- AAE dataset is unknown to the authors.
gone (or "gne", "gon"), and remote past been (or "bin"). Each of these features are detected by the use each term (or their variants) if they are not preceded by another auxiliary verb or preposition in the clause (i.e., "He be eating" contains a use of habitual be, but "Should he be cating?" does not because the auxiliary verb "should" precedes "be" in the clause). While standard POS taggers could potentially underperform on AAL, there were AAL-specific POS taggers available the time of dataset collection to our knowledge.
C Annotation Procedure
C.1 Counterpart Annotations
Annotators were asked to provide a semantically- equivalent rewriting (or counterpart) of a given text from the AAL dataset in WME. The specific set of guidelines provided to annotators were:
A.6 Text Characteristics
1. Change alternative spellings (i.e., "shoulda" for "should've" ve")
Among the data subsets, the focus group transcripts are the most topically focused. All focus groups primarily included discussion surrounding the ex- periences and responses to grief in the Harlem com- munity, focusing on experiences due to daily stres-
Maintain usernames, hashtags, and URLs if present
3. Ignore emojis unless speakers of WME may use them differently
sors, the death of loved ones, police shootings, and the COVID-19 19 pandemic. In the r/BPT posts and r/BPT comments subsets, texts were typically writ- ten in response to tweet by an African American Twitter user, ranging from political commentary to discussion of the experience of African Americans the United States. The hip-hop lyries subset not topically focused, but includes texts that follow specific rhyming patterns and meters. The remain- ing subsets of the data (TwitterAAE, CORAAL) span variety of topics and structures.
4. Conserve and un-censor profanity
5. Avoid unnecessary changes
6. Use your best judgement in special cases
As noted, annotators had the option label text as "Not Interpretable" it lacks a reasonable counterpart in WME.
C.2 Counterpart Judgment Instructions
In judging counterparts, annotators were provided with original text from the dataset in either WME or AAL and model-generated (or human- annotated) counterpart. Notably, judgments were assigned ensuring that no annotator received text they were involved in creating the counterpart for. Additionally, annotators were not given definitions or guidelines for terms such as "meaning" or "tone" to avoid biasing judgements and to encourage an- notators to use their own interpretation of the terms. The questions asked of annotators are as follows:
B
AAL Search Patterns
To better ensure our dataset includes use of AAL features, we use a set of regex and grammar-based search patterns as part of the sampling procedure. Regex patterns for AAL features are listed below.
AAI. Feature
Partern sinn Tt
oin't
Existential
it (?owaslis) alow n't (ook "i''t (*mobodylanyyody)) (Whtheyl', bweel bhe) (w"Ting
Nogative Concord
Dropped Copula
Determiner Lveling
a [acion]] we
1. Human-likeness: Is the interpretation more likely generated by a human language model?
Table 5: ALL feature search patterns.
The set also includes grammar-based patterns using the spacy POS tagger to detect the use of ha- bitual be, completive done (or "dun", "dne"), future
(1) Very Likely a Model, (2) Likely a Model,
(3) Neutral, (4) Likely a Human, (5) Very
Likely a Human

---

2. Linguistic Match: How well does the interpre-
E.2 Generation Hyperparameters
tation resemble AAL?
For all GPT-family models, we use the default tem- perature of 0.7 generations. For the BART and T5 model variants, we use a beam width of 3, tem- perature of 1, and no roprreear pop_reeeat_ggaam ngram_size of 3.
(I) Completely Unlike AAL, (2) Unlike AAL,
(3) Neutral, (4) Like AAL, (5) Completely Like AAL
3. Meaning Preservation: How well does the in- terpretation reflect the meaning of the original text?
F Full Counterpart Results
(1) Completely innacurate, (2) Inaccurate, (3)
Table and Table 7 show the raw automatic metric and human judgement scores for the counterpart generation task respectively. Table 8 shows the per- centage of toxic terms removed by the model when generating counterparts in AAL WME. Finally, Table 9 and Table 10 present the automatic metric and human judgments scores on the toxic (at least one term categorized as offensive) and non- toxic (no terms categorized as offensive) subsets of the corpus.
Neutral, (4) Accurate, (5) Completely Accu-
rate
4. Tone Preservation: How well does the coun- terpart accurately reflect the tone the origi- nal text?
(1) Completely innacurate, (2) Inaccurate, (3) Neutral, (4) Accurate, (5) Completely Accu-
rate
For judgment samples that involved judging WME counterparts, questions and response options that refer to "AAL" were changed accordingly.
G Full MSP Results
Table 11 presents the raw perplexity and entropy scores in the Masked Span Prediction task.
D Annotator Agreement Calculation
Because the task provided to annotators required generating lext, we use Levenshtein distance and Krippendorf's s Alpha to calculate annotator agree- ment based the general form as described in Braylan al. 2022. Annotator agreement is caleu- lated with the following formula:
H Additional Counterpart Generation Examples
The remaining appendices, Tables 12-19, provide further examples from the counterpart generation task. Examples are drawn randomly from subsets where the total score given to one of the models evaluated exceeds the ratings the original an- notated counterpart and where the total score of a model is lower.
D. Do
1
(1)
where D. represents the observed distance between annotations, D, represents the expected distance, and Levenshtein distance is used as the distance function (a, b). Expected distance between two annotators is calculated by randomly shuffling one set of annotations and calculating the average Lev- enshtein distance between the randomized pairs.
Model and Experiment Details
E.1 Checkpoints and Training
For GPT-3, ChatGPT, and GPT-4, we use the text- davinci-003, gpt-3.5-turbo, and gpt-4 checkpoints respectively.. For the T5 model variants, 15-large and google/flnn-i-tarre checkpoint. Flan- T5 is fine-tuned using a learning rate of 3c-5 for 5 epochs across the full set of 72 additional texts. Finally, for BART we use the facebookkbart-large checkpoint.
the

---

WATE AAT.
AXT.
WAR
Model
MA1
855
Repoat
77.4
53.1
Han 13 (FT)
77.2
03.0
76.1
53.3
0.21
-002
30.3
07.8
79.0
56.1
0.22
Fbe-TT
74.8
57.7
74.3
92.4
0.30
0.01
75.8
58.6
75.3
539
0.23
0.02
GPT3
26.5
32.9
55.2
11.5
0.02
0.80
08.5
51.0
082
53.5
0.13
0.92
ChatGPT
56.2
32.7
54.0
51.2
0.05
0.47
61.6
40.2
59.3
53.8
0.15
0.90
GPE4
Table 6: Full coverage and toxicity metrics of model counterpart generations for AAL and aligned WME. Toxicity scores are reported both as raw scores (T) and the percent change in scores (%AT) between model inputs to corresponding goneel-geeerate counterparts. Repeal row reports metrics computed on and between gold standard reference texts.
AAL + WME
Human-like
Dialect
Meaning
Tone
Model
AAL
WME
AAL
WME
AAL
WME
AAL
WME
Human
3,66
3.82
4.00
4.99
4.51
4.53
4.55
4.67
Flan-T5 (FT)
4.01
3.56
3.80
3.61
4.79
4.45
4.83
4.61
GPT-3
2.65
3.50
3.56
4.62
4.18
3.72
4.07
4.04
ChatGPT
3.18
3.62
4.09
4.67
3.99
3.80
4.11
4.19
GPT-4
3.17
3.51
4.05
4.74
4.67
3.96
4.65
4.21
Table 7: Full ratings counterpart responses for Flan-TS, GPT-3, ChatGPT, GPT-4, and the original human response in AAL and WME. Bolded values indicate scores exceeding judgments of human counterparts. The higher score between AAL and WME is colored green.
% Toxicity Removed
Model
WME AAL
AAL + WME
Flan-T5 (FT)
-0.02
0.37
Flan-T5
0.01
0.02
GPT-3
0.65
0.93
ChatGPT
0.62
0.88
GPT-4
0.21
0.84
Table 8: Percentage of toxicity removed from model inputs when generating counterparts in the WME AAL and AAL - WME directions. Negative values indicate more toxic terms were introduced in model-generated counterparts than were present in the input.
Toxic
Non-Toxic
Model
AAL
WME
Difference
AAL
WME
Difference
Flan-T5 (FT)
74.9
74.7
0.2
78.9
82.3
-3.4
Flan-T5
70.3
69.0
-0.01
75.6
77.5
0.02
GPT-3
54.0
62.5
-8.5
56.5
69.4
-12.9
ChaiGT
56.7
57.0
-0.3
57.5
62.9
-5.4
GPT-4
68.2
60.4
7.8
65.9
67.9
-2.0
Table Rouge-1 gaps Toxic and Non-Toxic subsets of the AAL dataset. Negative gaps indicate greater WME performance than AAL performance.
Toxic Dialect
Non-Toxic Dialect
Model
Human-like
Meaning
Tone
Human-like
Meaning
Tone
Human
-0.70
0.40
-0.08
-0.12
-0.06
00.10
-0.02
-0.14
Flan-T5 (FT)
0.45
0.49
0.83
0.57
0.44
0.11
0.21
0.14
GPT-3
-0.82
-1.04
0.61
0.19
-0.88
-1.07
0.40
-0.04
ChatGPT
-0.51
-0.60
0.11
-0.01
-0).42
-0.57
0.20
-0.10
GPT-4
0.33
-0.31
1.29
0.31
-0.50
-0.7
0.56
0.47
Table Gaps human judgments of counterparts in the toxic subset and non-toxic subset of the data. Negative scores indicate higher WME performance, and bolded scores indicate smaller gaps (less positive negative) compared to the other subset.

---

AAL
Perplexity Noun
Top-5 Entropy
Model
Random
Verb
Random
Noun
Verb
GPT-3 T5
21.0
12.2
12.24
15.86
13.80
1737.79 2559.9
3657 64
2420.74
1.53
1.51
1,50
BART
5856.9
3900.5
5.56
5.99
WME
Perplexity Noun
Top-S Entropy
Model
Random
Verb
Random
Noun
Verb
GPT-3
8.79
17.6
8,76
13.40
15.32
13.19
T5
1691.58
3391.09
2144.63
1.26
1.33
1.31
BART
2342.71
5437.39
3015.38
5,56
5.94
5.52
Table 11: Span perplexity and top-5 entropy AAL and aligned WME texts. Results BART are averaged across 5 trials with different random spans masked.
Source
Text
Human-like
Dialect
Meaning
Tone
Somebody is going to get robbed for that new money.
Human WME
Just watch because this niggas are broke and desperate
Counterpart
Somebody going get bust for them new breads watch
AAL Text
2.0
1.0
1.0
cause niggas be broke press.
Flan-T5 (FT)
Somebody gonna get robbed for that money. Just
5.0
4.0
5.0
5.0
Generated AAL
watch because these niggas broke and desperate.
GPT-3
Somebody gon' get robbed
new money. Just
3.0
3.0
4.5
4.5
Generated AAL
watch cause these niggas po" 'esperate.
Somebody finna get jacked for that new cash. Inuma tell ya, cause these niggas out here strugglin' and needin' bad.
ChatGPT
4.5
4.5
4.5
5.0
Generated AAL
GPT-4
Somebody gon' get got for that new money. Just watch
5.0
Generated AAI.
cuz these niggas broke and desperate.
Table
Source
Text
Hluman-like
Dialect
Mcaning
Tone
Human WME
And what does? People going be powerless stop
Counterpart
And what he does? Niggas gon be powerless to stop that shit teo.
AAL Text
1.0
3.0
2.0
2.0
Flan-T5 (FT)
And what if he does? People gonna be powerless to stop
4.5
5.0
5.0
5.0
Generated AAL
it.
GPT-3
An' if he do? Folks gon' be powerless stop
3.0
5.0
4.5
Generated AAI.
ChatGPT
And what it he do though? Ain nobody gonna have no
5.0
Generated AAI.
power stop it.
GPT-4
An' what he do? Folks gon' be powerless to stop it.
3.5
5.0
5.0
5.0
Generated AAL
Table 13

---

Text
Human like
Dialect
Meaning
Number one top is really number one thing that
Human WME
causes everything know all the violence, would
Counterpart
just go back and forth and social media.
Number one top really a number one thing that
AAL Text
causes everything know all the violence, he would
5.0
5.0
5.0
5.0
just go back and forth and social media.
Number one really number one thing that causes everything know all the violence, he would just go
Flan-T5 (FT)
5.0
5.0
5.0
5.0
Generated AAL
back and forth and social media.
be causin' all da violence know. He just keep goin' back and forth da social media, know mssaii?
Num top is suttin' that's really low-key
GPT-3
1.0
1.0
4.0
4.0
Generated AAL
numba one top thang causin' errythang, ya feel Dude jus'
ChatGPT
me? know da violence be 'cause on social media or whatever.
1.0
Generated AAL
goin' back
Numba one top straight up da numha one thing
GPT-4
causin' all dis violence know, he just be goin' back
4.0
3.0
5.0
5.0
Generated AAL
and forth on social media n' stuff.
Table 14
Source
Text
Human-like
Dialect
Mcaning
Tone
Human WME
My man, i díss in my freestyle rhyme
Counterpart
AAI. Text
my man, be dissin" in my freestyle rhyme.
4.5
4.5
4.5
Flan-Trs I'5 (FT)
my man, diss in my freestyle rhyme
5.0
5.0
5.0
5.0
Generated AAL
GPT-3
My dude, was killin' my freestyle rhyme!
2.0
Generated AAL
ChatGPT
My homie, dissed in my freestyle rap.
2.5
3.0
4.0
4.5
Generated AAL.
GPT-4
My dude, be dissin' in my freestyle flow
2.0
4.0
4.0
5.0
Generated AAL
Table 15
Source
Text
Human- like
Dialect
Mcaning
Tone
AAL Text
now they sayin' juice left some heads cracked.
Human WME
Now they're saying that juice left some heads cracked.
1.5
2.0
1.5
2.5
Counterpart
Flan-T5 (FT)
Generated
Now they say juice left some heads cracked.
5.0
WME
GPT-3
Now they are saying that Juice left some people with serious injuries.
Generated
5.0
4.0
5.0
WME
ChatGPT
Now they saying that Juice caused some people have head injuries.
Generated
2.0
4,0
5.0
5.0
WMR
GPT44
Generated
Now they saying that Juice injured some people.
5.0
5.0
4,0
4.0
WME
Table

---

Source
Text
Human-like
Dialect
Meaning
Moms always buyin groceries like they preparing for a nuclear winter.
AAL Text
Human WME
Mam is always buying groceries like they are preparing for a nuclear winter.
2.0
2.0
2.0
5.0
Counterpart
Flan- (FT)
Moms always buy groceries they are preparing nuclear winter.
Generated
3.5
4.5
5.0
5.0
WME
GPT-3
Mom always buys groceries like she's preparing for nuclear winter.
Generated
5.0
5.0
4.0
4.0
WME
ChatGPT
My mother always buys groceries if she is preparing for nuclear winter.
Generated
2.0
WME
GPT-4
Mom is always buying groceries as if she is preparing for nuclear winter.
Generated
2.0
WMF
Table 17
Source
Text
Human-lia like
Dialect
Meaning
Tone
AAL Text
used broken controllers. There used to be broken controllers.
Human WME
5.0
5,0
4.5
4.5
Counterpart
Flan-T5 (FT)
Generated
used to be broken controllers.
3.0
1.5
1.5
3.5
WME
GPT-3
Generated
It used to be malfunctioning controllers.
1.5
5.0
4.5
5.0
WME
ChatGPT
Generated
It used be controllers that were broken.
1.5
5.0
2.5
3.5
WME
GPT-4
Generated
It used that controllers would break.
2.5
5.0
1.5
WMF
Table 18
Source
Text
Human-like
Dialect
Mcaning
Tone
AAL Text
that shit be crazy, they "posed to protect
Human WME
That shit is often erazy, they supposed to protect us.
3.0
4.0
4.5
4.5
Counterpart
Flan-T5 (FT)
Generated
That's crazy, they're trying to protect
WME
GPT-3
That stuff is really outrageous, they' 're supposed protect us.
Generated
5.0
3.0
5.0
WME
ChatGPT
Generated
That is insane, they are supposed to protect us.
5.0
4.5
4.5
4.5
WMF
GPT-4
Generated
That situation is crazy, they supposed to protect us.
4.5
4.5
4.5
4.5
WME
Table


Reviews:
[{"review_id": "tEQs2ZwJFl", "reviewer": "Reviewer_raHm", "summary": "", "questions": "- Please report the generation hyperparameters used\n- Please report the results for FlanT5 without finetuning\n- Why did you use debertalargemnli for BERTScore and not another model?\n- How do you reconcile the fact that in the span masking, different spans are being masked so the results are not directly comparable?", "limitations": "", "rating": 5, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review.\n\n__Q1) Please report the generation hyperparameters used__\n\nWe will include further details concerning generation hyperparameters use in the appendix and update the corresponding reference in the Methods section. For GPT models, the generation hyperparameters are a temperature of 0.7, and all remaining hyperparameters were left as default (top p of 1, 0 frequency or presence penalty). For all other models (BART, T5, Flan-T5), the generation hyperparameters are 3 beams, a no_repeat_ngram_size of 3, and all other parameters left as default (temperature of 1, no sampling)\n\n\n__Q2) Please report the results for FlanT5 without fine-tuning__\n\nWe will update Figures 2, 4, and 5 as well as Tables 6, 8, and 9 in the Appendix to include the results for FlanT5 without fine-tuning. Results for fine-tuned Flan-T5 will be presented as “Flan-T5 (FT)” and without fine-tuning, “Flan-T5.” Across all automatic metrics and in both counterpart generation directions, Flan-T5 without fine-tuning performed slightly worse than with fine-tuning while still presenting performance gaps preferring WME over AAL. Additionally, similar trends were observed concerning toxicity, where Flan-T5 without fine-tuning presented a stronger preference for WME in the non-toxic subset of the data compared to the toxic subset.\n\n\n__Q3) Why did you use debertalargemnli for BERTScore and not another model?__\n\nWe use the ‘deberta-large-mnli' checkpoint for BERTScore over other model checkpoints because it has been found to be better correlated with human scores by the original BERTScore authors (Zhang et al., 2019). The associated GitHub repository lists all models’ correlation with human scores. We will update the Metrics section to explicitly state this reasoning.\n\nTianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert.\n\n\n__Q4) How do you reconcile the fact that in the span masking, different spans are being masked so the results are not directly comparable?__\n\nThis is an excellent point; we acknowledge that it is difficult to directly compare the masked span prediction results between dialects considering that different spans are masked. Due to the differences in word choice, word order, and other features of AAL, masking identical spans in both texts would be impossible, particularly while attempting to determine if the features of AAL affect model performance. To mitigate this issue, we repeat the masked span prediction experiment 5 times with different randomly masked spans (Lines 330-332), avoid directly comparing metrics at the sample level, and present results for different classes of spans (noun and verb phrases) to allow for more fine-grained and fair comparisons (Lines 228-230). We will add language highlighting the purpose of the repeated experiments in the Methods section.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Tackle an important problem of LLM understanding AAL, which is critical for AI inclusion.\n- Release a new AAL dataset that should aid research\n- Well written with comprehensive methodology that avoids several possible bias\n- Important results that show that LLMs struggle with AAL, although finetuning them helps.", "reasons_to_reject": "- Did not report generation hyperparameters for the LLMs\n- Missing important benchmark of FlanT5 without finetuning", "questions_to_authors": "- Please report the generation hyperparameters used\n- Please report the results for FlanT5 without finetuning\n- Why did you use debertalargemnli for BERTScore and not another model?\n- How do you reconcile the fact that in the span masking, different spans are being masked so the results are not directly comparable?", "paper_topic_and_main_contributions": "This paper investigates how well current popular LLMs understand African American language (AAL). They evaluate 6 LLMs on two language generation tasks, they also curate a new AAL dataset and perform extensive experiments showcasing the shortcomings of current LLMs on AAL.", "excitement": 5, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "T5a6WfFUPF", "reviewer": "Reviewer_gaMz", "summary": "", "questions": "In Figure 2, all the LLMs have comparable performance to Human in BERTScore. Does it mean LLMs are not worse than Human?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 1, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review.\n\n__Reason to Reject 1) The setting of the experiment is not valid. In Section 4.1, the authors are comparing the differences of the performance for WME-to-AAL and AAL-to-WME. This experiment is not valid since we cannot compare results obtained from different test data (e.g., comparing translation results of English-to-German and German-to-English).__\n\nWithin translation literature, we found that it is common practice to make claims based on a comparison of model performance across test sets in different languages (e.g. Belinkov et al., 2017; Macketanz et al., 2021; Koehn, 2005). We acknowledge that there is also work arguing against such comparison, namely, a study by Bugliarello et al. (2020) which cites two reasons that n-gram overlap metrics cannot be compared cross-lingually: (1) different languages may use drastically different numbers of words to convey the same meaning, and (2) different languages utilize different tokenization schemes.\n\nWe note first that while our task is framed as a translation task, the dialects we study are not distinct languages, which is why we use the name “counterpart generation” instead. Given the two criteria above, our task also avoids both of the weaknesses identified by Bugliarello et al. (2020). First, the AAL and WME texts in our dataset have a total of 5632 and 5665 words respectively, and aligned text samples differ by 0.095 words on average, making the first weakness of cross-lingual comparison unlikely to apply. Additionally, Bugliarello et al. (2020) finds that among linguistic properties, inherent difficulty of translating from a source language is most and negatively correlated with Type-Token Ratio (TTR). We have calculated the TTR for both the AAL (0.274) and WME (0.260) texts and find that there are no significant differences (p > 0.05) between the dialects in our dataset. Finally, because the comparisons in our experiments use the same models, they also use the same tokenization schemes, so the second weakness also does not apply. Therefore, we believe that the setting of the counterpart generation experiment and our comparisons using n-gram overlap metrics are valid.\n\nFurthermore, the counterpart generation task only represents a portion of our experiments and analyses, and thus does not impact the contributions of the entire paper. The conclusions resulting from this task are supported by the results of the masked span prediction task as well as qualitative analysis of models’ outputs. We will add a new section in the Appendix to include the above new analysis and discussion.\n\nYonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 861–872, Vancouver, Canada. Association for Computational Linguistics.\n\nEmanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos, Ryan Cotterell, and Naoaki Okazaki. 2020. It’s easier to translate out of English than into it: Measuring neural translation difficulty by cross-mutual information. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1640–1649, Online. Association for Computational Linguistics.\n\nPhilipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79–86, Phuket, Thailand.\n\nVivien Macketanz, Eleftherios Avramidis, Shushen Manakhimova, and Sebastian Möller. 2021. Linguistic evaluation for the 2021 state-of-the-art machine translation systems for German to English and English to German. In Proceedings of the Sixth Conference on Machine Translation, pages 1059–1073, Online. Association for Computational Linguistics  \n  \n\n__Reason to Reject 2.A) There are various dialects and minority languages. This paper considers African American Language, but it is not clear why the specific language is handled here. The paper may be more appropriate for a focused workshop.__\n\nWhile there are certainly other minority dialects that may experience similar biases from LLMs, we choose African American Language due to the historical treatment of African Americans in the United States and more specifically, because the dialect has in part reinforced the historical oppression of African Americans through its historical view as a “defective” form of English. With 13.6% of the US population (U.S. Census Bureau, 2020) self-identifying as African American, development of LLMs without consideration of their dialect would exclude a large community of potential users. There is a large body of prior work focusing on AAL in the NLP community and published as main conference papers, many of which we cite in the Introduction and Related Work sections (e.g. Lines 28-30; 539; 540-541; 554-556; 560-561; 566). \n\nU.S. Census Bureau. 2020. 2020 census. U.S. Department of Commerce.\n\n\n__Reason to Reject 2.B) In this paper, \"bias\" is defined as \"gaps in performance between one language and the other language\" (L108). It is not clear why such differences are considered as bias, and it can be misleading. This paper considers two simple tasks, but it is unclear if any social impacts can be estimated from the results.__\n\nWe note in the Introduction section that performance gaps can reflect representational harms (Line 109-113) and can potentially exacerbate societal inequalities, such as in mental health counseling or medical healthcare (Line 36-57). Additionally, Hovy and Prabhumoye (2021) explicitly mention “unequal performance for different user groups” as a reflection of “demographic bias”, and Shah et al. (2020) defines predictive bias as when “the label distribution of a predictive model reflects a human attribute in a way that diverges from a theoretically defined ‘ideal distribution,’” both of which encompass our analysis of performance gaps. We acknowledge in our limitations section that further work is necessary to evaluate LLMs in realistic use-cases (Lines 608-619), but because little prior work has investigated AAL and LLMs, we chose to perform an intrinsic evaluation as a foundation for future work on the topic. \n\nDirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing. Language and Linguistics Compass, 15(8):e12432.\n\nDeven Santosh Shah, H. Andrew Schwartz, and Dirk Hovy. 2020. Predictive biases in natural language processing models: A conceptual framework and overview. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5248–5264, Online. Association for Computational Linguistics. \n\n\n__Q1) In Figure 2, all the LLMs have comparable performance to Human in BERTScore. Does it mean LLMs are not worse than Human?__\n\nThe setting of the counterpart generation task required that the annotators and evaluated models produce semantically equivalent counterparts in the alternate language variety. Additionally, AAL and WME are largely similar language varieties beyond the set of unique features of AAL. So, the lack of differences in semantic similarity measures like BERTScore are expected due to these inherent similarities. While differences between LLMs and Human are small because of these similarities, the other metrics and tasks included do present performance gaps, so we cannot conclude from these results that LLM and Human outputs perform equally well.\n\n__Typos, Grammar, Style And Presentation Improvement 1) In the paper, expressions like \"model's interpretation\" and \"model's understanding\" are often used (e.g., L232). They are somewhat confusing. The statistical models just solve optimization problems for given tasks.__\n\nWe note that terms such as \"natural language understanding\" and \"interpretation\" have been used in the computational linguistics fields for decades in many different contexts (e.g., \"semantic interpretation\" is a common phrase). We can switch references from \"understanding\" to \"interpretation\", but we feel that the solution of an optimization problem for a given input in this case is indeed an interpretation of the input and thus, \"interpretation\" is an appropriate term.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A dataset for African American Language was made, though the size is small (346 samples).", "reasons_to_reject": "- The setting of the experiment is not valid.\n\nIn Section 4.1, the authors are comparing the differences of the performance for WME-to-AAL and AAL-to-WME. This experiment is not valid since we cannot compare results obtained from different test data (e.g., comparing translation results of English-to-German and German-to-English).\n\n- The motivation and importance of the study is not clear.\n\nThere are various dialects and minority languages. This paper considers African American Language, but it is not clear why the specific language is handled here. The paper may be more appropriate for a focused workshop.\nIn this paper, \"bias\" is defined as \"gaps in performance between one language and the other language\" (L108). It is not clear why such differences are considered as bias, and it can be misleading.\nThis paper considers two simple tasks, but it is unclear if any social impacts can be estimated from the results.", "questions_to_authors": "In Figure 2, all the LLMs have comparable performance to Human in BERTScore. Does it mean LLMs are not worse than Human?", "paper_topic_and_main_contributions": "This paper analyzes the performance of LLMs for African American Language. A dataset was created and experiments were run for two tasks (the counterpart generation task and the masked span prediction task).", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "In the paper, expressions like \"model's interpretation\" and \"model's understanding\" are often used (e.g., L232). They are somewhat confusing. The statistical models just solve optimization problems for given tasks."}}, {"review_id": "cKs8BQjC5g", "reviewer": "Reviewer_jtk5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review.\n\n__Reason to Reject 1) The paper would benefit from information about the demographics of the annotators and human evaluators. The authors demonstrate that LLMs perform worse on AAE-related tasks, but if these models are trained on predominantly SAE, individuals more familiar with SAE or AAE might misinterpret the samples.__\n\nAll annotators are African American undergraduate and graduate students, native AAL-speakers, and fluent WME-speakers. Given that certain cases of AAL can be extremely difficult for White Americans to understand, we elect to only choose annotators that are native AAL-speakers given they are able to better understand both dialects. We will update the discussion of both annotations and human judgments to explicitly note the race of annotators.\n\n__Typos, Grammar, Style And Presentation Improvement 1) For clarity, the authors could maintain the terminology used in previous literature. For instance, the abbreviation AAE could be used instead of AAL. However, the authors could still make it clear that they disagree with the current terminology.__\n\nThe terms we use, “African American Language” and “White Mainstream English”, are both found in recent sociolinguistics literature (Alim & Smitherman, 2012; Baker-Bell, 2020). The authors that introduced these terms did so deliberately to “foreground the relationship between language, race, anti-Black racism, and white linguistic supremacy” (Baker-Bell, 2020), and so we adopt this terminology to recognize the social context of the work. We will update the Introduction to make clear that both terms are derived from prior sociolinguistics literature.\n\nH Samy Alim and Geneva Smitherman. 2012. Articulate while Black: Barack Obama, language, and race in the US. Oxford University Press.\n\nApril Baker-Bell. 2020. Linguistic justice: Black language, literacy, identity, and pedagogy. Routledge.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper provides a comprehensive and rigorous evaluation of LLMs' understanding of AAE through well-designed tasks. The experimental design tests multiple LLMs, including GPT-3, GPT-3.5 (ChatGPT), GPT-4, T5, Flan-T5, and BART, encompassing both decoder-only and encoder-decoder transformer-based LLMs. \n\nThe authors also experiment with different tuning paradigms: few-shot and fine-tuning. This approach provides a broad perspective on the performance of LLMs under different conditions. \n\nThe methodology, experimental design, and results are clear, reasonable, and easy to follow. The authors' findings are well-supported by the data and contribute to our understanding of LLMs' performance on different English variants. \n\nThe creation of an AAE text sample dataset is a significant contribution that can facilitate further research in this area. \n\nDespite the predictable conclusion that LLMs perform better on English variants better represented in their training data, this paper provides empirical data supporting this for AAE, which is a valuable contribution to the field.", "reasons_to_reject": "The paper would benefit from information about the demographics of the annotators and human evaluators. The authors demonstrate that LLMs perform worse on AAE-related tasks, but if these models are trained on predominantly SAE, individuals more familiar with SAE or AAE might misinterpret the samples.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper conducts an in-depth investigation into the understanding of African-American English (AAE) by large language models (LLMs) in comparison to Standard American English (SAE). The authors assess a variety of LLMs, both fine-tuned and few-shot, on two tasks: (1) conversion between American English variants (e.g., AAE to SAE or vice versa), and (2) masked span prediction for AAE. \n\nIn the first task, the authors highlight the role of perceived toxicity in LLMs' understanding of AAE. They find that LLMs are more likely to infer toxicity in AAE text, possibly due to the unfamiliar context within SAE. This finding is significant as it sheds light on potential biases in LLMs. \n\nIn the second task, the authors demonstrate that LLMs have a poorer understanding of AAE compared to SAE. This is a crucial finding, as it underscores the need for more inclusive training data for LLMs. \n\nAdditionally, the authors contribute a dataset of AAE text samples, which is a valuable resource for future research in this area.", "excitement": 5, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "For clarity, the authors could maintain the terminology used in previous literature. For instance, the abbreviation AAE could be used instead of AAL. However, the authors could still make it clear that they disagree with the current terminology."}}]

Author rebuttal:

Meta-review: The paper does a thorough investigation of the ability of LLMs to understand AAE as compared to SAE. This is a very timely piece of work, as models get more intertwined with lives globally. Discrepancies in quality of service must be investigated, and this paper contributes to this soundly. In addition, they develop a resource for this evaluation. It would be useful to release more details of this such as annotator demographics, compensation etc in a datasheet.
I would also urge the authors to reflect a little however on the definition and usage of the word ‘bias’ in the paper and try to ground it in experienced harms. As noted by a lot of recent literature, the interchangeable or disconnected usage of these words can dilute the impact and utility of a work, and given the thoroughness in this paper, just some additional reflection of the concepts used and framed would amplify its message.
Decision: Accept-Main