Sociolectal Analysis of Pretrained Language Models (2021)

Abstract: Using data from English cloze tests, in which subjects also self-reported their gender, age, education, and race, we examine performance differences of pretrained language models across demographic groups, defined by these (protected) attributes. We demonstrate wide performance gaps across demographic groups and show that pretrained language models systematically disfavor young non-white male speakers; i.e., not only do pretrained language models learn social biases (stereotypical associations) – pretrained language models also learn sociolectal biases, learning to speak more like some than like others. We show, however, that, with the exception of BERT models, larger pretrained language models reduce some the performance gaps between majority and minority groups.

## introduction while speakers of english generally understand each other, our linguistic preferences may differ, depending on the linguistic varieties we were exposed to, and how susceptible we were to them at the time @xcite . linguistic varieties form a multi-dimensional continuum of dialects and sociolects @xcite . 1 such linguistic variation presents a real challenge: the linguistic preferences of english pretrained language models may align better with the linguistic preferences of some groups in society than with those of others. group disparities constitute a fairness problem @xcite : if our technologies provide end users with new opportunities, group disparities mean unequal opportunities across groups. moreover, if the groups are defined in terms of protected attributes, our technologies may discriminate between end users in ways that violate regulations. 1 dialects are mainly defined in terms of geography; sociolects in terms of demographics @xcite . after waiting three hours, cal whined and started to ___. we evaluate the sociolectal biases of a range of pre-trained english language models. unlike previous work on biases in pre-trained language models, we do not consider representational biases @xcite , but performance disparities; moreover, we do not consider downstream performance differences after fine-tuning for downstream tasks such as coreference resolution @xcite or machine translation @xcite , but performance differences across demographics of the language models themselves on cloze (fill-inthe-gap) problems. since the cloze task is how these pre-trained language models are trained, we can evaluate models directly without introducing biases from probes or downstream tasks. note that some sociolinguistic variables are salient (e.g., @xmath0, others are not @xcite . one strategy to evaluate the robustness of pre-trained language models across groups would be to identify salient variables and evaluate language models in the context of those @xcite . while such evaluations are easier to interpret than evaluations of performance parity, they typically only cover a small set of variables or lectal features and therefore run the risk of only scratching the surface of sociolectal variation. in contrast, we will not focus on salient lectal features, but on error rates across demographics (precision at k and mean reciprocal rank). ## experimental setup dataset we use a publicly available cloze-style word prediction dataset foot_0 for our experiments. the dataset consists of fill-in-the-gap (cloze-style) example sentences with (always) the last word is removed. the statistics of the annotators is shown in pre-processing each annotator was provided with about 1049 sentences on average. some examples were left unanswered. the data collectors pre-trained language models the probed models foot_1 are listed here: to evaluate the extent to which pretrained language models are aligned with annotator preferences. given a incomplete sentence v 1 . . . v n __, and @xmath1, w 2 , • • • , w r ] the r-most frequent continuations of v 1 . . . v n (within a group of human annotators). our probed language model ranks candidate words by their model likelihood @xmath2, c 2 , • • • , c p ] . the p@1 of the language model is then defined as: ## q1: outlier demographics? before comparing pretrained language models with human annotations, we first consider how continuations differ across demographic groups. we compare demographic groups by computing the average p@1 scores for individuals in each group relative to the overall majority vote (across all groups). note that we can also compute the variance within groups by computing the p@1 scores for each annotator. ## q2: unfair language models? we evaluate the pretrained language models on the cloze examples and obtain the logits vectors of the last layers corresponding to the masked tokens (gaps), perform softmax normalization to obtain the top-10 most likely candidate words and compute the fairness of the pretrained language models based on these predictions. our fairness metric is an instance of multi-group -fairness @xcite , sometimes referred to as min-max rawlsian fairness @xcite , and says a model is -fair if the risk across any two groups is approximately the same. to this end, we compare the pretrained language models' range (range) of p@1 and mrr scores across groups. for each pretrained language model, we compute the maximum performance difference across any two groups ( or range). if a language model m is -fair, for some value of , and no other language models are -fair, m is the most fair language model in our batch. foot_2 ## q3: demographics of models? finally, we explore how specific pretrained language models align with group-level preferences. in other words, are different pretrained language models aligned with different demographics, or are they all biased in similar ways? ## conclusion we compared pretrained language models to human cloze tests, and showed that models align much better with some groups of participants than others. for electra and gpt-2, larger models are more fair; for bert, albert, and roberta, the opposite is true. electra models are least fair. generally, models disfavor young non-white men the most. previous work has explored social biases in how pretrained language models represent concepts @xcite , but this is, to the best of our knowledge, the first work on whose language pretrained language models reflect, i.e., what sociolects models align best with. work on personalized language modeling @xcite ) is loosely related, e.g., @xcite present interesting to work to make word prediction sociolect-aware, showing significant keystroke savings by conditioning on sociolect.