Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias (2022)

Abstract: The size of pretrained models is increasing, and so is their performance on a variety of NLP tasks. However, as their memorization capacity grows, they might pick up more social biases. In this work, we examine the connection between model size and its gender bias (specifically, occupational gender bias). We measure bias in three masked language model families (RoBERTa, DeBERTa, and T5) in two setups: directly using prompt based method, and using a downstream task (Winogender). We find on the one hand that larger models receive higher bias scores on the former task, but when evaluated on the latter, they make fewer gender errors. To examine these potentially conflicting results, we carefully investigate the behavior of the different models on Winogender. We find that while larger models outperform smaller ones, the probability that their mistakes are caused by gender bias is higher. Moreover, we find that the proportion of stereotypical errors compared to anti-stereotypical ones grows with the model size. Our findings highlight the potential risks that can arise from increasing model size.

## introduction the growing size of pretrained language models has led to large improvements on a variety of nlp tasks @xcite @xcite . however, the success of these models comes with a price-they are trained on vast amounts of mostly web-based data, which often contains social stereotypes and biases that the models might pick up @xcite @xcite . combined with recent evidence that the memorization capacity of training data grows with model size @xcite , the risk of figure 1 : we study the effect of model size on occupational gender bias in two setups: using prompt based method (a), and using winogender as a downstream task (b). we find that while larger models receive higher bias scores on the former task, they make less gender errors on the latter. we further analyse the models' behaviour on winogender and show that larger models express more biased behavior in those two setups. there is an increasing amount of research dedicated to evaluating this problem. for example, several works studied the bias in models using downstream tasks such as coreference resolution @xcite , natural language inference (nli) @xcite and machine translation @xcite . other works measured bias in language models directly using masked language modeling (mlm) @xcite @xcite . in this paper, we examine how model size af-fects gender bias (fig. 1 ). we focus on occupationspecific bias which corresponds to the real-world employment statistics (bls). 2 we measure the bias in three model families (roberta; @xcite @xcite in two different ways: using mlm prompts and using the winogender benchmark @xcite . we start by observing a potentially conflicting trend: although larger models exhibit more gender bias than smaller models in mlm, foot_2 their winogender parity score, which measures gender consistency, is higher, indicating a lower level of gender errors. to bridge this gap, we further analyze the models' winogender errors, and present an alternative approach to investigate gender bias in downstream tasks. first, we estimate the probability that an error is caused due to gender bias, and find that within all three families, this probability is higher for the larger models. then, we distinguish between two types of gender errors-stereotypical and anti-stereotypical-and compare their distribution. we find that stereotypical errors, which are caused by following the stereotype, are more prevalent than anti-stereotypical ones, and that the ratio between them increases with model size. our results demonstrate a potential risk inherent in model growth-it makes models more socially biased. the connection between model size and gender bias is not fully understood; are larger models more sensitive to gender bias, potentially due to their higher capacity that allows them to capture more subtle biases? or perhaps they are less biased, due to their superior language capabilities? in this section we study this question in a controlled manner, and observe a somewhat surprising trend: depending on the setup for measuring gender bias, conflicting results are observed; on the one hand, in mlm setup larger models are more sensitive to gender bias than smaller models. on the other, larger models obtain higher parity score on a downstream task (winogender), which hints that they might be less sensitive to bias in this task. we describe our findings below. we measure the occupational gender bias in three models' families, using two methodsprompt based method @xcite and winogender schema @xcite . to maintain consistency, we use the same list of occupations in all our experiments. the gender stereotypicality of an occupation is determined by the u.s. bureau of labor statistics (bls). 4 pretrained models unless stated otherwise, we experiment with three families of pretrained language models: roberta-{base,large} @xcite , deberta-{base,large,xlarge} @xcite and t5-{base,large,3b} @xcite . we provide implementation details in app. b. ## winogender errors analysis unravels biased behavior we have so far shown that larger models make fewer gender errors compared to smaller models (sec. 2.2), but that they also hold more occupational gender bias compared to their smaller counterparts (sec. 2.1). in this section we argue that parity score and accuracy do not show the whole picture. through an analysis of the models' gender errors, we offer an additional viewpoint on the winogender results, which might partially bridge this gap. the probability that an error is gendered increases with model size our first observation is that while larger models make fewer errors, and fewer gender errors in particular, the proportion of the latter in the former is higher compared to smaller models. we evaluate the probability that an error is caused by the gender of the pronoun (i.e., that an error is gendered). we estimate this probability by the proportion of gender errors in total errors: we find for both deberta and roberta that this probability increases with model size (tab. 2, gender column). in the extreme case (debertaxlarge), 41% of the errors are gendered. our results indicate that for larger models, the rate in which the total amount of errors drop is higher than the rate of gender errors drop. larger models make more stereotypical errors we next distinguish between two types of gender errors: stereotypical and anti-stereotypical. as described in sec. to "gotcha" and "not gotcha" instances. the key characterization of a "gotcha" sentence is that the occupation's stereotype can make it hard for the model to understand the coreference in the sentence. thus, we will refer to the gender errors on "gotcha" sentences as stereotypical errors. 8 accordingly, we will refer to gender errors on "not gotcha" sentences as anti-stereotypical errors. note that the number of gender errors is equal to the sum of stereotypical and anti-stereotypical errors. we present in tab. 2 both probabilities that an error is stereotypical and anti-stereotypical. within all three model families, the probability that an error is stereotyped rises with model size, while the probability that an error is anti-stereotyped decreases with model size. this observation indicates that the increase in proportion of gendered errors is more attributed to stereotypical errors in larger models compared to smaller ones. indeed, when considering the distribution of gender errors (fig. 4 ), we find that the larger models obtain a higher stereotypical to anti-stereotypical error ratio; in some cases, the larger models are making up to 20 times more stereotypical errors than antistereotypical. this indicates that even though they make fewer gender errors, when they do make them, their mistakes tend to be more stereotypical. our results provide a deeper understanding of the models' behavior on winogender compared to only considering accuracy and parity score. combined with our mlm results (sec. 2.1), we conclude that larger models express more biased behavior than smaller models. ## related work measuring bias in pretrained language models earlier works presented evaluation datasets such as weat/seat, which measure bias in static word embedding using cosine similarity of specific target words @xcite . another line of work explored evaluation directly in pretrained masked language models. @xcite presented an association relative metric for measure gender bias. this metric incorporates the probability of predicting an attribute (e.g "programmer") given the target for bias (e.g "she"), in a generic template such as "<@xmath0. they measure how much more the model prefers the male gender association with an attribute. @xcite presented stereoset, a large-scale natural dataset to measure four domains of stereotypical biases in models using likelihood-based scoring with respect to their language modeling ability. @xcite introduced crows-pairs, a challenge set of minimal pairs that examines stereotypical bias in nine domains via minimal pairs. they adopted psuedolikelihood based scoring @xcite that does not penalize less frequent attribute term. in our work, we build upon @xcite 's measure in order to examine stereotypical bias to the specifics occupations we use, in different sizes of models. another method to evaluate bias in pretrained models is through downstream tasks, such as coreference resolution @xcite and sentiment analysis @xcite . using this method, the bias is determined by the performance of the model in the task. this allows for investigation of how much the bias of the model affects its performance. ## conclusion we investigated how a model's size affects its gender bias. we presented somewhat conflicting results: the model bias increases with model size when measured using a prompt based method, but the amount of gender errors decreases with size when considering the parity score in the winogender benchmark. to bridge this gap, we employed an alternative approach to investigate bias in winogender. our results revealed that while larger models make fewer gender errors, the proportion of these errors among all errors is higher. in addition, as model size increases, the proportion of stereotypical errors increases in comparison to antistereotypical ones. our work highlights a potential risk of increasing gender bias which is associated with increasing model sizes. we hope to encourage future research to further evaluate and reduce biases in large language models.