A Predictive Factor Analysis of Social Biases and Task-Performance in Pretrained Masked Language Models (2023)

Abstract: Various types of social biases have been reported with pretrained Masked Language Models (MLMs) in prior work. However, multiple underlying factors are associated with an MLM such as its model size, size of the training data, training objectives, the domain from which pretraining data is sampled, tokenization, and languages present in the pretrained corpora, to name a few. It remains unclear as to which of those factors influence social biases that are learned by MLMs. To study the relationship between model factors and the social biases learned by an MLM, as well as the downstream task performance of the model, we conduct a comprehensive study over 39 pretrained MLMs covering different model sizes, training objectives, tokenization methods, training data domains and languages. Our results shed light on important factors often neglected in prior literature, such as tokenization or model objectives.

## introduction masked language models (mlms) have achieved promising performance in many nlp tasks @xcite @xcite . however, mlms trained on massive amounts of textual training data have also been found to encode concerning levels of social biases such as gender and racial biases @xcite @xcite @xcite . in spite of the overall success of mlms across nlp tasks, such biases within mlms raise ethical considerations and underscore the need for debiasing methods to ensure fair and unbiased mlms. on the other hand, mlms are trained by considering and optimising various underlying factors that contribute to their performance on downstream tasks. these factors include but are not limited to parameter size, tokenization methods, training objectives and training corpora. the performance of mlms is affected by the interplay of such factors. nevertheless, it remains unclear as to how these factors influence social biases in mlms and their downstream task performance. evaluating the impact of these factors is challenging due to three main reasons: (a) the factors that we consider within a model are not independent, rather, they exhibit complicated interdependence and affect the performance of models simultaneously. (b) mlms are diverse with different architectures, configurations and parameters. the diversity across models requires the need for generalisation and abstraction when considering the values of factors. (c) many recent works proposed debiasing methods to mitigate social biases in mlms @xcite @xcite . however, most debiasing methods tend to worsen the performance of mlms in downstream tasks @xcite . therefore, it is crucial to consider the trade-off between social bias and downstream task performance when comparing mlms. to address the non-independent issue of factors, we propose a method using gradient boosting @xcite to consider dependencies among factors. moreover, we use the coefficient of determination (r 2 ; nagelkerke et al., 1991) as a measure to analyse the importance of factors. regarding the diversity of mlms, we converge a broad set of mlms across multiple languages and 4 domains, resulting in 39 mlms for evaluation in total. moreover, we incorporate tweeteval and glue to evaluate the downstream task performance of mlms, meanwhile, evaluating their social biases intrinsically using all unmasked likelihood with attention weights (aula; @xcite on two benchmarks stereoset (ss; @xcite and crowdsourced stereotype pairs benchmark (cp; @xcite . note that we are not proposing novel bias evaluation measures in this paper. instead, we use existing metrics such as aula to evaluate social biases. our experimental results indicate that model size, training objectives and tokenization are the three most important categories of factors that affect the social bias and downstream task performance of mlms. interestingly, we observe that models using byte-pair encoding (bpe; @xcite include lower level of social biases, while achieving the best downstream performance compared to models using other tokenization methods. overall, multilingual models tend to have less biases than their monolingual counterparts. ## related work as mlms have been successfully applied to diverse nlp tasks, it is important to study the factors that determine their social biases. @xcite reviewed the current state of knowledge regarding how bert works, what kind of information it learns and how it is encoded, typically alterations to its training objectives and architecture, the overparameterization issue and approaches to compression. @xcite studied contextualised encoders in various aspects and discussed the trade-off between task performance and the potential harms contained in the pretraining data. later, @xcite investigated the effect of pretraining data size on the syntactic capabilities of roberta and they showed that models pretrained with more data tend to encode better syntactic information and provide more syntactic generalisation across different syntactic structures. however, these studies focus on mlms in downstream tasks, while none consider the social biases in mlms. on the other hand, models trained for different downstream tasks have been found to exhibit social biases. @xcite evaluated gender and racial biases across 219 automatic sentiment analysis systems and discovered statistically significant biases occurring in several systems. @xcite studied age-related biases in sentiment classification and discovered that significant age bias is encoded in the output of many sentiment analysis systems as well as word embeddings. @xcite focused on gender bias in multilingual embeddings and its influence on the process of transfer learning for nlp applications. they showed that the level of bias in multilingual representations varies depending on how the embeddings are aligned to different target spaces, and that the alignment direction can also affect the bias in transfer learning. @xcite investigated the types of stereotypical information that are captured by pretrained language models and showed the variability of attitudes towards various social groups among models and the rapid changes in emotions and stereotypes that occur during the fine-tuning phase. existing bias evaluation methods use different strategies such as pseudo likelihood @xcite , cosine similarity @xcite , inner-product @xcite , to name a few. independently of any downstream tasks, intrinsic bias evaluation measures @xcite @xcite evaluate social biases in mlms stand alone. however, given that mlms are used to represent input texts in a variety of downstream tasks, multiple previous works have proposed that social biases should be evaluated with respect to those tasks @xcite . @xcite showed that there exists only a weak correlation between intrinsic and extrinsic social bias evaluation measures. in this paper, we use an intrinsic bias evaluation measure, namely aula, to evaluate social biases in mlms. aula has been shown to be the most reliable bias evaluation measure @xcite , hence we use it as our bias evaluation measure. although we specifically focus on mlms in this paper, evaluating the performance predictors for neural architecture search (nas) @xcite has been conducted for designing high performance neural networks. generalisability of the identified factors is an important in nas because the selected neural architecture would be trained on different datasets to obtain different models. although it would be ideal to perform a controlled training of mlms where we experimentally fix all other factors except for a single target factor and then analyse the trained mlm, this is an expensive undertaking given the computational cost of training mlms on large datasets. on the other hand, we consider already pre-trained mlms that are publicly made available and do not train any mlms during this work. ## analysis of factors in order to study the impact of different factors in mlms, we consider 30 factors and split them into 5 categories. the details of each individual factor are provided in appendix c. ## masked language models to conduct a comprehensive study of different factors of mlms affecting social bias and task performance, we evaluate 39 pretrained mlms, 1 which we divide into four categories as follows. ## mlm evaluation metrics and tasks our goal in this paper is to study the relationship between model factors and social bias as well as downstream task performance in pretrained mlms. for this purpose, we conduct our evaluations using three different types of tasks. ## regression analysis to investigate the importance of the different factors on social bias and the task performance of mlms, we train a regression model. ## qualitative analysis with the knowledge that model size, tokenization, and training objectives are the three primary categories of factors influencing social bias and task performance in mlms from §6.2, we further investigate the factors within each category to discern their individual contributions in mlms. for this purpose, we calculate the average performance of mlms when considering a specific feature associated with a factor for each task. to capture the overall trend, we extend our analysis to include not only monolingual mlms in the general domain but also domain-specific and multilingual mlms. ## conclusion despite the extensive prior work evaluating social bias in mlms, the relationship between social biases and the factors associated with mlms remains unclear. to address this gap, we conducted the first-ever comprehensive study of this type through predictive factor analysis to investigate the impact of different factors on social bias and task performance in pretrained mlms, considering 39 models with 30 factors. we found that model size, tokenization and training objectives are the three most important factors across tasks. in terms of social biases, domain-specific models do not appear to be more or less biased than general domains, while multilingual models, which are trained on corpora covering different languages and cultures, appear to be less socially biased than the monolingual ones.