An Empirical Study on Pseudo-log-likelihood Bias Measures for Masked Language Models Using Paraphrased Sentences (2022)

Abstract: In this paper, we conduct an empirical study on a bias measure, log-likelihood Masked Language Model (MLM) scoring, on a benchmark dataset. Previous work evaluates whether MLMs are biased or not for certain protected attributes (e.g., race) by comparing the log-likelihood scores of sentences that contain stereotypical characteristics with one category (e.g., black) versus another (e.g., white). We hypothesized that this approach might be too sensitive to the choice of contextual words than the meaning of the sentence. Therefore, we computed the same measure after paraphrasing the sentences with different words but with same meaning. Our results demonstrate that the log-likelihood scoring can be more sensitive to utterance of specific words than to meaning behind a given sentence. Our paper reveals a shortcoming of the current log-likelihood-based bias measures for MLMs and calls for new ways to improve the robustness of it

## introduction in recent years, pretrained transformer-based language models, from bert @xcite to palm @xcite , have shown remarkable results in many downstream natural languages processing (nlp) tasks such as question answering, natural language inference, reading comprehension, and text classification as demonstrated by many benchmarks. nevertheless, there is a growing concern if such language models contain social biases such as stereotyping negative generalizations of different social groups and communities, which might have been present in their training corpora @xcite . a cognitive bias, stereotyping, is defined as the assumption of some characteristics are applied to communities on the basis of their nationality, * the authors contributed equally to this work. @xcite . relatedly, fairness ("zero-bias"), in the context of nlp and machine learning is defined as preventing harmful, discriminatory decisions according to such unwanted, stereotypical characteristics @xcite . there are benchmarks and metrics @xcite @xcite @xcite defined for auditing and measuring biases in language models. in this paper, we focus on the crows-pairs dataset @xcite which contains pairs of free-form contrastive sentences where one is a stereotypical sentence which reflects a social bias towards a disadvantaged group and the other with a perturbation of the same with the advantaged group. it attempts to measure if the language model prefers or likely to produce more stereotypical sentence by calculating pseudolog-likelihood masked language model (mlm) scoring @xcite . nevertheless, some analysis such as @xcite questions the extent to which such measures exactly capture the bias of a model. in this paper, we conduct an empirical study to investigate the robustness of the log-likelihoodbased bias measure by paraphrasing the test sentences as in figure 1 and analysing if they produce consistent results. the goal of this study is to gain insights about whether the bias is measured at the meaning / sense of the sentence or at the utterance of specific words. the contributions of this paper are (a) an analysis of the impact of specific utterance of a sentence rather than its meaning in bias measurements, (b) a dataset of manual paraphrases derived from a subset of the crows-pairs dataset. ## experiment in this section, we describe the experimental setup and results. ## discussion overall, the experiment results demonstrate that the pseudo-log-likelihood differences within sentence pairs tend to be very small, so can easily change the direction (positivity/negativity) in response to word choices of input sentences. in the end, we want to ideally measure harmful biases or fairnesses of the underlying pretrained mlms against a set of examples including typical stereotypes. however, the experiment revealed some limitations of the pseudo-log-likelihood bias measurement because the scores fluctuate according to the word choice. therefore, we may not be able to conclude whether a pretrained masked language model like bert is biased or not given one sentence example. the results should consistently persist with paraphrased sentences that are semantically identical. therefore, we believe that we need to test the robustness, fragility, and/or sensitivity of bias measures by bootstrapping/perturbing sentences. the experiment shows one way to test the robustness, but future research can investigate more automated methods. we may conjecture some ways to improve the pseudo-log-likelihood differences used in previous research @xcite . instead of measuring relative likelihood between two sentences in a binary measure, one can think of multiple thresholds that define varying levels of likelihood differences within sentence pairs. we observed that the majority of sentence pairs we tested fall into the range between -.25 and .25. we may consider the sentences that fall into the range as "they are considered nearly likely" rather than "one is more likely than others." it will also be worthwhile to report the magnitude of log-likelihood differences as we showed in our experiment. this work provides a direction for new research: how to test the robustness of bias measures for pretrained masked language models (mlms). we plan to continue our efforts to conduct a large-scale experiment with more automated ways to test the sensitivity. first, in this experiment we used only a small subset of crows-pairs @xcite . we plan to extend our experiment to the entire dataset. second, we manually created paraphrases of given sentences. we plan to automatically detect target phrases and replace them with appropriate synonyms. third, we only used a log-likelihoodbased measure as a bias measuring score in this work. we plan to test the robustness of other scores. last, we also plan to test the statistical significance on the log-likelihood-based measures. ## related work garrido-muñoz et al. provide an extensive survey on biases in nlp @xcite . there are several benchmarks such as stere-oset @xcite , crows-pairs @xcite , winogender @xcite , winobias @xcite containing contrastive sentence pairs are defined for measuring stereotypical bias in mlms. for our experiments, we chose the crows dataset because it covered more bias types.