Quantifying Social Biases in NLP : A Generalization and Empirical Comparison of Extrinsic Fairness Metrics (2021)

Abstract: Measuring bias is key for better understanding and addressing unfairness in NLP/ML models. This is often done via fairness metrics, which quantify the differences in a model’s behaviour across a range of demographic groups. In this work, we shed more light on the differences and similarities between the fairness metrics used in NLP. First, we unify a broad range of existing metrics under three generalized fairness metrics, revealing the connections between them. Next, we carry out an extensive empirical comparison of existing metrics and demonstrate that the observed differences in bias measurement can be systematically explained via differences in parameter choices for our generalized metrics.

## introduction the prevalence of unintended social biases in nlp models has been recently identified as a major concern for the field. a number of papers have published evidence of uneven treatment of different demographics @xcite @xcite @xcite @xcite @xcite , which can reportedly cause a variety of serious harms, like unfair allocation of opportunities or unfavorable representation of particular social groups @xcite . measuring bias in nlp models is key for better understanding and addressing unfairness. this is often done via fairness metrics, which quantify the differences in a model's behavior across a range of social groups. the community has proposed a multitude of such metrics @xcite @xcite @xcite . in this pa-♠ work done during an internship at amazon ai. per, we aim to shed more light on how those varied means of quantifying bias differ and what facets of bias they capture. developing such understanding is crucial for drawing reliable conclusions and actionable recommendations regarding bias. we focus on bias measurement for downstream tasks, as @xcite have recently shown that there is no reliable correlation between bias measured intrinsically on, for example, word embeddings, and bias measured extrinsically on a downstream task. we narrow down the scope of this paper to tasks that do not involve prediction of a sensitive attribute. we survey 146 papers on social bias in nlp and unify the multitude of disparate metrics we find under three generalized fairness metrics. through this unification we reveal the key connections between a wide range of existing metricswe show that they are simply different parametrizations of our generalized metrics. next, we empirically investigate the role of different metrics in detecting the systemic differences in performance for different demographic groups, namely, differences in quality of service @xcite . we experiment on three transformer-based models-two models for sentiment analysis and one for named entity recognition (ner)-which we evaluate for fairness with respect to seven different sensitive attributes, qualified for protection under the united states federal anti-discrimination law: 1 gender, sexual orientation, religion, nationality, race, age, and disability. our results highlight the differences in bias measurements across the metrics and we discuss how these variations can be systematically explained via different parameter choices of our generalized metrics. our proposed unification and observations can guide decisions about which metrics @xcite to use, allowing researchers to focus on the pressing matter of bias mitigation, rather than reinventing parametric variants of the same metrics. while we focus our experiments on english, the metrics we study are language-agnostic and our methodology can be trivially applied to other languages. we release our code with implementations of all metrics discussed in this paper. 2 our implementation mirrors our generalized formulation (section 3), which simplifies the creation of new metrics. we build our code on top of check-list foot_2 @xcite , making it compatible with the checklist testing functionalities; that is, one can evaluate the model using the fairness metrics, as well as the checklist-style tests, like invariance, under a single bias evaluation framework. ## generalized fairness metrics we introduce three generalized fairness metrics that are based on different comparisons between protected groups and are model and task agnostic. they are defined in terms of two parameters: each of the three metrics is conceptually different and is most suitable in different scenarios; the choice of the most appropriate one depends on the scientific question being asked. through different choices for φ and d, we can systematically formulate a broad range of different fairness metrics, targeting different types of questions. we demonstrate this in section 4 and ## experimental details having introduced our generalized framework and classified the existing metrics, we now empirically investigate their role in detecting the systemic performance difference across the demographic groups. we first discuss the relevant experimental details before presenting our results and analyses (section 6). ## empirical metric comparison figure 1 shows the results for sentiment analysis for all attributes on bcm, pcm, and mcm metrics. in each table we report the original bias measurements and row-normalize the heatmap coloring using maximum absolute value scaling to allow for some cross-metric comparison. 11 figure 1 gives evidence of unintended bias for most of the attributes we consider, with disability and nationality being the most and least affected attributes, respectively. we highlight that because we evaluate on simple synthetic data in which the expressed sentiment is evident, even small performance differences can be concerning. figure 1 also gives an initial insight into how the bias measurements vary across the metrics. 11 even after normalization, bias measurements across metrics are not fully comparable-different metrics use different base measurements (tpr, tnr, etc.) and hence measure different aspects of bias. in figure 2 we present the per-group results for vbcm and bcm metrics for the example gender attribute. 12 similarly, in figure 3 we show results for ner for the relevant loc class. the first set of results indicates that the most problematic gender group is cis. for ner we observe a big gap in the model's performance between the most affluent countries and countries with lower gdp. in the context of those empirical results we now discuss how different parameter choices affect the observed bias measurement. ## related work to our knowledge, we are the first to review and empirically compare fairness metrics used within nlp. close to our endeavor are surveys that discuss types, sources, and mitigation of bias in nlp or ai in general. surveys of @xcite , @xcite , and chouldechova and roth (2020) cover a broad scope of literature on algorithmic fairness. @xcite offer both a survey of bias in nlp as well as a conceptual framework for studying bias. @xcite provide a comprehensive overview of addressing gender bias in nlp. there are also many task specific surveys, for example, for language generation @xcite or machine translation @xcite . @xcite outline a number of methodological issues, such as providing vague motivations, which are common for papers on bias in nlp. we focus on measuring bias exhibited on classification and sequence labeling downstream tasks. a related line of research measures bias present in sentence or word representations @xcite @xcite @xcite @xcite @xcite @xcite . however, such intrinsic metrics have been recently shown not to correlate with application bias @xcite . in yet another line of research, @xcite detect bias through identifying bias sensitive words. beyond the fairness metrics and significance tests, some works quantify bias through calculating a standard evaluation metric, for example, f1 or accuracy, or a more elaborate measure independently for each protected group or for each split of a challenge dataset @xcite @xcite @xcite @xcite @xcite @xcite . ## conclusion we conduct a thorough review of existing fairness metrics and demonstrate that they are simply parametric variants of the three generalized fairness metrics we propose, each suited to a different type of a scientific question. further, we empirically demonstrate that the differences in parameter choices for our generalized metrics have direct impact on the bias measurement. in light of our results, we provide a range of concrete suggestions to guide nlp practitioners in their metric choices. we hope that our work will facilitate further research in the bias domain and allow the researchers to direct their efforts towards bias mitigation. because our framework is language and model agnostic, in the future we plan to experiment on more languages and use our framework as principled means of comparing different models with respect to bias.