Evaluating Debiasing Techniques for Intersectional Biases (2021)

Abstract: Bias is pervasive for NLP models, motivating the development of automatic debiasing techniques. Evaluation of NLP debiasing methods has largely been limited to binary attributes in isolation, e.g., debiasing with respect to binary gender or race, however many corpora involve multiple such attributes, possibly with higher cardinality. In this paper we argue that a truly fair model must consider ‘gerrymandering’ groups which comprise not only single attributes, but also intersectional groups. We evaluate a form of bias-constrained model which is new to NLP, as well an extension of the iterative nullspace projection technique which can handle multiple identities.

## introduction text data reflects the social and cultural biases in the world, and nlp models and applications trained on such data have been shown to reproduce and amplify those biases. discrimination has been identified across diverse sensitive attributes including gender, disability, race, and religion @xcite @xcite @xcite . while early work focused on debiasing typically binarized protected attributes in isolation (e.g., age, gender, or race; @xcite ), more recent work has adopted a more realistic scenario with multiple sensitive attributes @xcite or attributes covering several classes @xcite . in the context of multiple protected attributes, gerrymandering refers to the phenomenon where an attempt to make a model fairer towards some group results in increased unfairness towards another group @xcite @xcite . notably, algorithms can be fair towards independent groups, but not towards all intersectional groups. despite this, debiasing approaches within nlp have so far been evaluated only using independent group fairness when modelling datasets with multiple attributes, disregarding intersectional subgroups defined by combinations of sensitive attributes (see figure 1 ). the primary goal of this work is to evaluate independent and intersectional identity debiasing approaches in relation to fairness gerrymandering for text classification tasks. to this end, we evaluate bias-constrained models @xcite and iterative nullspace projection (inlp; @xcite ), a post-hoc debiasing method which we extend to handle intersectional groups. the constrained model jointly optimizes model performance and model fairness, while inlp seeks to learn a hidden representation which is independent of the protected attributes. inlp does not consider the trade-off between accuracy and fairness, but rather it iteratively maximizes fairness in an unconstrained fashion. in this work, we address the following questions: ## background debiasing with respect to more than a single protected attribute (|z| > 1) requires grouping data points according to their associated protected values. in this work, debiasing approaches are trained wrt the three settings detailed below @xcite , evaluated based on aggregated violations over g gerry , which is a commonly used evaluation strategy for intersectional identities @xcite . independent groups (g indep ) each private attribute is treated as an independent group. in the case of all binary attributes, this gives 2|z| groups. as illustrated in figure 1 , for binary gender and race, 1 f, m, w, and ¬w are four independent groups, which are overlapping, e.g., with f referring to both white females and female persons of colour (c and d). intersectional groups (g inter ) nonoverlapping, exhaustive combinations of groups, which in the binary attribute case gives 2 |z| combinations. for our example in figure 1 , the intersectional groups are a, b, c and d. ## proposed approaches we aim to learn a model to predict main task labels (e.g., sentiment) from input data x (e.g., reviews) by means of a learnt hidden representation x h . x h will generally encode implicit biases, and our goal is to predict the output without the influence of social identities encoded in x h . we assume access to the protected attributes for each instance. below, we detail the two primary models targeted in this work: inlp, and the bias-constrained model. ## experimental results we evaluate using two binary classification datasets: (1) twitter hate speech detection; and (2) occupation classification using biography text. for inlp we use logistic regression to construct w. all the approaches have hyperparameters controlling the performance-fairness trade-off, and as no choice of hyperparameter will optimize both objectives simultaneously, we adopt the concept of pareto optimality @xcite . ## conclusion and future work we examined the impact of independent vs. intersectional groups on classifier fairness. we proposed an extended version of inlp, which we compared against bias-constrained models. inlp is a post-hoc debiasing method, while the constrained model requires full joint training. debiasing in inlp happens in an unconstrained way, where sensitive information is removed from the latent representations independent of predictive performances, leading to fairer models at the cost of overall accuracy @xcite . from a practical perspective, post-hoc debiasing may be the only option if it is impossible or undesirable to re-train a model from scratch with a regularized objective. our paper shows that lighter-weight post-hoc debiasing is indeed possible in the case of overlapping groups, and quantifies the additional advantage of joint training and debiasing in the context of two data sets. the constrained models especially are more robust to stereotyped settings, and using intersectional identities benefits inlp in particular.