Fairness-aware Class Imbalanced Learning (2021)

Abstract: Class imbalance is a common challenge in many NLP tasks, and has clear connections to bias, in that bias in training data often leads to higher accuracy for majority groups at the expense of minority groups. However there has traditionally been a disconnect between research on class-imbalanced learning and mitigating bias, and only recently have the two been looked at through a common lens. In this work we evaluate long-tail learning methods for tweet sentiment and occupation classification, and extend a margin-loss based approach with methods to enforce fairness. We empirically show through controlled experiments that the proposed approaches help mitigate both class imbalance and demographic biases.

## introduction class imbalance is common in many nlp tasks, including machine reading comprehension @xcite , authorship attribution @xcite , toxic language detection @xcite , and text classification @xcite . a skewed class distribution hurts the performance of deep learning models @xcite , and approaches such as instance weighting @xcite @xcite , data augmentation @xcite , and weighted max-margin @xcite are commonly used to alleviate the problem. bias in data often also manifests as skewed distributions, especially when considered in combination with class labels. this is often referred to as "stereotyping" whereby one or more private attributes are associated more frequently with certain target labels, for instance more men being employed as surgeons than women. prior work has identified several classes of bias, including 1 code available at: https://github.com/ shivashankarrs/classimb_fairness c figure 1 : example of a two-class problem where grey and blue points denote majority and minority classes, respectively, and circles and squares denote two sub-groups. imbalanced learning methods such as ldam @xcite maximise the (soft-)margin for minority classes and do not consider sub-groups within each class. this paper draws a connection between classimbalanced learning and stereotyping bias. most work has focused on class-imbalanced learning and bias mitigation as separate problems, but the unfairness caused by social biases is often aggravated by the presence of class imbalance @xcite . class-imbalanced learning approaches improve the performance of minority classes at some cost to the performance of majority classes. a common approach re-weights instances in the training objective to be proportional to the inverse frequency of their class. approaches such as focal @xcite and dice @xcite extend this approach by down-weighting "easy" instances. label-distribution-aware margin loss ("ldam": cao et al. (2019) ) is an alternative approach, which encourages a larger margin for the minority class, but it does not consider sub-group proportions (see figure 1 ). on the other hand, debiasing approaches do not typically focus on class imbalance explicitly. for instance, in toxicity classification, certain sub-groups are often predicted more confidently for toxicity (encouraging false negatives for the majority sub-group), which tend to be close to the margin for the non-toxic class (encouraging false positives; @xcite al. (2019) ). in this work, we modify the training objective of ldam as a state-of-the-art approach for imbalanced learning so that margins depend not just on class-imbalance, but also on the subgroup distribution within each class. specifically, we extend ldam with popular debiasing strategies. we show the effectiveness of our approach through several controlled experiments on two text classification data sets. ## proposed approaches let (x, y, g) denote a training instance, comprising an input, label, and group identifier, respectively. ldam @xcite addresses class imbalance by enforcing a larger margin for minority classes: l ldam (x, y; f ) = -log e zy-∆y e zy-∆y + @xmath0, . . . , k} where @xmath1, k is the number of classes, n j is the number of instances in class j, and c is a hyperparameter. smaller classes are associated with a larger ∆ y , which is subtracted from the model output z y , thus enforcing a larger margin. we propose three extensions to ldam, each of which takes into account imbalance in the distribution of private attributes across classes: ldam iw adds instance re-weighting, based on groups within each class: ldam adv adds an adversarial term, so that the learnt representations are a poor predictor of group membership @xcite : ldam reg adds a soft regularization term which encourages fairness as equalised odds by reducing maximum mean discrepancy @xcite across groups. the probability of predicting some class k for any individual group g should be close to k's probability over the whole data set: ## experimental results we perform experiments on the two tasks of emoji prediction and occupation classification, both of which are binary classification tasks with binary protected attributes. emoji prediction: we use the twitter dataset of @xcite , where tweets are associated with the private attribute race (black/white), and sentiment labels are derived from emoji usage (happy/sad) @xcite . we experiment with different levels of class and stereotyping imbalance in the emoji dataset, including its original distribution (see section 3.1). occupation classification: this data set consists of short biographies scraped from the web, annotated for private attribute gender (male/female) and target occupation labels @xcite . we focus on two occupations with well-documented gender stereotypes -surgeon and nurse. the resulting dataset is mildly class-imbalanced (59% surgeon: 41% nurse), with roughly symmetric natural gender splits (90% male for surgeon and 90% female for nurse). for emoji prediction we follow @xcite and use the deepmoji encoder, which was trained on millions of tweets and is known to encode demographic information @xcite . for occupation classification, we use the bert-base uncased model and classify via the last hidden state of the cls token @xcite . both encoders are followed by a single hidden layer mlp. we evaluate classification performance based on macro-averaged f-score (to account for class imbalance), and evaluate fairness using performance gap: the average of the true positive rate (tpr) and true negative rate (tnr) differences between the two subgroups @xcite . note that a wide variety of fairness measures (both on the group-and individual levels) have been proposed, which are impossible to satisfy simultaneously @xcite . often, a suitable measure is chosen based on the target application. here we use the popular equalised odds measure considering both tpr and tnr of classifiers, in order to address scenarios where certain subgroups are predicted more often with some classes (see section 1). we report fairness as 1-gap, such that higher numbers are better, and a perfectly fair model achieves 1-@xmath2. we compare our methods against the following benchmarks: vanilla: unweighted cross-entropy loss. ## conclusion and future work we explored the interplay of class-imbalance and stereotyping in two language classification data sets. we showed that vanilla class-imbalanced learning (iw, cw, focal, ldam and ldam cw ) can exacerbate unfairness. we extended classimbalanced learning approaches to handle fairness under stereotyping, and showed that our models provide consistent gains in fairness without sacrificing accuracy. both ldam reg which uses maximum mean discrepancy regularizer @xcite and ldam adv with adversarial loss @xcite are different ways to make the text representation independent of demographic attributes. consistent with previous work @xcite we find that ldam reg is robust and performs best across several test scenarios, except in extremely skewed (or stereotyped) settings where the gains of ldam reg over its adversarial counterpart (ldam adv ) diminishes. in addition, ldam adv introduces more parameters into the model, and is in general hard to train, hence ldam reg is more preferable overall. in the future, we plan to extend our methods to more complex tasks and multiple private attributes @xcite .