Feature-Wise Bias Amplification

1 INTRODUCTION . Bias amplification occurs when the distribution over prediction outputs is skewed in comparison to the prior distribution of the prediction target . Aside from being problematic for accuracy , this phenomenon is also potentially concerning as it relates to the fairness of a model ’ s predictions ( Zhao et al. , 2017 ; Burns et al. , 2018 ; Bolukbasi et al. , 2016 ; Stock & Cissé , 2017 ) as models that learn to overpredict negative outcomes for certain groups may exacerbate stereotypes , prejudices , and disadvantages already reflected in the data ( Hart , 2017 ) . Several factors can cause bias amplification in practice . The class imbalance problem is a wellstudied scenario where some classes in the data are significantly less likely than others ( Wallace et al. , 2011a ) . Classifiers trained to minimize empricial risk are not penalized for ignoring minority classes . However , as we show through analysis and experiments , bias amplification can arise in cases where the class prior is not severely skewed , or even when it is unbiased . Thus , techniques for dealing with class imbalance alone can not explain or address all cases of bias amplification . We examine bias amplification in the context of binary classifiers , and show that it can be decomposed into a component that is intrinsic to the model , and one that arises from the inductive bias of gradient descent on certain feature configurations . The intrinsic case manifests when the class prior distribution is more informative for prediction than the features , causing the the model to predict the class mode . This type of bias is unavoidable , as we show that any mitigation of it will lead to less accurate predictions ( Section 3.1 ) . Interestingly , linear classifiers trained with gradient descent tend to overestimate the importance of moderately-predictive , or “ weak , ” features if insufficient training data is available ( Section 3.2 ) . This overestimation gives rise to feature-wise bias amplification – a previously unreported form of bias ( see Section 2 for comparison to related work ) that can be traced back to the features of a trained model . It occurs when there are more features that positively correlate with one class than the other . If these features are given undue importance in the model , then their combined influence will lead to bias amplification in favor of the corresponding class . A concrete example of this would be predicting gender from face images . Suppose that there are several features that occur slightly more often with women ( i.e . are weakly correlated with women ) – long hair , eyeliner , purses , smiling , necklaces – but only a few that are more correlated with men – necktie , mustache . Then , our analysis suggests a model would exhibit feature-wise bias amplification towards female prediction , because there are more weak features oriented towards that class . In fact , we experimentally demonstrate that feature-wise bias amplification can happen even when the class prior is unbiased ; e.g. , even if the aforementioned hypothetical classification problem contained an equal number of male and female training instances . Our analysis sheds new light on real instances of the problem , and paves the way for practical mitigations of it . The existence of such moderately-predictive weak features is not uncommon in models trained on real data . Viewing deep networks as the composition of a feature extractor and a linear classifier , we explain some instances of bias amplification in deep networks ( Table 1 , Section 5 ) . Finally , this understanding of feature-wise bias amplification motivates a solution based on feature selection . We develop two new feature-selection algorithms that are designed to mitigate bias amplification ( Section 4 ) . We demonstrate their effectiveness on both linear classifiers and deep neural networks ( Section 5 ) . For example , for a VGG16 network trained on CelebA ( Liu et al. , 2015 ) to predict the “ attractive ” label , our approach removed 95 % of the bias in predictions . We observe that in addition to mitigating bias amplification , these feature selection methods reduce generalization error relative to an ` 1 regularization baseline for both linear models and deep networks ( Table 1 ) . 2 RELATED WORK . While the term bias is used in a number of different contexts in machine learning , we use bias amplification in the sense of Zhao et al . ( 2017 ) , where the distribution over prediction outputs is skewed in comparison to the prior distribution of the prediction target . For example , Zhao et al . ( 2017 ) and Burns et al . ( 2018 ) use the imSitu vSRL dataset for the MS-COCO task , i.e . to classify agents and actions in pictures . In the dataset , women are twice as likely to be the agent when the action is cooking , but the model was five times as likely to predict women to be the agent cooking . In a related example , Stock & Cissé ( 2017 ) identify bias in models trained on the ImageNet dataset . Despite there being near-parity of white and black people in pictures in the basketball class , 78 % of the images that the model classified as basketball had black people in them and only 44 % had white people in them . Additionally , 90 % of the misclassified basketball pictures had white people in them , whereas only 20 % had black people in them . Note that this type of bias over classes is distinct from the learning bias in machine learning ( Geman et al. , 1992 ) which has received renewed interest in the context of SGD and under-determined models ( Gunasekar et al. , 2018 ; Soudry et al. , 2017 ) . Naturally , bias has been studied in the context of fairness , and in particular in cases where preexisting societal biases are propagated by a model ’ s predictions . One recent example of this is Amazons gender-biased recruiting tool , which learned not to prefer candidates from all-female colleges based on past hiring data ( Dastin , 2018 ) . Removing such biases from models ’ behavior altogether using fairness criteria ( Bilal Zafar et al. , 2015 ; Binns , 2018 ; Zliobaite , 2015 ) requires deviating from patterns in the data , and may conflict with accuracy . However , bias can also come as a product of the model itself : a model could ostensibly take a relatively unbiased dataset and erroneously make more biased predictions due to some limitations of learning . Feature-wise bias falls into this class , as it is created by the mechanics of the learning rule itself rather than stemming from the data . This means that removing feature-wise bias would make the model more faithful to the data it was given , generally increasing accuracy . Consequently , this paper does not weigh in on which fairness guidelines to use in navigating the bias-accuracy tradeoff , since it does not apply to the type of bias we present . Bias amplification is often thought to be result of class imbalance in the training data , which is wellstudied in the learning community ( see He & Garcia ( 2009 ) and Buda et al . ( 2017 ) for comprehensive surveys ) . There are a myriad of empirical investigations of the effects of class imbalance in machine learning and different ways of mitigating these effects ( Maloof , 2003 ; Chawla , 2005 ; Mazurowski et al. , 2008 ; Oommen et al. , 2011 ; Wallace et al. , 2011b ) . It has been shown that neural networks are affected by class imbalance as well ( Murphey et al. , 2004 ) . Buda et al . ( 2017 ) point out that the detrimental effect of class imbalance on neural networks increases with scale . They advocate for an oversampling technique mixed with thresholding to improve accuracy based on empirical tests . An interesting and less common technique from Havaei et al . ( 2015 ) relies on a drastic change to neural network training procedure in order to better detect brain tumors : they first train the net on an even distribution , and then on a representative sample , but only on the output layer in the second half of training . In contrast to prior work , we demonstrate that bias amplification can occur without existing imbalances in the training set . Therefore , we identify a new source of bias that can be traced to particular features in the model . Since we remove bias feature-wise , our approach can also be viewed as method for feature selection . While feature selection is a well-studied problem , to the authors ’ knowledge , no one has looked at removing features to mitigate bias . Generally , feature selection has been applied for improving model accuracy , or gaining insight into the data ( Chandrashekar & Sahin , 2014 ) . For example , Kim et al . ( 2015 ) use feature selection for interpretability during data exploration . They select features that have high variance across clusters created based on humaninterpretable , logical rules . Differing from prior work , we focus on bias by identifying features that are likely to increase bias , but can be removed while maintaining accuracy . Naive Bayes classification models comprise a similarly well-studied topic . Rennie et al . ( 2003 ) point out common downfalls of Naive Bayes classifiers on datasets that do not meet Naive Bayes criteria : bias from class imbalance , and the problem of over-predicting classes with correlated features . Our work shows that similar effects can occur even on data that does match Naive Bayes assumptions . Zhang ( 2004 ) shows that the naive Bayes classifier is optimal so long as the dependencies between features over the whole network cancel each other out . Our work can mitigate bias in scenarios where these conditions do not hold . 3 BIAS AMPLIFICATION IN BINARY CLASSIFIERS . In this section , we define bias amplification for binary classifiers , and show that in some cases it may be unavoidable . Namely , a Bayes-optimal classifier trained on poorly-separated data can end up predicting one label nearly always , even if the prior label bias is minimal . While our analysis makes strong generative assumptions , we show that its results hold qualitatively on real data that resemble these assumptions . We begin by formalizing the setting . We consider the standard binary classification problem of predicting a label y ∈ { 0 , 1 } given features x = ( x1 , . . . , xd ) ∈ X . We assume that data are generated from some unknown distribution D , and that the prior probability of y = 1 is p∗ . Without loss of generality , we assume that p∗ ≥ 1/2 . The learning algorithm recieves a training set S drawn i.i.d . from Dn and outputs a predictor hS : X → { 0 , 1 } with the goal of minimizing 0-1 loss on unknown future i.i.d . samples from D. Definition 1 ( Bias amplification , systematic bias ) Let hS be a binary classifier trained on S ∼ Dn . The bias amplification of hS on D , written BD ( hS ) , is given by Equation 1 . BD ( hS ) = E ( x , y ) ∼D [ hS ( x ) − y ] ( 1 ) We say that a learning rule exhibits systematic bias whenever it exhibits non-zero bias amplification on average over training samples , i.e . it satisfies Equation 2 . E S∼Dn [ BD ( hS ) ] 6= 0 ( 2 ) Definition 1 formalizes bias amplification and systematic bias in this setting . Intuitively , bias amplification corresponds to be the probability that hS predicts class 1 on instances from class 0 in excess of the prior p∗ . Systematic bias lifts the definition to learners , characterizing rules that are expected to amplify bias on training sets drawn from D .

Summary: In this paper the authors identify a specific source of marginal class probability bias that occurs when using logistic regression models. Using synthetic and real datasets they demonstrate this bias and explore characteristics of the data that exacerbate the issue. Finally, they propose two methods for correcting this bias in logistic regression models and neural network models with logistic output layers and evaluate these methods on several benchmark datasets.