F EATURE -W ISE B IAS A MPLIFICATION
Anonymous authors
Paper under double-blind review
A BSTRACT
We study the phenomenon of bias ampliﬁcation in classiﬁers, wherein a machine learning model learns to predict classes with a greater disparity than the underlying ground truth.
We demonstrate that bias ampliﬁcation can arise via inductive bias in gradient descent methods resulting in overestimation of importance of moderately-predictive “weak” features if insufﬁcient training data is available.
This overestimation gives rise to feature-wise bias ampliﬁcation – a previously unreported form of bias that can be traced back to the features of a trained model.
Through analysis and experiments, we show that the while some bias cannot be mitigated without sacriﬁcing accuracy, feature-wise bias ampliﬁcation can be mitigated through targeted feature selection.
We present two new feature selection algorithms for mitigating bias ampliﬁcation in linear models, and show how they can be adapted to convolutional neural networks efﬁciently.
Our experiments on synthetic and real data demonstrate that these algorithms consistently lead to reduced bias without harming accuracy, in some cases eliminating predictive bias altogether while providing modest gains in accuracy.
I NTRODUCTION
Bias ampliﬁcation occurs when the distribution over prediction outputs is skewed in comparison to the prior distribution of the prediction target.
Aside from being problematic for accuracy, this phenomenon is also potentially concerning as it relates to the fairness of a model’s predictions (Zhao et al., 2017; Burns et al., 2018; Bolukbasi et al., 2016; Stock and Ciss ´ e, 2017) as models that learn to overpredict negative outcomes for certain groups may exacerbate stereotypes, prejudices, and disadvantages already reﬂected in the data (Hart, 2017).
Several factors can cause bias ampliﬁcation in practice.
The class imbalance problem is a wellstudied scenario where some classes in the data are signiﬁcantly less likely than others (Wallace et al., 2011a).
Classiﬁers trained to minimize empricial risk are not penalized for ignoring minority classes.
However, as we show through analysis and experiments, bias ampliﬁcation can arise in cases where the class prior is not severely skewed, or even when it is unbiased.
Thus, techniques for dealing with class imbalance alone cannot explain or address all cases of bias ampliﬁcation.
We examine bias ampliﬁcation in the context of binary classiﬁers, and show that it can be decomposed into a component that is intrinsic to the model, and one that arises from the inductive bias of gradient descent on certain feature conﬁgurations.
The intrinsic case manifests when the class prior distribution is more informative for prediction than the features, causing the the model to predict the class mode.
This type of bias is unavoidable, as we show that any mitigation of it will lead to less accurate predictions (Section 3.1).
Interestingly, linear classiﬁers trained with gradient descent tend to overestimate the importance of moderately-predictive, or “weak,” features if insufﬁcient training data is available (Section 3.2).
This overestimation gives rise to feature-wise bias ampliﬁcation – a previously unreported form of bias (see Section 2 for comparison to related work) that can be traced back to the features of a trained model.
It occurs when there are more features that positively correlate with one class than the other.
If these features are given undue importance in the model, then their combined inﬂuence will lead to bias ampliﬁcation in favor of the corresponding class.
Indeed, we experimentally demonstrate that feature-wise bias ampliﬁcation can happen even when the class prior is unbiased.
Our analysis sheds new light on real instances of the problem, and paves the way for practical mitigations of it.
The existence of such moderately-predictive weak features is not uncommon in models trained on real data.
Viewing deep networks as the composition of a feature extractor and a linear classiﬁer, we explain some instances of bias ampliﬁcation in deep networks (Table 1, Section 5).
Finally, this understanding of feature-wise bias ampliﬁcation motivates a solution based on feature selection.
We develop two new feature-selection algorithms that are designed to mitigate bias ampliﬁcation (Section 4).
We demonstrate their effectiveness on both linear classiﬁers and deep neural networks (Section 5).
For example, for a VGG16 network trained on CelebA (Liu et al., 2015) to predict the “attractive” label, our approach removed 95% of the bias in predictions.
We observe that in addition to mitigating bias ampliﬁcation, these feature selection methods reduce generalization error relative to an (cid:96) 1 regularization baseline for both linear models and deep networks (Table 1).  R ELATED W ORK
While the term bias is used in a number of different contexts in machine learning, we use bias ampliﬁcation in the sense of Zhao et al. (2017), where the distribution over prediction outputs is skewed in comparison to the prior distribution of the prediction target. For example, Zhao et al. (2017) and Burns et al.
(2018) use the imSitu vSRL dataset for the MS-COCO task, i.e. to classify agents and actions in pictures.
In the dataset, women are twice as likely to be the agent when the action is cooking, but the model was ﬁve times as likely to predict women to be the agent cooking.
In a related example, Stock and Ciss ´ e (2017) identify bias in models trained on the ImageNet dataset.
Despite there being near-parity of white and black people in pictures in the basketball class, 78% of the images that the model classiﬁed as basketball had black people in them and only 44% had white people in them.
Additionally, 90% of the misclassiﬁed basketball pictures had white people in them, whereas only 20% had black people in them.
Note that this type of bias over classes is distinct from the learning bias in machine learning (Geman, Bienenstock, and Doursat, 1992) which has received renewed interest in the context of SGD and under-determined models (Gunasekar et al., 2018; Soudry et al., 2017).
Bias ampliﬁcation is often thought to be result of class imbalance in the training data, which is wellstudied in the learning community (see He and Garcia (2009) and Buda, Maki, and Mazurowski (2017) for comprehensive surveys).
There are a myriad of empirical investigations of the effects of class imbalance in machine learning and different ways of mitigating these effects (Maloof, 2003;
Chawla, 2005; Mazurowski et al., 2008; Oommen, Baise, and Vogel, 2011; Wallace et al., 2011b).
It has been shown that neural networks are affected by class imbalance as well (Murphey, Guo, and
Feldkamp, 2004).
Buda, Maki, and Mazurowski (2017) point out that the detrimental effect of class imbalance on neural networks increases with scale.
They advocate for an oversampling technique mixed with thresholding to improve accuracy based on empirical tests.
An interesting and less common technique from Havaei et al.
(2015) relies on a drastic change to neural network training procedure in order to better detect brain tumors: they ﬁrst train the net on an even distribution, and then on a representative sample, but only on the output layer in the second half of training.
In contrast to prior work, we demonstrate that bias ampliﬁcation can occur without existing imbalances in the training set.
Therefore, we identify a new source of bias that can be traced to particular features in the model.
Since we remove bias feature-wise, our approach can also be viewed as method for feature selection.
While feature selection is a well-studied problem, to the authors’ knowledge, no one has looked at removing features to mitigate bias .
Generally, feature selection has been applied for improving model accuracy, or gaining insight into the data (Chandrashekar and Sahin, 2014).
For example, Kim, Shah, and Doshi-Velez (2015) use feature selection for interpretability during data exploration.
They select features that have high variance across clusters created based on human-interpretable, logical rules.
Differing from prior work, we focus on bias by identifying features that are likely to increase bias, but can be removed while maintaining accuracy.
Naive Bayes classiﬁcation models comprise a similarly well-studied topic.
Rennie et al.
(2003) point out common downfalls of Naive Bayes classiﬁers on datasets that do not meet Naive Bayes criteria: bias from class imbalance, and the problem of over-predicting classes with correlated features.
Our work shows that similar effects can occur even on data that does match Naive Bayes assumptions.
Zhang (2004) shows that the naive Bayes classiﬁer is optimal so long as the dependencies between features over the whole network cancel each other out.
Our work can mitigate bias in scenarios where these conditions do not hold.
B IAS A MPLIFICATION IN B INARY C LASSIFIERS
In this section, we deﬁne bias ampliﬁcation for binary classiﬁers, and show that in some cases it may be unavoidable.
Namely, a Bayes-optimal classiﬁer trained on poorly-separated data can end up predicting one label nearly always, even if the prior label bias is minimal.
While our analysis makes strong generative assumptions, we show that its results hold qualitatively on real data that resemble these assumptions.
We begin by formalizing the setting.
We consider the standard binary classiﬁcation problem of predicting a label y ∈ { 0 , 1 } given features x = ( x 1 , . . .
, x d ) ∈ X .
We assume that data are generated from some unknown distribution D , and that the prior probability of y = 1 is p ∗ .
Without loss of generality, we assume that p ∗ ≥ 1 / 2 .
The learning algorithm recieves a training set S drawn i.i.d.
from D n and outputs a predictor h S :
X → { 0 , 1 } with the goal of minimizing 0-1 loss on unknown future i.i.d.
samples from D .
Deﬁnition 1 (Bias ampliﬁcation, systematic bias) Let h S be a binary classiﬁer trained on S ∼ D n .
The bias ampliﬁcation of h S on D , written B D ( h S ) , is given by Equation 1.
[Equation]
We say that a learning rule exhibits systematic bias whenever it exhibits non-zero bias ampliﬁcation on average over training samples, i.e. it satisﬁes Equation 2.
[Equation]
Deﬁnition 1 formalizes bias ampliﬁcation and systematic bias in this setting.
Intuitively, bias ampliﬁcation corresponds to be the probability that h S predicts class 1 on instances from class 0 in excess of the prior p ∗ .
Systematic bias lifts the deﬁnition to learners, characterizing rules that are expected to amplify bias on training sets drawn from D .
3. S YSTEMATIC B IAS IN B AYES -O PTIMAL P REDICTORS
Deﬁnition 1 makes it clear that systematic bias is a property of the learning rule producing h S and the distribution, so any technique that aims to address it will need to change one or both.
However, if the learner always produces Bayes-optimal predictors for D , then any such change will result in suboptimal classiﬁers, making bias ampliﬁcation unavoidable .
In this section we characterize the systematic bias of a family of linear Bayes-optimal predictors.
Consider a special case of binary classiﬁcation in which x are drawn from a multivariate Gaussian distribution with class means µ ∗ 0 , µ ∗ 1 ∈ R d and diagonal covariance matrix Σ ∗ , and y is a Bernoulli random variable with parameter p ∗ .
Then D is given by Equation 3.
[Equation]
Because the features in x are independent given the class label, the Bayes-optimal learning rule for this data is Gaussian Naive Bayes, which is expressible as a linear classiﬁer (Murphy, 2012).
Making the ideal assumption that we are always able to learn the Bayes-optimal classiﬁer h ∗ for parameters µ ∗ y , Σ ∗ , p ∗ , we proceed with the question: does h ∗ have systematic bias?
Our assumption of h S = h ∗ reduces this question to whether B D ( h ∗ ) is zero.
[-(a) 0 .0 .0 .0 .
-]
Proposition 1 shows that B D ( h ∗ ) is strictly a function of the class prior p ∗ and the Mahalanobis distance D of the class means µ ∗ y .
Corollary 1 shows that when the prior is unbiased, the model’s predictions remain unbiased.
Proposition 1 Let x be distributed according to Equation 3, y be Bernoulli with parameter p ∗ , D be the Mahalanobis distance between the class means µ ∗ 0 , µ ∗ 1 , and β = − D − 1 log( p ∗ / (1 − p ∗ )) .
Then the bias ampliﬁcation of the Bayes-optimal classiﬁer h ∗ is:
[Equation]
Corollary 1 When x is distributed according to Equation 3 and p ∗ = 1 / 2 , B D ( h ∗ ) = 0 .
The proofs of both claims are given in the appendix.
Corollary 1 is due to the fact that when p ∗ = 1 / 2 , β = 0 .
Because of the symmetry Φ( − x ) = 1 − Φ( x ) , the Φ terms cancel out giving
Pr[ h ∗ ( x ) = 1] = 1 / 2 , and thus the bias ampliﬁcation B D ( h ∗ ) = 0 . [Table] [Table] [Table]
[Table]
0 .p ∗
B D ( h ∗ ) D = 14 D = D =(b) Figure 1a shows the effect on real data available on the UCI repository classiﬁed using Gaussian
Naive Bayes (GNB).
These datasets were chosen because their distributions roughly correspond to the naive Bayes assumption of conditional feature independence, and GNB outperformed logistic regression.
In each case, bias ampliﬁcation occurs in approximate correspondence with Proposition 1, tracking the empirical class prior and class distance to Figure 1b.
Figure 1b shows B D ( h ∗ ) as a function of p ∗ for several values of D .
As the means grow closer together, there is less information available to make reliable predictions, and the label prior is used as the more informative signal.
Note that B D ( h ∗ ) is bounded by 1/2, and the critical point corresponds to bias “saturation” where the model always predicts class 1.
From this it becomes clear that the extent to which overprediction occurs grows rather quickly when the means are moderately close.
For example when p ∗ = 3 / 4 and the class means are separated by distance 1 / 2 , the classiﬁer will predict Y = 1 with probability close to 1.
Summary :
Bias ampliﬁcation may be unavoidable when the learning rule is a good ﬁt for the data, but the features are less effective at distinguishing between classes than the prior.
Our results show that in the particular case of conditionally-independent Gaussian data, the Bayes-optimal predictor suffers from bias as the Mahalanobis distance between class means decreases, leading to a noticeable increase even when the prior is only somewhat biased.
The effect is strong enough to manifest in real settings where generative assumptions do not hold, but GNB outperforms other linear classiﬁers.
3. F EATURE A SYMMETRY AND G RADIENT D ESCENT
When the learning rule does not produce a Bayes-optimal predictor, it may be the case that excess bias can safely be removed without harming accuracy.
To support this claim, we turn our attention to logistic regression classiﬁers trained using stochastic gradient descent.
Logistic regression predictors for data generated according to Equation 3 converge in the limit to the same Bayes-optimal predictors studied in Proposition 1 and Corollary 1 (Murphy, 2012).
Logistic regression models make fewer assumptions about the data and are therefore more widelyapplicable, but as we demonstrate in this section, this ﬂexibility comes at the expense of an inductive bias that can lead to systematic bias in predictions.
To show this, we continue under our assumption that x and y are generated according to Equation 3, and consider the case where p ∗ = 1 / 2 .
According to Corollary 1, any systematic bias that emerges must come from differences between the trained classiﬁer h S and the Bayes-optimal h ∗ .
3.2. F EATURE A SYMMETRY
To deﬁne what is meant by “feature asymmetry”, consider the orientation of each feature x j as given by the sign of µ 1 j − µ 0 j .
The sign of each coefﬁcient in h ∗ will correspond to its feature orientation, so we can think of each feature as being “towards” either class 0 or class 1.
Likewise, we can view the combined features as being asymmetric towards y when there are more features oriented towards y than towards 1 − y .
As shown in Table 1, high-dimensional data with biased class priors often exhibit feature asymmetry towards the majority class.
This does not necessarily lead to excessive bias, and the analysis from the previous section indicates that if p ∗ = 1 / 2 then it may be possible to learn a predictor with no bias.
However, if the learning rule overestimates the importance of some of the features oriented towards the majority class, then variance in those features present in minority instances will cause mispredictions that lead to excess bias beyond what is characterized in Proposition 1.
This problem is pronounced when many of the majority-oriented features are weak predictors, which in this setting means that the magnitude of their corresponding coefﬁcients in h ∗ are small relative to the other features (for example, features with high variance or similar means between classes).
The weak features have small coefﬁcients in h ∗ , but if the learner systematically overestimates the corresponding coefﬁcients in h S , the resulting classiﬁer will be “out of balance” with the distribution generating the data.
Figure 2 explores this phenomenon through synthetic Gaussian data exemplifying this feature asymmetry, in which the strongly-predictive features have low variance σ s = 1 , and the weakly-predictive features have relatively higher variance σ w > 1 . Speciﬁcally, the data used here follows Equationwith the parameters shown in Equation 4.
[Equation] Figure 2c suggests that overestimation of weak features is precisely the form of inductive bias exhibited by gradient descent when learning logistic classiﬁers.
As h S converges to the Bayes-optimal conﬁguration, the magnitude of weak-feature coefﬁcients gradually decreases to the appropriate quantity.
As the variance increases, the extent of the overapproximation grows accordingly.
While this effect may arise when methods other than SGD are used to estimate the coefﬁcients, Figure 3 in the appendix shows that it occurs consistently in models trained using SGD.
3.2. P REDICTION B IAS FROM I NDUCTIVE B IAS
While the classiﬁer remains far from convergence, the cumulative effect of feature overapproximation with high-dimensional data leads to systematic bias.
Figure 2a demonstrates that as the disparity in weak features towards class y = 1 increases, so does the expected bias towards that class.
This bias cannot be explained by Proposition 1, because this data is distributed with p ∗ = 1 / 2 .
Rather, it is clear that the effect diminishes as the training size increases and h S converges towards h ∗ .
This suggests gradient descent tends to “overuse” the weak features prior to convergence, leading to systematic bias that over-predicts the majority class in asymmetric regimes.
Figure 2b demonstrates that for a ﬁxed disparity in weak features, the features must be sufﬁciently weak in order to cause bias.
This suggests that a feature imbalance alone is not sufﬁcient for causing systematic bias.
Moreover, the weak features, rather than the strong features, are responsible for the bias.
As the training size increases, the amount of variance required to cause bias increases.
However, when the features have sufﬁciently high variance, the model will eventually decrease their contribution, relieving their impact on the bias and accuracy of the model.
Summary : When the data is distributed asymmetrically with respect to features’ orientation towards a class, gradient descent may lead to systematic bias especially when many of the asymmetric features are weak predictors.
This bias is a result of the learning rule, as it manifests in cases where a Bayes-optimal predictor would exhibit no bias, and therefore it may be possible to mitigate it without harming accuracy.
M ITIGATING F EATURE -W ISE B IAS A MPLIFICATION
While Theorem 1 suggests that some bias is unavoidable, the empirical analysis in the previous section shows that some systematic bias may not be. Our analysis also suggests an approach for removing such bias, namely by identifying and removing the weak features that are systematically overestimated by gradient descent. In this section, we describe two approaches for accomplishing this that are based on measuring the inﬂuence (Leino et al., 2018) of features on trained models. In
Section 5, we show that these methods are effective at mitigating bias without harming accuracy on both logistic predictors and deep networks. [Figure] [Figure] [Figure]
P r [ h ( x ) = 1 ]
N [Figure] N
[Figure]
N [Figure] (a) 1 2 3 4 5 6 7 8 [-9 0 .0 .0 .0 .
-]{+
 [Figure]+}
.σ
w ( [Figure] feats.)
P r [ h ( x ) = 1 ]
N[-= 100 N = N =(b)    1 , 
 N (datasize) h S − h ∗ (-]
[Figure]
N
[Figure]
(b)
[Figure] [Figure] [Figure] N (datasize)
[Figure]
(c)
4. I NFLUENCE -D IRECTED F EATURE R EMOVAL
Given a model h : X 0 → R and feature x j the inﬂuence χ j of x j on h is a quantitative measure of feature j ’s contribution to the output of h .
To extend this notion to internal layers of a deep network h we consider the slice abstraction (Leino et al., 2018) comprised of a pair of functions f : X {+
+}
→ X and g : X → R , such that h =
g ◦ f .
We deﬁne f to be the network up to the penultimate layer, and g be the ﬁnal layer.
Intuitively, We can then think of the features as being precomputed by f , i.e., x = f ( x 0 ) for x 0 ∈ X 0 , allowing us to treat the ﬁnal layer as a linear model acting on features computed via a deep network.
Note that the slice abstraction encompasses linear models as well, by deﬁning f to be the identity function.
A growing body of work on inﬂuence measures (Simonyan, Vedaldi, and Zisserman, 2013; Sundararajan, Taly, and Yan, 2017; Leino et al., 2018) provides numerous choices for χ j , each with different tradeoffs.
We use the internal distributional inﬂuence (Leino et al., 2018), as it incorporates the slice abstraction naturally.
This measure is given by Equation 5 for a distribution of interest
P , which characterizes the distribution of test instances.
[Equation]
We now describe two techniques that use this measure to remove features causing bias.
Feature parity.
Motivated by the fact that bias ampliﬁcation may be caused by feature asymmetry, we can attempt to mitigate it by enforcing parity in features across the classes.
To avoid removing features that are useful for correct predictions, we order the features by their inﬂuence on the model’s output, and remove features from the majority class until parity is reached.
If the model has a bias term, we adjust it by subtracting the product of each removed coefﬁcient and the mean of its corresponding feature.
Experts Section 3.2 identiﬁes “weak” features as a likely source of systematic bias.
This is a somewhat artiﬁcial construct, as real data often does not exhibit a clear separation between strong and weak features.
Qualitatively, the weak features are less predictive than the strong features, and the learner accounts for this by giving less inﬂuence to the weak features.
Thus, we can think of imposing a strong/weak feature dichotomy by deﬁning the weak features to be those such that | χ j | < χ ∗ for some threshold χ ∗ .
This reduces the feature selection problem to a search for an appropriate χ ∗ that mitigates bias to the greatest extent without harming accuracy.
We parameterize this search problem in terms of α, β , where the α features with the most positive inﬂuence and β features with the most negative inﬂuence are “strong”, and the rest are considered weak.
This amounts to selecting the class-wise expert (Leino et al., 2018) for the dominant class. [Table]
Formally, let F α be the set of α features with the α highest positive inﬂuences, and F β the set of β features with the β most negative inﬂuences.
For slice h = g ◦ f , let g αβ be deﬁned as model g with its weights replaced by w αβ as deﬁned by Equation 6.
Then we deﬁne expert , g α ∗ β ∗ , to be the classiﬁer given by setting α ∗ and β ∗ according to Equation 7.
In other words, the α and β that minimize bias while maintaining at least the original model’s accuracy.
Here L S represents the 0-1 loss on the training set, S .
[Equation]
[Equation]
[Equation]
We note that this is always feasible by selecting all the features.
Furthermore, this is a discrete optimization problem, which can be solved efﬁciently with a grid search over the possible α and β .
In practice, even when there are many features, we can exhaustively search this space.
When there are ties we can break them by preferring the model with the greatest accuracy.
E XPERIMENTS
In this section we present empirical evidence to support our claim that feature-wise bias ampliﬁcation can safely be removed without harming the accuracy of the classiﬁer.
We show this on both logistic predictors and deep networks by measuring the bias on several benchmark datasets, and running the parity and expert mitigation approaches described in Section 4.
As a baseline, we compare against (cid:96) 1 regularization in the logistic classiﬁer experiments.
The results are shown in Table 1.
To summarize, on every dataset we consider, at least one of the methods in Section 4 proves effective at reducing the classiﬁer’s bias ampliﬁcation.
(cid:96) 1 regularization removes bias less reliably, and never to the extent that our methods do.
In all but two cases, the inﬂuence-directed experts show the best performance in terms of bias removal, and this method is able to reduce bias in all but one case.
In terms of accuracy, our methods consistently improve classiﬁer performance, and in some cases signiﬁcantly.
For example, on the prostate dataset, inﬂuence-directed experts removed 80% of the prediction bias while improving accuracy from 57.7% to 90.2%.
Data.
We performed experiments over eight binary classiﬁcation datasets from various domains (rows 3-11 in Table 1) and two image classiﬁcation datasets (CIFAR10-binary, CelebA).
Our criteria for selecting logistic regression datasets were: high feature dimensionality, binary labels, and rowstructured instances (i.e., not time series data).
Among the logistic regression datasets, arcene , colon , glioma , pc/mac , prostate , smokers were obtained from the scikit-feature repository (Li et al., 2016), and micromass was obtained from the UCI repository (Dheeru and Karra Taniskidou, 2017).
The synthetic dataset was generated in the manner described in Section 3.2, containing one strongly-predictive feature ( σ 2 = 1 ) for each class, 1,000 weak features ( σ 2 = 3 ), and p ∗ = 1 / 2 .
For the deep network experiments, we created a binary classiﬁcation problem from CIFAR10 (Krizhevsky and Hinton, 2009) from the “bird” and “frog” classes.
We selected these classes as they showed the greatest posterior disparity on VGG16 network trained on the original dataset.
For CelebA, we trained a VGG16 network with one fully-connected layer of 4096 units to predict the attractiveness label given in the training data.
Methodology.
For the logistic regression experiments, we used scikit-learn’s SGDClassiﬁer estimator to train each model using the logistic loss function.
Logistic regression measurements were obtained by averaging over 20 pseudorandom training runs on a randomly-selected stratiﬁed train/test split.
Experiments involving experts selected α, β using grid search over the possible values that minimize bias subject to not harming accuracy as described in Section 4. Similarly, experiments involving (cid:96) 1 regularization use a grid search to select the regularization paramter, optimizing for the same criteria used to select α, β .
Experiments on deep networks use the training/test split provided by the respective dataset authors.
Models were trained until convergence using Keras 2 with the Theano backend.
Logistic regression.
Table 1 shows that on linear models, feature parity always improves or maintains the model in terms of both bias ampliﬁcation and accuracy.
Notably, in each case where feature parity removes bias, the accuracy is likewise improved, supporting our claim that bias resulting from asymmetric feature regimes is avoidable.
In most cases, the beneﬁt from applying feature parity is, however, rather small.
arcene is the exception, which is likely due to the fact that it has large feature asymmetry in the original model, leaving ample opportunity for improvement by this approach.
The results suggest that inﬂuence-directed experts are the most effective mitigation technique, both in terms of bias removal and accuracy improvement.
In most datasets, this approach reduced bias while improving accuracy, often substantially.
Most notably on the prostate dataset, where the original model failed to achieve accuracy appreciably greater than chance and extreme bias.
The mitigation achieves 90% accuracy while removing 80% of the bias, improving the model signiﬁcantly.
Similarly, for arcene and smokers , this approach removed over 50% of the prediction bias while improving accuracy 5-11%.
(cid:96) 1 regularization proved least reliable at removing bias subject to not harming accuracy.
In many cases, it was unable to remove much bias ( glioma, micromass, PC/Mac ). On synthetic data (cid:96)gave the best bias reduction. Though it did perform admirably on several real datasets ( arcene, prostate, smokers ), even removing up to 40% of the bias on the prostate dataset, it was consistently outperformed by either the parity or expert method.
Additionally, on the colon dataset, it made bias signiﬁcantly worse (150%) for gains in accuracy.
Deep networks.
The results show that deep networks tend to have a less signiﬁcant feature asymmetry than data used for logistic models, which we would expect to render the feature parity approach less effective.
The results conﬁrm this, although on CIFAR10 parity had some effect on bias and a proportional positive effect on accuracy.
Inﬂuence-directed experts, on the other hand, continued to perform well for the deep models.
While this approach generally had a greater effect on accuracy than bias for the linear models, this trend reversed for deep networks, where the decrease in bias was consistently greater than the increase in accuracy.
For example, the 7.7% bias in the original CelebA model was reduced by approximately 98% to 0.2%, effectively eliminating it from the model’s predictions.
The overall effect on accuracy remained modest (0.3% improvement).
These results on deep networks are somewhat surprising, considering that the techniques described in Section 4 were motivated by observations concerning simple linear classiﬁers.
While the improvements in accuracy are not as signiﬁcant as those seen on linear classiﬁers, they align with our expectations regarding bias reduction.
This suggests that future work might improve on these results by adapting the approach described in this paper to better suit deep networks.
A P ROOFS
Proposition 1 Let x be distributed according to Equation 3, y be Bernoulli with parameter p ∗ , D be the Mahalanobis distance between the class means µ ∗ 0 , µ ∗ 1 , and β = − D − 1 log( p ∗ / (1 − p ∗ )) .
Then the bias ampliﬁcation of the Bayes-optimal classiﬁer h ∗ is:
[Equation]
Proof.
Note that the Bayes-optimal classiﬁer can be expressed as a linear weighted sum (Murphy, 2012) in terms of parameters ˆ w , ˆ b as shown in Equation 8.
[Equation]
The random variable w T X is a univariate Gaussian with variance w T Σw and mean w T µ y when
Y =
y .
Then the quantity we are interested in is shown in Equation 9, where Φ is the CDF of the standard normal distribution.
[Equation]
Notice that the quantity
[Equation]
is the square of the Mahalanobis distance between the class means.
[Equation]
Similarly
[Equation]
we can rewrite the standard deviation of w T X exactly as D .
Rewriting the numerator in the Φ term of (9),
[Equation]
Then we can write Pr (cid:2) w T X > − b (cid:3) as:
[Equation]
[Figure] Corollary 1 When x is distributed according to Equation 3 and p ∗ = 1 / 2 , B D ( h ∗ ) = 0 .
Proof.
Note that because p ∗ = 1 / 2 , the term β = 0 in Theorem 1.
Using the main result of the theorem, we have:
[Equation]
The third equality holds because Φ has rotational symmetry about (0 , 1 / 2) , giving the identity Φ( − x ) = 1 − Φ( x ) .
[Equation]