Understanding and Mitigating Accuracy Disparity in Regression

1 INTRODUCTION . Recent progress in machine learning has led to its widespread use in many high-stakes domains , such as criminal justice , healthcare , student loan approval , and hiring . Meanwhile , it has also been widely observed that accuracy disparity could occur inadvertently under various scenarios in practice ( Barocas & Selbst , 2016 ) . For example , errors are inclined to occur for individuals of certain underrepresented demographic groups ( Kim , 2016 ) . In other cases , Buolamwini & Gebru ( 2018 ) showed that notable accuracy disparity gaps exist across different racial and gender demographic subgroups on several real-world image classification systems . Moreover , Bagdasaryan et al . ( 2019 ) found out that a differentially private model even enlarges such accuracy disparity gaps . Such accuracy disparity gaps across demographic subgroups not only raise concerns in high-stake applications but also can be utilized by malicious parties causing information leakage ( Yaghini et al. , 2019 ) . Despite the ample needs of accuracy parity , most prior work limits its scope to studying the problem in binary classification settings ( Hardt et al. , 2016 ; Zafar et al. , 2017b ; Zhao et al. , 2019 ; Jiang et al. , 2019 ) . In a seminal work , Chen et al . ( 2018 ) analyzed the impact of data collection on accuracy disparity in general learning models . They provided a descriptive analysis of such parity gaps and advocated for collecting more training examples and introducing more predictive variables . While such a suggestion is feasible in applications where data collection and labeling is cheap , it is not applicable in domains where it is time-consuming , expensive , or even infeasible to collect more data , e.g. , in autonomous driving , education , etc . Our Contributions In this paper , we provide a prescriptive analysis of accuracy disparity and aim at providing algorithmic interventions to reduce the disparity gap between different demographic subgroups in the regression setting . To start with , we first formally characterize why accuracy disparity appears in regression problems by depicting the feasible region of the underlying group-wise errors . We also provide a lower bound on the joint error and a complementary upper bound on the error gap across groups . Based on these results , we illustrate why regression models aiming to minimize the global loss will inevitably lead to accuracy disparity if the input distributions or decision functions differ across groups ( see Figure 1a ) . We further propose an error decomposition theorem that decomposes the accuracy disparity into the distance between the label populations and the distance between conditional representations . To mitigate such disparities , we propose two algorithms to reduce accuracy disparity via joint distribution alignment with total variation distance and Wasserstein distance , respectively . Furthermore , we analyze the game-theoretic optima of the objective function and illustrate the principle of our algorithms from a game-theoretic perspective ( see Figure 1b ) . To corroborate the effectiveness of our proposed algorithms in reducing accuracy disparity , we conduct experiments on four real-world datasets . Experimental results suggest that our proposed algorithms help to mitigate accuracy disparity while maintaining the predictive power of the regression models . We believe our theoretical results contribute to the understanding of why accuracy disparity occurs in machine learning models , and the proposed algorithms provides an alternative for intervention in real-world scenarios where accuracy parity is desired but collecting more data/features is time-consuming or infeasible . 2 PRELIMINARIES . Notation We use X ⊆ Rd and Y ⊆ R to denote the input and output space . We use X and Y to denote random variables which take values in X and Y , respectively . Lower case letters x and y denote the instantiation ofX and Y . We useH ( X ) to denote the Shannon entropy of random variable X , H ( X | Y ) to denote the conditional entropy of X given Y , and I ( X ; Y ) to denote the mutual information between X and Y . To simplify the presentation , we use A ∈ { 0 , 1 } as the sensitive attribute , e.g. , gender , race , etc . LetH be the hypothesis class of regression models . In other words , for h ∈ H , h : X → Y is a predictor . Note that even if the predictor does not explicitly take the sensitive attribute A as an input variable , the prediction can still be biased due to the correlations with other input variables . In this work we study the stochastic setting where there is a joint distribution D over X , Y and A from which the data are sampled . For a ∈ { 0 , 1 } and y ∈ R , we use Da to denote the conditional distribution of D given A = a and Dy to denote the conditional distribution of D given Y = y . For an event E , D ( E ) denotes the probability of E under D. Given a feature transformation function g : X → Z that maps instances from the input space X to feature space Z , we define g ] D : = D ◦ g−1 to be the induced ( pushforward ) distribution of D under g , i.e. , for any event E′ ⊆ Z , g ] D ( E′ ) : = D ( { x ∈ X | g ( x ) ∈ E′ } ) . We use ( · ) + to indicate the value of a variable remains unchanged if it is positive or otherwise 0 , i.e. , ( Y ) + equals to Y if the value of Y is positive or otherwise 0 . Given a joint distribution D , the error of a predictor h under D is defined as ErrD ( h ) : = ED [ ( Y − h ( X ) ) 2 ] . To make the notation more compact , we may drop the subscript D when it is clear from the context . Furthermore , we also use MSED ( Ŷ , Y ) to denote the mean squared loss between the predicted variable Ŷ = h ( X ) and the true label Y over the joint distribution D. Similarly , we also use CED ( A ‖ Â ) denote the cross-entropy loss between the predicted variable Â and the true label A over the joint distribution D. Throughout the paper , we make the following standard assumption in regression problems : Assumption 2.1 . There exists M > 0 , such that for any hypothesis H 3 h : X → Y , ‖h‖∞ ≤M and |Y | ≤M . Problem Setup We study the fair regression problem : the goal is to learn a regressor that is fair in the sense that the errors of the regressor are approximately equal across the groups given by the sensitive attribute A . We assume that the sensitive attribute A is only available to the learner during the training phase and is not visible during the inference phase . We would like to point out that there are many other different and important definitions of fairness ( Narayanan , 2018 ) even in the sub-category of group fairness , and our discussion is by no means comprehensive . For example , two frequently used definitions of fairness in the literature are the so-called statistical parity ( Dwork et al. , 2012 ) and equalized odds ( Hardt et al. , 2016 ) . Nevertheless , throughout this paper we mainly focus accuracy parity as our fairness notion , due to the fact that machine learning systems have been shown to exhibit substantial accuracy disparities between different demographic subgroups ( Barocas & Selbst , 2016 ; Kim , 2016 ; Buolamwini & Gebru , 2018 ) . This observation has already brought huge public attention ( e.g. , see New York Times , The Verge , and Insurance Journal ) and calls for machine learning systems that ( at least approximately ) satisfy accuracy parity . Formally , accuracy parity is defined as follows : Definition 2.1 ( Accuracy Parity ) . Given a joint distribution D , a predictor h satisfies accuracy parity if ErrD0 ( h ) = ErrD1 ( h ) . The violation of accuracy parity is also known as disparate mistreatment ( Zafar et al. , 2017a ) . In practice the exact equality of on accuracy between two groups is often hard to ensure , so we define error gap to measure how well the model satisfies accuracy parity : Definition 2.2 ( Error Gap ) . Given a joint distributionD , the error gap of a hypothesis h is ∆Err ( h ) : = |ErrD0 ( h ) − ErrD1 ( h ) | . By definition , if a model satisfies accuracy parity , ∆Err ( h ) will be zero . Next we introduce two distance metrics that will be used in our theoretical analysis and algorithm design : • Total variation distance : it measures the largest possible difference between the probabilities that the two probability distributions can assign to the same event E. We use dTV ( P , Q ) to denote the total variation : dTV ( P , Q ) : = sup E |P ( E ) −Q ( E ) | . • Wasserstein distance : the Wasserstein distance between two probability distributions is W1 ( P , Q ) = sup f∈ { f : ‖f‖L≤1 } ∣∣∣∣∫ Ω fdP − ∫ Ω fdQ ∣∣∣∣ , where ‖f‖L is the Lipschitz semi-norm of a real-valued function of f and Ω is the sample space over which two probability distributions P and Q are defined . By the Kantorovich-Rubinstein duality theorem ( Villani , 2008 ) , we recover the primal form of the Wasserstein distance , defined as W1 ( P , Q ) : = inf γ∈Γ ( P , Q ) ∫ d ( X , Y ) dγ ( X , Y ) , where Γ ( P , Q ) denotes the collection of all couplings of P andQ , andX and Y denote the random variables with law P and Q respectively . Note that we use L1 distance for d ( · , · ) throughout the paper , but the extensions to other distance , e.g. , L2 distance , is straightforward . 3 MAIN RESULTS . In this section , we first characterize why accuracy disparity arises in regression models . More specifically , given a hypothesis h ∈ H , we first describe the feasible region of ErrD0 and ErrD1 by proving a lower bound of joint errors and an upper bound of the error gap . Then , we give a geometric interpretation to visualize the feasible region of ErrD0 and ErrD1 and illustrate how error gap arises when learning a hypothesis h that minimizes the global squared error . We further analyze the accuracy disparity by decomposing it into the distance between label populations and the distance between conditional representations . Motivated by the decomposition , we propose two algorithms to reduce accuracy disparity , connect the game-theoretic optima of the objective functions in our algorithms with our theorems , and describe the practical implementations of the algorithms . Due to the space limit , we defer all the detailed proofs to the appendix . 3.1 BOUNDS ON CONDITIONAL ERRORS AND ACCURACY DISPARITY GAP . When we learn a predictor , the prediction function induces X h−→ Ŷ , where Ŷ is the predicted target variable given by hypothesis h. Hence for any distribution D0 ( D1 ) of X , the predictor also induces a distribution h ] D0 ( h ] D1 ) of Ŷ . Recall that the Wasserstein distance is metric , hence the following chain of triangle inequalities holds : W1 ( D0 ( Y ) , D1 ( Y ) ) ≤W1 ( D0 ( Y ) , h ] D0 ) +W1 ( h ] D0 , h ] D1 ) +W1 ( h ] D1 , D1 ( Y ) ) Intuitively , W1 ( D0 ( Y ) , h ] D0 ) and W1 ( h ] D1 , D1 ( Y ) ) measure the distance between the true label distribution and the predicted one on A = 0/1 cases , respectively . This distance is related to the prediction error of function h conditioned on A = a : Lemma 3.1 . Let Ŷ = h ( X ) ∈ R , then for a ∈ { 0 , 1 } , W1 ( Da ( Y ) , h ] Da ) ≤ √ ErrDa ( h ) . With the above results , we can get the following theorem that characterizes the lower bound of joint error on different groups : Theorem 3.1 . Let Ŷ = h ( X ) ∈ R , we have ErrD0 ( h ) + ErrD1 ( h ) ≥ 12 [ ( W1 ( D0 ( Y ) , D1 ( Y ) ) − W1 ( h ] D0 , h ] D1 ) ) + ] 2 . In Theorem 3.1 , we see that if the difference between the label distribution across groups is large , then statistical disparity could potentially lead to a large joint error . Moreover , Theorem 3.1 could be extended to give a lower bound on the joint error incurred by h as well : Corollary 3.1 . Let Ŷ = h ( X ) ∈ R and α = D ( A = 0 ) ∈ [ 0 , 1 ] , we have ErrD ( h ) ≥ 12 min { α , 1− α } · [ ( W1 ( D0 ( Y ) , D1 ( Y ) ) −W1 ( h ] D0 , h ] D1 ) ) + ] 2 . Next , we upper bound the error gap to gain more insights on accuracy disparity . For a ∈ { 0 , 1 } , define the conditional variance VarDa [ Y |X ] = EDa [ ( Y − EDa [ Y |X ] ) 2|X ] and it shows up as the irreducible error of predicting Y when we only use the knowledge of X . We also know that the optimal decision function conditioned on A = a under mean squared error to be EDa [ Y |X ] . The following theorem characterizes the upper bound of the error gap between two groups : Theorem 3.2 . For any hypothesisH 3 h : X → Y , if the Assumption 2.1 holds , then : ∆Err ( h ) ≤ 8M2 dTV ( D0 ( X ) , D1 ( X ) ) + |ED0 [ VarD0 [ Y |X ] ] − ED1 [ VarD1 [ Y |X ] ] | + 4M min { ED0 [ |ED0 ( Y |X ) − ED1 ( Y |X ) | ] , ED1 [ |ED0 ( Y |X ) − ED1 ( Y |X ) | ] } . Remark Theorem 3.2 upper bounds the error gap across groups by three terms : the first term corresponds to the distance of input distribution across groups , the second term is the noise ( variance ) difference , and third term is the discrepancy of optimal decision functions across different groups . In an ideal and fair setting , where both distributions are noiseless , and the optimal decision functions are insensitive to the group membership , then Theorem 3.2 implies a sufficient condition to guarantee accuracy parity is to find group-invariant representation that minimize dTV ( D0 ( X ) , D1 ( X ) ) . Geometric Interpretation By Theorem 3.1 and Theorem 3.2 , in Figure 1a , we visually illustrate how accuracy disparity arises given data distribution and the learned hypothesis that aims to minimize the global squared error . In Figure 1a , given the hypothesis classH , we use the line ErrD0 + ErrD1 = B to denote the lower bound in Theorem 3.1 and the two lines |ErrD0 − ErrD1 | = A to denote the upper bound in Theorem 3.2 . These three lines form a feasible region ( the green area ) of ErrD0 and ErrD1 under the hypothesis classH . For any optimal hypothesis h which is solely designed to minimize the overall error , the best the hypothesis h can do is to intersect with one of the two bottom vertices . For example , the hypotheses ( the red dotted line and the blue dotted line ) trying to minimize overall error intersect with the two vertices of the region to achieve the smallest ErrD0-intercept ( ErrD1 -intercept ) , due to the imbalance between these two groups . However , since these two vertices are not on the diagonal of the feasible region , there is no guarantee that the hypothesis can satisfy accuracy parity ( ErrD0 = ErrD1 ) , unless we can shrink the width of green area to zero . Conditional Distribution Alignment Reduces Accuracy Parity In Theorem 3.2 , we illustrate how accuracy disparity arises in regression models due to noise , distance between representations , and distance between decision functions . However , it is nearly impossible to collect noiseless data with group-invariant input distribution . Moreover , there is no guarantee that the upper bound will be lower if we learn the group-invariant representation that minimizes dTV ( D0 ( X ) , D1 ( X ) ) alone , since the learned representation could potentially increase the variance . In this regard , we prove a novel upper bound which is free from the above noise term to motivate aligning conditional distributions to mitigate the error disparity across groups . To do so , we relate the error gap to the label distribution and the predicted distribution condition on Y = y : Theorem 3.3 . If Assumption 2.1 holds , then for ∀h ∈ H , let Ŷ = h ( X ) , the following inequality holds : ∆Err ( h ) ≤ 8M2dTV ( D0 ( Y ) , D1 ( Y ) ) + 3M min { ED0 [ |EDy0 [ Ŷ ] − EDy1 [ Ŷ ] | ] , ED1 [ |EDy0 [ Ŷ ] − EDy1 [ Ŷ ] | ] } . Remark We see that the error gap is upper bounded by two terms : the distance between label distributions and the discrepancy between conditional predicted distributions across groups . Note that this is different from the decomposition we have in Theorem 3.2 , where the marginal distribution is on X instead of Y . Given a dataset , the distance of label distributions is a constant since the label distribution is fixed . For the second term , if we can minimize the discrepancy of the conditional predicted distribution across groups , we then have a model that is free of accuracy disparity when the label distribution is well aligned .

Summary: This paper deals with a fair regression problem in which the accuracy disparity is employed as a fairness measure. The authors derived the upper and lower bounds on the difference of accuracy between groups to demonstrate that imbalance in the groups' sizes leads to accuracy disparity. Furthermore, they propose learning algorithms enabling us to mitigate the accuracy disparity, which is accomplished by minimizing the upper bound they derived. The empirical evaluations show that the present methods achieve a better trade-off between accuracy and fairness than some existing fair regression methods.