Understanding and Mitigating Accuracy Disparity in Regression

1 INTRODUCTION . Recent progress in machine learning has led to its widespread use in many high-stakes domains , such as criminal justice , healthcare , student loan approval , and hiring . Meanwhile , it has also been widely observed that accuracy disparity could occur inadvertently under various scenarios in practice ( Barocas & Selbst , 2016 ) . For example , errors are inclined to occur for individuals of certain underrepresented demographic groups ( Kim , 2016 ) . In other cases , Buolamwini & Gebru ( 2018 ) showed that notable accuracy disparity gaps exist across different racial and gender demographic subgroups on several real-world image classification systems . Moreover , Bagdasaryan et al . ( 2019 ) found out that a differentially private model even enlarges such accuracy disparity gaps . Such accuracy disparity gaps across demographic subgroups not only raise concerns in high-stake applications but also can be utilized by malicious parties causing information leakage ( Yaghini et al. , 2019 ) . Despite the ample needs of accuracy parity , most prior work limits its scope to studying the problem in binary classification settings ( Hardt et al. , 2016 ; Zafar et al. , 2017b ; Zhao et al. , 2019 ; Jiang et al. , 2019 ) . In a seminal work , Chen et al . ( 2018 ) analyzed the impact of data collection on accuracy disparity in general learning models . They provided a descriptive analysis of such parity gaps and advocated for collecting more training examples and introducing more predictive variables . While such a suggestion is feasible in applications where data collection and labeling is cheap , it is not applicable in domains where it is time-consuming , expensive , or even infeasible to collect more data , e.g. , in autonomous driving , education , etc . Our Contributions In this paper , we provide a prescriptive analysis of accuracy disparity and aim at providing algorithmic interventions to reduce the disparity gap between different demographic subgroups in the regression setting . To start with , we first formally characterize why accuracy disparity appears in regression problems by depicting the feasible region of the underlying group-wise errors . We also provide a lower bound on the joint error and a complementary upper bound on the error gap across groups . Based on these results , we illustrate why regression models aiming to minimize the global loss will inevitably lead to accuracy disparity if the input distributions or decision functions differ across groups ( see Figure 1a ) . We further propose an error decomposition theorem that decomposes the accuracy disparity into the distance between the label populations and the distance between conditional representations . To mitigate such disparities , we propose two algorithms to reduce accuracy disparity via joint distribution alignment with total variation distance and Wasserstein distance , respectively . Furthermore , we analyze the game-theoretic optima of the objective function and illustrate the principle of our algorithms from a game-theoretic perspective ( see Figure 1b ) . To corroborate the effectiveness of our proposed algorithms in reducing accuracy disparity , we conduct experiments on four real-world datasets . Experimental results suggest that our proposed algorithms help to mitigate accuracy disparity while maintaining the predictive power of the regression models . We believe our theoretical results contribute to the understanding of why accuracy disparity occurs in machine learning models , and the proposed algorithms provides an alternative for intervention in real-world scenarios where accuracy parity is desired but collecting more data/features is time-consuming or infeasible . 2 PRELIMINARIES . Notation We use X ⊆ Rd and Y ⊆ R to denote the input and output space . We use X and Y to denote random variables which take values in X and Y , respectively . Lower case letters x and y denote the instantiation ofX and Y . We useH ( X ) to denote the Shannon entropy of random variable X , H ( X | Y ) to denote the conditional entropy of X given Y , and I ( X ; Y ) to denote the mutual information between X and Y . To simplify the presentation , we use A ∈ { 0 , 1 } as the sensitive attribute , e.g. , gender , race , etc . LetH be the hypothesis class of regression models . In other words , for h ∈ H , h : X → Y is a predictor . Note that even if the predictor does not explicitly take the sensitive attribute A as an input variable , the prediction can still be biased due to the correlations with other input variables . In this work we study the stochastic setting where there is a joint distribution D over X , Y and A from which the data are sampled . For a ∈ { 0 , 1 } and y ∈ R , we use Da to denote the conditional distribution of D given A = a and Dy to denote the conditional distribution of D given Y = y . For an event E , D ( E ) denotes the probability of E under D. Given a feature transformation function g : X → Z that maps instances from the input space X to feature space Z , we define g ] D : = D ◦ g−1 to be the induced ( pushforward ) distribution of D under g , i.e. , for any event E′ ⊆ Z , g ] D ( E′ ) : = D ( { x ∈ X | g ( x ) ∈ E′ } ) . We use ( · ) + to indicate the value of a variable remains unchanged if it is positive or otherwise 0 , i.e. , ( Y ) + equals to Y if the value of Y is positive or otherwise 0 . Given a joint distribution D , the error of a predictor h under D is defined as ErrD ( h ) : = ED [ ( Y − h ( X ) ) 2 ] . To make the notation more compact , we may drop the subscript D when it is clear from the context . Furthermore , we also use MSED ( Ŷ , Y ) to denote the mean squared loss between the predicted variable Ŷ = h ( X ) and the true label Y over the joint distribution D. Similarly , we also use CED ( A ‖ Â ) denote the cross-entropy loss between the predicted variable Â and the true label A over the joint distribution D. Throughout the paper , we make the following standard assumption in regression problems : Assumption 2.1 . There exists M > 0 , such that for any hypothesis H 3 h : X → Y , ‖h‖∞ ≤M and |Y | ≤M . Problem Setup We study the fair regression problem : the goal is to learn a regressor that is fair in the sense that the errors of the regressor are approximately equal across the groups given by the sensitive attribute A . We assume that the sensitive attribute A is only available to the learner during the training phase and is not visible during the inference phase . We would like to point out that there are many other different and important definitions of fairness ( Narayanan , 2018 ) even in the sub-category of group fairness , and our discussion is by no means comprehensive . For example , two frequently used definitions of fairness in the literature are the so-called statistical parity ( Dwork et al. , 2012 ) and equalized odds ( Hardt et al. , 2016 ) . Nevertheless , throughout this paper we mainly focus accuracy parity as our fairness notion , due to the fact that machine learning systems have been shown to exhibit substantial accuracy disparities between different demographic subgroups ( Barocas & Selbst , 2016 ; Kim , 2016 ; Buolamwini & Gebru , 2018 ) . This observation has already brought huge public attention ( e.g. , see New York Times , The Verge , and Insurance Journal ) and calls for machine learning systems that ( at least approximately ) satisfy accuracy parity . Formally , accuracy parity is defined as follows : Definition 2.1 ( Accuracy Parity ) . Given a joint distribution D , a predictor h satisfies accuracy parity if ErrD0 ( h ) = ErrD1 ( h ) . The violation of accuracy parity is also known as disparate mistreatment ( Zafar et al. , 2017a ) . In practice the exact equality of on accuracy between two groups is often hard to ensure , so we define error gap to measure how well the model satisfies accuracy parity : Definition 2.2 ( Error Gap ) . Given a joint distributionD , the error gap of a hypothesis h is ∆Err ( h ) : = |ErrD0 ( h ) − ErrD1 ( h ) | . By definition , if a model satisfies accuracy parity , ∆Err ( h ) will be zero . Next we introduce two distance metrics that will be used in our theoretical analysis and algorithm design : • Total variation distance : it measures the largest possible difference between the probabilities that the two probability distributions can assign to the same event E. We use dTV ( P , Q ) to denote the total variation : dTV ( P , Q ) : = sup E |P ( E ) −Q ( E ) | . • Wasserstein distance : the Wasserstein distance between two probability distributions is W1 ( P , Q ) = sup f∈ { f : ‖f‖L≤1 } ∣∣∣∣∫ Ω fdP − ∫ Ω fdQ ∣∣∣∣ , where ‖f‖L is the Lipschitz semi-norm of a real-valued function of f and Ω is the sample space over which two probability distributions P and Q are defined . By the Kantorovich-Rubinstein duality theorem ( Villani , 2008 ) , we recover the primal form of the Wasserstein distance , defined as W1 ( P , Q ) : = inf γ∈Γ ( P , Q ) ∫ d ( X , Y ) dγ ( X , Y ) , where Γ ( P , Q ) denotes the collection of all couplings of P andQ , andX and Y denote the random variables with law P and Q respectively . Note that we use L1 distance for d ( · , · ) throughout the paper , but the extensions to other distance , e.g. , L2 distance , is straightforward . 3 MAIN RESULTS . In this section , we first characterize why accuracy disparity arises in regression models . More specifically , given a hypothesis h ∈ H , we first describe the feasible region of ErrD0 and ErrD1 by proving a lower bound of joint errors and an upper bound of the error gap . Then , we give a geometric interpretation to visualize the feasible region of ErrD0 and ErrD1 and illustrate how error gap arises when learning a hypothesis h that minimizes the global squared error . We further analyze the accuracy disparity by decomposing it into the distance between label populations and the distance between conditional representations . Motivated by the decomposition , we propose two algorithms to reduce accuracy disparity , connect the game-theoretic optima of the objective functions in our algorithms with our theorems , and describe the practical implementations of the algorithms . Due to the space limit , we defer all the detailed proofs to the appendix . 3.1 BOUNDS ON CONDITIONAL ERRORS AND ACCURACY DISPARITY GAP . When we learn a predictor , the prediction function induces X h−→ Ŷ , where Ŷ is the predicted target variable given by hypothesis h. Hence for any distribution D0 ( D1 ) of X , the predictor also induces a distribution h ] D0 ( h ] D1 ) of Ŷ . Recall that the Wasserstein distance is metric , hence the following chain of triangle inequalities holds : W1 ( D0 ( Y ) , D1 ( Y ) ) ≤W1 ( D0 ( Y ) , h ] D0 ) +W1 ( h ] D0 , h ] D1 ) +W1 ( h ] D1 , D1 ( Y ) ) Intuitively , W1 ( D0 ( Y ) , h ] D0 ) and W1 ( h ] D1 , D1 ( Y ) ) measure the distance between the true label distribution and the predicted one on A = 0/1 cases , respectively . This distance is related to the prediction error of function h conditioned on A = a : Lemma 3.1 . Let Ŷ = h ( X ) ∈ R , then for a ∈ { 0 , 1 } , W1 ( Da ( Y ) , h ] Da ) ≤ √ ErrDa ( h ) . With the above results , we can get the following theorem that characterizes the lower bound of joint error on different groups : Theorem 3.1 . Let Ŷ = h ( X ) ∈ R , we have ErrD0 ( h ) + ErrD1 ( h ) ≥ 12 [ ( W1 ( D0 ( Y ) , D1 ( Y ) ) − W1 ( h ] D0 , h ] D1 ) ) + ] 2 . In Theorem 3.1 , we see that if the difference between the label distribution across groups is large , then statistical disparity could potentially lead to a large joint error . Moreover , Theorem 3.1 could be extended to give a lower bound on the joint error incurred by h as well : Corollary 3.1 . Let Ŷ = h ( X ) ∈ R and α = D ( A = 0 ) ∈ [ 0 , 1 ] , we have ErrD ( h ) ≥ 12 min { α , 1− α } · [ ( W1 ( D0 ( Y ) , D1 ( Y ) ) −W1 ( h ] D0 , h ] D1 ) ) + ] 2 . Next , we upper bound the error gap to gain more insights on accuracy disparity . For a ∈ { 0 , 1 } , define the conditional variance VarDa [ Y |X ] = EDa [ ( Y − EDa [ Y |X ] ) 2|X ] and it shows up as the irreducible error of predicting Y when we only use the knowledge of X . We also know that the optimal decision function conditioned on A = a under mean squared error to be EDa [ Y |X ] . The following theorem characterizes the upper bound of the error gap between two groups : Theorem 3.2 . For any hypothesisH 3 h : X → Y , if the Assumption 2.1 holds , then : ∆Err ( h ) ≤ 8M2 dTV ( D0 ( X ) , D1 ( X ) ) + |ED0 [ VarD0 [ Y |X ] ] − ED1 [ VarD1 [ Y |X ] ] | + 4M min { ED0 [ |ED0 ( Y |X ) − ED1 ( Y |X ) | ] , ED1 [ |ED0 ( Y |X ) − ED1 ( Y |X ) | ] } . Remark Theorem 3.2 upper bounds the error gap across groups by three terms : the first term corresponds to the distance of input distribution across groups , the second term is the noise ( variance ) difference , and third term is the discrepancy of optimal decision functions across different groups . In an ideal and fair setting , where both distributions are noiseless , and the optimal decision functions are insensitive to the group membership , then Theorem 3.2 implies a sufficient condition to guarantee accuracy parity is to find group-invariant representation that minimize dTV ( D0 ( X ) , D1 ( X ) ) . Geometric Interpretation By Theorem 3.1 and Theorem 3.2 , in Figure 1a , we visually illustrate how accuracy disparity arises given data distribution and the learned hypothesis that aims to minimize the global squared error . In Figure 1a , given the hypothesis classH , we use the line ErrD0 + ErrD1 = B to denote the lower bound in Theorem 3.1 and the two lines |ErrD0 − ErrD1 | = A to denote the upper bound in Theorem 3.2 . These three lines form a feasible region ( the green area ) of ErrD0 and ErrD1 under the hypothesis classH . For any optimal hypothesis h which is solely designed to minimize the overall error , the best the hypothesis h can do is to intersect with one of the two bottom vertices . For example , the hypotheses ( the red dotted line and the blue dotted line ) trying to minimize overall error intersect with the two vertices of the region to achieve the smallest ErrD0-intercept ( ErrD1 -intercept ) , due to the imbalance between these two groups . However , since these two vertices are not on the diagonal of the feasible region , there is no guarantee that the hypothesis can satisfy accuracy parity ( ErrD0 = ErrD1 ) , unless we can shrink the width of green area to zero . Conditional Distribution Alignment Reduces Accuracy Parity In Theorem 3.2 , we illustrate how accuracy disparity arises in regression models due to noise , distance between representations , and distance between decision functions . However , it is nearly impossible to collect noiseless data with group-invariant input distribution . Moreover , there is no guarantee that the upper bound will be lower if we learn the group-invariant representation that minimizes dTV ( D0 ( X ) , D1 ( X ) ) alone , since the learned representation could potentially increase the variance . In this regard , we prove a novel upper bound which is free from the above noise term to motivate aligning conditional distributions to mitigate the error disparity across groups . To do so , we relate the error gap to the label distribution and the predicted distribution condition on Y = y : Theorem 3.3 . If Assumption 2.1 holds , then for ∀h ∈ H , let Ŷ = h ( X ) , the following inequality holds : ∆Err ( h ) ≤ 8M2dTV ( D0 ( Y ) , D1 ( Y ) ) + 3M min { ED0 [ |EDy0 [ Ŷ ] − EDy1 [ Ŷ ] | ] , ED1 [ |EDy0 [ Ŷ ] − EDy1 [ Ŷ ] | ] } . Remark We see that the error gap is upper bounded by two terms : the distance between label distributions and the discrepancy between conditional predicted distributions across groups . Note that this is different from the decomposition we have in Theorem 3.2 , where the marginal distribution is on X instead of Y . Given a dataset , the distance of label distributions is a constant since the label distribution is fixed . For the second term , if we can minimize the discrepancy of the conditional predicted distribution across groups , we then have a model that is free of accuracy disparity when the label distribution is well aligned .

Summary: This paper theoretically and empirically studies accuracy disparity in regression problems. It proves an information-theoretic lower bound on the joint error and a complementary upper bound on the error gap across groups to depict the feasible region of group-wise errors. It further proposes to achieve accuracy parity theoretically and empirically by learning conditional group-invariant representations using statistical distances.