Equalized Robustness: Towards Sustainable Fairness Under Distributional Shifts

1 INTRODUCTION . With the wide deployment of deep learning in modern business applications concerning individual lives and privacy , there naturally emerge concerns on machine learning fairness ( Podesta et al. , 2014 ; Muñoz et al. , 2016 ; Smuha , 2019 ) . Research efforts on various fairness evaluation metrics and corresponding enforcing methods have been carried out ( Edwards & Storkey , 2016 ; Hardt et al. , 2016 ; Du et al. , 2020 ) . Specifically , many such metrics require some form of “ equalized model performance ” across different groups on in-distribution data . Examples include Demographic parity ( DP ) ( Edwards & Storkey , 2016 ) , Equalized Opportunity ( EOpp ) , and Equalized Odds ( EO ) ( Hardt et al. , 2016 ) . Unfortunately , when deployed for real-world applications , deep models commonly encounter data with unforeseeable distribution shifts ( Hendrycks & Dietterich , 2019 ; Hendrycks et al. , 2020 ; 2021 ) . It has been shown that deep learning models can have drastically degraded performance ( Hendrycks & Dietterich , 2019 ; Hendrycks et al. , 2020 ; 2021 ; Taori et al. , 2020 ) and show unreliable behaviors ( Qiu et al. , 2019 ; Yan et al. , 2021 ) under unseen distribution shifts . Intuitively speaking , previous fairness learning algorithms aim to optimize the model to a local minimum where data from majority and minority groups have similar average loss values ( and thus similar in-distribution performance ) . However , those algorithms do not take into consideration the the stability or “ robustness ” of their found fairness-aware minima . Taking object detection in a self-driving car for example , it might have been calibrated over high-quality clear images to be “ fair ” with different skin colors ; however such fairness may severely break down when applied to data collected in adverse visual conditions , such as inclement weather , poor lighting , or other digital artifacts . Our experiments also find that previous state-of-the-art fairness algorithms would be jeopardized if distributional shifts are present in test data , as illustrated in Figure 1 ( b ) . The above findings beg the following question : How to achieve practically sustainable fairness , e.g. , even under unseen distribution shifts ? To answer that , we first propose a new fairness objective , termed Equalized Robustness ( ER ) , which aims to impose “ equalized robustness ” against unseen distribution shifts across the majority and minority groups , so that the learned fairness can sustain even with test data perturbed . ER explicitly considers a new dimension of fairness that is practically significant yet so far largely overlooked . In other words , ER assesses fairness on “ out-of-distribution ” . Therefore it works as a complement instead of a replacement for previous fairness metrics , which focus on assessing the “ in-distribution ” fairness . Previous research has shown that model robustness against input perturbation is highly correlated with loss curvature smoothness ( Bartlett et al. , 2017 ; Moosavi-Dezfooli et al. , 2019 ; Weng et al. , 2018 ) . Our experiments also observed that , the local loss curvature of minority group is often larger than that of majority group , leading to the two group ’ s robustness discrepancy against distribution shifts . To this end , we propose to empirically quantify the robustness discrepancy as the maximum mean discrepancy ( MMD ) ( Gretton et al. , 2012 ) distance between the local model smoothness distributions , for data samples from the majority and minority groups . We experimentally demonstrate that our new metric aligns well with model performance under real-world distribution shifts . On top of that , we further propose a new fair learning algorithm , termed Curvature Matching ( CUMA ) , to simultaneously achieve both traditional in-distribution fairness and ER . CUMA matches the local curvature distribution between data points from the two different groups , as illustrated in Figure 1 ( c ) , by adding a curvature-matching regularizer that can be efficiently computed via a one-shot power iteration method . Our codes will be released upon acceptance . Our contributions can be summarized as bellow : • We propose Equalized Robustness ( ER ) , a new fairness objective for machine learning models , to impose equalized model robustness against unforeseeable distributions shifts across majority and minority groups . • We further propose a new fairness learning algorithm dubbed Curvature Matching ( CUMA ) , which enforces ER during training by utilizing a one-shot power iteration method . • Experiments show that CUMA achieves much more robust fairness against distribution shifts , without more sacrifice on either overall accuracies or the in-distribution fairness , compared with traditional in-distribution fair learning methods . 2 PRELIMINARIES . 2.1 MACHINE LEARNING FAIRNESS . Problem Setting and Metrics Machine learning fairness can be generally categorized into individual fairness and group fairness ( Du et al. , 2020 ) . Individual fairness requires similar inputs to have similar predictions ( Dwork et al. , 2012 ) . Compared with individual fairness , group fairness is a more popular setting and thus the focus of our paper . Given input data X ∈ Rn with sensitive attributes A ∈ { 0 , 1 } and their corresponding ground truth labels Y ∈ { 0 , 1 } , group fairness requires a learned binary classifier f ( · ; θ ) : Rn → { 0 , 1 } parameterized by θ to give equally accurate predictions ( denoted as Ŷ : = f ( X ) ) on the two groups with A = 0 and A = 1 . Multiple fairness criteria have been defined in this context . Demographic parity ( DP ) ( Edwards & Storkey , 2016 ) requires identical ratio of positive predictions between two groups : P ( Ŷ = 1|A = 0 ) = P ( Ŷ = 1|A = 1 ) . Equalized Odds ( EO ) ( Hardt et al. , 2016 ) requires identical false positive rates ( FPRs ) and false negative rates ( FNRs ) between the two groups : P ( Ŷ 6= Y |A = 0 , Y = y ) = P ( Ŷ 6= Y |A = 1 , Y = y ) , ∀y ∈ { 0 , 1 } . Equalized Opportunity ( EOpp ) ( Hardt et al. , 2016 ) requires only equal FNRs between the groups : P ( Ŷ 6= Y |A = 0 , Y = 0 ) = P ( Ŷ 6= Y |A = 1 , Y = 0 ) . Based on these fairness criteria , quantified metrics are defined to measure fairness . Specifically , DP , EO and EOpp distances ( Madras et al. , 2018 ) are defined as follows : ∆DP : = |P ( Ŷ = 1|A = 0 ) − P ( Ŷ = 1|A = 1 ) | ( 1 ) ∆EO : = ∑ y∈ { 0,1 } |P ( Ŷ 6= Y |A = 0 , Y = y ) − P ( Ŷ 6= Y |A = 1 , Y = y ) | ( 2 ) ∆EOpp : = |P ( Ŷ 6= Y |A = 0 , Y = 0 ) − P ( Ŷ 6= Y |A = 1 , Y = 0 ) | ( 3 ) MMD has been previously used to define fairness metric in ( Quadrianto & Sharmanska , 2017 ) defines a more general fairness metric using MMD distance , and shows ∆DP , ∆EO and ∆EOpp to be spatial cases of their unified metric . All these metrics consider the in-distribution fairness , while our Equalized Generalizibility is the first fairness metric explicitly aware of robust generalization ability on unseen distributions . Bias Mitigation Methods Many methods have been proposed to mitigate model bias . Data preprocessing methods such as re-weighting ( Kamiran & Calders , 2012 ) and data-transformation ( Calmon et al. , 2017 ) have been used to reduce discrimination before model training . In contrast , Hardt et al . ( 2016 ) and Zhao et al . ( 2017 ) propose post-processing methods to calibrate model predictions towards a desired fair distribution after model training . Instead of pre- or post-processing , researchers have explored to enhance fairness during training . For example , Madras et al . ( 2018 ) uses a adversarial training technique and shows the learned fair representations can transfer to unseen target tasks . The key technique , adversarial training ( Edwards & Storkey , 2016 ) , was designed for feature disentanglement on hidden representations such that sensitive ( Edwards & Storkey , 2016 ) or domain-specific information ( Ganin et al. , 2016 ) will be removed while keeping other useful information for the target task . The hidden representations are typically the output of intermediate layers of neural networks ( Ganin et al. , 2016 ; Edwards & Storkey , 2016 ; Madras et al. , 2018 ) . Instead , methods , like adversarial debiasing ( Zhang et al. , 2018 ) and its simplified version ( Wadsworth et al. , 2018 ) , directly apply the adversary on the output layer of the classifier , which also promotes the model fairness . Observing the unfairness due to ignoring the worst learning risk of specific samples , Hashimoto et al . ( 2018 ) proposes to use distributionally robust optimization which provably bounds the worst-case risk over groups . Creager et al . ( 2019 ) proposes a flexible fair representation learning framework based on VAE ( Kingma & Welling , 2013 ) , that can be easily adapted for different sensitive attribute settings during run-time . Sarhan et al . ( 2020 ) uses orthogonality constraints as a proxy for independence to disentangles the utility and sensitive representations . Martinez et al . ( 2020 ) formulates group fairness with multiple sensitive attributes as a multi-objective learning problem and proposes a simple optimization algorithm to find the Pareto optimality . Another line of research focuses on learning unbiased representations from biased ones ( Bahng et al. , 2020 ; Nam et al. , 2020 ) . Bahng et al . ( 2020 ) proposes a novel framework to learn unbiased representations by explicitly enforcing them to be different from a set of pre-defined biased representations . Nam et al . ( 2020 ) observes that data bias can be either benign or malicious , and removing malicious bias along can achieve fairness . Li & Vasconcelos ( 2019 ) jointly learns a data re-sampling weight distribution that penalizes easy samples and network parameters . Applications in Computer Vision When many fairness metrics and debiasing algorithms are designed for general learning problems as aforementioned , there are a line of research and applications focusing on fairness-encouraged computer vision tasks . For instance , Buolamwini et al . ( Buolamwini & Gebru , 2018 ) shows current commercial gender-recognition systems have substantial accuracy disparities among groups with different genders and skin colors . Wilson et al . ( 2019 ) observe that state-of-the-art segmentation models achieve better performance on pedestrians with lighter skin colors . In ( Shankar et al. , 2017 ; de Vries et al. , 2019 ) , it is found that the common geographical bias in public image databases can lead to strong performance disparities among images from locales with different income levels . Nagpal et al . ( 2019 ) reveal that the focus region of face-classification models depends on people ’ s ages or races , which may explain the source of age- and race-biases of classifiers . On the awareness of the unfairness , many efforts have been devoted to mitigate such biases in computer vision tasks . Wang et al . ( 2019 ) shows the effectiveness of adversarial debiasing technique ( Zhang et al. , 2018 ) in fair image classification and activity recognition tasks . Beyond the supervised learning , FairFaceGAN ( Hwang et al. , 2020 ) is proposed to prevent undesired sensitive feature translation during image editing . Similar ideas have also been successfully applied to visual question answering ( Park et al. , 2020 ) .

Summary: The paper is motivated by a common problem in real world applications of deep models, distributional shifts, which can cause unreliable behavior in the deployed models. In particular,  state-of-the-art fairness algorithms would be affected by such distributional shifts in the test data. This poses the following question studied by the paper: how can one achieve fairness when there exist unseen distributional shifts?  Towards this end, the paper proposes a new objective: Equalized Robustness (ER), that imposes equalized model robustness against distributions shifts across majority and minority. Fruthermore, the paper develops a new algorithm called Curvature Matching (CUMA) that imposes ER during training, and tests it through experiments.