Equalized Robustness: Towards Sustainable Fairness Under Distributional Shifts

1 INTRODUCTION . With the wide deployment of deep learning in modern business applications concerning individual lives and privacy , there naturally emerge concerns on machine learning fairness ( Podesta et al. , 2014 ; Muñoz et al. , 2016 ; Smuha , 2019 ) . Research efforts on various fairness evaluation metrics and corresponding enforcing methods have been carried out ( Edwards & Storkey , 2016 ; Hardt et al. , 2016 ; Du et al. , 2020 ) . Specifically , many such metrics require some form of “ equalized model performance ” across different groups on in-distribution data . Examples include Demographic parity ( DP ) ( Edwards & Storkey , 2016 ) , Equalized Opportunity ( EOpp ) , and Equalized Odds ( EO ) ( Hardt et al. , 2016 ) . Unfortunately , when deployed for real-world applications , deep models commonly encounter data with unforeseeable distribution shifts ( Hendrycks & Dietterich , 2019 ; Hendrycks et al. , 2020 ; 2021 ) . It has been shown that deep learning models can have drastically degraded performance ( Hendrycks & Dietterich , 2019 ; Hendrycks et al. , 2020 ; 2021 ; Taori et al. , 2020 ) and show unreliable behaviors ( Qiu et al. , 2019 ; Yan et al. , 2021 ) under unseen distribution shifts . Intuitively speaking , previous fairness learning algorithms aim to optimize the model to a local minimum where data from majority and minority groups have similar average loss values ( and thus similar in-distribution performance ) . However , those algorithms do not take into consideration the the stability or “ robustness ” of their found fairness-aware minima . Taking object detection in a self-driving car for example , it might have been calibrated over high-quality clear images to be “ fair ” with different skin colors ; however such fairness may severely break down when applied to data collected in adverse visual conditions , such as inclement weather , poor lighting , or other digital artifacts . Our experiments also find that previous state-of-the-art fairness algorithms would be jeopardized if distributional shifts are present in test data , as illustrated in Figure 1 ( b ) . The above findings beg the following question : How to achieve practically sustainable fairness , e.g. , even under unseen distribution shifts ? To answer that , we first propose a new fairness objective , termed Equalized Robustness ( ER ) , which aims to impose “ equalized robustness ” against unseen distribution shifts across the majority and minority groups , so that the learned fairness can sustain even with test data perturbed . ER explicitly considers a new dimension of fairness that is practically significant yet so far largely overlooked . In other words , ER assesses fairness on “ out-of-distribution ” . Therefore it works as a complement instead of a replacement for previous fairness metrics , which focus on assessing the “ in-distribution ” fairness . Previous research has shown that model robustness against input perturbation is highly correlated with loss curvature smoothness ( Bartlett et al. , 2017 ; Moosavi-Dezfooli et al. , 2019 ; Weng et al. , 2018 ) . Our experiments also observed that , the local loss curvature of minority group is often larger than that of majority group , leading to the two group ’ s robustness discrepancy against distribution shifts . To this end , we propose to empirically quantify the robustness discrepancy as the maximum mean discrepancy ( MMD ) ( Gretton et al. , 2012 ) distance between the local model smoothness distributions , for data samples from the majority and minority groups . We experimentally demonstrate that our new metric aligns well with model performance under real-world distribution shifts . On top of that , we further propose a new fair learning algorithm , termed Curvature Matching ( CUMA ) , to simultaneously achieve both traditional in-distribution fairness and ER . CUMA matches the local curvature distribution between data points from the two different groups , as illustrated in Figure 1 ( c ) , by adding a curvature-matching regularizer that can be efficiently computed via a one-shot power iteration method . Our codes will be released upon acceptance . Our contributions can be summarized as bellow : • We propose Equalized Robustness ( ER ) , a new fairness objective for machine learning models , to impose equalized model robustness against unforeseeable distributions shifts across majority and minority groups . • We further propose a new fairness learning algorithm dubbed Curvature Matching ( CUMA ) , which enforces ER during training by utilizing a one-shot power iteration method . • Experiments show that CUMA achieves much more robust fairness against distribution shifts , without more sacrifice on either overall accuracies or the in-distribution fairness , compared with traditional in-distribution fair learning methods . 2 PRELIMINARIES . 2.1 MACHINE LEARNING FAIRNESS . Problem Setting and Metrics Machine learning fairness can be generally categorized into individual fairness and group fairness ( Du et al. , 2020 ) . Individual fairness requires similar inputs to have similar predictions ( Dwork et al. , 2012 ) . Compared with individual fairness , group fairness is a more popular setting and thus the focus of our paper . Given input data X ∈ Rn with sensitive attributes A ∈ { 0 , 1 } and their corresponding ground truth labels Y ∈ { 0 , 1 } , group fairness requires a learned binary classifier f ( · ; θ ) : Rn → { 0 , 1 } parameterized by θ to give equally accurate predictions ( denoted as Ŷ : = f ( X ) ) on the two groups with A = 0 and A = 1 . Multiple fairness criteria have been defined in this context . Demographic parity ( DP ) ( Edwards & Storkey , 2016 ) requires identical ratio of positive predictions between two groups : P ( Ŷ = 1|A = 0 ) = P ( Ŷ = 1|A = 1 ) . Equalized Odds ( EO ) ( Hardt et al. , 2016 ) requires identical false positive rates ( FPRs ) and false negative rates ( FNRs ) between the two groups : P ( Ŷ 6= Y |A = 0 , Y = y ) = P ( Ŷ 6= Y |A = 1 , Y = y ) , ∀y ∈ { 0 , 1 } . Equalized Opportunity ( EOpp ) ( Hardt et al. , 2016 ) requires only equal FNRs between the groups : P ( Ŷ 6= Y |A = 0 , Y = 0 ) = P ( Ŷ 6= Y |A = 1 , Y = 0 ) . Based on these fairness criteria , quantified metrics are defined to measure fairness . Specifically , DP , EO and EOpp distances ( Madras et al. , 2018 ) are defined as follows : ∆DP : = |P ( Ŷ = 1|A = 0 ) − P ( Ŷ = 1|A = 1 ) | ( 1 ) ∆EO : = ∑ y∈ { 0,1 } |P ( Ŷ 6= Y |A = 0 , Y = y ) − P ( Ŷ 6= Y |A = 1 , Y = y ) | ( 2 ) ∆EOpp : = |P ( Ŷ 6= Y |A = 0 , Y = 0 ) − P ( Ŷ 6= Y |A = 1 , Y = 0 ) | ( 3 ) MMD has been previously used to define fairness metric in ( Quadrianto & Sharmanska , 2017 ) defines a more general fairness metric using MMD distance , and shows ∆DP , ∆EO and ∆EOpp to be spatial cases of their unified metric . All these metrics consider the in-distribution fairness , while our Equalized Generalizibility is the first fairness metric explicitly aware of robust generalization ability on unseen distributions . Bias Mitigation Methods Many methods have been proposed to mitigate model bias . Data preprocessing methods such as re-weighting ( Kamiran & Calders , 2012 ) and data-transformation ( Calmon et al. , 2017 ) have been used to reduce discrimination before model training . In contrast , Hardt et al . ( 2016 ) and Zhao et al . ( 2017 ) propose post-processing methods to calibrate model predictions towards a desired fair distribution after model training . Instead of pre- or post-processing , researchers have explored to enhance fairness during training . For example , Madras et al . ( 2018 ) uses a adversarial training technique and shows the learned fair representations can transfer to unseen target tasks . The key technique , adversarial training ( Edwards & Storkey , 2016 ) , was designed for feature disentanglement on hidden representations such that sensitive ( Edwards & Storkey , 2016 ) or domain-specific information ( Ganin et al. , 2016 ) will be removed while keeping other useful information for the target task . The hidden representations are typically the output of intermediate layers of neural networks ( Ganin et al. , 2016 ; Edwards & Storkey , 2016 ; Madras et al. , 2018 ) . Instead , methods , like adversarial debiasing ( Zhang et al. , 2018 ) and its simplified version ( Wadsworth et al. , 2018 ) , directly apply the adversary on the output layer of the classifier , which also promotes the model fairness . Observing the unfairness due to ignoring the worst learning risk of specific samples , Hashimoto et al . ( 2018 ) proposes to use distributionally robust optimization which provably bounds the worst-case risk over groups . Creager et al . ( 2019 ) proposes a flexible fair representation learning framework based on VAE ( Kingma & Welling , 2013 ) , that can be easily adapted for different sensitive attribute settings during run-time . Sarhan et al . ( 2020 ) uses orthogonality constraints as a proxy for independence to disentangles the utility and sensitive representations . Martinez et al . ( 2020 ) formulates group fairness with multiple sensitive attributes as a multi-objective learning problem and proposes a simple optimization algorithm to find the Pareto optimality . Another line of research focuses on learning unbiased representations from biased ones ( Bahng et al. , 2020 ; Nam et al. , 2020 ) . Bahng et al . ( 2020 ) proposes a novel framework to learn unbiased representations by explicitly enforcing them to be different from a set of pre-defined biased representations . Nam et al . ( 2020 ) observes that data bias can be either benign or malicious , and removing malicious bias along can achieve fairness . Li & Vasconcelos ( 2019 ) jointly learns a data re-sampling weight distribution that penalizes easy samples and network parameters . Applications in Computer Vision When many fairness metrics and debiasing algorithms are designed for general learning problems as aforementioned , there are a line of research and applications focusing on fairness-encouraged computer vision tasks . For instance , Buolamwini et al . ( Buolamwini & Gebru , 2018 ) shows current commercial gender-recognition systems have substantial accuracy disparities among groups with different genders and skin colors . Wilson et al . ( 2019 ) observe that state-of-the-art segmentation models achieve better performance on pedestrians with lighter skin colors . In ( Shankar et al. , 2017 ; de Vries et al. , 2019 ) , it is found that the common geographical bias in public image databases can lead to strong performance disparities among images from locales with different income levels . Nagpal et al . ( 2019 ) reveal that the focus region of face-classification models depends on people ’ s ages or races , which may explain the source of age- and race-biases of classifiers . On the awareness of the unfairness , many efforts have been devoted to mitigate such biases in computer vision tasks . Wang et al . ( 2019 ) shows the effectiveness of adversarial debiasing technique ( Zhang et al. , 2018 ) in fair image classification and activity recognition tasks . Beyond the supervised learning , FairFaceGAN ( Hwang et al. , 2020 ) is proposed to prevent undesired sensitive feature translation during image editing . Similar ideas have also been successfully applied to visual question answering ( Park et al. , 2020 ) .

Summary: The paper proposed a new penalty term which is expected to make the final prediction model robust to distribution shift of test data. The proposed penalty is based on the distributions of local curvatures of two sensitive groups and enforces these two distributions to be similar. The proposed idea is motivated by robust learning methods where the prediction model is robust to distributional shift.