EQUALIZED ROBUSTNESS: TOWARDS SUSTAINABLE FAIRNESS UNDER DISTRIBUTIONAL SHIFTS
Year: 2021

Abstract:
Increasing concerns have been raised on deep learning fairness in recent years. Existing fairness metrics and algorithms mainly focus on the discrimination of model performance across different groups on in-distribution data. It remains unclear whether the fairness achieved on in-distribution data can be generalized to data with unseen distribution shifts, which are commonly encountered in real-world applications. In this paper, we first propose a new fairness goal, termed Equalized Robustness (ER), to impose fair model robustness against unseen distribution shifts across majority and minority groups. ER measures robustness disparity by the maximum mean discrepancy (MMD) distance between the loss curvature distributions of two groups of data. We show that previous fairness learning algorithms designed for in-distribution fairness fail to meet the new robust fairness goal. We further propose a novel fairness learning algorithm, termed Curvature Matching (CUMA), to simultaneously achieve both traditional in-distribution fairness and our new robust fairness. CUMA debiases the model robustness by minimizing the MMD distance between loss curvature distributions of two groups. Experiments on three popular datasets show CUMA achieves superior fairness in robustness against distribution shifts, without more sacrifice on either overall accuracies or the in-distribution fairness.

heading: 1 INTRODUCTION
text: With the wide deployment of deep learning in modern business applications concerning individual lives and privacy, there naturally emerge concerns on machine learning fairness (Podesta et al., 2014; Muñoz et al., 2016; Smuha, 2019). Research efforts on various fairness evaluation metrics and corresponding enforcing methods have been carried out (Edwards & Storkey, 2016; Hardt et al., 2016; Du et al., 2020). Specifically, many such metrics require some form of “equalized model performance” across different groups on in-distribution data. Examples include Demographic parity (DP) (Edwards & Storkey, 2016), Equalized Opportunity (EOpp), and Equalized Odds (EO) (Hardt et al., 2016).
Unfortunately, when deployed for real-world applications, deep models commonly encounter data with unforeseeable distribution shifts (Hendrycks & Dietterich, 2019; Hendrycks et al., 2020; 2021). It has been shown that deep learning models can have drastically degraded performance (Hendrycks & Dietterich, 2019; Hendrycks et al., 2020; 2021; Taori et al., 2020) and show unreliable behaviors (Qiu et al., 2019; Yan et al., 2021) under unseen distribution shifts. Intuitively speaking, previous fairness learning algorithms aim to optimize the model to a local minimum where data from majority and minority groups have similar average loss values (and thus similar in-distribution performance). However, those algorithms do not take into consideration the the stability or “robustness” of their found fairness-aware minima. Taking object detection in a self-driving car for example, it might have been calibrated over high-quality clear images to be “fair” with different skin colors; however such fairness may severely break down when applied to data collected in adverse visual conditions, such as inclement weather, poor lighting, or other digital artifacts. Our experiments also find that previous state-of-the-art fairness algorithms would be jeopardized if distributional shifts are present in test data, as illustrated in Figure 1 (b). The above findings beg the following question:
How to achieve practically sustainable fairness, e.g., even under unseen distribution shifts?
To answer that, we first propose a new fairness objective, termed Equalized Robustness (ER), which aims to impose “equalized robustness” against unseen distribution shifts across the majority and minority groups, so that the learned fairness can sustain even with test data perturbed. ER explicitly considers a new dimension of fairness that is practically significant yet so far largely overlooked. In other words, ER assesses fairness on “out-of-distribution”. Therefore it works as a complement instead of a replacement for previous fairness metrics, which focus on assessing the “in-distribution” fairness.
Previous research has shown that model robustness against input perturbation is highly correlated with loss curvature smoothness (Bartlett et al., 2017; Moosavi-Dezfooli et al., 2019; Weng et al., 2018). Our experiments also observed that, the local loss curvature of minority group is often larger than that of majority group, leading to the two group’s robustness discrepancy against distribution shifts. To this end, we propose to empirically quantify the robustness discrepancy as the maximum mean discrepancy (MMD) (Gretton et al., 2012) distance between the local model smoothness distributions, for data samples from the majority and minority groups. We experimentally demonstrate that our new metric aligns well with model performance under real-world distribution shifts. On top of that, we further propose a new fair learning algorithm, termed Curvature Matching (CUMA), to simultaneously achieve both traditional in-distribution fairness and ER. CUMA matches the local curvature distribution between data points from the two different groups, as illustrated in Figure 1 (c), by adding a curvature-matching regularizer that can be efficiently computed via a one-shot power iteration method. Our codes will be released upon acceptance.
Our contributions can be summarized as bellow:
• We propose Equalized Robustness (ER), a new fairness objective for machine learning models, to impose equalized model robustness against unforeseeable distributions shifts across majority and minority groups.
• We further propose a new fairness learning algorithm dubbed Curvature Matching (CUMA), which enforces ER during training by utilizing a one-shot power iteration method.
• Experiments show that CUMA achieves much more robust fairness against distribution shifts, without more sacrifice on either overall accuracies or the in-distribution fairness, compared with traditional in-distribution fair learning methods.

heading: 2 PRELIMINARIES

heading: 2.1 MACHINE LEARNING FAIRNESS
text: Problem Setting and Metrics Machine learning fairness can be generally categorized into individual fairness and group fairness (Du et al., 2020). Individual fairness requires similar inputs to have similar predictions (Dwork et al., 2012). Compared with individual fairness, group fairness is a more popular setting and thus the focus of our paper. Given input data X ∈ Rn with sensitive attributes A ∈ {0, 1} and their corresponding ground truth labels Y ∈ {0, 1}, group fairness requires a learned binary classifier f(·; θ) : Rn → {0, 1} parameterized by θ to give equally accurate predictions (denoted as Ŷ := f(X)) on the two groups with A = 0 and A = 1. Multiple fairness criteria have been defined in this context. Demographic parity (DP) (Edwards & Storkey, 2016) requires identical ratio of positive predictions between two groups: P (Ŷ = 1|A = 0) = P (Ŷ = 1|A = 1). Equalized Odds (EO) (Hardt et al., 2016) requires identical false positive rates (FPRs) and false negative rates (FNRs) between the two groups: P (Ŷ 6= Y |A = 0, Y = y) = P (Ŷ 6= Y |A = 1, Y = y),∀y ∈ {0, 1}. Equalized Opportunity (EOpp) (Hardt et al., 2016) requires only equal FNRs between the groups: P (Ŷ 6= Y |A = 0, Y = 0) = P (Ŷ 6= Y |A = 1, Y = 0). Based on these fairness criteria, quantified metrics are defined to measure fairness. Specifically, DP, EO and EOpp distances (Madras et al., 2018) are defined as follows:
∆DP := |P (Ŷ = 1|A = 0)− P (Ŷ = 1|A = 1)| (1) ∆EO := ∑ y∈{0,1} |P (Ŷ 6= Y |A = 0, Y = y)− P (Ŷ 6= Y |A = 1, Y = y)| (2)
∆EOpp := |P (Ŷ 6= Y |A = 0, Y = 0)− P (Ŷ 6= Y |A = 1, Y = 0)| (3) MMD has been previously used to define fairness metric in (Quadrianto & Sharmanska, 2017) defines a more general fairness metric using MMD distance, and shows ∆DP , ∆EO and ∆EOpp to be spatial cases of their unified metric. All these metrics consider the in-distribution fairness, while our Equalized Generalizibility is the first fairness metric explicitly aware of robust generalization ability on unseen distributions.
Bias Mitigation Methods Many methods have been proposed to mitigate model bias. Data preprocessing methods such as re-weighting (Kamiran & Calders, 2012) and data-transformation (Calmon et al., 2017) have been used to reduce discrimination before model training. In contrast, Hardt et al. (2016) and Zhao et al. (2017) propose post-processing methods to calibrate model predictions towards a desired fair distribution after model training. Instead of pre- or post-processing, researchers have explored to enhance fairness during training. For example, Madras et al. (2018) uses a adversarial training technique and shows the learned fair representations can transfer to unseen target tasks. The key technique, adversarial training (Edwards & Storkey, 2016), was designed for feature disentanglement on hidden representations such that sensitive (Edwards & Storkey, 2016) or domain-specific information (Ganin et al., 2016) will be removed while keeping other useful information for the target task. The hidden representations are typically the output of intermediate layers of neural networks (Ganin et al., 2016; Edwards & Storkey, 2016; Madras et al., 2018). Instead, methods, like adversarial debiasing (Zhang et al., 2018) and its simplified version (Wadsworth et al., 2018), directly apply the adversary on the output layer of the classifier, which also promotes the model fairness. Observing the unfairness due to ignoring the worst learning risk of specific samples, Hashimoto et al. (2018) proposes to use distributionally robust optimization which provably bounds the worst-case risk over groups. Creager et al. (2019) proposes a flexible fair representation learning framework based on VAE (Kingma & Welling, 2013), that can be easily adapted for different sensitive attribute settings during run-time. Sarhan et al. (2020) uses orthogonality constraints as a proxy for independence to disentangles the utility and sensitive representations. Martinez et al. (2020) formulates group fairness with multiple sensitive attributes as a multi-objective learning problem and proposes a simple optimization algorithm to find the Pareto optimality. Another line of research focuses on learning unbiased representations from biased ones (Bahng et al., 2020; Nam et al., 2020). Bahng et al. (2020) proposes a novel framework to learn unbiased representations by explicitly enforcing them to be different from a set of pre-defined biased representations. Nam et al. (2020) observes that data bias can be either benign or malicious, and removing malicious bias along can achieve fairness. Li & Vasconcelos (2019) jointly learns a data re-sampling weight distribution that penalizes easy samples and network parameters.
Applications in Computer Vision When many fairness metrics and debiasing algorithms are designed for general learning problems as aforementioned, there are a line of research and applications focusing on fairness-encouraged computer vision tasks. For instance, Buolamwini et al. (Buolamwini & Gebru, 2018) shows current commercial gender-recognition systems have substantial accuracy disparities among groups with different genders and skin colors. Wilson et al. (2019) observe that state-of-the-art segmentation models achieve better performance on pedestrians with lighter skin colors. In (Shankar et al., 2017; de Vries et al., 2019), it is found that the common geographical bias in public image databases can lead to strong performance disparities among images from locales with different income levels. Nagpal et al. (2019) reveal that the focus region of face-classification models depends on people’s ages or races, which may explain the source of age- and race-biases of classifiers. On the awareness of the unfairness, many efforts have been devoted to mitigate such biases in computer vision tasks. Wang et al. (2019) shows the effectiveness of adversarial debiasing technique (Zhang et al., 2018) in fair image classification and activity recognition tasks. Beyond the supervised learning, FairFaceGAN (Hwang et al., 2020) is proposed to prevent undesired sensitive feature translation during image editing. Similar ideas have also been successfully applied to visual question answering (Park et al., 2020).

heading: 2.2 MODEL ROBUSTNESS AND SMOOTHNESS
text: Model generalization ability and robustness has been shown to be highly correlated with model smoothness (Moosavi-Dezfooli et al., 2019; Weng et al., 2018). Weng et al. (2018) and Guo et al. (2018) use local Lipschitz constant to estimate model robustness against small perturbations on inputs within a hyper-ball. Moosavi-Dezfooli et al. (2019) proposes to improve model robustness by adding a curvature constraint to encourage model smoothness. Miyato et al. (2018) approximates model local smoothness by the spectral norm of Hessian matrix, and improves model robustness against adversarial attacks by regularizing model smoothness.

heading: 3 EQUALIZED ROBUSTNESS: A NEW METRIC FOR FAIR GENERALIZATION AND ROBUSTNESS
text: Consider a binary classifier f(·; θ) trained on two groups of data X1 and X2 respectively. Our goal is to define a metric to measure the gap of model robustness between the two groups. Formulating such a metric is highly non-trivial, with difficulties from mainly two aspects.
The first challenge is that we need to ensure fair generalization against multiple unseen distribution shifts that may encounter in real world applications. A trivial solution would be selecting a set of predefined distribution shifts and measuring the average performance gap (e.g., ∆EO) against them. However, this approach requires engineering overhead in handcrafting the predefined distribution shifts, and the predefined distribution shifts may not be representative enough to cover all unseen cases. Previous research (Miyato et al., 2018; Moosavi-Dezfooli et al., 2019; Guo et al., 2018; Weng et al., 2018) has shown both theoretically and empirically that deep model robustness scales with its model smoothness. Following (Miyato et al., 2018; Moosavi-Dezfooli et al., 2019), we use the spectral norm of Hessian matrix to approximate local smoothness as an indicator of model robustness. Specifically, given an input x, the Hessian matrix H(x) is defined as the second-order gradient of L(x) with respect to input x: H(x) = ∇2xL(x). The approximated local curvature C(x) at point x is thus defined as: C(x) = σ(H(x)), (4) where σ(H) is the spectral norm of H: σ(H) = supv:‖v‖2=1 ‖Hv‖2. Intuitively, C(x) measures the maximal directional curvature or change rate at x. Thus, smaller C(x) indicates better local smoothness around x (Miyato et al., 2018; Moosavi-Dezfooli et al., 2019).
For the second difficulty, unlike previous fairness metrics where the target random variable1 follows a Bernoulli distribution, the local curvature used in ER is a continuous random variable without a simple underlying distribution. The unknown distribution form makes it difficult to directly measure the difference between the curvature distributions by a parametric statistic test (e.g., t-test or KL divergence). To tackle this problem, we utilize maximum mean discrepancy (MMD) (Gretton et al.,
1Such as Y = 1 in DP and Y 6= Ŷ in EO and EOpp. (See Section 2.1.)
2012) to do a two-sample test on C(X1) and C(X2). MMD is a distribution distance measure, agnostic to the exact distribution formulation and only based on the mean difference. Formally, our new fairness metric for equalized robustness is defined as follows:
Our new fairness metric ∆ER Consider a machine learning model f trained on two groups of data X1 and X2 respectively. Suppose C(X1) ∼ P1 and C(X2) ∼ P2, then the model’s ∆ER is defined as the squared maximum-mean-discrepancy (MMD) distance between C(X1) and C(X2):
∆ER = MMD 2(P1,P2). (5)
MMD is widely used to measure the distance between two high-dimensional distributions in deep learning (Li et al., 2015; 2017; Bińkowski et al., 2018). The MMD distance between two distributions P and Q is defined as
MMD2(P,Q) = ‖µP − µQ‖2H = EP [k(X,X)]− 2EP,Q[k(X,Y )] + EQ[k(Y, Y )] (6)
where X ∼ P , Y ∼ Q and k(·, ·) is the kernel function. In practice, we use finite samples from P and Q to statistically estimate their MMD distance:
MMD2(P,Q) = 1 M2 M∑ i=1 M∑ i′=1 k(xi, xi′)− 2 MN M∑ i=1 N∑ j=1 k(xi, yj) + 1 N2 N∑ j=1 N∑ j′=1 k(yj , yj′) (7)
where {xi ∼ P}Mi=1, {yj ∼ Q}Nj=1, and we use the mixed RBF kernel function k(x, y) =∑ σ∈S e − ‖x−y‖ 2
2σ2 with hyperparameter S = {1, 2, 4, 8, 16}. Ablation studies on S values are conducted in Section 5.3.

heading: 4 CURVATURE MATCHING: FAIR MACHINE LEARNING TOWARDS EQUALIZED ROBUSTNESS

heading: 4.1 PRACTICAL CURVATURE APPROXIMATION
text: In order to achieve equalized robustness, one intuitive solution is to add ∆ER (Eq. (5)) as an regularization term in the loss function during training phase. However, it is non-practical to precisely calculate the spectral norm (which is equal to the absolute value of dominant eigenvalue) of Hessian matrix in ∆ER. To solve this problem, we use a one-shot power iteration method (PIM) for practical approximation of C(x) during training. First we rewrite C(x) with the following form: C(x) = σ(H(x)) = ‖H(x)v‖, where v is the dominant eigenvector with the maximal eigenvalue, which can be calculated by power iteration method. In practice, to increase training efficiency, we use a one-shot power iteration method. Specifically, we estimate the dominant eigenvector v by the gradient direction: ṽ := sign(g)‖sign(g)‖ ≈ v, where g = ∇xL(x). This is because previous works have observed a large similarity between the dominant eigenvector and the gradient direction (Miyato et al., 2018; Moosavi-Dezfooli et al., 2019). We further approximate Hessian matrix by finite differentiation on gradients: H(x)v ≈ ∇xL(x+hv)−∇xL(x)h where h is a small constant. As a result, the final approximation of curvature smoothness is
C(x) ≈ C̃(x) := ‖∇xL(x+ hṽ)−∇xL(x)‖ |h| . (8)

heading: 4.2 CURVATURE MATCHING
text: With the practical curvature approximation, now we can match the curvature distribution of the two groups by minimizing the MMD distance. Suppose C̃(X1) ∼ Q1 and C̃(X2) ∼ Q2, we define the curvature matching loss functions as:
Lcm = MMD2(Q1,Q2) (9)
We add Lcm to the traditional adversarially fair training (Ganin et al., 2016; Madras et al., 2018) loss function as a regularizer, in order to attain both in-distribution fairness and fair robustness. As
𝑥 𝑓𝑠(∙; θs) ℎ𝑡(∙; θ𝑡)
ℎ𝑎(∙; θ𝑎)
𝑓 ∙; θ = ℎ𝑡(𝑓𝑠(∙; θs); θ𝑡)
𝐿𝑎𝑑𝑣
𝐿𝑐𝑙𝑓
𝐶(∙) 𝐿𝑐𝑚𝑄1
𝑄2
Figure 2: The overall framework of CUMA. x is the input sample. ht is the utility head for the target task. ha is the adversarial head to predict sensitive attributes. fs is the shared backbone. C(·) is the curvature estimation function, as defined in Eq. (4). Q1 and Q2 are local curvature distributions of majority and minority groups, respectively. Lcm, Lclf and Ladv are three loss terms as defined in Eq. (9) and (11).
illustrated in Figure 2, our model follows the same “two-head” structure as traditional adversarial learning frameworks (Ganin et al., 2016; Madras et al., 2018), where ht is the utility head for the target task, ha is the adversarial head to predict sensitive attributes, and fs is the shared backbone.2 Suppose for each sample xi, the sensitive attribute is ai and the corresponding target label is yi, then our overall optimization problem can be written as:
min θs,θt max θa L = min θs,θt max θa (Lclf − αLadv + γLcm) (10)
where
Lclf = 1
N N∑ i=1 `(ht(fs(xi; θs); θt), yi),Ladv = 1 N N∑ i=1 `(ha(fs(xi; θs); θa), ai), (11)
`(·, ·) is the cross-entropy loss function, α and γ are trade-off hyperparameters, andN is the number of training samples.

heading: 5 EXPERIMENTS

heading: 5.1 EXPERIMENTAL SETUP
text: Datasets and pre-processing Experiments are conducted on three datasets widely used to evaluate machine learning fairness: Communities and Crime (C&C) (Redmond & Baveja, 2002), Adult (Kohavi, 1996), and CelebA (Liu et al., 2015).3 C&C dataset has 1,994 samples with neighborhood population statistics, where 1,500 are used for training and the rest for evaluation. The target task is to predict violent crime per capita, and we use “RacePctBlack” (percentage of black population in the neighborhood) and “FemalePctDiv” (divorce ratio of female in the neighborhood) as sensitive attributes. All features in C&C dataset are of continous values in [0, 1]. To fit in the fairness problem setting, we binarilize the target and sensitive attributes with the top-30% largest value as the threshold.4 We also do data-whitening on C&C. Adult dataset has 48,842 samples with basic personal information such as education and occupation, where 30,000 are used for training and the rest for evaluation. The target task is to predict the person’s annual income, and we use “gender” (male or female) as the sensitive attribute. The features in Adult dataset are of either continuous (e.g., age) or categorical (e.g. sex) values. We use one-hot encoding on the categorical features and then concatenate them with the continuous ones. We use data whitening on the concatenated features. CelebA has over 200,000 images of celebrity faces, with 40 attribute annotations. The target task is to predict the “attractiveness” attribute and the sensitive attributes to protect are “chubby” and “eyeglasses”. We randomly select 45, 000 as training samples and 5, 000 as testing samples. All images are center-cropped and resized to 128× 128, and pixel values are scaled to [0, 1].
2Thus the binary classifier f(·; θ) = ht(fs(·; θs); θt), with θ = θt ∪ θs. 3Traditional image classification datasets (e.g., ImageNet) are not directly applicable since they lack fairness
attribute labels. 4As a result P[A = 0] = 30% and P[Y = 0] = 30%.
Models For C&C and Adult datasets, we use two-layer MLPs for fs, ht and ha. For CelebA dataset, we use ResNet18 as backbone, where the first three stages are used as fs and the last stage (together with the fully connected classification layer) is used as ht. The auxiliary adversarial head ha has the same structure as ht. Detailed model structures are described in Appx. A.
Baseline Methods We compare CUMA with the following state-of-the-art in-distribution fairness algorithms. Adversarial debiasing (AdvDebias) (Zhang et al., 2018) is one of the most popular fair training algorithm based on adversarial training (Ganin et al., 2016). Madras et al. (2018) proposes a similar framework termed Learned Adversarially Fair and Transferable Representations (LAFTR), by replacing the cross-entropy loss used in (Zhang et al., 2018) with a group-normalized `1 loss, which is shown to work better on highly unbalanced datasets. We also include normal (fairnessignorant) training as a baseline.
Evaluation Metric We use three different groups of evaluation metrics: the overall accuracy, in-distribution fairness metrics, and robust fairness metrics. We report the overall accuracy on all test samples in the original test sets. To measure in-distribution fairness, we use ∆EOpp and ∆EO on the original test sets. To measure robust fairness under distribution shifts, we use our newly proposed ∆ER on the original test sets, and also ∆EOpp and ∆EO on a set of pre-defined real-world distribution shifts. We intend to show that ∆ER calculated on the original test sets aligns well with robust fairness under real-world distribution shifts. See the following paragraph for the details in constructing distributional shifts.
Distributional shifts On Adult and C&C datasets, we construct two distribution shifts by adding random Gaussian and uniform noises, respectively, to the test data. Specifically, following (Madras et al., 2018; Zhang et al., 2018), the categorical features in Adult and C&C datasets are first one-hot encoded and then whitened into float-value vectors, where noises are added. Both types of noises have mean µ = 0 and has standard derivation σ = 0.03 . On CelebA dataset, following (Hendrycks & Dietterich, 2019), we construct two distribution shifts by adding random Gaussian (with mean µ = 0 and standard derivation σ = 0.08) and impulse noise (with ratio p = 0.03), respectively. We report the fairness in robustness against other settings of distribution shifts in Appx. C.
Implementation Details Unless further specified, we set the loss trade-off parameter α to 1 in all experiments by default. We use Adam optimizer (Kingma & Ba, 2014) with initial learning rate 10−3 and weight decay 10−5. The learning rate is gradually decreased to 0 by cosine annealing learning rate scheduler (Loshchilov & Hutter, 2016). On both Adult and C&C datasets, we train for 50 epochs from scratch for all methods. On CelebA dataser, we first normally train a model for 100 epochs, and then finetune it for 20 epochs using CUMA. For fair comparison, we train for 120 epochs on CelebA for all baseline methods. The constant h in Eq. (8) is set to 1 by default. For more implementation details, please check Appx. A.

heading: 5.2 MAIN RESULTS
text: Experimental results on three datasets with different sensitive attributes are shown in Tables 3-5, where we compare CUMA with the baseline methods on three different groups of metrics as discussed in Section 5.1. “Normal” means standard training without any fairness regularization. All numbers are shown as percentages. Many intriguing findings can be concluded from the results.
First, we see that previous state-of-the-art fairness learning algorithms would be jeopardized if distributional shifts are present in test data. For example, on C&C dataset with “RacePctBlack” as sensitive attribute (Table 1), LAFTR achieves ∆EO = 28.83% on in-distribution test set, while that number is increased to 32.20% on the test set perturbed with uniform random noise. Similarly, for AdvDebias, it achieves ∆EO = 29.73% on the original CelebA test set with “chubby” as the sensitive attribute (Table 4), while that number is increased to 35.46% and 36.48% on test sets perturbed with Gaussian and impulse noises, respectively.
Second, we see that CUMA achieves the best robust fairness under distribution shifts on all three benchmark datasets with different sensitive attribute settings, while maintaining similar indistribution fairness and overall accuracy. For example, on C&C dataset with “RacePctBlack” as the sensitive attribute (Table 1), CUMA achieves 2.73% and 4.82% less ∆EO than the second-best performer (LAFTR) under distribution shifts by additive Gaussain and uniform noises, respectively. Moreover, for the same experiment setting, although CUMA and LAFTR achieve almost identical in-distribution fairness (the difference between their ∆EO on original test set is within 0.5%), CUMA keeps (and even increases) the fairness under distribution shifts (e.g., 1.33% smaller ∆EO under uniform noises), while the fairness achieved by LAFTR is jeopardized under both types of distribution shifts (e.g., 3.37% larger ∆EO under uniform noises). Similarly, on CelebA dataset with “Chubby” as the sensitive attribute, LAFTR has even slightly better in-distribution fairness than CUMA. However, when the test sets have distribution shifts, the fairness achieved by LAFTR is jeopardized (with 5.50% and 6.02% more ∆EO under Gaussian and uniform noises, respectively), while CUMA keeps its fairness and achieves better fairness under distribution shifts (e.g., 2.50% and 3.17% less ∆EO compared with LAFTR.).
Third, for all three datasets, the ∆ER calculated on the original test set highly correlates with traditional fairness metrics (e.g., ∆EOpp, ∆EO) calculated on the perturbed test sets: the smaller ∆ER on the in-distribution test set, the smaller ∆EO on perturbed test sets. This shows that our new metric ∆ER aligns well with robust fairness under real-world distribution shifts, and validates the rationality of using it as an indicator of model robustness discrimination.
More experimental results are shown in Appx. B (trade-off curves between fairness and accuracy) and Appx. C (results on other settings of distributional shifts).

heading: 5.3 ABLATION STUDY
text: Ablation Study on ∆ER In this section, we study how well can the ∆ER predict the robust fairness and the sensitivity of ∆ER with respect to S (the sampling set for σ in the mixed RBF kernel function, as described in Section 3). A small σ will make the ∆ER more sensitive to the difference between the two sample set, which could be caused by either the true discrepancy of distributions or the different noise introduced by sampling.
In contrast, a larger ones may under-estimate the discrepancy. Thus, a proper S should include a wide range of σ to avoid the domination of either large or small values. In this paper, we choose a geometric sequence with 2 as the base, i.e., S = {1, 2, 4, 8, 16}. Furthermore, we compare ∆ER values under three different sets: S1 = {0.25, 0.5, 1, 2, 4}, S2 = {1, 2, 4, 8, 16} (the default S as defined in Section 3), and S3 = {4, 8, 16, 32, 64}. Results are shown in
Table 6. As in Section 5.2, we empirically evaluate the robust fairness by ∆EO on the test set corrupted by uniform noise. From the results, we observe that with all three different S settings, ∆ER aligns well with the model fairness under distribution shifts (∆EO under uniform noise).
Ablation Study on CUMA In this section, we check the sensitivity of CUMA with respect to its hyperparameters: the loss trade-off parameters α and γ in Eq. (10) and h in Eq. (8). Results are shown in Table 7. When fixing γ = 1, ∆ER peeks at around α = 1, so we use
it as the default α value. When fixing α = 1, the best trade-off between overall accuracy and robust fairness is achieved at round γ = 1, which we use as the default γ. Varying the value of h hardly affects the performance of CUMA.

heading: 6 CONCLUSION
text: In this paper, we first propose a new fairness goal, termed Equalized Robustness (ER), to impose fair model robustness against unseen distribution shifts across different data groups. We further propose a novel fairness learning algorithm, termed Curvature Matching (CUMA), to simultaneously achieve both traditional in-distribution fairness and our new robust fairness. Experiments show CUMA achieves superior fairness in robustness against distribution shifts, without more sacrifice on either overall accuracies or the in-distribution fairness compared with traditional in-distribution fair learning methods. As a pioneer work, the new concept of ER proposed in this paper aims to measure a new dimension of fairness that is practically significant yet so far largely overlooked: ER assesses “out-of-distribution” fairness while previous metrics focus on “in-distribution” fairness. Therefor, ER works as a complement instead of a replacement for previous fairness metrics. We hope our work can open up more discussions on how to evaluate model fairness in a more complete spectrum.

heading: A MORE IMPLEMENTATION DETAILS
text: On C&C and Adult datsets, suppose the input feature dimension is d, then the dimensions of hidden layers in fs and ht are d → 100 → 64 and 64 → 32 → 2, respectively. ha has identical model structure with ht. For all three sub-networks, ReLU activation function and dropout layer with 0.25 dropout ratio are applied between the two fully connected layers. On CelebA dataset, we use the ResNet18 as backbone. The input feature size of ht and ha is 8×8×256 (with channel-last layout).

heading: B TRADE-OFF CURVES BETWEEN FAIRNESS AND ACCURACY
text: For CUMA and both baseline methods, we can obtain different trade-offs between fairness and accuracy by setting the loss function weights (e.g., α and γ) to different values. For example, the larger α, the better fairness and the worse accuracy. Such trade-off curves between fairness and accuracy of different methods are shown in Figure 3. The closer the curve to the top-left corner (i.e., with larger accuracy and smaller ∆EO), the better Pareto frontier is achieved. As we can see, our method achieves the best Pareto frontiers for both in-distribution fairness (left panel) and robust fairness under distribution shifts (middle and right panel).

heading: C RESULTS ON OTHER SETTINGS OF DISTRIBUTIONAL SHIFTS
text: In this section, we show that the conclusions drawn in Section 5.2 hold under different settings of distributional shifts. Specifically, we consider a new noise setting with mean µ = 0 and standard
derivation σ = 0.06 (other than the mean µ = 0 and standard derivation σ = 0.03 evaluated in the main text) for both random Gaussian and uniform noises. The results under these new distributional shifts on C&C dataset are shown in Tables 8 and 9.

Review summaries:
The paper is motivated by a common problem in real world applications of deep models, distributional shifts, which can cause unreliable behavior in the deployed models. In particular,  state-of-the-art fairness algorithms would be affected by such distributional shifts in the test data. This poses the following question studied by the paper: how can one achieve fairness when there exist unseen distributional shifts?  Towards this end, the paper proposes a new objective: Equalized Robustness (ER), that imposes equalized model robustness against distributions shifts across majority and minority. Fruthermore, the paper develops a new algorithm called Curvature Matching (CUMA) that imposes ER during training, and tests it through experiments.  

The paper proposed a new penalty term which is expected to make the final prediction model robust to distribution shift of test data. The proposed penalty is based on the distributions of local curvatures of two sensitive groups and enforces these two distributions to be similar. The proposed idea is motivated by robust learning methods where the prediction model is robust to distributional shift. 

This paper proposes a new fairness metric, Equalized Robustness (ER), to assess model robustness for sensitive subgroups. ER measures the maximum mean discrepancy (MMD) between the loss curvature of the two sensitive groups. The paper proposes using gradient direction to approximate the spectral norm and using finite differences to approximate the Hessian. The MMD is calculated using the RBF kernel.   The paper shows that existing in-distribution fairness promoting methods do not achieve parity with respect to the new metric. A new method, curvature matching (CUMA), is proposed to achieve both in-distribution and robust fairness. CUMA adds the MMD distance as a regularizer to the standard adversarially fair learning approach (which includes a cross entropy loss term and an adversarially fair loss term for in-distribution fairness).  The paper performs ablation studies for the hyper parameter used in the RBF used to compute MMD.