Title: Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach
Venue: NeurIPS 2023
Accepted: yes
Keywords: Model Weight Perturbation

fairness

distribution shift
Abstract: Fairness in machine learning has attracted increasing attention in recent years. The fairness methods improving algorithmic fairness for in-distribution data may not perform well under distribution shifts. In this paper, we first theoretically demonstrate the inherent connection between distribution shift,  data perturbation, and model weight perturbation.
Subsequently, we analyze the sufficient conditions to guarantee fairness (i.e., low demographic parity) for the target dataset, including fairness for the source dataset, and low prediction difference between the source and target datasets for each sensitive attribute group. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR) by considering the worst case within the model weight perturbation ball for each sensitive attribute group. We evaluate the effectiveness of our proposed RFR algorithm on synthetic and real distribution shifts across various datasets. Experimental results demonstrate that RFR achieves better fairness-accuracy trade-off performance compared with several baselines. The source code is available at \url{https://github.com/zhimengj0326/RFR_NeurIPS23}.

Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach

Abstract

Fairness in machine learning has attracted increasing attention in recent years. The fairness methods improving algorithmic fairness for in-distribution data may not perform well under distribution shifts. In this paper, we first theoretically demonstrate the inherent connection between distribution shift, data perturbation, and model weight perturbation. Subsequently, we analyze the sufficient conditions to guarantee fairness (i.e., low demographic parity) for the target dataset, including fairness for the source dataset, and low prediction difference between the source and target datasets for each sensitive attribute group. Motivated by these sufficient conditions, we propose robust fairness regularization (RFR) by considering the worst case within the model weight perturbation ball for each sensitive attribute group. We evaluate the effectiveness of our proposed RFR algorithm on synthetic and real distribution shifts across various datasets. Experimental results demonstrate that RFR achieves better fairness-accuracy trade-off performance compared with several baselines. The source code is available at https://github.com/zhimengj0326/RFR_NeurIPS23 .

1 Introduction

Previous research [1-4] has shown that a classifier trained on a specific source distribution will perform worse when testing on a different target distribution, due to distribution shift. Recently, many studies have focused on investigating the impact of distribution shift on machine learning models, where fairness performance degradation is even more significant than that of prediction performance [5]. The sensitivity of fairness over distribution shift challenges machine learning models in high stake applications, such as criminal justice [6], healthcare [7], and job marketing [8]. Thus the transferability of the fairness performance under distribution shift is a crucial consideration for real-world applications.

To achieve the fairness of the model (already achieved fairness on the source dataset ) on the target dataset, we first reveal that distribution shift is equivalent to model weight perturbation, and then seek to achieve fairness under distribution shift via model weight perturbation. Specifically, i) we reveal the inherent connection between distribution shift, data perturbation, and model weight perturbation. We theoretically demonstrate that any distribution shift can be equivalent to data perturbation and model weight perturbation in terms of loss value. In other words, the effect of distribution shift on model training can be attributed to data or model perturbation. ii) Given the established connection between the distribution shift and model weight perturbation, we next tackle fairness under the distribution shift problem. We first investigate demographic parity relation between source and target datasets. Achieving fair prediction (e.g., low demographic parity) in the source dataset is insufficient for fair prediction in the target dataset. More importantly, the average prediction difference between source and target datasets with the same sensitive attribute also matters for achieving fairness in target datasets.

Source

Target

…

Equivalence

(a)

(b)

Motivated by the established equivalence connection, we propose robust fairness regularization (RFR) to enforce average prediction robustness over model weight. In this way, the well-trained model can tackle the distribution shift problem in terms of demographic parity. Considering the expensive computation for the inner maximization problem in RFR, we accelerate RFR by obtaining the approximate closed-form model weight perturbation using first-order Taylor expansion. In turn, an efficient RFR algorithm, trading the inner maximization problem into two forward and backward propagations for each model update, is proposed to achieve robust fairness under distribution shift. Our contributions are highlighted as follows:

We theoretically reveal the inherent connection between distribution shift, data perturbation, and model weight perturbation. In other words, distribution shift and model perturbation are equivalent in terms of loss value for any model architecture and loss function.

We first analyze the sufficient conditions to guarantee fairness transferability under distribution shift. Based on the established connection, we propose RFR explicitly pursuing sufficient conditions for robust fairness by considering the worst case of model perturbation.

We evaluate the effectiveness of RFR on various real-world datasets with synthetic and real distribution shifts. Experiments results demonstrate that RFR can mostly achieve better fairnessaccuracy tradeoff with both synthetic and real distribution shifts.

2 Understanding Distribution Shift

In this section, we first provide the notations used in this paper. And then, we theoretically understand the relations between distribution shift, data perturbation, and model weight perturbation, as shown in Figure 1.

2.1 Notations

We consider source dataset D S and target dataset D T , defined as a probability distribution P S and P T for samples S ∈ S and T ∈ T , respectively. Each sample defines values for three random variables: features X with arbitrary domain X , binary sensitive attribute A ∈ A = { 0 , 1 } , and label Y arbitrary domain Y , i.e., S = T = X × A × Y . We denote δ as data perturbation on source domain S , where δ X ( X ) and δ Y ( Y ) data perturbation of features and labels. Using P ( · ) to denote the space of probability distributions over some domain, we denote the space of distributions over examples as P ( S ) . We use || · || p as L p norm. Let f θ ( x ) be the output of the neural networks parameterized with θ to approximate the true label y . We define l ( f θ ( x ) , y ) as the loss function for neural network training, and the optimal model parameters θ ∗ training on source data S is given by θ ∗ = arg min R S , where

θ

R S = E ( X,Y ) ∼ P S [ l ( f θ ( X ) , Y )] . Since the target distribution P T maybe be different with source distribution P S , the well-trained model f θ ∗ ( · ) trained on soure dataset S typically does not perform well in target dataset T .

2.2 Distribution Shift is Data Perturbation

Deep neural networks are immensely challenged by data quality or dynamic environments [9, 10]. The well-trained deep neural network model may not perform well if existing feature/label noise in training data or distribution shifts among training and target environments. In this subsection, we reveal the inherent equivalence between distribution shift and data perturbation for any neural networks f θ ( · ) , source distribution P S , and target distribution P T using optimal transport. We first provide the formal definition of optimal transport as follows:

Definition 1 (Optimal Transport [11]) . Considering two distributions with probability distribution P S and P T , and cost function moving from s to t as c ( s, t ) , optimal transport between probability distribution P S and P T is given by

where the distribution set Γ ( P S , P T ) is the collection of all possible transportation plans and given by Γ ( P S , P T ) = { γ ( s, t ) > 0 , ∫ γ ( s, t )d t = P S ( s ) , ∫ γ ( s, t )d s = P T ( t ) } . In other words, the distribution set consists of all possible joint distributions with margin distribution P S and P T .

Based on the definition of optimal transport, we demonstrate that distribution shift is equivalent to data perturbation in the following theorem:

Theorem 2.1. For any two different distributions with probability distribution P S and P T , adding data perturbation δ 2 on source data S can make perturbated source data and target data with the same distribution, where the distribution of data perturbation δ is given by

Additionally, for any positive p > 0 , data perturbation δ with minimal power E [ || δ || p p ] is given by Eq.(2) if optimal transport plan γ ∗ ( · , · ) is calculated based on Eq. (1) with cost function c ( s, t ) = || s -t || p p .

Proof sketch. Given two different distributions, there are many possible perturbations to move one distribution to another. Optimal transport can select the optimal feasible perturbations or transportation plan in terms of specific objectives (e.g., perturbations power). Given the optimal transportation plan, we can derive the distribution of perturbations based on basic probability theory.

Theorem 2.1 demonstrates the equivalence between distribution shift and data perturbation, where data perturbation distribution is dependent on optimal transport between source and target distribution. The intuition is that such distribution shift can be achieved via optimal transportation (i.e., data perturbation). Based on Theorem 2.1, we have the following Corollary 2.2 on the equivalent of neural network behavior for source and target data:

Corollary 2.2. Given source and target datasets with probability distribution P S and P T , there exists data perturbation δ so that the training loss of any neural network f θ ( · ) for target distribution equals that for source distribution with data perturbation δ , i.e.,

In other words, for the model trained with loss minimization on source data, the deteriorated performance on the target dataset stems from the perturbation of features and labels. The proof sketch is based on Theorem 2.1 since adding a perturbation on the source dataset can be consistent with the target dataset distribution. Therefore, the conclusion can hold for any loss function that is only dependent on the dataset.

2.3 Data Perturbation Equals Model Weight Perturbation

Although we understand that the distribution shift can be attributed to the data perturbation of features and labels in source data, it is still unclear how to tackle the distribution shift issue. A natural solution

is to adopt adversarial training to force the well-trained model to be robust over data perturbation. However, it is complicated to generate data perturbation on features and labels simultaneously, and many well-developed adversarial training methods are mainly designed for adversarial feature perturbation. Fortunately, we show that model weight perturbation is equivalent to data perturbation by Theorem 2.3.

Theorem 2.3. Considering the source dataset with distribution P S , suppose the source dataset is perturbed with data perturbation δ , and the neural network is given by f θ ( · ) , for general case, there exists model weight perturbation ∆ θ so that the training loss on perturbed source dataset is the same with that for model weight perturbation ∆ θ on source distribution:

Proof sketch. The loss under data perturbation and weight perturbation can be analyzed using first-order Tayler expansion. The critical step is to find the condition of the equivalence for the first order term of data perturbation and weight perturbation. Fortunately, the existence of such conditions can be easily proved using linear algebra.

Theorem 2.3 demonstrates that the training loss on perturbed data distribution (but fixed model weight) equals the training loss on perturbed model weight (but original data distribution). In other words, the training loss fluctuation from data perturbation can equal model weight perturbation. Furthermore, we conclude that chasing a robust model over data perturbation can be achieved via model weight perturbation, i.e., finding a 'flattened" local minimum in terms of the target objective is sufficient for robustness.

3 Methodology

In this section, we first analyze the sufficient condition to achieve robust fairness over distribution shift in terms of demographic parity. Based on the analysis, we propose a simple yet effective robust fairness regularization via explicit adopting group model weight perturbation. Note that the proposed robust fairness regularization involves a computation-expensive maximization problem, we further accelerate model training via first-order Taylor expansion.

3.1 Robust Fairness Analysis

We consider binary sensitive attribute A ∈ { 0 , 1 } and demographic parity as fairness metric, i.e., the average prediction gap of model f θ ( · ) for different sensitive attribute groups in target dataset ∆ DP T = | E T 0 [ f θ ( x )] -E T 1 [ f θ ( x )] | , where T 0 and T 1 represent the target datasets for sensitive attribute A = 0 and A = 1 , respectively. However, only source dataset S is available for neural network training. In other words, even though demographic parity on source dataset ∆ DP S = | E S 0 [ f θ ( x )] -E S 1 [ f θ ( x )] | is low, demographic parity on target dataset ∆ DP T may not be guaranteed to be low due to distribution shift.

To investigate robust fairness over distribution shift, we try to reveal the connection between demographic parity for source and target datasets. We bound the demographic parity difference for source and target datasets as follows:

where inequality (a) and (b) hold due to a -b ≤ | a -b | and ∣ ∣ | a -b | -| a ′ -b ′ | ∣ ∣ ≤ | a -a ′ | + | b -b ′ | , respectively, for any a, a ′ , b, b ′ . In other words, in order to minimize demographic parity for target dataset DP T , the objective of DP S minimization is insufficient. The minimization of prediction difference for source and target datasets given sensitive attribute groups A = 0 and A = 1 , defined as ∆ 0 = | E S 0 [ f θ ( x )] -E T 0 [ f θ ( x )] | and ∆ 1 = | E S 1 [ f θ ( x )] -E T 1 [ f θ ( x )] | , are also beneficial to achieve robust fairness over distribution shift in terms of demographic parity.

The bound in Eq. (5) is tight when both condition (a) DP S ≤ DP T and (b) maximum and minimum of value set min { E S 0 [ f θ ( x )] , E T 0 [ f θ ( x )] } , min { E S 1 [ f θ ( x )] , E T 1 [ f θ ( x )] } both are from source or target distribution. Even though conditions (a) and (b) may not hold for the neural network model, we

would like to that our goal is not to obtain a tight bound for demographic parity on target distribution. Instead, we aim to find sufficient conditions to guarantee low demographic parity and such low demographic parity can be achieved in model training without any target distribution information. The proposed upper bound Eq. (5) actually reveals sufficient conditions, which can be achieved by our proposed RFR algorithm.

3.2 Robust Fairness Regularization

Motivated by Section 3.1 and distribution shift understanding in Section 2, we develop a robust fairness regularization to achieve robust fairness over distribution shift. Section 3.1 demonstrates that fair prediction on the target dataset requires fair prediction on the source dataset and low prediction difference between the source and target dataset for each sensitive attribute, i.e., low ∆ 0 = | E S 0 [ f θ ( x )] -E T 0 [ f θ ( x )] | and ∆ 1 = | E S 1 [ f θ ( x )] -E T 1 [ f θ ( x )] | . Based on Theorem 2.3, there exists /epsilon1 0 so that the following equality holds:

Note that the distribution shift is unknown, we consider the worst case for model weight perturbation /epsilon1 0 within L p -norm perturbation ball with radius ρ as follows:

where ‖ · ‖ p represents L p norm, ρ and p are hyperparameters. Note that the feasible region of model weight perturbation /epsilon1 0 is symmetric, and the neural network prediction is locally linear around parameter θ , we have E S 0 [ f θ + /epsilon1 0 ( x )] -E S 0 [ f θ ( x )] ≈ -( E S 0 [ f θ -/epsilon1 0 ( x )] -E S 0 [ f θ ( x )] ) due to the local linearity.In other words, the absolute operation can be removed if we consider the maximation problem in a symmetric feasible region since there are always non-negative value for the pair perturbation /epsilon1 0 and -/epsilon1 0 . Therefore, we can further bound ∆ 0 as

Similarly, we can bound the prediction difference for source and target distribution with sensitive group A = 1 as follows:

Therefore, demographic parity for source and target distribution relation is given by DP T ≤ DP S + L RFR, S 0 + L RFR, S 1 , we propose robust fairness regualarization (RFR) to achieve robust fairness as

It is worth noting that our proposed RFR is agnostic to the training loss function and model architectures. Additionally, we follow [12] to accelerate model training using sharpness-aware minimization via trading maximization problem can be simplified as two forward and two backward propagations. More details on training acceleration are in Appendix E.

3.3 The Proposed Method

In this subsection, we introduce how to use the proposed RFR to achieve robust fairness, i.e., the fair model (low ∆ DP S ) trained on the source dataset will also be fair on the target dataset (low ∆ DP T ). Considering binary sensitive attribute A ∈ { 0 , 1 } and binary classification problem Y ∈ { 0 , 1 } , the classification loss is denoted as

To achieve fairness, we consider demographic parity as fairness regularization, i.e.,

ACS-I

ACS-E

Adult

(1.0, 2.0)

(1.5, 3.0)

Additionally, we also adopt L RFR as robust fairness regularization. The overall objective is given as follows:

where λ is a hyperparameter to balance the model performance and fairness. L CLF is the loss function for the downstream task, L DP is to ensure the demographic parity on the source dataset, and L RFR is to ensure the transferability of fairness from the source dataset to the target dataset.

In the proposed RFR algorithm, we directly apply approximation to accelerate model training. Note that the optimal model weight perturbation is dependent on the current model weight θ , two forward and backward propagations are required to calculate the final gradient.

4 Experiments

In this section, we conduct experiments to evaluate the effectiveness of our proposed RFR, aiming to answer the three research questions. Q1 : How effective is RFR to achieve fairness under distribution shift for synthetic distribution shift? Q2 : How effective is RFR for real spatial and temporal distribution shift? Q3 : How sensitive is RFR to the key hyperparameter λ ?

4.1 Experimental Setting

In this subsection, we present the experimental setting, including datasets, evaluation metrics, baseline methods, distribution shift generation, and implementation details.

Datasets. We adopt the following datasets in our experiments. UCI Adult [13] dataset contains information about 45 , 222 individuals with 15 attributes from the 1994 US Census. We consider

gender as the sensitive attribute and the task is to predict whether the income of the person is higher than $50 k or not. ACS-I ncome [14] is extracted from the American Community Survey (ACS) Public Use Microdata Sample (PUMS) with 3 , 236 , 107 samples. We choose gender as the sensitive attribute. Similar to the task in UCI Adult, the task is to predict whether the individual income is above $50 k . ACS-E mployment [14] also derives from ACS PUMS, and we also use gender as the sensitive attribute. The task is to predict whether an individual is employed.

Evaluation Metrics. We use accuracy to evaluate the prediction performance for the downstream task. For fairness metrics, we adopt two common-used quantitative group fairness metrics to measure the prediction bias [15, 16], i.e., demographic parity ∆ DP = | P ( ˆ Y = 1 | A = 0) -P ( ˆ Y = 1 | A = 1) | and equal opportunity ∆ EO = | P ( ˆ Y = 1 | A = 0 , Y = 1) -P ( ˆ Y = 1 | A = 1 , Y = 1) | , where A represents sensitive attribute, Y and ˆ Y represent the ground-truth label and predicted label, respectively.

Baselines. In our experiments, we consider vanilla multi-layer perceptron (MLP) and two widely adopted in-processing debiasing methods, including fairness regularization (REG), adversarial debiasing (ADV), and fair consistency regularization (FCR). MLP directly uses vanilla 3-layer MLP with 50 hidden unit and ReLU activation function [17] to minimize cross-entropy loss with source dataset. In the experiments, we adopt the same model architecture for all other methods (i.e., REG and ADV). REG adds a fairness-related metric as a regularization term in the objective function to mitigate the prediction bias [18, 19]. Specifically, we directly adopt demographic parity as a regularization term, and the objective function is L CLF + λ L DP , where L CLF and L DP are defined in Eqs. (11) and (12). ADV [20] employs a two-player game to mitigate bias, in which a classification network is trained to predict labels based on input features, and an adversarial network takes the output of the classification network as input and aims to identify which sensitive attribute group the sample belongs to. FCR [21] aims to minimize and balance consistency loss across groups.

ACS-E

ACS-I

2016

2018

MI

CA

Synthetic and Real Distribution Shift. We adopt synthetic and real distribution shifts. For synthetic distribution shift, we follow work [22, 23] to generate distribution shift via biased sampling. Specifically, we adopt applying principal component analysis (PCA) [24] to retrieve the first principal component C from input attributes. Subsequently, we estimate the mean µ ( C ) and standard deviation σ ( C ) , and set Gaussian distribution N S ( µ ( C ) , σ ( C )) to randomly sampling of target dataset. As for source dataset, we choose different parameters α and β to generate distribution shift using another Gaussian distribution N T ( µ ( C ) + α , σ ( C ) β ) for randomly sampling. The source and target datasets are constructed by sampling without replacement. For real distribution shift, we adopt the sampling

and pre-processing approaches following Folktables [14] to generate spatial and temporal distribution shift via data partition based on different US states and year from 2014 to 2018 .

Implementation Details. We run the experiments 5 times and report the average performance for each method. We adopt Adam optimizer with 10 -5 learning rate and 0 . 01 weight decay for all models. For baseline ADV, we alternatively train classification and adversarial networks with 70 and 30 epochs, respectively. The hyperparameters for ADV are set as { 0 . 0 , 1 . 0 , 10 . 0 , 100 . 0 , 500 . 0 } . For adding regularization, we adopt the hyperparameters set { 0 . 0 , 0 . 5 , 1 . 0 , 10 . 0 , 30 . 0 , 50 . 0 } .

4.2 Experimental Results on Synthetic Distribution Shift

In this experiment, we evaluate the effectiveness of our proposed Robust Fairness Regularization (RFR) method with synthetic distribution shifts via biased sampling with different parameters ( α , β ) . We compare the performance of RFR with several other baseline methods, including standard training, adversarial training, and fairness regularization methods. The results of performance value are presented in Table 1, and the results of fairness-accuracy tradeoff are presented in Figure 2. We have the following observations:

The results in Table 1 demonstrate that RFR consistently outperforms the baselines in terms of fairness for small distribution shifts, achieving a better balance between fairness and accuracy. For example, RFR achieves an improvement of 47 . 0% on metric ∆ DP and 84 . 8% on metric ∆ EO compared to the second-best method in the Adult dataset with (1 . 0 , 2 . 0) -synthetic distribution shift. Furthermore, we observed that the outperforms of RFR compared with baselines decrease as the distribution shift intensity increases. The reason is that the approximation RFR involved Taylor expansion over model perturbation and is effective with mild distribution shifts.

The results in Figure 1 show that RFR achieved a better fairness-accuracy tradeoff compared to the baseline methods for mild distribution shift, We observed that our proposed method achieved a better Pareto frontier compared to the existing methods, and the bias can be mitigated with tiny accuracy drop.

The experimental results show that our method can effectively address the fairness problem under mild distribution shift while maintaining high accuracy, outperforming the existing state-of-the-art baseline methods.

4.3 Experimental Results on Real Distribution Shift

In this experiment, we evaluate the performance of RFR on multiple real-world datasets with real distribution shifts. We use ACS dataset in the experiment. The distribution of this dataset varies across different time periods or geographic locations, which also causes fairness performance degradation under the distribution shift. The results are presented in Table 2, and the results of the fairness-accuracy tradeoff are presented in Figure 3. The results show that our method consistently outperforms the baselines across all datasets, achieving a better balance between fairness and accuracy. Specifically, we have the following observations:

Table 2 demonstrates that RFR consistently outperforms the baselines across all datasets in terms of fairness-accuracy tradeoff. This suggests that our method is effective in achieving robust fairness under real temporal and spatial distribution shifts. For example, RFR achieved an improvement of 34 . 9% on metric ∆ DP and 34 . 4% on metric ∆ EO compared to the second-best method in ACS-I dataset with temporal distribution shift.

Figure 3 shows that RFR can achieve better fairness-accuracy tradeoff than baselines, i.e., RFR is particularly effective in addressing the fairness problem on the ACS dataset with source/target data varying across different time periods or geographic locations. This is important for many practical applications, where fairness is a critical requirement, such as in credit scoring, and loan approval.

The variance for spatial shift on ACS-I is higher than that of temporal shift on all methods, which indicates neural networks easily converge to different local minima for spatial distribution shift. The proposed methods can achieve better fairness results for most cases and the tradeoff results clearly demonstrate the effectiveness.

Overall, the experimental results on temporal and spatial distribution shifts further support the effectiveness of our proposed method RFR and its potential for practical applications in diverse settings.

4.4 Hyperparameter Study

In this experiment, we investigate the sensitivity of the hyperparameter λ in Equation L all = L CLF + λ ( L DP + L RFR ) for spatial and temporal distribution shift across different datasets. Specifically, we tune the hyperparameter as λ = { 0 . 0 , 0 . 1 , 0 . 5 , 1 . 0 , 3 . 0 , 5 . 0 , 10 . 0 } . From the results in Figure 4, it is seen that the accuracy and demographic parity are both sensitive to hyperparameter λ , which implies the capability of accuracy-fairness control. With the increase of λ , accuracy decreases while demographic parity also decreases. When λ is smaller than 5 , accuracy drops slowly while demographic parity drops faster. Such observation represents that an appropriate hyperparameter can mitigate prediction bias while preserving comparable prediction performance.

5 Related Work

In this section, we present two lines of related work, including fairness in machine learning and distribution shift.

Fairness in Machine Learning. Fairness [25 ? -31] is a legal requirement for machine learning models for various high-stake real-world predictions, such as healthcare [7, 32, 33], education [3436], and job market [37, 38]. Achieving fairness, either from a data or model perspective [39-42], in machine learning is a challenging problem. As such, there has been an increasing interest in both the industrial and research community to develop algorithms to mitigate these issues and ensure that machine learning models make fair and unbiased decisions. Extensive efforts led to the development of various techniques and metrics for fairness and proposed various definitions of fairness, such as group fairness [43-47, 40], individual fairness [48-53], and counterfactual fairness [54-56]. In this paper, we focus on group fairness, and the widely used methods to achieve group fairness are fair regularization and adversarial debias method. [19, 18] proposed to add a fairness regularization term

to the objective function to achieve group fairness, and [20] proposes to jointly train a classification network and an adversarial network to mitigate the bias for different demographic groups to achieve group fairness. Overall, ensuring fairness in machine learning is a critical and ongoing research area that will continue to be an important focus for the development of responsible machine learning.

Distribution Shift. Previous work [1-4, 57] reveals that a classifier trained on a source distribution will perform worse on a given target distribution because of the distribution shift, and recently extensive works have explored the influence of distribution shift in model prediction. Moreover, distribution shift can significantly affect the fairness performance of machine learning models. The sensitivity of fairness to the distribution shift is notorious for the legal requirement. The degraded performance of fair models under a distribution shift would trigger new bias and discrimination issues. There are some works [23, 58, 21, 59, 60] that solve fairness under various distribution shifts. For example, [23] explore the fairness under covariate shift, where the inputs change with the in-distribution label. [60] proposes Shifty algorithms to hold fairness guarantees when the dataset in the deployment environment is out-of-distribution of the training datasets (distribution shift). Our work is different from those prior works from model weight perturbation perspective.

6 Conclusion

This paper aims the solve the fairness problem under the distribution shifts from the model weight perturbation perspective. We first establish a theoretical connection between distribution shift, data perturbation, and model weight perturbation, which allowed us to conclude that distribution shift and model perturbation are equivalent. We then propose a sufficient condition for ensuring fairness transference under distribution shift. To explicitly chase such sufficient conditions, we introduce Robust Fairness Regularization (RFR) method based on the established understanding, to achieve robust fairness. Our experiments on both synthetic and real distribution shifts demonstrate the effectiveness of RFR in achieving a better fairness-accuracy tradeoff compared to existing baselines. We believe that our understanding of distribution shift is valuable and intriguing to the development of robust machine learning models, and the proposed RFR approach can be of great practical value to build fair and robust machine learning models in real-world applications.

References

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 , 2019.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8340-8349, 2021.

Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Information Processing Systems , 33:18583-18599, 2020.

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems , 33:11539-11551, 2020.

L Elisa Celis, Anay Mehrotra, and Nisheeth Vishnoi. Fair classification with adversarial perturbations. Advances in Neural Information Processing Systems , 34:8158-8171, 2021.

Songül Tolan, Marius Miron, Emilia Gómez, and Carlos Castillo. Why machine learning may lead to unfairness: Evidence from risk assessment for juvenile justice in catalonia. In Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law , pages 83-92, 2019.

Muhammad Aurangzeb Ahmad, Arpit Patel, Carly Eckert, Vikas Kumar, and Ankur Teredesai. Fairness in machine learning for healthcare. In Proceedings of the 26th ACM SIGKDD , pages 3529-3530, 2020.

Jeff Johnson, Donald M Truxillo, Berrin Erdogan, Talya N Bauer, and Leslie Hammer. Perceptions of overall fairness: are effects on job performance moderated by leader-member exchange? Human Performance , 22(5), 2009.

Ainhize Barrainkua, Paula Gordaliza, Jose A Lozano, and Novi Quadrianto. A survey on preserving fairness guarantees in changing environments. arXiv preprint arXiv:2211.07530 , 2022.

Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre-Alvise Rebuffi, Ira Ktena, Krishnamurthy Dj Dvijotham, and Ali Taylan Cemgil. A fine-grained analysis on distribution shift. In International Conference on Learning Representations , 2022.

Cédric Villani et al. Optimal transport: old and new , volume 338. Springer, 2009.

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 , 2020.

Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.

Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. NeurIPS , 2021.

Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard Zemel. The variational fair autoencoder. arXiv preprint arXiv:1511.00830 , 2015.

Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H Chi. Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075 , 2017.

Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML , 2010.

Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Joint European conference on machine learning and knowledge discovery in databases . Springer, 2012.

Ching-Yao Chuang and Youssef Mroueh. Fair mixup: Fairness via interpolation. In ICLR , 2020.

Gilles Louppe, Michael Kagan, and Kyle Cranmer. Learning to pivot with adversarial networks. NeurIPS , 30, 2017.

Bang An, Zora Che, Mucong Ding, and Furong Huang. Transferring fairness under distribution shifts via fair consistency regularization. arXiv preprint arXiv:2206.12796 , 2022.

Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning , 3(4):5, 2009.

Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian D Ziebart. Robust fairness under covariate shift. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9419-9427, 2021.

Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics , 2(4):433-459, 2010.

Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) , 54(6):1-35, 2021.

Dana Pessach and Erez Shmueli. A review on fairness in machine learning. ACM Computing Surveys (CSUR) , 55(3):1-44, 2022.

Michael Madaio, Lisa Egede, Hariharan Subramonyam, Jennifer Wortman Vaughan, and Hanna Wallach. Assessing the fairness of ai systems: Ai practitioners' processes, challenges, and needs for support. Proceedings of the ACM on Human-Computer Interaction , 6(CSCW1):1-26, 2022.

Dexun Li and Pradeep Varakantham. Efficient resource allocation with fairness constraints in restless multi-armed bandits. In James Cussens and Kun Zhang, editors, Uncertainty in Artificial Intelligence, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, UAI 2022, 1-5 August 2022, Eindhoven, The Netherlands , volume 180 of Proceedings of Machine Learning Research , pages 1158-1167. PMLR, 2022.

Kirtan Padh, Diego Antognini, Emma Lejal Glaude, Boi Faltings, and Claudiu Musat. Addressing fairness in classification with a model-agnostic multi-objective algorithm. In Cassio P. de Campos, Marloes H. Maathuis, and Erik Quaeghebeur, editors, Proceedings of the ThirtySeventh Conference on Uncertainty in Artificial Intelligence, UAI 2021, Virtual Event, 27-30 July 2021 , volume 161 of Proceedings of Machine Learning Research , pages 600-609. AUAI Press, 2021.

Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, and Xia Hu. Fmp: Toward fair graph message passing against topology bias. arXiv preprint arXiv:2202.04187 , 2022.

Xiaotian Han, Zhimeng Jiang, Hongye Jin, Zirui Liu, Na Zou, Qifan Wang, and Xia Hu. Retiring ∆ DP: New distribution-level metrics for demographic parity. Transactions on Machine Learning Research , 2023.

Margrét Vilborg Bjarnadóttir and David Anderson. Machine learning in healthcare: Fairness, issues, and challenges. In Pushing the Boundaries: Frontiers in Impactful OR/OM Research , pages 64-83. INFORMS, 2020.

Thomas Grote and Geoff Keeling. Enabling fairness in healthcare through machine learning. Ethics and Information Technology , 24(3):39, 2022.

Steinar Bøyum. Fairness in education-a normative analysis of oecd policy documents. Journal of Education Policy , 29(6):856-870, 2014.

Paolo Brunori, Vito Peragine, and Laura Serlenga. Fairness in education: The italian university before and after the reform. Economics of Education Review , 31(5):764-777, 2012.

René F Kizilcec and Hansol Lee. Algorithmic fairness in education. In The ethics of artificial intelligence in education , pages 174-202. Routledge, 2022.

Lily Hu and Yiling Chen. A short-term intervention for long-term fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference , pages 1389-1398, 2018.

G Stoney Alder and Joseph Gilbert. Achieving ethics and fairness in hiring: Going beyond the law. Journal of Business Ethics , 68:449-464, 2006.

Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. Data-centric artificial intelligence: A survey. arXiv preprint arXiv:2303.10158 , 2023.

Qizhang Feng, Zhimeng Jiang, Ruiquan Li, Yicheng Wang, Na Zou, Jiang Bian, and Xia Hu. Fair graph distillation. 2023.

Chia-Yuan Chang, Yu-Neng Chuang, Zhimeng Jiang, Kwei-Herng Lai, Anxiao Jiang, and Na Zou. Coda: Temporal domain generalization via concept drift simulator. arXiv preprint arXiv:2310.01508 , 2023.

Chia-Yuan Chang, Yu-Neng Chuang, Kwei-Herng Lai, Xiaotian Han, Xia Hu, and Na Zou. Towards assumption-free bias mitigation. arXiv preprint arXiv:2307.04105 , 2023.

Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. NeurIPS , 29, 2016.

Sahil Verma and Julia Rubin. Fairness definitions explained. In 2018 ieee/acm international workshop on software fairness (fairware) , pages 1-7. IEEE, 2018.

Peizhao Li, Yifei Wang, Han Zhao, Pengyu Hong, and Hongfu Liu. On dyadic fairness: Exploring and mitigating bias in graph connections. In International Conference on Learning Representations , 2020.

Gaurush Hiranandani, Jatin Mathur, Harikrishna Narasimhan, and Oluwasanmi Koyejo. Quadratic metric elicitation for fairness and beyond. In James Cussens and Kun Zhang, editors, Uncertainty in Artificial Intelligence, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, UAI 2022, 1-5 August 2022, Eindhoven, The Netherlands , volume 180 of Proceedings of Machine Learning Research , pages 811-821. PMLR, 2022.

Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Ali Mostafavi, and Xia Hu. Generalized demographic parity for group fairness. In International Conference on Learning Representations , 2022.

Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair ml models with sensitive subspace robustness. International Conference on Learning Representations , 2020.

Debarghya Mukherjee, Mikhail Yurochkin, Moulinath Banerjee, and Yuekai Sun. Two simple ways to learn individual fairness metrics from data. In ICML , 2020.

Mikhail Yurochkin and Yuekai Sun. Sensei: Sensitive set invariance for enforcing individual fairness. In International Conference on Learning Representations , 2021.

Jian Kang, Jingrui He, Ross Maciejewski, and Hanghang Tong. Inform: Individual fairness on graph mining. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 379-389, 2020.

Debarghya Mukherjee, Felix Petersen, Mikhail Yurochkin, and Yuekai Sun. Domain adaptation meets individual fairness. and they get along. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Shantanu Das, Swapnil Dhamal, Ganesh Ghalme, Shweta Jain, and Sujit Gujar. Individual fairness in feature-based pricing for monopoly markets. In James Cussens and Kun Zhang, editors, Uncertainty in Artificial Intelligence, Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, UAI 2022, 1-5 August 2022, Eindhoven, The Netherlands , volume 180 of Proceedings of Machine Learning Research , pages 486-495. PMLR, 2022.

Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. NeurIPS , 30, 2017.

Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitnik. Towards a unified framework for fair and stable graph representation learning. In UAI 2021: Uncertainty in Artificial Intelligence , 2021.

Aoqi Zuo, Susan Wei, Tongliang Liu, Bo Han, Kun Zhang, and Mingming Gong. Counterfactual fairness with partially known causal graph. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Chia-Yuan Chang, Yu-Neng Chuang, Guanchu Wang, Mengnan Du, and Zou Na. Dispel: Domain generalization via domain-specific liberating. arXiv preprint arXiv:2307.07181 , 2023.

Jessica Schrouff, Natalie Harris, Oluwasanmi Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alex Brown, Subhrajit Roy, Diana Mincu, Christina Chen, et al. Maintaining fairness across distribution shift: do we have viable solutions for real-world applications? arXiv preprint arXiv:2202.01034 , 2022.

Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Fairness violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , pages 3-13, 2021.

Stephen Giguere, Blossom Metevier, Bruno Castro da Silva, Yuriy Brun, Philip Thomas, and Scott Niekum. Fairness guarantees under demographic shift. In International Conference on Learning Representations , 2022.

Jiongli Zhu, Sainyam Galhotra, Nazanin Sabri, and Babak Salimi. Consistent range approximation for fair predictive modeling. Proceedings of the VLDB Endowment , 16(11):2925-2938, 2023.

Jiawei Du, Hanshu Yan, Jiashi Feng, Joey Tianyi Zhou, Liangli Zhen, Rick Siow Mong Goh, and Vincent YF Tan. Efficient sharpness-aware minimization for improved training of neural networks. International Conference on Learning Representations , 2022.

Maksym Andriushchenko and Nicolas Flammarion. Towards understanding sharpness-aware minimization. In International Conference on Machine Learning , pages 639-668. PMLR, 2022.

Yuzhe Lu, Zhenlin Wang, Runtian Zhai, Soheil Kolouri, Joseph Campbell, and Katia Sycara. Predicting out-of-distribution error with confidence optimal transport. arXiv preprint arXiv:2302.05018 , 2023.

Yaodong Yu, Zitong Yang, Alexander Wei, Yi Ma, and Jacob Steinhardt. Predicting out-ofdistribution error with the projection norm. In International Conference on Machine Learning , pages 25721-25746. PMLR, 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the issue of fairness in situations where there are distribution shifts, viewing it from the angle of model weight perturbation. The authors establish a theoretical link between distribution shift, data perturbation, and model weight perturbation, leading to the conclusion that distribution shift and model perturbation are essentially the same. They propose a sufficient condition to ensure fairness transference under distribution shift and introduce the Robust Fairness Regularization (RFR) method to meet these conditions and achieve robust fairness. Experimental results on both synthetic and real distribution shifts show the RFR method outperforms existing baselines in terms of fairness-accuracy tradeoff.\n\nSTRENGTHS:\n1. The fairness under covariate shift is an important problem. The authors also manage to show that Data Perturbation equals Model Weight Perturbation and use this motivation to construct the following methodology descriptions and theoretical analysis. The motivation and writing is strong.   \n2. The method proposed by this work is novel about the insights from optimal transport to optimize the worst case within the model weight perturbation ball and Data Perturbation equals Model Weight Perturbation. This paper points out the limitations of previous works well.  \n3. The experiments on fairness accuracy tradeoff and hyperparameter study are sufficient. Although in the table it does not explicitly outperforms baselines, the tradeoff is clearly better.\n\nWEAKNESSES:\nI do not find important weaknesses on this paper.\n\nQUESTIONS:\nThe hyperparameter tuning seems important. Do you record the experimental setting variables and preferably provide means to reimplement the result?\n\n----------\nI have seen other reviews and am a little confused and quite frankly appalled why we need to assume zero access to data from the target distribution. Is it even possible if we do not even know data from the target distribution (or the distribution variation) to do transfer learning? Although it would be interesting to know a good $\\rho$ a priori by either prior knowledge or theoretical bounds, many papers out there also do not discuss hyperparameters like $\\rho$, and they are all fine. I fail to see why this could be a MAJOR problem. In my view, it is not even a weakness, but something that would simply be good to know and good to ask the authors about. I would like to raise my confidence to 4 and advocate the paper if needed.\n\nLIMITATIONS:\nThe authors do not talk about limitations of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is in the context of fairness measures under distribution shift. \nThere is first a theoretical connection between distribution shift, data perturbation and model perturbations.\nThe proposed method Robust Fairness Regularizatin (RFR), proposes a regularization using 3 terms L_CLF-- classifier loss function, L_DP ensuring DP on source data and L_RFR= L_RFR_S0 +  L_RFR_S1 which is the fairness transferability from training set.\nThe experiments are in  synthetic (PCA transformations) and real data (Adult and Folk)\n\nSTRENGTHS:\nThe paper is extremely well-written and clear. Enjoyable lecture.\n\nThe method proposed is simple. \n\nThe work seems extensive and thorough. \n\nExperiments are well-redacted and easy to understand.\n\nThe work is one of the few (to the best of my knowledge) that are tackling distribution shift and fairness, without strong assumptions like causal graph availability.\n\nWEAKNESSES:\nThe method is designed/tested in Neural Network architectures in tabular datasets. In tabular data algorithms like gradient boosting decision trees achieve SOTA[1] and are restricted to Linear Regression in regulated environments. \n\n\n\nThe paper calls fairness to just one metric --Demographic Parity-- Fairness is not just one metric. L_RFR is not performance based but group disparate treatment based. \n\nThe work does not acknowledge the difference between algorithmic fairness and fairness in broader socio-technical systems.\n\n\n\n## Minor\n\nL72 Citation [9] is a specific \"fairness distribution shift\" citation, not a general one about distribution shift, which is what is about the sentence. \n\nI wonder how different are actually Adult and ACS Income.\n\nThere seem to be many citations coming from Arxiv. Perhaps it could be good to check if there are more recent proceedings versions. \n\n\n[1] https://proceedings.neurips.cc/paper_files/paper/2022/file/0378c7692da36807bdec87ab043cdadc-Paper-Datasets_and_Benchmarks.pdf\n\nQUESTIONS:\nThe work states that \"any distribution shift can be equivalent to data perturbations.\" Does this approach (Sec2.2) considers pure concept shift? When $P(X_S)=P(X_T)$ and $P(Y_S)=P(Y_T)$, but the only thing that changes is the relationship between covariate and target? $X\\Rightarrow Y$.\n\nIs the method strictly limited to NN? If so (since its tabular data) it may be worth stating it earlier and clearer in the paper. \nI wonder if the theoretical results from section 2 can be aligned with the previous empirical findings of https://arxiv.org/pdf/2302.05018.pdf\n\nLIMITATIONS:\nFairness is just one metric. I wonder if this limitation should be stated early on the paper or even affect the title. \n\nThe work does not acknowledge the difference between algorithmic fairness and fairness in broader socio-technical systems.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

comments: SUMMARY:
theoretical proof: prove that distribution shifting can be written as data perturbation, and the model trained with the perturbed data can be written as the perturbed model trained with original source data.
methodology: motivated by the connection between distribution shifting and data weight perturbation, the author proposed RFR(robust fairness regularization), which is an upper bound of the difference of demographic parity between source and target.

STRENGTHS:
well written; easy to follow. 
well-structured and coherent framework that facilitates easy comprehension.

WEAKNESSES:
- I think that the objective function which the author suggested did not reflect the true effect of $L_{RFR}$. Please add the experiments without $L_{DP}$
- It would be better if the notation $\delta_X, \delta_Y$ were changed. The author writes $\delta_X, \delta_Y$ to denote the perturbation for feature and label respectively, however, in equation (3), it seems like they depend on each random variable X and Y. It can make confusion for readers.
- some typos: for example, line 495(appendix), $E_{\delta_Y}[\delta_X]$ → $E_{\delta_X}[\delta_X]$, line 152 $\|\cdot\|_2$ →$\| \cdot\|_p$

QUESTIONS:
- I cannot understand the proof of theorem 2.3, lines 499-501; is that sufficient? Please give more details.
- I think it would be better if the graph of EO was also included in appendix.

LIMITATIONS:
refer weakness and questions.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors seek to learn neural networks that are fair not just w.r.t. the observed distribution of individuals, but also under shifts to this distribution. In order to achieve this, they propose robust fairness regularization (RFR) which uses adversarial weight-space perturbations at train time to create more robustly fair models (w.r.t. distribution shifts). RFR being a reliable regularizer for distribution shifts relies on a correspondence between distribution and point-wise data perturbations as well as a correspondence between point-wise data perturbations and model weight perturbations. Finally, the paper proposes that for each group, the difference between the predictive performances on the source and target distribution should be the same. By enforcing this over all groups, the authors have the RFR loss that they propose.\n\nSTRENGTHS:\nThe paper focuses on a very important topic for trustworthy ML. \n\nThe paper is well-written and the notation is clear and easy to follow.\n\nThe experimental results that are reported are strong and at the core evaluate the method well, though I have some suggestions below in the weaknesses for improvements here.\n\nWEAKNESSES:\nMajor weakness: The major weakness that I see with this paper is the underlying theoretical motivation. In particular Theorem 2.3 appears to have a critical flaw that is never addressed in the paper (as far as I can tell). I elaborate further at the bottom of this section, but the key point is this, though there exists weight perturbation that give us the same loss as any arbitrary input perturbations, there is no given relationship between the magnitude of the perturbations in either domain. Therefore, even given that you know the Wasserstein distance is bounded by some value $\\epsilon$ there is no way to select the value of $\\rho$ before you have observed the target distribution, rendering the method difficult to use in practice. Moreover, I find it troubling that this is (1) not addressed in the paper, and (2) the authors do not explain how they select $\\rho$ either theoretically or experimentally. It is obvious to me that given access to the target distribution, one can optimize $\\rho$ to give good results, but this is the same as training on your test distribution and claiming good results. Without a clear and theoretically grounded argument for selecting $\\rho$ and an open source implementation showing that this works well across datasets, I do not think I can vote to accept this paper. \n\n\nAnother weakness I see, is that the authors do not compare their weight perturbation method to the corresponding input perturbation method (as far as I can see). The ADV method used in the experiments refers to methods like LAFTR which are adversarial, but not in the same spirit as crafting worst-case perturbations.\n\n\nElaborated major weakness: The  first claim (Corollary 2.2)  is that there exists a perturbation map that makes the classification loss between source, $P_S$, and target $P_T$ the same. This statement seems clear and intuitive; however, we do not, a priori know how the magnitude by which we will have to perturb each data point. Let us assume, for sake of argument, that we know a priori that the Wasserstein distance between $P_S$ and $P_T$ is bounded by a real value $\\epsilon > 0$. Then for an empirical distribution consisting of $n$ atoms, we know that we won’t perturb any given point by more than $n\\epsilon$, so this gives us some practical guidance on how to set the data perturbation value, assuming we know a bound on the Wasserstein distance. The second claim made by the paper, Theorem 2.3, is that there exists a weight perturbation such that the model has the same loss on the source distribution (after its weights have been perturbed) as it does on the target distribution. However, though we may have a bound on how large the input perturbation may be, there is absolutely no correspondence between the magnitude of input perturbations, $n\\epsilon$ and the magnitude we should set for the weight space perturbations $\\rho$.\n\nQUESTIONS:\nPlease address my key point on how one picks $\\rho$ in a principled way.\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

comments: SUMMARY:
The paper studies the problem of training a fair ML model in the presence of distribution shift. The proposed method aims to bound the biases on the target distribution and use it as a regularizer.

STRENGTHS:
S1 The problem of distribution shift is quite fundamental.
S2 The idea of bounding the biases and using it as a regularization term is underexplored.

WEAKNESSES:
W1 The approach of using the worst-case bound as a regularizer has been explored before for selection bias [1]. Please discuss the differences with this approach.

W2 The experiments do not explain the observations in detail. Specifically, the variance is quite high for spatial shift. Are the results staistically significant? Why are the baselines doing worse?

It would be worth adding an experiment where source and target are exact same. I would expect the baselines to perform much better in this case. More generally, only when the source and target are very different, I expect RFR to perform better than the rest. The current experiments do not seem to clarify any such aspect.

W3 Please show how loose are the estimated bounds. Are the bounds tight?

W4 If possible, consider adding proofs or sketch of theorems. 
[1] https://arxiv.org/pdf/2212.10839.pdf

QUESTIONS:
Why does RFR not have the best fairness for spatial distribution shift? Is it because lambda is small?

LIMITATIONS:
The paper does not discuss the limitations of the approach. I think the loose bound of fairness estimates can adversely affect model accuracy. Authors should discuss the practical implications of the presented approach.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-27