Transferring Fairness under Distribution Shifts via Fair Consistency Regularization
Authors: Bang An
Zora Che
Mucong Ding
Furong Huang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Fairness
Distribution Shifts
Self-training
Transfer Learning

Abstract: The increasing reliance on ML models in high-stakes tasks has raised a major concern about fairness violations. Although there has been a surge of work that improves algorithmic fairness, most are under the assumption of an identical training and test distribution. In many real-world applications, however, such an assumption is often violated as previously trained fair models are often deployed in a different environment, and the fairness of such models has been observed to collapse. In this paper, we study how to transfer model fairness under distribution shifts, a widespread issue in practice. We conduct a fine-grained analysis of how the fair model is affected under different types of distribution shifts and find that domain shifts are more challenging than subpopulation shifts. Inspired by the success of self-training in transferring accuracy under domain shifts, we derive a sufficient condition for transferring group fairness. Guided by it, we propose a practical algorithm with fair consistency regularization as the key component. A synthetic dataset benchmark, which covers diverse types of distribution shifts, is deployed for experimental verification of the theoretical findings. Experiments on synthetic and real datasets, including image and tabular data, demonstrate that our approach effectively transfers fairness and accuracy under various types of distribution shifts.

Transferring Fairness under Distribution Shifts via Fair Consistency Regularization
Bang An Department of Computer Science University of Maryland, College Park bangan@umd. edu
Zora Che
Department of Computer Science
Boston University zche@bu.
Mucong Ding
Furong Huang
Department of Computer Science University of Maryland, College Park mcding@umd. edu
Department of Computer Science University of Maryland, College Park furongh@umd.com edu
Abstract
The increasing reliance on MI. models in high-stakes tasks has raised major concern about fairness violations. Although there has been a surge of work that improves algorithmic fairness, most are under the assumption of an identical training and test distribution. In many real-world applications, however, such an assumption is often violated as previously trained fair models are often deployed in a different environment, and the fairness of such models has been observed collapse. In this paper, we study how to transfer model fairness under distribution shifts, widespread issue practice. We conduct fine- mnee-graied analysis of how the fair model is affected under different types of distribution shifts and find that domain shifts are more challenging than subpopulation shifts. Inspired by the success of self-training in transferring accuracy under domain shifts, we derive a sufficient condition for transferring group fairness. Guided by it, we propose practical algorithm with fair consistency regularization as the key component. A synthetic dataset benchmark, which covers diverse types of distribution shifts, is deployed for experimental verification of the theoretical findings. Experiments on synthetic and real datasets, including image and tabular data, demonstrate that approach effectively transfers fairness and accuracy under various types of distribution shifts!
Introduction
Machine learning's 's social impact has broadened as it is widely used to aid decision-making in real-world applications, such as hiring, loan approval, facial recognition, and criminal justice. To avoid discrimination against a subset of the population (e.g., w.r.t race or gender), many efforts on algorithmic fairness have been carried out [12, 21, 65, 44, 46, 15, 7]. Although existing work has achieved remarkable success in ensuring fairness, most of them assume the distribution data at test time is identical to that in the training set. However, recent studies show that the fairesss of model is likely to collapse when encountering a distribution shift. For example, [19] observes that a fair income predictor trained with data from state might not be fair when used in other states. [50] tries to maintain fairness in healthcare settings, but model that performs fairly according to the metric evaluated in "Hospital A" shows unfaimess when applied to "Hospital B" Such observations
Cooe is available at https///ithub..cm//ubb
hab//raasfer-fairee
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

2
graup
Source
Target
Sampled from
UTKFace [71]
FairFace [30]
exparaioo
Class
Gender
Sensitive attribute
Race
Group
Gender Race
Shared
Underlying data generation process
group
Differ
Capturing-bian bius, e.g. diff angles, facial expressions
Figure 1: Illustration of intra-group expansion assumption in the input space. An example of gender classification task with the sensitive attribute being race. Intra-group expansion assumes that different groups are separated but every group is self-connected under certain transformations. If model has consistent predictions under those transformations, we can propagate labels within each group. Under this assumption, we propose obtain faimess and accuracy in both domains by self-training algorithm with fair consistency regularization.
motivate us to find the reason behind the collapse of fairness and investigate how to transfer faireess under distribution shifts. Specifically, when we have labeled data in the source domain and unlabeled data in the target domain, we investigate how to adapt the fair source model to target domain with the goal of achieving both accuracy and fairness both domains.
Intuitively, the fairness of a model in the target domain strongly depends on the nature of distribution shifts. In this paper, we only consider cases where the oracle model is the same in two domains. We characterize distribution shifts by assuming two domains share the same underlying data generation process where data is generated from set of latent factors with fixed generative model, and the shift is caused by the shift of the marginal distribution of some factors. We categorize distribution shifts into three types [32]: 1) Domain shift where source and target distributions comprise data from related but distinct domains (e.g.. train model in hospital A but test it in hospital B). 2) Subpopulation shift where two domains overlap, but relative proportions of subpopulations differ (e.g., the proportion of female candidates increases at test time). 3) Hybrid shift where domain shift and subpopulation shift happen at the same time. We find domain shift more challenging for transferring fairness since the model's 'ss performance is unpredictable in unseen domains. Such finding is supported empirically on a synthetic dataset that is developed to simulate diverse types of distribution shifts. While recent work explores methods to transfer fairness [54, 48, 23], most considered settings fall into subpopulation shifts. In this paper, we consider all three types of distribution shifts. Our analysis suggests we encourage consistent fairness under different factor values.
We draw inspiration from recent progress on self-training in transferring accuracy under domain shifts [61, 5, 70, 3, 49, 55]. The success of self-training is due to an expansion axsumption and consistency regularization algorithm. The expansion assumption also assumes two domains share one underlying generative model and the support of the distribution on each class is connected compact set under data transformations (i.c., has a good continuity). Under the expansion assumption, [61] and [5] prove that self-training, which enforces consistent predictions for the same input under different transformations (i.e., under shifts of nuisance factors), can propagate labels from the source to the target domain. This approach exhibits superior performance in transferring accuracy [70, 49], but does not consider fairness.
Taking demography into consideration, we relax the expansion assumption to more realistic intra- group expunsion assumption, as shown in Figure I, which only requires continuity of the underlying distribution within every group (i.e., data points with the same class and sensitive attribute) rather than the entire class. Based on the intra-group expansion assumption, we derive a sufficient condition that guarantees fairness in both source and target domains. This sufficient condition suggests that ensuring the trained model gains the same consistency across groups under a fair teacher classifier guarantees fairness in both domains. However, such teacher classifier is not available in practice, and we need a practical treatment.
Guided by the theoretical algorithm, we propose practical self-training algorithm to minimize and balance consistency loss across groups. Our algorithm builds upon Laftr [42], an adversarial learning method for fairness, and FixMatch [55], a self-training framework. To encourage similar consistency different groups, we propose a novel fair consistency regularization. By reweighting the consistency loss of each group dynamically according to the model's performance, the algorithm
2

---

encourages the model to pay more attention to the high-error group while training. Our method results in a model that is fair in source and has similar consistency across groups. As indicated by our theory, it would have similar accuracy across groups in the target domain so that we can transfer fairness. We evaluate our method under different types of distribution shifts with the synthetic and real datasets. Experiments show that our approach achieves high accuracy and fairness in the target domain without sacrificing performance in the source domain. To the best of our knowledge, this is the first work using self-training to transfer fairness under distribution shifts.
Summary of contributions: (1) We provide a fine-grained analysis of fairness under distribution shifts and develop a synthetic dataset to study model faimess under different types of distribution shifts. (2) Theoretically, we derive sufficient condition for transferring fairness under distribution shifts. (3) Algorithmically, we propose
algorithm for transferring fairness with a fair consistency regularization as the key component. (4) Experimentally, we evaluate our method on synthetic data, real image data, and real tabular data. All results show the effectiveness of our approach in transferring fairness.
2 Preliminaries and Notations
Transfer Fairness. Let A, Y and X', A, denote random variables and sample space of input features, sensitive attribute, and label. For simplicity, we assume binary sensitive attribute and binary classification, while our method can easily extend to multi-sensitive attributes and multi-class cases (see Appendix E). We aim learn a classifier g : X + y and are interested in its fairness under distribution shifts. Specifically, with S and T denoting source and target domains, we study how to transfer fairness and accuracy when IPs (Ps(X, A, Y) * (X, A, Y), with the access to X, A, Y in the source domain, but only X, A in the target domain. In the self-training algorithm, we use Itc to denote a teacher classifier, and g" to denote the oracle classifier. We use the word "group" to denote the set of data that has the same label and sensitive attribute.
Fairness Metric. Since we consider classification problems in this paper, we expect the fairness metrics could encourage models to achieve similar classification performance across groups We use two metrics in this paper, equalized odds and variance of group accuracy. Equalized odds [27] is widely used unfairness metric in classification problems that requires the true positive rate and the true negative rate to be the same among groups. It is defined as Aodds
Σ |P(Y y) 0,
3) PÝ = y|A 1,Y v)}. where Ý = 9(X) is the prediction. Additionally, we also evaluate the variance group accuracy which is defined as Vacc Var (P(() v a, Y = y), Va, y) Smaller Vace indicates the model is fairer since it performs similarly across groups. Note that the variance of group accuracy can help avoid trivial fairness where a model with constant output has Aodds = 0, but such fairness is meaningless.
3 Fairness under Distribution Shifts
In this section, provide fine-grat fine-grained analysis of faimess under various types of distribution shifts based on a unified framework of distribution shift characterization.
A Unified Framework to Characterize Distribution Shifts. Following [62]. we characterize distribution shifts by assuming a unified latent variable model for the underlying data generation process. We denote the underlying factors as Y Y ..., Y and data point as X. Two of the factors are label Y (i.e. Y) and sensitive attribute Y (i.c. A). We call other factors nuisance sissanceefcctrrs since they are irrelevant to the classification task.
Assumption 1. (Underlying data generation process) We assume the data is generated from latent generative model as y l:K ~ P(y and x ~ PX -yl:K). The generative model is fixed Ps(X y l:K) Pa (X =y" ) but the marginal distribution of factors varies in two domains (t + Pr (yl:k) causing the distribution shift PY 1:K ,X) + (Y 1:K XX)
realistic to assume two domains share the same data generation process. For example, the underlying physical process of cell imaging is fixed, while the distribution of underlying factors (e.g. gender, age or equipment) may vary in two hospitals (i.e. two domains), resulting in the distribution shift of the observed tissue images. Based on the unified framework, consider two major types
3

---

of distribution shifts, namely subpopulation shift and domain shift, which are widely considered in many practical applications [32].
Definition 3.1. (Subpopulation shift) We say it is a subpopulation shift, if for any factor Y the sample space of it remains the same in two domains (i.e., V'' = =Y,). but the marginal distribution of at least one factor changes (e.g., (Y)) # Pr(Y")), resulting in Ps 1: ) + P PY 1: and
(Y )) + Pr (Y .)).
Definition 3.2. (Domain shift) We say it a domain shift, if at least one nuisance factor Y i o 1, o a, has different sample space in two domains, Byy E 3r,, but y¹ & V's. resulting in Is(( 111)) of Pr(YL: and Ps PYLL ,) + (T(Y1 ,X)
Intuitively, under subpopulation shift, the sample space overlaps, and only the marginal distributions of factors vary in the two domains. For example, the proportion of females versus males in training and deployment time differs. In contrast, under domain shift, the source model has never seen the data with factor values that only exist in the target domain. For instance, the source model is unaware of the equipment used for cell imaging at deployment time.
Why do distribution shifts cause unfairness? Suppose the marginal distributions of a binary nuisance factor Y differ in two domains with Pss(Y") * (Y+). The unfairness in two domains are
A odds 1s'ss'' 0) x A r'= + Ps(Y' = 1) x Addd
(1)
odds PY = 0) x A odds Aaddly + (Y' 1) x
Due to the same generation process where Ps(XIy* = y²) = Pr = y²), we have odds y-=y', Vy' E {0. 1). Under subpopulation shift, Y has the same sample space in two domains but with different proportions (e.g.. Ps(Y'= 0) = 0.9, P. 1) = 0.1, PT = 0) = 0.1. Pr = 1) =09), while under domain shift the sample space differs (e.g. Ps = 0) 1, 1,Pr(Y' 1) = 1). is easy to see from (1) that if model is perfectly fair on data with Y 0 but unfair data with Y 1, then the model is highly fair in the source domain but highly unfair in the target domain under both cases. Therefore, the model has inconsistent performance on data generated from different nuisance factor values, then the shifted marginal distribution of those factors may cause fairness collapse.
A ooddll''=y' = A A
How to transfer fairness under distribution shifts? Based on the above analysis, one way is to train the model be fair under any values of factors. is possible under subpopulation shift as stated in the following proposition (see proof and discussion in Appendix B).
Proposition 3.1. (Transfer fairness under subpopulation shift) Consider the subpopulation shift that caused by the shifted marginal distribution of nuisance factor Y (i.e., (Y' o (Y while
O's yr- = >>* If model f is strictly fair in source domain under any value of factor Y satisfying
(x( = y 0, O,Y = y². Y = y²) = Ps(o(XX= = y' = 1, 5¹ Y = v²), Vyy
,", s E {0,1) then model is alsofairr in target domain with Andds 0.
Our empirical results (Figure 3) also support this finding. However, domain shift is more challenging. The source model's performance on target data is unpredictable due to the distinct sample space. One promising way to tackle domain shift is to enforce the model's 's invariance to nuisance factors so that the source model would have the same behavior on target data. Note that this solution also works for subpopulation shift since it leads to the case in Proposition 3.1 directly. The above analysis motivates us to transfer fairness by encouraging consistent fairness under different nuisance factor values.
4 Transfer Fairness via Fair Consistency Regularization
4.1 Theoretical Analysis: A Sufficient Condition for Transferring Fairness
In reality, distribution shifts are usually hybrid, and we may not know all the underlying factor values. In this section, we consider a general case where we only have access to input X, label Y. and sensitive attribute A. We use data transformations to simulate the shift of nuisance factors. Our theory based on [61] and [5] which prove that encouraging consistency under transformations can propagate labels so that to transfer accuracy. In this section, we find that in order to transfer fairness, we need a fair label propagation process that requires the model to have similar consistency across groups. We introduce assumptions and our findings as follows.

---

Assumption 2 (Separability of the input). Let Sa and TN denote the sample space of X Y= in source and target domains. The ground truth class and sensitive attribute for x E Sy U TT are consistent, which are y E {0. 1) and a E [0.1]. We assume the sample spaces of X in two domains are = Uy Uo S and T = Uy Ua Ty, where groups are separated with 1) S n S = TØ n T
Rs nT = 0. Vy, a # d', and 2) S n S TH TT SW N 0, Va, d' y y..
This is a realistic assumption as illustrated in Figure where the data from two domains are from the same underlying conditional distribution Xly,A, and groups are separated by label and sensitive attribute. We define U (S% + T as the group distribution, and U as the population distribution on the entire data. Next, we characterize the good continuity of group distributions with the definition of neighbor and intra-group expansion assumption.
Definition 4.1 (Neighbor). Let 'I denote a set of input transformations and define the transformation set of x as B(x) A 3t E T, s.t. || - -Ix)]l < r). For any E S% UTH, we define the neighbor of x as (xx) == (5% UTY) (a ((x) n B(x)) # 0} and define the neighbor of a set V E X as N(V) = U Uwe.nu,, Sy SUTTY) (x)).
Intuitively, two examples are neighbors if they are near each other after applying some transformations. Note that we only consider neighbors that have the same class and sensitive attribute (i.c.. from the same group). Based on this definition, we characterize the continuity of group distribution with intra- intra-group expansion assumption where any small set has large neighbor in its group.
Assumption 3 (Intra-grou expansion). We say that U2 satisfies (a, c)-mltipllcaative expansion for some constant a E (0. 1) and e. > 1, if for all V C Ul with Pug (V) S a, the following holds: Puz(W(V) 2 min(clPug uy (V), 13.
Different from the expansion assumption proposed in [61] which considers the class continuity, intra- group expansion assumes group continuity. As shown in Figure 1, this is more realistic since groups separated by both label and sensitive attribute. We can also interpret it as the transformations that change the value of nuisance factors will generate neighbors within the same group.
This assumption allows us to propagate labels within the group from one domain to another by encouraging consistency under transformations. We use Rgy (g) A Pux [Ex' (zx)) s.t. g(x) * g(x')] denote the consistency loss of classifier g on the group distribution UY, which is the fraction of examples where g is not robust to input transformations. Since we only have partial supervision (i.e., no labels in the target domain), we use a self-training framework to obtain a model that is accurate and fair in both domains (i.e., on U2). Based on the theory of self-training in [61], we derive a sufficient condition in Theorem 4.1 that bounds the unfairness and error on the population distribution. We use 0-1 loss to evaluate the error of g as Eug (g) - Pug (xx) & g" (x')), and the disagreement between g and a teacher classifier 9te as Lux (g, 9te) - Fug ((x)) # (z)))
Theorem 4.1 (Guarantee fairness). Suppose we have a teacher classifier 9te with bounded unfairness such that | (gte) Egy; (gtc)) S Y, Va, a' A and y. v E V. We assume intra- strop expansion where Uy satisfies (ā, millpliiliie expansion and Suz (ghe)
< 1/3 and ē 3, Va, y- We define e 2 min(L/a, č}, and set μ S Euy (gte), Va, y. If we train our classifier with the algorithm min max Ray (g): s.t. Luy (g. 9hc) S Yo, gEG
then the error and unfairness of the optimal solution 9 on the distribution U are bounded with 2
2c
E(g) S c =Ev(e + c
(2)
2 Aodde (g) S c (7 + 11 emax ceaaxayyf Roy(())
(3)
a.y
Remark. This sufficient condition suggests we fit teacher classifier which is fair on the population distribution and minimize the consistency loss in every group. The unfairness of the resulting model is bounded by the quality (unfairness and error) of the teacher classifier and the worst-group consistency loss. Intuitively, we can understand the consistency loss as the model invariance to the nuisance factors. With a group-balanced consistency loss, the model would have similar invariance to the nuisance factors resulting in similar group performance on the unseen data so that to transfer accuracy and fairness. We also bound the variance of group accuracy with the variance of consistency loss (Appendix C). Both bounds suggest we balance and minimize the consistency loss across groups
5

---

4.2 Practical Algorithm: Fair Consistency Regularization
There are two challenges in realizing the theoretical algorithm in Theorem 4.1. First, we need high-quality teacher model, but the model trained with labeled source data is only fair and accurate in the source domain. Second, existing consistency regularization methods do not consider fairness. We tackle the first problem by leveraging the iterative self-training paradigm that updates the teacher model with the student model while training, thus making it fairer and fairer. We tackle the second problem by proposing a novel fair consistency regularization.
Algorithm. Figure 2 shows the overall training diagram. There are three ma- jor components:
Adversary
Srar (ooure oniy!
(1) In every training epoch, we use the student model obtained in the last epoch as the teacher model and auto- matically fit the teacher model by ini- tializing the student model to be the same as the teacher model. In other words, only one model is training itself iteratively.
gradine
Fes
prosiation
Encoder
Clasiifier
(nstint
sbarld
lyoruss
precktion
Encoder
2
Classifier
Figure 2: Training diagram.
(2) To ensure the accuracy and fairness in the source domain, we adopt Laftr
[42], an adversarial learning method consisting of a classification loss Lats and a fairness loss faar- (3) To transfer fairess and accuracy. we do consistency training on all unlabeled data (including source and target data). Following FixMatch [55], we use the pseudo-labels generated by the teacher model as supervision for consistency training where the model should have consistent predictions under transformations. Different from FixMatch, we propose a fair consistency regularization with a balanced group consistency loss feonsis.
We train the model with the weighted summation of these three losses as shown in Figure 2. We defer the detailed loss functions of Lets and Lfair with detailed algorithm description to Appendix D.
Fair Consistency Regularization. To tighten the upper bound of the unfairness in Theorem 4.1, we need to minimize and balance consistency loss across groups. However, the consistency regularization in FixMatch [55] does not distinguish groups and might amplify the bias as observed in [76] and our experiments. Instead, we propose to use fair consistency regularization that evaluates the consistency loss per group and minimizes the balanced consistency loss I.foonsis defined as below.
Lyconsis (g)
=ΣX
(g)
(4)
Y=u
where [g(g)
1
Σ
(max((9r 2 (argmaa (zt))) ((((x))))
(5)
Σnn
where zy denotes an input with sensitive attribute = a and class Y = y. LX(g) is model g's consistency in the group of {xx]}, and Xg is the corresponding weight of the group consistency loss. Here, we abuse the usage of g(x) to denote the output logits of model g on input x and thus, argmax((ie(x")) is the pscudolabel generated by teacher classifier. t(x²) is the transformed input as defined in Definition 4.1. We use a cross-entropy loss H-)) to encourage the consistency under transformation t(-) and only consider examples that the teacher model has high confidence in with confidence threshold r. Note that data is classified into groups according to the true sensitive attribute and pseudolabels. To balance the group consistency loss, we propose to weigh each group inversely with the number of confident pseudolabels, and set Ag as
Σ A2.
(6)
End (max(gro
G.
The weights will dynamically change while training. Heuristically, if the teacher model only confident in a few examples in group, the model's 's consistency in this group is more likely to be low. With the proposed weights, a larger penalty will be applied to such groups. Therefore, the proposed fair consistency regularization will enforce the model to pay more attention to high-error groups. By doing so, the trained model would enjoy similar consistency loss across groups Together with the self-training algorithm, it would have similar accuracy across groups in the target domain.

---

Sshift2
Sshift2
Dshift
Hsnif
Accurocy
Accuracy
Accurcio
Accurdio
2
25
28
for ThE
Unferrmess (equalized pdds)
Unfeimmess (equalized odos)
Unfeimess (equalized zed odds)
Unfelimess requill zed odds)
Base DANN
Laftra
Laftr (5)
Liftr-Fimmtch (5)
Base (T)
BANt--DANNI (T)
Ours (T)
Figure 3: Accuracy and unfairness (error bar denotes the standard deviation) in two domains under subpopulation shifts (Sshift 1, Sshift 2), domain shift (Dshift), and hybrid shift (Hshif). (S) and (T) denotes the evaluation in the source and target domains respectively. Results show that domain shift is more challenging than subpopulation shift, and our method can effectively transfer accuracy and fairness under all the distribution shifts considered.
Related Work
This section features related work for transferring faimesss Another discussion of related work in fair machine learning, domain adaptation, and self-training is deferred to Appendix A. Duutof-distribution fairness remains under-explored area. We categorize prior works into five classes. 1) Group-wise distribution matching. [51] derives an upper bound for fairness in the target domain which suggests training fair model in the source domain and matching the distributions of relevant groups from two domains in feature space at the same time. [64] also applies group-wise distribution matching but with Wasserstein distance. Such methods are hard to achieve if we do not have supervision in the target domain and it also shares the drawback of distribution matching methods. 2) Reweighting. When the proportions of groups differ in two domains, reweighting the examples in the source domain can approximate the target distribution. [16] uses reweighting to deal with fairness problems under covariate shift and [23] uses reweighting together with fairness test to guarantee fairness under demographic shift. Reweighting methods strongly rely on the support cover assumption which is not satisfied under domain shift. 3) Distributionally robust optimization (DRO). This line of work considers unknown target data that can be any arbitrary weighted combinations of the source dataset and train a fair model that is robust to the worst-case shift [48, 43]. These methods also assume subpopulation shift instead of domain shift. 4) Causal inference. [54] conducts causal domain adaptation and DRO based on well-characterized causal graph that describes the data construction and distribution shift. Causal methods highly rely on the correct causal graph which is hard to obtain in reality. For example, [50] finds that the causal graph in real applications (e.g. predicting the skin condition in dermatology) is far more complicated which violates normal assumptions, thus making those approaches inapplicable. 5) Others. [10] derives bound for fairness violation under distribution shifts. There are also studies that aim to maintain fairness under distribution shifts through online learning [69], and loss curvature matching [59]. To the best of our knowledge, this is the first work that uses self-training to transfer fairness. Some work also studies self-supervised learning and fairness, yet they use unlabeled data and self-training to improve the in-distribution faimess [14, 68, 9] which is different from our goal.
6 Experiments
6.1 Evaluation under Different Types of Distribution Shifts with a Synthetic Dataset
In order to study the fairness under distribution shifts and verify our theoretical findings, we develop a synthetic dataset to simulate different types of distribution shifts.
Synthetic dataset. The synthetic dataset is adapted from the 3dshapes dataset [31] which contains im- ages of 3D objects generated from six independent latent factors (shape, object hue, scule, orientation, floor hue, wall hue). This dataset satisfies our assumption on the shared underlying data generation process. We simulate different types of distribution shifts by varying the marginal distributions of the latent factors and sample the data accordingly (see Appendix D. for details).
7

---

Distribution shifts. We set the image as input X, and select three latent factors to be class (Y shape), sensitive attribute (A = object hue), and a nuisance factor that might shift (D scale). We consider four widely observed distribution shifts in reality ( (Pps(X, (X, Y. A, D) * Pr(X, Y. A, D));
(1) Sshift Subpopulation shift where only the nuisance factor shift (i.e. more small objects in source but more large objects in target), Ips sY, A) = (Y, A), 1 () * Pr
(2) Sshift 2: Subpopulation shift where A and Y have different correlations in two domains (i.e. most red objects are cubes in source but are capsules in target), PY, A) o Pr(Y, (A) Ps(D) = (D).
(3) Dshift: Domain shift where the nuisance factor has different sample spaces (i.e. only small objects
in source but only large objects in target), Ps(Y, A) = Pr A), (D( F Pr (D), V/ + y%-
(4) Hshift: Hybrid shift of (2) and (3).
Baselines. We do shape classification task with an MLP model and compare our method with four baselines: Base (standard ERM); Laftr; LafN++DANN (a combination of Laftr and a domain adaptation method [22]); Laftr+FixMatch. In our method, we also use Laftr and FixMatch but with the proposed fair consistency regularization. Since the shifted nuisance factor is scale, we use random padding and cropping as transformations in our method and Laftr+FixMatchh. We train Base and Laftr with labeled source data and train others with unlabeled target data as well.
Domain shift is more challenging than subpopulation shift. Figure 3 shows that under subpopula- tion shifts, the fair source model trained with Laftr also has high accuracy and fairness in the target do- main although it has not seen any target date. This is because the sample space is shared (e.g. small and large objects both exist in the source data), and the model has similar performance under all factor val- ues. Thus, good performance remains even if the proportion of data changes, verified Proposition 3.1. In contrast, under domain shift and hybrid shift, the fair source model performs poorly in the target domain where data is sampled from a different sample space, suggesting the difficulty of domain shift.
Our method can transfer fairness and accuracy under various types of distribution shifts. Under domain shift, the domain adaptation method DANN does not help in transferring fairness or accuracy, Consistency regularization forces the model to behave consistently under cropping and padding, resulting in a model that has similar predictions regardless of the object's 's scale and thus transfers accuracy. However, it may cause bias as shown in the results of Laftr+FixMatch. With the proposed fair consistency regularization, the model gains similar consistency across groups, resulting in similar accuracy in all groups in the target domain and thus transfers fairness. Therefore, our method achieves high accuracy and fairness in two domains under all the considered distribution shifts.
Source
Target
Ace
Unfaimess
Ace
Unfaimess
Method
Vace
Aodda
Vace
Auddo
Base
92.85
2.30a0.97
4.812069
74.4910.83
5,79a349
9.90s1
Laftr
93.24+0.41
1.19+0.66
2.44=0.1
74.35+1.46
6.92+0.2
9.79=1
Crair
92.51+0.22
1.76+0.53
4.75-0.5
73.53+0.8
1.51+
7.26-1 95
LaNN++DANN
91.33a0.08
2.12a1.72
2.7020.57
74.28a1.63
6.25a2.59
8.27
CFair+DANN
90.8920.76
2.01a0.70
4.4321.36
74.6211.06
6,2310.90
5.2622.07
I Mafts+FiMMacch
96.62+9.06
0.77+0.21
2.23-044
83,8740.48
8.21+0067
9.32-1
Trair+FixMatch
96. 1330,53
1.28,0.53
2.78
83. 83.11 1
7.87a186
7.89:040
Ours (w/ Laftr)
96.08 06.0810.07
0,9620.39
2.5920.35
85,5210.10
2.8220.87
5.7020.52
Ours (w/ CFair)
95.65+0.22
1.56+0.37
3.85-0.97
84.48+0.42
2.88+0.99
6.43-065
Table 1: Transfer faimness and accuracy from UTKFace to FairFace
6.2 Evaluation on Real Datasets
Evaluation on images. We use UTKFaac [71] as the source data and FairFace [30] as the target data. Although both are facial images, there is a distribution shift between them due to different image sources. We consider gender classification task with race as the sensitive attribute. We use VGG16 [53] as the model and RandAugment [18] (excluding transformations that may change the group) as the transformation function. Additional to previous baselines, we also use CFair [73] as the method for in-distribution fairness. As shown in Table 1, there is indeed distribution shift as the source model trained with Laftr or CFair is no longer accurate or fair in the target domain. The domain adaptation method has a limited effect on transferring accuracy and fairness. As expected, self-training Flxmath+Fxmmach and CFair+ Fixmatch) significantly improves the accuracy in the target domain, but the unfairness is high. With the proposed fair consistency regularization, method
8

---

DANN
Laftr+ DANN
Laftr+ Ridtttt
Accuracy
Aerurary
Ours
79-
Infairess (varianca acturacy)
(a) Parcto frontiers of ours (w/ Laftr) and baselines in the target domain.
85
Clair DANN
Accunity
Pacunesty
CFaina FixMatch
70
Unfattees lequeitzed
Unterrness (verience
(b) Pareto fronticrs of ours (w/ Cfair) and baselines in the target domain.
Figure 4: Comparison of Pareto frontiers. Upper left preferred. Our method outperforms baseline methods in achieving accuracy and fairness at same time.
Unfoinesz rcqualood Oodsl
(equatord
12 De d
-0.2
0.0
0.2
Accuracy
Accuracy
(a) Laftr
(b) Ours
(c) Decreuse unfairmess
Figure 5: Unfairness and accuracy tested on NewAdult. CA as the source domain (red star) and other states as the target domain (blue dots). Red lines indicate the average of unfairness. The relative decrease is calculated by comparing with Laftr.
outperforms it remarkably on fairness with a 70% decrease in the variance of group accuracy and a 30% decrease in the equalized odds. We further sweep the weights of losses and draw Pareto frontiers. As shown in Figure 4, our method significantly outperforms others in achieving accuracy and faimesss the same time.
Evaluation on tabular data. We further evaluate our method on the NewAdult dataset [19] which contains census data from all states of the United States. We consider gender the sensitive attribute and do income classification with an MLP as the model. We set CA as the source domain and all the other states the target domain. We use random perturbation on tabular data (see details in Appendix D) as the transformations. Results are shown in Figure 5. When applied to other states, the fair model trained on CA becomes unfair (Figure 5a). Our method improves the fairness in most states with a slight improvement accuracy (Figure 5b). Compared with the one without fair consistency regularization, method achieves better fairness with decrease in unfairness in most states (Figure 5c).
6.3 Ablation Study
The role of transformation. We design transformation functions based on our domain knowledge of latent factors. investigate the importance of transformations, we test a weaker set of trans- formations, which includes only cropping and flipping, on the TTTKFace-FairFFF report the performance in Table 2. Compared with RandAugment Table consistency under weak transformations leads to less effective transfer of accuracy since the neighbor generated transformations is much smaller. The limited transformations also restrict the performance of our method tabular data (see Appendix E). Though the ability to transfer accuracy is limited by weak
experiment and

---

Source
Target
Acc
Unlairness
Acc
Unfaimess
Method
Vana
Vanc
Anddd
Laftr-FixMatth
94.08e0.70
1.6420446
3.51+1.46
77.05e026
12.23
6.55a1.54
CTair-FixMatch
94. 39.333
0.97.0.36
2.16.1.97
77.25.021
12.93.2.66
9.7740.95
Ours (w/ Laftr)
94,25g022
1.0620446
2.0920.55
77.32=021
2.35
4.27at.41
Ours (w/ CFair)
94.24a026
1.67s0338
4.43a0.63
77.96=038
3.34
5.70at.14
Table 2: Transfer fairness and accuracy from UTKFace FairFace with weak transformations
transformations, our method can still make the transfer process fair as there's significant decrease in unfairness, as shown in Table 2.
Fair consistency is essential in transferring fairness.
FOR
FCR
To see whether enhanced consistency improves accuracy and whether unbalanced consistency leads to unfaimess as suggested by Theorem 4.1, we evaluate the accuracy and consistency of each group in the FiXKFaeFFFirFuce experiment on the target data. The consistency is mea- sured by testing the model's agreement the outputs under two random transformations. As shown in Figure 6, groups that obtain higher consistency have higher accuracy, which validates the ability of consistency regularization for transferring accuracy. The training methods that use standard consistency regularization (e.g. Laftr+FixMaiin have been observed be unfair in the target domain. Fig- ure 6 shows that it is because the model has imbalanced consistency across groups. With our fair consistency reg- ularization, the model gains similar consistency for all groups, resulting in similar group accuracy.
accuracy
A113
100
250
53
500
152
200
spoch
SCR
SCR
accursIy
100 opoch
250
200
152
Figure 6: Per-grouy accuracy and consis- tency. Compared with the standard consis- tency regularization (SCR), the model trained with fair consistency regularization (FCR) has more balanced consistency and accuracy.
The role of components in fair consistency regu- larization. Table 3 shows the ablation study. We can see that the consistency in both domains matters. Giving every group the same weight instead of using dynamic weights leads to increased unfaimesss Fix- ing the teacher classifier to the fair source model, we observe a significant decrease in the accuracy, sug- gesting the important role of iterative self-training in our algorithm.
Acc
Unfairness
Method
Vece
Ours
85,5240.00
2.82a037
5.70a0:3
wro consistency in target
82.43:1.05
6.80
5.85.0.40
consistency wio dynamic weights
82,5=1.5
6.6341.7
8.18
84.34.0.19
6.86.0.90
7.68:0331
wro updating
79,13ua.32
3.49u0.3
6.65a1.31
Table 3: Ablation study on OTTKFac--FarFaace task
Conclusion
In this paper, explore how to transfer fairness under distribution shifts. We derive sufficient condition and present a theory-guided self-training algorithm based on an intra-group expansion assumption. The key component of algorithm is fair consistency regularization. We simulate different types of distribution shifts with a synthetic dataset and examine our theoretical findings with it. Abundant experiments with synthetic data and real data have shown that method has superior performance in transferring fairness and accuracy. Like other self-training methods, one limitation of our method is the reliance on well-defined data transformation set. Future work will relax this limitation for application to more real-world problems.
Acknowledgements
This work is supported by National Science Foundation NSF-IIS-FAI program, DOD-ONR-Ofice of Naval Research, DOD-DARPA-DDfensee Advanced Research Projects Agency Guaranteeing Al Robustness against Deception (GARD), and Adobe, Capital One and JP Morgan faculty fellowships.
10

---

References
[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna M. Wallach. A reductions approach to fair classification. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Leurning, ICML 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 60-69. PMLR, 2018.
[2] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. A theory of learning from different domains. Mach. Learn., 79(1-
2):151-175, 2010.
[3] David Berthelot, Rebecca Roclofs, Kihyuk Sohn, Nicholas Carlini, and Alex Kurakin. Adamatch: A unified approach to semi-supervised learning and domain adaptation, 2021.
[4] Alex Beutel, Jilin Chen, Zhe Zhao, and Ed H. Chi. Data decisions and theoretical implications when adversarially learning fair representations, 2017.
[5] Tianle Cai, Ruiqi Gao, Jason Lee, and Qi Lei. A theory of label propagation for subpopulation shift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Confer- ence on Machine Learning. volume 139 of Proceedings of Machine Learning Research, pages 1170-1182. PMLR, 18 24 Jul 2021.
[6] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with independency constraints. In 2009 IEEE International Conference on Data Mining Workshops, pages 13-18, 2009.
[7] Simon Caton and Christian Haas. Fairness in machine learning: A survey, 2020.
[8] L. Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nisheeth K. Vishnoi. Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT '19, page 319-328, New York, NY, USA, 2019. Association for Computing Machinery.
[9] Joymallya Chakraborty, Huy Tu, Suvodeep Majumder, and Tim Menzies. Can we achieve fairness using semi-supervised learning? arXiv preprint arXiv:2111.02036 2021.
[10] Yatong Chen, Reilly Raab, Jialu Wang. and Yang Liu. Faimess transferability subject to bounded distribution shift, arXiv preprint UrXiv:2206.00129, 2022.
[11] Yining Chen, Colin Wei, Ananya Kumar, and Tengyu Ma. Self-training avoids using spurious features under domain shift. Advances in Neural Information Processing Systems, 33:21061- 2 21071, 2020.
[12] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments, 2016.
[13] Ching- Ching-Yao Chuang and Youssef Mroueh. Fair mixup: Fairnsss via interpolation. In 9th International Conference on Learning Representations, ICLR 2021, Viriual Event, Austria, May
3-7, 2021. OpenRevie,.net, 2021.
[14] Evgenii Chzhen. Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Leveraging labeled and unlabeled data for consistent fair binary classification. Advances in Neural Information Processing Systems, 32, 2019.
[15] Sam Corbett-Davies and Sharad Goel. The measure and mismeasure of f'arrees: A critical review of fair machine learning, 2018.
[16] Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R. Varshney, Skyler Speakman, Zairah Mustahsan, and Supriyo Chakraborty. Fair transfer learning with missing protected attributes. In Vincent Conitzer, Gillian K. Hadfield, and Shannon Vallor, editors, Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2019, Honolulu, HI. USA, January 27-28, 2019, pages 91-98. ACM, 2019.
[17] Elliot Creager, David Madras, Jörn-Henne Jacobsen, Marissa A. Weis, Kevin Swersky, Toniann Pitassi, and Richard S. Zemel. Flexibly fair representation learning by disentanglement. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICMI. 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 1436-1445. PMLR, 2019.

---

[18] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoe V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops, pages 702-703, 2020.
[19] Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021.
[20] Michele Donini, Luca Oneto, Shai Ben-David, John Shawe-Taylor, and Massimiliano Pontil. Empirical risk minimization under fairesss constraints. In Samy Bengio, Hanna M. Wal- lach, Hugo Larochelle, Kristen Grauman, Nicolò
and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Anmual Conference on Neural Informa- tion Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 2796-2806, 2018.
[21] Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. On the (im)possibility of fairness, 2016.
[22] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-aavversrrial training of neural networks. The journal of machine learning research, 17(1):2096-200) 2030, 2016.
[23] Stephen Giguere, Blossom Metevier, Yuriy Brun, Philip S. Thomas, Scott Niekum, and Bruno Castro da Silva. Fairness guarantees under demographic shift. In International Confer- ence on Learning Representations, 2022.
[24] Bruce Glymour and Jonathan Herington. Measuring the biases that matter: The ethical and casual foundations for measures of fairess in algorithms. In danah boyd and Jamie H. Morgen- stern, editors, Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT 2019, Atlanta, GA, USA, January 29-31, 2019, pages 269-278. ACM, 2019.
[25] Karan Goel, Albert Gu, Yixuan Li, and Christopher Re. Model patching: Closing the sub- group performance gap with data augmentation. In International Conference on Learning Representations, 2021.
[26] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems, 33:21271-21284 2020.
[27] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
[28) Judy Hoffman, Eric Tzeng, Taesung Park, Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. CyCADA: sycle-consistent adversarial domain adaptation. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1989-1998. PMLR, 10-15 Jul 2018.
[29] Taotao Jing, Bingrong Xu, and Zhengming Ding. Towards fair knowledge transfer for imbal- anced domain adaptation. IEFE Transactions on Image Processing, 30:8200 0018200-2211, 2021.
[30] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age. arXiv preprint arXiv: 1908. 04913, 2019.
[31] Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2649-2658. PMLR, 10-15 Jul 2018.
[32] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in til- distribution shifts. In International Conference on Machine Learning, pages 5637-5664. PMLR, 2021.
[33] Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. In International Conference Machine Learning, pages 5468-5479. PMLR, 2020.
12

---

[34] Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. Counterfactual faimess, In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4066-4076, 2017.
[35] Chao Ln and Jun Huan. Discriminatory transfer. arXiv preprint arXiv: 1707.00780 2017.
[36] Bo Li, Yezhen Wang, Tong Che, Shanghang Zhang, Sicheng Zhao, Pengfei Xu, Wei Zhou, Yoshua Bengio, and Kurt Keutzer. Rethinking distributional matching based domain adaptation. arXiv preprint arXiv:2006. I 13352, 2020.
[37] Hong Liu, Jeff Z. HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning is more robust to dataset imbalance. In NeurlPS 202/ Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
[38] Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self-training for domain adaptation, 2021.
[39] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International conference on machine learning, pages 97-105, PMLR, 2015.
[40] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in neural information processing systems, 31, 2018.
[41] Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard S. Zemel. The variational fair autoencoder. In Yoshua Bengio and Yann LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
[42] David Madras, Elliot Creager. Toniann Pitassi, and Richard S. Zemel. Learning adversarially fair and transferable representations. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässsn, Stockholm, Sweden, July 10, 2018, volume 80 of Proceedings of Machine Learning Research, pages 3381-3390. PMLR, 2018.
[43] Debmalya Mandal, Samuel Deng. Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fairness beyond the training data. Advances in neural information processing systems, 33:18445-
18456, 2020.
[44] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. CoRR, abs/1908 0000199089900 00635,96555 2019.
[45] Luca Oneto and Silvia Chiappa. Fairness in machine learning. Studies in Computational Intelligence, page 155-196, 2020.
[46] Alvin Rajkomar, Michaela Hardt, Michael D Howell, Greg Corrado, and Marshall H Chin. Ensuring fairness in machine learning to advance health equity. Annals of internal medicine, 169(12):866887. 2018
[47] Charan Reddy, Deepak Sharma, Soroush Mchri, Adriana Romero-Soriant Samira Shabanian, and Sina Honaii. Benchmarking bias mitigation algorithms in representation learning through fairness metrics. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
[48] Ashkan Rezaei, Angi Liu, Omid Memarrast, and Brian D. Ziebart. Robust fairness under covariate shift. CoRR, abs/2010.05166, 2020.
[49] Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori B. Hashimoto, Sergey Levine, Chelsea Finn, and Percy Liang. Extending the wilds benchmark for unsupervised adaptation. ArXiv, abs/21 12.55900 2021.
[50] Jessica Schrouff. Natalie Harris, Oluwasanmi Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alex Brown, Subhrajit Roy, Diana Mincu, Christina Chen, Awa Dieng, Yuan Liu, Vivek Natarajan, Alan Karthikesalingam, Katherine Heller, Silvia Chiappa, and Alexander D' Amour. Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?, 2022.
13

---

[51] Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed H. Chi. Transfer of machine learning fairness across domains. CoRR, hts//990.0000 09688, 2019.
[52] Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. arXiv preprint 18X2::880.0087553 2018.
[53] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
[54] Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Fairness violations and mitigation under covariate shift. In Madeleine Clare Elish, William Isaac, and Richard S. Zemel, editors, FAccT '21: 2021 ACM Conference Fairness, Accountability, and Transparency, Virtual Event Toronto, Canada, March 3-10, 2021, pages 3-13, ACM, 2021.
[55] Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001. N0KV::200.00685,, 2020.
[56) Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. In Kamalika Chaudhuri and Masashi Sugiyama, editors, The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pages 2164-2173, PMLR, 2019.
[57] Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey J Gordon. Domain adaptation with conditional distribution matching and generalized label shift. Advances in Neural Information Processing Systems, 33:19276-1928, 2020.
[58] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 7167-7176, 2017.
[59] Haotao Wang, Junyuan Hong, Jiayu Zhou, and Zhangyang Wang. Equalized robustness: Towards sustainable fairnsss under distributional shifts, 2022.
[60] Tongxin Wang. Zhengming Ding, Wei Shao, Haixu Tang. and Kun Huang. Towards fair cross- domain adaptation via generative learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 454 463, 2021.
[61] Colin Wei, Kendrick Shen, Yining Chen, and Tengyu Ma. Theoretical analysis of self-trriming with deep networks on unlabeled data. In 9th International Conference on Learning Represen- tations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[62] Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre-Alvee Alvise Rebuffi, Ira Ktena, Krishna- murthy Dj Dvijotham, and Ali Taylan Cemgil. A fine-grained analysis on distribution shift. In International Conference on Learning Representations, 2022.
[63] Yifan Wu, Ezra Winston, Divyansh Kaushik, and Zachary Lipton. Domain adaptation with asymmetrically--relaxe distribution alignment. In International Conference on Machine Learn- ing, pages 6872668228 18811 PMLR, 2019.
[64] Tae-Ho Yoon, Jaewook Lee, and Woojin Lee. Joint transfer of model knowledge and fairness over domains using wasserstein distance. IEEE Access, 8: 8:123383-123798, 2020.
[65] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P. Gummadi. Fairness Constraints: Mechanisms for Fair Classification. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 962-970. PMLR, 20-22 Apr 2017.
[66] Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representa- tions. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 325-333, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR.
[67] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 335-340 2018
14

---

[68] Tao Zhang, tianqing zhu, Jing Li, Mengde Han, Wanlei Zhou, and Philip Yu. Fairness in semi semi-supervised learning: Unlabeled data help to reduce discrimination. IEEE Transactions on Knowledge and Data Engineering, pages 1-1, 2020.
[69] Wenbin Zhang, Albert Bifet, Xiangliang Zhang, Jeremy C Weiss, and Wolfgang Nejdl. Farf: A fair and adaptive random forests classifier. Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 245-256 Springer, 2021.
[70]
Yabin Zhang. Haojian Zhang, Bin Deng, Shuai Li, Kui Jia, and Lei Zhang. Semi-supervised models are strong unsupervised domain adaptation learners, 2021.
[71] Zhifei Zhang, Yang Song, and Hairong Qi. Age progression/regrssionn by conditional adver- sarial autoencoder. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2017.
[72] Han Zhao, Amanda Coston, Tameem Adel, and Geoffrey J Gordon. Conditional learning of fair representations. In International Conference on Learning Representations, 2019.
[73] Han Zhao, Amanda Coston, Tameem Adel, and Geoffrey J. Gordon. Conditional learning of fair representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. Open 2020.
[74] Han Zhao, Remi Tachet Des Combes, Kun Zhang, and Geoffrey Gordon. On learning invariant representations for domain adaptation. In International Conference on Machine Learning, pages 7523-7532. PMLR, 2019.
[75] Sicheng Zhu, Bang An, and Furong Huang. Understanding the generalization benefit of model invariance from data perspective. Advances in Neural Information Processing Systems, 34, 2021.
[76] Zhaowei Zhu, Tianyi Luo, and Yang Liu. The rich get richer: Disparate impact of semi- supervised learning. In International Conference on Learning Representations, 2022.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section F
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] see Section D
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] see Section D
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
15

---

(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "wDUkqhkflV", "reviewer": "Reviewer_bbdE", "summary": "In this work, the author analyses the problem of how model fairness is affected under different types of distribution shifts. \nFor this, they use equalized odd fairness metrics and propose several types of distribution shifts in the synthetic, real image, and real tabular datasets.\nThe authors also propose a \"theory-minded algorithm\" for transferring fairness with a fair consistency regularization. \n\n", "questions": "Other authors have clearly limited what relies upon to be possible or not to predict model performance degradation under distribution shift [1]\nDo these limitations still apply when evaluating for transferring fairness?\n", "limitations": "Other authors have clearly limited what relies upon to be possible or not to predict model performance degradation under distribution shift [1]. This limitation is eluded by stating assumption #1 \n\nThe abstract presents a \"fine-grained analysis of how the fair model is affected under different types of distribution shift\". There is no discussion about concept shift. This should be acknowledged. ", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper deals with a highly relevant topic (fairness and distribution shift) that remains largely unexplored. \n\nThe authors select one measure of fairness equal opportunity odds, that is very relevant and widely used but without any social/redistributive justice justification.\n\nThe paper is in general hard to understand. Some of the mathematical formulations are not the traditional ones and they use overly complicated notations that difficult the reading of the paper.\n\nThe paper is also not self-contained and over relies on the appendix and external work that is not properly introduced.\n\nAssumption #1 This assumption makes the problem solvable but in real cases the assumption does not hold, as one of the most common and challenging types of shift is concept shift, where the data generation process actually changes.\nThis happens more often with tabular data, and distinguishing covariate shift from concept shift is not feasible in many situations.\nFor the folktables data, one could arguably say that predicting US income by training in CA and evaluating has low fairness variations.\nIf the problem has a higher distribution shift e.g. by changing the prediction task to  \"ACSTravelTime: predict whether an individual has a commute to work that is longer than 20 minutes\" training in Hawaii and predicting in the rest of the states, assumption #1 will not hold. How would the proposed method work in this case? Would it flag that is not properly working? Will achieve better results than a baseline.\n\n\n\nMinor remarks:\n - Line 15 \"A synthetic dataset benchmark... is deployed\" Is deployed the right verb here?\n - The figures are too small and the font size is considerably smaller than the text\n\n[1] https://arxiv.org/abs/2201.04234", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "HYGqRu4pAs0", "reviewer": "Reviewer_FM6i", "summary": "This paper aims to transfer fairness under distribution shift. Specifically, a comprehensive analysis of distribution shift is provided and then defines two types of distribution shift, named subpopulation shift and domain shift. Given the powerful self-training tool to solve the distribution shift problem, the authors first identify a sufficient condition for fairness transfer and then propose an algorithm with fair consistency regularization. Experiments on synthetic\nand real datasets demonstrate the effectiveness in terms of fairness and accuracy under various distribution shifts.\n\nOverall, this paper is generally good, in terms of technical novelty and evaluations.\n", "questions": "Please see Weaknesses.", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n1.\tThis paper is well-written. The authors first provide a fine-grained analysis of fairness under distribution shift and identify the sufficient condition for fairness transfer. Motivated by the analysis of fairness, fair consistency regularization is introduced to transfer fairness under distribution shift. \n\n2.\tThe idea of fair consistency regularization is simple yet effective with (partial) theoretical support.\n\nWeakness\n\n1.\tThere is a gap between theory (Theorem 4.1) and proposed algorithms. Theorem 4.1 implies that minimizing the worst-group consistency loss is beneficial to transfer fairness. However, the proposed regularization employs a balanced consistency loss in practice. It is still unclear, empirically or theoretically, why the worst-group consistency loss cannot work well in practice. Additionally, the dynamic weight seems to be important in the proposed regularization, what’s the rationale to design such dynamic weight?\n\n2.\tMarginal gain for the proposed algorithm. In figure 3 and table 1, the accuracy-fairness tradeoff performance seems to be marginal compared with baseline Laftr+FixMatch. Current results only provide one single accuracy and EO performance, it is hard to identify the superiority of the proposed method. A similar issue exists in Ablation study part. It would be better to plot the Pareto frontier of different methods with variable hyperparameters.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "CVb5pe2KJFW", "reviewer": "Reviewer_15g2", "summary": "This paper proposes to use consistency regularization to improve fairness in ML models under distribution shifts. Specifically, the paper extends upon LAFTR (an adversarial learning algorithm for fairness) and FixMatch to impose the consistency constraints via self-training, and for the consistency constraints the authors extend the existing consistency notion with \"intra-group expansion\", i.e., the transformation/augmentation should be applied within each group and label.\n\nThe authors performed experiments on both a synthetic dataset and two real datasets, and show sometimes the fairness in the target domain can be improved compared to existing approaches.", "questions": "- How does the result in Table 1 show the superiority of the proposed methods on target fairness? From the numbers it is unclear if the proposed method works better than Laftr+FixMatch or CFair+FixMatch.\n\n- Why do augmentations designed for robustness definitely improve fairness? Would some of the augmentations (other than obvious ones like color jittering) hurt fairness? The authors should perform a more detailed study on this.", "limitations": "N/A, this paper aims to improve fairness in ML models.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Fairness under distribution shift is an important research topic, and this paper proposes to use the consistency regularization as a novel way to tackle this problem.\n- In general the paper is clearly written, the method and related work are well presented.\n\nWeaknesses:\n- The proposed algorithm seems to only work well on synthetic data (Figure 3). When transferred to real datasets, the experiments results are rather weak. \n\n(1) from UTKFace to FairFace (Table 1), the proposed approach (either with Laftr or CFair) does not achieve the best fairness in terms of equalized odds. For example, Laftr+FixMatch / CFair + FixMathc both achieve good accuracy and lower equalized odds than the proposed approaches. The group accuracy variance does decrease for the proposed approach, but why is that important compared to the fairness metrics? In addition, all the numbers have high standard deviations, are there any significance test done to show the superiority of some of the methods?\n\n(2) on the tabular dataset, the only results shown are in Figure 4. Other than LAFTR, no other baselines are shown, how do we know if the proposed methods work better than other baselines? Even just compared to LAFTR, from the figures (a) and (b) it is very hard to tell if the proposal made the results more fair (as most data points are almost in the same range, and there is no one-to-one correspondence). For Figure 4(c), there is no significance test either, so it is unclear if the results are significant.\n\n- The transformation/augmentation is an important piece in the overall framework, but it is not discussed in detail at all in this paper. The authors only briefly mentioned what transformations are excluded in Section 6.3. As the authors reuse existing augmentations designed for robustness (not necessarily for fairness), they should perform a more detailed study on how each of those transformations affect fairness, and if the effect is positive or negative. \n\nMinor:\n- Figure 3 (a), LAFTER -> LAFTR", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept