Transferring Fairness under Distribution Shifts via Fair Consistency Regularization
Abstract
The increasing reliance on ML models in high-stakes tasks has raised a major concern on fairness violations.
Although there has been a surge of work that improves algorithmic fairness, most of them are under the assumption of an identical training and test distribution.
In many real-world applications, however, such an assumption is often violated as previously trained fair models are often deployed in a different environment, and the fairness of such models has been observed to collapse.
In this paper, we study how to transfer model fairness under distribution shifts, a widespread issue in practice.
We conduct a fine-grained analysis of how the fair model is affected under different types of distribution shifts, and find that domain shifts are more challenging than subpopulation shifts.
Inspired by the success of self-training in transferring accuracy under domain shifts, we derive a sufficient condition for transferring group fairness.
Guided by it, we propose a practical algorithm with a fair consistency regularization as the key component.
A synthetic dataset benchmark, which covers all types of distribution shifts, is deployed for experimental verification of the theoretical findings.
Experiments on synthetic and real datasets including image and tabular data demonstrate that our approach effectively transfers fairness and accuracy under various types of distribution shifts 1 .
Introduction
Machine learning’s social impact has broadened as it is widely used to aid decision-making in real-world applications, such as hiring, loan approval, facial recognition, and criminal justice.
To avoid discrimination against a subset of population (e g., w.r.t race or gender), many efforts on algorithmic fairness have been carried out [11, 20, 64, 43, 45, 14, 7].
Although existing work has achieved remarkable success in ensuring fairness, most of them assume the distribution of data at test time is identical to that in the training set.
However, recent studies show that the fairness of a model is likely to collapse when encountering a distribution shift.
For example, [18] observes that a fair income predictor trained with data from one state might not be fair when used in other states.
[49] tries to maintain fairness in healthcare settings, but a model that performs fairly according to the metric evaluated in “Hospital A” shows unfairness when applied to “Hospital B”.
Such observations
[Figure]
motivate us to find the reason behind the collapse of fairness, and investigate how to transfer fairness under distribution shifts.
Specifically, when we have labeled data in source domain and unlabeled data in target domain, we investigate how to adapt the fair source model to a target domain with the goal of achieving both accuracy and fairness in both domains.
Intuitively, the fairness of a model in target domain strongly depends on the nature of distribution shifts.
In this paper, we only consider cases where the oracle model is the same in two domains.
We characterize distribution shifts by assuming a unified latent variable model for the underlying data generation process.
We categorize distribution shifts into three types [31]: 1) Domain shift where source and target distributions comprise data from related but distinct domains (e.g., train a model in hospital A but test it in hospital B).
2) Subpopulation shift where two domains overlap, but relative proportions of subpopulations differ (e.g., the proportion of female candidates increases at test time).
3) Hybrid shift where domain shift and subpopulation shift happen at the same time.
Taking a closer look at fairness under them, we find that domain shift is more challenging since the model’s performance is unpredictable in unseen domains.
Such finding is supported empirically on a synthetic dataset we develop to simulate diverse types of distribution shifts.
While recent work explores methods to transfer fairness [53, 47, 22], most considered settings fall into subpopulation shifts.
In this paper, we consider more challenging but practical settings: domain shifts and hybrid shifts.
We draw inspirations from recent progress on self-training in transferring accuracy under domain shifts [60, 5, 69, 3, 48, 54].
The success of self-training is due to an expansion assumption and an input consistency regularization algorithm.
The expansion assumption assumes two domains share one underlying generative model and the support of the distribution on each class is a connected compact set under data transformations (i.e., has a good continuity).
Under the expansion assumption , [60] and [5] prove that self-training which enforces consistent predictions for the same input under different transformations can propagate labels from source to target domain.
This approach exhibits superior performance in transferring accuracy [69, 48], but does not consider fairness.
Taking demography into consideration, we relax the expansion assumption to a more realistic intragroup expansion assumption , as shown in Figure 1, which only requires continuity of the underlying distribution within every group (i.e., data points with the same class and sensitive attribute) rather than the entire class.
Based on intra-group expansion assumption, we derive a sufficient condition that guarantees fairness in both source and target domains.
This sufficient condition suggests that ensuring the trained model to gain the same input consistency on different groups under a fair teacher classifier guarantees fairness in both domains.
However, such a teacher classifier is not available in practice and we need a practical treatment.
Guided by the theoretical algorithm, we propose a practical self-training algorithm which builds upon
Laftr [41], an adversarial learning method for fairness, and FixMatch [54], a self-training framework.
To encourage similar input consistency on different groups as suggested by our theoretical algorithm, we propose a novel fair consistency regularization.
By reweighting the consistency loss of each group dynamically according to model’s performance, it encourages the model to pay more attention to high-error group while training.
We evaluate our method under different types of distribution shifts with the synthetic and real datasets.
Experiments show that our approach achieves high accuracy and fairness in target domain without sacrificing the performance in source domain.
To the best of our knowledge, this is the first work using self-training to transfer fairness under distribution shifts.
Summary of contributions: (1) We provide a fine-grained analysis of fairness under distribution shifts and develop a synthetic dataset to study model fairness under different types of distribution shifts.
(2) Theoretically, we derive a sufficient condition for transferring fairness under distribution shifts.
(3) Algorithmically, we propose a theory-guided algorithm for transferring fairness with a fair consistency regularization as the key component.
(4) Experimentally, we evaluate our method on synthetic data, real image data and real tabular data.
All results show the effectiveness of our approach in transferring fairness.
Preliminaries and Notations
Transfer Fairness.
Let X, A, Y and X , A , Y denote random variables and sample space of input features, sensitive attribute and label.
For simplicity, we assume binary sensitive attribute and binary classification, while our method can easily extend to a more general setting (see Appendix E).
We aim to learn a classifier g : X → Y and are interested in the fairness of it under distribution shifts.
Specifically, with S and T denoting source and target domains, we study how to transfer the fairness when P S ( X,
A, Y ) ̸
= P T ( X, A, Y ) , with the labeled source data and unlabeled target data.
In self-training algorithm, we use g tc to denote a teacher classifier.
We use the word “group” to denote the set of data that has the same label and sensitive attribute.
Unfairness Metric.
We use equalized odds [26] to measure unfairness, ∆ odds = 12 (cid:80) 1 y =0 (cid:12)(cid:12) P ( ˆ Y =
y
|
A = 0 , Y = y ) − P ( ˆ Y =
y
|
A = 1 , Y = y ) (cid:12)(cid:12)
where ˆ Y = g ( X ) is the prediction.
To avoid trivial fairness where a model with constant output has ∆ odds = 0 , we also evaluate the variance of group accuracy, V acc
= V ar ( { P ( ˆ Y =
y
|
A = a, Y = y ) , ∀ a, y } ) , in our experiments.
A Unified Framework to Characterize Distribution Shifts.
We characterize distribution shifts by assuming a unified latent variable model for the underlying data generation process.
We denote the underlying factors as Y 1 , Y 2 , ..., Y K (with Y i as the sample space of Y i ), and data point as X .
Assumption 1.
[Underlying data generation process] We assume the data is generated from a latent generative model as y 1: K ∼ P ( Y 1: K ) and x ∼ P ( X | Y 1: K = y 1: K ) .
The generative model is fixed P
S ( X | Y 1: K = y 1: K ) =
P T ( X | Y 1: K = y 1: K ) but the marginal distribution of factors varies in two domains P S ( Y 1: K ) ̸
= P T ( Y 1: K ) , causing the distribution shift P S ( Y 1: K , X )
̸ = P T ( Y 1: K , X ) .
It assumes that two domains share the same data generation process.
It is realistic, for example, the underlying physical process of cell imaging is fixed, but the distribution of underlying factors (e.g. gender , age or equipment ) may vary in two hospitals (i.e. two domains), resulting in the distribution shift of the observed tissue images.
Two of the factors are label Y l (i.e. Y ) and sensitive attribute Y a (i.e. A ).
We call other factors nuisance factors since they are irrelevant to the classification task.
Fairness under Distribution Shifts
In this section, we provide a fine-grained analysis of fairness under various types of distribution shifts.
Based on the unified framework, we consider two major types of distribution shifts, namely subpopulation shift and domain shift , which are widely considered in many applications [31].
Definition 3.1.
(Subpopulation Shift)
We say it is a subpopulation shift, if for any factor Y i , the sample space of it remains the same in two domains (i.e., Y iS =
Y iT ), but the marginal distribution of at least one factor changes resulting in P S ( Y 1: K ) ̸
= P T ( Y 1: K ) and P S
( Y 1: K , X )
̸ = P T ( Y 1: K , X ) .
Definition 3.2.
(Domain Shift)
We say it is a domain shift, if at least one nuisance factor Y i , i ̸
= l, i ̸
= a , has different sample space in two domains,
∃ y i ∈ Y iT , but y i / ∈
Y iS , resulting in P S ( Y 1: K ) ̸ =
P T ( Y 1: K ) and P S ( Y 1: K , X )
̸ = P T ( Y 1: K , X ) .
Intuitively, under subpopulation shift, the sample space overlaps, and only the marginal distributions of factors vary in two domains.
For example, the proportion of young people versus elder people in training and deployment time differs.
In contrast, under domain shift, the source model has never seen the data with factors’ values that only exist in the target domain.
For instance, the source model is unaware of the equipment used for cell imaging at deployment time.
Why do distribution shifts cause unfairness?
Suppose the marginal distributions of a binary nuisance factor Y i differ in two domains with P S ( Y i )
̸
= P T ( Y i ) .
The unfairness in two domains are
[Equation]
Since P S ( X | Y i = y i )
=
P T ( X | Y i
=
y i )
, we have ∆ Sodds | Y i =
y i =
∆ Todds | Y i
=
y i , ∀ y i ∈ { 0 , 1 } .
Under subpopulation shift, Y i has the same sample space in two domains but with different proportions (e.g., P S ( Y i = 0)
= 0 .
9 , P S ( Y i = 1) = 0 .
1 , P T ( Y i = 0) = 0 .
1 , P T ( Y i = 1) =0 . 9 ), while under domain shift the sample space differs (e.g. P S ( Y i = 0)
= 1 , P T ( Y i = 1) = 1 ).
It is easy to see from (1) that if a model is perfectly fair on data with Y
i = 0 but totally unfair on data with Y i = 1 , then the model is highly fair in the source domain but highly unfair in the target domain under both cases.
Therefore, if the model has inconsistent performance on data generated from different nuisance factors’ values, then the shift marginal distribution of those factors may cause fairness collapse.
How to transfer fairness under distribution shifts?
Based on the above analysis, one way is to train the model to be fair under any factors’ values.
It is possible under subpopulation shift as states in the following proposition (see proof and discussion in Appendix B).
Proposition 3.1.
[Transfer of fairness under subpopulation shift] Consider the subpopulation shift that is caused by the shifted marginal distribution of nuisance factor
Y i (i.e., P S ( Y i )
̸
= P T ( Y i ) ), while Y iS =
Y iT
=
Y i .
If model f is strictly fair in source domain under any value of factor Y i satisfying P S ( f ( X ) =
y l | Y a = 0 , Y l = y l , Y i =
y i ) =
P S ( f ( X ) =
y l | Y a = 1 , Y l = y l , Y i =
y i ) ,
∀ y i ∈ Y i , y l ∈ { 0 , 1 } , then f is also fair in target domain with ∆ odds = 0 .
Our empirical results (Figure 3) also support this finding.
However, domain shift is more challenging.
The source model’s performance on the target data is unpredictable due to the distinct sample space.
One promising way to tackle domain shift is to enforce model’s invariance to nuisance factors, so that source model would have the same behavior on target data.
Note that, this solution also works for subpopulation shift since it will lead to the case in Proposition 3.1 directly.
Above analysis motive us to transfer accuracy and fairness by doing consistency regularization in the next section.
Transfer Fairness via Fair Consistency Regularization 4.1 Theoretical Analysis: A Sufficient Condition for Transferring Fairness
In reality, distribution shifts are usually hybrid and we may not know all the attributes, while the underlying data generation process remains unchanged.
In this section, we consider a general distribution shifts case with access to only input X , label Y , and sensitive attribute A .
Due to the fixed underlying data generation process, we assume that the data X from two domains are from the same underlying conditional distribution of X | Y,A where groups are separated (i.e. by label and sensitive attribute) which is characterized as follows.
Assumption 2 (Separability of the input) .
Let S ya and T ya denote the sample space of X |
A = a
,Y = y in source and target domains.
The ground truth class and sensitive attribute for x ∈ S ya ∪ T ya are consistent, which are y ∈ { 0 , 1 } and a ∈ { 0 , 1 } .
We assume the sample spaces of X in two domains are S = ∪ y ∪ a S ya and T = ∪ y ∪ a T ya , where groups are separated where 1) S ya ∩ S ya ′ =
T ya ∩ T ya ′ =
S ya ∩ T ya ′
= ∅ , ∀ y, a ̸ = a ′ , and 2) S ya ∩ S y ′
a ′ =
T ya ∩ T y ′
a ′ =
S ya ∩ T y ′
a ′ = ∅ , ∀ a, a ′ , y ̸
= y ′ .
This is a realistic assumption as illustrated in Figure 1.
We define U ya =
12 ( S ya + T ya ) as the group distribution, and U as the distribution on the entire data.
Next, we characterize the good continuity of group distributions with the definition of neighbor and intra-group expansion assumption.
Definition 4.1 (Neighbor) .
Let T denote a set of input transformations (e.g. via data augmentation) and define the transformation set of x as B ( x ) ≜ { x ′ |∃ t ∈ T , s.t.
∥ x ′ − t ( x ) ∥ ≤ r } .
For any x ∈ S ya ∪ T ya , we define the neighbor of x as N ( x ) :
= ( S ya ∪ T ya ) ∩ { x ′ |B
( x ) ∩ B ( x ′ ) ̸ = ∅} and define the neighbor of a set V ∈ X as N ( V ) :
= ∪ x ∈ V ∩ ( ∪ y ∪ a S ya ∪ T ya ) N ( x ) .
Intuitively, two examples are neighbors if they are near each other after applying some transformations.
Note that, we only consider neighbors that have the same class and sensitive attribute (i.e. from the same group).
Based on this definition, we characterize the continuity of group distribution with intra-group expansion assumption where any small set has a large neighbor in its group.
Assumption 3 (Intra-group expansion) .
We say that U ya satisfies ( α, c ) -multiplicative expansion for some constant α ∈ (0 , 1) and c > 1 , if for all V ⊂
U ya with P
U
ya
( V ) ≤ α , the following holds:
[Equation]
Different from the expansion assumption proposed in [60] which considers the class continuity, intragroup expansion assumes group continuity.
As shown in Figure 1, this is more realistic since groups are separated by both label and sensitive attribute.
We can also interpret it as the transformations that change the value of nuisance factors will generate neighbors from the same group.
This assumption allows us to propagate labels within the group from one domain to another by encouraging the consistency under transformations.
We use R U ya ( g ) ≜
P U ya
[ ∃ x ′ ∈ B ( x ) , s.t. g ( x ) ̸
= g ( x ′ )] to denote the input consistency loss of classifier g on the group distribution U ya , which is the fraction of examples where g is not robust to input transformations.
Since we only have partial supervision (no labels in the target domain), we use a self-training framework to obtain a model that is accurate and fair in both domains (i.e. on U ya ).
Based on the theory of self-training in [60], we derive a sufficient condition in Theorem 4.1 that bounds the unfairness and error on the population distribution.
We use 0-1 loss to evaluate the error of g as ε U y a ( g ) ≜ P U y a [ g ( x ) ̸ =
g ∗ ( x ′ )] , and the disagreement between g and a teacher classifier g tc as L U ya ( g, g tc ) ≜
P U ya [ g ( x ) ̸
= g tc ( x ′ )] .
Theorem 4.1 (Guarantee fairness) .
Suppose we have a teacher classifier g tc with bounded unfairness such that | ε U
[Equation]
, ∀ a, a ′ ∈ A and y, y ′ ∈ Y .
We assume intra-group expansion where U ya satisfies (¯ α, ¯ c ) -multiplicative expansion and ε U ya ( g tc ) ≤ ¯ α < 1 / 3 and ¯ c > 3 , ∀ a, y .
We define c ≜ min { 1 / ¯ α, ¯ c } , and set µ ≤ ε U ya ( g tc ) , ∀ a, y .
If we train our classifier with the algorithm
[Equation]
g ∈ G
[Equation]
a,y
[Equation]
U
[Equation]
then the error and unfairness of the optimal solution ˆ g on the distribution U are bounded with
[Equation]
Remark.
This sufficient condition suggests us to fit a teacher classifier which is fair on the population distribution, and minimize the consistency loss of each group.
The unfairness of the resulting model is bounded by the quality (unfairness and error) of the teacher classifier, and the worst-group consistency loss.
Intuitively, we can understand the consistency loss as the model invariance to the nuisance factors.
With a group-balanced consistency loss, the model would have similar invariance to the nuisance factors resulting in similar group-performance on the unseen data, so that to transfer accuracy and fairness.
4.2 Practical Algorithm: Fair Consistency Regularization
There are two challenges in realizing the theoretical algorithm in Theorem 4.1.
First, we need a high-quality teacher model but the model trained with labeled source data is only fair and accurate in source domain.
Second, existing consistency regularization methods do not take fairness into consideration.
We tackle the first problem by leveraging the iterative self-training paradigm that updates teacher model with student model while training, and thus makes it fairer and fairer.
We tackle the second problem by proposing a novel fair consistency regularization.
Algorithm.
Figure 2 shows the overall training diagram.
There are three major components: (1) In every training
epoch, we use the student model obtained in last epoch as the teacher model
and automatically fit the teacher model by initializing the student model to be the same as the teacher model.
In other words, there is only one model training itself iteratively.
(2) To ensure the accuracy and fairness in the source domain, we adopt Laftr [41], an adversarial learning method consisting of a classification loss L cls and a fairness loss L fair .
(3) To transfer fairness and accuracy, we do consistency training on all the data.
Following FixMatch [54], we use the pseudo-labels generated by the teacher model as supervision for consistency training where the model should have consistent predictions under transformations.
Different from FixMatch, we propose a fair consistency regularization with a balanced group consistency loss L fconsis .
[Figure]
We train the model with the weighted summation of these three losses as shown in Figure 2.
We defer the detailed lose functions of L cls and L fair with a detailed algorithm desciption to Appendix D.
Fair Consistency Regularization.
To tighten the upper bound of the unfairness in Theorem 4.1, we need to minimize the worst-group consistency loss.
However, the consistency regularization in
FixMatch [54] does not distinguish groups which might amplify the bias as observed in [75] and our experiments.
Instead, we propose to use a fair consistency regularization that evaluates the consistency loss per group and minimize the balanced consistency loss L fconsis defined as below.
[Equation]
where L ya ( g ) = (cid:80) x ya
[Equation]
where x ya denotes an input with sensitive attribute
A = a and class Y =
y .
L ya ( g ) is model g ’s consistency in the group of { x ya } , and λ ya is the corresponding weight of the group consistency loss.
Here, we abuse the usage of g ( x ) to denote the output logits of model g on input x and thus, argmax( g tc ( x ia )) is the pseudolabel generated by teacher classifier.
t ( x ya ) is the transformed input as defined in Definition 4.1.
We use a cross-entropy loss H ( · ) to encourage the consistency under transformation t ( · ) and only consider examples that the teacher model has high confidence in with a confidence threshold τ .
Note that data is classified into groups according to the true sensitive attribute and pseudolabels.
To balance the group consistency loss, we propose to weight each group inversely with the number of confident pseudolabels, and set λ ya as
[Equation]
The weights will dynamically change while training.
Heuristically, if the teacher model is only confident in a few examples in a group, the model’s consistency in this group is more likely to be low.
With the proposed weights, larger penalty will be applied to such groups.
Therefore, the proposed fair consistency regularization will enforce the model to pay more attention to higher-error groups.
Related Work
This section features related work for transferring of fairness.
Other discussion of related work in fair machine learning, domain adaptation and self-training is deferred to Appendix A. Out-of-distribution fairness remains an under-explored area.
We categorize prior works into five classes.
1) Group-wise distribution matching .
[50] derive an upper bound for fairness in the target domain which suggests training a fair model in source domain and matching the distributions of relevant groups from two domains in feature space at the same time.
[63] also do group-wise distribution matching but with
Wasserstein distance.
Such methods are hard to achieve if we do not have supervision in target domain and it also shares the drawback of distribution matching methods. 2) Reweighting .
When the proportions of groups differ in two domains, reweighting the examples in source domain can approximate the target distribution.
[15] use reweighting to deal with fairness problem under covariate shift and [22] use reweighting together with a fairness test to guarantee the fairness under demographic shift.
Reweighting methods strongly rely on the support cover assumption which is not satisfied under domain shift.
Distributionally robust optimization (DRO) .
This line of work considers unknown target data that can be any arbitrary weighted combinations of the source dataset, and train a fair model that is robust to the worst-case shift [47, 42].
These methods also assume subpopulation shift instead of domain shift.
4) Causal inference .
[53] do causal domain adaptation and DRO based on a well-characterized causal graph that describes the data construction and distribution shift.
Causal methods highly rely on the correct causal graph which is hard to obtain in reality.
For example, [49] find that the causal graph in real applications (e.g. predicting the skin condition in dermatology) is far more complicated which violates normal assumptions, and thus making those approaches inapplicable.
5) Others .
There are also studies aim to maintain fairness under distribution shifts through online learning [68] and loss curvature matching [58].
To the best of our knowledge, this is the first work that uses self-training to transfer fairness.
Some work also studies self-supervised learning and fairness, yet they use unlabeled data and self-training to improve the in-distribution fairness [13, 67, 9] which is different from our goal.
Experiments 6.1 Evaluation under Different Types of Distribution Shift with a Synthetic Dataset
In order to study the fairness under distribution shifts and verify our theoretical findings, we develop a synthetic dataset to simulate different types of distribution shifts.
Synthetic dataset.
The synthetic dataset is adapted from the 3dshapes dataset [30] which contains images of 3D objects generated from six independent latent factors ( shape , object hue , scale , orientation , floor hue , wall hue ).
This dataset satisfies our assumption on the underlying data generation process very well.
We simulate different types of distribution shift by shifting the marginal distributions of latent factors.
By sampling images according to different marginal distributions of latent factors, we are able to simulate distribution shifts (see Appendix D.1 for details).
Distribution shifts.
We set the image as input X , and select three latent factors to be class ( Y = shape ), sensitive attribute ( A = object hue ), and a nuisance factor that might shift ( D = scale ).
We consider four widely observed distribution shifts in reality ( P S ( X, Y, A, D )
̸ = P T ( X, Y, A, D ) ):
(1) Sshift 1 : Subpopulation shift where only the nuisance factor shift (i
e. more small objects in source but more large objects in target), P S ( Y, A )
= P T ( Y, A ) , P S ( D ) ̸
= P T ( D ) .
(2) Sshift 2 : Subpopulation shift where A and Y have different correlations in two domains (i
e. most red objects are cubes in source but are capsules in target), P S ( Y, A )
̸ = P T ( Y, A ) , P S ( D ) =
P T ( D ) .
(3) Dshift :
Domain shift where the nuisance factor has different sample spaces (i
e. small objects in source but large objects in target), P S ( Y, A )
= P T ( Y, A ) , P S ( D ) ̸
= P T ( D ) , Y d S ̸ = Y d T .
(4) Hshift : Hybrid shift of (2) and (3).
Baselines.
We do shape classification task with an MLP model and compare our method with four baselines: Base (standard ERM); Laftr; Laftr+DANN (a combination of Laftr and a domain adaptation method [21]); Laftr+FixMatch.
In our method, we also use Laftr and FixMatch but with the proposed fair consistency regularization.
Since the shifted nuisance factor is scale , we use random padding and cropping as transformations in our method and Laftr+FixMatch.
We train Base and Laftr with labeled source data, and train others with unlabeled target data as well.
Domain shift is more challenging than subpopulation shift.
Figure 3 shows that under subpopulation shifts, the source fair model trained with Laftr also has high accuracy and fairness in the target domain although it has not seen any target date.
We observe that this is because the sample space is shared (e.g. small and large objects both exists in the source data), and the model has similar performance under all factor values.
Thus, the good performance remains even if the proportion of data changes, verified Proposition 3.1.
In contrast, under domain shifts and hybrid shifts, the source fair model performs poorly on data from a different sample space, suggesting the difficulty of domain shift.
Our method can transfer fairness and accuracy under various types of distribution shift.
Under domain shifts, the domain adaptation method DANN does not help in transferring fairness or accuracy.
Consistency regularization enforces the model to behave consistently under cropping and flipping, resulting in a model that has similar predictions regardless of the object’s scale, and thus to transfer accuracy.
However, it may cause bias as shown in the results of Laftr+FixMatch.
With the proposed fair consistency regularization, the model gains similar consistency for all groups, resulting in similar performance on all groups in the target domain.
Therefore, our method achieves both high accuracy and fairness in two domains under all the considered distribution shifts.
[Table]
[Figure]
6.2 Evaluation on Real datasets
Evaluation on images.
We use UTKFace [70] as the source data and FairFace [29] as the target data.
Although both of them are facial images, there is a distribution shift between them due to different [Figure] image sources.
We consider a gender classification task with the race as the sensitive attribute.
We use VGG16 [52] as the model and RandAugment [17] (excluding transformations that may change the group) as the transformation function.
Additional to previous baselines, we also compare with
Laftr+MMD [38] (a domain adaptation method), and use CFair [72] as the method for in-distribution fairness.
As shown in Table 1, there is indeed a distribution shift as the source model trained with
Laftr or CFair is no longer accurate or fair in the target domain.
Two domain adaptation methods are not able to transfer accuracy and may harm the fairness.
As expected, self-training (Laftr+Fixmatch and CFair+Fixmatch) significantly improves the accuracy in the target domain but the unfairness is high.
With the proposed fair consistency regularization, our method outperforms it remarkably on fairness with a 70% decrease of the group accuracy variance.
Evaluation on tabular data.
We further evaluate our method on NewAdult dataset [18] which contains census data from all states of the United States.
We do income classification task with an
MLP as the model.
We set CA as the source domain and all the other states as the target domain.
We use random perturbation on tabular data (see details in Appendix D) as the transformations.
Results are shown in Figure 5.
When applying to other states, the fair model trained on CA becomes unfair, as shown in Figure 5a.
Our method improves the fairness in most of the states with a slight improvement of accuracy as well.
6.3 Ablation Study
Fair consistency is essential in transferring fairness.
To see whether boosted consistency improves accuracy and whether imbalanced consistency causes unfairness as Theorem 4.1 suggests, we evaluate the accuracy and consistency for every group on target data in UTK-FairFace.
The consistency is measured by testing the model’s agreement on the outputs under two different random transformations.
As shown in Figure 6, groups that obtain higher consistency have higher accuracy, which validates the ability of consistency regularization for transferring accuracy.
The training methods that use standard consistency regularization (e.g. Laftr+FixMatch) have been observed to be unfair in the target domain.
Figure 6 shows that it is because the model has imbalanced group consistency.
With our fair consistency regularization, the model gains similar consistency for all groups, resulting in similar group accuracy.
Such finding verifies that fair consistency is essential in transferring fairness.
The role of transformation.
We design transformation functions based on our domain knowledge of the latent factors.
To satisfy intra-group expansion assumption , the transformations should be group-preserving (i.e. do not change class and sensitive attribute), so we exclude transformations such as color jittering in RandAugment when race is the sensitive attribute.
To investigate the importance of transformations, we test a weaker set of transformations (cropping and flipping) and report the performance in Table 2.
Comparing with
RandAugment (Table 1), consistency to only cropping and flipping is less effective in transferring [Table] since the neighbor generated by them is much smaller
[Table]
The role of components in fair consistency regularization.
Table 3 shows that the consistency in both domains matters.
Giving every group the same weight instead of using dynamic weights results in increased unfairness.
Fixing the teacher classifier to be the fair source model, we observe a significantly decrease of the accuracy, suggesting the important role of iterative self-training in our algorithm.
Conclusion
In this paper we explore transferring fairness under distribution shifts.
We present a theory-guided method of self-training based on an intra-group expansion assumption.
To provide fair consistency regularization, we suggest dynamically reweighting consistency loss with the number of confident pseudolabels.
We also offer a synthetic dataset simulating different types of distribution shifts to examine this topic.
Our technique and other self-training methods rely on a well-defined data transformation set.
Future work will relax this limitation for application to more real-world problems
Acknowledgements
This work is supported by National Science Foundation NSF-IIS-FAI program, DOD-ONR-Office of Naval Research,
DOD-DARPA-Defense Advanced Research Projects Agency
Guaranteeing AI
Robustness against Deception (GARD) and Adobe, Capital One and JP Morgan faculty fellowships
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section F (c)
Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] see Section D (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] see Section D 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Supplementary Material
A More on Related Work
Fair machine learning.
Generally, fair machine learning methods fall into three categories: preprocessing, in-processing, and post-processing [43, 7].
In this paper, we focus on in-processing methods that modify learning algorithms to remove discrimination during the training process.
As for fair classification, several approaches have been proposed including fair representation learning [65, 40, 4, 66, 41, 55, 16, 72], fairness-constrained optimization [19, 1], causal methods [33, 23, 44], and many other approaches with different techniques [8, 12, 24].
All of those works are for indistribution fairness and we investigate out-of-distribution fairness in this paper.
We use LAFTR [41], an adversarial learning method that shows advanced performance on fairness [46], to learn a fair model in the source domain and adapt it to the target domain.
We also test CFair[71] in our experiments.
Many metrics of fairness have been proposed [14] including demographic parity [6], equalized opportunity, and equalized odds [26] which are most widely adopted.
In this paper, we use equalized odds to measure unfairness in both domains.
Distribution shifts.
In many real-world applications, distribution shifts are unavoidable.
The goal of existing work addressing distribution shifts is simply to transfer accuracy.
[31] propose a benchmark of in-the-wild datasets to study the real distribution shifts.
We follow their category of distribution shifts, including subpopulation shifts and domain shifts.
Their empirical results on many state-of-theart methods show that self-training outperforms others on image datasets significantly while have limited performance due to the limited data augmentation on non-image modalities [48].
This finding aligns with our experimental results.
[61] conduct a fine-grained analysis of various distribution shifts based on an underlying data generation assumption which is similar to ours.
They also use 3dshapes dataset to simulate different types of distribution shifts.
Additional to accuracy, we aim at transferring fairness at the same time in this paper.
Domain adaptation and self-training.
Inspired by the theoretical work [2], numerous distribution matching approaches have emerged for domain adaption over the past decade.
Domain-adversarial training [21] and many of its variants [57, 39, 27, 56] that aim at matching the distribution of two domains in the feature space have shown encouraging results in many applications.
However, recent studies [62, 73, 35] show that such methods may fail in many cases since they only optimize part of the theoretical bound.
We test DANN [21] and MMD [38], two distribution matching methods in our experiments, and also find them less effective in transferring accuracy and fairness.
Recently, another line of work that uses self-training draws increasing attention [69, 3].
Those methods enjoy guarantees [60, 5] and demonstrate superior empirical results with desirable properties such as robustness to spurious features [32, 10, 37] and robustness to dataset imbalance [36].
However, all of those work on domain adaptation only aims at transferring accuracy.
Although there is work that study fairness issues in current domain adaptation methods [34] and propose to alleviate it by balancing the data [28, 59, 75], fair domain adaptation is still under-explored.
Based on the findings that model’s consistency to input transformations is important to generalization [74] and is a core component of self-training [51, 54, 25], we improve the consistency regularization in [54] to achieve fair transferring.
B Proof and More Discussion of Fairness under Distribution Shifts
Lemma B.1.
Under Assumption 1, for a subpopulation shift that caused by the shift of marginal distribution of factor Y i , we have P S ( X | Y i
=
y i ) =
P T ( X | Y i
=
y i ) ,
∀ y i ∈ Y
i .
Proof.
Under Assumption 1, P S ( X | Y 1: K = y 1: K ) =
P T ( X | Y 1: K = y 1: K ) .
Since the shift only happens on the factor Y i , the marginal distribution of other factors remains the same in two domains,
P S ( Y { 1: K }\ i ) =
P T ( Y { 1: K }\ i ) where we use { 1 : K } \ i to denote 1 , .., i − 1 , i + 1 , ..., K .
Then
P S ( X | Y i =
y i ) =
[Equation]
where the second line holds because of the independence of the latent factors Y 1 , ..., Y K .
Now, we restate Proposition 3.1 and provide the proof.
Proposition B.1 (Transfer of fairness under subpopulation shift) .
Consider the subpopulation shift that is caused by the shifted marginal distribution of a nuisance factor
Y i (i.e., P S ( Y i )
̸
= P T ( Y i ) ), while Y iS =
Y iT
=
Y i .
If model f is strictly fair in source domain under any value of factor Y i satisfying P S ( f ( X ) =
y l | Y a = 0 , Y l = y l , Y i =
y i ) =
P S ( f ( X ) =
y l | Y a = 1 , Y l = y l , Y i =
y i ) ,
∀ y i ∈ Y i , y l ∈ { 0 , 1 } , then f is also fair in target domain with ∆ odds = 0 .
Proof.
In the target domain, the equalized odds (unfairness) is defined as
[Equation]
y l = (cid:12)(cid:12) P T ( f ( X ) =
y l | Y a = 0 , Y l = y l ) − P T ( f ( X ) =
y l | Y a = 1 , Y l = y l ) (cid:12)(cid:12) .
Since all latent factors are independent, we have
P T (cid:0) f ( X ) =
y l | Y a = 0 , Y l = y l (cid:1) =
(cid:88)
y i ∈Y i
P T ( Y i =
y i )
P T (cid:0) f
( X ) =
y l | Y a = 0 , Y l = y l , Y i =
y i (cid:1) and
P T (cid:0) f ( X ) =
y l | Y a = 1 , Y l = y l (cid:1) =
(cid:88)
[Equation]
i (cid:1) .
Therefore, the ∆ odds in the target domain can be decomposed into
[Equation]
−
[Equation]
Since two domains share the same underlying data generative model, and the distribution shift is caused by the shift of the marginal distribution of factor Y i , from Lemma B.1, we have
[Equation]
| Y a = 0 , Y l = y l , Y i =
y i (cid:1) and
P T (cid:0) X | Y a = 1 , Y l = y l , Y i =
y i (cid:1) =
P S (cid:0) X | Y a = 1 , Y l = y l , Y i =
y i (cid:1) .
Thus the conditional distribution of the model’s prediction also remains, as
P T (cid:0) f
[Equation]
X ) | Y a
[Equation]
0 , Y l
[Equation]
y l , Y i
[Equation]
y i
[Equation]
= P
[Equation]
X
[Equation]
| Y a
[Equation]
0 , Y l = y l , Y i =
y
i
[Equation]
and
P
[Equation]
In this case, if the source model is strictly fair that ∀
y i ∈
Y i , y l ∈ { 0 , 1 } the following holds
P S ( f ( X ) =
y l | Y a = 0 , Y l = y l , Y i =
y i ) =
P S ( f ( X ) =
y l | Y a = 1 , Y l = y l , Y i =
y i )
, then it is also fair in the target domain with ∆ odds = 11 (cid:88)
y l
[Equation]
(cid:12)(cid:12)
[Equation]
y i ∈Y i
P
[Equation] [Equation]
l |
[Equation]
0 , Y l
[Equation]
y l , Y
[Equation]
y
[Equation]
−
[Equation]
y
[Equation]
Y
[Equation]
y
[Equation]
= 0 .
This proposition explains why the fair source model is also fair in the target domain under Sshift 1 in our experiments (see Section 6.1).
In addition to shifts of nuisance factors, the subpopulation shifts can also be caused by the marginal distribution shift of the label and sensitive attribute.
The following proposition argues that the fair model is also in the target domain under such distribution shifts.
Proposition B.2 (Transfer of fairness under subpopulation shift of sensitive attribute) .
Consider the subpopulation shift that is caused by the shifted marginal distribution of sensitive attribute Y a (i.e., P
S ( Y a )
̸ = P T ( Y a ) ), while Y aS =
Y aT =
Y a = { 0 , 1 } .
If model f is fair in the source domain with ∆ Sodds = 0 , then it is also fair in the target domain with ∆ Todds = 0 .
Proof.
The proof is similar to the previous one.
Since
[Equation]
and from Lemma B.1 we know that
[Equation]
thus,
[Equation]
Such result also holds for subpopulation shifts that caused by the shift of label Y l .
This proposition explains why the fair source model is also fair in the target domain under Sshift 2 in our experiments (see Section 6.1).
It suggests that encouraging fairness is able to alleviate spurious correlation.
We leave more studies on this interesting finding to future work.
Remark.
All the above analyses are based on the population distribution where P
S ( f ( X ) =
y l | Y a = 0 , Y l = y l ) =
E P S ( X,Y 1: K ) ( f ( X ) =
y l | Y a = 0 , Y l = y l ) .
In practice, it is estimated by finite samples.
Insufficient samples would cause estimation error of fairness and bring another challenge for transferring fairness.
In this paper, we only consider the fairness measured by population distribution.
Future work will investigate the impact of estimation error on transferring fairness, and the way to resolve it.
C Proof of the Sufficient Condition for Transferring Fairness
Our proof is based on the theory in [60].
Theorem C.1.
(Restatement of Lemma A.8 in [60])
We assume that U ya satisfies (¯ α, ¯ c ) -multiplicative expansion for ε U ya
( g tc ) ≤ ¯ α < 1 / 3 and ¯ c > 3 .
We define c ≜ min { 1 / ¯ α, ¯ c } .
Then for any classifier g : X → Y , the error of it on the group U ya is upper bounded as:
[Equation]
Theorem C.2.
(A restricted version of the above theorem)
We assume that U y a satisfies (¯ α, ¯ c ) - multiplicative expansion for ε
U ya ( g tc ) ≤ ¯ α < 1 /
and ¯ c > 3 .
We define c ≜ min { 1 / ¯ α, ¯ c } .
Then for any classifier g : X → Y satisfies L U ya ( g, g tc ) ≤ ε U ya ( g tc ) , the error of it on the group
U ya is upper bounded as:
[Equation]
Proof.
[Equation]
L U ya
[Equation]
g
[Equation]
g tc
[Equation]
U
[Equation]
g
[Equation]
Theorem C.3.
If L U ya ( g, g tc ) ≤ ε U ya ( g tc ) , we have
[Equation]
Proof.
By triangle inequality.
Now, we restate Theorem 4.1 and provide the proof.
Theorem C.4.
Suppose we have a teacher classifier g tc with bounded unfairness such that | ε U ya ( g tc ) − ε U y ′
a ′ ( g tc ) | ≤ γ, ∀ a, a ′ ∈ A and y, y ′ ∈ Y .
We assume intra-group expansion where
U ya satisfies (¯ α, ¯ c ) -multiplicative expansion and ε U ya ( g tc ) ≤ ¯ α < 1 /
and ¯ c > 3 , ∀ a, y .
We define c ≜ min { 1 / ¯ α, ¯ c } , and set µ ≤ ε U y a ( g tc ) , ∀ a, y .
If we train our classifier with the algorithm
[Equation]
then the error and unfairness of the optimal solution ˆ g on the distribution U are bounded with
[Equation]
Proof.
The upper bound of error is derived from Theorem C.2.
For the unfairness, by definition
[Equation]
Based on the upper bound of group error from Theorem C.2, and the lower bound of it from
Theorem C.3, we have
[Equation] [Equation]
Therefore, ∆ odds ≤
[Equation]
Upper bound of V acc .
From Theorem C.2 we know that the group accuracy is upper bounded by ε U ya ( g ) ≤ 2 c −
ε U ya ( g tc ) + 2 c c − 1 R U ya ( g ) .
The variance of group accuracy is defined as
[Equation]
If we assume the same estimation error for all the groups when we use the upper bound to estimate the group accuracy with ε U ya ( g ) = 2 c − 1 ε U ya
( g tc ) + 2 c c − 1 R U ya ( g ) , then
[Equation]
When the teacher classifier has bounded unfairness with | ε U ya ( g tc ) − ε U y ′
a ′ ( g tc ) | ≤ γ, ∀ a, a ′ , y, y ′ , the variance of group accuracy would be mainly affected by the variance of group consistency loss
V ar ( { R U ya ( g ) , ∀ a, y } ) .
Therefore, this upper bound also suggests us to balance the consistency loss while minimizing it.
Multi-sensitive attribute and multi-class cases.
It is obvious that the Theorem 4.1 still holds for the binary-sensitive attribute and multi-class case
where Y = { 1 , 2 , .., M } .
For the multi-sensitive attribute case, the key problem is how to define the unfairness.
If we define the equalized odds in general cases to be the following one, then it is easy to see that Theorem 4.1 still holds.
[Equation]
[Table]
[Equation]
[Table]
D Details of Experiments D.1 Synthetic Dataset
The 3dshapes dataset 2 [30] contains 480000 RGB images (the size is 64 × 64 × 3 ) of 3D objects.
Every image is generated by six latent factors ( shape , object hue , scale , orientation , floor hue , wall hue ) which are annotated along with images.
The sample spaces of these factors are shown in Table 4.
[Table]
How to simulate different types of distribution shift?
By varying the marginal distribution of latent factors and then sample images according to the distribution of latent factors, we can simulate different distribution shifts.
In this paper, we set the image as input X , and set class Y = shape , sensitive attribute
A = object hue , and a nuisance factor that might shift to be D = scale .
We consider a binary case, and restrict the shape to be in { 0 , 1 } (i.e. { cube , cylinder } ) and object hue to be in { 0 , 1 } (i.e. { red , yellow } ).
In our experiments, we simulate four types of distribution shift.
Their specific settings are shown in Table 5.
We show examples from two domains under different shifts in Figure 7.
D.2 Real Datasets
UTKFace 3 [70] is a face dataset with images annotated with age, gender, and race.
The data is collected from MORPH, CACD and Web.
In our experiments, we use the aligned and cropped face images with ages larger than 10.
We do gender classification which is a binary classification task, and set sensitive attribute to be the race.
We consider binary-sensitive attribute case in our experiment by setting race to be white or non-white.
The statistics of this dataset are shown in Table 6.
S ou r ce
T a r g e t
[Figure]
S ou r ce
T a r g e t
[Figure]
S ou r ce
T a r g e t
[Figure]
S ou r ce
T a r g e t
[Figure]
Factor
Source
Target
Sshift
P ( Y, A ) [0 . 1 , 0 . 4 , 0 . 4 , 0 . 1] same
P ( D ) [ 416 , 416 , 316 , 116 , 116 , 116 , 116 , 116 ] [ 116 , 116 , 116 , 116 , 116 , 316 , 416 , 416 ]
Sshift
P ( Y, A ) [0 . 1 , 0 . 4 , 0 . 4 , 0 . 1] [0 . 4 , 0 . 1 , 0 . 1 , 0 . 4]
P ( D ) [ 18 , 18 , 18 , 18 , 18 , 18 , 18 , 18 ] same
Dshift
P ( Y, A ) [0 . 1 , 0 . 4 , 0 . 4 , 0 . 1] same
P ( D ) [ 12 , 12 , 0 , 0 , 0 , 0 , 0 , 0] [ 18 , 18 , 18 , 18 , 18 , 18 , 18 , 18 ]
Hshift
P ( Y, A ) [0 . 1 , 0 . 4 , 0 . 4 , 0 . 1] [0 . 4 , 0 . 1 , 0 . 1 , 0 . 4]
P ( D ) [ 12 , 12 , 0 , 0 , 0 , 0 , 0 , 0] [ 18 , 18 , 18 , 18 , 18 , 18 , 18 , 18 ] distributions.
Images in two domains are sampled according to the marginal distributions of six latent factors.
FairFace 4 [29] is another large-scale face dataset with images annotated with age, gender, and race as well.
Different from UTKFace, the data in FairFace is collected from Flickr, Twitter, Newspapers
[Table]
and Web.
We also use images with ages larger than 10 in our experiments.
We set the label to be the gender and the sensitive attribute to be the race.
See statistics in Table 6.
All face images in UTK and
FairFace are resized to 96 × 96 × 3 in our experiments.
NewAdult 5 [18] is a suite of datasets derived from US Census surveys.
The data spans multiple years and all states of the United States which is a good fit for studying distribution shifts.
In our experiments, we use 2018 data that span all states, and do income classification with a threshold of 50,000 dollars.
We set the gender to be sensitive attribute.
We consider a problem that we train a fair classifier in California (source domain) and deploy it in other states (target domain).
The statistics are shown in Table 7.
The input contains 10 features (see Appendix B.1 in [18]) which are preprocessed to one-hot embeddings in our experiments.
[Table]
D.3 Experimental Settings D.3.1 Experiments on 3dshapes
Model.
We use a two-layer MLP with 512 hidden units as the encoder and one linear layer as the classifier.
The adversaries used in Laftr, CFair and DANN are also two-layer MLP with 512 hidden units.
ReLU is used as the activation function.
Transformations.
We use random center cropping and padding as the transformation functions in consistency regularization.
Such transformations can perfectly change the scale of the objects, so that to propogate labels from the source domain to the target domain.
Setup.
We use SGD as the optimizer, and set the learning rate to be 0.01.
We train every model with 200 epochs and select the best model according to the model’s performance on the validation set.
For
Base and Laftr that only have access to the source data, the model selection is based on the source validation set, and for other methods that can access to the unlabeld target data, the model selection is based on the labeled target validation set.
Since accuracy and fairness are both important metrics, we select the best model according to the value of accuracy minus unfairness which is Acc − ∆ odds .
The coefficients of the fairness loss and consistency loss are both set to be 1.
We run every method for five times and report the mean and the standard deviation.
D.3.2 Experiments on UTK-FairFace
Model.
We use VGG16 and ResNet18 as the model in our experiments.
The last linear layer is the classifier and all the previous layers construct the encoder.
When we use VGG16 as the model, we set every adversary used in Laftr, CFair and DANN to be a two-layer MLP with 1024 hidden neurons.
When ResNet18 is the model, the adversary has 512 hidden neurons.
Transformations.
We use RandAugment [17] as the transformation function which contains data augmentations that are the best for CIFAR-10 dataset.
To restrict the transformations to be grouppreserving, we exclude augmentations that may change the color (so to change the race).
The transformation function used in our experiments contains AutoContrast, Brightness, Equalize, Identity,
Posterize, Rotate, Sharpness, ShearX, ShearY, TranslateX, TranslateY.
In this experiment, we use a weak augmented (with random cropping and flipping) image as the original input x , and a strong augmented (with RandAugment) image as the transformed input t ( x ) .
Setup.
We use SGD as the optimizer.
The learning rate for Base, Laftr, and CFair is 0.001 and is 0.001 for other methods.
We train every model with 200 epochs and use the way introduced in the 3dshapes experiment to select the best model.
The coefficients of the fairness loss and consistency loss are both set to be 1.
We run every method for five times and report the mean and the standard deviation.
D.3.3 Experiments on NewAdult
Model.
We use a 3-layer MLP with hidden sizes of (256, 512, 256) as the encoder, and a 2-layer
MLP with the hidden size of 128 as the classifier.
Every adversary is a two-layer MLP withhidden neurons.
Transformations.
Studies on data augmentations for tabular data is very limited.
In this paper, we use random corruptions on the input features as the transformation function.
There are ten features in the input, and every time we only corrupt half of them.
Additionally, for important factors that highly correlated with the label, including the OCCP (occupation), COW (class of worker), we do not do any corruptions.
For factor SEX (gender), we also do not do any corruptions to preserve the group.
For continues factors including AGEP (age), SCHL (educational attainment) and WKHP (work hours), we do perturbations within a range.
For other factors, we do uniformly sampling from their value spaces as corruptions.
We do such transformations based on our assumption that they do not change the label.
For example, two individuals that have a 5 years of age gap but have the same other features should have similar income, and two individuals that only differ in the place of birth should have similar income.
We admit that such transformations may not be the best ones.
We need better domain knowledge on the income prediction to design more powerful transformations.
We leave the improvement of transformations for tabular data to future work.
Setup.
We use SGD as the optimizer.
The learning rate is set to be 0.01.
We train every model with 200 epochs and use the metric introduced in the 3dshapes experiment to select the best model.
The coefficients of the fairness loss and consistency loss are both set to be 1.
We run every method for five times and report the mean and the standard deviation.
D.4 Baselines
Laftr is an adversarial learning method for algorithmic fairness.
The adversary aims at accurately predicting the sensitive attribute based on the representation, while the encoder aims at making it hard.
By adversarial learning, the representation will not contain information of sensitive attribute, so that the prediction based on it will be fair.
We denote the data in each group to be D ya = { x ∈
D| A = a, Y = y } .
The fairness loss is designed to be
[Equation]
where f is the encoder, h is the adversary.
[41] prove that this loss is an upper bound of the equalized odds.
The adversary minimizes this loss while the encoder maximizes this loss with a gradient reversal layer.
CFair is similar to Laftr but uses two adversaries h ′ and h ′′ for two classes with a balanced error rate (BER) defined as follows.
We denote the data from one class to be D y
= { x ∈ D| Y = y } .
[Equation]
where BER D
( h ′ ( f ( x ))
∥ A ) = 12 P D 0 ( h ′ ( f ( x )) ̸
= A | A = 0) + 12 P D 0 ( h ′ ( f ( x ))
̸
= A | A = 1) .
In practice, the balanced error rate is estimated by the following cost-sensitive cross-entropy loss.
[Equation]
Laftr+FixMatch uses the same framework as our method but with a standard consistency regularization that does not care about group performance.
The consistency loss is defined as
[Equation]
where D denotes the entire dataset.
D.5 Time and Space Complexity
Comparing with Base, Laftr and CFair that only uses labeled source data, our method need more training time and memory since we use unlabeled target data as well.
Comparing with other baselines that also use target data, such as Laftr+DANN, the time complexity of our method is comparable to theirs.
Nevertheless, our method needs much less parameters than Laftr+DANN since it requires an adversary to do domain classification while we do not need it.
Actually, our method has the same model parameters as that in Laftr but with an additional consistency loss.
E More Experimental Results E.1 Additional Results on UTKFace-FairFace with a Different Data Setting
To evaluate our method in extreme circumstances, we conduct the UTK-FairFace experiment with less labeled source data and more unlabeled target data (see Table 8).
We also consider the race to be "white" and "non-white".
Are shown in Table 9, we get consistent results that our method outperforms all baselines, and can effectively transfer accuracy as well as fairness.
[Table]
E.2 Additional Results on UTKFace-FairFace with Different Transformations
To investigate the effect of using different transformations in our method, we evaluate 14 transformations in RandAugment and report the results in Table 9.
All the transformations can improve the accuracy in both domains.
The effect on fairness varies.
We find that Solarize , Color and TranslateX increase the unfairness in the source domain the most, and Contrast , Color and Solarize have the highest unfairness in the target domain.
Note that, it does not mean that these augmentations always lead to unfairness but that they are not suitable for our method.
Recall that our theory and algorithm are built upon the intra-group expansion assumption.
Transformations like Contrast , Color and Solarize may change the sensitive attribute "race" and break this assumption.
Thus, in our experiments (Table 1) we use all the transformations excluding Contrast , Color and Solarize . [Table] [Table]
E.3 Additional Results on NewAdult with Different Transformations
[Figure]
Pre-defined transformation functions require good domain knowledge on the task which is hard to obtain for the NewAdult dataset.
To alleviate this issue, we try to use Mixup 6 as the transformation function on the tabular data.
Mixup does extrapolation between training examples as the data augmentation which is applicable to any type of data without requiring any prior knowledge.
We apply Mixup on NewAdult dataset.
Since Mixup requires the access to the label, we only use labeled source data to train the model in this experiment.
We use CA as the source domain, and evaluate on all the other states.
We set race (white and non-white) to be the sensitive attribute.
Figureshows that Mixup improves the in-distribution accuracy and fairness but harms the out-of-distribution fairness with neglectable improvement in accuracy.
This preliminary experiment suggests that Mixup that only applies to the source data can only benefit the accuracy and fairness in the source domain.
We suspect that it is because the augmented data generated by Mixup is hard to expand to other domains, and they are not group-preserving.
One interesting future work is to do intra-group Mixup and apply it to the unlabeled target data as well by using confident pseudolabels.
All the experiments on NewAdult dataset encourage the community to design more powerful transformations for tabular data, so that to transfer accuracy.
After that, the proposed fair consistency regularization is able to help making this transferring process fair.
E.4 A Byproduct: Alleviate the Disparate Impact of Semi-supervised Learning
[75] find that semi-supervised learning methods may have disparate impact.
The classes that have high accuracy on labeled data is likely to benefit more from semi-supervised learning on unlabeled data.
We test this argument on CIFAR-10 with FixMatch as the semi-supervised learning method.
We use ResNet18 as the model.
We randomly sample 500 images to be the labeled data and treat others as unlabeled data.
We use the benefit ration proposed in [75] as the metric for the benefit of semi-supervised learning, defined as
[Equation]
8) where D denotes the data from one class.
a semi ( D ) is the model’s test accuracy after semi-supervised learning, a baseline ( D ) is the test accuracy of the base model that is trained on labeled data, and a ideal ( D ) is the test accuracy of the ideal model where all data are labeled.
We evaluate the benefit ratio of FixMatch on ten classes.
As the blue line in Figure 10 shows, the rich gets richer and the poor gets poorer after semi-supervised learning.
Our method (without using Laftr) can directly apply to this task.
By using the proposed fair consistency regularization (red line in Figure 10), we can significantly improve the benefit ratio of the poor classes.
Therefore, the fair consistency regularization is a byproduct of this paper which is able to alleviate the disparate impact of semi-supervised learning.
[Figure] and Limitations
Fairness of machine learning is a critical problem we are facing in todays real-world applications.
When distribution shifts happen, the collapse of fair systems would cause unexpected discrimination, resulting in severe negative social impact.
The fairness that robust to distribution shifts is essential but is less explored.
In this paper, the theoretical analysis of how the fairness changes under different distribution shifts sheds light on the deep reasons of the collapse of fairness.
The theory-guided self-training algorithm proposed in this paper explores a promising way to tackle distribution shifts.
We hope our work would inspire more algorithms for this important and practical task.
The major limitation of our method is that it strongly relies on the pre-defined transformations as all the other self-training methods.
The transformations are designed to be group-preserving based on our prior knowledge.
Our experiments show that self-training with less powerful transformations have limited ability in propagating labels from source to target (i.e. transfer accuracy).
Valid transformation functions on image data are thoroughly studied in exiting work, while transformations on non-image data such as tabular data are much less explored.
Our method with more powerful transformations on tabular data is expected to have significant improvement.
Future work is encouraged to relax this limitation, such as by using a generative model as the transformation function.