F AIRNESS AND A CCURACY UNDER D OMAIN G ENERALIZATION
A BSTRACT
As machine learning (ML) algorithms are increasingly used in high-stakes applications, concerns have arisen that they may be biased against certain social groups.
Although many approaches have been proposed to make ML models fair, they typically rely on the assumption that data distributions in training and deployment are identical.
Unfortunately, this is commonly violated in practice and a model that is fair during training may lead to an unexpected outcome during its deployment.
Although the problem of designing robust ML models under dataset shifts has been widely studied, most existing works focus only on the transfer of accuracy.
In this paper, we study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from never-before-seen domains.
We first develop theoretical bounds on the unfairness and expected loss at deployment, and then derive sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning.
Guided by this, we design a learning algorithm such that fair ML models learned with training data still have high fairness and accuracy when deployment environments change.
Experiments on real-world data validate the proposed algorithm.
Model implementation is available at https://github.com/pth1993/FATDM .
I NTRODUCTION
Machine learning (ML) algorithms trained with real-world data may have inherent bias and exhibit discrimination against certain social groups.
To address the unfairness in ML, existing studies have proposed many fairness notions and developed approaches to learning models that satisfy these fairness notions.
However, these works are based on an implicit assumption that the data distributions in training and deployment are the same, so that the fair models learned from training data can be deployed to make fair decisions on testing data.
Unfortunately, this assumption is commonly violated in real-world applications such as healthcare e.g., it was shown that most US patient data for training ML models are from CA, MA, and NY, with almost no representation from the otherstates (Kaushal et al., 2020).
Because of the distribution shifts between training and deployment, a model that is accurate and fair during training may behave in an unexpected way and induce poor performance during deployment.
Therefore, it is critical to account for distribution shifts and learn fair models that are robust to potential changes in deployment environments.
The problem of learning models under distribution shifts has been extensively studied in the literature and is typically referred to as domain adaptation/generalization, where the goal is to learn models on source domain(s) that can be generalized to a different (but related) target domain.
Specifically, domain adaptation requires access to (unlabeled) data from the target domain at training time, and the learned model can only be used at a specific target domain.
In contrast, domain generalization considers a more general setting when the target domain data are inaccessible during training; instead it assumes there exists a set of source domains based on which the learned model can be generalized to an unseen, novel target domain.
For both problems, most studies focus only on the generalization of accuracy across domains without considering fairness, e.g., by theoretically examining the relations between accuracy at target and source domains (Mansour et al., 2008; 2009; Hoffman et al., 2018;
Zhao et al., 2018; Phung et al., 2021; Deshmukh et al., 2019; Muandet et al., 2013; Blanchard et al., 2021; Albuquerque et al., 2019; Ye et al., 2021; Sicilia et al., 2021; Shui et al., 2022) or/and developing practical methods (Albuquerque et al., 2019; Zhao et al., 2020;
Li et al., 2018a; Sun &
Saenko, 2016; Ganin et al., 2016; Ilse et al., 2020; Nguyen et al., 2021).
To the best of our knowledge, only Chen et al. (2022); Singh et al.
; Coston et al.
Rezaei et al.
; Oneto et al.
; Madras et al.
; Schumann et al.
; Yoon et al.
(2020) considered the transfer of fairness across domains.
However, all of them focused on domain adaptation, and many also imposed rather strong assumptions on distributional shifts (e.g., covariate shifts (Singh et al., 2021;
Coston et al., 2019; Rezaei et al., 2021), demographic shift (Giguere et al., 2022), prior probability shift (Biswas & Mukherjee, 2021)) that may be violated in practice.
Among them, most focused on empirically examining how fairness properties are affected under distributional shifts, whereas theoretical understandings are less studied (Schumann et al., 2019; Yoon et al., 2020).
Details and more related works are in Appendix A.
[Figure]
In this paper, we study the transfer of both fairness and accuracy in domain generalization via invariant representation learning, where the data in target domain is unknown and inaccessible during training.
A motivating example is shown in Figure 1.
Specifically, we first establish a new theoretical framework that develops interpretable bounds on accuracy/fairness at a target domain under domain generalization, and then identify sufficient conditions under which fairness/accuracy can be perfectly transferred to an unseen target domain.
Importantly, our theoretical bounds are fundamentally different from the existing bounds, compared to which ours are better connected with practical algorithmic design, i.e., our bounds are aligned with the objective of adversarial learning-based algorithms, a method that is widely used in domain generalization.
Inspired by the theoretical findings, we propose Fairness and Accuracy Transfer by Density Matching ( FATDM ), a two-stage learning framework such that the representations and fair model learned with source domain data can be well-generalized to an unseen target domain.
Last, we conduct the experiments on real-world data; the empirical results show that fair ML models trained with our method still attain a high accuracy and fairness when deployment environments differ from the training.
Our main contributions and findings are summarized as follows:
• We consider the transfer of both accuracy and fairness in domain generalization.
To the best of our knowledge, this is the first work studying domain generalization with fairness consideration.
• We develop upper bounds for expected loss (Thm. 1) and unfairness (Thm. 3) in target domains.
Notably, our bounds are significantly different from the existing bounds as discussed in Appendix A.
We also develop a lower bound for expected loss (Thm. 2); it indicates an inherent tradeoff of the existing methods which learn marginal invariant representations for domain generalization.
• We identify sufficient conditions under which fairness and accuracy can be perfectly transferred from source domains to target domains using invariant representation learning (Thm. 4).
• We propose a two-stage training framework (i
e., based on Thm. 5) that learns models in source domains (Sec. 4), which can generalize both accuracy and fairness to target domain.
• We conduct experiments on real-world data to validate the effectiveness of the proposed method.
P ROBLEM F ORMULATION
Notations.
Let X , A , and Y denote the space of features, sensitive attribute (distinguishing different groups, e.g., race/gender), and label, respectively.
Let Z be the representation space induced from
X by representation mapping g : X → Z .
We use X , A , Y , Z to denote random variables that take values in X , A , Y , Z and x, a, y,
z the realizations.
A domain D is specified by distribution
P D :
X × A × Y → [0 , 1] and labeling function f D : X → Y ∆ , where ∆ is a probability simplex over Y .
Similarly, let h D : Z → Y ∆ be a labeling function from representation space for domain D .
Note that f D , h D , g are stochastic functions and f D = h D ◦ g .
1 For simplicity
, we use P VD (or P V | U D ) to denote the induced marginal (or conditional) distributions of variable V (given U ) in domain D .
Error metric.
Consider hypothesis (cid:98) f
= (cid:98) h ◦ g : X → Y ∆ ,
where (cid:98) h : Z → Y ∆ is the hypothesis directly used in representation space.
Denote (cid:98) f
( x ) y as the element on y -th dimension which predicts the probability that label Y =
y given X = x .
Then the expected error of (cid:98) f in domain D is defined as ϵ AccD ( (cid:98) f ) =
E D
[ L ( (cid:98) f
( X ) , Y )] for some loss function L : Y ∆ × Y → R + (e.g., 0-1 loss, cross-entropy loss).
Similarly, define the expected error of (cid:98) h in representation space as ϵ AccD ( (cid:98) h ) =
E D
[ L ( (cid:98) h
( Z ) , Y )] .
Note that most existing works (Albuquerque et al., 2019; Zhao et al., 2018) focus on optimizing ϵ AccD ( (cid:98) h ) , while our goal is to attain high accuracy in input space, i.e., low ϵ AccD
( (cid:98) f ) .
Unfairness metric.
We focus on group fairness notions (Makhlouf et al., 2021) that require certain statistical measures to be equalized across different groups; many of them can be formulated as (conditional) independence statements between random variables (cid:98) f
( X ) , A, Y , e.g., demographic parity ( (cid:98) f
( X ) ⊥ A : the likelihood of a positive outcome is the same across different groups)
(Dwork et al., 2012) , equalized odds ( (cid:98) f
( X ) ⊥
A | Y : true positive rate (TPR) and false positive rate (FPR) are the same across different groups), equal opportunity ( (cid:98) f
( X ) ⊥ A | Y = 1 when Y = { 0 , 1 } : TPR is the same across different groups) (Hardt et al., 2016).
In the paper, we will present the results under equalized odds ( EO ) fairness with binary Y = { 0 , 1 } and A = { 0 , 1 } , while all the results (e.g., methods, analysis) can be generalized to multi-class, multi-protected attributes, and other fairness notions.
Given hypothesis (cid:98) f
= (cid:98) h ◦ g : X → Y ∆ , the violation of EO in domain D can be measured as ϵ EO D ( (cid:98) f ) =
(cid:80) y ∈Y D (cid:16) P (cid:98) f
( X ) 1 | Y
=
y,A =0 D || P (cid:98) f
( X ) 1 | Y
=
y,A =1 D (cid:17) for some distance metric D ( ·||· ) .
Problem setup.
Consider a problem of domain generalization where a learning algorithm has access to data { ( x k , a k , y k , d k ) } mk =1 sampled from a set of N source domains { D Si } i ∈ [ N ] , where d k is the domain label and [ N ] = { 1 , · · · , N } .
Our goal is to learn a representation mapping g : X → Z and a fair model (cid:98) h : Z → Y ∆ trained on source domains such that the model (cid:98) f
= (cid:98) h ◦ g can be generalized to an unseen target domain D T in terms of both accuracy and fairness.
Specifically, we investigate under what conditions and by what algorithms we can guarantee that attaining high accuracy and fairness at source domains { D Si } Ni =1 implies small ϵ Acc D T ( (cid:98) f ) and ϵ EO D T ( (cid:98) f ) at unknown target domain.
T HEORETICAL R ESULTS
In this section, we present the results on the transfer of accuracy/fairness under domain generalization via domain-invariant learning (proofs are shown in Appendix E).
We first examine that for any model (cid:98) h : Z → Y ∆ and any representation mapping g : X → Z , how the accuracy/fairness attained at source domains { D Si } Ni =1 can be affected when (cid:98)
f = (cid:98) h ◦ g is deployed at any target domain D T .
Specifically, we can bound the error and unfairness at any target domain based on source domains.
Before presenting the results, we first introduce the discrepancy measure used for measuring the dissimilarity between domains.
Discrepancy measure.
We adopt Jensen-Shannon (JS) distance (Endres & Schindelin, 2003) to measure the dissimilarity between two distributions.
Formally, JS distance between distributions P and P ′ is defined as
[Equation]
where D JS ( P || P
[Equation]
= 12 D KL ( P || P + P
[Equation]
2 ) + 12 D KL ( P ′ ||
P + P ′ 2 ) is JS divergence defined based on
Kullback–Leibler (KL) divergence D KL ( ·||· ) .
Note that unlike KL divergence, JS divergence is symmetric and bounded: 0 ≤ D JS ( P || P ′ ) ≤ 1 .
While different discrepancy measures such as H and H ∆ H divergences (Ben-David et al., 2010) (i.e., definitions are given in Appendix A) were used in prior works, we particularly consider JS distance because (1) it is aligned with training objective for discriminator in generative adversarial networks (GAN) (Goodfellow et al., 2014), and many existing methods (Ganin et al., 2016; Albuquerque et al., 2019) for invariant representation learning are built based on GAN framework; (2) H and H ∆ H divergences are limited to settings where the labeling functions f D are deterministic (
Ben-David et al., 2010; Albuquerque et al., 2019; Zhao et al., 2018)
In contrast, our bounds admit the stochastic labeling functions.
The limitations of other discrepancy measures and existing bounds are discussed in detail in Appendix A.
Theorem 1 (Upper bound: accuracy) For any hypothesis (cid:98) h : Z → Y ∆ , any representation mapping g : X → Z , and any loss function L :
Y ∆ × Y → R + that is upper bounded by C , the expected error of (cid:98) f
= (cid:98) h ◦ g : X → Y ∆ at any unseen target domain D T is upper bounded:
[Equation]
The upper bound in Eq.
(1) are interpretable and have three terms: term (i) is the averaged error of source domains in input space; term (ii) is the discrepancy between the target domain and the source domains in input space; term (iii) is the discrepancy between the source domains in representation space.
3 It provides guidance on learning the proper representation mapping g : X → Z : to ensure small error at target domain ϵ Acc D T ( (cid:98) f ) , we shall learn representations such that the upper bound of ϵ Acc D T ( (cid:98) f ) is minimized.
Because term (ii) depends on the unknown target domain D T and it’s evaluated in input space
X × Y , it is fixed and is out of control during training, we can only focus on term (i) and term (iii) , i.e., learn representations Z such that errors at source domains
ϵ Acc D Si ( (cid:98) f ) and the discrepancy between source domains in the representation space d JS ( P Z,YD Si , P Z,YD Sj ) are minimized.
Corollary 1.1 ∀ i, j , JS distance between P Z,YD Si and P Z,YD Sj in Eq.
(1) can be decomposed:
[Equation]
where P YD Si,j = 12 (cid:16) P YD Si + P YD Sj (cid:17) .
Our algorithm in Sec.
4 is designed based on above decomposition: because P YD Si solely depends on source domain D Si , we learn representations by minimizing d JS ( P Z | Y D Si , P Z | Y D Sj ) , ∀ i, j .
Combining
Thm. 1 and Corollary 1.1, to ensure high accuracy at unseen target domain D T , we learn the representation mapping g and model (cid:98) h such that P Z | Y D Si is invariant across source domains, and meanwhile (cid:98) f
= (cid:98) h ◦ g attains high accuracy at source domains.
Note that unlike our method, many existing works (Phung et al., 2021; Albuquerque et al., 2019;
Ganin et al., 2016) suggest that to ensure high accuracy in domain generalization, representation mapping g should be learned such that P ZD Si is same across domains, i.e., small d JS ( P ZD Si , P ZD T ) .
However, we show that the domain-invariant P ZD Si may adversely increase the error at target domain, as indicated in the Thm.
2 below.
Theorem 2 (Lower bound: accuracy) Suppose L ( (cid:98) f
( x ) , y ) =
(cid:80) ˆ y ∈Y (cid:98) f
( x ) ˆ y L (ˆ y, y ) where function
L : Y × Y → R + is lower bounded by c
when ˆ y ̸ = y , and is
when ˆ y = y .
If d JS ( P YD Si , P YD T ) ≥ d JS ( P ZD Si , P ZD T ) , the expected error of (cid:98) f
at source and target domains is lower bounded:
[Equation]
The above lower bound shows an inherent trade-off of approaches that minimize d JS ( P ZD Si , P ZD T ) when learning the representations.
Specifically, with the domain-invariant P ZD Si , the right hand side of Eq.
(2) may increase, resulting in an increased error at target domain ϵ Acc D T ( (cid:98) f ) .
|
(cid:1)
, where p
In fact, a tighter upper bound for the loss at target domain can be established using strong data processing inequality (Polyanskiy & Wu, 2017), as detailed in Appendix D
Similar to the loss, the unfairness at target domain can also be upper bounded, as presented in Thm. 3.
Theorem 3 (Upper bound: fairness) Consider a special case where the unfairness measure is defined as the distance between means of two distributions:
[Equation]
then the unfairness at any unseen target domain D T is upper bounded:
[Equation]
Similar to Thm. 1, the upper bound in Thm.
3 also has three terms and the second term is out of control during training because it depends on the unseen target domain and is defined in input space.
Therefore, to maintain fairness at target domain D T , we learn the representation mapping g and model (cid:98) h such that P Z | Y,A D Si is invariant across source domains, and meanwhile (cid:98) f
= (cid:98) h ◦ g attains high fairness at source domains.
The results above characterize the relations between accuracy/fairness at any target and source domains under any representation mapping g and model (cid:98) h .
Next, we identify conditions under which the accuracy/fairness attained at sources can be perfectly transferred to a target domain.
Theorem 4 (Sufficient condition for perfect transfer) Consider N source domains { D Si } Ni =1 and an unseen target domain D T .
Define set Λ =
{ D t : D t = (cid:80) Ni =
1 π i D Si , { π i } ∈ ∆ N − 1 } .
(Transfer of fairness) ∀ D T ∈ Λ , if g is the mapping under which P Z | Y,A D Si is the same across all source domains, then ϵ EO D Si ( (cid:98) h ) =
ϵ EO D T ( (cid:98) h ) =
ϵ EO D Si ( (cid:98) f ) =
ϵ EO D T ( (cid:98) f ) , ∀ i. 2.
(Transfer of accuracy) ∀ D T ∈ Λ , if P YD Si is the same and if g is the mapping under which P Z | Y D Si is the same across all source domains, then ϵ Acc D Si ( (cid:98) h ) =
ϵ Acc D T ( (cid:98) h ) =
ϵ Acc D Si ( (cid:98) f ) =
ϵ Acc D T ( (cid:98) f ) , ∀ i.
Thm. 4 indicates the possibility of attaining the perfect transfer of accuracy/fairness and examples of such representation mappings are provided.
Note that these results are consistent with Thm. 1 and
Thm. 3, which also suggest learning domain-invariant representations P Z | Y D Si and P Z | Y,A D Si .
P ROPOSED A LGORITHM
[Table]
(cid:17) and
[Equation]
and
[Equation]
The accuracy and fairness upper bounds in Sec.shed light on designing robust ML model that can preserve high accuracy and fairness on unseen target domains.
Specifically, the model consists of representation mapping g : X → Z and classifier (cid:98) h : Z → Y such that
(1) the prediction errors and unfairness of (cid:98)
f = (cid:98) h ◦ g on source domains are minimized; and (2) the discrepancy of learned conditional representations (i.e, P Z | Y D Si and P Z | Y,A D Si ) among source domains is minimized.
That is,
[Equation]
where L cls , L fair , and L inv are expected losses that penalize incorrect classification, unfairness, and discrepancy among source domains.
Hyper-parameters ω > 0 and γ > 0 control the accuracyfairness trade-off and accuracy-invariant representation trade-off, respectively.
The usages of these three losses are summarized in Table 1.
Adversarial learning framework (Goodfellow et al., 2014).
L inv in Eq.
(3) can be optimized directly with adversarial learning.
This is because the training objective of the discriminator in GAN is aligned with our goal of minimizing JS distance between P Z
| Y D Si (or P Z | Y,A D Si ) among source domains, as mentioned in Sec.
Specifically, define a set of discriminators K = { k y : y ∈ Y} ∪ { k y,a : y ∈
Y , a ∈ A} ; each discriminator k y (resp.
k y,a ) aims to distinguish whether a sample with label y (resp.
label y and sensitive attribute a ) comes from a particular domain (i.e., maximize L inv ).
The representation mapping g should be learned to increase the error of discriminators (i.e., minimize
L inv ).
Therefore, the model and discriminators can be trained simultaneously by playing a two-player minimax game (i.e., min g max K L inv ( g ) ).
Combine with the objective of minimizing prediction error and unfairness (i.e., min g, (cid:98)
h
L cls ( g, (cid:98) h ) + ω
L fair ( g, (cid:98) h ) )
, the overall learning objective is:
[Equation]
However, the above adversarial learning framework for learning domain-invariant representation may not work well when |Y × A| is large: as the label space and sensitive attribute space get larger, the number of discriminators to be learned increases and the training can be highly unstable.
A naive solution to tackling this issue is to use one discriminator ∀ y ∈ Y , a ∈ A .
However, this would result in the reduced mutual information between representations and label/sensitive attribute, which may hurt the accuracy.
We thus propose another approach to learn the domain-invariant representations
[Figure]
resentation.
To transfer accuracy and fairness to target domains, we need to find representation z
such that P z | y D Si and P z | y,a D Si are domain-invariant.
Proposed solution to learning invariant representations.
For any domain D , we have:
[Equation]
where P Z | x is domain-independent so we drop
D in subscript.
Given any two source domains D Si and D Sj , in general P X |
y D Si ̸ =
P X |
y D Sj and P X
| y,a D Si ̸
=
P X | y,a D Sj so that it is nontrivial to achieve domain-invariant representations P Z |
y D Si
= P Z |
y D Sj and P Z
| y,a D Si = P Z | y,a D Sj .
However, if there exist invertible functions m yi,j : X → X and m y,ai,j : X → X that can match the density functions of X from D Si to D
Sj such that P X |
y D Si = P m yi,j ( X ) | y D Sj and P X
| y,a D Si = P m y,ai,j ( X ) | y,a D Sj , and if we can find the representation Z such that P Z | x D Si = P Z | m yi,j ( x ) D Si and P Z |
x D Si = P Z | m y,ai,j ( x ) D Si , then ∀ y ∈ Y , a ∈ A , we have:
[Equation]
where x ′ = m yi,j ( x ) and x ′′ = m y,ai,j ( x ) .
This observation suggests that to minimize the discrepancy of representation distributions among source domains, we can first find the density mapping functions m yi,j and m y,ai,j , ∀ y, a, i, j , and then minimize the discrepancies between P Z | x , P Z | x ′ , and P Z | x ′′ , ∀ x .
This is formally shown in Thm.
5 below.
Theorem 5 If there exist invertible mappings m y i,j and m y,a i,j such that P X | y D S i =
P m yi,j ( X ) | y D S j and
P X | y,a D Si =
P m y,ai,j ( X ) |
y,a D Sj , ∀ y, a, i, j , and if the representation mapping are in the form of g := P Z | x =
N ( µ ( x ) , σ 2 I d ) , where µ ( x ) is the function of x and d is the dimension of the representation space
Z , then minimizing d JS (cid:16) P Z | y D Si , P Z | y D Sj (cid:17) and d JS (cid:16) P Z | y,a D Si , P Z | y,a D Sj (cid:17) can be reduced to minimizing (cid:13)(cid:13) µ ( x ) − µ (cid:0) m y i,j ( x ) (cid:1)(cid:13)(cid:13) 2 and (cid:13)(cid:13) µ ( x ) − µ (cid:0)
[Table]
Based on Thm.
5, we propose a two-stage learning approach FATDM , as stated below.
Remark 1 (Fairness and Accuracy Transfer by Density Matching ( FATDM ))
Given the existence of density matching functions m yi,j and m y,ai,j , and representation mapping g :=
N ( µ ( x ) , σ 2 I d ) , domain-invariant representations can be learned via a two-stage process: (i) finding these mapping functions m yi,j and m y,ai,j ; (ii) minimizing the mean squared errors between µ ( x ) and µ (cid:0) m yi,j ( x ) (cid:1) , and µ ( x ) and µ (cid:0) m y,ai,j ( x ) (cid:1) , ∀ i, j ∈ [ N ] , x ∈ X , y ∈ Y , a ∈ A .
Stage 1: learning mapping functions m yi,j and m y,ai,j across source domains.
Many approaches can be leveraged to estimate m yi,j and m y,ai,j from data.
In our study, we adopt StarGAN (Choi et al., 2018) and CycleGAN (Zhu et al., 2017) as examples; both frameworks are widely used in multi-domain image-to-image translation and can be leveraged.
In our algorithm, we independently train two translation models DensityMatch Y and DensityMatch Y,A using StarGAN or CycleGAN, with each used for learning { m yi,j } y ∈Y ,i,j ∈ [ N ] and { m y,ai,j } y ∈Y ,a ∈A ,i,j ∈ [ N ] , respectively.
[Figure]
Specifically, DensityMatch Y (or DensityMatch Y,A ) consists of a generator G : X × [ N ] × [ N ] → X and a discriminator D : X → [ N ] × { 0 , 1 } .
The generator takes in real image x and a pair of domain labels i, j as input and generates a fake image; the discriminator aims to predict the domain label of the image generated by the generator and distinguish whether it is fake or real.
G and D are learned simultaneously by solving the minimax game, and their loss functions are specified in Appendix B.
When the training is completed, we obtain two optimal generators from DensityMatch Y and
DensityMatch Y,A , denoted as G Y and G Y,A .
We shall use
G Y ( · , i, j ) (resp.
G Y,A ( · , i, j ) ) directly as the density mapping function { m yi,j ( · ) } y ∈Y (resp.
m y,ai,j ( · ) } y ∈Y ,a ∈A ).
Stage 2: learning domain-invariant representation.
Given
G Y and G Y,A learned in stage 1, we are ready to learn the invariant representation Z by finding g : X → Z such that g := N ( µ ( x ) , σ 2 I d ) and minimizes the following:
[Equation]
where d, d ′ , d ′′ are domain labels sampled from source domains, X is features sampled from domain d , X ′ =
G Y ( X, d, d ′ ) , X ′′ =
G Y,A ( X, d, d ′′ ) , L mse is mean squared error.
The pseudo-code of our proposed model ( FATDM ) is in Algorithm 1.
The detailed architecture of FATDM is in Appendix B.
[Table]
[Table]
/* Procedure for training G Y is similar but not presented */
Remark 2 (Summary of theoretical results and proposed algorithm)
Thm. 1 and Thm.
3 suggest a way to ensure high accuracy and fairness in target domain: by minimizing the source error ϵ Acc D si (i.e.,
L cls in Eq.
(3)), the source unfairness ϵ EO D si (i.e., L fair in Eq.
(3)), and the discrepancies between source domains d JS (cid:16) P Z | Y =
y D Si , P Z | Y =
y D Sj (cid:17) and d JS (cid:16) P Z
| Y
=
y,A = a D Si , P Z | Y
=
y,A = a D Sj (cid:17) (i.e., L inv in Eq.
The common way to optimize Eq.
(3) using adversarial learning (Eq.
(4)) is not stable when |Y × A| is large.
Thm.
5 states that instead of using adversarial learning, Eq.
(3) can be optimized via 2-stage learning: (i) find mappings m yi,j and m y,ai,j ( Density_Matching in Alg. 1) and (ii) minimize
Eq.
(3) with L inv defined in Eq.
(5) ( Invariant_Representation_Learning in Alg. 1).
E XPERIMENTS
We conduct experiments on MIMIC-CXR database (Johnson et al., 2019), which includes 377, chest X-ray images associated with 227,827 imaging studies about 14 diseases performed at the Beth
Israel Deaconess Medical Center.
Importantly, these images are linked with MIMIC-IV database (Johnson et al., 2021) which includes patients’ information such as age, and race; these can serve as sensitive attributes for measuring the unfairness.
Based on MIMIC-CXR and MIMIC-IV data, we construct two datasets on two diseases:
• Cardiomegaly disease: we first extract all images related to Cardiomegaly disease, and the corresponding labels (i
e., positive/negative) and sensitive attributes (i
e., male/female); then we partition the data into four domain-specific datasets based on age (i
e., [18 , 40) , [40 , 60) , [60 , 80) , [80 , 100) ).
We consider age as domain label because it captures the real scenario that there are distribution shifts across patients with different ages.
• Edema disease: we extract all images related to Edema disease, and corresponding labels (i
e., positive/negative) and sensitive attributes (i.e., age with ranges [18 , 40) , [40 , 60) , [60 , 80) , [80 , 100) ).
Unlike Cardiomegaly data, we construct the dataset for each domain by first sampling images from
Edema data followed by θ degree counter-clockwise rotation, where θ ∈ { 0 ◦ , 15 ◦ , 30 ◦ , 45 ◦ , 60 ◦ } .
We consider rotation degree as domain label to model the scenario where there is rotational misalignment among images collected from different devices.
Next, we focus on Cardiomegaly disease and the results for Edema disease are shown in Appendix C.
Baselines.
We compare our method (i.e., FATDM-StarGAN and FATDM-CycleGAN ) with existing methods for domain generalization, including empirical risk minimization, domain invariant representation learning, and distributionally robust optimization, as detailed below
• Empirical risk minimization ( ERM ): The baseline that considers all source domains as one domain.
• Domain invariant representation learning: Method that aims to achieve the invariant across source domains.
We experiment with G2DM (Albuquerque et al., 2019), DANN (Ganin et al., 2016), CDANN (Li et al., 2018c), CORAL (Sun & Saenko, 2016), IRM (Arjovsky et al., 2019).
These models focus on accuracy transfer by enforcing the invariance of distributions P ZD Si or P Z | Y D Si .
• Distributionally robust optimization: Method that learns a model at worst-case distribution to hope it can generalize well on test data.
We experiment with GroupDRO (Sagawa et al., 2019) that minimizes the worst-case training loss over a set of pre-defined groups through regularization.
• ATDM :
A variant of FATDM-StarGAN that solely focuses on accuracy transfer.
That is, we only enforce the invariance of P Z | Y D Si during learning which is similar to Nguyen et al.
The implementations of these models except G2DM are adapted from DomainBed framework (Gulrajani & Lopez-Paz, 2020).
For G2DM , we use the author-provided implementation.
For all models, we use ResNet18 (He et al., 2016) as the backbone module of representation mapping g : X → Z ; and fairness constraint L fair is enforced as a regularization term added to the original objective functions.
Experiment setup.
We follow leave-one-out domain setting in which 3 domains are used for training and the remaining domain serves as the unseen target domain and is used for evaluation.
Several metrics are considered to measure the unfairness and error of each model in target domain, including:
• Error: cross-entropy loss (CE), misclassification rate (MR), AUROC :
= 1 − AUROC, AUPR := 1 − AUPR, F 1 := 1 − F 1 , where AUROC, AUPR, F 1 are area under receiver operating characteristic curve, area under precision-recall curve, F 1 score, respectively.
Unfairness: we consider both equalized odds and equal opportunity fairness notion, and adopt mean distance (MD) and earth mover’s distance (EMD) as distance metric D ( ·||· ) .
Fairness and accuracy on target domains.
We first compare our method with baselines in terms of the optimal trade-off (Pareto frontier) between accuracy and fairness on target domains under different metric pairs.
Figure 4 shows the error-unfairness curves (as ω varies from 0 (no fairness constraint) to 10 (strong fairness constraint)), with AUROC and MR as error metric, and equalized odds (measured under distance metrics MD and EMD) as fairness notion; the results for other error metrics are similar and shown in Appendix C. Our observations are as follows: ( As expected, increasing
(2) Among all methods, the Pareto frontiers of FATDM-StarGAN and FATDM-CycleGAN are the bottom leftmost, implying that our method attains a better fairness-accuracy trade-off than baselines.
(3) Although fairness constraint is imposed during training for all methods, the fairness attained at source domains cannot be well-generalized to the target domain under other methods.
These results validate our theorems and show that enforcing the domain-invariant P Z | Y D Si and P Z | Y,A D Si when learning representations ensures the transfer of both accuracy and fairness.
It is worth-noting that under this dataset, the domain-invariant P Z | Y D Si (accuracy transfer) does not imply the domain-invariant P Z | Y,A D Si (fairness transfer).
This is because domain D Si (i.e., age) is correlated with label Y (i.e., has a disease) and
sensitive attribute A (i.e., gender), making the distribution P Y,AD Si different across domains.
Impact of density mapping model.
To investigate whether the performance gain of our method is due to the use of any specific density mapping model, we adopt StarGAN and CycleGAN architectures to learn density mapping functions in our method and compare their performances.
Figure 4 shows that FATDM-StarGAN and CycleGAN achieve similar fairness-accuracy trade-off at the target domains and both of them outperform the baselines.
This result shows that our method is not limited to any specific density mapping model and is broadly applicable to other architectures.
Impact of invariant representation constraints.
We also examine the impact of L inv on the performance of FATDM-StarGAN at target domains, where we vary the hyper-parameter γ ∈ [0 , 5 e 2 ] at different levels of fairness (i.e., fix ω = 1 , 5 , 10 ) and examine how the prediction performances (i.e.,
AUROC, AUPR, accuracy and F 1 ) could change.
Figure 5 shows that enforcing domain-invariant constraint L inv helps transfer the performance from source to target domain, and γ that attains the highest accuracy at target domain can be different for different levels of fairness.
The results also indicate the fairness-accuracy trade-off, i.e., for any γ , enforcing stronger fairness constraints (large ω ) could hurt prediction performances.
C ONCLUSION
In this paper, we theoretically and empirically demonstrate how to achieve fair and accurate predictions in unknown testing environments.
To the best of our knowledge, our work provides the first theoretical analysis to understand the efficiency of invariant representation learning in transferring both fairness and accuracy under domain generalization.
In particular, we first propose the upper bounds of prediction error and unfairness in terms of JS-distance, then design the two-stage learning method that minimizes these upper bounds by learning domain-invariant representations.
Experiments on the real-world clinical data demonstrate the effectiveness of our study.
R EPRODUCIBILITY S TATEMENT
The original chest X-ray images and the corresponding metadata can be downloaded from PhysioNet ( https://physionet.org/content/mimic-cxr-jpg/2.0.0/ ; https: //physionet.org/content/mimiciv/2.0/ ).
Codes for data processing and proposed algorithms are in supplementary materials.
Technical details of the proposed algorithms and experimental settings are in Appendix B. Additional experimental results are in Appendix C. Lemmas used in proofs of the theorems in the main paper are in Appendix D. Complete proofs of the theorems in the main paper and the corresponding lemmas are in Appendix E A CKNOWLEDGEMENTS
This work was funded in part by the National Science Foundation under award number IIS-2145625, by the National Institutes of Health under award number UL1TR and by The Ohio State
University President’s Research Excellence Accelerator Grant.
A R ELATED W ORKS
Domain generalization/Domain adaptation: In many real scenarios of machine learning, data in training phase is sampled from one or many source domains, while in the testing phase, data is sampled from an unseen target domain.
Many works have been proposed to design robust ML models that can achieve good performances in deployment environment depending on whether they can access to the target data (domain adaptation) or not (domain generalization).
However, most of these models focus only on transfering accuracy from source to target domains and can be categorized into five main approaches: (1) data manipulation (Volpi et al., 2018;
Qiao et al., 2020;
Zhou et al., 2020; Zhang et al., 2018; Shankar et al., 2018); (2) domain-invariant representation learning (Li et al., 2018b;a; Ganin & Lempitsky, 2015; Ganin et al., 2016; Phung et al., 2021; Nguyen et al., 2021); (3) distributional robustness (Krueger et al., 2021;
Liu et al., 2021; Koh et al., 2021; Wang et al., 2021;
Sagawa et al., 2019; Hu et al., 2018), (4) gradient operation (Huang et al., 2020; Shi et al., 2021;
Rame et al., 2021; Tian et al., 2022), and (5) self-supervised learning (Carlucci et al., 2019;
Kim et al., 2021; Jeon et al., 2021; Li et al., 2021).
Fairness in Machine Learning:
Many fairness notions have been proposed to measure the unfairness in ML model, and they can be roughly classified into two classes:
Individual fairness considers the equity at the individual-level and it requires that similar individuals should be treated similarly (Biega et al., 2018;
Bechavod et al., 2020; Gupta & Kamble, 2021; Dwork et al., 2012).
Group fairness attains a certain balance in the group-level, where the entire population is first partitioned into multiple groups and certain statistical measures are equalized across different groups (Hardt et al., 2016; Zhang et al., 2019; 2020).
Various approaches have also been developed to satisfy these fairness notions, they roughly fall into three categories: (1) Pre-processing : modifying training dataset to remove bias before learning an ML model (Kamiran & Calders, 2012; Zemel et al., 2013).
(2) In-processing : attain fairness during the training process by imposing certain fairness constraint or modifying loss function.
(Zafar et al., 2019; Agarwal et al., 2018) (3) Post-processing : altering the output of an existing algorithm to satisfy a fairness constraint after training (Hardt et al., 2016).
However, most of these methods assume the data distributions at training and testing are the same.
In contrast, we study fairness problem under domain generalization in this paper.
Fairness under Domain Adaptation: There are some studies proposed to achieve good fairness when the testing environment changes but all of them focused on the domain adaptation setting.
The most common adaptation setup is learning under the assumption of covariate shift.
For example,
Singh et al.
(2021) leveraged a feature selection method in a causal graph describing data to mitigate fairness violation under covariate shift of distribution in testing data.
Coston et al.
(2019) proposed the weighting methods that can give fair prediction under covariate shift between source and target distribution when access to the sensitive attributes is prohibited.
Rezaei et al.
(2021) sought fair decisions by optimizing a worst-case testing performance.
Besides convariate shift, there are some works proposed to handle other types of distribution shift including demographic shift and prior probability shift.
Instead of learning fair model directly, Oneto et al.
(2019) and Madras et al.
(2018) find fair representation that can generalize to the new tasks.
Aside from empirical studies, Schumann et al.
(2019) and Yoon et al.
(2020) developed theoretical frameworks to examine fairness transfer in domain adaptation setting and then offered modeling approaches to achieve good fairness in the target domain.
Comparison with existing bounds in the literature: We compare our bounds with most commons bound in the fields of domain adaptation and domain generalization as follows.
Accuracy bounds in domain adaptation.
• Bounds in Ben-David et al.
(2010): ϵ Acc D T (cid:16) (cid:98) f (cid:17)
[Equation]
This bound is for binary classification problem under domain adaptation.
The classification error in target domain is bounded by the error in source domain, the total variation distance of feature distribution between source and target domain, and the misalignment of the labeling function between source and target domain.
The limitation of this bound is that (1) it’s only applicable to settings with zero-one loss function and deterministic labeling function; (2) estimating the total variation distance is hard in practice and it doesn’t relate the feature and representation spaces.
This paper also provides another accuracy bound based on H ∆ H divergence:.
[Equation]
where D H ∆ H
[Equation]
P
[Equation]
is the
H ∆ H divergence.
However, it has the same limitations as total variation distance mentioned above.
Accuracy bounds in domain generalization.
[Table] [Table]
[Equation]
where D H (cid:0) P XD S
[Equation]
P XD
[Equation]
(cid:1)
=
sup
[Equation]
is the H divergence,
[Equation]
XD
[Equation]
is the mixture of source domains that is closest to target domain with respect to H divergence.
In this bound, the classification error in target domain is bounded by the convex combination of errors in source domains, the H divergence between source domains, the H divergence between target domain and its nearest mixture of source domains, and the misalignment of the labeling function between mixture source domains and target domain.
Because this bound is constructed based on H divergence, it also has the limitations for the bounds in domain adaptation (Ben-David et al., 2010) as we mentioned.
This bound can be transformed to the representation space Z by replacing X by Z in its formula.
Then, this bound suggests enforcing invariant constraint of marginal distribution of representation Z across source domains, which has inherent trade-off as shown in Thm. 2.
Because the target domain is unknown during training, the mixing weights { π i } Ni =1 are not useful for algorithmic design.
• Bounds in Phung et al.
(2021): ϵ
[Equation]
) y (cid:12)(cid:12)(cid:12)(cid:105) |Y| y =1 (cid:13)(cid:13)(cid:13)(cid:13)(cid:21) +
N (cid:88) i =
N (cid:88) j =
C (cid:112) 2 π j
N d 1 / 2 (cid:16) P ZD T , P ZD Si (cid:17)
+
N (cid:88) i = N (cid:88) j =
C (cid:112) 2 π j
N d 1 / 2 (cid:16) P ZD Si , P ZD Sj (cid:17) where d 1 / 2 (cid:16) P X D S
i , P X D S j (cid:17) =
(cid:114)
D 1 / 2 (cid:16) P X D S i
∥
P X D S j (cid:17) is Hellinger distance defined based on
Hellinger divergence D 1 / 2 (cid:16) P XD Si ∥ P XD Sj (cid:17) =
2 (cid:82) X (cid:18)(cid:113)
P XD Si − (cid:113)
P XD Sj (cid:19)dX .
This bound relates the feature and representation spaces that the classification error of target domain defined in feature space is bounded by classification errors of source domains defined in feature space, the misalignment of labeling function between target and source domains, and the Hellinger distances between source and target domains and between source domains of marginal distribution of representation Z .
While this bound is not limited to zero-one loss and the labeling function can be stochastic, it suggests the alignment of marginal distribution of representation Z across source domains for generalization.
Moreover, estimating Hellinger distance can be hard in practice.
The mismatch between existing bounds and adversarial learning approach for domain generalization.
All existing bounds mentioned above suggest minimizing the distances between representation distributions across source domains with respect to some discrepancy measures such as H divergence, total variation distance, and Hellinger distance.
Based on these bounds, adversarial learning-based models are often proposed to minimize these distances.
However, there is a misalignment between the objectives of adversarial learning and the bounds which results in the gap between theoretical findings and practical algorithms.
In particular, it has been shown that the objective of the minimax game between the representation mapping and the discriminator is equivalent to minimizing the JS divergence between representation distributions across source domains (Goodfellow et al., 2014).
However, minimizing JS divergence does not guarantee the minimization of common distances used in the existing bounds.
The details are as follows.
• H divergence: We show that JS divergence is not the upper bound of H divergence.
Consider an example with two distributions P ( X ) and Q ( X ) where (cid:26) P ( X ) =
w.p 1 /P ( X ) = 1 w.p 2 / 3 and (cid:26) Q ( X )
=
w.p 1 /Q ( X ) = 1 w.p 2 / 3 .
By definition, D H ( P ∥ Q ) ∼ 0 .
33 > D JS ( P ∥ Q ) ∼ 0 .
• Total variation distance: We have D JS ( P ∥ Q ) ≤ D TV ( P ∥ Q ) ∀ P, Q where D JS and D TV are JS divergence and total variation distance, respectively.
Then, minimizing JS divergence does not guarantee the minimization of total variation distance.
• Hellinger distance: We have D JS ( P ∥ Q ) ≤ √ 2 d 1 /
( P, Q ) ∀ P, Q where d 1 / 2 is Hellinger distance and total variation distance, respectively.
Then, minimizing JS divergence does not guarantee the minimization of Hellinger distance.
Different from the existing bounds, our bounds are based on JS divergence/distance.
Then they align with the adversarial learning approach for domain generalization in general, and with our proposed method FATDM in particular.
Advantages of our proposed bounds in domain generalization.
In summary, our proposed bounds has several advantages in terms of the following:
• Most existing bounds (Ben-David et al., 2010; Albuquerque et al., 2019) do not relates feature and representation spaces so it is not clear how performance in input space is affected by the representations.
In contrast, our bounds connect the representation and input spaces; this further guides us to find representations that lead to good performances in input space.
• Most prior studies adopt H divergence to measure the dissimilarity between domains, which is limited to deterministic labeling functions and zero-one loss (Ben-David et al., 2010; Albuquerque et al., 2019).
In contrast, our bound is more general and is applicable to settings where domains are specified by stochastic labeling functions and general loss functions.
• Distant metrics (i.e., total variation distance, H divergence, Hellinger divergence, etc.) used in existing bounds (Ben-David et al., 2010; Albuquerque et al., 2019; Phung et al., 2021) are hard to compute in practice.
In contrast, our bounds use JS divergence which is aligned with training objective for discriminator in adversarial learning Goodfellow et al.
• Existing bounds for domain generalization only imply the alignment of marginal distribution of feature across source domains (Albuquerque et al., 2019; Phung et al., 2021).
As shown in Thm. 2, methods that learn invariance of marginal distribution have an inherent trade-off and may increase the lower bound of expected loss.
In contrast, our bounds suggest the alignment of label-conditional distribution of feature across source domains which has been verified to be more effective in empirical studies (Li et al., 2018b;c; Zhao et al., 2020; Nguyen et al., 2021).
• Regarding the fairness, our work is the first that bounds the unfairness in domain generalization.
In particular, our bounds suggest enforcing the invariant constraint of feature distribution given label and sensitive attribute across source domains to transfer fairness to the unseen target domain.
B D ETAILS OF A LGORITHM FATDM
FATDM consists of density mapping functions
m y i,j and m y,a i,j , ∀ y ∈ Y , a ∈ A , i, j ∈ [ N ] (learned by two DensityMatch models), feature mapping function g (ResNet18 model) and the classifier (cid:98) h .
In our study, we experiment with two different DensityMatch architectures: StarGAN (i.e., in FATDM-StarGAN ) and CycleGAN (in FATDM-CycleGAN ).
We show the details of FATDM-StarGAN below.
For FATDM-CycleGAN , the only difference is we used CycleGAN as DensityMatch instead of StarGAN.
The details of CycleGAN were presented in the original paper (Zhu et al., 2017).
For FATDM-StarGAN , each DensityMatch Y (or DensityMatch Y,A ) consists of a generator
G : X × [ N ] × [ N ] → X and a discriminator D : X → [ N ] × { 0 , 1 } .
The generator takes in real image x and a pair of domain labels i, j as input and generates a fake image; the discriminator aims to predict the domain label of the image generated by the generator and distinguish whether it is fake or real.
G and D are learned simultaneously by solving the following optimizations:
[Equation]
where L StarGAN adv is the adversarial loss, L StarGAN cls ( fake ) , L StarGAN cls ( real ) are domain classification loss with respect to fake and real images respectively, L StarGAN rec is the reconstruction loss.
The specific formulations of these loss functions are in Choi et al.
λ cls and λ rec are hyper-parameters that control the relative importance of domain classification and reconstruction losses, respectively, compared to the adversarial loss.
In our experiments, input images are resized to (256 , 256) and normalized into the range [ − 1 , 1] .
The dimension of representation space Z is set to 512 .
ω (hyper-parameter that controls accuracy-fairness trade-off) varies from 0 to 10 with step sizes 0 .
0002 for ω ∈ [0 , 0 .
002 for ω ∈ [0 .
1] and 0 .
2 for ω ∈ [0 . 2 , 10] , and γ (hyper-parameter that controls accuracy-invariance trade-off) is set to 0 .
1 (after hyper-parameter tuning).
Models ( FATDM and baselines) are implemented by PyTorch library version 1.11 and is trained on multiple computer nodes (each model instance is trained on a single node which has 4 CPUs, 8GB of memory, and a single GPU (P100 or V100)).
One domain’s data is used for testing and the other domains’ data is used for training ( 10% of training data is used for validation).
Each model is trained with 10 epoches and the results are from the epoch with best performance on the validation set.
Figure 6 visualizes the two-stage training process of
FATDM-StarGAN .
The detailed architectures of FATDM-StarGAN are shown in Tables 2-5.
We have also provided all code for these models in supplemental material.
[Table] [Table]
[Table] [Table]
C A DDITIONAL E XPERIMENTS
Experimental results with all unfairness and error metrics.
In this section, we provide more experimental results about fairness and accuracy under domain generalization.
In particular, we investigate fairness-accuracy trade-off on the two clinical image datasets including Cardiomegaly and
Edema diseases with respect to different fairness criteria (i.e., Equalized Odds, Equal Opportunity), and unfairness (i.e., MD and EMD) and error (i.e., CE, MR, AUROC, AUPR, F 1 ) measures.
Figure(Cardiomegaly disease - Equalized Odds), Figure 8 (Cardiomegaly disease - Equal Opportunity),
Figure 9 (Edema disease - Equalized Odds), and Figure 10 (Edema disease - Equal Opportunity) show the unfairness-error curves of our models as well as baselines for these two datasets.
As we can see, our model outperforms other baselines in terms of fairness-accuracy trade-off.
The curve of our model is the bottom-leftmost compared to other baselines in all measures showing the clear benefit of (1) enforcing conditional invariant constraints for accuracy and fairness transfer and (2) using the two-stage training process to stabilize training compared to adversarial learning approach.
We also quantify our observations by calculating the areas under these unfairness-error curves, in which the smaller area indicates the better accuracy-fairness trade-off.
As shown in Tables 6 and 7, our model has the smallest areas under the curve and achieves significantly better fairness-accuracy trade-off for both equalized odd and equal opportunity compared to other methods.
Impact of the number of source domains.
Our work focuses on transferring fairness and accuracy under domain generalization when the target domain data are inaccessible during training.
Instead, it relies on a set of source domains to generalize to an unseen, novel target domain.
We investigate the relationship between the fairness-accuracy trade-off on the target domain and the number of source domains during training.
In particular, we evaluate the performances of FATDM and ERM on
Edema dataset with different numbers of source domains.
Similar to the previous experiment, we first construct the dataset for each domain by rotating images with θ degree, where θ ∈ { 0 ◦ , 15 ◦ , 30 ◦ } when the number of domain is 3 , θ ∈ { 0 ◦ , 15 ◦ , 30 ◦ , 45 ◦ } when the number of domain is 4 , and θ ∈ { 0 ◦ , 15 ◦ , 30 ◦ , 45 ◦ , 60 ◦ } when the number of domain is 5 .
The number of images per domain is adapted to ensure the training set size is fixed for the three cases.
We follow the leave-one-out domain setting in which one domain serves as the unseen target domain for evaluation while the rest domains are for training; the average results across target domains are reported.
Figure 11 shows error-unfairness curves of FATDM and ERM when training with 2 , 3 , and 4 source domains.
We observe that training with more source domains does not always help the model achieve better fairness-accuracy trade-off on unseen target domains.
In particular, the performances of both
FATDM and ERM are the best when training with 2 source domains and the worst when training with 3 source domains.
We conjecture the reason that adding more source domains may help reduce the discrepancy between source and target domains (term (ii) in Thm. 1 and Thm. 3), but it may make it more difficult to minimize the source error and unfairness (term (i) in Thm. 1 and Thm. 3) and to learn invariant representation across the source domains (term (iii) in Thm. 1 and Thm. 3).
Thus, our suggestion in practice is to conduct an ablation study to find the optimal number of source domains.
Simultaneous and sequential training comparison.
In all experiments we conducted so far, the fairness constraint L fair is optimized simultaneously with the prediction error L acc and the domain-invariant constraint L inv for all methods.
To investigate whether FATDM still attains a better accuracy-fairness trade-off when the processes of invariant representation learning and fair model training are decoupled, we conduct another set of experiments where models ( FATDM (i.e.,
FATDM-StarGAN ) and baselines G2DM , DANN , CDANN ) are learned in a sequential matter: for each model, we first learn the representation mapping g by optimizing L inv and L acc ; using the representations generated by the fixed g , we then learn the fair classifier by optimizing L acc and L fair .
The models trained based on the above procedure are named FATDM-seq , G2DM-seq , DANN-seq , and CDANN-seq ; and their corresponding error-unfairness curves are shown in Figure 12.
The results show that FATDM-seq still attains the best accuracy-fairness trade-off at target domain compared to G2DM-seq , DANN-seq , CDANN-seq .
Our method is effective no matter whether
L fair and L inv are optimized simultaneously or sequentially.
The reason that our method consistently outperforms the baselines for both settings is that the invariant-representation learning in baseline methods only guarantees the transfer of accuracy but not fairness.
Even though a fairness regularizer is imposed to ensure the model is fair at source domains (no matter whether invariant representations and fair classifier are trained simultaneously or sequentially), this fairness cannot be preserved at the target domain due to the potential distributional shifts.
The key to ensuring the transfer of fairness is to learn representations such that P ( Z | Y, A ) is domain-invariant; this must be done during the representation learning process.
From Thm 3, we can see that unfairness at target domain ϵ EO D T can still blow up if P Z | Y,A is different across domains, regardless of how fair the model is at source domains (i.e., small ϵ EO D Si ).
[Table] [Table]
[Table] [Table]
[Table] [Table]
E qu a l
O ppo r t un it y
AUROC
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
E q u a l
O ppo r t un it y AUROC - MD 0. AUPRC
[Table]
[Table]
[Table]
[Table]
MD 0.
MR
[Table]
MD 0.
[Table]
MD 0.
[Table]
AUROC
[Table]
EMD 0.
AUPRC
[Table]
EMD
[Table]
[Table] [Table] EMD 0. MR
[Table]
MD 0.
[Table]
[Table]
[Table]
[Table]
MD 0.
[Figure]
D A DDITIONAL R ESULTS & L EMMAS D. T IGHTER UPPER BOUND FOR ACCURACY
Corollary 5.1 We can replace term (ii) in Thm.
1 with the following term to attain a tighter upper bound for accuracy:
[Equation]
where η TV = sup
[Equation]
D
[Equation]
1 is called Dobrushin’s coefficient (Polyanskiy & Wu, 2017).
This result suggests that we can further optimize term (ii) in Thm. 1 by minimizing η TV .
It has been shown in Shui et al.
(2022) that η TV can be controlled by Lipschitz constant of the feature mapping
g : X → Z when g follows Gaussian distribution.
The Lipschitz constant of g , in turn, can be upper bounded by the Frobenius norm of Jacobian matrix with respect to g (Miyato et al., 2018).
However, in practice, we found that computing Jacobian matrix of g is computationally expensive when dimension of representation Z is large, and optimizing it together with invariant constraints does not improve the performances of models in our experiments.
D. L EMMAS FOR PROVING T HEOREM
Lemma 6 Let X be the random variable in domains D i and D j , and E be an event that P XD j ≥ P
XD i , then we have: (cid:90)
[Equation]
where E is the complement of event E .
Lemma 7 Let X be the random variable in domains D i and D j , let f : X → R + be a non-negative function bounded by C , then we have:
[Equation]
where D KL ( · ∥ · ) is the KL-divergence between two distributions.
Lemma 8 Suppose loss function L is upper bounded by C and consider a classifier (cid:98) f : X → Y .
the expected classification error of (cid:98) f in domain D j can be upper bounded by its error in domain D i :
[Equation]
where X, Y are random variables denoting feature and label in domains D i and D j .
Lemma 9 Consider two distributions P XD i and P
XD j over X .
Let P ZD i and P ZD j be the induced distributions over Z by mapping function g : X → Z , then we have:
[Equation]
Lemma 10 (Phung et al., 2021) Consider domain D with joint distribution P X,YD and labeling function
f D : X → Y ∆ from feature space to label space.
Given mapping function g : X → Z from feature to representation space, we define labeling function h D : Z → Y ∆ from representation space to label space as h D ( Z ) Y =
f D
[Equation]
X ) Y ◦ g −
[Equation]
Z
[Equation]
=
[Equation]
[Equation] Z ) P XD dX .
Similarly, let (cid:98) f be the hypothesis from feature space, then the corresponding hypothesis (cid:98) h from representation space under the mapping function g is computed as (cid:98) h ( Z ) Y =
(cid:82) g − 1( Z ) (cid:98) f
( X ) Y P XD dX (cid:82) g − 1( Z ) P XD dX .
Let ϵ Acc D ( (cid:98) f ) =
E D (cid:2) L ( (cid:98) f
( X ) , Y ) (cid:3) and ϵ Acc D ( (cid:98) h ) =
E D (cid:2) L ( (cid:98)
h
( Z ) , Y ) (cid:3) be expected errors defined with respect to feature space and representation space, respectively.
We have:
[Equation]
D. L EMMAS FOR PROVING C OROLLARY 1.
Lemma 11 Consider two random variables X, Y .
Let P X,YD i , P X,YD j be two joint distributions defined in domains D i and D j , respectively.
Then, JS-divergence D JS (cid:16) P X,YD i ∥ P X,YD j (cid:17) and KL-divergence
D KL (cid:16) P X,Y D i ∥
P X,Y D j (cid:17) can be decomposed as follows:
[Equation]
D. L EMMAS FOR PROVING T HEOREM
Lemma 12 Under Assumption in Theorem 2, the following holds for any domain D :
[Equation]
where (cid:98) Y is the prediction made by randomized predictor (cid:98) f .
D. L EMMAS FOR PROVING T HEOREM
Definition
Given domain D i with binary random variable
A denoting the sensitive attribute, the unfairness measures that evaluate the violation of equalized odd ( EO ) and equal opportunity ( EP ) criteria between sensitive groups of this domain are defined as follows.
[Equation]
where R y,aD i (cid:16) (cid:98) f (cid:17) =
E D i (cid:104) (cid:98) f
( X
[Equation]
| Y =
y
[Equation]
a (cid:105)
[Equation]
Lemma
Given two domains D i and D j , under Definition 13, R y,aD j (cid:16) (cid:98) f (cid:17) can be bounded by
R y,aD i (cid:16) (cid:98) f (cid:17) as follows.
[Equation]
Lemma
Given two domains D i and D j , under Definition 13, the unfairness in domain D j can be upper bounded by the unfairness measure in domain D i as follows.
[Equation]
Lemma 16 Consider domain D with distribution P X,YD and labeling function
f D : X → Y ∆ .
Given mapping function g : X → Z from feature to representation space, we define labeling function h D : Z → Y ∆ from representation space to label space as h D ( Z ) Y =
f D ( X ) Y ◦ g − 1 ( Z ) =
(cid:82)
[Equation]
X
[Equation]
XD dX (cid:82) g
[Equation]
[Equation]
XD dX .
Similarly, let (cid:98) f be the hypothesis from feature space, then the corresponding hypothesis (cid:98) h from representation space under the mapping function g is computed as (cid:98) h ( Z ) Y =
(cid:82)
[Equation]
g −
[Equation]
Under Definition 13, we have:
[Equation]
D.
L EMMAS FOR PROVING T HEOREM
Lemma 17 Consider two domains D i and D j , if there exist invertible mappings m yi,j and m y,ai,j such that P X | y D i
= P m yi,j ( X ) |
y D j and P X
| y,a D i =
P m y,ai,j ( X ) | y,a D j , ∀ y ∈ Y , a ∈ A , then D JS (cid:16) P Z | y D i
∥ P Z | y D j (cid:17) and D JS (cid:16) P Z | y,a D i ∥
P Z | y,a D j (cid:17) can be upper bounded by (cid:82) x
P x | y D
i D JS (cid:16) P Z | x ∥ P Z | m yi,j ( x ) (cid:17) dx and (cid:82) x
P x | y,a D
i
D JS (cid:16) P Z | x ∥ P Z | m y,ai,j ( x ) (cid:17) dx , respectively.
E P ROOFS E. P ROOFS OF T HEOREMS
Proof of Theorem 1.
First, we get the upper bound based on the representation space Z .
Then, we relate it with the feature space X .
Let D S ∗ ∈
{ D S i } N i =1 be the source domain that’s nearest to the target domain D T .
According to Lemma 8, we have upper bound of the expected classification error for the target domain based on each of the source domain as follows.
[Equation]
Taking average of upper bounds based on all source domains
[Equation]
we have:
[Equation]
Here we have (1) ≤ by using triangle inequality for JS-distance: d JS ( P, R ) ≤ d JS ( P, Q ) + d JS ( Q, R ) with P, Q, and R = P D T , P D S ∗ and P D Si , respectively.
We have (2) ≤ because D S ∗ ∈ { D Si } Ni =1 then d JS (cid:16) P Z,YD S ∗ , P Z,YD Si (cid:17) ≤ max i,j ∈ [ N ] d JS (cid:16) P Z,YD Si , P Z,YD Sj (cid:17) .
Similarly, we can obtain the upper bound based on the feature space X as follows.
[Equation]
However, the bounds in Eq.
(7) and Eq.
(8) are based on either feature space or representation space, which is not readily to use for practical algorithmic design because the actual objective is to minimize ϵ Acc D T (cid:16) (cid:98) f (cid:17) in feature space by controlling Z in representation space.
According to Lemmas 9 and 10, we can derive the bound that relates feature and representation spaces as follows.
ϵ
[Equation]
Proof of Corollary 1.1.
[Equation]
Here we have (1) ≤ by using Lemma 11 to decompose the JS-divergence of the joint distributions and (2) ≤ by using inequality √ a + b ≤ √ a + √ b .
This new upper bound, combined with Thm. 1 suggests learning representation Z such that P Z | Y D Si is invariant across source domains, or in another word, Z ⊥ D | Y .
This result is consistent with
Thm. 4: when the target domain D T is the mixture of source domains { D Si } Ni =1 , and when P YD Si and
P Z | Y D Si are invariant across source domains, we have d JS (cid:16) P Z,YD T , P Z,YD Si [Equation] =
[Equation]
(cid:16) P Z,YD Si , P Z,YD Sj (cid:17)
= 0 , implying
[Equation]
Proof of Corollary 5.1 (tighter upper bound for accuracy).
The bound in Eq.
(9) is constructed using Lemma 9.
Indeed, we can make this bound tighter using the strong data processing inequality for JS-divergence (Polyanskiy & Wu, 2017), as stated below.
[Equation]
where Z is random variable induced from random variable X , and P XD i and P
XD i are two distribution over X , and η JS = sup
[Equation]
is the total variation distance.
η TV is called the Dobrushin’s coefficient (Polyanskiy & Wu, 2017).
Apply Lemma 11 and this inequality to the second term in the right hand side of Eq.
(7) (similar to the proof of Corollary 1.1), we have:
[Equation]
Proof of Theorem 2.
Consider a source domain D Si and target domain D T .
Because JS-distance d JS ( · , · ) is a distance metric, we have triangle inequality:
[Equation]
Since X
[Equation]
Z
[Equation]
(cid:98) Y , we have d JS ( P (cid:98) YD Si [Equation] P (cid:98) YD [Equation]
[Equation]
d [Equation] ( P ZD Si , P ZD T ) .
Using Lemma 12, the following holds when d JS
[Equation]
P YD
[Equation]
The last inequality is by AM-GM inequality.
Therefore, when d JS ( P YD Si , P YD T ) ≥ d JS ( P ZD Si , P ZD T ) , we have
[Equation]
The above holds for any source domain D Si .
Average over all N source domains, we have
[Equation]
Proof of Theorem 3.
The proof is based on Lemmas 15 and 16 and similar to the proof of Thm. 1.
Let D S ∗ ∈
{ D Si } Ni =1 be the source domain nearest to the target domain D T .
According to Lemma 15, we have upper bound of the unfairness measured with respect to the representation space for the target domain based on each of the source domain.
For equal opportunity (EP) , we have:
[Equation]
Taking average of upper bounds based on all source domains
[Equation]
we have:
[Equation]
According to Lemmas 9 and 16.
we can relate this bound to the feature space as follows
[Equation]
Similarly
[Equation]
we got the upper bound for unfairness measure with respect to equalized odds as follows.
[Equation]
Proof of Theorem 4.
Consider two source domains, D Si and D Sj , if P YD Si = P YD Sj , we can learn the mapping function g =
P θ ( Z | X ) such that P Z
| Y D Si
=
P
Z |
Y D Sj .
Note that this mapping function always exists.
In particular, the trivial solution for Z that satisfies P Z |
Y D
Si
=
P
Z |
Y D Sj is making Z ⊥ Y, D (e
[Equation]
g.,
P θ ( Z | X ) =
N ( 0 , I ) ).
Then we have:
[Equation]
For unseen target domain D T in Λ , we have:
[Equation]
By Lemma 10, we have
[Equation]
For fairness, we only give the proof for equalized odds ( EO ), we can easily get the similar derivation for equal opportunity.
For any Z that satisfies P Z | Y =
y,A = a D Si
[Equation]
} , we have:
[Equation]
For unseen target domain D T in Λ , we have:
[Equation]
h
[Equation]
=
[Equation]
y
[Equation]
[Equation]
D
[Equation]
i
[Equation]
[Equation]
Similar to the proof of accuracy, Z that satisfies P
[Equation]
[ N ] always exists.
The trivial solution for is Z that satisfies Z ⊥ Y, A, D .
By Lemma 16, we have
[Equation]
For equal opportunity ( EP ), Z only need to satisfy the condition for positive label, i.e., P Z | Y =1 ,A = a D Si
[Equation]
P Z | Y =1 ,A = a D Sj ∀ a ∈ { 0 , 1 } , i, j ∈ [ N ] .
Proof of Theorem 5.
According to Lemma 17, we have:
[Equation]
Then, minimizing
[Equation]
can be achieved by minimizing
[Equation]
We can upper bound D JS (cid:16) P Z | x ∥ P Z | m yi,j ( x ) (cid:17) as follows
[Equation]
where D TV and d 1 / 2 are total variation distance and Hellinger distance between two distributions, respectively.
We have (1) =
because of our choice for representation mapping g ( x ) :
= P Z | x =
N (cid:0) µ ( x ); σ 2 I d (cid:1) .
According to Devroye et al.
[Equation] the Hellinger distance between two multivariate normal distributions over R d has a closed form as follows
[Equation]
where µ 1 , µ 2 , Σ 1 , Σ 2 are mean vectors and covariance matrices of the two normal distributions.
In
Eq.
[Equation]
let µ 1 = µ ( x ) , µ 2 = µ (cid:0) m y
i,j ( x ) (cid:1) , Σ 1 = Σ 2 = σ 2 I d [Equation] then we have:
[Equation]
From Eq.
(15), we can see that Helinger distance between two representation distributions P Z | x and P Z | m yi,j ( x ) is the function of their means µ ( x ) and µ (cid:0) m yi,j ( x ) (cid:1) .
Combining this with Eq.
(12) and Eq.
(13), we conclude that minimizing d JS (cid:16) P Z | y D Si , P Z | y D Sj (cid:17) can be reduced to minimizing (cid:13)(cid:13) µ ( x ) − µ (cid:0) m yi,j ( x ) (cid:1)(cid:13)(cid:13) 2 which can be implemented as the mean square error between µ ( x ) and µ (cid:0) m yi,j ( x ) (cid:1) in practice.
Proof for d JS (cid:16) P Z | y,a D Si , P Z | y,a D Sj (cid:17) is derived in the similar way.
E. P ROOFS OF L EMMAS
Proof of Lemma 6.
We have:
[Equation]
Proof of Lemma 7.
We have:
[Equation]
where E is the event that P XD j ≥ P
XD i and E is the complement of E .
We have (1) ≤ because (cid:82) E f ( X ) (cid:16) P
XD j − P
XD i (cid:17) dX ≤ 0 ; (2) ≤ because f ( X ) is non-negative function and is bounded by
C ; (3) ≤ by using Lemma 6; (4) ≤ by using Pinsker’s inequality between total variation norm and KLdivergence.
Proof of Lemma 8.
Applying Lemma 7 and replacing X by ( X, Y ) , f by loss function L , D i by
D i,j , we have:
[Equation]
Applying Lemma 7 again and replacing X by ( X, Y ) , f by loss function L , D j by D i,j , we have:
[Equation]
Adding Eq.
(16) to Eq.
(17), we have:
[Equation]
Here we have (1) ≤ by using Cauchy–Schwarz inequality.
Proof of Lemma 9.
Note that the JS-divergence D JS (cid:16) P
XD i ∥ P XD j (cid:17) can be understood as the mutual information between a random variable X associated with the mixture distribution P
XD i,j = 12 (cid:16) P
XD i + P XD j (cid:17) and the equiprobable binary random variable T used to switch between P XD i and
P XD j to create the mixture distribution P XD i,j .
In particular, we have:
[Equation]
where H ( X ) is the entropy of X , H
( X | T ) is the entropy of X conditioned on T , and I ( X ; T ) is the mutual information between X and T .
Similarly, we also have D JS (( P ZD i ∥ P ZD j ))
=
I ( Z ; T ) .
Because Z is induced from X by the mapping function
h then we have Z ⊥ T | X
and the Markov chain T → X → Z .
According to data processing inequality for mutual information (Polyanskiy &
Wu, 2014), we have I ( X ; T ) ≥ I ( Z ; T ) which implies D JS (( P XD i ∥
P XD j )) ≥ D JS (( P ZD i ∥
P ZD j )) .
Taking square root on both sides, we have d JS ( P X D i , P X D j ) ≥ d JS ( P Z D i , P Z D j ) .
Proof of Lemma 10.
We have:
[Equation]
Proof of Lemma 11.
We show the decomposition for KL-divergence first and then use the result to derive the decomposition for JS-divergence.
We have:
[Equation]
[Table]
[Equation]
Proof of Lemma
[Equation]
Here we have (1) ≥ is because of the assumption that L ( (cid:98) y, y ) is lower bounded by c when (cid:98)
y ̸
= y ; (2) = is because (cid:98)
f
( X ) T 1 = || (cid:98) f
( X ) || 1 = 1 ; (3) ≥ is because || (cid:98) f
( X ) ||
2 ≤ || (cid:98) f
( X ) || 1 = 1 ; (4) ≥ is because || (cid:98) f
( X ) ||
2 ≥ 1 √ |Y| || (cid:98) f
( X ) || 1 ; (5) ≥ is by using Jensen’s inequality; (6) ≥ is by using JS-divergence lower bound of total variation distance.
Proof of Lemma 14.
Similar to the proof in Lemma 8, we apply Lemma 7 for R y,aD i and R y,aD j and note that (cid:98) f
( X ) y is bounded by 1 .
Then ∀ y, a ∈ { 0 , 1 } , we have:
[Equation]
Adding Eq.
(18) to Eq.
(19), we have:
[Equation]
Proof of Lemma 15.
We give the proof for unfairness measure w.r
[Equation]
t. to equal opportunity first and then use this result to derive the proof for unfairness measure
w
[Equation]
r
[Equation]
t. to equalized odd.
Without loss of generality, assign group indices 1 , 0 be such that R 1 , 0 D j (cid:16) (cid:98) f (cid:17) ≥ R 1 , 1 D j (cid:16) (cid:98) f (cid:17) .
Then we have:
[Equation]
where 1 is vector with all 1’s.
By Lemma 14, we have:
[Equation]
Sum above two inequalities and add − 1 at both sides, we have,
[Equation]
Similarly, we have:
[Equation]
Sum both Eq.
(20) and Eq.
(21), we have:
[Equation]
Proof of Lemma 16.
Similar to the proof of Lemma 10, R y,aD i (cid:16) (cid:98) f (cid:17)
[Equation]
y, a ∈ { 0 , 1 } [Equation]
Then, we have:
[Equation]
Proof of Lemma 17.
∀ y ∈ Y , we have:
[Equation]
Here we have (1) = is because of law of total probability and Z ⊥ Y | X ; (2) = is because m yi,j is invertible function; (3) = is because P x | y i
= P m yi
[Equation]
j ( x ) | y j ∀ x ∈ X ; (4) ≤ is because of joint complexity of JS divergence.
By similar derivation, ∀ y ∈ Y , a ∈ A , we have:
[Equation]