Fairness and Accuracy under Domain Generalization
Authors: Thai-Hoang Pham
Xueru Zhang
Ping Zhang
Venue: ICLR 2023 poster (2023)
Keywords: fairness
accuracy
domain generalization
js-divergence
invariant representation
equalized odds
equal opportunity
regularization

Abstract: As machine learning (ML) algorithms are increasingly used in high-stakes applications, concerns have arisen that they may be biased against certain social groups. Although many approaches have been proposed to make ML models fair, they typically rely on the assumption that data distributions in training and deployment are identical. Unfortunately, this is commonly violated in practice and a model that is fair during training may lead to an unexpected outcome during its deployment. Although the problem of designing robust ML models under dataset shifts has been widely studied, most existing works focus only on the transfer of accuracy. In this paper, we study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from never-before-seen domains. We first develop theoretical bounds on the unfairness and expected loss at deployment, and then derive sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning. Guided by this, we design a learning algorithm such that fair ML models learned with training data still have high fairness and accuracy when deployment environments change. Experiments on real-world data validate the proposed algorithm.

Published as a conference paper at ICLR 2023
FAIRNESS AND ACCURACY UNDER DOMAIN GENERAL- IZATION
Thai-Hoang Pham, Xueru Zhang, Ping Zhang The Ohio State University, Columbus, OH 43210, USA (pham.377 zhang. 1 12807, zhang. 1 10631] @osuuuuee.. edu
ABSTRACT
As machine learning (ML) algorithms are increasingly used in high-stakes applica- tions, concerns have arisen that they may be biased against certain social groups. Although many approaches have been proposed to make ML models fair, they typically rely on the assumption that data distributions training and deployment are identical. Unfortunately, this is commonly violated in practice and a model that is fair during training may lead to an unexpected outcome during its deployment. Although the problem of designing robust ML models under dataset shifts has been widely studied, most existing works focus only on the transfer of accuracy. In this paper, study the transfer of both fairness and accuracy under domain generalization where the data at test time may be sampled from never- seeee--efor--eee domains. We first develop theoretical bounds on the unfairness and expected loss at deployment, and then derive sufficient conditions under which fairness and accuracy can be perfectly transferred via invariant representation learning. Guided by this, we design a learning algorithm such that fair ML models learned with training data still have high fairness and accuracy when deployment environments change. Experiments on real-world data validate the proposed algorithm. Model implementation is available https: //github. com waa.m//phhl0e 993/FATTM
INTRODUCTION
Machine learning (ML) algorithms trained with real-world data may have inherent bias and exhibit discrimination against certain social groups. To address the unfairness in ML, existing studies have proposed many fairness notions and developed approaches to learning models that satisfy these fairness notions. However, these works are based on an implicit assumption that the data distributions in training and deployment are the same, so that the fair models learned from training data can be deployed to make fair decisions testing data. Unfortunately, this assumption is commonly violated in real-world applications such as healthcare e.g., it was shown that most US patient data for training ML models are from CA, MA, and NY. with almost representation from the other 47 states (Kaushal et al., 2020). Because of the distribution shifts between training and deployment, a model that is accurate and fair during training may behave in an unexpected way and induce poor performance during deployment. Therefore, is critical to account for distribution shifts and learn fair models that are robust to potential changes in deployment environments.
The problem of learning models under distribution shifts has been extensively studied in the literature and is typically referred to as domain seapptaaioongenrralzziiinn on source domain(s) that can be generalized to a different (but related) target domain. Specifically, domain adaptation requires access (unlabeled) data from the target domain at training time, and the learned model can only be used at specific target domain. In contrast, domain generalization considers a more general setting when the target domain data are inaccessible during training; instead it assumes there exists set of source domains based on which the learned model can be generalized to an unseen, novel target domain. For both problems, most studies focus only on the generalization of accuracy across domains without considering fairness, e.g., by theoretically examining the relations between accuracy at target and source domains (Mansour et al. 2008 2009 Hoffman et al., 2018
where the goal is learn models
Zhao al.) 2018; Phung al., 2021 Deshmukh et al.] 2019 Muandet et al.] [013 Blachardd et al. 2021 Albuquerque et al. 2019; Ye et al. |021 Sicilia et al. [i.]2022]] Shui et al. (022) or/and
developing practical methods (Albuquerque et al. 2019 Zhao al. 2020 Li et al."" 2018a Sun &

---

Published as a conference paper at ICLR 2023
Ganin et al. 2016 llse al. 2020 Nguyen et al. 2021 To the best of our knowledge, only Chen et al. (2022); Singh et al. (202 I Coston et al. (2019) 9); Rezaci et al. (2021); Oneto et al. (2019); Madras et al. (2018); Schumann et al. (2019); Yoon et al. (2020) considered the transfer of fairness across domains. However, all of them focused on domain adaptation, and many also imposed rather strong assumptions on distributional shifts (e.g.. covariate shifts (Singh al. 202T Coston et al. 2019 Rezaei et al., (222)) demographic shift (Giguere et al., (022). prior probability shift (Biswas& & Mukherjee] (202T) D) that may be violated in practice. Among them, most focused
Saenko 2016;
on empirically examining how fairness properties are affected under distributional shifts, whereas
theoretical understandings are less studied (Schumann al., 2019 Yoon et al. 2020). Details and
more related works are in Appendix
In this paper, we study the transfer of both fairness and accuracy in domain generalization via invariant represen- tation learning, where the data in target domain is unknown and inaccessible during training. A motivating example shown in Figure D Specifically, we first establish a new theoretical framework that develops interpretable bounds on accuracyffairness at a target domain under domain gen- eralization, and then identify sufficient conditions under which fairness/accuracy can be perfectly transferred to an unseen target domain. Importantly, our theoretical bounds are fundamentally different from the existing bounds, com- pared to which ours are better connected with practical algorithmic design, i.e., our bounds are aligned with the ob- jective of adversarial learning- -baeed algorithms, method that is widely used in domain generalization.
Source Domain
Source Domain
CA
NY
Rad
Adaan
While
Training Data
[Fair ML
Model
Unsean
TX
Target
Figure I: An example of domain gener- alization in healthcare: (fair) ML model trained with patient data in CA, NY, etc.. can be deployed in other states by main- taining high accuracy/fairness.
Inspired by the theoretical findings, we propose Fairness and Accuracy Transfer by Density Matching (FATDM), two-stage learning framework such that the representations and fair model Iearned with source domain data can be well-generalized to unseen target domain. Last, we conduct the experiments on real-worke data; the empirical results show that fair ML models trained with our method still attain a high accuracy and fairness when deployment environments differ from the training. Our main contributions and findings are summarized as follows:
We consider the transfer of both accuracy and fairness in domain generalization. To the best of our knowledge, this is the first work studying domain generalization with fairness consideration.
We develop upper bounds for expected loss (Thm. I and unfairness (Thm. 3 in target domains. Notably, our bounds are significantly different from the existing bounds as discussed in Appendix
A We also develop a lower bound for expected loss (Thm. 2 it indicates an inherent tradeoff of the existing methods which learn marginal invariant representations for domain generalization.
We identify sufficient conditions under which fairness and accuracy can be perfectly transferred from source domains target domains using invariant representation learning (Thm. 4
We propose a two-stage training framework (i.e., based on Thm. 5) that learns models in source domains (Sec. D. which generalize both accuracy and faireess to target domain.
We conduct experiments on real-world data to validate the effectiveness of the proposed method.
PROBLEM FORMULATION
Notations. Let X, .A, and Y denote the space of features, sensitive attribute (distinguishing different groups, c.g., race/gender), and label, respectively. Let Z be the representation space induced from X' by representation mapping g : X' + Z. We use X, A, Y, Z to denote random variables that take values in X, Y, 2 and x, y, the realizations. A domain D is specified by distribution PD : X x A x y + [0, 1] and labeling function fD + YA, where is a probability simplex over y. Similarly, let hp Z + د be a labeling function from representation space for domain D. Note that fD. hpp. are stochastic functions and fD = hn g For simplicity, we use Pý (or / to denote the induced marginal (or conditional) distributions of variable V (given U) in domain D.
'The deterministic laheling function is special case when it follows Dirac delta distribution in A.
2

---

Published as a conference paper at ICLR 2023
Error metric. Consider hypothesis f= h thog. X + VA, where h 2 - is the hypothesis directly used in representation space. Denote f(z), as the element on y-th dimension which predicts the probability that label Y = given X = x. Then the expected error of in domain D is defined as (f) Ep [n([(f(X), Y)] for some loss function L V x y - R (e.g., 0-1 loss, eross-entropy loss). Similarly, define the expected error of h in representation space
(7) ED ED(<(((()) Y)]-
Note that most existing works (Albuquerque et al. 2019 Zhao et al. 2018) focus on optimizing
<b<((h). while our goal is to attain high accuracy in input space, i.e., low = (f).
Unfairness metric. We focus on group fairness notions (Makhlouf et al. 2021) that require certain statistical measures to be equalized across different groups; many of them can be formulated as (conditional) independence statements between random variables f(X), A, Y, e.g., demographic parity (f() A: the likelihood of positive outcome the same across different groups) (Dwork et al. 2012). equalized odds (f(X) 1. A|Y: true positive rate (TPR) and false positive rate (FPR) the same across different groups), equal opportunity (f(X) 1 LA 1 when y {0, 1}: TPR is the same across different groups) (Hardt et al.) 2016). the paper, we will present the results {0,1} and A = {0, 1}, while all the results (e.g..
under equalized odds (EO) fairness with binary methods, analysis) can be generalized to multi-class, multi-protected attributes, and other fairness notions. Given hypothesis f f= hog:& X + YA, the violation of EO in domain D can be measured as (f) E D (
(X)) Bally-BBAAA v.A |IP (x))xxXX 2.4=1) for some distance metric D(-1I-).
Problem setup. Consider a problem of domain generalization where learning algorithm has access to data {T ake yk. d& ) sampled from set of N source domains hEtN, where da is the domain label and [AV] {1, J.. Our goal is to learn representation mapping g X + Z and fair model h Z + y trained on source domains such that the model = can be generalized to an unseen target domain D in terms of both accuracy and fairness. Specifically, we investigate under what conditions and by what algorithms
guarantee that attaining high accuracy and fairness source domains {D Ji=1 implies small D (f) and D2 at unknown target domain.
3 THEORETICAL RESULTS
In this section, we present the results on the transfer of accuracy/fairness under domain generalization via domain-invariin invariant learning (proofs are shown in Appendix E We first examine that for any model h: - Y and any representation mapping 9 X - Z, how the accuracy/fairnes attained at source domains {D$ J.V can be affected when f = h g is deployed at any target domain D", Specifically, we can bound the error and unfairness at any target domain based on source domains. Before presenting the results, we first introduce the discrepancy measure used for measuring the dissimilarity between domains.
Discrepancy measure. We adopt measure the dissimilarity between two distributions. Formally, JS distance between distributions P and Pt is defined as
(JS) distance (Endres & Schindelin 2003) to
dJs(PPP') = (D)s(P
where Dss(PiIP') == (PL P+P' + } DKL P (
[ is JS divergence defined based on Kullback-Leibie (KL) divergence (-1.. Note that unlike KL divergence, JS divergence is symmetric and bounded: 0 < Dos 1.
While different discrepancy measures such as 7. and HAH divergences (Ben-David al. 2010) (i.e.,
definitions are given in Appendix (A) were used in prior works, we particularly consider JS distance because (1) it is aligned with training objective for discriminator generative adversarial networks (GAN) Gooddfelow et al., 2014), and many existing methods (Ganin et al. 2016] Albuquerque et al.) 2019) for invariant representation learning are built based on GAN framework; (2) H and HAH divergences are limited to settings where the labeling functions fo are deterministic (Ben-Davin cct al. 2010 Albuquerque et al., 2019 Zhao et al. 2018). In contrast, our bounds admit the stochastic labeling functions. The limitations of other discrepancy measures and existing bounds are discussed
in detail in Appendix A
3

---

Published as a conference paper at ICLR 2023
Theorem (Upper bound: accuracy) For any hypothesis h: Z - ,'a, any representation map- ping : + 2, and any loss function C YA x - R. that is upper bounded by C. the expected
error of = hog::: X + y at any unseen target domain D" is upper bounded 2
AAao Dr
$ N
Acc D
(
+ 2C min ds (PDr P.Y.Y
/20 max d3s s.JE[N]
( P.Y. Dj
DD
(1)
iE[M]
term (i)
term (ii)
torm (ti))
The upper bound in Eq. ① are interpretable and have three terms: term (i) is the averaged error of source domains in imput space; term (ii) is the discrepancy between the target domain and the source domains in input space; term (iii) is the discrepancy between the source domains in representation
space It provides guidance on learning the proper representation mapping X + 2: to ensure small error at target domain b (f), we shall learn representations such that the upper bound of
Acc (f) is minimized. Because term (it) depends on the unknown target domain D and it's evaluated DD
in input space X' y, it is fixed and is out of control during training, we can only focus term (i) and term (iii), i.e., learn representations Z such that errors at source domains b (J) and the discrepancy between source domains in the representation space d.ss(P?)) Y PD are minimized.
Corollary 1.1 Vi JS distance between pZy D DD.
and p2, in Eq. ① can be decomposed:
dls (P P
dJs (Pbę, pbg Dg) +
2E yr
(22l dss
Dē D
p
where DS
Pk: + PY
Our algorithm in Sec.
4 is designed based on above decomposition: because PT solely depends
D$$ D$ P ), Vi, Combining Thm. and Corollary 1.1 to ensure high accuracy at unseen target domain DT, learn the representation mapping and model h such that pž! invariant across source domains, and D
on source domain D we learn representations by minimizing dys (P21)
meanwhile -= Ti y attains high accuracy at source domains.
Note that unlike our method, many existing works (Phung al.) 2021 Albuquerque et al. 2019, Ganin et al. 2016) suggest that to ensure high accuracy in domain generalization, representation mapping g should be learned such that Pb is same across domains, i.e., small d (P2 D Prr)).
However, we show that the domain-invariant P3 may adversely increase the error at target domain, as indicated in the Thm. 2 below.
Theorem (Lower bound: accuracy) Suppose ((f()),))= EsEy L(r) where function L: x y + R4 is lower bounded by e when ÿ # and is 0 when ý = y. If dIs(P) D. pY Z
d (P% Ds. PTTr), the expected error of f source and target domains is lower bounded:
Σ D ( + (8)())
4DJN Σ (dis(Phs. Pbr) dos( Pz )br)))
C
(2)
The above lower bound shows an inherent trade-off of approaches that minimize d (P) p when learning the representations. Specifically, with the Komain-invarian P/,,, the right hand side of Eq. (2) may increase, resulting in an increased error at target domain 2 (f)
'Th condition the bounded loss is mild and be satisfied by many loss functions. For example, cross- entropy loss be bounded by modifying the softmax output from (p1, (2..P..... .Ppr))
Pb, -P>1).
where f. = [1 (1- exp(- exp(-C))D1) + exp(-C),
E |//
IIn fact, tighter upper bound for the loss at target domain can he established using strong data processing
incquality (Polyanskiy Wul 2017), as detailed in Appendix [T]
4

---

Published as a conference paper at ICLR 2023
Similar to the loss, the unfairmess target domain also be upper bounded, as presented in Thm.
Theorem (Upper bound: fairness) Consider a special case where the unfairness measure is defined as the distance between means of two distributions:
< ( ED [F(X) V,A ED -
then the unfairness at any unseen target domain DJ is upper bounded:
D
D
/2 min
iE[M]
E
Σ
d1s
(P (PNYY=uAzO
pp
p
WE (E (E(0.))
+ max [JE[N]
Σ DE(0,1) (E(1,1)
das (
p2
D.
Similar to Thm. [ the upper bound Thm. 3 also has three terms and the second term out of control during training because it depends on the unseen target domain and is defined in input space. Therefore, to maintain fairness at target domain D", we learn the representation mapping g and model h such that p is invariant across source domains, and meanwhile f h g attains high fairness source domains.
The results above characterize the relations between accuracy/fairness at any target and source domains under any representation mapping and model h. Next, identify conditions under which the accuracy/faimess attained at sources can be perfectly transferred to a target domain.
Theorem (Sufficient condition for perfect transfer) Consider N source domains (DS) and an unseen target domuin D Define set
(D': D' = = T.DS, {TC} AN-I).
1. (Transfer of fairness) V A, ifg is the mapping under which P2YYA is the same across all source domains, then
80 (h) = Dr (h) D
= = De
D (f), Vi.
2. (Transfer of accuracy) VD E A, P is the same and if g is the mapping under which p Y is the same across all source domains, then Aue (h) = d (T) = D. ( De (i), Vi.
DE
Thm. 4 iindicates the possibility of attaining the perfect transfer of accuracy/fairness and examples of such representation mappings are provided. Note that these results are consistent with Thm. and Thm. which also suggest learning domain-invarian representations p>l and pd Y,A
PROPOSED ALGORITHM
The accuracy and fairness upper bounds in Sec.
Table 1: Usages of terms in Eq. 3 guarantee the fairness and accuracy in target domain.
shed light on designing robust ML model that can preserve high accuracy and faimess unseen tar- get domains. Specifically, the model consists of representation mapping g X' Z and classi- fier h 2 + y such that (1) the prediction errors and unfairness of f h o on source domains minimized; and (2) the discrepancy of learned P conditional representations (i.e, p2!"" and DD among source domains is minimized. That is.
Loss terms
Usages
Mimimize Ade Df
Ljuir
30 Mimimize Df
Minimize das (P p p.IY..
Live
and d (pZYA .
D3
Leta Ljair
Mimimize 6 and Ep
min (ei(g,h) h) fa Ri) ( Linv g
where Lets, Cyair. and Line are expected losses that penalize incorrect classification, unfairness, and diserepancy among source domains. paper--prrmmters >> 0 and 7 > 0 control the accuracy- fairness trade-off and accuracy-invarians representation trade-off, respectively. The usages these three losses are summarized in T
5

---

Published as a conference paper at ICLR 2023
Ad Adversarial learning framework (Goodfellow et al. 2014). Liny in Eq. can be optimized directly with adversarial learning. This is because the training objective of the discriminator in GAN is aligned with our goal of minimizing JS distance between Po (or P among source domains, as mentioned in Sec. Specifically, define set of discriminators K = {ky E }} (kg,a e V', a A]; each discriminator ky (resp. ky,a) aims to distinguish whether sample with label y (resp. label y and sensitive attribute a) comes from a particular domain (i.e., maximize Line). The representation mapping g should be learned to increase the error of discriminators (i.e., minimize Linv). Therefore, the model and discriminators can be trained simultaneously by playing two-player minimax game (i.e., ming maxk, Lino(g)). Combine with the objective of minimizing prediction error and unfairness (i.e., min 9. Cassgg h)) + wh h)), the overall learning objective is:
min max (.) h) + Wh (g, h) + (fir (g)
g.h
However, the above adversarial learning framework for learning domain invariint representation may not work well when ()/ x A| is large: as the label space and sensitive attribute space get larger, the number of discriminators to learned increases and the training can be highly unstable. naive solution to tackling this issue is to use one discriminator Vy E ) a E A. However, this would result in the reduced mutual information between representations and label/sensitive attribute, which may hurt the accuracy. We thus propose another approach to learn the domain- invariant representations.
Proposed solution to learning invariant rep- resentations. For any domain D, we have:
source Jonain
source damain
p
pay
p2w = [ p2. v dx
/ p2lz pllppplaa dx
Owelt
P
Y
PD pBlv. S Ph pa.vuaa wx = 1 p71 polu dr
aly
pau
pize
Denry
by MEe
where pZle is domain-independennt so we drop D in subseript. Given any two source do- mains D and D,S, in general p * p. and plln D
23,
Figure 2: ID illustration of domain-invarisn rep- resentation. To transfer accuracy and fairness to target domains, we need to find representation such that pely and paly.o are domain-invarian D.
* pxlua so that it is non-
D trivial achieve domain-invariant representa- tions p P and p = = -221,0 How-
ever, if there exist invertible functions m X - X' and m.j
X - X' that can match the density
functions of X from D to D such that x = (X)y and ply. D
(X)l,,a and
D. P
DØ
if find the representation Z such that p P (x) and p7" l Ds
(a)
P D
then
vy E V,a a E A, we have:
p2ly
DZppply dr D$
/ p7la pelly de' p21y
D3
x
D2
Z2a D
p Xlyia dx
/ p2 a" pr dx" = Al D
Dg
x
where x'= 30'mm; m (z) and x² m3 (x)) This observation suggests that to minimize the discrepancy of representation distributions among source domains, we can first find the density mapping functions n.a mmy and mz Vy, a, i.j. and then minimize the discrepancies between pž(# æ p2 and p2|=". Vz. This formally shown Thm. below.
(z))
Theorem 5 If there exist invertible mappings m and m such that P x
Pm
(x1lv De
and
mi (X)luaa Vy, a, i, and if the representation mapping are in the form of := p2 pxln = PD. N(((s),, T2T)) where p(x) is the function of and d is the dimension of the representation space
2, then minimizing d.s (p P and dus (p? .p
pZ1y.a) can be reduced to minimizing
[m(x)) a (m? (z)) ||, and [|ı(x)) μ (m (x)) respectively.

---

Published as a conference paper at ICLR 2023
Based on Thm.
we propose two-stage learning approach FATDM, as stated below.
Remark 1 (Fairness and Accuracy Transfer by Density Matching (FATDM)) Given the exis- tence of density matching functions mi, and m.. domain-inwe invariant representations can be learned via two-stage process: (i) finding these mapping functions m and m v.o, (ii) minimizing the mean squared errors between μ(z) and y (m)))())) and p(x) and u (m (1)) Vi.j E [N], E V,a A.
v.a. and representation mapping 9 := NN((x)), oala),
Stage 1: learning mapping functions m and m across source domains. Many approaches can v.a be leveraged to estimate m² and m. from data, In our study, we adopt StarGAN (Choi et al. 2018) and CycleGAN (Zhu et 2017) examples; both frameworks are widely used in multi-domam image-to-image translation and can be leveraged. In our algorithm, we independently train two translation models DensityMatch" and DensityMatch" 4:4 using StarGAN or CycleGAN, with each used for learning {m,, [EENAAJ[[]] and {m (EAkEE) respectively.
Specifically, DensityMatch' Y (or DensityMatch V con- sists of a generator G: x [N] x [N] + X and a discrim- inator D X + [N] x {0, 1}. The generator takes in real image x and a pair of domain labels i, as input and generates fake image; the discriminator aims to predict the domain label of the image generated by the generator and distinguish whether it is fake or real. G and D are learned simultaneously by solving the minimax game, and their loss functions are
source domain
source domain
Densty
zly pe
Stage Finding ml,
pay Vy
2
specified Appendix B When the training completed, we obtain two optimal generators from DensityMatch" and DensityMatch" Y.4 denoted as GY and GY.A, We shall use GY (, (resp. G..A(,,i,)) i,))) directly as the density map- ping function (m),,())ev> (.)eey (resp. {m (.)EEA
Dersity
Stage Enfording plaki paint (+) Va X, y
Stage 2: learning domain- representation. Given Figure 3: FATDM: two-stage training GY and GY.A learned in stage 1, we are ready to learn the invariant representation Z by finding g: X - Z such that g == N (u(x), o T²Ia) and minimizes the following:
Line Ed,d' ,''' 1f heM [mmse (M()), ((X))) + Lmsse (MX),
(5)
where d, d', d" are domain labels sampled from source domains, X is features sampled from domain X' = GŌ GY(XX d. d, d'), X" = GY.A (X, d, d'), Lisee is mean squared error. The pseudo-code of our The detailed architecture of FATDM is in Appendix B
proposed model (FATDM) is in Algorithm [
Algorithm 1: Fairness and Accuracy Transfer by Density Matching (FATDM)
Input: Training dataset Diros from N source domains (D fi=1
Output: representation mapping g. classifier h, density matching functions GY.A (Dirain)
Procedure Density_Mat
/x Procedure for training GY is s similat but not presented //
while training DensityMatch" V,A is not end do
Sample y ~ y, a ~ AA and data batch B = {ax, dylas aa = a, y)
k-1 from Dirain
[8]
Update G.AA hased the objectives of the minimax game (Appendix B
Procedure Invariars ant Representation__t Lnriienrii_nnngggeeaaaaeaeeegggg (Dirain, GY, G
while training FATDM is not end do
Sample data batch B {xe, Ok Uk. del from Dirain
Sample lists of domain lahels [d])[1 and {d] kkmi [d])
Generate sets artificial images (xf [== and {a [81 by GY and GY-A,
10
Update g. h by optimizing Eq. with Liov defined Eq.
Remark 2 (Summary of theoretical results and proposed algorithm) Thm. I Dand Thm. 3 suggest
a way to ensure high accuracy and fairness target domain: by minimizing the source error DD (i.e.,
Lote in Eq. (3). the source unfairness DD; (i.e., in Eq. 3 ), and the discrepancies between source
(p/ Y= Y= pz and d
(p22 (p2 =y pZy = (i.e., Linn in Eq. (3)). The
domains d.s
PP
D2
7

---

Published as a conference paper at ICLR 2023
common way optimize Eq. (3) using adversarial learning (Eq. (4)) is not stable when |>/ x Al is
large. Thm. states that instead of using adversarial learning. Eq. (3) can be optimized via 2-stage
learning: (i) find mappings m and m
/ Waanstt_Maatchaee in Alg. 7 and (ii) minimize
Eq. with Line defined in Eq. (5) Invariantt TetaaraattRReeeeettttiiiooo Lea andaaning Tearning in Alg.
5 EXPERIMENTS
We conduct experiments on MIMIC-CXR database (Johnson al., 2019). which includes 377,1 110 chest X-ray images associated with 227,827 imaging studies about 14 diseases performed at the Beth Israel Deaconess Medical Center. Importantly, these images are linked with MIMIC-IV database (Johnson et al. 2021) which includes patients' information such as age, and race; these can serve as sensitive attributes for measuring the unfairness. Based on MIMIC-CXR and MIMIC-IV data, we construct two datasets on two diseases:
Cardiomegaly disease: we first extract all images related to Cardiomegaly disease, and the corre- sponding labels (i.e.. positiveennegatie) and sensitive attributes (i.e., male/female); then we partition the data into four domain-speciinn datasets based on age (i.e., [18, 40), [40, 60), [60, 80), [80, 100)). We consider age as domain label because captures the real scenario that there are distribution shifts across patients with different ages.
Edema disease: we extract all images related Edema disease, and corresponding labels (i.e., pos- itive/negative) and sensitive attributes (i.e., age with ranges [18, 40), [40, 60), [60, 80), [80, 100)). Unlike Cardiomegaly data, we construct the dataset for each domain by first sampling images from Edema data followed by 0 degree counter-clockwise rotation, where 0 E {0', 15 ,0,, 45" 60%) We consider rotation degree as domain label to model the scenario where there rotational misalignment among images collected from different devices.
Next, we focus on Cardiomegaly disease and the results for Edema disease are shown in Appendix
C
Baselines. We compare our method (i.e., PATDM-Sttaraa and FATDW-CycleGAN) with exist- ing methods for domain generalization, including empirical risk minimization, domain invariant representation learning, and distributionally robust optimization, as detailed below.
Empirical risk minimization (ERM): The baseline that considers all source domains as one domain. Domain invariant representation learning: Method that aims to achieve the invariant across source domains. We experiment with G2DM (Albuquerque et al. 2019), DANN (Ganin al. 2016), CDANN et al. 2018c). CORAL (Sun & Saenko (0166), IRM (Arjovsky et al. 2019). These models focus
accuracy transfer by enforcing the invariance of distributions PZ: P21x
Distributionally robust optimization: Method that learns a model at worst-case distribution to hope it can generalize well on test data. We experiment with GroupDRO (Sagawa et al.) 2019) that minimizes the worst-case training loss over set of pre-defined groups through regularization. ATDM: A variant of FAIDM- StarGAN that solely focuses on accuracy transfer. That is, we only enforce the invariance of PO! during learning which is similar to Nguyen et al. (2021).
The implementations of these models except G2DM adapted from DomainBed framework (Gulra jaan & Lopez-Paz] 2020). For G2DM, we use the author- provided implementation. For all models, we use R8 (He et al. 2016) as the backbone module of representation mapping y X' - Z: and fairness constraint Leair is enforced as a regularization term added to the original objective functions.
Experiment setup. We follow leave-one-out domain setting in which 3 domains are used for training and the remaining domain serves as the unseen target domain and is used for evaluation. Several metrics are considered to measure the unfairness and error of each model in target domain, including:
Error: cross-entropy loss (CE), misclassification rate (MR), AUROC == AURROC, AUPR := 1 1-AUPR, Fj := - Fi, where AUROC, AUPR, Fi are area under receiver operating characteristic curve, area under precision-recall curve, Fi score, respectively.
we consider both equalized odds and equal opportunity fairness notion, and adopt mean distance (MD) and earth mover's distance (EMD) as distance metric D(-1I-).
Unfairness:
Fairness and accuracy on target domains. We first compare method with baselines in terms of the optimal crade-off (Pareto frontier) between accuracy and fairness on target domains under different metric pairs. 4shows the error-ufaarress curves (as varies from 0 (no faimess constraint) to 10 (strong fairness constraint)), with AUROC and MR as error metric, and equalized odds (measured under distance metrics MD and EMD) faimess notion; the results for other error metrics are similar
8

---

Published as a conference paper at ICLR 2023
Unfoimes (MD)
Unfarmen ( JEMD)
Unfaimea JEMD))
Unfaimees
Erer AUROC
Error (MK)
Error AUROC
Error (MK)
Figure 4: Fairess-accuracy trade-off (Pareto frontier) of FATDM-StarGAN FATDM- MATDM-CycleGAM
and bascline methods: eror-unfairness curves are constructed by varying w E [0, 10) and the values of error and unfairness are normalized [0. 1]. Lower-left points indicate the model has better fairness- accuracy trade-off (Pareto optimality).
0175-
3.0
3.8
075
0.33
Figure 5: Prediction performances (AUROC, AUPR, Accuracy, Fi) of FATDM-StarGAN Car- diomegaly disease data when varying hyper-parameter 7 at different levels fairness constraint w.
and shown in Appendix C Our observations as follows: (1) As expected, there is trade-off between fairness and accuracy: for all methods, increasing w improves fairness but reduces accuracy. (2) Among all methods, the Pareto frontiers of FATDM-Sta rGAN and FATDM-Cyc. eGAN are the bottom leftmost, implying that our method attains better aucness--ccurrcy trade-off than baselines. (3) Although fairness constraint is imposed during training for all methods, the faireess attained at source domains cannot be well-generalized to the target domain under other methods. These results validate our theorems and show that enforcing the domain-invarian p Y and 2 when learning worth-noting that under this P2 (accuracy transfer) does not imply the domain-invarian P
representations ensures the transfer of both accuracy and fairness. dataset, the (fairness transfer). This is because domain D (i.e., age) is correlated with label Y (i.e., has a disease) and sensitive attribute (i.e., gender), making the distribution PY,,A different across domains.
Dz
Impact of density mapping model. To investigate whether the performance gain of our method is due to the use of any specific density mapping model, we adopt StarGAN and Cycl eGAN architectures to learn density mapping functions in our method and compare their performances. Figure 4 shows that PATDM-Stazee at the target domains and both of them outperform the baselines. This result shows that our method is not limited any specific density mapping model and is broadly applicable to other architectures.
and CycleGAN achieve similar fairness-act accurracy trade-off'
Impact of invariant representation constraints. We also examine the impact of Live on the perfor- mance of FATDM-St arGAN target domains, where we vary the different levels of fairness (i.e., fix w 1, 5, 10) and examine how the prediction performances (i.e., AUROC, AUPR, accuracy and Fi) could change. Figure 5 shows that enforcing domain-invariann constraint Line helps transfer the performance from source target domain, and that attains the highest accuracy at target domain can be different for different levels fairness. The results also indicate the fairness-accuracy accuracy trade-off, i.e., for any 7, enforcing stronger fairesss constraints (large w) could hurt prediction performances.
y E [o, 5e at
6 CONCLUSION
In this paper, we theoretically and empirically demonstrate how to achieve fair and accurate predic- tions in unknown testing environments. To the best of our knowledge, our work provides the first theoretical analysis to understand the efficiency of invariant representation learning in transferring both fairness and accuracy under domain generalization. In particular, we first propose the upper bounds of prediction error and unfaimeess in terms of JS-distance, then design the two-stage learning method that minimizes these upper bounds by learning domain-invariant representations. Experiments on the clinical data demonstrate the effectiveness of our study.

---

Published as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
The original chest X-ray images and the corresponding metadata can be downloaded from PhysioNet (attps://physionett, http:///eee...oooooooaaaa/aa///a
https: htt.://te.....ceooooooooooocna/a 0/). Codes for data processing and proposed algorithms are in supplementary materials. Technical details of the proposed algorithms and experi- mental settings are in Appendix B. Additional experimental results are in Appendix C. Lemmas used proofs of the theorems in the main paper are in Appendix D. Complete proofs of the theorems in the main paper and the corresponding lemmas in Appendix E.
ACKNOWLEDGEMENTS
This work was funded in part by the National Science Foundation under award number IIS-2145625, by the National Institutes of Health under award number UL URIOR002733 and by The Ohio State University President's 's Research Excellence Accelerator Grant.
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. In International Conference on Machine Learning, pp. 60-69. PMLR, 2018.
Isabela Albuquerque, João Monteiro, Mohammad Darvishi, Tiago H Falk, and Ioannis Mitliagkas. Generalizing to unseen domains via distribution matching. arXiv preprint arXiv:1911.0080 2019.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint 1rXiv:1907.0289) 2019.
Yahav Bechavod, Christopher Jung, and Steven Z Wu. Metric-free individual faimeess in online learning. Advances neural information processing systems, 33:11214-11225, 2020.
Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1):151-175, 2010.
Asia J Biega, Krishna P Gummadi, and Gerhard Weikum. Equity of attention: Amortizing individual fairness rankings. In The 41 st international aem sigir conference on research & development in information retrieval, pp. 405 414, 2018
Arpita Biswas and Suvam Mukherjee. Ensuring fairness under prior probability shifts. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 414-424 2021.
Gilles Blanchard, Aniket Anand Deshmukh, Urun Dogan, Gyemin Lee, and Clayton Scott. Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22:1-55. 2021.
Fabio M Carlucci, Antonio D' D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2229-2238, 2019.
Yatong Chen, Reilly Raab, Jialu Wang, and Yang Liu. Fairness transferability subject to bounded distribution shift. arXiv preprini noXiv:2206.00/29 2022.
Yunjey Choi. Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jaegul Choo. Star- gan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8789-8797, 2018.
Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R Varshney, Skyler Speakman, Zairah Mustahsan, and Supriyo Chakraborty. Fair transfer learning with missing protected attributes. In Proceedings of the 2019 AAAI/ACM Conference AI, Ethics, and Society, pp. 91-98, 2019.
10

---

Published as a conference paper at ICLR 2023
Aniket Anand Deshmukh, Yunwen Lei, Srinagesh Sharma, Urun Dogan, James W Cutler, and Clayton Scott. A generalization error bound for multi-class domain generalization. arXiv preprint 1035::2.05..039.. 2019.
Luc Devroye, Abbas Mehrabian, and Tommy Reddad. The total variation distance between high- dimensional gaussians. arXiv preprint arXiv: 1818.0800000,.. 2018.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214-226, 2012.
D.M. Endres and J.E. Schindelin. A new metric for probability distributions. JEEE Transactions on Information Theory, 49(7):1858. (9(7))1858-186)) 1860, 2003. doi: 10.110997T1122058813566.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180-1189. PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of muchine learning research, 17(1):099-20300 2016.
Stephen Giguere, Blossom Metevier, Bruno Castro da Silva, Yuriy Brun, Philip S Thomas, and Scott Niekum. Fairness guarantees under demographic shift. In International Conference on Learning Representations, 2022.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. arXiv preprint arXiv:2007.01434, 2020.
Swati Gupta and Vijay Kamble. Individual fairness in hindsight. Journal of Machine Learning Research, 22(144):1-35, 2021.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, Advances in neural information processing systems, 29, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. Algorithms and theory for multiple-source adaptation. Advances in Neural Information Processing Systems, 31, 2018.
Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning, pp. 2029-2033 PMLR, 2018
Zeyi Huang, Haohan Wang, Eric Xing, and Dong Huang. Self-challenging improves domminn generalization. In European Conference on Computer Vision, pp. 124-140 Springer, 2020.
Maximilian Ilse, Jakub M Tomezak, Christos Louizos, and Max Welling. Diva: Domain invariant PMLR, 2020.
variational autoencoders. In Medical Imaging with Deep Learning, pp.
Seogkyu Jeon, Kibeom Hong, Pilhyeon Lee, Jewook Lee, and Hyeran Byun. Feature stylization and domain-awart contrastive learning for domain generalization. In Proceedings of the 29th ACM International Conference Mullimedia, pp. 22-31, 2021.
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Homg, Leo Anthony Celi, and Mark Roger. Mimic-iy. httpp::/ //doi wwg/10. 1302
2021.

---

Published as a conference paper at ICLR 2023
Alistair EW Johnson, Tom J Pollard, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth Berkowitz, and Steven Homg. Mimic-cxr-jpp jpg. a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07044 070422. 2019.
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimi- nation. Knowledge and information systems, 33(1):1-33, 2012.
Amit Kaushal, Russ Altman, and Curt Langlotz. Geographic distribution of us cohorts used to train deep learning algorithms. Jama, (24(11):1212-121) 2020.
Dachee Kim, Youngjun Yoo, Seunghyun Park, Jinkyu Kim, and Jackoo Lee. Selfreg: Self-supervised contrastive regularization for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1619-9628, 2021.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, al. Wilds: A benchmark of in-the-will wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664. PMLR, 2021.
David Krueger, Ethan Caballero, Joern- Heerik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapola- tion (rex). In International Conference on Machine Learning, pp. 5815-5826. PMLR, 2021.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5400-5409, 201 8a.
Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao. Domain generalization via conditional invariant representations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018b.
Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 624-639, 2018c.
Zheren Li, Zhiming Cui, Sheng Wang, Yuji Qi, Xi Ouyang. Qitian Chen, Yuezhi Yang. Zhong Xue, Dinggang Shen, and Jie-Zhi Cheng. Domain generalization for mammography detection via multi-style and multi-view contrastive learning. International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 108.08. Springer, 2021.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792. PMLR, 2021.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3383.3393. PMLR, 2018.
Karima Makhlouf, Sami Zhioua, and Catuscia Palamidessi. Machine learning fairness notions: Bridging the gap with real-world applications. Information Processing & Management, 58(5): 102642, 2021.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation with multiple sources. Advances in neural information processing systems, 21, 2008
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Multiple source adaptation and the rényi divergence. In Proceedings of the Twenty-Fiph Conference on Uncertainty in Artificial Intelligence, pp. 367-374, 2009.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv: 180.00557, 2018.
12

---

Published as a conference paper at ICLR 2023
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. International Conference on Machine Learning, pp. 10-18. PMLR, 2013.
A Tuan Nguyen, Toan Tran, Yarin Gal, and Atilim Gunes Baydin. Domain invariant representation learning with domain density transformations. Advances in Neural Information Processing Systems, 34, 2021.
Luca Oneto, Michele Donini, Andreas Maurer, and Massimiliano Pontil. Learning fair and transfer- able representations. arXiv preprint arXiv: /906. 10673, 2019.
Trung Phung, Trung Le, Tung-Long Vuong, Toan Tran, Anh Tran, Hung Bui, and Dinh Phung. On learning domain-invariant representations for transfer learning with multiple sources. Advances in Neural Information Processing Systems, 34, 2021.
Yury Polyanskiy and Yihong Wu. Lecture notes on information theory. Lecture Notes for ECE563 (UIUC) and, 6(2012-2016))7, 2014.
Yury Polyanskiy and Yihong Wu. Strong data- sittapprceesing inequalities for channels and bayesian networks. In Convexity and Concentration, pp. 211-249. Springer, 2017.
Fengchun Qiao, Long Zhao, and Xi Peng. Learning to learn single domain generalization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12556-12565, 2020.
Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of-distribution generalization. arXiv preprint arXiv:2109002934, 2021.
Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian D Ziebart. Robust faimesss under covariate shift. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9419-94499 9427, 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2019.
Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed H Chi. Transfer of machine learning fairness across domains. arXiv preprint arXiv: arXiv:1966.098 09688, 2019.
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. Generalizing across domains via cross- cross-gradient training. In International Conference on Learming Representations, 2018.
Yuge Shi, Jeffrey Seely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. arXiv preprint arXiv:2104.09937, 2021.
Changjian Shui, Boyu Wang, and Christian Gagné. On the benefits of representation regularization in invariance based domain generalization. Machine Learning, pp. 1-21, 2022.
Anthony Sicilia, Xingchen Zhao, and Scong Jae Hwang. Domain adversarial neural networks for domain generalization: When it works and how to improve. arXiv preprint arXi::2102.03924 2021.
Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Faimess violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 3-13, 2021.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European conference on computer vision, pp.
Springer, 2016.
Chris Xing Tian, Haoliang Li, Xiaofci Xie, Yang Liu, and Shiqi Wang. Neuron coverage-guided domain generalization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Riccardo Volpi. Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. Advances in neural information processing systems, 31, 2018.
13

---

Published as a conference paper at ICLR 2023
Jingge Wang, Yang Li, Liyan Xie, and Yao Xie.
domain generalization via
wasserstein distributional robust optimization. arXiv preprint arXiv:2109.03676, 2021.
Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards theoretical framework of out-of-distribution generalization. Advances in Neural Information Processing Systems, 34, 2021.
Taeho Yoon, Jaewook Lee, and Woojin Lee. Joint transfer of model knowledge and fairness over domains using wasserstein distance. IEEE Access, 8:123783-123798, 2020.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguuz and Krishna P Gummadi. Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research, 20(1):273727788, 2019.
Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. In International conference on machine learning. pp. 325-333. PMLR, 2013.
Hongyi Zhang. Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
Xueru Zhang, Mohammadmahdi Khaliligarekani, Cem Tekin, al. Group retention when using machine learning in sequential decision making: the interplay between user dynamics and fairness. Advances in Neural Information Processing Systems, 32, 2019.
Xueru Zhang, Ruibo Tu, Yang Liu, Mingyan Liu, Hedvig Kjellstrom, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualification? Advances in Neural Information Processing Systems, 33:18457-18866 2020.
Han Zhao, Shanghang Zhang, Guanhang Wu, José MF Moura, Joao Costeiia, and Geoffrey Gordon. Adversarial multiple source domain adaptation, Advances in neural information processing systems, 31, 2018.
Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain generalization via entropy regularization. Advances Neural Information Processing Systems, 13:16696-16107,
2020.
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In European conference on computer vision, pp. 561-578. Springer, 2020.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference computer vision, pp. 2223-2233 2017.
14

---

Published as a conference paper at ICLR 2023
A RELATED WORKS
adaptation: In many real scenarios of machine learning, data in training phase is sampled from one or many source domains, while in the testing phase, data is sampled from an unseen target domain. Many works have been proposed to design robust ML models that can achieve good performances deployment environment depending on whether they can access to the target data (domain adaptation) or not (domain generalization). However, most of these models focus only on transfering accuracy from source to target domains and can be categorized into five main approaches: (1) data manipulation (Volpi et al. 2018: Qiao et al. 2020; Zhou et al. 2020 Zhang et al., 2018 (Shankar et al., 2018); (2) domain-invariant representation learning (Li et al.
Domain generalization/Donas
2018b a Ganin & Lempitsky| [013]] Ganin al. 2016 Phung al. 2021 Nguyen et al. 202 0; (3)
distributional robustness (Krueger et al. 2022] Liu et al. [202]] Koh et al. 2021 Wang et al.
202
Sagawa et al.
2019; Hu et al.
(0118), (4) gradient operation (Huang et al. 2020; Shi et al.
202
Rame et al.
2021 ran et al.
2022). and (5) self-supervised learning (Carlucci el al. 2019
Kim
ct al.
2021 Jeon et al.
2021 Li et al. 2021).
Fairness in Machine Learning: Many fairmess notions have been proposed to measure the unfairness in ML model, and they can be roughly classified into two classes: Individual fairness considers the equity at the individual-level and it requires that similar individuals should be treated similarly
(Biega et al., 2018] [Bechavod et al.) 2220)] Gupta & Kamble| [02T] Dwork et al.) 2012). Group fairnesy attains a certain balance in the group-level, where the entire population is first partitioned into multiple groups and certain statistical measures are equalized across different groups (Hardi et al., 2016: Zhang et al., 2019 2020). Various approaches have also been developed to satisfy these fairness notions, they roughly fall into three categories: (1) Pre-processing: modifying training
dataset to remove bias before learning ML model (Kamiran & Calders 2012 Zemel et al. 2013).
(2) In-processing: attain fairness during the training process by imposing certain fairness constraint or modifying loss function. (Zafar al., 2019 Agarwal et al. 2018) (3) Post-processing: altering the output of an existing algorithm to satisfy a fairness constraint after training (Hardt et al. (0016).
However, most of these methods assume the data distributions at training and testing are the same. In contrast, we study fairness problem under domain generalization in this paper.
Fairness under Domain Adaptation: There are some studies proposed to achieve good fairness when the testing environment changes but all of them focused on the domain adaptation setting. The most common adaptation setup is learning under the assumption of covariate shift. For example, Singh et al. (2021) leveraged a feature selection method in causal graph describing data to mitigate fairness violation under covariate shift of distribution in testing data. [Coston et al. (2019) proposed the weighting methods that can give fair prediction under covariate shift between source and target distribution when access to the sensitive attributes is prohibited. |Reaai et al. (202T) sought fair decisions by optimizing a worst-case testing performance. Besides convariate shift, there are some works proposed to handle other types of distribution shift including demographic shift and prior probability shift. Instead of learning fair model directly, Oneto et al. (201)) and Madras et al. (2018) find fair representation that can generalize to the new tasks. Aside from empirical studies, [Schumann) al. (2019) and Yoon al. (2020) developed theoretical frameworks examine fairness transfer in domain adaptation setting and then offered modeling approaches to achieve good fairness in the target domain.
Comparison with existing bounds in the literature: We compare our bounds with most commons bound in the fields of domain adaptation and domain generalization as follows.
Accuracy bounds in domain adaptation.
Bounds in Ben-David et al. (0100):
Ace
f S
i
+ DrV (PD ) min ED (X) - fpr (X)Il
DE(DS,DT) .DT
This bound is for binary classification problem under domain adaptation. The classification error in target domain is bounded by the error in source domain, the total variation distance of feature distribution between source and target domain, and the misalignment of the labeling function between source and target domain. The limitation of this bound is that (1) it's only applicable to settings with zero-one loss function and deterministic labeling function; (2) estimating the total variation distance is hard in practice and it doesn't 't relate the feature and representation spaces.
15

---

Published as a conference paper at ICLR 2023
This paper also provides another accuracy bound based on HAH divergence:
Acc DD"
f S EDS (f) + DHAN (PDs || PDx)
+ inf
f DD
Acc
t0s (f)
where DHAH (PY || PDx) sup PDs
-Pr
Por (R(X) + fa(x)) is the
HAH divergence. However, it has the same limitations as total variation distance mentioned above.
Accuracy bounds in domain generalization.
Bounds in Albuquerque et al. 2019):
N
Acc DD
ACC (f)
max Dzc (PDg PD + Dz (PA5 || Pbr)
THe
D
J.KE[N]
t=1
min DE(DE.DT)
ED (X) [xr(X)]
where Dz (PTs p = sup PDs (f(X) -1) Por Por (f(x)=1) the H divergence, /
P.P = argminD EE T.P || PD is the mixture of source domains that is closest to D
target domain with respect to H. divergence. In this bound, the classification error in target domain is bounded by the convex combination of errors in source domains, the H divergence between source domains, the H. divergence between target domain and its nearest mixture of source domains, and the misalignment of the labeling function between mixture source domains and target domain. Because this bound is constructed based H divergence, it also has the limitations for the bounds in domain adaptation (Ben-David et al. 2010) as we mentioned. This bound can be transformed to the representation space by replacing X by Z in its formula. Then, this bound suggests enforcing invariant constraint of marginal distribution of representation Z across source domains, which has inherent trade-off as shown in Thm. 2 Because the target domain is unknown during training, the mixing weights (T;) i= are not useful for algorithmic design.
Bounds in Phung et al. 2021):
Aap DD
N
Acc (T)
CinaEEDs iE[N]
[[
JDz (x)
31 y=
THe
D
C 2 d1/2
N
C CV/ d2/2 (P75, p?3))
pZ Psg
N
where d1/2 Pxx POY)) )
/D122
(pxx
is Hellinger distance defined based on
fxx (vP lates the feature and representation spaces that the classification error of target domain defined in feature space is bounded by classification errors of source domains defined in feature space, the misalignment of labeling function between target and source domains, and the Hellinger distances between source and target domains and between source domains of marginal distribution of rep- resentation Z. While this bound not limited to zero-one loss and the labeling function can be stochastic, it suggests the alignment of marginal distribution of representation Z across source domains for generalization. Moreover, estimating Hellinger distance can be hard in practice.
Hellinger divergence D1/2
PX D
P.X D3
dx. This bound re-
The mismatch between existing bounds and adversarial learning approach for domain generalization.
All existing bounds mentioned above suggest minimizing the distances between representation distributions across source domains with respect to some discrepancy measures such as H divergence, total variation distance, and Hellinger distance. Based on these bounds, adversarial learning based models often proposed to minimize these distances. However, there is misalignment between the objectives of adversarial learning and the bounds which results in the gap between theoretical findings and practical algorithms.
16

---

Published as a conference paper at ICLR 2023
In particular, it has been shown that the objective of the minimax game between the representation mapping and the discriminator is equivalent to minimizing the JS divergence between representation distributions actoss souree domains (Goodtellow et al. 2014). However, minimizing JS divergence does not guarantee the minimization of common distances used in the existing bounds. The details follows.
H divergence: We show that JS divergence is not the upper bound of H divergence. Con-
sider an example with two distributions P(X) and Q(X) where P(X) w.p 1/3 and P(() = w.p 2/3
[Q(X)= = 0 w.p 1/3 1Q(x= =1 w.p 2/3"
By definition, (x Q) 0.33> Djs(P Q) 0.08.
Total variation distance: We have (P Q) S Drv Q) VP.Q where DJs and Drv JS divergence and total variation distance, respectively. Then, minimizing JS divergence does not guarantee the minimization of total variation distance.
Hellinger distance: We have (P Q) (2d1)2(P,Q) vP,Q where d1/2 / is Hellinger distance and total variation distance, respectively. Then, minimizing JS divergence does not guarantee the minimization of Hellinger distance.
Different from the existing bounds, our bounds are based on JS divergence/distance. Then they align with the adversarial learning approach for domain generalization in general, and with our proposed method FATDM particular.
Advantages of our proposed bounds in domain generalization.
In summary, our proposed bounds has several advantages in terms of the following:
Most existing bounds (Ben-David et al. 2010 Albuquerque et al., 2019) do not relates feature and representation spaces so it is not clear how performance input space is affected by the representations. In contrast, our bounds connect the representation and input spaces; this further guides us to find representations that lead to good performances input space.
Most prior studies adopt H divergence measure the dissimilarity between domains, which is limited to deterministic labeling functions and zero-one loss (Ben-David et al.) 2010 Albuquerque et al. 2019). In contrast, our bound is more general and is applicable to settings where domains specified by stochastic labeling functions and general loss functions.
Distant metrics (i.e., total variation distance, H divergence, Hellinger divergence, etc.) used in existing bounds (Ben-Davis et al. 2010 Albuquerque al. [019 Phung et al. [2021) are hard compute in practice. contrast, our bounds use JS divergence which is aligned with training objective for discriminator in adversarial learning Goodfellow et al. (2014).
Existing bounds for domain generalization only imply the alignment of marginal distribution of feature across source domains (Albuquerque al., 2019 |hung et al. 202T). As shown Thm. 2 methods that learn invariance of marginal distribution have an inherent trade-off and may increase the lower bound expected loss. In contrast, our bounds suggest the alignment of label-conditional distribution of feature across source domains which has been verified to be more effective in empirical studies (Li et al., 2018b c Zhao et al.) 2020 Nguyen et al.) 202T).
Regarding the fairness, our work the first that bounds the unfairness in domain generalization. In particular, our bounds suggest enforcing the invariant constraint of feature distribution given label and sensitive attribute across source domains to transfer fairness to the unseen target domain.
B DETAILS OF ALGORIITM FATDM
FATDM consists of density mapping functions m and m Vy e V. A,i,j j E [N] (learned by two DensityMate ch models), feature mapping function (ResNet18 model), and the clas- sifier h. In our study, we experiment with two different DensityMaton architectures: Star- GAN (i.e., in FATDM-StarGAN and CycleGAN (in FATDM-CycleGAN), We show the details of PATDM-SStaGaL below. For FATDM-CycleGAl the only difference is we used CycleGAN as bensityMate ch instead of StarGAN. The details of CycleGAN were presented in the original paper (Zhu al. 2017)
17

---

Published as a conference paper at ICLR 2023
For FATDM- each DensityMatchi (or DerDeittytaten X.A) consists of generator X x [N] x [N] + X' and a discriminator X - [N] [0,1}. The generator takes in real image z and pair of domain labels i.j imput and generates fake image; the discriminator aims to predict the domain label of the image generated by the generator and distinguish whether is fake real. G and D are learned simultaneously by solving the following optimizations:
Discriminator's objective: min D L
ade
Aota (reral)
Generator's objective: min G
C CStarGAK
ude
+ Ants clsakes
CStarGAN
+ AreaCrec
StarGAN
where Ca is the adversarial loss, cle(fake) Cetarenn are domain classification loss with respect
to fake and real images respectively, CSterGAN is the reconstruction loss. The specific formulations of these loss functions are in Choi al. (2018). Xcts and Arec are hyper-parameters that control the relative importance of domain classification and reconstruction losses, respectively, compared to the adversarial loss.
In our experiments, input images are resized to (256, 256) and normalized into the range [-1, 1]. The dimension of representation space Z is set to 512. (hyper-parameter that controls accuracy-fairest trade-off) varies from to 10 with step sizes 0.0002 for w E [o, 0.002]. 0.002 for w E [0.002, 0.1 trade-off) is set
and 0.2 for [0.2, 10), and y (hyper-parameter that controls accuracy-invariane to 0.1 (after hyper-parameter tuning). Models (FATDM and baselines) are implemented by PyTocch multiple computer nodes (each model instance is trained single node which has 4 CPUs, 8GB of memory, and a single GPU (P100 or V100)). One domain's s data is used for testing and the other domains' data used for training (10% of training data is used for validation). Each model is trained with 10 epoches and the results are from the epoch with best performance on the validation set. Figure [ visualizes the two-stage training process of FATDM-Sta rGAN. The detailed architectures of FATDM-St.n StarGAN are shown in Tables] 25 We have also provided all code for these models supplemental material.
library version 1.11 and is trained
18

---

Published as a conference paper at ICLR 2023
Stage 1
langet
Sumple
Target Domain Label
Source Domein abei
Fake image
D
V. Dia. A
Sample dill
A
Raconstructad Image
G²
Real Image
Source Domai n Label
Targei Domain Label
Balch of images with y,
Image
image
Image
Source
Target
Source
Target
Sample data for training
Discriminator Training
Generator Training
Stage 2
Lino
Image
Represerbttoo
Eonsittve Ara buue
Rtaal image
Target Domai Lubel (Ger craked)
Clacalliariion Latell
Source [Joma Laber
Sturrar Domsin Label
Domain Labe
Real Image
Hent Regresentation
images
Image
Fate Represstraaion
Stemetivee
Sample data training
Stage 2 Training
Figure 6: Two-stage training of FATDM-StarGAN, For stage we only show the training process for MensityMatch! Y.A (training process for DensityMatchi ch is similar.)
Table 2: Architecture of StarGAN generators GY and GY.A Density mapping functions m and maaa Vy y1 A, i [N]. This architecture similar the one in the original paper Choi et al. 2018) except for the first convolution layer where number of input channels is (for grayscale images) and input shape is (h, w, + 2nc).. (h, w) is the size of input images, IN is instance batchnorm, and ReLU is Rectified Linear Unit. N: number of output channels, K: kernel size, S: stride szie, padding size are convolution and deconvolution layers' hyper-parameetes.
Part
Input Output Shape
Layer Information
(h, + 2m.) (h,, 10,6666 64)
CONV-(N64, K7x7, $1, P3), IN, ReL.U
Down-sampling
(h, w. 64) + (b) N. 128)
CONV-(N128, K4x4, 82, PT), IN, ReLU
(h. w 2 128)
+ (+) 256)
CONV-(N256, K4x4, $2 PI), IN, ReLU
(4) 256)
(4. 256)
Residual Block: CONV-(N256, K3x3, S1, PI), IN, ReLU
(), (,225)
+ (4, 4, 256)
Residual Block: CONV-(N256, K3x3, S1, P1), IN, ReLU
(), ( 2.66)
+ + 1 256)
Residual Block: CONV-NN266, K3x3, S1, P1), IN, ReLU
Bottleneck
(), (.56)
+ (#) 7. (52)
Residual Block: CONV-(N256, K3x3, S1, PI). IN, ReLU
( w 256)
+ ($) 7. 256)
Residual Block: CONV-(N256, K3x3, P1). IN, ReLU
(4. 4. 256)
- +4 4. (56)
Residual Block: CONV-NN266, K3x3, S1, P1), IN, ReLU
[4, w. 256)
(b. 2 128)
DECONV-NN18, K4x4, S2, P1), IN, ReLU
Up- Sa-sampling
(b. 5. 128)
(l,, w.....6 64)
DECONV-(N64, K4x4, $2, P1), IN, ReLU CONV-N3, K7X7, P3), IN. ReLU
(A, w. 64) + (4, w.
19

---

Published as a conference paper at ICLR 2023
Table 3: Architecture of StarGAN discriminators. This architecture is similar to the one in the original paper Choi et al. (2018) except for the first convolution layer where number of input channels is I (for grayscale images). (h, w) is the size of input images, Rdd is the number of domains, and Leaky ReLU is Leaky Rectified Linear Unit. N: number of output channels, K: kernel size, S: stride szie, P: padding size are convolution layers' hypperpaarameters
Layer
Input -> Output Shape
Layer Information
Input Layer
(h, w. 1) + (2 3 64)
CONV-(N64, K4x4, S2, Pl), Leaky ReLU
Hidden Layer
(3. 1. 64) 6.4)
- (#. 7 128)
CONV-(N128, K4x4, S2, P1), Leaky ReLU
Hidden Layer
(). 1, 128)
+ (š w
8: 256)
CONV-(N256, K4x4, S2, P1), Leaky ReLU
Hidden Layer
(8) 8 256)
+ (h 16: 512)
CONV-(N512, K4x4, S2, P1), Leaky ReLU
Hidden Layer
A w
512)
+ (32 32 1024)
h
CONV-(N1024, K4x4, S2, P1), Leaky ReLU
16 16
Hidden Layer
37 52 1024)
(A 6 2048)
CONV-(N2048, K4x4, S2, Pt), Leaky ReLU
Output Layer aDrcc)
(64 61 2048)
+ (6) 64 1)
CONV-(NI, K3x3, S1, PI) CONV-(N(ma). K 04 61 S1, PO)
Output Layer (Das)
64 61. 2048)
A
- (1, n.)
Table 4: Architecture of feature mapping g. This architecture is similar to ResNet18 model He et al. (2016) except for the first convolution layer where number input channels is (for grayscale images) and the last layer where output dimension is dimension of representation space 2. (h, w) the size of input images, BN is batchnorm, MaxPool is max pooling, AvePool average pooling, and ReLU is Rectified Linear Unit. N: number of output channels, K: kernel size, S: stride szie, P: padding size are convolution layers' typee-parameters.
Part
Input + Output Shape
Layer Information
Input
(h, w. 1) + (6) 2 64)
CONV-(N64, K7x7, S2, P3), BN, ReLU, MaxPool
Residual Block: CONV-(N64, K3x3, S1, PI), BN, ReLU,
(b) 61) + (1- T. 6-1)
CONV-(N64, K3x3, $1, P1), BN
Residual Block: CONV-NN128, K3x3, S1, PI), BN, ReLU,
(h) 1 64)
(h 128)
CONV-NN128, K3x3, S1, Pl), BN
Bottleneck
Residual Block: CONV-(N256. K3x3, S1, PI), BN, ReLU,
($ w 128)
-
T6
A
16:
256)
CONV-(N256, K3x3, S1, PI), BN
Residual Block: CONV-(N512, K3x3, S1, P1), IN, ReLU, CONV-N5112, K3x3, S1, PI), BN, AvgPool
16 16: 256)
A
+ (1, 1, 512)
Output
(1,1,512)
+ n
LINEAR-(512,, m.)
Table 5: Architecture of classifier h. n, is the dimension of representation space Z.
Layer
Input Output Shape
Layer Information
Hidden Layer
n2 >
R2
LINEAR- (n2, n:), ReLU
Hidden Layer
4
LINEAR 2 *4), ReLU
Output Layer
a. +
LINEAR- (4, 1), Sigmoid
20

---

Published as a conference paper at ICLR 2023
C ADDITIONAL EXPERIMENTS
Experimental results with all unfairness and error metries. In this section, we provide more experimental results about fairesss and accuracy under domain generalization. In particular, we investigate fairness-accuree
trade-off on the two clinical image datasets including Cardiomegaly and Edema diseases with respect to different fairness criteria (i.c., Equalized Odds. Equal Opportunity], and unfairness (i.e., MD and EMD) and error (i.e., CE, MR, AUROC, AUPR, F1) measures. Figure (Cardiomegaly disease- Equalized Odds), Figure [ (Cardiomegaly disease Equal Opportunity). (Edema disease- Equalized Odds), and Figure I0 (Edema disease- Equal Opportunity)
Figure 9 show the unfairness-erve curves of our models as well as baselines for these two datasets. As we can see, our model outperforms other baselines in terms of fairness-accuracy trade-off. The curve of our model is the bottom- lettom-leftmost compared to other baselines in all measures showing the clear benefit of (1) enforcing conditional invariant constraints for accuracy and faimesss transfer and (2) using the two-stage training process stabilize training compared to adversarial learning approach. We also quantify our observations by calculating the areas under these unfairness-errov curves, in which the smaller area indicates the better aecuracy-fairness trade-off. As shown in Tables and 7 our model has the smallest areas under the curve and achieves significantly better fairness-accures trade-off for both equalized odd and equal opportunity compared to other methods.
Impact of the number of source domains. Our work focuses on transferring fairness and accuracy under domain generalization when the target domain data are inaccessible during training. Instead, it relies on set of source domains to generalize to unseen, novel target domain. We investigate the relationship between the fairness-accuracy trade-off on the target domain and the number of source domains during training. In particular, we evaluate the performances of FATDM and ERM on Edema dataset with different numbers of source domains. Similar to the previous experiment, we first construct the dataset for each domain by rotating images with @ degree, where 0 E {00, 15 33%) when the number of domain is 3, 0 E {0?, 15 1,0 -45) when the number of domain is 4, and 0 E f0², 15", 30 455 60%) when the number of domain is 5. The number of images per domain is adapted to ensure the training set size is fixed for the three cases. We follow the leave-one-out domain setting in which one domain serves as the unseen target domain for evaluation while the rest domains are for training; the average results across target domains are reported.
Figure T shows error- unfor-unfarrnes curves of FAIDM and ERM when training with 3. and source domains. We observe that training with more source domains does not always help the model achieve better fairness-accuran
trade-off on unseen target domains. In particular, the performances of both FATDM and ERM are the best when training with 2 source domains and the worst when training with 3 source domains. We conjecture the reason that adding more source domains may help reduce the discrepancy between source and target domains (term (ii) in Thm. and Thm. 3). but it may make it more difficult to minimize the source error and unfairness (term (1) Thm. [nd Thm. (3 and to learn invariant representation across the source domains (term (iii) in Thm. [ and Thm. B) Thus, our suggestion in practice is to conduct an ablation study to find the optimal number of source domains.
Simultaneous and sequential training comparison. In all experiments we conducted so far, the fairness constraint Lfair is optimized simultaneously with the prediction error Lace and the domain-invarian constraint Line for all methods. To investigate whether FATDM still attains better accuracy-fain -firnesss trade-off when the processes of invariant representation learning and fair model training are decoupled, we conduct another set of experiments where models (FATDM (i.e.. FATDM-Star GAN) and baselines G2DM, DANN, CDANN) are learned in sequential matter: for each model, we first learn the representation mapping g by optimizing Linv and Laces using the representations generated by the fixed g. we then learn the fair classifier by optimizing Lace and fair- The models trained based on the above procedure are named FATDM-seq, G2 seg. DANN-seq, seq. and CDANN seq; and their corresponding results show that FATDM- seq still attains the best accuracy-fairne fairness trade-off at target domain compared to G2DM- DANN seq, CDANN-Seq, Our method is effective no matter whether ffair and Linv are optimized simultaneously or sequentially.
unfaimesss curves are shown in Figure [2 The
The reason that our method consistently outperforms the baselines for both settings is that the invaariattreprrssntttioon learning in baseline methods only guarantees the transfer of accuracy but not faimesss. Even though fairness regularizer is imposed ensure the model is fair at source
21

---

Published as a conference paper at ICLR 2023
domains (no matter whether invariant representations and fair classifier trained simultaneously sequentially), this fairness cannot be preserved at the target domain due to the potential distributional shifts. The key to ensuring the transfer of fairness is to learn representations such that P(Z AA) is domain-inye nonain-invariant; this must be done during the representation learning process. From Thm 3 we can see that unfairness at target domain el Dr can still blow up if |2 (x,4 different across domains, regardless of how fair the model is at source domains (i.e., small D
194 C164 CHAN
CDANN
-B- CORN)
-6- AIDV
PLTIM CHNCON RITIU SEHONS
Unfairness (EMD)
Unfairness (MD)
0c
Error AUROC
Error AUROC
Unfairness (EMD)
Unfairness (MD)
Error AUPR
Error AUPR
Unfairness (EMD)
Unfairness (MD)
02
40 Error (CE)
Error (CE)
Unfaireess (EMD)
Unfairness (MD)
0.-
8G
Error (MR)
Error (MR)
Unfairness (EMD)
Unfairness (MD).
05-
Error
Error F
Figure 7: Error-unfairness curves with respect to equalized odds of FATDM and baselines on Car- diomegaly disease dataset.
22

---

Published as a conference paper at ICLR 2023
ENTERAOOMOGA
Unfairness (EMD)
Unfairness (MD)
AUROC
AUROC
Unfairness (EMD)
Unfairness (MD)
Crror AUPR
Crror AUPR
Unfairness (EMD)
Unfairness (MD)
Error (CE)
Error (CE)
Unfairness (EMD)
Unfairness (MD)
Error (MR)
Error (MR)
Unfairness (EMD)
Unfairness (MD)
Error
Figure Error-unfairness curves with respect to equal opportunity of FATDM and baselines on Cardiomegaly discase dataset.
23

---

Published as a conference paper at ICLR 2023
BPN
120M
DVW
OPPN
-#- GruaEin
CORVL
--- ENTOM Star
Unfairness (MD)
Unfairness (EMD)
0.1
0.4
Error AUROC
Error AUROC
Unfairness (EMD)
Unfairness (MD)
0.1
131
Error AUPR
Error AUPR
Unfairness (EMD)
Unfairness (MD)
Error (CE)
Error (CE)
Unfairness (EMD)
Unfairness (MD)
Error (MR)
Error (MR)
Unfairness (MD)
Unfairness (EMD)
Error (F)
Error (F1)
Figure 9: Error-unfaireess curves with respect to equalized odds of FATDM and baselines on Edema
disease dataset.
24

---

Published as a conference paper at ICLR 2023
BPN
120M
DVW
OPMi
-#- Gruatio
CORVL
--- INTDM Sta
Unfairness (MD)
Unfairness (EMD)
8.1
04-
Error AUROC
Error AUROC
Unfairness (EMD)
Unfairness (MD)
0.4-
0.1
11
131
Error AUPR
Error AUPR
Unfairness (EMD)
Unfairness (MD)
Error (CE)
Error (CE)
Unfairness (EMD)
Unfairness (MD)
Error (MR)
Error (MR)
Unfairness (MD)
Unfairness (EMD)
Error (F)
Error
(F))
Figure 10: Error-unfairres curves with respect to equal opportunity of FATDM and baselines on
Edema disease dataset.
25

---

Published as a conference paper at ICLR 2023
Table 6: Area under the error-unfairness curves (Cardiomegalyy disease dataset).
Method
Error Unfairness
ERM
G2DM
DANN
CDANN
CORA
GroupDRO
IRM
FATDM
AUROC MD
0.5575
0.6093
0.7571
0.7224
0.7239
0.7039
0.6784
0.0935
AUPRC- MD
0.5463
0.6301
0.7730
0.6883
0.7300
0.7152
0.6967
0.0291
CE MD
0.2861
0.2601
0,4622
0.4232
0.4424
0.3148
0.3370
0.2152
Equalized Odds
MR- MD
0.6312
0.4906
0.6795
0.6667
0.6683
0.6382
0.5721
0.2439
F--MD MD
0.5901
0,4150
0,6507
0.6547
0.5745
0,5360
0.5025
0.3365
AUROC FMD)
0.7326
0.7106
0.8342
0.7931
0.8075
0.7845
0.7991
0.1099
AUPRC- EMD
0.6901
0.7146
0.8308
0.7577
0.7918
0.7806
0.7945
0.0437
CH-HMD
0.5158
0,4443
0.6143
0.5788
0.5873
0.4911
0.5274
0.3384
MR EMD
0.7056
0.5795
0.7137
0.6979
0.6902
0.6571
0.6483
0.2045
P- EMD
EMD
0,6866
0.5328
0.7279
0.7019
0.6515
0.6027
0.6120
0.2888
AUROC MD
0.5128
0.6001
0.6999
0.6686
0.5935
0.6288
0.5910
0.0750
AUPRC- MD
0.5419
0.6718
0.7086
0.7189
0.6423
0.6761
0.6435
0.0262
CE-MD
0.3690
0.4272
0.5094
0.4492
0.3780
0.2737
0.3582
0.2754
Equal Opportunity
MR MD
0.3203
0.5068
0.5252
0.5512
0.4897
0.4368
0.4173
0.1778
F MD
0.2134
0.4570
0.4608
0.5207
0.4017
0.3561
0,3510
0.2737
AUROC EMD
0.6119
0.7184
0.7649
0.7517
0.6720
0.7068
0.6780
0.0947
AUPRC- EMD
0.6321
0.7684
0.7718
0.7877
0.6912
0.7335
0.7200
0.0448
CE EMD
0.5092
0.6093
0.6264
0.6141
0.4737
0.4340
0.4917
0.3070
MR EMD
0.4619
0.6420
0.6325
0.6532
0.5790
0.5515
0.5298
0.1918
F EMD
0.3876
0.6122
0.5942
0.6496
0.5101
0.4898
0.4889
0.3108
Table 7: Area under the error-unfairness curves (Edema disease dataset).
Method
Eroo Unfairness
ERM
G2DM
DANN
CDANN
CORAL
GroupDRO
FATDM
AUROC- MD
0.3395
0.2765
0.2972
0.2548
0.3642
0.3627
0.0633
AUPRC- MD
0.2865
0.2446
0.2561
0.2304
0,3052
0.2998
0.0771
CE MD
0.1096
0.1266
0.1243
0.1
0.1269
0.1179
0.0341
Equalized Odds
MR MD
0,3929
0.3525
0.3509
0.3302
0.4303
0.4240
0.0656
Fí MD
0.4213
0.3219
0.3527
0.4178
0.4283
0.4189
0.1369
AUROC- EMD
0.4277
0.3813
0.3637
0.3419
0.4419
0,4394
0.2729
AUPRC- EMD
0.3868
0.3588
0.3285
0.3245
0,3958
0.3921
0.3041
CE EMD
0.2366
0.2401
0.2348
0.2334
0.2447
0.2339
0.1827
MR MD
0.4592
0.4435
0.4017
0.3904
0.4942
0.4792
0.2802
F MD
0.5186
0.4642
0.4132
0.4827
0,5180
0.5029
0.3855
AUROC MD
0.2488
0.2139
0.2085
0.1806
0.2696
0.2625
0.0218
AUPRC- MD
0.2606
0.2381
0.2297
0.2035
0,2937
0.2874
0.0168
CE MD
0.1540
0.1839
0.1689
0.1572
0.1487
0.1446
0.0234
MR MD
0.2967
0.2652
0.2620
0.2516
0.3101
0.2999
0.0468
Equal Opportunity
F MD
0.2848
0.2195
0.2534
0.2613
0.2973
0.2975
0,0502
AUROC- EMD
0.2736
0.2472
0.2449
0.2155
0.2897
0.2841
0.1121
AUPRC- EMD
0.2653
0,2451
0.2429
0.2176
0,2852
0,2812
0.0912
CE EMD
0.2083
0.2318
0.2355
0.2147
0.2055
0.2003
0.1159
MR MD
0.3409
0.3162
0.3258
0.3026
0.3442
0.3388
0.1872
Fi MD
0.3237
0.2756
0.3031
0.3008
0.3215
0.3271
0.1779
26

---

Published as a conference paper at ICLR 2023
ERV-3
FATDM-2
FATDM-33
FATOM-4
Unfairness (MD)
Unfairness (EMD)
10
42
Error AUROC
Error AURÓC
Unfairness (MD)
Unfairness (EMD)
00
04
0.0
91
Error AUPR
Error AUPR
Unfairness (EMD)
Unfairness (MD)
06
02
644
6.0
93
Error (CE)
Error (CE)
Unfairness (EMD)
Unfairness (MD)
00
02
60
93
611
Error (MR)
Error (MR)
Unfairness (EMD)
Unfairness (MD)
06
60
04
Error F1)
Error
(F1)
Figure 11: Error-unfaimest curves with respect equalized odds of FATDM and ERM Edema disease dataset when training with different numbers of source domains. Names in the figure legend in the form of X-Y where X the model and Y is the number of source domains (e.g., ERM-2 means training ERM on two source domains.)
27

---

Published as a conference paper at ICLR 2023
Uniairmss (MD)
Un Unlarmas (MD)
Unfoirness (EMD)
Undairnes: (ESM12)
Emrr AUROC
Error (MR)
Error AUROC
Eror (MR)
(a) Equalized Odds
Dott
KITTM ENTEM
Unlaimus (MD)
Unlaimes (MD)
Sommmnn (END)
AUROC
(MR)
Error AUROC
(MR)
(b) Equal Opportunity
Figure 12: Fairness-accuracy trade-off (Pareto frontier) of models trained with simultaneous and sequential (i.e., models with '-seq' suffix) approaches, and FAIDM- CycleGAN (i.e., use CycleGAN instead StarGAN as density mapping functions) on Cardiomegaly disease dataset: error-unfairness curves are constructed by varying E [0, 10) and the values of error and unfairness are normalized to [0, 1]. Lower-left points and the smaller area under the curve indicate the model has a better faimess--accraccy trade-off (Pareto optimality).
D ADDITIONAL RESULTS & LEMMAS
D.I TIGHTER UPPER BOUND FOR ACCURACY
Corollary 5.1 We can replace term (ii) in Thm. 7 with the following term to attain tighter upper bound for accuracy:
V2C min sE[M]
d.s (p)r, P3$)
+ 2 Byrr dus (PDI P xix 27
)
Drv Pb,
where TTV
sup PN #
I is called Dobrushin's coefficient Polyynskiy Wi
Drv PX PN,)
2017
This result suggests that we can further optimize term (ii) in Thm. I by minimizing 'rv. It has been shown in (Shui mapping X' + Z when follows Gaussian distribution. The Lipschitz constant of g, in turn, can be upper bounded by the Frobenius norm of Jacobian matrix with respect to g (Miyato et al. 2018). However, in practice, we found that computing Jacobian matrix of is computationally expensive when dimension of representation Z is large, and optimizing it together with invariant constraints does not improve the performances of models in our experiments.
(2022) that 7rV can be controlled by Lipschitz constant of the feature
D.2 LEMMAS FOR PROVING THEOREM 1
Lemma Let X be the random variable in domains D; and Dj. and be an event that P 2 PX then we have:
-18 Px | [|Pt,
| /
Pp
dx
P dX
2
where E is the complement of event E.
28

---

Published as a conference paper at ICLR 2023
Let X be the random variable in domains D; and D,. let f :X + R4 be non-negative function bounded by C, then we have:
Lemma 7
C V2V min (DEL (PB. l1 Pp,)) DKL (P)) Pb.))
- -
where D | is the KL-divvegencee between distributions.
Lemma Suppose loss function is upper bounded by C and consider a classifier f X + y. the expected classification error of J in domain D5 can be upper bounded by its error in domain Dj:
Acc (i) DD
A00 ED.
(f)+ V2CdJs (Po) .P
where X, Y random variables denoting feature and label in domains D; and Dj.
Lemma 9 Consider two distributions P and PD over X. Lel PR, and Pß, be the induced distributions over Z by mapping function X + Z, then we have:
dis(PE D. Pθ,) 2 P,, PB,)
Lemma 10 (Phung al. 2021 Consider domain D with joint distribution P.XYY and labeling function fD X y from feature space label space. Given mapping function g X Z from feature to representation space, we define labeling function hD Z - y from representation
space label space hn(Z)) fo(X)y g-g (2) 5--1 fn(X)y
Similarly, let
Ju-1(z; Pg
f be the hypothesis from feature space, then the corresponding hypothesis h from representation
space under the mapping function g is computed as h(Z)) = J F
Let e2 (f)
1,-1(2) PB
Ep [ff( and (h) = ED (Z), Y)] be expected errors defined with respect to feature space and representation space, respectively. We have:
Acc
1 = Acc
D.3 LEMMAS FOR PROVING COROLLARY
Lemma 11 Consider two random variables X,Y Let X.Y.Y P.Y. be two joint distributions defined D. in domains D; and Dj. respectively. Then, JS-divergence DJs (Px. || P) and KL-divergence
DKL. (pX.y || Pb;Y) can be decomposed follows:
DKL PXXY P DRL PX PK,) ED, DKL (Px (PXM))
DJs XXY (P))) < DJs
(P%, PY.) + EED (DJs (pxl p
+ En, DJs (X || ||
D.4 LEMMAS FOR PROVING THEOREM 2
Lemma 12 Under Assumption in Theorem
the following holds for any domain D:
Vetc=(F) = -VD
2c / (P))
vf
where & is the prediction made by randomized predictor f.
29

---

Published as a conference paper at ICLR 2023
D.5 LEMMAS FOR PROVING THEOREM 3
Definition 13 Given domain Dj with binary random variable A denoting the sensitive attribute, the unfairness measures that evaluate the violation of equalized odd (EO) and equal opportunity (EP)
criteria between sensitive groups of this domain are defined as follows.
(
RD: (1) RD: (f)
+ RD: 0 (1) -Rb. (f)
RD
CD
where D (T) =
[(X)) -nd En.
a)
13 Rb (f)
Lemma 14 Given two domains Dj and Dj- under Definition as follows.
can be bounded by
D.D.
R' (i) S R. (i) + V2djs
(DD)
(XXIEvy.Aza pxII=EAA=a
Vy,a E [0
Lemma 15 Given two domains Dr and Dj. under Definition 13 the unfairness in domain Dj can be upper bounded by the unfairness measure in domain D; as Jollows.
80 D
S
2D D
( + v
Σ dis (PTs Y=M.A=a , pXIY=viA=))
Σ
a==.1
EF S
ED
+ V2
Σ
dJs
(pxiY=l,Aza
(PD)
0=1,1
Lemma 16 Consider domain D with distribution PT and labeling function fD X + YA, Given
mapping function g : X Z from feature representation space, we define labeling function
hp 2 + ydd from representation space to label space as hn(Z)y fo(X)y g- (Z) =
l (--1(2) fD(X) ax
Similarly, lei ] be the hypothesis from feature space, then the corresponding
1,-1 P2
hypothesis h from representation space under the mapping function g is computed as h(Z)y
(--1(2) P( ax
Under Definition 13 we have:
1g-1(z) P7
h
tD
C
h
D.6 LEMMAS FOR PROVING THEOREM
Lemma 17 Consider two domains D, and Dj, if there exist invertible mappings m and m such
that D. = =P (X
(X)]y and P-Xlu.a m (X))a Vy E V.aa E A, then DJs (p || PD)
PD,
and DJs (p?.y.a v.a || p?].vo) can be upper bounded by S Dsl (p2la || p2)m²? (1)) dx and
{ pelu. DJs Js 1/ p2lmpg((0)) dx, respectively.
D.
E PROOFS
E PROOFS OF THEOREMS
Proof of Theorem First, we get the upper bound based on the representation space Z. Then, relate it with the feature space X. Let D. (D$) be the source domain that's nearest the
30

---

Published as a conference paper at ICLR 2023
target domain D According to Lemma 8 have upper bound of the expected classification error for the target domain based on each of the source domain as follows.
Acp
h
Ace D
h
+
v2CdJs (P P [
Taking average of upper bounds based on all source domains, we have:
N
22
N
Acc CDr
Acc
D
h
+
N
Σ
dis
(Pbr
DJ
P
i=1
i=1
N
20
N
V2C N
ACE $
h
+
dJs
(Prr .PA))
Σ dss
(P
p
N
N
D
1=1
(2)
h
v 2C
mindys KE[N]
(PAY, D + V2C [JE[N]
max dJs
(PDz) .
D
(1)
Here we have S. by using triangle inequality for JS-distance: R) S dos( Q) + dds( R)
(2) with P,Q, and R = Ppr, PDE and PDS. respectively. We have < because D E (DS) then dss (PD-
ph.)) Similarly, we can obtain the upper bound based DS P S max djs (Pbr Ds
[JE[N]
on the feature space X' as follows.
AaC (i) + V20 min (PXY, PXY) + 2C max dJs (PXy px.)) DØ
DD
(E(N]
[JE[M]
(8)
However, the bounds in Eq. 7 and Eq. (8) are based on either feature space or representation space, which is not readily to use for practical algorithmic design because the actual objective is to minimize
AcE (7) in feature space by controlling Z in representation space. According to Lemmas 9 and 10
we can derive the bound that relates feature and representation spaces as follows.
Acc Dr
Ace D
h
Acc 52
h
+
22C min ds NE(N)
(pby, pbp²)
V2C max dAs (JE[N]
(PZ)) p.a))
N
i=1
Acc D
+ 2C
min dss iE[N]
pxx)) (pxy,
V2C max d.s ijE[N
(pZYy p.Z.
, n₂
D.
Proof of Corollary 1.1
p2,Y dss (pZY
V DJs (p2. 1.P..
(1)
V DJs (pyr || |b))) +2E Dg (x)
(pzl DJs
DD || p }
dJs (pZL ) )²]
x s (p)g Pbz) +
2E Dg (2)
Here have by using Lemma 11
to to decompose the JS-divergence of the joint distributions and
(2)
by using inequality Va t bs Va vb.
31

---

Published as a conference paper at ICLR 2023
This new upper bound, combined with Thm. suggests learning representation Z such that p Y is invariant across source domains, or in another word, Z I D Y. This result is consistent with Thm. when the target domain D is the mixture of source domains {Di Ji=1 and when PY and
D
DD
p D
are invariant across source domains, we have dJs (P De dJs (P pZy)
0.
implying ASc
(f) S N
ACc D
f
D. ACE (F) (F)
[N].
Proof of Corollary 5.1 (tighter upper bound for accuracy). The bound in Eq. 9 is constructed
using Lemma Indeed, we can make this bound tighter using the strong data processing inequality for JS- di-eivereeece (Polyanskiy & Wu 2017). as stated below.
DJs PB. PB,) $ nJs (PS) PB.) S TDVDJs (PE)
|| PE.))
where Z is random variable induced from random variable X, and PD. and PD are two distribution
over X, and nas sup Dss (B,,PE p PP, DJs Pž Pb,)
Drv PD P8,
S UTV
sup P P *P Drv P Pb
1, Drv is the
total variation distance. "/V is called the Dobrushin's coefficient PPolynnkkiy & Wu 2017).
Apply Lemma 11 and this inequality to the second term in the right hand side of Eq. 7 (similar to the proof of Corollary
I.I). we have:
20 min dJs 1E[N]
[p2.y P
dJs (Prr, Pb:)) + 2E 2BzwPp)(() dJs (p -pp)) 27 )
I V2C min NE(N]
+
dis PD),, Pbx) + P dis (pxty, P )))
S V20 min (E(N]
(10)
Proof of Theorem 2 Consider a source domain D and target domain D Because JS-distance ( :) is distance metric, we have triangle inequality:
d Dy. prr) S dis(P)) PT:) + d drs(ppp De: Pbx) + ddss( P Pir) D
Since X . R Y, we have dss(p) ppr) I dys( D.S Pbr). Using Lemma 12 the
following holds when ds (P)) Djr pTx) 2 dis(P% s& P.r))
(dis(Ppp. Pbr dss(Pp pb.)pppp)))) PB.))
(dss(Ppp Dy. Ppę d (P). Pbr) )² (dis((pss Ds, Ppg) d (p8 Dr, P
(20s (f) + v / (
2c y
4
(eaog ( + ACG (f)
The last inequality is by AM-GM inequality. Therefore, when d 5 prr) 2 dds(Ppy,
hyg Pbr), we have
b ( + ACR ())
/ (dis( (dy(dis(P Dg, pkr) dss(P) P DØ. pBr)
c
The above holds for any source domain D.S. Average over all N source domains, we have
N
N (dss(P)e, Pbr) dss(Ph pbr) 4
1
Acc
(f) + ASE (f) 2 A
Σ
N
D$
32

---

Published as a conference paper at ICLR 2023
Proof of Theorem
3
The proof is based on Lemmas 15 and and similar to the proof of Thm.
Let D E {D;) N.
J==1 be the source domain nearest to the target domain D According to Lemma 15 have upper bound of the unfairmess measured with respect the representation space for the target domain based on each of the source domain. For equal opportunity (EP), we have:
B2 D
h
V
Σ
dss
1,4ma p Dr
D
Df
ac (0,1)
Taking average of upper bounds based on all source domains, we have:
N
N
EP D
h
N
+
N
Σ
dis
p21==1AA=o -2
Dr
DD
p
i=
aE (0,1)
N
N
<
N
D2
+
N
Σ
dis
p2
Dr
D.
-
im1
aE (0.1)
N
Σ
dis
(p21Y=1A=a
p2 p21==1.
Dj
aE {0,1}
N
D2
D2
+
2 min
Σ
dJs p
12
121y=1,A=a
NN(N]
(0
+ 2 max [JE[[]
Σ d
p
P
-1
DJ
(E
According to Lemmas and [6 we can relate this bound to the feature space as follows.
f
TD
= πT
N 1=1
FP
D
11
+
min iE[M]
Σ
dss
DJ
p p
aE(O,1)
2 max
Σ
drg
p
( A1,Aaa
p²
D
aE(O,1}
BP D
+
min
iE[N]
Σ
d1s
pxIY=l,AAa
DJ
DJ
1=1
aE(0.1)
2 max MJEM
Σ
drg
(2,
DE
p
D
aE(O.1)
Similarly, got the upper bound for unfaimess measure with respect to equalized odds follows.
+ V2 min
Σ
Σ d3s
(px
(pxIY=v.A=a )
Dr
5
D
NE(N]
yE(0.1) aE(0.)
2 max
Σ Σ
dis (pZIY=y,A=
p21
(11)
[JE[N]
gE(0,1) aE(u,1)
Proof of Theorem 4
DE D. we can learn the mapping function g = Po (Z such that p2ly =2 Note that this mapping function always
Consider two source domains, DX and DJ. if PY PP
D.
exists. In particular, the trivial solution for Z that satisfies pl2d -p21 is making Z 2. Y, D (e.g..
D. Dj
33

---

Published as a conference paper at ICLR 2023
Po (ZIX) /V (0, D)) Then we have:
Roc D
g p
vr. DJ
[ [<(h(z),)) r)
Dg
[[((2))))]
E
y
py
21
Dj
[c [T((()))r]
E pr 221
Dg
Dg
Eg ps w 1g [ (T()))))
Acc DJ
For unseen target domain D² in A, we have:
Ace D
h
E Dr c (b)()))))
C (T()))))
pyz dz
2xY
N
/
L
(T((2),)) Y)
Σ
D... dYdz dz D.
2xy
i=1
Σ
C (T((Z). r)
P dY
Ti
2xy
Σ T. EDS
[c (c(T(()) (Z)) x)]
D
[c
(Th (2))x)]
[
Acc D
h
vi E [N]
By Lemma 10 we have DD
Ace D2
Acc Dr
Acc D.
For fairness, we only give the proof for equalized odds (EO), we can easily get the similar derivation for equal opportunity. For any Z that satisfies p =g,A=a 2 = Vy, E (0, 1}, we have:
h
E D
I= p P
ppp ,
D3
D
yE 10.1)
Σ D
N-y.AA0
p
D3
v
D
h
For unseen target domain DT in A, we have:
h
Σ
D
|| p
(()) IV (mgyAmm)
D
(E(11
Σ
D
T
TiP D2
Σ
n. p22 =y
vE(U,1}
Σ
D
((2) Y ( DJ
WEIN
D
yE (0.1)
E [N]
34

---

Published as a conference paper at ICLR 2023
Similar the proof of accuracy, Z that satisfies P
=y, -2 -p
Vy,aE (0,1),i,1
[N] always exists. The trivial solution for is Z that satísfies Z 1 D.
By Lemma 16 we have DD
h
B0
80 D$
D
D2
For equal opportunity (EP), Z only need to satisfy the condition for positive label, i.e., Pp
p21Y=1,A=0
p2y==I,A Va i, E (N].
Dj
Proof of Theorem According to Lemma
we have:
DJS Pb." P Σ
D (Pb. PD. m ml) dr
(12)
DJs pbyy Pb)
Then,
minimizing
can be achieved by minimizing
DJs 1 p2)m
Va E X. We can upper bound DJs (p x pm)m?
as
follows
DJs (pzle) pllm?i(e))
< Drv (p2lk p2)mi)(e)
I V2 d1/2 (pala plimi
/2 d1/2 (N (u(x); La) .N (n (mm)(x)) ;o!l,)
(13)
where Drv and d1/2 are total variation distance and Hellinger distance between two distributions,
respectively. We have because of our choice for representation mapping g(z) = p²|= = N (u(x); According to Devroye al. (2018), the Hellinger distance between two multivariate normal distributions over Rd has a closed form as follows
d1/2 (N (11;E1), .N (122 E2))
det (2)) 1/4 det (2z) 1/4
E + 2i
exp
(m) -22)²
(Mi + u2)
(14)
det + 1/2
2
where M1, U2, E1, En mean vectors and covariance matrices of the two normal distributions. In Eq. (14,, let #1 = ((x), P2 = μ (m (z)). Σ₁ = Σg = I then we have:
d1/2 (N (u(z); .N (μ (m 0 o241)
exp (-
_do (~sdoz (1(x) (m) (x))) (1 a (mi,
1
Il (x) a (m (z:) (1)
exp
(15)
8do2
see that Helinger distance between two representation distributions p2|= and p2 m (.) the function of their means y (2) and i (x)). Combining this with Eq.
From Eq. 15
12 and Eq. 13 we conclude that minimizing d.s (P7 p/ can be reduced to minimizing [u(z)= (m, (x:) | which can be implemented as the mean square error between μ(xx) and (m²)(x)) in practice. Proof for dıs (p?lue pZlbia is derived the similar way.
35

---

Published as a conference paper at ICLR 2023
E.2
PROOFS OF LEMMAS
Proof of Lemma 6
We have:
/ (P/, -PB.) dx
PD.
PD.
dx
8
08
(P8, - -PX.) dX-
1
dX
EVEE
PX -Pt) dx
58
PD P dx
08
/
PDJ P dx
2
Proof of Lemma
We have:
ED, [f(x)] = J, F(X))) dx -,-/5p/0 1(X)),,XX dx + / F(x) (PE, - -PX.) dx
x
[[D))( + f ((x) (pX,- -P%.) dx
-ED, [Ex]I + - 11x) (PD, - -PB.) dx + / [(x) (P)) - PB..) dx
_ En, [(()] + /1 1(x) (Pt) -PA.) dx
(
- ED. [f(x)) +C| / (P%) -P%.) dx
ED, [[(x]) +C / /PS -P dx
2 ED. [s(X) + C / |8j -P dx
(1) I ED. [5(x)++ + C
min (DKL (PX, P..))) DEL (PY, || )T.))
ED, [f(x)) + v2 V
C
min (DKL (P) ll Pp.) DEL (PX, || PX.))
(1) S
where& E is the event that PT, 2 PA and E the complement of E. We have
because
dx 0: because f(X) is non-negative function and is bounded by (E(()) Pp.))
(3)
C: S by using Lemma divergence.
6 by using Pinsker's inequality between total variation norm and KI--
Proof of Lemma 8
Applying Lemma 7 and replacing X by (X,Y), / by loss function C. D, by
Da. we have:
f ED.,, [(F(x))))] = ED, [()( -ED,, [(F(
D,
< 2 V min (DKL (PAS" pX.)) DEL (PDY |
C
C
DKL (Pb,Y || (PX.))
/2 V
(16)
36

---

Published as a conference paper at ICLR 2023
Applying Lemma 7
again and replacing X by (X,Y), f by loss function L, Dj by Di.j. we have:
ED.,, [(f(()))))
Acc
(f) = ED.,,
[(F(x),)) - ED, [((P(x))))
C
min (DKL (PD. Po..)) DEL (PDi (1pb.t)
V2 V
C
DKL. (Pū, || Ppis)
<
v2 V
(17)
Adding Eq. 116 to Eq.
17 17, we have:
Acc DD
8
Acc DD
(f)
C
V
DKL
(PD. II pDt.)) + V DKL (PD;" Il PD!)
V2
(1)
C
(DKL (Po. PD.))) + DKL (Po,") Py.)))
V2
v2V
V2VADJs
ADJS (PD. || PD)))
V2Cdjs (PD. Pp)))
Here we have S by using Cauchy-Schwarzz inequality.
Proof of Lemma 9 Note that the JS-divergence Djs (PX,
Px.) can be understood as the mutual information between a random variable associated with the mixture distribution PD.., Pτ,) and the equiprobable binary random variable T used to switch between PAY, and PD to create the mixture distribution P In particular, we have:
PXX
DJs (P),
|| Pp.))
DKL (PK. Pi...) + +DJs (PY) ll 1
2
/ [log Pit. - llog Pis P.Y. dX
1
/ (log Pi², - -log PA Pi
+ 2
1
/ log (PY.) PAA dx +
1 / log (Pb,))),,3) Py,,x
= 2
2
/ log (PX..) PK.,, dx
-H(XXT) + H(X)
I(X;T)
where H (XX is the entropy of X, H (X is the entropy of X conditioned on T' and [(X;T) is the mutual information between X and T. Similarly, also have ()s((P} (b,)) )) (Z;
Because Z is induced from X by the mapping function h then we have Z 1. T' X and the Markov chain 7 + X + Z. According data processing inequality for mutual information (Polyanskiy &
Wu 2014), we have T(X;T) 2 I(Z; T) which implies ()s((P). PK )) 2 ( l (b,)).
Taking square root both sides, have d PXX PH.) 2 d (PB, PB,).
37

---

Published as a conference paper at ICLR 2023
Proof of Lemma 10
We have:
Acc h = 2-- p/ Ny mD())
( ((((22)))))
[c( v) (2))
D
/ (Ti(Z) 1DD PZd2
Σ
2=1
|>
(0-12) (D((), (x)) dx
/ c (h(2))))
PD dxdZ
19-1(2) PD g-1(Z)
-
D
(T((2)))))
L
Pf 1(2)
DI
Σ
/ /
C ( v)) PFxx),
2 9-12))
1>1
Σ 2 xx
/ ((x PX
dxdzz
131
Σ /
1 1(2 12 (2==(()) fo(X),
x 2
/ (T(6(x)))))
Σ
In(X),
[ (F(())))) fD(X)
Σ x
Acc
We show the decomposition for KL-divergence first and then use the result to derive the decomposition for JS-divergence. We have:
Proof of Lemma 11
DeL
PNY. || (
log XxY. log X.. D,
Ep.
log DD: + log p> x|r
Ep, log PÝ + +lo PX XIV
ED,
ED.
log PY log p
+ ED: log xxl log x|X
ED.
log PY log PT]
B Bewpoii [log pixl log p. X|X (P), || P))) + ED, [DKL (PD. || P
+ Eyynpy
DKL
38

---

Published as a conference paper at ICLR 2023
DJs
p.XY
||
p...))
Di
1
DKL ( || 11 + PKL PXxY || 1
2
1
DKL (P%, || PP...) )) ED. DKL (PD. PoT)))
2
[KL (P%) PD. ))
[B, [KKL (PD)) P
+ 2
+ 2
DJS (P%, || Pb,) + ED. DKL (POA poTT)) + 2 EDD [KK (PD) || p
2
S DJs (P%, || Pb,
BBD. [DKL (PD.V prtr))) + 1 (ED. [DKL (PO, Pp).)) ))
+ 2 (ED (DKL [ P
)
(ED, [DKL] (Poh"" || | )
-Dss (P), Pb,) ED: Dos
pxxly | )] +ED, [Dis (Pot. || p
Proof of Lemma 12
ED [((()))))
ED
((X)
Dey
Ex
Σ f(X)s Pr(Y + T
DEY
Ex
1 [(x)
C Ex
[| - -f(x) 27
1/]
2
c 1
[117(00) - (0x))1)
Ex
2D
c 1
(1 [||x [F(x) F(X) (1,) )²
2 2
c
|/ - 1311
2
(
2c DJs (P) p$)
2c
(P% P8)
dJs
Here we have 2 is because of the assumption that L(G, v) is lower bounded by c when T + y;
(2) is because f(x) = [1 = 1; 2 is because [1 < [lf(x)) = 1: 2 is because
(3)
(4)
(5) [If(x)]12 2 - [[xx)) 2 is by using Jensen's 's inequality; 2 is by using JS-divergence lower
(6)
bound of total variation distance.
39

---

Published as a conference paper at ICLR 2023
Proof of Lemma 14
Similar to the proof in Lemma 8 we apply Lemma 7 for RD, and RD, and
note that f(X), y is bounded by 1. Then Vy, {0, 1), we have:
D, ED.s f(x) Y = y, A =
EE, [(x))lxx =. = a) - ED... ((X) = =v,A=q
V2 V min (DKL (p:s P Pis p DEL (Piiin) (DINIY p
1
1
/DKL ( || P
(18)
V2
[(x))y = y, = R.
ED.,
= ED,,, ((X)gyY = -y = EED -
1
min (DKL PD. P PD. P DEL (PD..) (PDiY
V2 V
PD;
1
DKL PXXIY=MAAz Priy XIY=
V2 V
(19)
Adding Eq.
(88) to Eq.
we have:
R8.0 DD
D.
VDKL (PD.
P p (DX
DDL (PD) (pxIY=y,Aza PD., px
$ V2
S <vddjs (PD) p
PP...
px
Proof of Lemma 15
We give the proof for unfairness measure w.r.. to equal opportunity first and then use this result to derive the proof for unfairness measure w.r.t. to equalized odd. Without loss of
generality, assign group indices 1, 0 be such that 12/1 (f) > Rb) (f) Then we have:
R1.0 D2
RD f
D2
=
1.0
1
RD
(i)
RD,
ED;
F(x))ly=1,44
Rho
+ ED;
1
= -1,=11
RD
+ RD,
1 f- -1
where 1 vector with all I's. By Lemma 14 we have:
RD f < Rb. (f) + 2d.5 (pxI==I .p. (IA==
Rb)
11 1 < Rb (1- 7) + v2d (pX p =1
Sum above two inequalities and add 1 at both sides, we have,
D,
BP
f
f +
RD,
1- (1-)- -)-- -1
R
110
f + RD.
p1.1
1 (1- f) -1+ V2 Σ dss (P)) (PxIY=l,A=o 1an=il _
a=0,1
go D
( + 22 Σ dis (px
(pxIY=I.A=a -x
XXI==IAA=0)
(20)
0=11
40

---

Published as a conference paper at ICLR 2023
Similarly, have:
RDg (f) BD, (f) -
RD
(
+v2
+
v2 Σ
(P;;
,.
2=0,1
(21)
Sum both Eq. (20) and Eq.
(1
we have:
BO
2
Σ
Σ
dJs
PDD
.PD.
y=0,1
Proof of Lemma 16
Similar to the proof of Lemma
10. Ii (J)
=
D.
h
Vy, a E 10,
Then, we have:
R
000
f
RD.
00.1
f
+
R
RD,
RD0
h
R
7
R
h
Rb!
h1
2D
DD
f
R1.0
RD!
f
DD
R1.0
h
h
D.
Proof of Lemma 17
Vy y, we have:
DJs
p2
(1) DJs
p palppeldd / (oll prit) (o) 'dm (tt)
p p do { iiilzd (o) prit) (1)) vdx
DJs
p2 pallpptied f plimil (c) puludd
DJs
(4)
Dr (p2lz p2lm))(t)) dx
11) because of law of total probability and Z 1 Y because m is invertible
Here we have
function: is because pillv = pm (a)ll Vz E X: S is because of joint complexity of JS
(4)
divergence. By similar derivation, Vy E &..EE E we have:
DJs (p??um p s ( pelvia (p2lz p2)m))()) dz
41


Reviews:
[{"review_id": "cB4Xs5R_ZV", "reviewer": "Reviewer_PEwK", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper considers the representation learning problem in domain generalization setting with fairness constraints (DP, EO). The paper presents upper and lower bounds for accuracy, and also an upper bound for fairness violation. Empirically, the paper aims to solve a minmax problem to account for fairness when learning invariant representations.", "summary_of_the_review": "As presented in the paper, the bounds for accuracy when there are domain shifts have been discussed extensively. Fairness bounds are actually also discussed in some recent literature (as I mentioned in \"Strength and Weakness\"). I think largely the contribution of the paper comes from the empirical invariant representation learning algorithm. Additional clarifications of the connection between theoretical and empirical results would be very helpful.\n\n=== Post-rebuttal ===\n\nThank authors for the additional discussions and clarifications. I have updated my score accordingly.", "strength_and_weaknesses": "## Strength\n\nThe paper takes effort to make theoretical results mathematically rigorous and provides multiple bounds w.r.t. accuracy and fairness. An empirical two-stage algorithm is also presented for the purpose of learning invariant representation.\n\n## Weakness\n\n### 1. w.r.t. dense notations and clarity of presentation\n\nOverall, while I appreciate the effort of presenting rigorous arguments, I find the paper hard to follow from time to time because of the dense notations and (occasionally unnecessarily) complex presentation. For example, in Section 4, the overall optimizing objective, the description of the proposed two-stage algorithm, the calculation of density of learnt representation are not presented in a clearly organized way. Personally, I would benefit a lot from, e.g., the numbering of the key equation (e.g., the objective), the implication and/or importance of the theoretical results, and the correspondence between the specific step in algorithm to previous theoretical results. As another example, the sufficient conditions provided in Theorem 4 seem to be rather straightforward, I think a concise summary would also serve the purpose without sacrificing mathematical rigor.\n\n### 2. the relation between the proposed FATDM algorithm and previous theoretical results\n\nWhile I appreciate the mathematical rigor when deriving upper-/lower- bounds for accuracy and upper bound for fairness. I am having some difficulty drawing connection between the theoretical results and the empirical two-stage algorithm. Does the theoretical characterization provides the ground for framing the problem in terms of invariant representation learning? Or, does the theoretical analysis motivate the design of algorithm? After all, as authors already noted, in the domain generalization setting, we do not have access to target domain, in this case, I am wondering if authors can share some insight regarding how to parse the theoretical results in the evaluation of empirical algorithm.\n\n## Additional comment\n\n### w.r.t. the theoretical analysis on Fairness transferability\n\nThe paper discusses previous approaches in extensive detail, which is clear and informative. Only for the purpose of completeness, I would like to provide a pointer to a previous work that also provided fairness transferability bounds for DP and EO notions: \"Fairness Transferability Subject to Bounded Distribution Shift\" by Chen et al., (2022). I am aware of the fact that the paper is recent (arxiv June 2022, to appear in NeurIPS), but considering that they are considering the same fairness notions in a very similar problem setting -- fairness bounds for DP and EO under domain shift, I want to bring their paper to the attention of authors. I am just a little bit hesitant to agree with the claim that the paper is \"the first work studying domain generalization with fairness consideration\". In fact, their bounds are more general, e.g., they do not require $A$ and $Y$ to be binary for EO when deriving the fairness bound.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "a1P1lW0TjO4", "reviewer": "Reviewer_nacW", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper aims to develop fair and accurate models in unseen environments. The use of multiple environments and invariant representation learning for training fair models is an interesting approach. The paper also provides a theoretical analysis on the efficiency of invariant representation learning in transferring fair and accurate models across domains while using multiple source domains. The empirical results support the theoretical claims. \n\n \n", "summary_of_the_review": "The paper tries to solve the important issue of achieving fair and accurate predictions in unseen environments. As the paper gives several insights into their theoretical discussion, I vote for accepting this paper.", "strength_and_weaknesses": "Strength And Weaknesses:\n[Strength]\n\nS1: The paper tries to demonstrate how to achieve fair and accurate predictions in unseen environments without relying on assumptions about a specific type of distribution shifts across domains, which is an important and practical issue.\n\nS2: The theoretical analyses in the paper give several nice insights. For example, I enjoy reading Sections 3 and 4.\n\nS3: The proposed algorithm is effective in specific empirical scenarios, as shown in their theoretical results.\n\n[Weakness]\n\nAdding discussion about how the tradeoff between accuracy and fairness is affected by the number of source domains would be helpful. A major issue in certain settings could be acquiring multiple domain data, in which case one is restricted to a single domain. It would be helpful to address this point. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DqZJlJySiQ", "reviewer": "Reviewer_dQPF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considers the problem of domain generalization with both fairness and accuracy guarantees. To this end, the authors establish an upper bound for accuracy (resp. fairness) for general domain generalization algorithms, which in turn provides guidance on designing their proposed algorithm. A complementary lower bound is also provided in the paper, explaining why previous representation learning methods may not be good choices. The proposed method is tested on real data, showing substantial improvement over previous practices. ", "summary_of_the_review": "In general, this is a well-written paper, with a clear presentation and sufficient discussion on previous works. The theory presented here is novel and interesting; the advantage of the proposed algorithm is validated by numerical experiments.\n", "strength_and_weaknesses": "In general, this paper is well-written. The presentation is clear, references are well-cited and motivation is sufficiently discussed. The upper and lower bounds on accuracy and fairness transferability are interesting because they serve as good guidance for algorithm design. Below I have some comments:\n1. Section 2 is a bit dense with too many notations and concepts, which may be hard for readers not super familiar with the terminologies. It would be better if some of the definitions/terminologies can be introduced within a context.\n2. On page 3, it is clear what $\\mathcal{H}$ and $\\mathcal{H}\\Delta \\mathcal{H}$ are (they seem to be defined in the Appendix). Maybe add a reference here.\n3. On page 4, the footnote mark 2 and 3 should be put after punctuations.\n4. Above Theorem 2, \"bellow\" -> \"below\"\n5. On page 5, maybe add a sentence or two to explain the claim \"Note that these results are consistent with Thm. 1 and Thm. 3 above.\"\n6. I assume a similar lower bound as Thm 2 can be established for EO as well?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster