Unsupervised Adaptation for Fairness under Covariate Shift
Authors: Jatin Chauhan
Shreyas Havaldar
Karthikeyan Shanmugam
Jay Nandy
Aravindan Raghuveer
Venue: Submitted to ICLR 2023 (2023)
Keywords: Out of Distribution
Fairness
Unsupervised
Adaptation

Abstract: Training fair models typically involves optimizing a composite objective accounting for both prediction accuracy and some fairness measure. However, due to a shift in the distribution of the covariates at test time, the learnt fairness tradeoffs may no longer be valid, which we verify experimentally. To address this, we consider an unsupervised adaptation problem of training fair classifiers when only a small set of unlabeled test samples is available along with a large labeled training set. We propose a novel modification to the traditional composite objective by adding a weighted entropy objective on the unlabeled test dataset. This involves a min-max optimization where weights are optimized to mimic the importance weighting ratios followed by classifier optimization. We demonstrate that our weighted entropy objective provides an upper bound on the standard importance sampled training objective common in covariate shift formulations under some mild conditions. Experimentally, we demonstrate that Wasserstein distance based penalty for representation matching across protected sub groups together with the above loss outperforms existing baselines. Our method achieves the best accuracy-equalized odds tradeoff under the covariate shift setup. We find that, for the same accuracy, we get upto 2x improvement in equalized odds on notable benchmarks.

Under review as a conference paper at ICLR 2023
UNSUPERVISED ADAPTATION FOR FAIRNESS UNDER COVARIATE SHIFT
Anonymous authors Paper under double-blind review
ABSTRACT
Training fair models typically involves optimizing a composite objective account- ing for both prediction accuracy and some faimess measure. However, due to shift in the distribution of the covariates test time, the learnt fairness trade- offs may longer be valid, which we verify experimentally. To address this, we consider an unsupervised adaptation problem of training fair classifiers when only small set of unlabeled test samples is available along with large labeled training set. We propose novel modification to the traditional composite objec- tive by adding weighted entropy objective the unlabeled test dataset. This involves a min-max optimization where weights are optimized to mimic the im- portance weighting ratios followed by classifier optimization. We demonstrate that our weighted entropy objective provides an upper bound on the standard im- portance sampled training objective common in covariate shift formulations under some mild conditions. Experimentally. we demonstrate that Wasserstein distance based penalty for representation matching across protected sub groups together with the above loss outperforms existing baselines. Our method achieves the best accuracy-eco eeuuacc-equalized odds tradeoff under the covariate shift setup. We find that, for the same accuracy, we get up to x improvement in equalized odds on notable benchmarks.
INTRODUCTION
Moving away from optimizing only prediction accuracy, there lot of interest in understanding and analyzing Machine Learning model performance along other dimensions like robustness (Silva Najafraad 2020), model generalization (Wiles et al. 2021 and fairness (Oneto & Chiappa 2020) In this work, we focus on the algorithmic fairness aspect. When the prediction of a machine learning classifier is used make important decisions that have societal impact, like in criminal justice, loan approvals, to name a few; how decisions impact different protected groups needs to be taken into account. Datasets used for training could biased the sense that some groups may under- represented, biasing classifier decisions towards the over-represented group or the bias could be in terms of undesirable causal pathways between sensitive attribute and the label in the real world data generating mechanism (Oneto & Chiappal 2020). It has often been observed (Bolukbasi al., 2016). (Buolamwini & Gebru (018) that algorithms that optimize predictive accuracy that are fed pre-existing biases further learn and then propagate the same biases.
While there are various approaches for fair machine learning, a class of methods called in-processing methods have been shown to perform well (Wan al.) 2021). These methods regularize training of fair models typically through a composition of loss objective accounting for a specific fairness mea- sure along with predictive accuracy. Popular fairness measures are based on notions of demographic parity, equal opportunity, predictive rate parity and equalized odds. After regularized training, the model attains a specific fairness- accuraay tradeoff. When the test distribution close or identical to the training distribution, fairness there could be non-trivial distributional shifts due to which tradeoffs achieved train may not hold in the test. For example, Ding et al. 2222l) highlights how classifier's bainess--acuurae tradeoff trained on input samples derived from one state does not extend to predict income in other states for the Adult Income dataset. Similarly, Rezaei et al. (2021); Mandal et al. (2020) demonstrate that the tradeoffs achieved by state of the art faimesss techniques do not generalize to test data under shifts. In figure complement these claims by analyzing the under-performance for Pate-of-the-art
tradeoffs typically hold. However, in practical scenarios,

---

Under review as a conference paper at ICLR 2023
faireess method Adversarial Debiasing (hang et al. 2018). We also see similar drop in perfor- mance under covariate shift in other baselines we consider, which we highlight in our experimental analysis.
In this work, we study covariate shift where the distribution of covariates across training and testing changes, however the optimal label predic- tor conditioned on input remains the same. We address the following question for unsupervised adaptation of training fair classifiers: Under the covariate shift setup, given suffi- cient amount of labeled training sam- ples and only few unlabeled test samples, how can we ensure good faimess-accuracy trade-offs on the test distribution? While this question has not received much attention, some recent works like Rezaei et al.] (2021) have begun to address this problem. Prior works rely on explicit density estimation which then used to adapt to test data. In our work, we focus on avoiding density estimation steps that do not scale well in high dimensions. Here, we propose novel unsupervised adaptation training objective that is theoretically justified. The ob- jective depends on labeled training samples and unlabeled test samples along with standard fairness objective involving representation matching across the groups on the test. We report the results on equalized odds in our experiments and use the related notion of accuracy parity to motivate our algorithmic design with empirical evidence. Our key contributions are listed as follows:
Pooemer Conttar Eait
Bussser Cauation
Figure 1: Both Error (in % left) and Equalized Odds (right) for SOTA faimess method Adversarial Debiasing exhibit strong degradation on increasing the magnitude of covariate shift. Three scenarios corresponding to no shift, intermedi- ate shift and high shift are plotted (details on shift construc- tion are provided in experiments).
1. We show that under a scenario of asymmetric covariate shift, where one group exhibits large co- variate shift while the other does not, accuracy parity degrades despite perfect representation match- ing across protected groups highlighting the need to tackle covariate shift explicitly. (Section 4
We introduce a composite objective for prediction that involves a novel weighted entropy objec- tive the set of unlabeled test samples along with standard ERM objective the labeled training samples for tackling covariate shift. We optimize the weights using min-max optimization: The outer minimization optimizes the classifier with the composite objective, while the inner maximiza- tion finds the appropriate weights for each sample that are related to importance sampling ratios determined implicitly with no density estimation steps. We prove that our composite objective pro- vides an upper bound on the standard importance sampled training objective common covariate shift formulations under some mild conditions. We then combine the above composite objective with a representation matching loss to train fair classifiers. (Section
3. We experiment on four benchmark datasets, including Adult, Arrhythmia, Communities and Drug. We demonstrate that, by incorporating our proposed weighted entropy objective, with the Wasserstein based penalty for representation matching across protected sub-groups, we outper- form existing fairness methods under covariate shifts. In particular, we achieve the best accuracy- equalized odds tradeoff: for the same accuracy, we achieve up to Ri 2x improvement in equalized odds metric. (Section(8
2 RELATED WORK
There have been works studying different types of fairness criterion. Group Fairness metrics have been studied in Hardt et al. (2016b); Kleinberg et al. (2016) while Individ- ual Fairness metrics were studied in [Dorrk et al. (2012); Shariff-Malvajerdi et al. (2019). Causal
Fairness Metrics:
Fairness criterions has been studied in Kilbertus et al. (2017); Kusner al. (2017); Galhotra et al. (022);/Chiappe 2019); Nabi et al.) (2019); Salimi et al. 2019) where causal mechanisms that gener- ate data are leveraged. Our work addresses questions surrounding statistical Group Fairness metrics where we address the effect of covariate shift on fairness- tradeoffs.
Techniques for imposing fairness: Pre- pr-prccessiigg techniques that aim to transform the dataset (Calmon et al.) 2017] (Swersky et al. 2013] Feldman et al.) 2015); Kamiran & Calders 2012) fol- lowed by standard training have been studied. In-processing methods directly modify the learning
2

---

Under review as a conference paper at ICLR 2023
algorithms using techniques, such as, adversarial learning (Madras et al. 2018 8 Zhang al. 2018
(Agarwal et al. 2018: Cotter et al. 2019 Donini et al., 2018 Fish et al. 2016 Zafar et al.
2017
Celis et al. 2019). Post-processing approaches, primanly focus modifying the outcomes of the predictive models in order to make unbiased predictions (Pleiss et al. 2017 Zhao et al. 2017 Hardt
et al. 2016b). [Bellamy al. (2019) provides comprehensive survey containing broad variety of these algorithms. Our method is an in-processing technique and is different from the above methods in that operates with a small unlabeled test set along with a standard labeled training set.
Distribution Shift: Research addressing distribution shift in machine learning is vast and is grow- ing. The general case considers a joint distribution shift between training and testing data (Ben- David et al.) |006)] BBlizer et al.] [2077] Moreno-Torres et al.] (012) resulting in techniques like domain adaptation (Ganin & Lempitsky, 2015), distributionally robust optimization (Sagawa et al. 2019 Duchi & Namkoong 2021) and invariant risk minimization and its variants (Arjovsky et al.,
2019 Krueger et al. |021 | Shi al. (20TT) A survey of various methods and their relative perfor-
mance is discussed by Wiles et al. (222I) We focus on the problem of Covariate Shift where the Conditional Label distribution is invariant while there is a shift in the marginal distribution of the covariates across training and test samples. This classical setup is studied by [Shimodaira (2000);
Sugiyama et al.] (2007b); Gretton et al. (2009). Importance Weighring is one of the prominently used techniques for tackling covariate shifts (Sugiyama et al.) 2007aa] [am al. 2019). However, they are known to have high variance under minor shift scenarios (Cortes et al. (0110).. Recently
methods that emerged the de-facto approaches tackle distribution shifts include popular entropy minimization (Wang et al. (221a)). pseudo-labeling (French et al. 2017 Xie et al. 20200), batch nor- malization adaptation (Schneider et al., 2020 Nado al. 2020). because of their wide applicability and superior performance. Our work provides connection between version of weighted entropy minimization and traditional importance sampling based loss which may be of independent interest.
Fairness under Distribution shift: The work by |Rezaci et al. (2021) is by far the most aligned to ours as they propose a method that is robust to covariate shift while ensuring fairness when unlabeled test data is available. However, this requires the density estimation of training and test distribution that is not efficient at higher dimensions and small number of test samples. In contrast our method avoids density estimation and uses weighted version of entropy minimization that is constrained suitably to reflect importance sampling ratios implicitly. [Mandal et al. (2020) proposed a method for fair classification under the worst-case weighting of the data via iterative procedure, but it is in the agnostic setting where test data is not available. [Singh et al. (202T) studied fairness under shifts through a causal lens but the method requires access to the causal graph, separating sets and other non-trivial data priors. Zhhng et al.] (202T) proposed FARF, an adaptive method for learning in an online setting under fairness constraints, but clearly different from the static shift setting considered in our work. Slack et al. (2020) proposed a MAML based algorithm to learn under fairness constraints, but it requires access to labeled test data. An et al. (2022) propose a consistency regularization technique to ensure fairness under label shifts, while we consider covariate shift.
PROBLEM SETUP
Let X C. Rd be the d dimensional feature space for covariates, A be the space of categorical group attributes and y be the space of class labels. In this work, we consider A = {0, 1} and ) ) (0. Let X X, A E A, Y E Y' be realizations from the space. We consider training dataset D = ((,,, A,, Yd)li E |n]} where every tuple (X,, A,, Y.) E X x A x V. We also have an unlabeled test dataset, DT The training samples (X,, A,, Y, E DS) are sampled i.i.d from distribution p&5 (X. Y,A) while the unlabeled test instances are sampled from P'T (X.A).
X,,, Asli E [m]]. We focus on the setup where m << n.
Let J. X' + [0. be the space of soft prediction models. In this work, we will consider F E F of the form being learnt while h(g())) E [0, 1] provides the soft prediction. Note that we don't consider A as an input to F, as explained in the work of (Zhao 2021). The parameters of F' are denoted as ((P). We denote the class prediction probabilities from F with P(Ý = y|X.), where E {0, 1}.
o g where g(X) E R (for some dimension >> O), is a representation that is
The supervised in nn-disribution training of F is done by minimizing the empirical risk, ER as the proxy for population risk, R Both risk measures are computed using the Cross Entropy (CF) loss
3

---

Under review as a conference paper at ICLR 2023
for classification (correspondingly we use ER T
and R² over the test distribution for F).
Rd -ES (-1og P(o ER
1
Σ
(-log P-log P(Y =Y,==)((.)))
(X.))E
(I)
3.1 COVARIATE SHIFT ASSUMPTION
For our work, we adopt the covariate shift assumption as in Shimodaira (000). Covariate shift as- sumption implies that P. (Y = P (Y|X, A). In other words, shift distribution only affects the joint distribution of covariates and sensitive attribute, i.e. (X, + p (X, A). We note that our setup identical also define and focus on a special case of covariate shift called asymmetric covariate shifl.
recent work of fairness under covariate shift by Rezaei al. (202I). We
Definition (Asymmetric Covariate Shift). Asymmetric covariate shift occurs when distribution of covariates of one group shifts while the other does not, i.e. p (X = 1) + (X = 1) while p" P"(XIA 0) = (X 0) in addition to ps (Y( IP" (Y]
This type of covariate shift occurs when sub-group is over represented (sufficiently capturing all parts of the domain of interest in the training data) while the other sub-group being under rep- resented and observed only in one part of the domain. In the test distribution, covariates of the under-represented group assume a more drastic shift.
3.2 FAIRNESS MEASURE
To quantify fairness, we follow Rezaci et al. (202T) and use Equalized Odds (EOdds), proposed by
Hardt et al. (016 Agodds = >
Eye (0.1) P((Y = 1 = Y, P( = 1|A = 1,
y]1. FOdds requires parity in both true positive rates and false positive rates across the groups. Hardt et al. al. (201 have raised several concerns regarding other widely used fairness metrics, e.g., Demographic Parity (DP) and Equalized Opportunity (EOpp). Therefore,
don't emphasize them in this work. Another way to interpret EOdds is that it requires I(Ý; A\Y) to be small, where I(; 1-) the conditional mutual information measure. Ideally. we are interested in classifier, F that minimizes the objective: R" + (Y; where Ir(-) is the mutual information measure with respect to the test distribution. However, EOdds metric requires the true labels Y from the test distribution. Therefore, we consider optimizing for a related weaker notion, called accuracy parity, ie. A Apar = IP(Y + Y|A = 0) - P(Y + Y|A = 1)}. In information theoretic terms, minimizing accuracy parity entails keeping ly-(Y * A) small. We now state the main goal of this work:
Objective
min RT + AAAApar O(F)
REPRESENTATION MATCHING AND COVARIATE SHIFT
Our objective is to learn a highly accurate classifier on the test distribution while ensuring accuracy parity as 2). Despite the lack of test labels, accuracy parity admits simpler sufficient condition: Train a classifier F - / g(X) by matching representation g(X) across the protected sub groups and learning classifier on top of that representation (Zhao & Gordon 2019). Several variants for representation matching loss have been proposed in the literature for both classification (Jiang et al.) 2020 Wang et al., (021c) and regression (Zhao) 202T Chzhen et al. (020). For implementation ease, we pick Wasserstein-2 metric to impose representation matching. We recall the definition of Wasserstein distance:
Definition 2. Let (M,d) be a metric space and (MM denote the collection of all probability measures on M with finite thh moment. Then the p-th Wasserstein distance between measures μi and both (MM given by: Wy(mv)) (inf., JMx d(z, v) v)) I(p,v), where T T(u, v) denotes the collection of all measures on Mxx M with marginals μ and respectively.
4

---

Under review as a conference paper at ICLR 2023
We minimize the W22 between the representation g(:) of the test samples from both groups. Empiri- cally, our representation matching loss is given by:
Wass (D²) W,(,.D), á A(=)E" (g(X.)
-(x:A::I)eD" 69(X.)
(3)
(X, = E A, 1) E
We arrive at the following objective:
ER T + XLWass
min 0 ((F=ho
(4)
However, we still don't have labeled samples from DJ for realizing the first term. It is natural to op-
timize the following objective: ER" + +&W aeeaaeeD") where the first term leverages labeled training data while the second term matches representation across groups using unlabeled test. We illustrate that under covariate shift, using the test set only for representation matching alone is ineffective. We also provide strong experimental justification to support this claim in section AA... We now quote a result from existing literature that bounds accuracy parity under representation matching.
Theorem (Zhao & Gordon 2019)). Consider any soft classifier F = h o g(X) E [0, and the hard decision rule Y 1 Let the Bayes optimal classifier for group under representation
g(.) be: ( = 8a (X).. Let the Bayes error for group under representation
g(.) be erra- Then we have:
A Apar S Σ erra + ||PT (X()) = 1) - P" (x())A 0)1 + min (in((er 80 (X))
a
Here, [[P()) Q1-1l1 is the total variation distance between measures P and Q
This suggests applying loss for representation matching to enforce accuracy parity as it would drive the purely label independent middle term to zero. However, we argue that, under asymmetric covariate shift (Definition [] accuracy parity is approximately the third term in Theorem I even when the second term set to O.
Representation Matching does not work under Asymmetric Covariate Shift: Consider the co- variate shift scenario given by Definition Suppose one is also able to find a representation 9()) that matches across groups exactly in the test, Lc. 1x (g(+) 1) P" (g() A = 0). Due to the asym- metric covariate shift assumption between train and test, we have Pd" (g( | A
0) = P (g(.){4 1) pa (g(-)] 0). Since there is covariate shift for group = 0, optimal scoring function so (X) remains the same even for the training set, given the representation.
(g(())
Since a classifier is learnt top of representation g, and only training distribution of group under overlaps (completely) with the test, classifier would be trained overwhelmingly with the correct labels for A = in the region where test samples are found. Over the test distribution, the hard decision score function will be approximately so(X) Therefore, the error in the group 0 would be small. While the test error in group will be approximately (-IAA-1=1) (Iso (X) ss (X)]) which matches the third term in Theorem
Therefore, in this setting is essential to use training samples and unlabeled test samples address covariate shift problem for group 1. Samples for group in the training distribution (not just that belong to group = 0) must be emphasized more. This motivates the need for performing unsu- pervised adaptation using unlabeled test samples focusing on accuracy improvement and combining with representation matching.
5 METHOD AND ALGORITHM
Recall that the objective are interested in is 4). One needs proxy for the first term due to lack of labels. From considerations in the previous section, training has to be done in a manner that tackle covariate shift despite using representation matching. Building over the analysis from the previous section, we derive a novel objective in Theorem based on the weighted entropy over instances in D" along with empirical loss over D and show that is upper bound to RT.
5

---

Under review as a conference paper at ICLR 2023
Theorem 2. Suppose that P*(+) and pS)(() are absolutely continuous with respect each other domain X. Let E IR+ be such that p² (Y==))X) distribution p" (X). Then, we can upper bound RJ using R$ along with an unsupervised objective over ps as:
P(Y=y(X) S 6, for E {0, 1 almost surely with respect to
( (x0
Rd 1 R +ex (x)
e
(T)) H(
(5)
where H(Y(x)
E Eyeto,11 -P(Y x|X) (P(Y= y x is the conditional entropy of the label given sample X.
Proof Sketch. The proof is relegated to the appendix A.1 To arrive at this bound, we manipulate the importance sampled population training loss (Sugiyama al. 2007b).
We emphasize that this result also provides important connection and a rationale for us- ing entropy based objectives an unsupervised adaptation objective from an importance sam- pling point of view that has been missing in the literature (Wang et al. 2021a Sun et al. 2019).
x
Entropy objective is imposed on points that are more typical with respect to the test than the training. Conversely, in the region where sam- ples are less likely with respect to the test dis- tribution, since it has been optimized for label prediction part of training, the entropy objec- tive is not imposed strongly. The above bound however hinges predictor by at most a constant factor e. To ensure a small value of e, we resort to pre-trainnt pre-training F with only D samples for few epochs before imposing any other type of regularization.
Figure 2: High level architecture of our method. Colored blocks represent parameterized sub- networks.
the assumption that pointwise in the domain X, F approximates the true soft
5.1 WEIGHTED ENTROPY OBJECTIVE
de (X)
Implementing the objective in 5 requires computation of the Radon-Nikodyy derivative This challenging when m (amount of unlabeled test samples) is small and typical way of den- sity estimation high dimensions is particularly hard. Therefore, propose to estimate the ratio (X)(X)
(X] by a parametrized network F. X + I. where Fn (X) shall satisfy the following con- straints: (x~ Fu (X)] 1, and (x) (1((P.(X)) = By definition of the Radon- Nikodym derivative, these constraints must be satisfied.
Building
5 solve for the following upper bound in Theorem
max R + ExT(X) ((F.)
(e(=F)(X))
s.t. EX- Fu (X) = Ex- (x (x) =
(6)
Finally, we plug in the empirical risk estimator for R., approximate the expectation in second term with the empirical version over DT, posit as a hyperparameter and add the unfairness objective in eq 3 to minimize the following:
min max ((O(F)) ER O(P) ((Fw)
A
Σ
((. + XeLwass
X,EDT
Σ (X) 1, and C2
Σ F. (X,) =1
s.t. Ci
X.EDT
X.EDS
Here λ and X2 are hyperparameters governing the objectives. C1 and C2 refer to the constraints.

---

Under review as a conference paper at ICLR 2023
Algorithm 1: Gradient Updates for the proposed objective to learn fairly under covariate shift
Input: Training data D Unlabelled Test data D model F, weight estimator Fa,, decaying learning rate 7,t number of pre- pre-training steps &, number of training steps & for A2
Output: Optimized parameters 0* (F)
(F) < random initialization
for t + to & do
(F) - 0 (F) N V EF)
o (Fww <- random initialization
for t f- 1 to do
(Fw) +
n (( (()))( subject to C1 and C2
+ + 0
n VAF) ( (F), ( (Fw));: /0 We apply gradient stopping through
Fu during backpropagation in this step */
8r (F) - ( (F)
Since the function has a representation layer followed by a classifier, i.e. F = h g. in our imple- mentation we apply the weighing function on g(-). Therefore, we have the following formulation:
min max ((((F), ER ((P) ((F..)
A1 1
Σ [cl-Po00xx,))
m X,E
1 Σ (x)))( s.t. C1 = m X,E
1, and C2
1 n
Σ X;EDS Fm (o
(8)
We use alternating gradient updates to solve the above min-max problem. Our entire learning proce- dure consists of stages: (1) pre-training I for some epochs with only DS and (2) further training F with (8. The procedure is summarized in Algorithm and a high level architecture is provided Figure
EXPERIMENTS
We demonstrate our method on widely used benchmarks the fairness literature, i.c. Adult, Communities and Crime, Arrhythmia and Drug Datasets with detailed description in appendix A.2
The baseline methods used for comparison are: MLP, Adversarial Debias (AD) (Zhang al.. 2018),
Robust Fair (RF) (Mandal al. 2020), Robust Shift Fair (RSF) (Rezaci et al. 2021 and Z-Score Adaptation (ZSA) with detailed description in appendix A.3 The implementation details of all the methods with relevant hyperparameters are provided in section A.5 The procedure for constructing the covariate shift described section A.4 To summarize, we use the Principal Component Analysis (PCA) direction to generate covariate shifted test set similar to Rezazi et al. (2021); Gretton ett al. (2008). The evaluation of our method against the baselines is done via the trade-off between Tairness violation (using Agodds) and error (which 100- accuracy). All algorithms are run 50 times before reporting the mean and the standard deviation in the results.
6.1 COMPARATIVE RESULTS
The experimental results for the shift constructed using procedure in Section[A.4]ar shown in Figure The results closer to the bottom left corner in each plot are desirable. In some cases, the standard deviation bars in the figure stretch beyond 0 in due to skewness when we plot standard error bars, however all the numbers across the runs positive.
3
Our method provides better error and fairness tradeoffs against the baselines on all the benchmarks. For example, the Adult dataset, have the lowest error rate around 15% with Agodds at almost 0.075 while the closest baselines MLP and RF fall short on either of the metrics. On Ar- rhythmia and Communities, our method achieves very low Agodas (best Anhhthhia with
7

---

Under review as a conference paper at ICLR 2023
Sevat
Figure 3: Comparison of our method against the baselines under Covariate Shift. The bars provide the standard deviation intervals both error (vertical) and Agodds (horizontal).
margin of ~ 30%) with only marginally higher error as compared to MLP and RF respectively. On the Drug dataset, we achieve the best numbers for both the metrics. For the same accuracy, we obtain 1.3x-2x improvements against the baselines methods on most of the benchmarks. Similarly for the same Agodda. we achieve up to 1.5x lower errors, is also important note that all the other unsu- pervised adaptation algorithms perform substantially worse and are highly unreliable. For example, ZSA performs well only on the Drug dataset, but shows extremely worse errors (even worse than random predictions) on Communities and Adult. The adaptation performed by ZSA is insufficient to handle covariate shift. RSF basclinn is consistently worse across the board. This is because tries to explicitly estimate (X) and p (X) which is extremely challenging whereas we implicitly estimate the importance ratio. While there is extensive evidence in the literature suggesting that fair- ness is achieved at the expense of performance (Menon & Williamson| 2015), we attribute the low errors achieved by our method to the novel entropy formulation, where in fact minimize the worst case weighting of entropy under the constraints. The saddle-point solution optimizes the entropy on points far from the training distribution, via appropriate scaling (importance weighting).
Zhao 2021] Zliobaite
6.2 RESULTS ON ASYMMETRIC SHIFT
We also study the problem of covariate shift under new lens where the degree of shift is substan- tially different across the groups which also motivates our novel formulation (section 4
To construct this, follow the same procedure described in section A.4 but operate on data for the two groups differently. The shift is introduced in one of the groups while for the other group, resort splitting it randomly into train-val-test. Figure 4 provides the results for the setup when shift is created in group A 0 whereas figure provides the result for shift in group A = 1.
Figure 4: Comparison of our method against the baselines under asymmetric covariate Shift for group A- 0.
We again observe that our method provides better tradeoffs across the board. For the shift in group A = 0, we have substantially better results on Adult and Arrhythmia with up to ~ 2x improvements on Agodds for similar error and up to ~ 1.4x improvements in error for similar Agodds- On the Communities dataset, MLP and AD show similar performance to ours, but much worse on the Drug dataset for both the metrics. ZSA performs comparably to our method only on Drug, but is substantially worse on other datasets. This confirms the inconsistency of the baselines under this setup as well. For the shift in group A = 1, we observe a similar behavior. On the Drug dataset, clearly obtain the best tradeoff compared to all other baselines. MLP and AD achieve similar performance to our method on Communities, but show up to 2x worse A Apodds on Arrhythmia with marginal improvements in On the Adult dataset, we observe up to 1.5x improvements against
8

---

Under review as a conference paper at ICLR 2023
Figure 5: Comparison of our method against the baselines under asymmetric covariate Shift for group A =
MLP and AD in Agodda RF baseline performs strictly worse than ours on Adult and Arrhythmia datasets where it's marginally better on either metrics on Communities and Drug, but at the expense of the other metric. is also important note that the errors are lower for all the methods as compared to figure since only one group exhibits substantial shift while degradation in equalized odds higher. This in line with the reasoning provided section based on theorem
6.3 RATIO ESTIMATED VIA (x()))
DP:
We empirically justify the of (X)) by compar- ing the distribution of the learned ratio across samples from Dd and D in figure 6
It is evident that the parametrized weight network can approximately learn importance ratios w.r.t IP" and IpS The ratio computed for the test points lie mostly be- tween and in order satisfy C1 (in eq whereas the ratio computed for the training points are mostly in order satisfy C2 (in eq 8). More importantly, F. learned end to end via optimization and doesn't incur any significant overhead compared to explicit density estimation.
Figure Comparison of the ratio esti- mated via (x()) across D and D The network learns the importance ratios w.r.t PT and PS.
6.4 EXTENDED ANALYSIS
Extensive experimental results and analysis across multiple settings are provided in Appendix (due lack of space). We empirically justify the motivation for unsupervised adaptation (described in section in section[A.6]
Ablation studies for the hyperparameters A₁ and A2 are performed in section A.6.2 for the magnitude shift in section A.6.3 and for value of m section AA.6.4] In section
A.6.5 we derive the connection standard entropy loss over unlabeled test samples (akin to the work by Wang et al. (2021a)) and demonstrate that our formulation achieves substantially better trade-off.
7 CONCLUSION
In this work, we considered the problem of unsupervised test adaptation under covariate shift to achieve good fairness-accuracy trade-offs when a small amount unlabeled test data is available. We showed how fair representation matching alone is insufficient due covariate shift. We pro- posed composite objective that involves weighted entropy loss on the unsupervised test and a rep- resentation matching loss across protected groups. Finally, we experimentally demonstrate that our composite objective outperforms many baselines on benchmarks in achieving non trivial accuracy- fairness trade-offs.

---

Under review as a conference paper at ICLR 2023
8 REPRODUCIBILITY STATEMENT
We have described all the relevant implementation details required to reproduce the experiments in the appendix. The details for all the benchmarks as well as the baselines are provided comprehen- sively. We will publicly release the source code after the review process.
ETHICS STATEMENT
This work aims to address the concerns related to the unfairness and bias issues that manifest when there is a shift in distribution across training and the testing phase of model. With the ever in- creasing real-world deployment of machine learning models, especially in life-altering scenarios like jurisdiction and college admissions, we hope to tackle these issues with this work and expect cumulative social gain.
REFERENCES
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. A reduc- tions approach to fair classification. Jennifer Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 60-66-6666 69. PMLR, 10-15 Jul 2018. URL/atts::/proceedaaee press/v80 http://aaaaaallaaaa 8a. html
.mlr.
Bang An, Zora Che, Mucong Ding, and Furong Huang. shifts via fair consistency regularization, 2022. URL
Transferring faimess under distribution
12796
Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXhv:1907.02893, 2019.
Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffiman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilović, et al. Ai fairesss 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5):4-1, 2019.
Shai Ben- Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances neural information processing systems, 19, 2006.
John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman. Learning bounds for domain adaptation. Advances in neural information processing systems, 20, 2007.
Tolga Bolukbasi, Kai- Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
Olivier Bousquet, Stéphane Boucheron, and Gábor Lugosi. Introduction statistical learning the- ory. In Summer school on machine learning, pp. 169 207. Springer, 2003.
Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commer- cial gender classification. Conference on fairness, accountability and transparency, pp. 77-91. PMLR, 2018.
Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney, Optimized pre-processing for discrimination prevention. Advances in neural informa- tion processing systems, 30, 2017.
Elisa Celis, Lingxiao Huang, Vijay Keswani, and Nisheeth K Vishnoi. Classification with fairness constraints: A meta-algorithm with provable guarantees. In Proceedings of the conference on fairness, accountability, and transparency, pp. 319-328, 2019.
Silvia Chiappa. Path-specific counterfactual fairness. Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 7801-7808. 2019.
10

---

Under review as a conference paper at ICLR 2023
Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, and Massimiliano Pontil. Fair regression with wasserstein barycenters. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Bal- can, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 7321-7331. Curran Associates, Inc., 2020. URL paper/2020/Ftle/ cdds226 wtw.:c/eeee...eeeeeeeeeeee 4
neur meuurps.... cc/
Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Learning bounds for importance weighting. In Lafferty, C. Williams, J. Shawe- Tayeer,aylr, R. Zemel, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 23. Curran Asso- ciates, Inc., 2010a. URL https:::///...........:: htttthttss://ttttt http:://///.ooceenggg....o neurips............p............ httpap//pps.ccaappe///////////// 59c3301 686466664462222 6be 3
pdd
Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Learning bounds for importance weighting. Advances in neural information processing systems, 23, 2010b.
Andrew Cotter, Heinrich Jiang, Maya Gupta, Serena Wang. Taman Narayan, Seungil You, and Karthik Sridharan. Optimization with non-differentiable constraints with applications to fair- ness, recall, churn, and other goals. Journal of Machine Learning Research, 20(172):1-59, 2019.
URL http:// /mrr wwwp:.orgppppppeee http://////////// 116118--61.111 61t.mm11
Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances Neural Information Processing Systems, 34:6478 6490, 2021.
Michele Donini, Luca Oneto, Shai Ben-Daviae John Taylor, and Massimiliano Pontil. Em- pirical risk minimization under fairness constraints. Advances in Neural Information Processing Systems, 31, 2018.
John Duchi and Hongseok Namkoong. Learning models with uniform performance via distribu- tionally robust optimization. The Annals of Statistics, 49(3):1378-1406, 2021.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Con- ference, ITCS 12, pp. 214-226, New York, NY, USA, 2012. Association for Computing Machin- doi: 10.1145/2090023622000000 20900255.
ery. ISBN 9781450311151
URL htwph:tt////cc http::tt/o....oog////
1145/2090236. 2090255
Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubra- manian. Certifying and removing disparate impact. In proceedings of the 2 11th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 259-268, 2015.
Benjamin Fish, Jeremy Kun, and Ádám D Lelkes. A confidence-hased approach for balancing fairness and accuracy. In Proceedings of the 2016 SIAM international conference on data mining, pp. 144-152. SIAM, 2016.
Geoffrey French, Michal Mackiewicz, and Mark Fisher. Self-ensembling for visual domain adapta- tion. arXiv, 2017.
Sainyam Galhotra, Karthikeyan Shanmugam, Prasanna Sattigeri, and Kush R. Varshney. Causal feature selection for algorithmic fairness. In Proceedings of the 2022 International Conference on Management of Data, SIGMOD 22, pp. 276 285, New York, NY, USA, 2022, Association for Computing Machinery. ISBN 9781450392495. doi: 10.1 10.1145/3514221.35112 799099 URL https:
//dbi. 1141 351 3517909
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180-1189, PMLR, 2015.
Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. 131 Covariie Shift by Kernel Mean Matching. In Dataset Shift in Machine Learning- The MIT Press, 12 2008 ISBN 97802621 70055. doi: 10.755 / .003. 0008. URL hntttt::: //ooi. org/10..010. 7551 wtt.://mmmiieesseeeeeeeeeeeeeeee /tte:///8222222..22 003. 0008
Arthur Gretton, Alex Smola, Jiayuan Huang, Marcel Schmittfull, Karsten Borgwardt, and Bernhard Schölkopf. Covariate shift by kernel mean matching. Dataset shift in machine learning. 3(4)55, 2009.
11

---

Under review as a conference paper at ICLR 2023
Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning, 2016a. URL
org/ abs/1610. 02413
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning, Advances in neural information processing systems, 29, 2016b.
Ray Jiang, Aldo Pacchiano, Tom Stepleton. Heinrich Jiang, and Silvia Chiappa. Wasserstein fair classification. In Ryan P. Adams and Vibhav Gogate (eds.), Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, volume 115 of Proceedings of Machine Learning Research, pp. 862-872. PMLR, 22-25 Jul 2020. URL httpt:tt////t... /ttpas//procceddd ngs. ml www..rpresssvvl1v Jiang20a.hti html
Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrim- ination. Knowledge and information systems, 33(1):1-33, 2012.
Takafumi Kanamori, Shohei Hido, and Masashi Sugiyama. A least-squares approach to direct im- portance estimation. Journal of Machine Learning Research, 10(48):1911-1445, 2009, URL
http:// wttp://rtr..ppp.p..//pep/emeees/ http://////////////e//e/////
Kwnamorianss.... html
Niki Kilbertus, Mateo Rojas-Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. 2017. doi: 10.
AB550/ARXIV.1766.022 027444
URL htttthttp//aaatt
02744
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2014. URL
https://arxs /ttp:v/aavvvvv//aa/ org/ abs/ / 1 1412. 6980
Jon Kleinberg. Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determi- nation of risk scores. arXiv preprint arXiv: (166666080077 2016.
David Krueger, Ethan Caballero, Joern-Hent Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. distrf--istributionn generalization via risk extrapo- lation (rex). In International Conference on Machine Learning, pp. 5815-5826 PMLR, 2021.
Matt Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL https: / http:////oceeeeeeeeeeeee newrisenerppps. http://eeia....etrr.....o/e////d a486cd077//acdd2 700002775055700 71177666223333 6ec5 Paper. pdf
Henry Lam, Fengpei Li, and Siddharth Prusty. Robust importance weighting for covariate shift, 2019. UtLpttps:: /ttpvv/aavvvvv//v////a//......aa org abs 011///99100..33
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normaliza- tion for practical domain adaptation, 2017. URL
new. wtw.:/rummeeeeoommmmeem forummird
BJ BJuysoFeg]
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In In- ternational Conference Learning Representations, 2017. URL https:://pperrevier
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Learning adversarially fair and transferable representations. In International Conference on Machine Learning, pp. 3384-3393. PMLR, 2018
Debmalya Mandal, Samuel Deng, Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fair- ness beyond the training data. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 18445-18456. Cur- ran Associates, Inc., 2020. URL /ttpstsss/////pooeeeeeiaoo newrips. htw.ipp.cccppppppp wttpppsscccppppp//22000
Aditya Menon and Cheng Soon Ong. Linking losses for density ratio and class-probability es- timation. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learn- ing Research, pp. 304 313, New York, New York, USA, 20-22 Jun 2016. PMLR URL
https: https://proceeeee
6.htm]
12

---

Under review as a conference paper at ICLR 2023
Aditya Krishna Menon and Robert C. Williamson. The cost of faimess in binary classification. In FAT, pp. 107-118, 2018. URL http::/prcceeenggs.... mlr. www.ll..peee//v/mmmeeeee html
Jose G Moreno-Torres, Troy Raeder, Rocío Alaiz-Rodriguez, Nitesh V Chawla, and Francisco Her- rera. A unifying view on dataset shift classification. Puttern recognition, 45(1):521-530, 2012.
Razich Nabi, Daniel Malinsky, and alya Shpitser. Learning optimal fair policies. In International Conference on Machine Learning, pp. 4674-4682. PMLR, 2019.
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D' Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv, 2020.
Luca Oneto and Silvia Chiappa. Fairness in machine learning. In Recent Trends in Learning From Data, pp. 155-196. Springer, 2020.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian D Ziebart. Robust faimess under covariate shift. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9419- 9427, 2021.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. arXiv preprint arXiv: /9/ 1.08731, 2019.
Babak Salimi, Luke Rodriguez, Bill Howe, and Dan Suciu. Capuchin: Causal database repair for algorithmic fairness. arXiv preprint urXiv:/902.000
2019.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 11539-11551, Curran Asso- ciates, Inc., 2020. URL 85690F81aadc1749111 18771a17777777111777778878888131
neurips ips. cc/pppper/220//11e .ddf
Saeed Sharifi Sharifi-Malvajeddi Michael Kearn,, and Aaron Roth. Average individual fairness: Algo- rithms, generalization and experiments. H. Wallach, H. Larochelle, A. Beygelzimer, F. dAlché- Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, vol- ume 32. Curran Associates, Inc., 2019. URL https: /tt.ttts///oocceeeiiiiii http:///ooeeeeeii//ceee.. ngs nwwrieeuipp..e cc/ pddf
paper/2019/FA1ee// otae:r/00001ffaeeooaaaeeeeee5000 feae
Yuge Shi, Jeffrey Seely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. arXiv preprint
2021.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-
likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
Samuel Henrique Silva and Peyman Najafirad. Opportunities and challenges deep learning ad- versarial robustness: A survey. arXiv preprint arXiv:2007.00753 2020.
Harvinect Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Fairesss violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Ac- countability, and Transparency, pp. 13, 2021.
Dylan Slack, Sorelle A Friedler, and Emile Givental. Fairness warnings and fair-maml: learning fairly with minimal data. In Proceedings of the 2020 Conference Fairness, Accountability, and Transparency, pp. 200-209, 2020.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56):1929-199) 2014. URL http:// htt../mlr.../ .ow....rr//go. satg//apprss/15//
sri vastava html
13

---

Under review as a conference paper at ICLR 2023
Masashi Sugiyama, Matthias Krauledat, and Klaus-Robee Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8335):88-10005 2007a.
wttp://///aaaaaaaaaaaaaaa html
Masashi Sugiyama, Matthias Krauledat, and Klaus- Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007b.
Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. In J. Platt, D. Koller, Y. Singer, and S. Roweis (eds.). Advances in Neural Information Processing Systems, volume 20. Curran Asso- ciates, Inc., 2007c. URL htt b
www.inniiisss... wttpc//pppee/2///eeee.... pdf
Yu Sun, Xiaolong Wang. Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts, 2019. URL https:
wwwxiv.org/abs////..aaaaa 13231
Rich Zemel Yu Wu Kevin Swersky, Toni Pitassi, and Cynthia Dwork. Learning fair representations. ICML PMLR, 2013.
Mingyang Wan, Daochen Zha. Ninghao Liu, and Na Zou. Modeling techniques for machine learning fairness: A survey, 2021. /tl.htttp:///rxxi..orrg /21111 03015
Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully
test-time adaptation by entropy minimization. In International Conference Learning Repre- sentations, 2021a.
net wet//ooummidduu- 3bb13kk
Haotao Wang, Junyuan Hong, Jiayu Zhou, and Zhangyang Wang. Equalized robustness: Towards sustainable fairness under distributional shifts. 2021b.
Yijie Wang, Viet Anh Nguyen, and Grani Hanasusanto. Wasserstein robust classification with
fairness constraints, 2021c. Uttpttps: http::://xxxvvvooo wwtpx/iirrr/abs///oe atl//bb//2100 /2t...213...00 06828
Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebuffi, Ira Ktena, Taylan Cemgil, et al. A fine-grained analysis on distribution shift. arXiv preprint arXiv:2/ 13388, 2021.
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In CVPR, 2020.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fair- ness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference world wide web, pp. 1171-
1180, 2017.
Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with ad- versarial learning. In Proceedings of the 2018 AAAT/ACM Conference AI, Ethics, and Soci- ety, AIES *18 pp. 335-340, New York, NY, USA, 2018, Association for Computing Machin- ery. ISBN 9781450360128.
doi: 10.1145/377872112277777 URL httL::ttt///:
1145/3278 0007722...777777 3278779
Wenbin Zhang, Albert Bifet, Xiangliang Zhang, Jeremy C Weiss, and Wolfgang Nejdl. Farf: A fair and adaptive random forests classifier. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 245-256. Springer, 2021.
Han Zhao. Costs and benefits of fair regression, 2021. URLLIttps://awxwvoor//////2222/ 08812
Han Zhao and Geoff Gordon. Inherent tradcoffs in learning fair representations. Advances in neural information processing systems, 32, 2019.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai- Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint arXiv: (707:07777 2017.
14

---

Under review as a conference paper at ICLR 2023
Indre Zliobaite. On the relation between accuracy and fairness in binary classification, 2015. URL pittps://ariis htt.v//.org//b//155..ooo2
15

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 PROOFS
Proof of Theorem We start with rewriting the expected cross entropy loss the test impor- tance sampled loss on the training distribution.
RT= = (x)(x)
Σ -PT x| log( 2|Xi) =
(9)
8E(0,1}
dIP (X) (X)
Eps(X)
Σ ME(0,1)
-ps (Y = x|X) log = x/Xx)
-
(10)
(a) is the Importance Weighting technique proposed by (Sugiyama et al. 2007b) and dr (X) Radon-Nikodym derivative because the two distributions are absolutely continuous with respect to each other.
(x) is the
dP (
R + (x)
-Ps log) P =\/X))
(1I)
dPS dPS(X)
vE(0,1}
dps (X)
= =R$ + Exr(x)
Σ
- PP lo =//xi)
(12)
dPP (x)
DE
b
(1-
dFs dxs(()
S R + x EpT(x)
(13)
dPT ]PP(X)
(E(0.1)
(x) H((Ix)
& R + x (x)(x)
(x)
(14)
(b) is because of the assumption that (T r( x)
s almost surely with respect to X ~ PJ
(c) This is because 1 - - x20. 0.
A.2 DATASET DESCRIPTION
The detailed description of the datasets used this work follows:
Adult is a dataset from the UCI repository containing details of individuals. The output variable is the indicator of whether the adult makes over $50k year. The group attribute is gender. Following Mandal et al. (2020), we use the processed data with 2213 examples and 97 features.
Arrhythmia is a dataset from the UCI repository where each example is classified be- tween the presence and absence cardiac arrhythmia. The group attribute gender. Fol- lowing Rezaci et al. (2021), we used the dataset used containing 452 examples and 279 features.
Communities and Crime dataset from the UCI repository where each example rep- resents community. The output variable is the community having violent crime rate in the thth percentile of all the communities. The group attribute is the binary indicator of the presence of the majority white population. Following Mandal et al. (2020), we use the dataset with 2185 examples and 122 features.
Drug is dataset from the UCI repository where the task is to classify the type of drug consumer based on personality and demographics. The group attribute is race. Follow- ing Rezaei al. (2021), used the dataset with 1885 samples and 11 features.
16

---

Under review as a conference paper at ICLR 2023
A.3 BASELINES
We use the following baselines for comparison. This covers the exhaustive set of relevant methods described in section
MLP is the standard Multi Layer Perceptron classifier that doesn't 't take into account shift and fairness properties. In the standard in-distribution evaluation settings, such a model usually provides the upper bound the accuracy without considering fairness, however the scenario differs as we are dealing with distribution shifts.
Adversarial Debiasing (AD) ZZhng et al., 2018) is one of the most popular debiasing methods in the literature. This method performs well on the fairness metrics under the standard in-distribution evaluation settings, but fails to do so in the shift setting.
Robust Fair (RF) (Mandal et al. 2020) proposes a framework to learn classifiers that are fair not only with respect to the training distribution, but also for a broad set of distributions characterized by any arbitrary weighted combinations of the dataset.
Robust Shift Fair (RSF) (Rezaci et al.) 2021) is recent and most relevant baseline to this work. The authors propose a method to robustly learn a classifier under covariate shift, with fairness constraints. A severe limitation of this method is that it requires explicit estimation of both source and target covariates' distributions.
Z-Score Adaptation (ZSA) following the thread of work under Batch Norm Adaptation (Li et al., 2017 Schneider et al. 2020) literature, we implement baseline that adapts the pa- rameters of the normalizing layer by recomputing the z-score statistics from the unlabeled test data points.
A.4 SHIFT CONSTRUCTION
To construct the covariate shift the datasets, i.e., to introduce PS (X, o PT (X A). we utilize the following strategy akin to the works Rezaci al. (2021): Gretton et al. (2008). First, all the non-categorical features are normalized by z-score, We then obtain the first principal component of the of the covariates and further project the data onto it. denoting it by Pc. We assign score each point Pe[i using the density function E Pe[[i] - e7-(Pe(1)-)) 1)/1)-b)//2 /2. Here, is hyperparameter controlling the level of distribution shift under the split, b the 60th% (percentile) of Pc and 2 the normalizing coefficient computed empirically. Using this, we sample 40% instances from the dataset as the test and remaining 60% as training. To construct the validation set, we further split the training subset make the final trainvalidattion.test ratio as 5 shifted.
where the test is distribution
Note that for large values of % all the points with Paμ > b will have high density thereby increasing the probability of being sampled into the test set. This generates a sufficiently large distribution shift. Correspondingly, for smaller values of 7, the probability of being sampled is not sufficiently high for these points thereby leading to higher overlap between the train and test distributions.
A.5 IMPLEMENTATION DETAILS
We use the same model architecture across MLP and our method in order to ensure consistency. Following Wang et al. (2021b), Fully Connected Network (FCN) with 4 layers is used, where the first two layers compose y and the subsequent layers compose h. For AD, we use additional 2 layer FCN that serves as the adversarial head a ((X) -> A (similar to (Wang et al. (021b)).
Without further specification, we use the following hyperparameters to train MLP, AD and ZSA. The number of epochs is set to 50 with Adam as the optimizer (Kingma & Ba 2014) and weight decay of 1 (for Adult dataset, the weight decay is 44. The learning rate is set to the value of Le - initially and is decayed to 0 using the Cosine Annealing scheduler (Loshchilov & Hutter, 2017). A batch size of 32 is generally used train the models. The gradients clipped at the value of 5.0 to avoid explosion during training. The dropout (Srivastava et al.] 2014) rate is set to 0.25 across the layers. For AD, the adversarial loss hyperparameter post grid search is used.
17

---

Under review as a conference paper at ICLR 2023
RF and RSF works have tuned their model for the specific architecture and corresponding hyperpa- rameters (different from the aforementioned specifics). We perform another grid search over these hyperparameters and report the best results for comparison.
For our proposed method, pre-train the model for 15 epochs with only ER For the next 35 epochs we use the objective in eq [ but with a higher training data batch size to reduce variance in the Monte Carlo Estimation of the second constraint ( Ex,ED (x))( =1) The value of m (size of D²) kept at 50 for the main experiments, which << size of DS The primary experiments are run with the shift magnitude ?y 10 (with ablations provided in vection[A.6.3 The constraints C₁ and C2 as mentioned in arre are implemented as squared error terms where we minimize
(( -X.ED (9(x:)) -1 c +22-((; Ex.EDs Fo(o(X.))) -1)² where and C2 are hyperparameters to control the relative importance of each constraint. The values of the tuple (A1, A2) are set to the following Adult (1, 0.01): Arthythmia (0.01, 0.005) Communities: (0.005, 0.0001) and Drug: (0.1, 0.1) post grid search. All experiments are run on single NVIDIA Tesla V100 GPU.
A.6 ANALYSIS
A.6.1 UNSUPERVISED ADAPTATION WITH OUR ENTROPY FORMULATION UNDER ASYMMETRIC SHIFT
The asymmetric shift setup described in section 3.1 provides a well grounded motivation (section 4p and use case for explicitly handling shifts along with the unfairness objective. We complement the claim with empirical evidence here. The results in table [ provide comparison of the performance across the metries with and without proposed formulation. The wasserstein objective in eq retained in both settings. We observe significant improvements on both error and Apodds with our formulation. Particularly the Drug dataset, we see improvement of almost 4% in and around 13x in the Agodds, which is also notable on Arrhythmia.
Table 1: Comparison of the performance on using the unfairness objective without and with the unsupervised adaptation (our proposed entropy formulation). We observe substantial improvements in both error and Aoodss Numbers in the parenthesis represent standard deviation across the 50 runs.
Dataset
Arrhythmia
Drug
Entropy Variation
Without Entropy
With Entropy 27.617 (2.978) 0.071 (0.037)
Without Entropy 35.859 (3.437) 0.076 (0.043)
With Entropy (og 31.910 (0.166)
Error %
28.6 23.648 (3.079) 0.080 (0.032)
Apodda
0.006 (0.013)
A.6.2 VARIATION OF À1 AND A2
In this section, we study the variation of the performance of our method against the hyperparame- ters governing error (A1) and Agodds (A2). While studying the effect of either, we keep the other constant.
Table 2: Variation of the performance of our method with Entropy Regularizer A on Adult dataset.
0.001
0.005
0.01
1.0
Error
23.819 (8.819 (8.593)
22.047 (6.631)
20.510 706)
20.85I (7.829)
(1.118
14.787 (I.326)
Agudda
0.131 (0.038)
0.126 (0.037)
0.129 (0.037)
0.129 (0.029)
0.104 (0.033)
0.075 (0.30)
Table 2 reports the variation for A₁ keeping X2 = 0.01 fixed. is evident from the numbers that increasing λ has strong correlation with the reduction in error, which exhibits a saturation at 0.1. Higher values of A emphasize the minimization of the worst-case weighted entropy thus helping in calibration of the network in regions across IPT Furthermore, ments in Apodds which is inline with the motivation of handling shifts along with an unfairness objective (section 4.. Increasing \ doesn't 't help post a threshold value the correct estimation of
observe significant improve-
18

---

Under review as a conference paper at ICLR 2023
the true class for given X under pd becomes harder, particularly in regions far from the labeled in-distribution data. Imposing very strong A can hurt the model performance.
The variation against λ2, keeping λ fixed is reported in table As X increases, observe a gradual improvement in Agodds- This exhibits a maxima after which the performance degrades drastically. This because strongly penalizing Lwass (D") with a small number of samples m leads overfitting (illustrated by the large standard deviation) while matching IPz (X]A). This also hurts the optimization as demonstrated by the substantial increase in error.
Table 3:
Variation the performance of our method with Wasserstein Regularizer X2 on Adult dataset.
Ag Error %)
0.001
0.005
0.01
0.1
15.049 .244)
848849 (1,437)
14,901 (1.352)
14.787 (1.326)
17.936
(15.962)
42.280 (32.581)
ARGads
0.091 (0.031)
0.099 (0.034)
0.098 (0.032)
0.075 (0.030)
0.074 (0.036)
0.093 (0.064)
A.6.3 SHIFT MAGNITUDE
We study the variation the performance our method against the magnitude of shift 7y on Ar- rhythmia. A comparison against the best baseline ZSA is also provided. The variation of error is plotted in the left subfigure of With no shift in the data, exhibit small emoos D follows in-distribution. With the increase in the value of 7, ZSA shows sudden increment in the eπrоr with an unstable pattern whereas our method exhibits more gradual pattern and lower error as compared ZSA. This justifies that the weighted entropy objective helps.
O, we observe that both the methods
0.50
Eree 2
29
0.06
Magritude of the eviti
Magnitude of the eniti
Figure 7: Variation error against 7 (left subfigure) and Agodds against y (right subfigure) on Arrhythmia Dataset. We observe that our method performs better in both metrics against the best baseline ZSA. While the error increases gradually, but we observe substantially better Agodds for method.
On the contrary, observe that method is highly stable over ArOdds and performs consistently better for larger shifts as compared to ZSA. We attribute this effect to the proposed objective which optimizes the model learn fairly under the shift and over the worst case scenario.
A.6.4 VARIATION OF SIZE OF TDT
Here, we study the dependence of the methods the size of DJ', The left subfigure in g plots the variation error against m. The eтror gradually decreases for our method and RSF as the estimation of the true test distribution improves and the optimization procedure covers larger region of IPT This also makes the approximation by F,, much more reliable and closer to true ratios. Although, the results don't show notable improvements after certain threshold are dealing in
19

---

Under review as a conference paper at ICLR 2023
unsupervised regime over P", It becomes increasingly harder to correctly estimate the true class for given X under P", particularly in regions far from the labeled in- in-distribution data. Interestingly, ZSA doesn't 't exhibit any improvements which demonstrates that merely matching first and second order moments across the data not sufficient to handle covariate shifts.
The right subfigure in X plots the variation of Agodds against Here, observe consistent reduction in Agodds as more data from Pr helps matching representations via improved approx- imation of Ip A). Further this objective only deals with matching representations across the groups and doesn't stagnate as quickly with increasing m as the error margins, which suffers from lack of reliable estimation in regions far from in-distribution.
We consistently outperform RSF in both very small and larger regimes of m, partly verifying the importance of Fw rather than direct estimation of ps and Ip" as RSF does. ZSA is substantially worse than both RSF and our method in terms of error.. In terms of goddd its only marginally better than our method for m 10 and m 20, but huge expense of prediction performance.
18
75A
20
10 10
20
52
106
23
20
50
100
Numbee
Figure 8: Variation of error against m (left subfigure) and ABodds against (right subfigure) on Adult Dataset. We see reduction in both error and Arodds with increasing value of m.
A.6.5 UNWEIGHTED ENTROPY OUR WEIGHTED ENTROPY FORMULATION
easy to observe that we can recover the standard unlabeled test entropy minimization using our derivation. Formally specifyying upper bound eq 14 to obtain entropy follows:
R 5 + x (x)
H(YIX)
< R +e Eer(x) (((()))) vee S
e
(15)
Our formulation particularly provides tighter bound compared to standard entropy and implicitly
accounts for points DT that are close to ps by assigning low weight. The experimental results comparing the settings both with and without the unfairness objective are provided in table 4
Our formulation achieves substantially better results with a relative improvement of around 33% in error. Note that due the fairness-error tradeoff, the standard (unweighted) entropy achieves better Agodde. but that is achieved at the expense of a nearly random classifier as evident from the error rate of nearly 50%. We also highlight the large standard deviation in the results achieved by unweighted entropy. This is largely because it seeks to minimize entropy across all m points whereas our objective more adaptive based on the approximation of importance ratio.
Table Comparison of the performance of Standard Unweighted Entropy v/s our Weighted Entropy formulation on Communities dataset.
Without Wasserstein Objective Unweighted Entropy Weighted Entropy (Ours)
With Wasserstein Objective (eq
Entropy Variation
Unweighted Entropy
Weighted Entropy (Ours)
Erroo %
78778770 (12.900)
34.291 (4.463)
45.654 (13.090)
35.549 (3. (3.748)
Agodd
0.204 (0. 194)
0.359 (0.074)
0.201 (0.200)
0.328 (0.073)
20

---

Under review as a conference paper at ICLR 2023
A.6.6 COMPARISON TO OTHER DENSITY RATIO ESTIMATION METHODS
As the explicit computation of the density values can be hard, we estimate the ratio d (X)
(X) via
parametrized network for this class of baselines. Density ratio estimation methods were previously Kanamorii & Ong (2016) analysed these methods in unifying framework.
proposed in the works by [Sugiyama et af. (2007c) (KLIEP), Kanamori al. (2009) (LSIF). Menon
To experimentally demonstrate the efficacy of our method over the aforementioned, we use the density ratio estimation methods of KLIEP and LSIF in the following manner. First, the importance ratio dP" (X)
[ is estimated using unsupervised test samples and the training samples available based on the KLIEP and LSII losses (given in Menon & Ong) (2016) via parameterized weight network
s(X). Then, we train classifier based on the following instance weighted cross entropy loss and representation matching loss:
min ((F=hog)
Σ (.A)EEE
s(X.) (olo PO(F)
(Y= (8 =Y(x))) + aawass (D=)
(16)
where s(X) is non-negative function which is obtained by minimizing:
LKLIEP(s(X)) m X
Σ
- log (X.)+
Σ
s(Xi) sxx))
(17)
XIEDS
or,
(LSIF(((X)
Σ (X,, 1
Σ (x(X,)))
(18)
m
2
n
X
X;EDS
The results stated in tables 5 and First, observe that our method consistently outperforms these algorithms across the datasets. The relative improvement of our method is as high as ~ 31% error on Adult dataset and 32.5x in AgOdds on Drug dataset against LSIF, Similar non-trivial margins can be noted on other datasets. Second, the variance in accuracies of the KLIEP and LSIF based importance is very high on the Drug dataset. Particularly, both KLIEP and LSIF exhibit up to 20 40 times higher variance in error and up to 10 12 times in Arodds.
We can attribute this to the phenomenon that in the small sample regime, impor- tance weighted training on training dataset alone may not bring any improvements for covariate shift due to variance issues and thus estimating the ratio can be insufficient. In fact, this has been pointed out in Menon & Ong (2016). We, on the other hand propose new formulation to optimize for an upper bound based on the ratio estimation but due to the negative exponent of the importance ratio, the importance ratio's 's effect on the loss does not induce such high variance and it also leverages unsupervised test samples at training time.
Key Takeaway:
Table 5: Comparison of our method against popular density ratio estimation methods: KLIEP and LSIF on Drug and Adult datasets.
Dataset
Drug
Adult
Method
Error %
Agodd=
Error %
Agodde
KLIEP
34.782 (3.879)
0.043 (0.042)
20.787 (4.368)
0.124 (0.028)
LSIF
39.517 (5.944)
0.065 (0.049)
19.428 (3.203)
0.107 (0.028)
Ours
32.928 (0.143)
0.002 (0.004)
14.787 (1.326)
0.075 (0.030)
A.6.7 FAIRNESS-ERROR TRADEOFF CURVES
To further demonstrate the effectiveness of our method, we plot the Pareto Frontier figure g (variance bars are removed to retain clarity), similar to Agarwal al. (2018) Achievable trade- offs for the baselines are plotted along with our Pareto curve for comparison. We observe that the curve corresponding to method is closer to the left axis with high gradient. The error reduces drastically for a small increase in Apoads while providing better tradeoffs as compared to the optimal performance of the baselines.
21

---

Under review as a conference paper at ICLR 2023
Table 6: Comparison of method against popular density ratio estimation methods: KLIEP and LSIF on Communities and Arrhythmia datasets.
Dataset
Communities
Arrhythmia
Method
Error %
Agodd=
Error %
AROdd
KLIEP
38,466 (4,403)
0.323 (0.062)
30.630 (3.605)
0.085 (0.044)
LSIF
38.842 (3.518)
0.340 (0.055)
30,972 (3.467)
0.081 (0.042)
Ours
34.549 (3.748)
0.328 (0.073)
29.746 (3,519)
0.077 (0.036)
Coeen
Figure 9: Fairness-Erre Tradeoff Curves for our method (Pareto Frontier) against the optimal per- formance of the baselines. Our method provides better tradeoffs all cases. (On Drug dataset, the performance is concentrated around the optimal point). Variance bars are removed to retain clarity in the plots.
A.6.8 EMPIRICAL INVESTIGATION THE BOUND THEOREM
We compare the ratio of the prediction probabilities for the classes (y E {0, 1}) on the validation set (which is not available during training to our algorithm) between a classifier trained only on the training set (Train) and classifier trained only on the held-out test set (Test).
We plot the ratios in figure T0 with outliers removed. The subfigures (a). demonstrate the ratio for the true class label for the samples. Subfigures (c).(d) demonstrate the ratio for class and subfigures (f) demonstrate the ratio for class y = 1. Correspondingly, in figure II we plot the ratios with outliers. Note that atmost 4 points in every plot outliers with ratios > 5. This empirically justifies that c can be set not too high with high probability except for a few outliers.
TrakiTed
Tutiihal
TutiPaak Ride
'1 Parie
Figure 10: The subfigures demonstrate the ratio of the prediction probabilities for the classes (y (0, 1)) on the validation set between a classifier trained only on the training set (Train) and classifier trained only on the held-out test set (Test), with outliers removed. Note that = 5 provides a reasonable threshold and holds for all the samples but for 4 outliers (shown figure [
A.6.9 COMPARISON OF ACCURACY PARITY
We further demonstrate that our method better as compared to the baselines when the fairness metric is Accuracy Parity. The results for Adult dataset provided in table
22

---

Under review as a conference paper at ICLR 2023
Figure 11: The subfigures demonstrate the ratio of the prediction probabilities for the classes (y E {0, 1)) on the validation set between a classifier trained only on the training set (Train) and a classifier trained only the held-out test (Test), with outliers. Atmost 4 points in every plot outliers with ratios
5.
Table 7: Comparison of Accuracy Parity as well as Error for all the methods. We outperform the baselines, particularly KLIEP and LSIF that are prone to poor results due to high variance.
Method ->
MLP
AD
RF
RSF
ZSA
KLIEP
LSIF
Ours
Error %
17.735
18.356
22.525
33.591
63.396
20.787
19.428
14.787
Accuracy Parity %
5.764
3.156
6.094
11,851
11.131
4.417
6.677
2.990
A.7 COMPARISON OF GENERALIZZTIN BOUNDS BETWERN IMPORTANCE, SAMPLED TRAINING LOSS AND OUR OBJECTIVE
In this section, we would like contrast Generalization bounds between our objective (Right Hand Side 5) and the left hand side which is importance sampled training loss.
The main intention is to bring out dependence on the variance of importance ratios. Therefore, we make the following simplifying assumptions:
Assumption 3. Xx).
Let ④ {0] ...... be parameters of finite of classifiers of the form
Let us assume the loss function (x log (Y bounded between [0, 1] in the domain 10.1} x X for all 0 E θ.
Let the loss function (2(X; Σ (0 -P&(Y = y/x) Po(Ý = y(X) also be bounded between [0, 1] in the domain {0,1] x X' for all 0 E e.
Let assume we have access the exact importance weight w(X) p (X)
(x)) Since,
assume (-)(.) and (.) are absolutely continuous with respect to each other, w(X) >
5 (X)
0, vX E X. For convenience of notation, let ((X)
(x
Let sup w(X) = M. Let the variance of the importance ratio with respect to the training XEX
distribution be Ex-es [w= (X() = 0²
Remark: We have assumed é is bounded in [0, 1]. If the log loss over a suitable function class is Lipschitz and domain is bounded, then the loss is also bounded. Therefore, it is not very heavy assumption and we wanted keep the analysis simple and normalized.
There are two loss functions we compare:
Rrs(0)= = Σ
w ((X,)& X); and
(X.NN
2. Rw (w)
Σ
4(Y,X,;0)
+
Σ
6 (xx)) (x;
(X
23

---

Under review as a conference paper at ICLR 2023
(f) is the empirical importance sampled loss while RwE is the weighted entropy objective of Theorem 2
We recall some generalization bounds for finite hypothesis classes with bounded risks.
Definition 3. Rademacher complexity R(AA) for finite set A {ai, a2 ... aN) Rn given by:
((A) E, (sup Σ o,a a
(19)
where a is sequence of n i..d Rademacher variables each uniformly sampled from {-1, -1] and afi] is the i-th coordinate of vector a.
Empirical Rademacher complexity of class of finite number of functions FF on data set DD with m samples is given by R(F(D)) where F(D) = {vee[f(x), Vo E D), vf E F}. Here, vec(.) is vector of entries.
Theorem Boussque al. (2003)). R(F(D)) &
[f(z)) SEF
sup
2 Iogi P
Theorem 5. [Bousquet et al. [2003)] When a dataset D is sampled i..d from distribution (X) and is uniformly bounded by L over the domain of P. then with probability - 5 over the draw of D,
In(2/5)
E Ep-rlf(x)
S
Σ 2R(((D) + 3L
P
2|D]
JED
2 log
In(2/8) 2/D
Σ RED
2L
+ +3L,
VSEF
(20)
D
[D]
Theorem 6. Under Assumption and D' p we have 40
we have that with probability
1 26 over the draws of DS
Ips
2 log θ D
2 log O
(2/5))
In(2/8)
Eys pr [RwE(0)] S (w 2
+ 22
+3,
+ +x
DT
2
2
(21)
Proof. We apply Theorem 5 to (1(-) (which is bounded by 1) and e e= where (2(-)
<
1, e em)i)) < 1 with the appropriate datasets in Assumption 3 We then use union bound over the two error events that result from application of the theorem twice.
For finite hypothesis classes, we recall generalization bounds for importance sampled losses from Cortes et al. (2100b)
Theorem 7 (Cortes et al. (2010b)). Suppose that dataset D is sumpled i.id from distribution ((X). f is uniformly bounded by over the domain of P, and a fixed weighing function w(x) is such that sup w(x) = M, Fx<p(w(x)) I o². Consider the loss function f(x) = w(x)f(x). We
denote f = wo =wo
f. then with probability 1 8 over the draw of D. we have:
2M (log] 0 + log(1/0)) + LV 202 (log| + log (1/6))
En-r(((>) < Σt()
3DS
Vf fwof
(22)
Applying Theorem to Ris(0) we have the following result.
Theorem 8. Under Assumption we have that with probability over the draws DS ~ P.S. DS
have 40 E A
Eps (R)( Ris(0)+ + 2ML(log|O) + log(1/)))
202 (log]O) + log( (1/))
(23)
3/DS
D
24

---

Under review as a conference paper at ICLR 2023
Proof. The proof is direct application of Theorem to Ris(f) under Assumption
Key Takeaways: Comparing Theorem 6 and Theorem g we see that the generalization bounds for importance sampled training loss depends variance of importance ratio and also the worst ratio over the training set (M and o²). In contrast, our objective does not depend on these parameters primarily due negative exponential dependence on wh We also note that Rw depends on size of test set also while the other does not seem to. However, Ris needs to estimate importance ratios whiich will depend on the test set We have analyzed both losses when the importance ratios are assumed to be known just to bring the difference in dependencies other parameters.
Remark: In Assumption 3 we have assumed a finite hypothesis class O. However, our result for Theorem [ would generalize (as is) with rademacher complexity or covering number based arguments of infinite functions classes & and lz. Cortes et al. (2010b) also point out analogous generalization for the importance sampling loss.
25


Reviews:
[{"review_id": "by2hOdJcnFt", "reviewer": "Reviewer_iDgW", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper considered the problem of unsupervised test adaptation under covariate shift to achieve good fairness-accuracy trade-offs when a small amount of unlabeled data is available. The authors proposed a new weighted entropy based loss function to account for covariate shift, in combination with a representation matching term to address fairness. A set of empirical experiments were conducted on four popular datasets to show the performance of their proposed method in achieving non trivial accuracy-fairness trade-offs.\n", "summary_of_the_review": "As mentioned above, I saw several strengths of this paper:\n\n1. Novel formulation to address covariate shift along with fairness\n2. Solid theoretical results to support the formulation\n3. Comprehensive empirical results to further support the formulation\n4. Thorough literature review for relevant works\n\nI also like the connection they made between a weighted entropy and importance sampling. ", "strength_and_weaknesses": "Overall this is a well-written paper with solid theoretical results and comprehensive empirical findings. \n\nStrength:\n1. Novel formulation to address covariate shift along with fairness\n2. Solid theoretical results to support the formulation\n3. Comprehensive empirical results to further support the formulation\n4. Thorough literature review for relevant works\n\nWeakness:\nI have several questions that I hope the authors can further clarify.\n1. The new proposed method utilizes the test set information to address covariate shift. In some scenarios the test set is not known a priori. I am wondering in the experiments section, whether the baselines that the authors compared to, use the test set information or not. If they did not use the test set information, can we call it a fair comparison?\n2. Furthermore, when considering the baseline choices, have you considered any of the methods mentioned in Section 2: Fairness under Distribution shift? I feel like those methods are more relevant to what you studied in this paper. I am interested in seeing the performance comparison with those methods if possible.\n3. The representation matching bears a similar spirit with the paper: Shalit, U., Johansson, F.D. and Sontag, D., 2017, July. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning (pp. 3076-3085). PMLR, where they also tried to minimize a distributional distance between representations in two groups. I am curious why you chose the Wasserstein metric, even though Theorem 1 suggests the total variation distance. \n4. In Section 5.1, why do we maximize (instead of minimize) over \\theta in Eq. (6)?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "1wuukfFqxX", "reviewer": "Reviewer_DiLH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper focuses on the fair learning problem under the covariate shift. The main contribution is the weighted entropy loss component of the regularized objective function. In the experiments, the proposed method achieves a better tradeoff between accuracy and fairness. ", "summary_of_the_review": "I am leaning towards rejection as the novel contribution to fair learning is limited, even though the weighted entropy is an effective method under the covariate shift. ", "strength_and_weaknesses": "Strength\n\nFairness under the covariate shift is an important under-investigated topic and very relevant to the community.\n\nThe empirical results are significant.\n\nWeakness\n\nThe lack of the target labels means the proposed method cannot be applied to all the fairness metrics that requires the knowledge of the true labels. So the metric considered in the paper is only an accuracy disparity. This will significantly affect the applicability of the method. Also, in the experimental results, I think the x-axis should be changed as the metric is not the real equalized odds. \n\nThe main contribution of the proposed formulation is actually in the first term, which is the covariate shift correction term, not the fairness regularization term. So it seems the resulting formulation is a weighted entropy loss for covariate shift correction (ratio estimation) in a mini-max end-to-end training. The novelty seems to be limited. For example, without fairness regularization, we can also apply the first term to learning problems under covariate shift. The application of weighted entropy under covariate shift is also not new. How are the fairness constraints connected to the covariate shift?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0rUXLa8hZ-", "reviewer": "Reviewer_HTwH", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors study the fairness issue under the covariate shift setting. To address this problem, they first show that the previous bound in (Zhao & Gordan (2019)) can not handle the asymmetric covariate shift setting. Afterward, they propose a novel bound on the performance in the test distribution. They further develop a min-max optimization framework to calculate the proposed bound. Finally, they conduct experiments on several datasets to prove the effectiveness of their method.", "summary_of_the_review": "Although the authors study an important problem, the methods have several unclear parts and the experiments should also be improved. As a result, I vote for the rejection in this round.", "strength_and_weaknesses": "The authors study an important question, which generalizes standard fairness issues to distribution shift scenarios (in particular, covariate shift). I value the importance of this setting. However, I think the methods have several unclear parts and the experiments should also be improved.\n\nFor the method part:\n1. The first main issue is the unclear fairness notion throughout the paper. Equalized odds and accuracy parity correspond to two different fairness targets and there are different methods to deal with them. The authors should highlight the main fairness notion they target and show why their method could guarantee or approximate the fairness notion.\n2. Theorem 2 needs further explanation. Firstly, the assumption on the constant $\\epsilon$ is unclear. The value of $\\epsilon$ depends on the model parameter and I think it can easily tend to infinity if a model outputs a small $P(\\hat{Y}=y|X)$. Secondly, the superiority of the proposed theorem compared with Theorem 1 (Zhao & Gordon (2019)) is unclear. As I mentioned before, I think the value of $\\epsilon$ may be large in general settings and the second term in the RHS of Theorem 2 may also be large, making the bound loose. The authors should discuss how Theorem 2 could guarantee handling the asymmetric covariate shift setting.\n3. The comparison with density ratio estimation methods is unclear, such as the methods proposed in [1]. The authors claim that \"the typical way of density estimation in high dimensions is particularly hard\". However, Equation (5) requires the estimation of density ratio only and several methods could deal with the problem (such as the KLIEP and LSIF loss mentioned in [1]). As a result, it would be better if the authors can compare their algorithm with these methods.\n4. I am unclear why we need the max step to optimize the density ratio function. The $F_w(X)$ should be the density ratio between the training and test distribution and the authors are encouraged to demonstrate why the max step could lead to the estimation of the true density ratio.\n\nFor the experimental part:\n1. As mentioned before, the authors should compare the methods that use typical density ratio estimation methods [1] to estimate the function $F_w(X)$ instead of the max step.\n2. The fairness-accuracy trade-off curves are encouraged to be plotted. See examples in [2].\n\nOther minor typos:\n1. Throughout the paper: upto -> up to\n2. Second paragraph in the introduction: Similarly, (missing a comma here)\n3. The cross-entropy loss in Equation (1) is wrong. The equations miss the $Y_i$ term.\n\n[1] Menon, Aditya, and Cheng Soon Ong. \"Linking losses for density ratio and class-probability estimation.\" International Conference on Machine Learning. PMLR, 2016.\n\n[2] Agarwal, Alekh, et al. \"A reductions approach to fair classification.\" International Conference on Machine Learning. PMLR, 2018.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject