Fairness Transferability Subject to Bounded Distribution Shift
Authors: Yatong Chen
Reilly Raab
Jialu Wang
Yang Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Fairness transferability
bounded distribution shift
demographic parity
equal opportunity
covariate shift
label shift

Abstract: Given an algorithmic predictor that is "fair"' on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound? In this paper, we study the transferability of statistical group fairness for machine learning predictors (i.e., classifiers or regressors subject to bounded distribution shift. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre-trained models in new settings. Herein, we develop a bound that characterizes such transferability, flagging potentially inappropriate deployments of machine learning for socially consequential tasks. We first develop a framework for bounding violations of statistical fairness subject to distribution shift, formulating a generic upper bound for transferred fairness violations as our primary result.  We then develop bounds for specific worked examples, focusing on two commonly used fairness definitions (i.e., demographic parity and equalized odds) and two classes of distribution shift (i.e., covariate shift and label shift). Finally, we compare our theoretical bounds to deterministic models of distribution shift and against real-world data, finding that we are able to estimate fairness violation bounds in practice, even when simplifying assumptions are only approximately satisfied.

Fairness Transferability Subject to Bounded Distribution Shift
Yatong Chen', Reilly Raab', Jialu Wang, Yang Liu"
University of California, Santa Cruz (ychen592, reilly, faldict, yangliul@ucsc.edu
Abstract
Given algorithmic predictor that is "fair" on some source distribution, will it still be fair on an unknown targer distribution that differs from the source within some bound? In this paper, study the transferability of statistical group fairness for machine learning predictors (i.e., classifiers or regressors) subject to bounded distribution shift. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre- pre-trained models in new settings. Herein, develop a bound that characterizes such transfcrability, flagging potentially inappropriate deployments machine learning for socially consequential tasks. We first develop framework for bounding violations of statistical fairness subject to distribution shift, formulating generic upper bound for transferred fairness violations our primary result. We then develop bounds for specific worked examples, focusing on two commonly used fairness definitions (i.e., demographic parity and equalized odds) and classes of distribution shift (i.e., covariate shift and label shift). Finally, we compare our theoretical bounds to deterministic models of distribution shift and against real-world data, finding that we are able to estimate fairness violation bounds in practice, even when simplifying assumptions are only approximately satisfied.
Introduction
Distribution shift a common, real-world phenomenon that affects machine learning deployments when the targel distribution of examples (features and labels) ultimately encountered by data- driven policy diverges from the source distribution it was trained for. For socially consequential decisions guided by machine learning, such shifts the underlying distribution can invalidate fairness guarantees and cause harm by exacerbating social disparities. Unfortunately, distribution shift can be technically difficult or impossible model at training time (e.g., when depending on complex social dynamics or unrealized world events). Nonetheless, we still wish to certify the robustness of faimesss metries for a policy on possible target distributions.
In this paper, we provide a general framework for quantifying the robustness of statistical group fairness guarantees. We assume that the target distribution is adversarially drawn from bounded domain, thus reducing the hard problem modelling distribution shift dynamics to a more tractable, static problem. With this framework, can detect potentially inappropriate policy applications, prior to deployment, when fairness violation bounds are not sufficiently small.
This work bridges gap between recent literature domain adaptation, which has largely focused on the transferability of prediction accuracy (rather than fairness), and algorithmic fairness, fairiss, which
These authors contributed equally to this work. Corresponding author; yangliu @ @ucse.edu
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

has typically considered static distributions or prescribed models of distribution shift. Our work is the first to systematically bound quantifiable violations of statistical group fairness while remaining agnostic to (1) the mechanisms responsible for distribution shift, (2) how group-specific distribution shifts are quantified, and (3) the specific statistical definition of group faimess applied.
Our primary result is bound on a policy's potential "violation of statistical group hairesss-define in terms of the differences in policy outcomes between groups-whtn when applied to a target distribution shifted relative to the source distribution within known con- straints. Such settings naturally arise whenever train- ing data represents a random sample of target pop- ulation with different statistics or a sample from dy- namic environments, when policy is reused a new distribution without retraining, or whenever pol- icy deployment itself induces distribution shift. As an example of this last case, strategic individuals seek- ing loans might change their features abstain from future application (thus shifting the distribution of ex- amples) in response to policies trained on historical data 18 38 43 Beyond policy selection, exogenous pressure such as economic trends and noise may also drive distribution shift in this example.
CA
MI
PA
of Denoorptee
WA
0.05
0.00
Prod
Figure 1: In Section 7 we evaluate our bounds against historical, temporal distribution shifts in demographics and income recorded by the US Census Bureau [3]. The above figure depicts changes to income- predocctreeiiioon accuracy and demo- graphic parity violation when classifier initially trained on US state-specific demographic data for 2014 is reused 2018 data, thus exemplifying the negative potential effects of distribution shift.
In Figure [ we show how real-world distribution shift in demographic and income data for US states be- tween 2014 and 2018 may increase faimess violations while decreasing accuracy for a hypothetical classifier trained on the 2014 distribution. In such settings, it is useful to quantify how fairness guarantees transfer across distributions shifted within some bound, thus allowing the deployment of unfair machine learning policies to be avoided.
1.1 Related Work
Our work considers a setting similar to recent studies of domain adaptation, which have largely focused on characterizing the effects of distribution shift on prediction performance rather than fairness. Our work also builds on efforts in algorithmic fairness, especially dynamical treatments of distribution shift in response to deployed machine learning policies 23 [ 30 We reference
specific prior work in these domains in Appendix B, and here discuss existing work that focuses on how certain measures of fairness are affected when policies are subject to specific distribution shift.
Fairness subject to Distribution Shift: A number of recent studies have considered specific examples of fairness transferability subject to distribution shift [34 9 36 31 21). In particular. Schumann et al. [4] examine equality of opportunity and equalized odds as definitions of group fairness subject to distribution shifts quantified by an H-divergence function; Coston et al. (9] consider demographic parity subject to a covariate shift assumption while group identification remains unavailable to the classifier; Singh et al. 36 focus on common group fairness definitions for binary classifiers subject to a class of distribution shift that generalizes covariate shift and label shift by preserving some conditional probability between variables; and Rezaei et al. BI similarly consider common binary classification fairesss definitions such as equalized odds subject to covariate shift. While we address similar settings to these works as special cases of our bound, we propose a unifying formulation for a broader class of statistical group fairness definitions and distribution shifts. In doing so, we recognize that particular settings recommend themselves to more natural measures of distribution shift, providing examples in Section 4.1 Section 4.2 and Section 5
Another thread in existing literature is the development of robust models with the goal of guaranteeing fairness on modelled target distribution (e.g., 32 26 3 (1)). for example, by assuming covariate In particular, Singh et al. (36) focus on learning stable models that will preserve prediction accuracy and fairness, utilizing a causal graph to describe anticipated distribution shifts. Rezaei et al. [1 takes a robust optimization approach, and and farget-fair covariate shift method for getting the
shift and the availability of some unlabelled target data 9 36
Coston et al. [9] develops prevalence-constraie robust model. In contrast, our goal is to quantify fairness violations after an adversarial distribution shift for any given policy, including those not trained with robustness in mind.

---

1.2 Our Contributions
Our primary contribution is formulating general, worst-case upper bound for given policy's violation of statistical group fairness subject to group-dependent distribution shifts within presupposed bounds (i.e., Equation (9)). Bounding violations of fairesss subject to distribution shift allows us to recognize and avoid potentially inappropriate deployments of machine learning when the potential disparities of prospective policy eclipse given threshold within bounded distribution shifts of the training distribution.
We first characterize the space of statistical group fairness definitions and possible distribution shifts by appeal to premetric functions (Definition 2.T After formulating the worst-case upper bound, we explore common sets of simplifying assumptions for this bound as special cases, yielding tractable calculations for several familiar combinations of fairness definitions and subcases of distribution shift (Theorem 4.1 Theorem 5.2) with readily interpretable results. Finally, we compare our theoretical bounds to prescribed models of distribution shift in Section and to real-world data in Section 7 The details for reproducing our experimental results can be found at attps://githul: com SEtp/UCSC-RRALLE REA--ERRLL/Faaa Faith///aaeeesssaa BaaeeessTTraateee Trans frrabiirybl
2
Formulation
The appendices include table of notation (Appendix A) and all proofs (Appendix F).
2.1 Algorithmic Prediction
We consider two distributions, S (source) and T (target), each defined as a probability distribution for examples, where each example defines values for three random variables: X, feature (e.g., x) with arbitrary domain X: Y, a label (e.g., y) with arbitrary domain V: and G. a group (e.g., or h) with finite, countable domain G. The predictor's ' policy æ, intended for S but used on T, defines fourth variable for each example: viz., Y, a predicted label (e.g., ú) with domain y =yy
Using P.-) to denote the space of probability distributions over some domain, we denote the space of distributions over examples as I) == P(x x y x 9), such that T E D. It will also be useful for us to notate the space of distributions over example outcomes associated with a given policy as O = P(X' x y x )) and the space of distributions over of group-specific examples as G = P )).
Without loss of generality, we allow the prediction policy क to be stochastic, such that, for any combination (I, the predictor effectively samples Ý from a corresponding probability distribution [2, g). Stochastic classifiers arise in various constrained optimization problems and proven useful for making problems with custom losses or fairness constraints tractable [10 17 40).
We denote the space of nondeterministic policies as = (X g - P())) (e.g., क E II) and utilize the natural transformations that relate the spaces of distributions ID, policies II, and outcomes O:
Pr - G=g)
Pr
(Ý=Ú)- Pr (X=x,G=))
(1)
{~T((zg)
X.G
We abuse the Pr notation for both probability density and probability mass functions as appropriate.
2.2
Statistical Group-Fairness
We next define broad class of disparity functions A : II D + R representing how "unfair" given policy is for a given distribution (e.g., writing A A T)), noting that this notion of fairness is limited capturing statistical discrepancies of outcomes between groups.
Definition 2.1. We define a premetrid v on the space of distributions with respect to by the properties (p || g) > 0 and (p || p) for all p, q, and refer the value of v as a "shift".
Definition 2.2. We define a statistical group disparity A for policy π and distribution T in terms of the symmetrized shifts between group-specific outcome distributions. We measure shifts between outcome distributions with given premetric V: 0² + R.
A (T, T) = Σ V (,,,X, G-9) | P | G-h)
(2)
n,
n,
g.bgg
'Despite use on Wikipedia, this is not a standard term in the literature. In general, the axioms of a premetric defined in Definition are are subset (thus "pre") of those that define metric.
3

---

In Definition 2.2 v quantifies the specific statistical differences in outcomes between groups that are "unfair", where a value of 0 implies perfect fairess.. In this work, we assume that the same for all g, h and that A is insensitive to relative group size Pr(G)).
Examples Familiar applications of Definition include demographic parity (DP) and equalized odds (EO). A policy satisfying DP, in expectation, assigns a given binary classification y E {0. 13 to the same fraction of examples in each group. We may measure the violation of DP as
A DP T) = Σ
Pr
| G-g) - Pr G-
(3)
x.7
g.hEG
The associated premetric YP for p,g E is 4p q) (r,((=1) Pr (Y
To satisfy EO, for binary y = [0,1]; must maintain group-invariant true positive and false positive classification rates. We may measure the violation of EO as
A A'EO(T,
E Σ
Pr F.T (r G=g. Pr (a-1, J
(4)
g.hEG DEY
The associated premetric is WEo(p () = Eney Pr - -1 Y yvy Note
Pr, (r,
that the restriction of EO to the (Y = 1) case is known as Equal Opportunity (EOp).
We remark that Definition 2.2 provides unifying representation for a wide array of statistical group "unfairness" definitions and may be used with inequality constraints. That is, we may recover many working definitions of fairness that effectively specify maximum value of disparity:
Definition 2.3. A policy и is c-fair with respect to A distribution T iff A (T, Sc.
2.3 Vector-Bounded Distribution Shift
Suppose, after developing policy π for distribution S, we realize some new distribution T on which the policy is actually operating. This realization may be the consequence of sampling errors during the learning process, strategic feedback to our policy, random processes, or the reuse of our policy on new distribution for which retraining is impractical. Our goal to bound A A T) given knowledge of A (T,, S) and some notion of how much T possibly differs from S.
Definition 2.4. K(p || q) is divergence if and only if for all p and q, K(p 1) 2 and K(p || 2) = > Definition 2.5. Define the group-vectorized shift D, as S mutates into T, as
p.
D(T |1 S)
(X (Pp(X G=g) || Pr(X, Y G=9))
(5)
5
where e, represents a unit vector indexed by g. and each D,: G2 - R a divergence (Definition 2.4 Note that each D, also defines premetric (but not necessarily divergence) on D. Assumption 2.6. Let there exist some vector B bounding D(T S) = B, where = and > denote element wise inequalities.
In Assumption 2.6 B limits the possible distribution shift as S mutates into T, without requiring us to specify model for how distributions evolve. When modelling distribution shift requires complex dynamics (e.g., when agents learn and respond to classifier policy), we reduce potentially difficult dynamical problem to a more tractable, adversarial problem to achieve a bound.
Lemma 2.7. For all n, A and D. when B = 0, (T S) = (TT
Lemma 2.7 indicates that, for fixed policy change in disparity requires a measurable shift in distributions from to T, confirming intuition.
Restricted Distribution Shift Common assumptions that restrict the set of distribution shifts include covariate shift and label shift. For covariate shift, the distribution of labels conditioned on features is preserved across distributions for all groups, while for label shift, the distributions of features conditioned on labels is preserved across distributions for all groups
Covariate shift implies Pr(Y X, X,G)
Pr(Y X,G)
(6)
T
5
Label shift implies Pr(X Y,G) = Pr Y.G) 3
(7)
7

---

In Section 4 we explore a deterministic model of population 's response to classification as an example covariate shift. We do the same in Section for label shift.
General Bounds
We first define primary bound in Definition 3.1 before considering simplifying special cases.
Given an element-wise bound B on the vector-valued shift D || S) (Assumption 2.6) we may bound the disparity A of policy
any realizable target distribution T by its supremum value.
Definition 3.1. Define the supremum value for A subject to (T |S) B as
o( D,D,,,,,,),
sup (T, T) D(T 8)
D(T S) =) 3BBBB - (T, D.
In general, our strategy is to exploit the mathematical structure of the setting encoded by A' (i.e., V) and D obtain an upper bound for v defined in Equation (8). We first explore general cases of simplifying assumptions before presenting worked special examples for frequently encountered settings. Finally, we compare the resulting theoretical bounds numerical results and simulations.
3.1 Lipshitz Conditions
The value of defines scalar field in B and therefore a conservative vector field F VBU.
For any curve in from S to T, bounds of the form F 3 L for some constant L along the curve imply Lipshitz bound on A'. We visualize a bound in Figure for all possible curves in the region D(T || S) 3B. B.
Theorem 3.2 (Lipshitz Upper Bound). If there exists such that Vbv(A*, D, n, S, b) < L, everywhere along some curve as varies from B, then
A (T, T) E (T, S) L
(10)
Figure 2: Lipshitz bound for all curves parameterized by distribution shift bound in the (0. B) D-hyperrectangle on the sur- face v. the figure, for groups i (g. h). max θ,v L,, and the colored dotted lines corresponds to L,b,, which, when summed, equal L B.
Succinctly, if we are guaranteed that disparity can never increase faster than certain rate in some measure of distribution shift, then, given a maximum distribution shift, this rate bounds the maximum possible disparity. The utility of Theorem [3.2] arises when a Lipshitz condition L is known, but direct computation of is difficult. We provide an example of a Lipshitz bound in Section 5
3.2 Subadditivity Conditions
Definition 3.3. Define w the maximum increase in disparity subject to D(T || w(A wAA D, n. S, v(A D, n. S, (T,
S) = B, i.e.
Theorem 3.4. Suppose, in the region D(T S) B, that is subadditive in its last argument. That is, w(... a) w(..., c) 2 w(...,a + c. for a, 2 and + c 3 B. If w is also locally differentiable, then a first-order approximation of w(..., b) evaluated
i.e..
L = Vpw( b) b)lb= = Vbo(.., b)
(11)
provides an upper bound for vi.., B), i.c.
v A D. n. D. S.) (),
(12)
Theorem 3.4 notes that "diminishing returns" in the change of A the difference of T with respect to S is increased implies a bound on A terms of its local sensitivity to D at S (i.e., using a first- order Taylor approximation). Note that, if is concave in the bounded region, it is also subadditive in the bounded region, but the converse is not true, nor does the converse imply Lipshitzness.
5

---

3.3 Geometric Structure
may happen that v: 0 + R and each Dg: G² + R share structure that permits geometric interpretation of distribution shift. While the utility of this observation depends on the specific properties of v and D, we demonstrate a worked example building on Section Appendix D, in which we allow ourselves to select suitable D for ease of interpretation. We proceed to consider worked examples that adopt common assumptions limiting the form of distribution shift and apply common definitions of statistical group fairness.
4
Covariate Shift
We now present our faimess transferability results subject to covariate shift for both demongraphic parity (Section 4.1 and equalized opportunity (Section 4.2) as faimess criteria.
4.1
Demographic Parity
The simplest way to work with Equation 8 is to bound the supremum v. We first consider demo- graphic parity (Equation 33) for y (0. 1 and g = {g, h], subject to covariate shift (Equation ()) We find that the form of Aop subject to covariate shift recommends itself to natural choice of vector
divergence, D. First, define re-weighting coefficient Wy(T, "T, x) = Pro(X==[G=a)
Ptr(X==|G=g)
Theorem 4.1, For demographic parity between two groups under covariate shift (denoting, for each 9g = PYa,s(Y=1 G-g)),
A AP(TT, T) A DP(T, S)+ + Σ (3,(1- Varlw)(T,S,
(13)
We notice that Vars(w, (T,, recommends itself as a suitable divergence Dg from S to T. Using basis vectors eg. for this example, we could define D(T || 5) Eg Og Vars (T,
When Vars[w, S.)] (T, - Bg it follows A (T,, 7) < (F) E, (3, - -,),B,) Comparing the inequality in Theorem 4.1 and the consequent Equation (10). we can interpret Pr,,s (Y=,s(Y=1)) in Theorem 4.1 as an upper bound for the average value of V DP. D. n, T, b) along any curve from S to T. Interpreting this result, the closer Pr(Ý=1) is to 0.5 for any group, the more potentially sensitive the fairness of the policy to distribution shifts for that group. We can further generalize the results to multi-class and multi group setting:
Corollary 4.2. Theorem 4.1 may be generalized to multiple classes y = (1, 2, m) and multiple Pr
groups G E (1, .......,, ..., n., where Ba.v
y -y G=9) and assuming Vars (W, (T,S, By:
Abp(T, T)=
Σ DEY g.hs
Σ
Pr (Ý =y G=9) Pr Pr G=h) T.T
(14)
n.
(3g. - (1-3) A A'DP(T, T) < A' DP(TT, S) + Σ
) Bg) 3/2
(15)
y
g
We remark that in general, binary classification bounds may frequently be generalized to multi-class bounds by redefining fairness violations as a sum of binary-class fairness violations (i.e., same-class vs. different-class labels) and summing the bounds on each.
4.2 Equal Opportunity
Consider an example using the (Yoni)-conditioned case of Equalized Odds termed Equal Opportu- nity (EOp). Denoting, for each group g. the true positive rate B = (Y= Y=1, G=g) as an implicit function of π and T. we define disparity for EOp as A EO, T) = Eg.vgg 18 1/t
We may bound the realized value of A EO, T) by bounding 3+ for each group: Theorem 4.3. Subject to covariate shift and any given D, B, assume extremal values for B i.e.
Vg, (T S1 < BB) (Lg S 0 (T,, T) S ug)
(16)

---

it follows that
v(A EOp. D, n, S,
max
Σ 2ig
(17)
E Mu Ja (1
9.h
Corollary 4.4. The disparity measurement A EOp cannot exceed 10
In Appendix D, we bound the extremal values of 8 by geometrically interpreting this quantity an inner product on an appropriate vector space, utilizing the freedom to select an appropriate D.
Label Shift
Under label shift (Xrs(X - PrT((X))), violations of EO and EOp are invariant, because the independence of y and Y given X implies Prr,T(Y||)
(Y We therefore focus on the violation of demographic parity (DP) (Equation (3)) subject to the label shift condition, treating binary classification task over two groups for simplicity.
In this setting, we choose to measure group-specific distribution shifts from T by the change in proportion of ground-truth positive labels, which we refer to as the group qualification rate (7) = Pr = G g):
(T - 1118)) Q,()) s Bg
(18)
Theorem 5.1. A Lipshitz condition bounds Vbr(A* D, n, S b) when
(T = Q2(5) =
Q₂(T) Bg
(19)
Specifically,
θ v(A DP. D, S, - (19) 1) - 3/
(20)
0bg
for true positive rates B5 and false positive rates By:
8 = (11--|Y-1,G--j)) 87 P:--(-111Y-00
(21)
Because 35 and 8 are invariant under label shift given constant policy æ, we elide their explicit dependence on the underlying distribution.
Theorem 5.2. For DP under the bounded label-shift assumption Vg. (Qg
Qg(T) By.
Abp(T, T) S Abp(T, S) + (( - Σ B 3 33g
(22)
Intuitively, the change in A 'P subject label shift depends on 1 - the marginal change in acceptance rates as agents change their qualifications Y. We measure the distribution shift as agents change their qualifications by (8) Q, (T)). When 35 close 3, the policy looks like random classifier, and a label shift has limited effect on statistical group disparity. When |3 S is large, indicating high classifier accuracy, the effect on supremal disparity is larger. Our bound thus exposes a direct tr between accuracy and fairness transferability guarantees.
6 Comparisons to Synthetic Distribution Shifts (Demographic Parity)
To further interpret our results, this section, we consider specific and popular agent models to characterize distribution shift and instantiate our bounds for particular forms of D, B, and A
6.1
Covariate Shift via Strategic Response
Let us consider specific example of covariate shift (Equation (6) caused by deterministic, group- independent model of strategic response in which agents react to binary classification policy π characterized by group-speet speeisie feature thresholds:
{}
with probability 1 if Tg
(23)
with probability otherwise

---

For simplicity, we assume the feature domain A' = [0, 1]. In response to threshold g. agents in each group g may modify their feature a to x' by incurring a cost eg (z. x²) 0. Similar [8] we define the utility Mg for agents in group y to be
u1(x,z')= = 3y(x)) - [z(2) - (z(2x By(x) = Pr Pr(ff Pr(Y=11 X=x, -=9) Vg.
(24)
Contrary to the standard strategic classification setting, we do not assume that feature updates represent false reports, but that such updates may correspond to actual changes underlying the true qualification Y of each agent. This assumption has been made in a recent line of research in incentivizing improvement from human agents subject to such classification [5]
Next, we assume all agents are rational utility maximizers (Equation (24). For a given threshold To and manipulation budget mg. the best response of an agent with original feature æ is
s' = argmax uz(2, z), such that Ca 2) S mg
(25)
To make the problem tractable, we make additional assumptions about the agents' best responses. Assumption 6.1. An agent's 's original feature x is sampled as X ~ Mi.
Assumption 6.2. The cost function (2 x²) is monotone in |x x'| as Cg (I, x') = |x' z/.
Under Assumption 6.2 only those agents with features a E ITg 1Tg--M9 mg Ty) will attempt to change their feature. We also assume that feature updates are deneeeeemiiistic, such that agents with features closer to the decision boundary Tg have a greater chance of updating their feature and each updated feature x' is sampled from a uniform distribution depending on Tg. mg. and z:
Assumption 6.3. For agents who attempt to update their features, the probability of a successful feature update is Pr(X # X') = [x
m2
Assumption 6.4. An agent's 's updated feature x', given original feature x, manipulation budget my. and classification boundary Tg. is sampled as X" ~ (1 -x]-
With the above setting, we can specify the reweighting coefficient wi)()) for our setting (Equa- tion (102) in Appendix F.1 and get the following bound for the strategic response setting])
Proposition 6.5. For our assumed setting of strategic response involving DP for two groups {g, h), Theorem implies
2
(T, T) I Abp(T, S) +
-1- - -g)) T1
Tm) mm
(26)
The above result shows that two factors lead to smaller difference between the source and target fairness violations: less stochastic classifier (when the threshold Tg far away from 0.5) and smaller manipulation budget my (diminishing agents' ability to adapt their feature). In this case, Bg (1
(1- - Tg)) These factors lead to less potential manipulation and result in a tighter upper bound for the fairness violation on T.
6.2 Label Shift via Replicator Dynamics
We now evaluate theoretical bound for demographic parity subject to label shift (Theorem 5.2 on the replicator dynamics model of Raab and Liu (30). Briefly, replicator dynamics assumes that the proportion of agents in population choosing one strategy over another grows in proportion to the ratio of average utilities realized by the two strategies. The cited model additionally assumes
a Pts(X=z[Y=1) R. R/ (0, }}. and monotonicity condition for S given by dæ (r=(X= > 0.
X =
Label shift under the discrete-time (t) replicator dynamics may be expressed in terms of group
qualification rates Q2 == (Y= G G=g) and agent utilities (i.e., group- and -indeee-indeeendentt values Un. such that, in each group, the popularity and average utility associated with a label determines its frequency at the next time step 1+1.
' 'where represents the uniform distribution.
Seee Figure in Appendix C for demonstration of Theorem 4.1
8

---

Thoortal Boand
(a) A stereoscopic (cross-eye view) comparison be- tween the bound of Section 4.1 (gradated) and sim- ulated results for the model of Section 6. (bluue in response fair classifier with different initial croup-independent acceptance rates.
(b) A policy satisfying DP is subject to distribution shift prescribed by replicator dynamics (Section 6.2). Realized disparity increases (blue) are compared the thcoretical bound (Theorem 5.2 gradated), which is tight when group have dissimilaa qualification rates.
Figure 3: Comparisons to synthetic distribution. Larger versions are provided Appendix
Denote the fractions of croup-conditioned, feature innluurreiddeeddeent outcomes with the expression py:0 = Pr,(F=ū, Y-y G-g) and abbreviate the fraction- fraction-weightedd utility as (1) = Uv.9 p.. We may then represent the replicator dynamics as
1. + (t)
Qgit +
1]
(27)
Uig
(t)
+ Ug (1) + ui (t) + Ug (t)
0.0 0.1
To apply Theorem 5.2 we also observe that 1 3 1 - 1ith where 35 and 3y represent the +
true positive rate and talse positive rate for group g. respectively, and we use the change in qualification rate as our measurement of label shift, i.e., B, = 1Q [ 1] - Q,14]I. When demographic parity is perfectly satisfied, we note that the acceptance rate (pg) 1 + pg is group-independent.
Theorem 6.6. For DP subject to label replicator dynamics,,
10g1 1.1
Pg
ADP(T, T) - Abp(T, S) + 11
(28)
Pg
+ Pg
0.1
In Figure 3(b) we graphically represent all possible states of an initially fair system (thus determining B and result of the monotonicity condition) by the tuple of qualification rates for each group. With the dynamics prescribed by Equation (27). we depict the rate of change of disparity given a fixed, locally farfair policy, and compare this to the theoretical bound when B3 Q2
1,1
Interpreting our results, we note that the bound lacks information about the relative directions of the change in acceptance rates for each group, and thus over-approximates possible fairness violations when group acceptance rates shift the same direction. When group acceptance rates move in opposing directions, however, the bound gives excellent agreement with the modelled replicator dynamics.
Comparisons to Real- Distribution Shifts
We now compare our special-case theoretical bounds (i.e., label/covariate shift) to real-world distribu- tion shifts and hypothetical classifiers. We use American Community Survey (ACS) data provided by the US Census Bureau [16]. We adopt the sampling and propesccssiing approaches following the Folktables package provided by Ding et [13 obtain 1,599,229 data points. The data partitioned by (1) all fifty US states and (2) years from 2014 4 to 2018. We use 10 features covering the demographic information used in the UCI Adult dataset 44, including age, occupation, education, etc., as X for our model, select sex binary protected group, i.e., G E {g- female, h male). We set the label Y to whether an individual's annual income is greater than $50K.
To apply our label-shift or covariate-shift bounds, we first need to verify whether the two datasets satisfy either of these assumptions. We adopted a conditional independence test [22), which takes data from source and target domains as input and returns a divergence score for each covariate and label variable, reflecting to what extent the variable shifted between distributions. We find that the likelihood that the covariates shift across US states is approximately two orders of magnitude higher
'This package is available httt::///ggtauub..m/////////////

---

than for labels. More specifically, there are 4 covariates, including class of worker (probabilistic divergence score of 2.67e-2), hours worker per week (3,56e-2), sex (3.56e-2) and race (2.55e-1), 1). that are more likely to be shifted than the label variable (1.2% (1.29e-4). For temporal shifts within states, we find that the label variable more likely to be shifted (0. 1) than all the other covariates (which are below 0.01), approximately two orders of magnitude in favor of label shift over covariate shift. We therefore compare the disparities of hypothetical policies on these distributions to bounds generated from the corresponding, approximately satisfied assumptions.
On this data, we train set of group-dependent, linear threshold classifiers PTo((.g)) (Y=1) = 1[o -r) > Tyl. for range of thresholds To and тд for each source distribution. Here, a(.) is the logistic function and w denotes weight vector. We then consider two types of real-world distribution shift: (1) geographic, in which model trained for one state is evaluated on other US state in the same year, and (2) temporal, in which model trained for 2014 is evaluated on the same state in 2018.
(a) CA - IL
(b) CA NV
(c) CA 2014 2018
(d) TX: 2014 2018
Figure 4: Simulated change in DP violation (blue mesh) subject to geographic and temporal distribution shifts vs. direct application of bounds for approximately satisfied assumptions (respectively, Theorem 4.1 and Theorem 5.2 (gradated mesh). The z-axis and y-axis of both figures represent the policy thresholds Tg and Th-
We graphically compare the theoretical bounds of Theorem 4.1 and Theorem 5.2 for the increased violation of DP subject to covariate and label shift, respectively, to the simulated violations for our model and data in Figure 4 We provide additional examples and an evaluation bounds for EO subject to covariate shift (noting that label shift preserves EO in theory) in Appendix E.2. Despite the fact that geographic or temporal distribution shifts only approximately satisfy the assumptions of covariate label shift, these comparisons demonstrate that our theoretical bounds are not vacuous, approximately bounding the change of fairess violation across real-world domain shifts. For geographic shifts, the covariate shift EO bounds (Appendix E.2) correctly overestimate disparity and tighten near accurate policies, while our DP bounds are useful only for a subset of policy thresholds (Figure ) add specific pointer. e.g. 4.a, that one is 4.b For temporal shift, the label shift bound for DP correctly overestimates the real change of DP violations but still remains at the same order of magnitude (Figure 4[c) and 4dd).
Conclusion and Discussion
In this paper, we have developed a unifying framework for bounding the violation of statistical group fairness guarantees when the underlying distribution shifts within presupposed bounds. We hope that this work can generate meaningful discussion regarding the viability of fairness guarantees subject to distribution shift, the bounds of adversarial attacks against algorithmic fairness, and evaluations of robustness with respect to algorithmic fairness. We believe that, just as published empirical measurements are of limited use without reported uncertainties, fairness guarantees must be accompanied by bounds on their robustness to distribution shift.
Future work remains to apply our framework for to problem of faimess transferability in settings with more complicated distribution shift dynamics. For example, compound distribution shifts 33), which compose covariate shifts and label shifts, cannot be treated by composing the thcoretical bounds developed herein without additional information regarding intermediate distributions. Another potential future direction is to develop reasonable bounds on anticipated distribution shift from models of human behavior and exogenous pressures.
Acknowledgement This work is supported by the National Science Foundation (NSF) under grants IIS 24- 143895, IIS-2040800 (FAI program in collaboration with Amazon), and CCF-2023495.
10

---

References
[1] Bang An, Zora Che, Mucong Ding, and Furong Huang. Transferring fairness under distribution shifts via fair consistency regularization. arXiv preprint arXiv:2206. 2022.
[2] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Vaughan. A theory of learning from different domains. Machine Learning, 79:151-1755 2010
[3] Arpita Biswas and Suvam Mukherjee. Ensuring fairness under prior probability shifts. In Proceedings of the 2021 AAAT/ACM Conference on AI, Ethics, and Society, pages 414-424, 2021.
[4] Catherine Blake. Uci repository of machine learning datahases. 1998.
[5] Yatong Chen, Jialu Wang, and Yang Liu. Linear classifiers that encourage constructive adapta- tion. In Algorithmic Recourse workshop at ICML'21, 2021.
[6] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153-163, 2017.
[7] Stephen Coate and Glenn C Loury. Will affirmative-actton action policies eliminate negative stereo- types? The American Economic Review, pages 1220-1240, 1993.
[8] Sam Corbett-Daviee Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pages 797-806, 2017.
[9] Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R. Varshney, Skyler Speakman, Zairah Mustahsan, and Supriyo Chakraborty. Fair transfer learning with missing protected attributes. In Proceedings of the 2019 AAAT/ACM Conference on Al, Ethics, and Sociery, AIES '19. page 91 91-98, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 978 4533644233 doi: 10. N0.1145/330661 8.33 https:////d/ org/ 10.1145/3306618. 3314236.
[10] Andrew Cotter, Maya Gupta, and Harikrishna Narasimhan. On making stochastic classifiers deterministic. In Advances in Neural Information ProiiionnnPreeee Processing Systems, volume 32. Curran Associates, Inc., 2019.
[11] Elliot Creager, David Madras, Toniann Pitassi, and Richard Zemel. Causal modeling for fairness in dynamical systems. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR .org, 2020.
[12] Alexander D' Amour, Hansa Srinivasan, James Atwood, Pallavi Baljekar, D Sculley, and Yoni Halpem. Fairness is not static: deeper understanding of long term faimesss via simulation studies. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 525 5344,34,22 2020.
[13] Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021. URL.tttps:: //tp:rrevieeeeeeeeee http://ppwaiiiieeeeeeeeaaaaaaaaa
2708mKK
[14] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedingy of the 3rd innovations in theoretical computer science conference, pages 214-226, 2012.
[15] Michael Feldman. Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference knowledge discovery and data mining, pages 259-268. 2015.
[16] Sarah Flood, Miriam King, Renae Rodgers Steven Ruggles, and J. Robert Warren. Integrated public use microdata series, current population survey: Version 8.0 [dataset], 2020. URL https: //www//www../ ipums. ottp/ppoojectss/psssss/pp//////p v

---

[17] Nina Grgić-Hlača, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. On fairness, diversity and randomness in algorithmic decision making, 2017.
[18] Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. Strategic classifi- cation. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, page 111-122, New York, NY, USA, 2016. Association for Computing Machinery.
[19] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315-3323, 2016.
[20] Lily Hu and Yiling Chen. short-term intervention for long-term fairness in the labor market. In Proceedings of the 2018 World Wide Web Conference on World Wide Web, pages 1389-13988 International World Wide Web Conferences Steering Committee, 2018.
[21] Mintong Kang, Linyi Li, Maurice Weber, Yang Liu, Ce Zhang, and Bo Li. Certifying some distributional fairness with subpopulation decomposition. arXiv preprint arXiv:2205.15494, 2022.
[22] Sean Kulinski, Saurabh Bagchi, and David Inouye. Feature shift detection: Localizing which features have shifted via conditional distribution tests. In H. Larochelle, M. Ranzato, R. Had- sell, M.F. Balcan, and H. Lin, editors, Advances Neural Information Processing Systems, volume 33, pages 19523-19533. Curran Associates, Inc., 2020. http::://prcccee//// pdf
[23] Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In International Conference on Machine Learning, pages 3150-3158. PMLR, 2018
[24] Lydia T Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer Chayes. The disparate equilibria of algorithmic decision making when individuals invest rationally. In Proceedings of the 2020 Conference on Fairness, Accountability, and Trunsparency, pages 381-391, 2020.
[25] Yang Liu, Yatong Chen, Zeyu Tang, and Kun Zhang. Model transferability with responsive decision subjects, 2021.
[26] Debmalya Mandal, Samuel Deng, Suman Jana, Jeannette Wing, and Daniel J Hsu. Ensuring fairness beyond the training data. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18445-18456. Curran Associates, Inc., 2020.
[27] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms, 2009.
[28] Hussein Mouzannar, Mesrob Ohannessian, and Nathan Srebro. From fair decision making to social equality. In Proceedings of the Conference on Fairness, Accountability, and Trunsparency, pages 359-368. ACM, 2019.
[29] Harikrishna Narasimhan. Learning with complex loss functions and constraints. In Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 1646-1654. PMLR, 09-11 Apr 2018.
[30] Reilly Raab and Yang Liu. Unintended selection: Persistent qualification rate disparities and interventions. Advances in Neural Information Processing Systems, 34, 2021.
[31] Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian D. Ziebart. Robust fairness under covariate shift. In AAAI, 2021.
[32] Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. Sample selection for fair and robust training. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 8 815-827. Curran Associates, Inc., 2021.
12

---

[33] Jessica Schrouff, Natalie Harris, Oluwasanmi Koyejo, Ibrahim Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alex Brown, Subhrajit Roy. Diana Mincu, Christina Chen, Maintaining fairness across distribution shift: do we have viable solutions for real-world applications? arXiv preprint arXiv:2202.01003, 2022.
[34] Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed H. Chi. Transfer of machine learning fairness across domains, 2019.
[35] Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
[36] Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara. Faimness violations and mitigation under covariate shift. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, New York, NY, USA, 2021. Association for Computing Machinery.
[37] Masashi Sugiyama, Taiji Suzuki, Shinichi Nakajima, Hisashi Kashima, Paul von Bünau, and Motoaki Kawanabe. Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60(4):699-746, 2008.
[38] Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classification. In Proceedings of the Conference Fairness, Accountability, and Transparency, pages 10-19, 2019.
[39] Min Wen, Oshert Bastani, and Ufuk Topeu. Fairness with Dynamics. arXiv preprint 1rX1v:1901.08568, 2019.
[40] Jimmy Wu, Yatong Chen, and Yang Liu. Metric-fai classifier derandomization. Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, PMLR, 17-23 Jul 2022.
[41] Rich Zemel, Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork, Learning fair representa- tions. In International conference machine learning, pages 325-333. PMLR, 2013.
[42] Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. International Conference on Machine Learning, pages 819-827. PMLR, 2013.
[43] Xueru Zhang, Ruibo Tu, Yang Liu, Mingyan Liu, Hedvig Kjellström, Kun Zhang, and Cheng Zhang. How do fair decisions fare in long-term qualification? In NeurIPS, 2020.
13


Reviews:
[{"review_id": "ufIhQ1lzvod", "reviewer": "Reviewer_84wj", "summary": "This paper considers fairness under domain shift, specifically, bounding the change in the violation of group fairness criteria when the evaluation (target) data distribution differs from the one (source) on which the model is trained.\n\n- First, a framework/distribution shift assumption is proposed for studying this problem.  Namely, the domain shift should be bounded w.r.t. some probability (pre)metric.  Then the worse-case (or adversarial) fairness bound is considered under this assumption.\n- The authors derive some results under several simplifying assumptions in section 3, then work out the results for covariate shift and label shift in sections 4 and 5, respectively.\n- In section 6, two examples of distribution shifts are considered, which are instances of covariate and label shift.\n- Finally in section 7, a visualization of the bounds in sections 4 and 5 are provided on a real-world dataset, and are compared to the true change in fairness violation incurred by a model under simulated domain shift.", "questions": "My main questions are embedded in the section above.\n\n1. line 230 missing $x_h\\in\\{l_h,u_h\\}$ under $\\max$?\n", "limitations": "The problem studied is of interest to the field, but the potential impact of the present work may be limited given the assumptions required in the results.  I am willing to hear the opinions of the other reviewers.\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n\n1. The paper is complete, clearly written, and easy to follow.\n2. The problem studied, fairness under domain shift, is current and of importance to the field.\n\nWeaknesses.\n\n1. I feel that some of the results are weak.\n\n    - It is unclear whether the results in section 3 under the simplifying assumptions would be generally useful; as the authors pointed out on line 183 “direct computation of [terms in theorem 3.2] is difficult”, and the same could be said of theorem 3.4.\n\n        Although the proof of theorem 5.2 uses theorem 3.2, I feel that a direct proof with potentially simpler arguments is also possible, given the label shift assumption.\n\n    - I feel that the conclusion in theorem 4.3 is trivial given the assumption that the change in true positive rate is bounded.  Evaluating the change in TPR requires labels from the target distribution; is there a bound if this information is not available?\n\n2. It is not clear what is being visualized in fig. 4 in section 7.  Is the blue mesh the ground-truth change in DP, and the colormapped mesh the modeled change?  Are the left two plots produced assuming covariate shift, and the right two label shift?  Are there any remarks/comments on the results?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Sh7OFi8Jfrt", "reviewer": "Reviewer_ifHr", "summary": "This paper studies the transferability of group fairness under distribution shift which is an important problem in reality. Under bounded distribution shift, this paper develops a general bound for the fairness violation in the test time. It also derives bounds when considering particular fairness metrics (including demographic parity and equalized odds) and types of distribution shift (including covariate shift and label shift). \n", "questions": "1. The theoretical bounds depend on the source and target data distribution, but in practice, we only have finite samples. What’s the estimation error when we measure the bound in practice? Are there any factors, such as the data size, that affect the measurement in practice?\n2. Based on the theoretical bounds, do you have any ideas or suggestions for the algorithms to tackle the distribution shift? How to design policy \\pi so that to have a smaller upper bound?\n3. Does figure 4 indicates that the actual distribution shift is not a covariate shift, and the bounds become less useful in practice?\n\nMinor issue:\nIn equation (22), is there a typo under $\\max$? Do you miss $x_h$?\n", "limitations": "One limitation of this paper is that the theoretical bounds are based on some impractical assumptions which hinder their use in reality. I encourage authors to discuss some potential algorithms inspired by the theoretical bounds.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThis is the first paper that provides guarantees of fairness under bounded distribution shift. This paper considers multiple types of distribution shift and multiple fairness metrics. They also compare the derived theoretical bounds with the real change in fairness violation. The theoretical analysis can be applied to any models.\n\nWeakness:\n1. This paper has too many notations. I got lost many times. \n2. It would be better if there are more intuitions and explanations. For example, what’s the message we can get from theorem 3.2? Figure 2 is hard to understand without explanation.\n3. Many contents are delayed to the appendix, but the appendix is not included in the supplementary material. (Please point it out if I missed it.)\n4. I’m concerned about the practical use of the derived bounds. This is because the assumptions this paper considers including the particular distribution shift (covariate shift or label shift) and the availability of the target data are hard to be satisfied in reality. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KXrRzqIVUrL", "reviewer": "Reviewer_k7a3", "summary": "The paper studies the fair violations of predictive models under bounded distribution shifts. The main contributions are bounds on the change in fairness metrics (like demographic parity and equal opportunity) between train and test distributions that undergo either covariate shift or label shift. The bounds are further specialised to two applications in covariate or label shifts due to individuals strategically responding classifiers.", "questions": "Questions to address in the response\n\nHow can a practitioner use these to judge whether to use a model? Do the quantities included in the expressions (like Eq 17) plausibly known beforehand to estimate the max change in fairness?\n\nHow does the bounds or the analysis, in general relate to previous work? In what ways does the formulation unify those in the previous work, as claimed?\n\nWhat are the takeaways from Adult Income results in Figure 4? Since the real data shifts do not neatly fit in either covariate or label shift, how to interpret the difference between the empirical and theoretical changes? I would suggest estimating the type of shifts and doing the comparison between states or time periods which have close to covariate or label shift. Or, to perform a more controlled setting where distribution shifts are simulated but the training data is real. Conditional independence test can be used to detect the types of shifts such as the methods used in this paper https://proceedings.neurips.cc/paper/2020/hash/e2d52448d36918c575fa79d88647ba66-Abstract.html\n\nRelation of the results in Section 3.1, 3.2, and 3.3 to the results in Section 4 is not clear. Are the results in Section 4 special cases of these results?\n\n\nMinor points and suggestions (I do not expect a response)\n\nSome more related work\nhttps://proceedings.neurips.cc/paper/2020/hash/d6539d3b57159babf6a72e106beb45bd-Abstract.html\nhttps://arxiv.org/abs/2005.03474\nhttps://proceedings.neurips.cc/paper/2021/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html\nhttps://arxiv.org/abs/2206.12796\n\n\nNotation can be eased. For example, consider skipping \\Psi or reusing \\Delta for it since \\Psi is not used in the results.\n\nPlease consider discussing how to bound the fairness change when both covariate and label distributions shift.\n\nDescribe the divergence metric D_g in Eq (21).\n\nFigure 2 is not clear.\n\nDescribe the term replicator dynamics briefly.\n\nConsider explaining why the divergence between distributions p and q is named ‘pre’metric in Definition 2.1.", "limitations": "Limitations of the work are not discussed. Please discuss limitations of the way distribution shifts are characterized that is the class of shifts considered and the parameters to quantify the shift, discuss the applicability of the results to help in deployment decisions, and the experiments required to validate the results on real data.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Proposes a generic way to write down and analyze the maximum changes in fairness properties under shifts.\n- Results are original. Analysis for three common fairness metrics and two broad types of shifts.\n- The paper is well-written in general.\n\nWeaknesses\n- Significance of the bounds is not clear. The interpretation of the theoretical results does not describe how to apply them in practice.\n- The experimental results on Adult Income data are not summarized.\n\nThe list of results seem to have non-obvious implications, but these are hard to see. Please address the questions below that will help me to decide whether to improve the evaluation to 6 or 7. Particularly, convey the insights from one of the bounds in more detail.\n\n---\n\n## After the response\n\nThe response aptly addressed my concerns on the practical utility and interpretation of the results. So I have increase my score to 8, Strong Accept.\n\nThe work makes a great contribution on anticipating robustness of fairness guarantees under distribution shift. It may encourage further work on making the shift assumptions more interpretable and practically useful.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "-gkhZsVRfW", "reviewer": "Reviewer_fUG9", "summary": "This paper's primary contribution is a theoretical upper bound on certain fairness definitions, when the target distribution has drifted from the source distribution. The paper then works through how label shift and covariate shift would affect the bounds on certain fairness definitions, leading to a brief empirical analysis of the results. The following are the main contributions:\n\n1) The paper provides a theoretical result that shows an upper bound on the violations distribution shifts can achieve under several different fairness definitions. (Significance: High)\n\n2) The paper provides a walk through of how certain distribution shift characteristics can be analyzed with their framework. (Significance: Medium)\n\n3) The paper provides a unifying framework for several other related works trying to assess the impacts of distribution shift (Significance: High)", "questions": "1) Could you give a more granular description of Appendix E.2. and why it was relegated to the appendix instead of being written in section 7?", "limitations": "As this paper is theoretical, there doesn't seem to be any limitations to discuss. If anything, the paper provides upper bounds on the limitations of other papers that explore how distribution shifts affect the violation of fairness definitions.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Quality:\n\nThe paper is of high quality overall, but section 7 needs a bit more work. The paper suggests in the abstract that empirical justification would be given for both demographic parity and equalized odds. However, section 7 seems to only deal with the demographic parity definition without much discussion. Recommendations:\n\n1) Clarify the abstract as to what empirical justification will be given\n\n2) Bolster section 7 with either examples concerning equalized odds and/or give more discussion as to how to interpret the outputs of the empirical justification.\n\nPossibly, including details from Appendix E.2. in section 7 may solve this issue.\n\nClarity:\n\nOverall, the paper is clear, but the over reliance on notation makes it hard to read at times. Many definitions have overused notation (such as implication symbols) rather than plain English, making interpretation of the definition hard to gauge. As an example, please look at definition 2.4, where the if and only if symbol and plain English are both there.\n\nOriginality:\n\nBased on my knowledge of the field and the authors’ citations, I believe that this is original work that will move the field forward. However, I do find that some critical dynamics in fairness papers are missing from the citations, such as Liu et. al. 2018 Delayed Impact of Fair Machine Learning and Creager et. al. 2020 Causal Modeling for Fairness in Dynamical Systems, which takes papers like Liu et. al. 2018 and simplifies them into a single framework as well. These citations will provide more credibility and offer other avenues for unifying frameworks for dynamics in fairness.\n\nSignificance:\n\nThis paper provides a significant result for the field that could provide a general bound for how much dynamics can violate fairness definitions on a static dataset. Although I am unsure right now how to use it in practice, I believe having these theoretical tools will be helpful in the future. However, the paper could be made more legible by decreasing the usage of notation. Moreover, having a more robust narrative in section 7 will show the results in the real world better.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept