Fairness Transferability Subject to Bounded Distribution Shift
Abstract
Given an algorithmic predictor that is “fair” on some source distribution, will itstill be fair on an unknown target distribution that differs from the source withinsome bound?
In this paper, we study the transferability of statistical group fairnessfor machine learning predictors ( i.e. , classiﬁers or regressors) subject to boundeddistribution shift, a phenomenon frequently caused by user adaptation to a deployedmodel or a dynamic environment.
Herein, we develop a bound characterizing suchtransferability, ﬂagging potentially inappropriate deployments of machine learningfor socially consequential tasks.
We ﬁrst develop a framework for bounding violations of statistical fairness subject to distribution shift, formulating a generic upperbound for transferred fairness violation as our primary result.
We then developbounds for speciﬁc worked examples, adopting two commonly used fairness deﬁnitions ( i.e. , demographic parity and equalized odds) for two classes of distributionshift ( i.e. , covariate shift and label shift).
Finally, we compare our theoreticalbounds to deterministic models of distribution shift and against real-world data,ﬁnding that we are able to estimate fairness violation bounds in practice, even whensimplifying assumptions are only approximately satisﬁed.
Introduction
Distribution shift is a common, real-world phenomenon that affects machine learning deploymentswhen the target distribution of examples (features and labels) encountered by a learned policy divergesfrom the source distribution it was trained for.
For socially consequential decisions guided by machinelearning, such shifts in the underlying distribution can invalidate fairness guarantees and cause harmby exacerbating social disparities.
Unfortunately, distribution shift can be technically difﬁcult orimpossible to model at training time.
As we shall explain later, these changes may involve a variety offactors that require highly customized treatments.
Nonetheless, we still wish to certify the robustnessof the fairness of a policy on possible target distributions.
In this paper we provide a general framework encompassing a wide array of settings for quantifyingthe robustness of statistical group fairness guarantees.
We assume that the target distribution is drawnfrom a bounded domain, thus reducing the hard problem of modelling distribution shift dynamics toa more tractable, adversarial problem.
By bounding fairness violations in this way, we can detectpotentially inappropriate policy applications prior to deployment.
This work bridges a gap between recent literature on domain adaptation , which has largely focusedon the transferability of prediction accuracy rather than fairness, and algorithmic fairness , whichhas typically considered static distributions or prescribed models of distribution shift.
Our work isthe ﬁrst to systematically bound quantiﬁable violations of statistical group fairness while remainingagnostic to (1) the mechanisms responsible for distribution shift, (2) the method of measurement ofsuch change between distributions, and (3) the speciﬁc statistical deﬁnition of fairness applied.
[Figure]
V i
o
l a t i o n o f D e m o g
r a p
h
i c P a r i t y
CAMINCTXWA
Our primary result is a bound on a policy’s potential “violation of statistical group fairness”, deﬁned in terms ofthe differences in policy outcomes between groups, whenapplied to a target distribution shifted relative to the sourcedistribution within known constraints.
Such bounds areuseful when a learned policy will be deployed on a distribution that differs from the training or target distributionwith some uncertainty, e.g. , whenever a training populationrepresents only a random sample of the target population.
Such settings also arise when a target distribution mayonly be approximately modelled or forecast.
For example, upon deducing loan-approval policies, strategic agentswith unknown utilities may respond by changing their features or abstaining from classiﬁcation, thus shifting thedistribution of examples [19, 38, 42].
In addition, exogenous pressures, such as economic trends and noise, mayalso drive distribution shift.
In Figure 1, we show how areal-world distribution shift in demographic and incomedata for US states between 2014 and 2018 may increasefairness violations while decreasing accuracy for a hypothetical classiﬁer trained on the 2014 distribution.
In suchsettings, it is useful to quantify how fairness guarantees transfer across distributions shifted withinsome bound, thus allowing the deployment of unfair machine learning policies to be avoided.
1.1 Related Work
Our work relates to a number of recently explored domains of study in the literature: Speciﬁcally,we share a similar setting with recent efforts on domain adaptation , which have largely focusedon characterizing the effects of distribution shift on prediction performance rather than fairness.
Additionally, our work builds on investigations of algorithmic fairness as well as the dynamicmodeling or response of distributions in response to deployed machine learning policies (e.g.,[23, 11]).
We reference speciﬁc prior work in these domains in Appendix B, in the supplementarymaterial.
The most closely related works to the present study focus on how certain measures offairness are affected when policies are subject to speciﬁc types of distribution shift.
Fairness subject to Distribution Shift: A number of recent studies have considered speciﬁc examplesof fairness transferability subject to distribution shift [34, 9, 36, 31].
In particular, Schumann et al.[34] examine equality of opportunity and equalized odds as deﬁnitions of group fairness subject todistribution shifts quantiﬁed by an H -divergence function; Coston et al.
[9] consider demographicparity subject to a covariate shift assumption while group identiﬁcation remains unavailable to theclassiﬁer; Singh et al.
[36] focus on common group fairness deﬁnitions for binary classiﬁers subjectto a class of distribution shift that generalizes covariate shift and label shift by preserving someconditional probability between variables; and Rezaei et al.
[31] similarly consider common binaryclassiﬁcation fairness deﬁnitions, and equalized odds in particular, subject to covariate shift.
Whilewe address similar settings to these works as special cases of our bound, we propose a unifyingformulation for a broader class of statistical group fairness deﬁnitions and distribution shifts.
Indoing so, we recognize that particular settings recommend themselves to more natural measures ofdistribution shift, providing examples in Section 4.1, Section 4.2, and Section 5).
Another focus is to develop robust model that will preserve fairness for the target distribution (see, e.g.[1, 32, 26, 3]), for example, by assuming covariate shift and the availability of some unlabelled targetdata [9, 36, 31].
In particular, Singh et al.
[36] focus on learning stable models that will preserveprediction accuracy and fairness, utilizing a causal graph to describe anticipated distribution shifts.
Rezaei et al.
[31] takes a robust optimization approach, and Coston et al. [9] develops prevalenceconstrained and target-fair covariate shift method for getting the robust model. In contrast, our goal isto quantify fairness violations after an adversarial distribution shift. 1.2 Our Contributions
Our primary contribution is formulating a general, worst-case upper bound for a given policy’s violation of statistical group fairness subject to group-dependent distribution shifts within presupposed bounds (i.e., Equation (9)).
We characterize the space of statistical group fairness deﬁnitions and possible distribution shifts by appeal to premetric functions (Deﬁnition 2.1).
Our work explores commonsets of simplifying assumptions for this bound as special cases, yielding tractable calculations forseveral familiar combinations of fairness deﬁnitions and subcases of distribution shift (Theorem 4.1, Theorem 5.2) with readily interpretable results.
We compare our theoretical bounds to prescribedmodels of distribution shift in Section 6 and to real-world data in Section 7.
Bounding violationsof fairness subject to distribution shift allows us to recognize and avoid potentially inappropriatedeployments of machine learning when the potential disparities of a prospective policy eclipse agiven threshold within bounded distribution shifts of the training distribution.
Formulation
The supplementary material includes a table of notation (Appendix A) and all proofs (Appendix F).
2.1 Algorithmic Prediction
We consider two distributions, S ( source ) and T ( target ), each deﬁned as a probability distributionover the space of “examples”, where each example deﬁnes values for three random variables: X : afeature , e.g. , x , with arbitrary domain X .
Y : a label , e.g. , y , with arbitrary domain Y .
G : a group ,e.g. , g or h , with ﬁnite, countable domain G .
The predictor’s policy, π , intended for S but used on T ,deﬁnes a fourth variable for each example, viz.
, ˆ Y : a predicted label , e.g. , ˆ y , with domain ˆ Y = Y .
Using P ( · ) to denote the space of probability distributions over some domain, we denote the spaceof distributions over examples as D :=
P ( X × Y × G ) , such that S , T ∈ D .
It will also be 110 useful for us to notate the space of distributions over example outcomes associated with a givenpolicy as O :=
P ( X × Y ×
ˆ Y ) and the space of distributions over of group-speciﬁc examples as 112 G :=
P ( X × Y ) .
Without loss of generality, we allow the prediction policy π to be stochastic, such that, for anycombination ( x, g ) , the predictor effectively samples ˆ Y from a corresponding probability distributionπ ( x, g ) .
Stochastic classiﬁers arise in various constrained optimization problems and have been shownto be useful for making problems with custom losses or fairness constraints tractable [10, 18, 29].
We denote the space of nondeterministic policies as Π :=
( X × G → P ( ˆ Y )) , e.g. , π ∈ Π , and utilize 118 the natural transformations that relate the spaces of distributions D , policies Π , and outcomes O :
[Equation]
We will abuse the Pr notation for both probability density and probability mass functions whereappropriate and distributions when the random variables inside the expression are unconstrained.
2.2 Statistical Group-Fairness
We next deﬁne a broad class of disparity functions ∆ (cid:63) : Π × D
→ R representing how “unfair” agiven policy is for a given distribution, e.g. , writing ∆ (cid:63) ( π, T ) , albeit limited to notions of fairnesscaptured by the statistical discrepancies of outcomes between groups.
Deﬁnition 2.1.
We deﬁne a premetric Ψ on the space of distributions p with respect to q by theproperties Ψ( p (cid:107) q ) ≥ 0 and Ψ( p (cid:107) p ) = 0 for all p, q , and refer to the value of Ψ as a “shift”.
Deﬁnition 2.2.
We deﬁne statistical group disparity ∆ (cid:63) in terms of the symmetrized shifts betweengroup-speciﬁc outcome distributions, each mapped from a policy π and a distribution T .
We measureshifts between outcome distributions with a given premetric Ψ:
O 2 → R .
[Equation]
In Deﬁnition 2.2, Ψ quantiﬁes the statistical differences in outcomes between groups that are "unfair",where a value of 0 implies perfect fairness.
In this work, we assume that Ψ is the same for all g, hand that ∆ (cid:63) is insensitive to relative group size Pr( G ) .
Examples Familiar applications of Deﬁnition 2.2 include demographic parity ( DP ) and equalizedodds ( EO ).
A policy satisfying DP gives agents chosen at random any group a uniform chance ofreceiving a given binary classiﬁcation, i.e. ˆ Y = { 0 , 1 } .
We may measure the violation of DP as
[Equation]
The associated premetric 1 Ψ DP on p, q ∈
O is Ψ DP ( p (cid:107) q ) =
(cid:12)(cid:12)(cid:12)
Pr p ( ˆ Y =1) − Pr q ( ˆ Y =1) (cid:12)(cid:12)(cid:12) . Tosatisfy EO , for binary Y = { 0 , 1 } , π must maintain group-invariant true positive and false positive 138 classiﬁcation rates.
We may measure the violation of EO as
[Equation]
The associated premetric Ψ EO on p, q ∈
O is Ψ EO [Equation] p (cid:107) q ) = (cid:80) y ∈Y
[Equation]
Pr p ( ˆ Y =1 | Y = y ) − Pr q ( ˆ Y =1 | Y = y ) (cid:12)(cid:12)(cid:12) . Note that the restriction of EO to the ( Y = 1) case is known as Equal Opportunity ( EOp ).
We remark that Deﬁnition 2.2 provides a unifying representation for a wide array of statistical group“unfairness” deﬁnitions and may be used with inequality constraints.
That is, we may recover manyworking deﬁnitions of fairness that effectively specify a maximum value of some ∆ (cid:63) :
Deﬁnition 2.3.
A policy π is (cid:15) -fair with respect to ∆ (cid:63) on distribution T iff ∆ (cid:63) ( π, T ) ≤ (cid:15) .
2.3 Vector-Bounded Distribution Shift
Suppose, after developing policy π for distribution S , we “realize” some new distribution T onwhich the policy is actually operating.
This realization may be the consequence of sampling errorsduring the learning process, strategic feedback to our policy, random processes, or the reuse of ourpolicy on a new distribution for which retraining is impractical.
Our goal is to bound ∆ (cid:63) ( π, T ) givenknowledge of ∆ (cid:63) ( π, S ) and some notion of how much T differs from S .
Deﬁnition 2.4.
K ( p (cid:107) q ) is a divergence if and only if for all p and q , K ( p (cid:107) q ) ≥ 0 and
K ( p (cid:107) q ) =
⇐⇒ q = p .
Deﬁnition 2.5.
Deﬁne the group-vectorized shift D as S mutates into T as
[Equation]
(5) where e g represents a unit vector indexed by g , and each D g : G 2 → R is a divergence.
Note thateach D g also deﬁnes a premetric, but not necessarily a divergence, on D .
Assumption 2.6.
Let there exist some vector B (cid:23) 0 bounding D ( T (cid:107) S ) (cid:22) B , where (cid:22) and (cid:23) 157 denote element-wise inequalities.
In Assumption 2.6, B limits the possible distribution shift as S mutates into T , without requiring usto specify a model for how distributions evolve.
When modelling distribution shift requires complexdynamics ( e.g. , when agents learn and respond to classiﬁer policy), we reduce a potentially difﬁcultdynamical problem to a more tractable, adversarial problem to achieve a bound.
Lemma 2.7.
For all π , ∆ (cid:63) , and D , when B = 0 , ∆ (cid:63) ( π, S ) =
∆ (cid:63) ( π, T ) .
Lemma 2.7 indicates that, for a ﬁxed policy π , a change in disparity requires a measurable shift indistributions, conﬁrming intuition.
Restricted Distribution Shift Common assumptions that restrict the set of allowed distributionshifts include covariate shift and label shift .
For covariate shift, the distribution of labels conditionedon features is preserved across distributions for all groups, while for label shift, the distributions offeatures conditioned on labels is preserved across distributions for all groups. [Equation] implies
[Equation]
Label shift implies
[Equation]
In Section 4, we explore a deterministic model of a population’s response to classiﬁcation, resultingin covariate shift.
We do the same in Section 5 for label shift.
General Bounds
We formulate a primary bound in Equation (9), which we subsequently consider under simplifyingspecial cases.
We defer all subsequently omitted proofs for this manuscript to the Appendix.
Given an element-wise bound B on the vector-valued shift D ( T (cid:107) S )
(Assumption 2.6) we maybound the disparity ∆ (cid:63) of policy π on T by its supremum value over all possible T within the bound.
Deﬁnition 3.1.
Deﬁne the supremum value v for ∆ (cid:63) subject to D ( T (cid:107) S ) (cid:22) B as
[Equation]
In general, our strategy is to exploit the mathematical structure of the setting encoded by ∆ (cid:63) ( i.e. ,Ψ ) and D to obtain an upper bound for v deﬁned in Equation (8).
We ﬁrst explore general casesof simplifying assumptions before presenting worked special examples for frequently encounteredsettings.
Finally, we compare the resulting theoretical bounds to numerical results and simulations.
3.1 Lipshitz Conditions
The value of v deﬁnes a scalar ﬁeld over the domain T and thus a conservative vector ﬁeld F = ∇ b v .b
g b h v ( ..., b ) b g = B g b h = B
h max ∂ g v max ∂ h v
LipshitzBound( L · B )
B v ( ..., 0) v ( ..., B )
Figure
For any curve in D from S to T , bounds of the form
F (cid:22) L for some constant L along the curve imply a
Lipshitz bound on ∆ (cid:63) .
We visualize a bound in Figure 2for all possible curves in the region D ( T (cid:107) S ) (cid:22) B .
Theorem 3.2 (Lipshitz Upper Bound) .
If there exists an
L such that ∇ b v (∆ (cid:63) , D , π, S , b ) (cid:22) L , everywhere along 189 some curve from
S to T then
[Equation]
Succinctly, if we are guaranteed that disparity can neverincrease faster than a certain rate in some measure ofdistribution shift, then, given a maximum distribution shift,this rate bounds the maximum possible disparity.
Theutility of Theorem 3.2 arises when a Lipshitz condition
L is known, but direct computation of v is difﬁcult.
Weprovide an example of a Lipshitz bound in Section 5.
3.2 Subadditivity Conditions
Deﬁnition 3.3.
Deﬁne w as the maximum increase in disparity subject to D ( T (cid:107) S ) (cid:22) B , i.e. ,w (∆ (cid:63) , D , π, S , B ) :
= v (∆ (cid:63) , D , π, S , B ) − ∆ (cid:63) ( π, S ) .
Theorem 3.4.
Suppose, in the region D ( T (cid:107) S ) (cid:22) B , that w is subadditive in its last argument.
201 That is, w ( ..., a ) + w ( ..., c ) ≥ w ( ..., a + c ) for a , c (cid:23) 0 and a + c (cid:22) B .
If w is also locally differentiable, then a ﬁrst-order approximation of w ( ..., b ) evaluated at 0 , i.e. , L = ∇ b w ( ..., b ) (cid:12)(cid:12) b
=0 =∇ b v ( ..., b ) (cid:12) (cid:12) b =0 provides an upper bound for v ( ..., B ) , i.e. , v (∆ (cid:63) , D , π, S , B ) ≤ ∆ (cid:63) ( π, S ) + L · B .
Theorem 3.4 notes that “diminishing returns” in the change of ∆ (cid:63) as the difference of T with respectto S is increased implies a bound on ∆ (cid:63) in terms of its local sensitivity to D at S ( i.e. , using a ﬁrstorder Taylor approximation).
Note that, if w is concave in the bounded region, it is also subadditivein the bounded region, but the converse is not true, nor does the converse imply Lipshitzness.
3.3 Geometric Structure
It may happen that Ψ: O 2 → R and each D g : G 2 → R share structure that permits a geometricinterpretation of distribution shift.
While the utility of this observation depends on the speciﬁcproperties of Ψ and D , we demonstrate a worked example building on Section 4.2 in Appendix D, inwhich we allow ourselves to select a suitable D for ease of interpretation.
We proceed to considerworked examples that adopt common assumptions limiting the form of distribution shift and applycommon deﬁnitions of statistical group fairness.
Covariate Shift
4.1 Demographic Parity
The simplest way to work with Equation (8) is to bound the supremum v .
We ﬁrst consider demographic parity (Equation (3)) for Y = { 0 , 1 } and G = { g, h } , subject to covariate shift (Equation (6)).
219 We ﬁnd that the form of ∆ (cid:63)
DP subject to this simplifying assumption recommends itself to a naturalchoice of D .
We ﬁrst deﬁne a reweighting coefﬁcient ω g
( x ) := Pr T ( X = x | G = g ) Pr S ( X = x | G = g )
Theorem 4.1.
For demographic parity between two groups under covariate shift, denoting, for each g ,β g : =
Pr π, S ( ˆ Y =1 | G = g ) , it holds ∆ (cid:63) DP ( π, T ) ≤ ∆ (cid:63) DP ( π, S )+ (cid:80) g (cid:0) β g (1 − β g )
Var S [ ω g ( x )] (cid:1) 1 / 2 .
We notice that Var S [ ω g ( x )] recommends itself as a suitable divergence D g from S to T .
Using basis 224 vectors e g , we select D DP ( T (cid:107) S )
= (cid:80) g
e g Var S [ ω g ( x )] .
Comparing the inequality in Theorem 4.1and the consequent of Equation (10), we can interpret Pr π, S
( ˆ Y =1) in Theorem 4.1, as an upperbound for the average value of ∇ b v (∆ (cid:63) , D , π, T , b ) along any curve from S to T .
To interpret thisresult, the closer Pr( ˆ Y =1) is to 0.5 for any group, the more potentially sensitive the fairness of thepolicy is to distribution shifts for that group.
Corollary 4.2.
Theorem 4.1 may be generalized to multiple classes Y = { 1 , 2 , ..., m } and multiplegroups G ∈ { 1 , 2 , ..., n } , where β g,y = Pr (cid:16)
[Equation]
Y
[Equation]
g
[Equation]
[Equation]
We remark that in general, binary classiﬁcation bounds may frequently be generalized to multi-classbounds by redeﬁning fairness violations as a sum of binary-class fairness violations ( i.e. , same-classvs.
different-class labels) and summing the bounds on each.
4.2 Equal Opportunity
Consider an example using the ( Y =1) -conditioned case of Equalized Odds—termed Equal Opportunity ( EOp ).
Denoting, for each group g , the true positive rate β + g :=
Pr π, T ( ˆ Y =1 | Y =1 , G = g ) asan implicit function of π and T , we deﬁne disparity for EOp as ∆ [Equation] EOp [Equation]
[Equation]
|
[Equation]
− [Equation] | We may bound the realized value of ∆ (cid:63) EOp ( π, T ) by bounding β + g for each group.
Theorem 4.3.
Subject to covariate shift and given D , B , assume extremal values for β + g , i.e. ,
[Equation]
It follows that
[Equation]
Corollary 4.4.
∆ (cid:63) EOp cannot exceed |G| 2 4 .
In Appendix D, we bound the extremal values of β + g by geometrically interpreting this quantity as aninner product on an appropriate vector space, utilizing the freedom to select an appropriate D .
Label Shift
Under label shift ( Pr T ( X | Y ) =
Pr S ( X | Y ) ), violations of EO and EOp are invariant, because theindependence of ˆ Y and Y given X implies Pr π, T ( ˆ Y | Y ) =
Pr π, S ( ˆ Y | Y ) .
We therefore focus on the 247 violation of demographic parity ( DP ) (Equation (3)) subject to the label shift condition, treating abinary classiﬁcation task over two groups for simplicity.
In this setting, we choose to measure group-speciﬁc distribution shifts between S and T by thechange in proportion of true positive labels, which we refer to as the group qualiﬁcation rate
Q g [Equation]
[Equation]
Theorem 5.1.
A Lipshitz condition bounds ∇ b v (∆ [Equation] DP , D ( DP ) , π, S , b ) when
[Equation]
for true [false] positive rates β + g :=
Pr π ( ˆ Y =1 | Y =1 , G = g ) [ β − g :
=
Pr π ( ˆ Y =1 | Y =0 , G = g ) ].
Because β + g and β − g are invariant under label shift given a constant policy π , we elide their explicitdependence on the underlying distribution.
Theorem 5.2.
For DP under label-shift, ∀ g, | Q g ( S ) − Q g ( T ) | ≤ B g and this implies
[Equation]
[Equation]
[Equation]
Intuitively, the change in ∆ (cid:63) DP subject to label shift is given by | β + g − β − g | , the marginal change 258 in acceptance rates as agents change their qualiﬁcations Y , a distribution shift we measure by| Q g ( S ) − Q g ( T ) | .
When β + g is close to β − g , the policy looks like a random classiﬁer, and a label shifthas limited effect on statistical group disparity.
When | β + g − β − g | is large, indicating high classiﬁer 261 accuracy, the effect is larger.
Our bound thus exposes a direct trade-off between accuracy and fairnesstransferability guarantees.
Comparisons to Synthetic Distribution Shifts (Demographic Parity)6.1 Covariate Shift via Strategic Response
Let us consider a speciﬁc example of covariate shift (see Equation (6)) caused by a deterministic,group-independent model of strategic response in which agents are classiﬁed by and react to a binaryclassiﬁcation policy characterized by group-speciﬁc feature thresholds:
[Equation]
probability [Equation] [Equation] Agents in each group g may modify their feature x to x (cid:48) , after learning τ g , by incurring a costc g ( x, x (cid:48) ) ≥ 0 .
Similar to [19], we deﬁne the agent’s utility to be
[Equation]
Contrary to the standard strategic classiﬁcation setting, we do not assume that feature updates area misreporting, but that such updates also have the potential of corresponding to updates to theunderlying true qualiﬁcation Y of each agent.
This assumption has been made in a recent line ofresearch in incentivizing improvement from human agents subject to such classiﬁcation [5].
We will, however, assume all agents are rational utility (Equation (19)) maximizers.
For a giventhreshold τ g and manipulation budget m g , the best response of an agent with original feature x is:
[Equation]
such that [Equation] Under some additional assumptions on the agent’s best response behaviors (see Assumption F.3- Assumption F.5 in Appendix F.1) we can specify the reweighting coefﬁcient ω g ( x ) for our setting(see Figure 5 in the supplementary material) for use with Theorem 4.1.
[Figure] [Figure]
Bound - Modelled Shift
[Figure]
[Figure]
[Figure]
Proposition 6.1.
For our assumed setting of strategic response involving DP for two groups { g, h } ,
Theorem 4.1 implies ∆ (cid:63) ( π, T ) ≤ ∆ (cid:63) ( π, S ) + τ g (1 − τ g ) 23 m g
+ τ h (1 − τ h ) 23 m h .
The above result shows that two factors lead to a smaller difference between the source and targetfairness violations: a less stochastic classiﬁer (when the threshold τ g is far away from 0 . 5 ) and asmaller manipulation budget m g (diminishing agents’ ability to adapt their feature).
These factorslead to less potential manipulation and result in a tighter upper bound for the fairness violation on T .
6.2 Label Shift via Replicator Dynamics
We now evaluate our theoretical bound, Theorem 5.2, for demographic parity subject to label shiftas prescribed by the replicator dynamics model of Raab and Liu [30].
Brieﬂy, replicator dynamicsassumes that the proportion of agents in a population choosing one strategy over another grows inproportion to the ratio of average utilities realized by the two strategies.
The cited model additionallyassumes X =
R , Y = { 0 , 1 } , and a monotonicity condition for S given by dd x Pr S ( X = x | Y =1)
Pr S ( X = x | Y =0) > 0 .
Label shift under the discrete-time ( t ) replicator dynamics may be expressed in terms of groupqualiﬁcation rate Q g :=
Pr t ( Y =1 |
G
=
g ) and agent utilities, i.e. , group- and feature- independentvalues
U y, ˆ y , such that agents adopt new labels with frequencies afﬁne in the current popularity andaverage utility of the label in the agent’s group.
Denote the fractions of group-conditioned, featureindependent outcomes with ρ y, ˆ y g :
=
Pr t ( ˆ Y =
ˆ y, Y =
y | G = g ) and abbreviate the fraction-weightedutility as u y, ˆ y g ( t ) :
= U y, ˆ y ρ y, ˆ y g .
We may then represent the replicator dynamics as
[Equation]
[Equation] To apply Theorem 5.2, we also observe that | β + g − β − g | =
| ρ 1 , 1 g − ρ 0 , 1 g | ρ 1 , 1 g + ρ 0 , 1 g , where β
+ g and β − g represent thetrue positive rate and false positive rate for group g , respectively, and we use the change in qualiﬁcationrate as our measurement of label shift, i.e. , B g = | Q g [ t + 1] − Q g [ t ] | .
When demographic parity isperfectly satisﬁed, we note that the acceptance rate ( ρ 1 , 1 g + ρ 0 , 1 g ) is group-independent.
Theorem 6.2.
For DP subject to label replicator dynamics,
[Equation]
In Section 6.1, we graphically represent all possible states of an initially fair system (thus determiningβ and ρ as a result of the monotonicity condition) by the tuple of qualiﬁcation rates for each group.
With the dynamics prescribed by Equation (21), we depict the local evolution of the state subject toa ﬁxed, locally DP -fair policy, as well as the rate of change for the violation of fairness, which wecompare to the theoretical bound.
Interpreting our results, we note that the bound lacks information about the relative directions of thechange in acceptance rates for each group, and thus over-approximates possible fairness violationswhen group acceptance rates shift the same direction.
When group acceptance rates move in opposingdirections, however, the bound gives excellent agreement with the modelled replicator dynamics.
Comparisons to Real-World Distribution Shifts
We now compare our special-case theoretical bounds ( i.e. , label/covariate shift) to real-world distribution shifts and hypothetical classiﬁers.
We use American Community Survey (ACS) data provided bythe US Census Bureau [17].
We adopt the sampling and pre-processing approaches following the Folktables package provided by Ding et al.
[14] 2 to obtain 1,599,229 data points.
The data ispartitioned by (1) all ﬁfty US states and (2) years from 2014 to 2018.
We use 10 features coveringthe demographic information used in the UCI Adult dataset [4], including age, occupation, education,etc. .
as X for our model, select sex as binary protected group, i.e., G ∈ { g = female , h = male } ,and set the label Y to correspond to whether an individual’s annual income is greater than $50K.
To apply our label-shift or covariate-shift bounds, we ﬁrst need to verify whether the two datasetssatisfy either of these assumptions.
Through a conditional independence test [22], we ﬁnd that thelikelihood that the covariates shift across US states is approximately two orders of magnitude higherthan for labels.
For temporal shifts within states, approximately the same ratio of probability favorslabel shift over covariate shift.
We therefore compare the disparities of hypothetical policies on thesedistributions to bounds generated from the corresponding, approximately satisﬁed assumptions.
On this data, we train a set of group-dependent, linear threshold classiﬁers Pr π ( x,g ) ( ˆ Y =1) =
327[ σ ( w · x ) > τ g ] , for a range of thresholds τ g and τ h for each source distribution.
Here, σ ( · ) is thelogistic function and w denotes a weight vector.
We then consider two types of real-world distributionshift: (1) geographic , in which a model trained for one state is evaluated on other US state in the sameyear, and (2) temporal , in which a model trained for 2014 is evaluated on the same state in 2018. [Figure] [Equation] g 0.0 0.2 0.4 0.6 0.8 1.
t a u _ h 0.00.20.40.60.81. C h a n g e i n V i o l a t i o n 0.080.060.040.020. [Figure]
−→ IL t a u _ g 0.0 0.2 0.4 0.6 0.8 1.
t a u _ h 0.00.20.40.60.81.
C h a n g e i n V i
o
l a t i o n 0.04 0.02 0.
[Figure]
t a u _ g 0.0 0.2 0.4 0.6 0.8 1.
t a u _ h 0.00.20.40.60.81.0 C h
a n g e i n V i
o
l a t i o n o f D e m o g
r a p
h
i c P a r i t y 0.010.000.010.020.
{+(c) CA: 
+}
→t a u _ g 0.0 0.2 0.4 0.6 0.8 1.
t a u _ h 0.00.20.40.60.81.0 C h a n g e i n V i o l a t i o n o f D e m o g
r a p
h
i c P a r i t y 0.01 0.00 0. [Figure] We graphically compare the theoretical bounds of Theorem 4.1 and Theorem 5.2 for the increasedviolation of DP subject to covariate and label shift, respectively, to the simulated violations for ourmodel and data in Figure 4. We provide additional examples and an evaluation of bounds for EOsubject to covariate shift (while label shift preserves EO ) in Appendix E.2.
Despite the fact thatgeographic or temporal distribution shifts only approximately satisfy the assumptions of covariate orlabel shift, these comparisons demonstrate that our theoretical bounds are not vacuous, approximatelybounding the change of fairness violation in the real world domain shifts.
For geographic shifts, thecovariate shift EO bounds (Appendix E.2) correctly overestimate disparity and tighten near accuratepolicies, while our DP bounds are useful only for a subset of policy thresholds. For temporal shift,the label shift bound for DP correctly overestimates the real change of DP violations but still remainsat the same order of magnitude.
Conclusion and Discussion
In this paper, we have developed a unifying framework for boundingthe violation of statistical group fairness guarantees when the underlying distribution subject toalgorithmic prediction is changed.
We hope that this work can generate meaningful discussionregarding the viability of fairness guarantees subject to distribution shift, the bounds of adversarialattacks against algorithmic fairness, and evaluations of robustness with respect to algorithmic fairness.
We believe that, just as published measurements are of limited use without reported uncertainties,fairness guarantees must come with bounds on robustness to distribution shift.
One direction of futurework is to consider fair transferability problem in a more complicated distribution shift dynamics.
Forexample, consider compound shift [33], which is a combination of covariate shift and label shift, forwhich the bounds herein cannot be composed without information about the intermediate distribution.
Another potential future direction is to develop reasonable bounds on anticipated distribution shiftfrom models of human behavior and exogenous pressures.
Checklist
For all authors...
Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contri- butions and scope?
[Yes]
Did you describe the limitations of your work?
[Yes]
Did you discuss any potential negative societal impacts of your work?
[Yes]
Have you read the ethics review guidelines and ensured that your paper conforms to them?[Yes]
If you are including theoretical results...1.
Did you state the full set of assumptions of all theoretical results?
[Yes]2.
Did you include complete proofs of all theoretical results?
[Yes]3.
If you ran experiments...
Did you include the code, data, and instructions needed to reproduce the main experimentalresults (either in the supplemental material or as a URL)?
[Yes] We submit the code in thesupplementary material.
We attach a README ﬁle for instructions to reproduce the results.
Did you specify all the training details (e.g., data splits, hyperparameters, how they werechosen)?
[Yes] We specify the data splits in the experiment details.
Did you report error bars (e.g., with respect to the random seed after running experimentsmultiple times)?
[N/A]
Did you include the total amount of compute and the type of resources used (e.g., type of
GPUs
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... [Table]
If your work uses existing assets, did you cite the creators?
[Yes] [Table]
Did you mention the license of the assets?
[N/A] [Table]
Did you include any new assets either in the supplemental material or as a URL?
[N/A] No [Table] new assets curated or released.
[Table]
Did you discuss whether
and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A]
No new assets curated or released.
[Table] Did you discuss whether the data you are using/curating contains personally identiﬁable [Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] Did you include the full text of instructions given to participants and screenshots, if applicable? [Table] [N/A]
[Table] Did you describe any potential participant risks, with links to Institutional Review Board (IRB) [Table]
approvals, if applicable?
[N/A]
[Table]
Did you include the estimated hourly wage paid to participants and the total amount spent onparticipant compensation?
[N/A]
[Table]