Fairness Transferability Subject to Bounded Distribution Shift
Abstract
Given an algorithmic predictor that is “fair” on some source distribution, will it still be fair on an unknown target distribution that differs from the source within some bound?
In this paper, we study the transferability of statistical group fairness for machine learning predictors ( i.e. , classiﬁers or regressors) subject to bounded distribution shift. Such shifts may be introduced by initial training data uncertainties, user adaptation to a deployed predictor, dynamic environments, or the use of pre-trained models in new settings.
Herein, we develop a bound that characterizes such transferability, ﬂagging potentially inappropriate deployments of machine learning for socially consequential tasks.
We ﬁrst develop a framework for bounding violations of statistical fairness subject to distribution shift, formulating a generic upper bound for transferred fairness violations as our primary result.
We then develop bounds for speciﬁc worked examples, focusing on two commonly used fairness deﬁnitions ( i.e. , demographic parity and equalized odds) and two classes of distribution shift ( i.e. , covariate shift and label shift).
Finally, we compare our theoretical bounds to deterministic models of distribution shift and against real-world data, ﬁnding that we are able to estimate fairness violation bounds in practice, even when simplifying assumptions are only approximately satisﬁed.
Introduction
Distribution shift is a common, real-world phenomenon that affects machine learning deployments when the target distribution of examples (features and labels) ultimately encountered by a datadriven policy diverges from the source distribution it was trained for.
For socially consequential decisions guided by machine learning, such shifts in the underlying distribution can invalidate fairness guarantees and cause harm by exacerbating social disparities.
Unfortunately, distribution shift can be technically difﬁcult or impossible to model at training time ( e.g. , when depending on complex social dynamics or unrealized world events).
Nonetheless, we still wish to certify the robustness of fairness metrics for a policy on possible target distributions.
In this paper, we provide a general framework for quantifying the robustness of statistical group fairness guarantees.
We assume that the target distribution is adversarially drawn from a bounded domain, thus reducing the hard problem of modelling distribution shift dynamics to a more tractable, static problem.
With this framework, we can detect potentially inappropriate policy applications, prior to deployment, when fairness violation bounds are not sufﬁciently small.
This work bridges a gap between recent literature on domain adaptation , which has largely focused on the transferability of prediction accuracy (rather than fairness), and algorithmic fairness , which has typically considered static distributions or prescribed models of distribution shift.
Our work is the ﬁrst to systematically bound quantiﬁable violations of statistical group fairness while remaining agnostic to (1) the mechanisms responsible for distribution shift, (2) how group-speciﬁc distribution shifts are quantiﬁed, and (3) the speciﬁc statistical deﬁnition of group fairness applied.
[Figure]
Our primary result is a bound on a policy’s potential “violation of statistical group fairness”—deﬁned
in terms of the differences in policy outcomes between groups—when applied to a target distribution shifted relative to the source distribution within known constraints.
Such settings naturally arise whenever training data represents a random sample of a target population with different statistics or a sample from dynamic environments, when a policy is reused on a new distribution without retraining, or whenever policy deployment itself induces a distribution shift.
As an example of this last case, strategic individuals seeking loans might change their features or abstain from future application (thus shifting the distribution of examples) in response to policies trained on historical data [18, 38, 43].
Beyond policy selection, exogenous pressure such as economic trends and noise may also drive distribution shift in this example.
In Figure 1, we show how a real-world distribution shift in demographic and income data for US states between 2014 and 2018 may increase fairness violations while decreasing accuracy for a hypothetical classiﬁer trained on the 2014 distribution.
In such settings, it is useful to quantify how fairness guarantees transfer across distributions shifted within some bound, thus allowing the deployment of unfair machine learning policies to be avoided.
1.1 Related Work
Our work considers a setting similar to recent studies of domain adaptation , which have largely focused on characterizing the effects of distribution shift on prediction performance rather than fairness.
Our work also builds on efforts in algorithmic fairness , especially dynamical treatments of distribution shift in response to deployed machine learning policies [23, 11, 30].
We reference speciﬁc prior work in these domains in Appendix B, and here discuss existing work that focuses on how certain measures of fairness are affected when policies are subject to speciﬁc distribution shift.
Fairness subject to Distribution Shift: A number of recent studies have considered speciﬁc examples of fairness transferability subject to distribution shift [34, 9, 36, 31, 21].
In particular, Schumann et al. [34] examine equality of opportunity and equalized odds as deﬁnitions of group fairness subject to distribution shifts quantiﬁed by an H -divergence function; Coston et al.
[9] consider demographic parity subject to a covariate shift assumption while group identiﬁcation remains unavailable to the classiﬁer; Singh et al.
[36] focus on common group fairness deﬁnitions for binary classiﬁers subject to a class of distribution shift that generalizes covariate shift and label shift by preserving some conditional probability between variables; and Rezaei et al.
[31] similarly consider common binary classiﬁcation fairness deﬁnitions such as equalized odds subject to covariate shift.
While we address similar settings to these works as special cases of our bound, we propose a unifying formulation for a broader class of statistical group fairness deﬁnitions and distribution shifts.
In doing so, we recognize that particular settings recommend themselves to more natural measures of distribution shift, providing examples in Section 4.1, Section 4.2, and Section 5).
Another thread in existing literature is the development of robust models with the goal of guaranteeing fairness on a modelled target distribution (e.g., [1, 32, 26, 3, 21]) , for example, by assuming covariate shift and the availability of some unlabelled target data [9, 36, 31].
In particular, Singh et al.
[36] focus on learning stable models that will preserve prediction accuracy and fairness, utilizing a causal graph to describe anticipated distribution shifts.
Rezaei et al.
[31] takes a robust optimization approach, and Coston et al.
[9] develops prevalence-constrained and target-fair covariate shift method for getting the robust model.
In contrast, our goal is to quantify fairness violations after an adversarial distribution shift for any given policy, including those not trained with robustness in mind.
1.2 Our Contributions
Our primary contribution is formulating a general, worst-case upper bound for a given policy’s violation of statistical group fairness subject to group-dependent distribution shifts within presupposed bounds (i.e., Equation (9)).
Bounding violations of fairness subject to distribution shift allows us to recognize and avoid potentially inappropriate deployments of machine learning when the potential disparities of a prospective policy eclipse a given threshold within bounded distribution shifts of the training distribution.
We ﬁrst characterize the space of statistical group fairness deﬁnitions and possible distribution shifts by appeal to premetric functions (Deﬁnition 2.1).
After formulating the worst-case upper bound, we explore common sets of simplifying assumptions for this bound as special cases, yielding tractable calculations for several familiar combinations of fairness deﬁnitions and subcases of distribution shift (Theorem 4.1, Theorem 5.2) with readily interpretable results.
Finally, we compare our theoretical bounds to prescribed models of distribution shift in Section 6 and to real-world data in Section 7.
The details for reproducing our experimental results can be found at https://github.com/UCSC-REAL/Fairness_Transferability .
Formulation
The appendices include a table of notation (Appendix A) and all proofs (Appendix F).
2.1 Algorithmic Prediction
We consider two distributions, S ( source ) and T ( target ), each deﬁned as a probability distribution for examples , where each example deﬁnes values for three random variables: X , a feature ( e.g. , x ) with arbitrary domain X ; Y , a label ( e.g. , y ) with arbitrary domain Y ; and G , a group ( e.g. , g or h ) with ﬁnite, countable domain G .
The predictor’s policy π , intended for S but used on T , deﬁnes a fourth variable for each example: viz.
, ˆ Y , a predicted label ( e.g. , ˆ y ) with domain ˆ Y = Y .
Using P ( · ) to denote the space of probability distributions over some domain, we denote the space of distributions over examples as D :=
P ( X × Y × G ) , such that S , T ∈ D .
It will also be useful for us to notate the space of distributions over example outcomes associated with a given policy as O :=
P ( X × Y ×
ˆ Y ) and the space of distributions over of group-speciﬁc examples as G :=
P ( X × Y ) .
Without loss of generality, we allow the prediction policy π to be stochastic, such that, for any combination ( x, g ) , the predictor effectively samples ˆ Y from a corresponding probability distribution π ( x, g ) .
Stochastic classiﬁers arise in various constrained optimization problems and proven useful for making problems with custom losses or fairness constraints tractable [10, 17, 29, 40].
We denote the space of nondeterministic policies as Π :=
( X × G → P ( ˆ Y )) ( e.g. , π ∈ Π ) and utilize the natural transformations that relate the spaces of distributions D , policies Π , and outcomes O :
[Equation]
We abuse the Pr notation for both probability density and probability mass functions as appropriate.
2.2 Statistical Group-Fairness
We next deﬁne a broad class of disparity functions ∆ (cid:63) : Π × D
→ R representing how “unfair” a given policy is for a given distribution ( e.g. , writing ∆ (cid:63) ( π, T ) ), noting that this notion of fairness is limited to capturing statistical discrepancies of outcomes between groups.
Deﬁnition 2.1.
We deﬁne a premetric 3 Ψ on the space of distributions p with respect to q by the properties Ψ ( p (cid:107) q ) ≥ 0 and Ψ( p (cid:107) p ) = {+
+}
for all p, q , and refer to the value of Ψ as a “shift”.
Deﬁnition 2.2.
We deﬁne a statistical group disparity ∆ (cid:63) for policy π and distribution T in terms of the symmetrized shifts between group-speciﬁc outcome distributions.
We measure shifts between outcome distributions with a given premetric Ψ:
O 2 → R .
[Equation]
In Deﬁnition 2.2, Ψ quantiﬁes the speciﬁc statistical differences in outcomes between groups that are "unfair", where a value of 0 implies perfect fairness.
In this work, we assume that Ψ is the same for all g, h and that ∆ (cid:63) is insensitive to relative group size Pr( G ) .
Examples Familiar applications of Deﬁnition 2.2 include demographic parity ( DP ) and equalized odds ( EO ).
A policy satisfying DP , in expectation, assigns a given binary classiﬁcation y ∈ { 0 , 1 } to the same fraction of examples in each group.
We may measure the violation of DP as
[Equation]
The associated premetric Ψ DP for p, q ∈
O is Ψ DP
[Equation]
To satisfy EO , for binary Y = { 0 , 1 } , π must maintain group-invariant true positive and false positive classiﬁcation rates.
We may measure the violation of EO as
[Equation]
The associated premetric is Ψ
[Equation]
Pr
[Equation]
|
[Equation]
y ) (cid:12)(cid:12)(cid:12) .
Note that the restriction of EO to the ( Y = 1) case is known as Equal Opportunity ( EOp ).
We remark that Deﬁnition 2.2 provides a unifying representation for a wide array of statistical group “unfairness” deﬁnitions and may be used with inequality constraints.
That is, we may recover many working deﬁnitions of fairness that effectively specify a maximum value of disparity:
Deﬁnition 2.3.
A policy π is (cid:15) -fair with respect to ∆ (cid:63) on distribution T iff ∆ (cid:63) ( π, T ) ≤ (cid:15) .
2.3 Vector-Bounded Distribution Shift
Suppose, after developing policy π for distribution S , we realize some new distribution T on which the policy is actually operating.
This realization may be the consequence of sampling errors during the learning process, strategic feedback to our policy, random processes, or the reuse of our policy on a new distribution for which retraining is impractical.
Our goal is to bound ∆ (cid:63) ( π, T ) given knowledge of ∆ (cid:63) ( π, S ) and some notion of how much T possibly differs from S .
Deﬁnition 2.4.
K ( p (cid:107) q ) is a divergence if and only if for all p and q , K ( p (cid:107) q ) ≥ 0 and
K ( p (cid:107) q ) =
⇐⇒ q = p .
Deﬁnition 2.5.
Deﬁne the group-vectorized shift D , as S mutates into T , as
[Equation]
where e g represents a unit vector indexed by g , and each D g : G 2 → R is a divergence (Deﬁnition 2.4).
Note that each D g also deﬁnes a premetric (but not necessarily a divergence) on D .
Assumption 2.6.
Let there exist some vector B (cid:23) 0 bounding D ( T (cid:107) S ) (cid:22) B , where (cid:22) and (cid:23) denote element-wise inequalities.
In Assumption 2.6, B limits the possible distribution shift as S mutates into T , without requiring us to specify a model for how distributions evolve.
When modelling distribution shift requires complex dynamics ( e.g. , when agents learn and respond to classiﬁer policy), we reduce a potentially difﬁcult dynamical problem to a more tractable, adversarial problem to achieve a bound.
Lemma 2.7.
For all π , ∆ (cid:63) , and D , when B = 0 , ∆ (cid:63) ( π, S ) =
∆ (cid:63) ( π, T ) .
Lemma 2.7 indicates that, for a ﬁxed policy π , a change in disparity requires a measurable shift in distributions from S to T , conﬁrming intuition.
Restricted Distribution Shift Common assumptions that restrict the set of distribution shifts include covariate shift and label shift .
For covariate shift, the distribution of labels conditioned on features is preserved across distributions for all groups, while for label shift, the distributions of features conditioned on labels is preserved across distributions for all groups.
Covariate shift implies Pr
[Equation]
Label shift implies Pr
[Equation]
In Section 4, we explore a deterministic model of a population’s response to classiﬁcation as an example of covariate shift.
We do the same in Section 5 for label shift.
General Bounds
We ﬁrst deﬁne a primary bound in Deﬁnition 3.1 before considering simplifying special cases.
Given an element-wise bound B on the vector-valued shift D ( T (cid:107) S )
(Assumption 2.6) we may bound the disparity ∆ (cid:63) of policy π on any realizable target distribution T by its supremum value.
Deﬁnition 3.1.
Deﬁne the supremum value v for ∆ (cid:63) subject to D ( T (cid:107) S ) (cid:22) B as
[Equation]
In general, our strategy is to exploit the mathematical structure of the setting encoded by ∆ (cid:63) ( i.e. , Ψ ) and D to obtain an upper bound for v deﬁned in Equation (8).
We ﬁrst explore general cases of simplifying assumptions before presenting worked special examples for frequently encountered settings.
Finally, we compare the resulting theoretical bounds to numerical results and simulations.
3.1 Lipshitz Conditions
The value of v deﬁnes a scalar ﬁeld in B and therefore a conservative vector ﬁeld F = ∇ B v .
b g b h v ( ..., b ) b g = B g b h = B
h max ∂ g v max ∂ h v
LipshitzBound( L ·
[Figure] [Figure]
For any curve in D from S to T , bounds of the form
F (cid:22) L for some constant L along the curve imply a
Lipshitz bound on ∆ (cid:63) .
We visualize a bound in Figurefor all possible curves in the region D ( T (cid:107) S ) (cid:22) B .
Theorem 3.2 (Lipshitz Upper Bound) .
If there exists an
L such that ∇ b v (∆ (cid:63) , D , π, S , b ) (cid:22) L , everywhere along some curve as b varies from 0 to B , then
[Equation]
Succinctly, if we are guaranteed that disparity can never increase faster than a certain rate in some measure of distribution shift, then, given a maximum distribution shift, this rate bounds the maximum possible disparity.
The utility of Theorem 3.2 arises when a Lipshitz condition
L is known, but direct computation of v is difﬁcult.
We provide an example of a Lipshitz bound in Section 5.
3.2 Subadditivity Conditions
Deﬁnition 3.3.
Deﬁne w as the maximum increase in disparity subject to D ( T (cid:107) S ) (cid:22) B , i.e. , w [Equation]
[Equation] D [Equation]
Theorem 3.4.
Suppose, in the region D ( T (cid:107) S ) (cid:22) B , that w is subadditive in its last argument.
That is, w ( ..., a ) + w ( ..., c ) ≥ w ( ..., a + c ) for a , c (cid:23) 0 and a + c (cid:22) B .
If w is also locally differentiable, then a ﬁrst-order approximation of w ( ..., b ) evaluated at 0 , i.e. , [Equation]
[Equation] provides an upper bound for v ( ..., B ) , i.e. , [Equation] [Equation]
Theorem 3.4 notes that “diminishing returns” in the change of ∆ (cid:63) as the difference of T with respect to S is increased implies a bound on ∆ (cid:63) in terms of its local sensitivity to D at S ( i.e. , using a ﬁrstorder Taylor approximation).
Note that, if w is concave in the bounded region, it is also subadditive in the bounded region, but the converse is not true, nor does the converse imply Lipshitzness.
3.3 Geometric Structure
It may happen that Ψ: O 2 → R and each D g : G 2 → R share structure that permits a geometric interpretation of distribution shift.
While the utility of this observation depends on the speciﬁc properties of Ψ and D , we demonstrate a worked example building on Section 4.2 in Appendix D, in which we allow ourselves to select a suitable D for ease of interpretation.
We proceed to consider worked examples that adopt common assumptions limiting the form of distribution shift and apply common deﬁnitions of statistical group fairness.
Covariate Shift
We now present our fairness transferability results subject to covariate shift for both demongraphic parity (Section 4.1) and equalized opportunity (Section 4.2) as fairness criteria.
4.1 Demographic Parity
The simplest way to work with Equation (8) is to bound the supremum v .
We ﬁrst consider demographic parity (Equation (3)) for Y = { 0 , 1 } and G = { g, h } , subject to covariate shift (Equation (6)).
We ﬁnd that the form of ∆ (cid:63)
DP subject to covariate shift recommends itself to a natural choice of vector divergence, D .
First, deﬁne a re-weighting coefﬁcient ω g ( T , S , x ) : =
Pr T ( X = x | G = g )
Pr S
X = x | G = g ) .
Theorem 4.1.
For demographic parity between two groups under covariate shift (denoting, for each g , β g :=
Pr π, S ( ˆ Y =1 | G = g ) ), [Equation]
We notice that Var S [ ω g ( T , S , x )] recommends itself as a suitable divergence D g from S to T .
Using basis vectors e g , for this example, we could deﬁne D ( T (cid:107) S ) =
(cid:80)
[Equation] When Var S [ ω g ( T , S , x )] ≤ B g , it follows [Equation]
Comparing the inequality in Theorem 4.1 and the consequent of Equation (10), we can interpret
Pr π, S ( ˆ Y =1) in Theorem 4.1 as an upper bound for the average value of ∇ b v (∆ (cid:63) DP , D , π, T , b ) along any curve from S to T .
Interpreting this result, the closer Pr( ˆ Y =1) is to 0.5 for any group, the more potentially sensitive the fairness of the policy is to distribution shifts for that group.
We can further generalize the results to multi-class and multi-group setting:
Corollary 4.2.
Theorem 4.1 may be generalized to multiple classes Y = { 1 , 2 , ..., m } and multiple groups G ∈ { 1 , 2 , ..., n } , where β g,y =
Pr (cid:16) ˆ Y =
y | G = g (cid:17) and assuming Var S [ ω g ( T , S , x )] ≤ B g :
[Equation]
[Equation]
We remark that in general, binary classiﬁcation bounds may frequently be generalized to multi-class bounds by redeﬁning fairness violations as a sum of binary-class fairness violations ( i.e. , same-class vs. different-class labels) and summing the bounds on each.
4.2 Equal Opportunity
Consider an example using the ( Y =1) -conditioned case of Equalized Odds—termed Equal Opportunity ( EOp ).
Denoting, for each group g , the true positive rate β + g :=
Pr π, T ( ˆ Y =1 | Y =1 , G = g ) as an implicit function of π and T , we deﬁne disparity for EOp as ∆ (cid:63) EOp ( π, T ) :
= (cid:80) g,h ∈G | β + g − β
+ h | .
We may bound the realized value of ∆ (cid:63) EOp ( π, T ) by bounding β + g for each group:
Theorem 4.3.
Subject to covariate shift and any given D , B , assume extremal values for β + g , i.e. ,
[Equation]
it follows that
[Equation]
Corollary 4.4.
The disparity measurement ∆ (cid:63) EOp cannot exceed |G| 2 4 .
In Appendix D, we bound the extremal values of β + g by geometrically interpreting this quantity as an inner product on an appropriate vector space, utilizing the freedom to select an appropriate D .
Label Shift
Under label shift ( Pr S ( X | Y ) =
Pr T ( X | Y ) ), violations of EO and EOp are invariant, because the independence of ˆ Y and Y given X implies Pr π, T ( ˆ Y | Y ) =
Pr π, S ( ˆ Y | Y ) .
We therefore focus on the violation of demographic parity ( DP ) (Equation (3)) subject to the label shift condition, treating a binary classiﬁcation task over two groups for simplicity.
In this setting, we choose to measure group-speciﬁc distribution shifts from S to T by the change in proportion of ground-truth positive labels, which we refer to as the group qualiﬁcation rate
Q g ( T ) :=
Pr T ( Y = 1 | G = g ) :
[Equation]
Theorem 5.1.
A Lipshitz condition bounds ∇ b v (∆ (cid:63) DP , D , π, S , b ) when
[Equation]
Speciﬁcally,
[Equation]
for true positive rates β + g and false positive rates β − g :
[Equation]
Because β + g and β − g are invariant under label shift given a constant policy π , we elide their explicit dependence on the underlying distribution.
Theorem 5.2.
For DP under the bounded label-shift assumption ∀ g, | Q g ( S
[Equation]
− Q g ( T
[Equation]
≤ B
[Equation]
Intuitively, the change in ∆ (cid:63) DP subject to label shift depends on | β + g − β − g | , the marginal change in acceptance rates as agents change their qualiﬁcations Y . We measure the distribution shift as agents change their qualiﬁcations by | Q g ( S ) − Q g ( T ) | .
When β + g is close to β − g , the policy looks like a random classiﬁer, and a label shift has limited effect on statistical group disparity.
When | β + g − β − g | is large, indicating high classiﬁer accuracy, the effect on supremal disparity is larger.
Our bound thus exposes a direct trade-off between accuracy and fairness transferability guarantees.
Comparisons to Synthetic Distribution Shifts (Demographic Parity)
To further interpret our results, in this section, we consider speciﬁc and popular agent models to characterize distribution shift and instantiate our bounds for particular forms of D , B , and ∆ (cid:63) .
Let us consider a speciﬁc example of covariate shift (Equation (6)) caused by a deterministic, groupindependent model of strategic response in which agents react to a binary classiﬁcation policy π characterized by group-speciﬁc feature thresholds:
[Equation]
For simplicity, we assume the feature domain X = [0 , 1] .
In response to threshold τ g , agents in each group g may modify their feature x to x (cid:48) by incurring a cost c g ( x, x (cid:48) ) ≥ 0 .
Similar to [18], we deﬁne the utility u g for agents in group g to be
[Equation]
Contrary to the standard strategic classiﬁcation setting, we do not assume that feature updates represent false reports, but that such updates may correspond to actual changes underlying the true qualiﬁcation Y of each agent.
This assumption has been made in a recent line of research in incentivizing improvement from human agents subject to such classiﬁcation [5].
Next, we assume all agents are rational utility maximizers (Equation (24)).
For a given threshold τ g and manipulation budget m g , the best response of an agent with original feature x is
[Equation]
such that c
[Equation]
x
[Equation]
z
[Equation]
To make the problem tractable, we make additional assumptions about the agents’ best responses. Assumption 6.1. An agent’s original feature x is sampled as X ∼ U [0 , 1] 4 . Assumption 6.2.
The cost function c g ( x, x (cid:48) ) is monotone in | x − x (cid:48) | as c g ( x, x (cid:48) )
=
| x (cid:48) − x | .
Under Assumption 6.2, only those agents with features x ∈ [ τ g − m g , τ g ) will attempt to change their feature. We also assume that feature updates are non-deterministic, such that agents with features closer to the decision boundary τ g have a greater chance of updating their feature and each updated feature x (cid:48) is sampled from a uniform distribution depending on τ g , m g , and x : Assumption 6.3. For agents who attempt to update their features, the probability of a successful feature update is Pr( X (cid:54) =
X (cid:48) )
= 1 − | x − τ g | m g . Assumption 6.4. An agent’s updated feature x (cid:48) , given original feature x , manipulation budget m g , and classiﬁcation boundary τ g , is sampled as X (cid:48) ∼ U [ τ g ,τ g + m g − x ] .
With the above setting, we can specify the reweighting coefﬁcient ω g ( x ) for our setting (Equation (102) in Appendix F.1 and get the following bound for the strategic response setting 5 :
Proposition 6.5.
For our assumed setting of strategic response involving DP for two groups { g, h } ,
Theorem 4.1 implies
[Equation]
The above result shows that two factors lead to a smaller difference between the source and target fairness violations: a less stochastic classiﬁer (when the threshold τ g is far away from 0 . 5 ) and a smaller manipulation budget m g (diminishing agents’ ability to adapt their feature). In this case, B g =
2 3 τ g (1 − τ g ) .
These factors lead to less potential manipulation and result in a tighter upper bound for the fairness violation on T .
6.2 Label Shift via Replicator Dynamics
We now evaluate our theoretical bound for demographic parity subject to label shift (Theorem 5.2) on the replicator dynamics model of Raab and Liu [30].
Brieﬂy, replicator dynamics assumes that the proportion of agents in a population choosing one strategy over another grows in proportion to the ratio of average utilities realized by the two strategies.
The cited model additionally assumes X =
R , Y = { 0 , 1 } , and a monotonicity condition for S given by dd x Pr S ( X = x | Y =1)
Pr S ( X = x | Y =0) > 0 .
Label shift under the discrete-time ( t ) replicator dynamics may be expressed in terms of group qualiﬁcation rates Q g :=
Pr t ( Y =1 |
G
=
g ) and agent utilities ( i.e. , group- and feature-independent values
U y, ˆ y ) such that, in each group, the popularity and average utility associated with a label determines its frequency at the next time step t +1 . [Figure] [Table] [Figure]
[Table] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table] [Figure] [Table] [Figure]
Denote the fractions of group-conditioned, feature-independent outcomes with the expression ρ y, ˆ y g :=
Pr t ( ˆ Y =
ˆ y, Y =
y | G = g ) and abbreviate the fraction-weighted utility as u y, ˆ y g ( t ) :
= U y, ˆ y · ρ y, ˆ y g .
We may then represent the replicator dynamics as
[Equation]
To apply Theorem 5.2, we also observe that | β + g − β −
[Equation]
| [Equation]
1 g − ρ 0 , 1 g | [Equation] 1 , 1 g + ρ 0 , 1 g , where β
+ g and β − g represent the true positive rate and false positive rate for group g , respectively, and we use the change in qualiﬁcation rate as our measurement of label shift, i.e. , B g = | Q g [ t + 1] − Q g [ t ] | .
When demographic parity is perfectly satisﬁed, we note that the acceptance rate ( ρ 1 , 1 g + ρ 0 , 1 g ) is group-independent.
Theorem 6.6.
For DP subject to label replicator dynamics,
[Equation]
In Figure 3(b), we graphically represent all possible states of an initially fair system (thus determining β and ρ as a result of the monotonicity condition) by the tuple of qualiﬁcation rates for each group.
With the dynamics prescribed by Equation (27), we depict the rate of change of disparity given a ﬁxed, locally DP -fair policy, and compare this to the theoretical bound when B g = | Q g [ t + 1] − Q g [ t ] | .
Interpreting our results, we note that the bound lacks information about the relative directions of the change in acceptance rates for each group, and thus over-approximates possible fairness violations when group acceptance rates shift the same direction.
When group acceptance rates move in opposing directions, however, the bound gives excellent agreement with the modelled replicator dynamics.
Comparisons to Real-World Distribution Shifts
We now compare our special-case theoretical bounds ( i.e. , label/covariate shift) to real-world distribution shifts and hypothetical classiﬁers.
We use American Community Survey (ACS) data provided by the US Census Bureau [16].
We adopt the sampling and pre-processing approaches following the Folktables package provided by Ding et al.
[13] 6 to obtain 1,599,229 data points.
The data is partitioned by (1) all ﬁfty US states and (2) years from 2014 to 2018.
We use 10 features covering the demographic information used in the UCI Adult dataset [4], including age, occupation, education, etc. , as X for our model, select sex as binary protected group, i.e., G ∈ { g = female , h = male } . We set the label Y to whether an individual’s annual income is greater than $50K.
To apply our label-shift or covariate-shift bounds, we ﬁrst need to verify whether the two datasets satisfy either of these assumptions.
We adopted a conditional independence test [22], which takes data from source and target domains as input and returns a divergence score for each covariate and label variable, reﬂecting to what extent the variable is shifted between distributions.
We ﬁnd that the likelihood that the covariates shift across US states is approximately two orders of magnitude higher than for labels.
More speciﬁcally, there are 4 covariates, including class of worker (probabilistic divergence score of 2.67e-2), hours worker per week (3.56e-2), sex (3.56e-2) and race (2.55e-1), that are more likely to be shifted than the label variable (1.29e-4). For temporal shifts within states, we ﬁnd that the label variable is more likely to be shifted (0.1) than all the other covariates (which are below 0.01), approximately two orders of magnitude in favor of label shift over covariate shift.
We therefore compare the disparities of hypothetical policies on these distributions to bounds generated from the corresponding, approximately satisﬁed assumptions.
On this data, we train a set of group-dependent, linear threshold classiﬁers Pr π ( x,g ) ( ˆ Y =1) =[ σ ( w · x ) >
τ g ] , for a range of thresholds τ g and τ h for each source distribution.
Here, σ ( · ) is the logistic function and w denotes a weight vector.
We then consider two types of real-world distribution shift: (1) geographic , in which a model trained for one state is evaluated on other US state in the same year, and (2) temporal , in which a model trained for 2014 is evaluated on the same state in 2018.
[Table]
b) CA −→ NV t a u _ g 0.0 0.2 0.4 0.6 0.8 1.
t {+a u _ h 0.00.20.4 0.
+}
0.81.0 C h a n g e i n V i
o
l a t i o n o f D e m o g
r a p
h
i c P a r i t y 0.010.000.010.020.
(c) CA:
→t a u _ g 0.0 0.2 0.4 0.6 0.8 1.
t a u _ h {+0.00.20.4 0.
 0.81.0+} C h a n g e i n V i
o
l a t i o n o f D e m o g
r a p
h
i c P a r i t y 0.01 0.00 0.
[Figure]
) TX: 2014 →
We graphically compare the theoretical bounds of Theorem 4.1 and Theorem 5.2 for the increased violation of DP subject to covariate and label shift, respectively, to the simulated violations for our model and data in Figure 4.
We provide additional examples and an evaluation of bounds for EO subject to covariate shift (noting that label shift preserves EO in theory) in Appendix E.2.
Despite the fact that geographic or temporal distribution shifts only approximately satisfy the assumptions of covariate or label shift, these comparisons demonstrate that our theoretical bounds are not vacuous, approximately bounding the change of fairness violation across real-world domain shifts.
For geographic shifts, the covariate shift EO bounds (Appendix E.2) correctly overestimate disparity and tighten near accurate policies, while our DP bounds are useful only for a subset of policy thresholds (Figure 4(b)). add speciﬁc pointer.
e.g, 4.a, that one is 4.b For temporal shift, the label shift bound for DP correctly overestimates the real change of DP violations but still remains at the same order of magnitude (Figure 4(c) and 4(d)).
Conclusion and Discussion
In this paper, we have developed a unifying framework for bounding the violation of statistical group fairness guarantees when the underlying distribution shifts within presupposed bounds.
We hope that this work can generate meaningful discussion regarding the viability of fairness guarantees subject to distribution shift, the bounds of adversarial attacks against algorithmic fairness, and evaluations of robustness with respect to algorithmic fairness.
We believe that, just as published empirical measurements are of limited use without reported uncertainties, fairness guarantees must be accompanied by bounds on their robustness to distribution shift.
Future work remains to apply our framework for to problem of fairness transferability in settings with more complicated distribution shift dynamics.
For example, compound distribution shifts [33], which compose covariate shifts and label shifts, cannot be treated by composing the theoretical bounds developed herein without additional information regarding intermediate distributions.
Another potential future direction is to develop reasonable bounds on anticipated distribution shift from models of human behavior and exogenous pressures.
Acknowledgement This work is supported by the National Science Foundation (NSF) under grants
IIS-2143895, IIS-2040800 (FAI program in collaboration with Amazon), and CCF-2023495.