Bounding and Approximating Intersectional Fairness through Marginal Fairness
Abstract
Discrimination in machine learning often arises along multiple dimensions (a.k.a. protected attributes); it is then desirable to ensure intersectional fairness —i.e., that no subgroup is discriminated against.
It is known that ensuring marginal fairness for every dimension independently is not sufficient in general.
Due to the exponential number of subgroups, however, directly measuring intersectional fairness from data is impossible.
In this paper, our primary goal is to understand in detail the relationship between marginal and intersectional fairness through statistical analysis.
We first identify a set of sufficient conditions under which an exact relationship can be obtained.
Then, we prove bounds (easily computable through marginal fairness and other meaningful statistical quantities) in highprobability on intersectional fairness in the general case.
Beyond their descriptive value, we show that these theoretical bounds can be leveraged to derive a heuristic improving the approximation and bounds of intersectional fairness by choosing, in a relevant manner, protected attributes for which we describe intersectional subgroups.
Finally, we test the performance of our approximations and bounds on real and synthetic data-sets.
Introduction
Research on fairness in machine learning has been very active in recent years, in particular on fair classification under group fairness notions, see e.g., [16, 30, 34, 33, 7, 28].
Such notions define demographic groups based on so-called protected attributes (e g., gender, race, religion) and impose that some statistical quantity be constant across the groups.
For instance, demographic parity imposes that the class-1 classification rate is the same for all groups but other notions were defined such as equal opportunity [16] or calibration by group [7]—see a survey in [4].
As exact fairness is too constraining, one often measures unfairness , which roughly quantifies the distance to the fairness constraint.
Most works on fair classification consider a single protected attribute and hence only two (or a small number of) groups.
Then, they use measures of unfairness to evaluate and penalize classifiers in order make them more fair.
This is making an implicit but very fundamental assumption that one can estimate the unfairness measure from the data at hand.
With only a few groups, this assumption is indeed easily satisfied as there are sufficiently many data points for each group.
In many—if not most—real-world applications, there are multiple protected attributes (typically 10-20) along which discrimination is prohibited [1, 2].
It is then desirable to consider the strong notion of intersectional fairness , which roughly specifies that no subgroup (defined by an arbitrary combination of protected attributes) is unfavorably treated.
In that case, however, estimating the unfairness measure becomes very challenging: as the number of groups is exponentially large (e.g., 2 10 for 10 binary protected attributes), it is very likely that the dataset has at least one subgroup for which there are very few (or zero) data point.
A potential solution is to treat each protected attribute separately through its marginal unfairness (which is easy to estimate); but it was observed in several real-world and algorithmic examples that it is not sufficient to ensure intersectional fairness [8, 5, 21, 22].
This raises the question: How to estimate intersectional fairness from data, and what is its precise relation to marginal fairness?
To date, only very few papers have tackled this issue.
[21, 22] adopt a definition of intersectional fairness that weights the unfairness of each group by its size.
This allows them to get large-samples generalization guarantees of empirical estimates (hence solving the estimation issue), but then it does not protect minorities since it allows a very high unfairness for tiny subgroups—which is contradictory to the intuitively desired behavior.
[17] makes a similar assumption by considering only subgroups above a minimum size, which eases estimate generalization.
[14] on the other hand uses the more natural definition of intersectional fairness based on the worst treated group irrespective of its size; but they consider only a few protected attributes, precisely to have enough data points on each subgroup to estimate intersectional unfairness.
[13] extends this work by proposing methods to interpolate for subgroups for which too few points are available, based on Bayesian machine learning models.
However, this work is empirical and does not give any guarantee on the estimates obtained.
In this paper, we also use the natural (strong) definition of intersectional fairness but we take instead a purely statistical approach.
We view the protected attributes as random variables to understand intersectional fairness and how it related to marginal fairness more finely.
Contributions: We identify sufficient conditions under which intersectional fairness can be exactly derived from marginal densities, which clarifies when marginal unfairness is a good estimate of intersectional unfairness.
We prove probabilistic bounds on intersectional unfairness based on marginal densities and independence measures of the protected attributes, that we show are easy to estimate.
We propose a method to improve the approximation of intersectional unfairness and the theoretical bound based on grouping carefully some of the protected attributes together, which we do through a heuristic by leveraging the independence measures exhibited in our bounds.
We perform experiments on real and synthetic datasets that illustrate the performance of our approach.
In particular, we show that grouping with our heuristic does improve the approximation of intersectional unfairness.
To the best of our knowledge, our work is the first work to exploit statistical information to better understand and estimate intersectional (un)fairness.
Our work is fairly general and can be instantiated for a variety of standard fairness notions (demographic parity, equal opportunity, etc.).
For simplicity, we focus on discrete protected attributes and on classification, but most of the core results can be extended to other cases.
Further Related Works: [31] proposes a unified framework to train a fair classifier under intersectional fairness metrics, but without taking into account regimes with sparse group membership data.
Some works propose to audit the accuracy of fairness metrics in contexts other than intersectionality, when there are missing data [35] or when there are unlabeled examples [18].
Others tackle the problem of intersectionality beyond group fairness, e.g., [32] considers causal intersectional fairness.
Finally there has been some interest [24, 10] in a different formulation of intersectional group fairness as a multi-objective optimization problem where each objective is the discrimination faced by a given protected group.
Another interesting approach to fairness is individual fairness developed in [12], however this is quite different from group fairness metrics on which we focus on and our techniques do not apply.
Setting and Models 2.1 Basic Setting
Notational convention : Wherever useful, for any two random variables V and W , we will use the shorthand p V ( v )=
Pr( V = v ) , p V,W ( v, w )=
Pr( V = v, W = w ) and p V | W ( v | w )=
Pr( V = v | W = w ) .
Consider a multi-class classification task.
A given individual is described by a tuple of random variables ( X, A, Y ) drawn according to a distribution D where X is the features vector, Y is the label with values in Y , and A =
( A 1 , ..., A d ) is a d -tuple of protected attributes.
The only variable used to make a prediction is X and the only variable to measure unfairness is A , but otherwise there are no constraints and A can be a part of X .
We denote the support of X , Y , A and A k for 1 ≤ k ≤ d , by X , Y , A and A k respectively.
We assume that A is finite (hence discrete).
For a deterministic classifier h , ˆ Y = h ( X ) is the predicted class for a random individual.
The classifier h is fixed, as we are interested in measuring its fairness and not finding a fair classifier.
To compare the discrimination between groups, we consider a second random variable A ′ such that ( A ′ | ˆ Y ) is independent and identically distributed (i.i.d.) to ( A | ˆ Y ) .
Some authors look at the difference in the treatment of protected groups as a ratio [14], and some others as a difference [21].
Here we choose to study discrimination in terms of ratio.
We further apply a logarithm to symmetrize the discrimination measure between two protected groups and for ease of computation.
We will consider Statistical Parity for simplicity of exposition, but other group fairness metrics can be either derived directly or adapted using the methods developed in this paper (see Appendix A.1).
We define our measure of unfairness as follows:
Definition 2.1.
For a distribution D and a classifier h , we define the intersectional unfairness and the k th protected attribute marginal unfairness as:
[Equation]
with u ( y, a, a ′ )=
(cid:12)(cid:12)(cid:12) log (cid:16)
[Equation]
( ˆ Y
[Equation] [Equation]
One could think that if the marginal unfairness of each protected attribute is smaller than some ϵ > 0 , then the overall unfairness is smaller than ϵ ; measuring u M =sup k u ∗ k corresponds to this idea.
As stated in the introduction this is not sufficient to describe unfairness and we can still have u ∗ > u M .
We can rewrite (1) as u ∗ =sup Y log(sup
A p
ˆ Y | A / inf A p ˆ Y | A ) , and similarly for u ∗ k .
This means that to measure unfairness we only need to analyze the function p
ˆ Y | A .
2.2 Estimation of Unfairness
If we want to estimate unfairness, the most straightforward approach is to estimate the probability mass function p A, ˆ Y and then to compute the unfairness over these estimated distributions.
The main difficulty in estimating the unfairness is estimating inf p ˆ Y | A , as we can upper bound the sup by 1 , but we cannot easily lower bound the inf .
For a data-set of n samples and d protected attributes, we denote for ( a, y ) ∈ A×Y the counts by group and prediction as N a,y = (cid:80) ni
=1 1 [( A ( i ) , ˆ Y ( i ) )=( a, y )]
where ( A ( i ) , ˆ Y ( i ) ) is the i -th i.i.d.
realization of ( A, ˆ Y ) .
The empirical probability is then defined as ˆPr( ˆ Y =
y, A = a )=
N a,y /n .
[14] shows in Theorem VIII.3 that the error made by using empirical estimates is decreasing in N a , which means that there needs to be sufficient data for each protected group to estimate u ∗ .
When there are many protected groups the probability that at least one group receives no sample is high, and in this case there is at least one a in A for which the empirical probability ˆPr( ˆ Y =
y | A = a ) is undefined, hence the inf and sup cannot be computed.
[13] and [14] alleviate this issue of 0 -counts by using a Dirichlet prior of uniform parameter α > 0 .
This yield the Bayesian estimates ( N a,y + α ) / ( n + |A||Y| α ) , that are then used to compute the estimator u B .
They also propose other methods to estimate p ˆ Y |
A which empirically performs better, but without guarantees; whereas u B has the nice property that u B is a consistent estimator of u ∗ .
This is because of the consistency of the Bayesian probability estimates and by applying the Continuous Mapping
Theorem for max and min which are continuous.
Note that the empirical estimator (with α = 0 is also consistent, but has infinite bias.
Nonetheless, u B has the drawback that for a low amount of samples and when the number of protected groups is high, it is almost determined deterministically by the parameter α and cannot be trusted.
If
N a =0 for a protected group a , the estimated distribution is uniform on ˆ Y |
A = a
and this group does not affect the computation of the sup and inf .
Hence if the most discriminated group is among the undiscovered one, we risk making an important error on the estimation.
When N a increases, we gain more information on the distribution of ˆ Y | A .
However, when N a is still low for all groups, the estimated distribution of the inf of ˆ Y |
A = a is almost entirely determined by the prior parameter α .
2.3 Probabilistic Unfairness
When the number of protected subgroups grows arbitrarily large, it may be useless to try to guarantee fairness for every single one of them, regardless on how many people this truly affects.
Should a decision maker sacrifice any potential predictive performance in order to guarantee fairness?
It could be argued that an algorithm which discriminates 1 person among a 1000 can be described as fair to an extent.
We may even be able to directly compensate the small amount of persons discriminated against if possible.
Let us consider another example: if a company has clients on which it leverages machine learning predictions to make decisions, it would seem very limiting to guarantee fairness for clients among specific protected groups for whom we will almost never deal with.
Nevertheless, if the underlying clients distribution changes, our decision making process should also reflect this change in terms of fairness.
This motivates looking at unfairness probabilistically in ( ˆ Y , A, A ′ ) .
To do that we define U =
u
( ˆ Y , A, A ′ )
the random variable which corresponds to randomly choosing a prediction, and then independently selecting two protected groups according to p
A | ˆ Y to compare them.
We now define our notion of probabilistic unfairness:
Definition 2.2.
For ϵ ≥ 0 and δ ∈ [0 , 1] , we say that classifier h over distribution D is ( ϵ, δ ) -probably intersectionally fair if Pr( U > ϵ ) ≤ δ .
It can be seen for some given ϵ as a statement on the expected size of the population that is not being discriminated too much against.
Probable intersectional fairness corresponds to searching for quantiles of U .
We define the δ -probabilistic unfairness as ϵ ∗ ( δ )=
min { ϵ ∈ R | Pr( U > ϵ ) ≤ δ } .
It is the (1 − δ ) -quantile of U .
We also know by definition that any classifiers over any distributions is ( u ∗ , 0) -probably intersectionally fair, as we have U ≤ u ∗ with probability 1 .
This shows that probabilistic fairness is a relaxed version of the hard intersectional unfairness as lim δ
→ 0 ϵ ∗ ( δ )=
u ∗ , and thus can be made arbitrarily close to intersectional fairness.
In order to give more intuition on what this measure of fairness represents, we will briefly only for this paragraph consider discrimination of protected groups compared to the predictions distribution p
ˆ Y instead of between groups, meaning that we now measure | log( p ˆ Y |
A /p ˆ Y ) | .
Suppose that a prediction model will be deployed over a population of n individuals.
Then if the classifier is ( ϵ, δ ) -probably intersectionally fair, this means that E A, ˆ Y [ (cid:80) ni
=1 1 [ u ( ˆ Y ( i ) , A ( i ) ) >
ϵ ]] the expected number of people that faces a discrimination more than ϵ is less than nδ .
This allows us to measure and control the size of the population that may face a difference in treatment that would be deemed too high.
It corresponds to the notion of fairness we were searching for.
For more comparisons between these different notions, see Appendix A.3.
As a remark, looking at E [ U ] , it can serve as a lower bound of u ∗ because E [ U ] = (cid:80) y,a,a ′ p ˆ Y ,A,
A ′ ( y, a, a ′ ) u
( y, a, a ′ ) ≤ u ∗ .
This represents the average discrimination in a population between two protected groups.
This is weaker than the notion presented above and is only mentioned in passing.
Probabilistic fairness can be especially relevant in the context where A are continuous sensitive attributes.
Indeed, even for a very basic multivariate normal distribution on A , we will end up with u ∗ = ∞ which is unhelpful.
Yet by considering this notion of probabilistic fairness we end up with finite (hence comparable) measures of unfairness where the discriminated population size can be explicitly controlled; see Appendix A.4 for some examples.
All in all, this notion of probabilistic unfairness, beyond its main interest of being a relaxed version of intersectional unfairness, could be in itself helpful for decision makers.
Measures of Independence and Theoretical Bounds
We now focus on providing valid ( ϵ, δ ) couples for probable intersectional fairness.
First note that while the intersectional unfairness u ∗ is hard to estimate, it is much easier to estimate the marginal unfairness u M .
The work done by [22] in the different setting of weighted unfairness, however, shows through experiments that across multiple classifiers and data-sets, u ∗ and u M can be uncorrelated, correlated, or even equal.
Building on this observation, we would like to approach u ∗ using marginal quantities estimable for reasonably-sized data-sets.
3.1 Intersectional Unfairness with Independence
Since the intersectional unfairness takes into account the interactions between all the protected attributes A k , one could guess that if the A k are mutually independent, this implies that u ∗ is close to u M .
Our first result is not far from this intuition, but we also need to take into account the influence from the classifier h .
Indeed, even if the protected attributes are independent, since the classifier makes predictions based on X which may encode redundant information from some A k , there can be interaction between those protected attributes through the classifier.
See Appendix B.1 for a counter example with the independence of the A k only but no clear relationship between marginal and intersectional fairness.
Proposition 3.1.
If the protected attributes A k are mutually independent and mutually independent conditionally on ˆ Y , then
[Equation]
Sketch of proof.
The main idea is to decompose p A and
p A | ˆ Y as their products of marginals using the independence assumptions, and using the fact that the sup taken over a product of functions with independent variables is distributed over the product.
The inequality is obtained because the sup of a sum is smaller than the sum of the sup .
See proof in Appendix B.2.
This theorem gives us a first sense on how intersectional unfairness relates with marginal unfairness in some contexts.
This shows us that if the independence conditions are fulfilled, then u ∗ becomes easy to estimate.
What we provide here are conditions and a equation to derive a direct relationship between the intersectional unfairness and the marginal unfairness of each A k .
These are unfortunately too strong conditions to actually expect and are almost never randomly satisfied, but they help us give insight into the relationship between marginal and intersectional fairness.
It also drives the analysis conducted in the next sub-section.
We would like to relax the independence criteria while still using marginal information from the problem.
3.2 Bounds on Probable Intersectional Fairness
In order to bound the probable intersectional unfairness and relate it with the strictly independent case, we want to use some measure of independence.
We want to bound in probability the joint probability density Pr( A = a ) with the product of its marginals (cid:81) dk
=1 Pr( A k = a k ) .
We will use one of the possible multivariable generalization of Mutual Information known as Total Correlation [29]:
[Equation]
where H ( A ) is the Shannon Entropy of A .
Similarly we define the conditional total correlation as
C
[Equation]
( p A | ˆ Y ( A | ˆ Y ) / (cid:81) k
p
A k | ˆ Y ( A k | ˆ Y ))]
= ( (cid:80) dk
=1 H ( A k | ˆ Y ))
− H ( A | ˆ Y ) where
H ( A | ˆ Y ) is the conditional entropy of A given ˆ Y .
Note that both can also be written in terms of a KL or expectation in ˆ Y over conditional KL divergence, which means that C ( A ) ≥ 0 and C
( A | ˆ Y ) ≥ 0 .
From these measures of independence, we intuitively define the following two random variables,
L = log( p A ( A ) / (cid:81) k
p
A k
( A k ))
and L y = log( p A | ˆ Y ( A | ˆ Y ) / (cid:81) k
p
A k | ˆ Y ( A k | ˆ Y )) .
By definition we have that E [ L ]= C ( A ) and E [ L y ]= C
( A | ˆ Y ) .
We denote σ and σ y the standard deviation of these two variables.
We have the following property:
[Equation]
The equivalence between independence and C ( A )=0 comes from rewriting C ( A ) as a KL and the fact that KL( P ∥ Q )=
if and only if P = Q almost everywhere.
For C ( A | ˆ Y )=
E y [KL( p A | ˆ Y =
y ∥ ⊗ p A i
| ˆ Y = y )] we also use that the expectation of a positive random variable is 0 if and only the variable is 0 almost everywhere.
When σ =0 then L = c is a constant which means that
p A = (cid:81) k
p
A k e c , and using that the probabilities must sum to 1 we have e c
=1 hence L = c =0 .
The same arguments apply for σ y .
We denote I ( V, W )
=
H ( V ) − H
( V | W ) the mutual information between a variable V and
W .
With these definitions, we can now derive the following theorem which bounds the probable intersectional fairness with independence measures and functions of marginal densities:
Theorem 3.2.
For δ ∈ (0 , 1] , any classifier h over a distribution D is ( ϵ 1 , δ ) and ( ϵ 2 , δ ) -probably intersectionally fair with
[Equation]
where s ∗ = ( σ 2 / 3 + σ 2 / 3 y ) 3 /and
γ = C ( A ) −
[Equation]
( A
[Equation]
Sketch of proof.
We apply Chebyshev’s inequality to L and L y for some introduced parameters α to bound the tails of these random variables, while making sure that overall the probability bounds stay larger than 1 − δ .
We can then compute inequalities on p A and p A | ˆ Y , and take the inf for a and sup for α .
This leads to a constrained minimization problem that can be solved, which yields s ∗ .
The full proof is in Appendix B.3.
For ϵ 2 we additionally use that p ˆ Y | A ≤ 1 as Y is discrete.
We observe that both ϵ 1 and ϵ 2 are composed of one term in s ∗ related with the δ -confidence, and a quantity with marginal information.
Aditionnaly ϵ 2 also includes a term in γ that corresponds to some form of mutual information correction.
We can control the confidence in this bound with the parameter δ .
Because s ∗ =
if and only if σ = σ y = 0 and combined with (5) we can see that s ∗ somewhat measures how far we are from the conditions of Proposition 3.1.
With ϵ 1 we see that when s ∗ goes to zero, we recover exactly the conditions of Proposition 3.1.
In order to prove Theorem 3.2, we used Chebyshev’s inequality.
We can derive a similar proof for other concentration inequalities, specifically with Chernoff bounds through the estimation of the moment generating function, which often leads to tighter bounds.
However this leads to harder quantities to estimate in addition to having to solve a non-convex optimization problem, see Appendix
B.4.
To conclude this section we provide additional intuition on the relationship between marginal and intersectional fairness.
For this we temporarily change our definition of unfairness for this paragraph only: suppose we are now only interested in the unfairness regarding one outcome y ∈ Y , say y = 1 , and let us redefine our unfairness, probabilistic unfairness, γ and s ∗ accordingly (see in Appendix (15)).
We can then derive the following corollary from the proof of the above Theorem:
Corollary 3.3.
Denoting (Ω , T , Pr) the probability space on which ( A, A ′ ) is defined, there exists an event F so that for f ( a )
=
(cid:81) dk
=1 p ˆ Y =
y | A k ( a ) /p
d ˆ Y
=
y we have
[Equation]
and
Pr( F | ˆ Y = y ) ≥ 1 − δ,
[Equation]
11) and the same inequalities hold for sup ω ∈ F
p ˆ Y =
y | A ( A ′ )( ω ) for the same event F .
The proof can be found in Appendix B.3.
This means that there is a fraction of the relevant pairs population of size bigger than 1 − δ , for which we can give intervals for the extreme values of p ˆ Y =
y | A A and p
ˆ Y =
y | A A ′ over this fraction F .
These intervals are centered and reduce around a unique quantity as s ∗ goes to 0 .
We also have that | log(sup ω ∈ F
p
ˆ Y =
y | A ( A )( ω ) / inf ω ∈ F p
ˆ Y =
y | A ( A )( ω )) | goes to u ∗ and that | log(sup ω ∈ F f ( A )( ω ) / inf ω ∈ F f ( A )( ω )) | goes toward the quantity derived in the corresponding Proposition 3 .
1 as δ goes to 0 .
This tells us that in a way we approach the quantity derived in Proposition 3 .
1 when s ∗ and δ go to 0 .
3.3 Estimation of the measures of independence
Theorem 3.2 trades the precise estimation of u ∗ with an upper bound, but with much easier quantities to estimate.
More specifically, as they are information measures, we can leverage the extensive literature on statistical estimators and entropy estimation.
We can intuitively see that the estimation of s ∗ and γ will be easier to handle because even the estimation with the empirical distribution ˆ p A, ˆ Y is always well defined, and is a Maximum Likelihood Estimator (MLE) as continuous functions of MLE.
They are well defined because s ∗ and γ are functions of entropies and of the quantities
Q ( P ) =
(cid:80) i p i log( p i ) 2 for a probability distribution P , which is finite event for p i
= 0 because x (cid:55)→ x log( x ) and x (cid:55)→ x log( x ) 2 are continuous at 0 .
Contrarily to u B we do not have to use any prior to obtain a well defined estimator.
In addition, using the delta method on the sum of entropies, for which the MLE is asymptotically normal (See [27] 3.1), shows that ˆ γ is asymptotically normal.
For more information on the estimation of entropy, mutual information or total correlation we defer to [27, 26, 3, 6, 15] to name but a few.
Moreover even with the very simple MLE, we can obtain Lerror upper-bounds for H ( P ) in O (log( | P | ) 2 /n ) where | P | is the number of outcomes for a discrete distribution P [20].
This bound depends only on the number of outcomes (supposed known), and not the actual distribution.
Using the same tools, we derive a rough error bound for Q ( P ) :
Proposition 3.4.
[Equation]
Sketch of proof.
We apply the methods described in [20] that bounds the bias using approximation theory for Bernstein polynomials and bounds the variance using the Efron-Stein inequality.
See proof in Appendix B.6.
More efficient estimators can be created using methods of [19], nevertheless the main interest of this proposition is to show that these quantities have an error rate depending on the number of samples n , and not the number of samples per group N a , which is much better.
Beyond the practical use of these inequalities and approximations, these theorems also show one crucial idea: we can relate intersectional and marginal unfairness with the help of information on the independence of the protected attributes.
Refined approximations and inequalities
In the previous section, we have derived conditions for marginal unfairness to directly relate to u ∗ , and bounds on probable intersectional fairness.
We now would like to propose an approximation of u ∗ using similar ideas.
Looking at (6), (3), and indirectly through Corollary 3.3 it seems natural to propose as one possible approximation of u ∗ the following quantity:
[Equation]
For the rest of the article, we thus now only focus on s ∗ and u I .
Compared with u B the estimator with Bayesian prior, it does not depend on a prior parameter, and is usually well defined as we only need N a i ,y the number of samples per a i and y to be strictly positive instead of all N a,y .
However this estimator of u ∗ is not consistent.
We will show that the previous bounds can be improved and that we can make our estimator consistent by gradually grouping together the protected attributes as the number of samples increases.
4.1 Grouping protected Attributes together
Until now, we have always decomposed the protected attributes A on their marginals A i .
However it may be that we have more than just marginal information available.
Take the example of 4 protected attributes A =(
A 1 , A 2 , A 3 , A 4 ) .
For a set t ⊆ { 1 , 2 , 3 , 4 } , we define A t =( A k ) k ∈ t .
We may not have enough data to compute the full intersectional unfairness, but it may be possible to compute it for the grouped protected attributes A { 1 , 2 } =(
A 1 , A 2 ) and A { 3 , 4 } .
We can use the same decomposition as we did before on the new marginals attributes (which corresponds to flattening A 1 and A 2 together) with support A { 1 , 2 } =
A 1 ×
A 2 and A { 3 , 4 } =
A 3 × A 4 .
More generally, let q be a partition of { 1 , ..., d } =[ d ] .
For a partition q , we denote A ( q ) =(
A t ) t ∈ q .
This is only a different way to group together the marginal attribute, and is the same as A .
Whenever quantities are changed according to some partition q , it will be indicated with ( q ) .
For each of the new marginal attributes defined by a set t of q , the new marginal unfairness u ∗ t corresponds to the intersectional unfairness of the ( A k ) k ∈ t .
If the A t are independent, and independent conditionally on ˆ Y , we can apply Proposition 3.1 and obtain directly u ∗ through the newly defined marginals.
If we relax the independence conditions, the same arguments of the previous section still apply, and we can look at the bounds and approximations defined by these new marginal densities.
We denote the new approximation with partition q by u ( q ) I =
[Equation]
where we are using the new marginals defined by q .
If we use the partition q of singletons then u ( q ) I = u I , and if we use the trivial partition then u ( q ) I = u ∗ .
The constraints of independence for these new marginals should be more feasible than the original marginals, hence it is possible that the A t fulfill the independence conditions, even if the A k do not (the trivial partition is such an example).
If we have enough data to compute the marginal densities derived from q and the A t fulfill the independence conditions, we can then compute u ∗ through the partition q .
Of course most of the time the independence conditions are not satisfied satisfied for a partition q .
Nonetheless because s ∗ measures how far we are from the independence conditions, we can more carefully select a partition among those for which we can compute the new marginal densities.
4.2 Efficient Partition Selection
Let Q be the set of all feasible partitions q , that is Q = { q ∈ P
([ d ]) | ∀ t ∈ q, ∀ ( a t , y ) ∈
A t ×
Y , N a t ,y > 0 } with P ([ d ]) the set of all partitions of [ d ] .
This set represents the set of partitions for which we can compute the newly defined marginals without having to use a prior parameter.
Note that Q is a random set that converges to P ([ d ]) almost surely as the number of samples n increases.
If q ∈ Q , then any partitions q ′ finer (meaning that any element of q ′ is a subset of an element of q ) than q is in Q as well.
We will say that q can be merged further if there exists a partition q ′ ∈ Q so that q is finer than q ′ .
Note that the choice of a partition q does not change the value of u ∗ but only that of u ( q ) I .
We therefore want to find a good feasible partition q in Q so that we can expect heuristically | u ( q ) I − u ∗ | to be the lowest among the partitions.
There are two criteria that should help us decide which partition q ∈ Q to choose from.
Algorithm 1 Greedy Partition Finder input: Protected attributes data and occurrences of ˆ Y require: The partition of singletons is feasible q ∗ ← the partition of singletons repeat
M = {{ t 1 ∪ t 2 }∪ q ∗ \ ( { t 1 }∪{ t 2 } ) , ( t 1 , t 2 ) ∈
q ∗ 2 , t 1 ̸ = t 2 } s ∗
min ← + ∞ for q in M do if q is feasible and s ∗ ( q ) <
s ∗ min then ( s ∗ min , q ∗ ) ← ( s ∗ ( q ) , q ) end if end for until M = ∅ or s ∗ min = ∞
(Nothing possible to merge) return: q ∗
If a partition q ′ is coarser than q (which means that q is finer than q ′ ), then reasonably the approximation is better with q ′ than q .
The reasoning is that by taking coarser partitions, we are taking more interactions between the protected attributes into account.
For example the coarsest partition which is the whole set gives us the intersectional unfairness as mentioned earlier.
However because the ‘finerthan’ relationship is only a partial order, we are not able to choose between any two sets.
Because Theorem 3 .seems to hint that there is a relationship between the error | u ( q ) I − u ∗ | , and the distance to the independence conditions s ∗ , the second criterion will be to select the partitions q with the smallest s ∗ ( q ) defined as s ∗ but taking the marginals in q .
These two criteria are closely linked.
Selecting coarser partitions does tend to yield partitions with smaller s ∗ , but not always.
We give some details on relationship between s ∗ ( q ) of a partition q compared to a coarser one in Appendix
C.2.
More crucially, finding a good partition with a small s ∗ ( q ) will also improve our inequalities as they are a function of s ∗ ( q ) which decreases on average as shown in Figure 5 as the number of sample grows (and as the partitions get coarser).
In principle, finding the best partition according to our criteria requires enumerating all feasible partitions which is computationally intractable.
Instead we propose a greedy heuristic that we describe in Algorithm 1.
We start from the finest partition (the partition of singletons), look at all the feasible partitions (with enough data) that can be obtained from merging two elements of the current partition, select the one with the smallest s ∗ , and repeat until there are no coarser partitions with enough data.
Note that when we want to verify that there is enough data available, we may need to do it multiple times for the same subset of protected attributes.
This is an expensive call so it is more efficient to do memoization and remember if there is enough data available for a given subset once encountered, which we do using a hash table to reduce the lookup time.
We denote this partition q ∗ .
We have the following property with the proof in Appendix C.1:
Proposition 4.1.
The estimator u ( q ∗ )
I is a consistent estimator of u ∗ .
This proposition shows that u ( q ∗ )
I is relevant in estimating u ∗ , while not needing to use a Bayesian prior with parameters that may overwhelmingly affect the estimation.
Note that instead of using Q which ensures that N a t ,y > 0 , we can instead use Q τ for τ ∈ N which ensures that for any q ∈ Q τ ,
N a t ,y > τ for t ∈ q .
Experiments
In this section, we present experimental results that show how our inequalities and approximations perform on real and synthetic data-sets, and compare their estimation error rates as the number of samples grow.
All the code used in our experiments can be found in the supplementary material or at https://github.com/mathieu-molina/BoundApproxInterMargFairness.
5.1 Data-sets and processing
In order to compare how well u B and u I perform as estimators on data on datasets with a high number of protected attributes, we need to compute u ∗ which is as discussed above inherently difficult.
We will always measure the unfairness with respects to the empirical distribution of the dataset.
For this empirical distribution to yield a well defined fairness measure, we need that N a,y > 0 for all a and y .
This means that if we want to take into account a high number of sensitive attributes, we have to pick a very large dataset.
We used US Census data from 1990 [11] which contains n =2 , 458 , 285 samples, and for which we identified many potential protected attributes.
We then train a Random Forest binary classifier on a poverty binary label, where we weight the labels differently so as to obtain about the same number of predictions for each outcome.
However we still do not have N a,y > 0 on the whole dataset.
To alleviate this issue, we will consider subsets of the protected attributes for which this is true, and we will measure fairness with respect to these subsets.
We obtain about 100 different subsets with d =protected attributes, that we denote as D i which is the original dataset where we only kept the i -th subset of protected attributes and the predictions ˆ Y .
Each of these subsets yield different values of u ∗ and s ∗ .
We pick 12 (for computational reasons) different D i with various values of u ∗ and s ∗ .
Some examples of the final protected attributes include sex, not speaking English at home, being overweight, being Hispanic, and others.
We will always take δ =0 .
1 when relevant.
We also conduct experiments on synthetic data.
We generate ( A, ˆ Y ) probability distributions from a
Dirichlet distribution, thus we can directly compute u ∗ without dealing with a very large dataset.
We take d =10 .
This synthetic data is one of the worst case for the approximation of u ∗ with u I , as the marginal distributions are a sum of 2 d − 1 i.i.d.
random variables that all converges to 1 / 2 as d grows.
We therefore will not plot u I for the synthetic data (it is close to 0 ).
Nonetheless, this synthetic data remains useful in order to compare the error rates between u B and ˆ s ∗ .
We denote by P i a generated probability distribution.
We generate 12 of them.
5.2 Experiments Results
We first want to compare the convergence rate of u B , s ∗ and u I to their asymptotic value.
To do that, and because they can take different values, we compute for each estimator
ˆ T n that converges in probability to T the relative expected L 2 error rate L r 2 ( ˆ T n ) =
E [( ˆ T n − T ) 2 /T 2 ] .
We fix a number of available samples n from 100 to 2 , 000 , and we sample without replacement from the datasets.
From these available samples, we compute all our estimators.
We denote by ˆ u I the estimator of u I computed with the empirical marginal densities for n samples.
In order to compute L r 2 , for each subset and each sample size n , we sample from D i and P i 20 times for each fixed number of samples n .
We see in Figure 1 on the left-most plot, that ˆ u I is reasonably close to u ∗ on average.
Still the gap between ϵ ∗ and ϵ 1 is quite big.
Other bounds such as ϵ 2 or with other concentration inequalities are generally a bit more efficient, but we focus here on comparing the error rates between s ∗ and u B , and on how u I performs.
The other two plots look at the L r 2 for the various estimators, with the middle one being with the real datasets D i , and the right on the synthetic datasets P i .
We can see that ˆ s ∗ and ˆ u I converges much faster than u B .
Moreover, it seems that the difference in error rate will only grow bigger as d increases, as there is a bigger gap for P i .
We can also see that u B is unreliable, because the error rate varies a lot depending on α , and can even increase.
This is because the parameter α dominates the computation of u B as discussed earlier.
[Figure]
We now conduct similar experiments, but this time using partitions.
We can see in the middle plot of
Figure 2 that ˆ u ( q ∗ ) I performs better.
The choice of τ the count threshold for grouping always gives reasonable approximations, with τ = 1 being close to u B , and τ big makes it close to u I .
Most importantly, the apparent good error rate of u B is merely an artifact of the current range of u ∗ being above the starting values of u B for these α .
It is clear that u B is unreliable by looking at the left plot in Figure 2 : the estimation with u B at n = 2000 for different values of α varies very little when u ∗ varies (it is almost not a function of u ∗ ).
This means that u B depends very little on the data for low amount of samples.
Even if it is not perfect, u ( q ∗ ) I still has better performance and is more coherent.
We note that the approximation performs well comparatively only when d is high, and considering more sensitive attributes should make an even bigger difference.
These results combined with Proposition 4.1 show that u ( q ∗ ) I is a relevant estimator of u ∗ with scarce data and high number of protected attributes.
Concerning s ∗ ( q ∗ ) the right-most plot shows that while it is not completely monotone, s ∗ ( q ∗ ) does decreases on average when using partitions as the number of sample increases.
The upper bound will become tighter as n grows, which will make bigger groupings of protected attributes possible.
[Figure]
Discussion
In this work, we presented new methods to approximate and to bound (in high probability) a strong intersectional unfairness measure, based on statistical information computable from a reasonable dataset.
Our results highlight the key role of independence of the protected attributes conditionally to the classifier, and propose to approach it via a smart grouping of some attributes—which our theoretical bound allows us to compute via an efficient heuristic.
Our experiments show that the approximations proposed here perform reasonably well for data-sets with a high number of protected attributes, but that our bounds are not very effective.
However their main interest is that it gives insight into the link between marginal and intersectional fairness, which was the main goal of this work.
It also helps us derive the proposed approximation.
We expect that more effective bounds could be derived for our notion of probabilistic fairness, for instance by making additional assumptions on the distribution, but presumably without an explicit dependence on independence measures and marginal densities, making the link between marginal and intersectional fairness harder to see.
In order to train fair models using the proposed approximations or bounds of this paper, we can use soft counts to compute the empirical densities (based on the classifier score for instance) as suggested in [14].
This makes the approximations and bounds differentiable, and ensure that we can apply gradient based methods so as to solve a constrained or penalized optimization problem using these quantities.
We hope that our approach will enable the development of improved bounds, raise interest in the proposed notion of probabilistic unfairness which we think is crucial to the development of fair algorithms, as well as the use of our approximations to penalize classifiers in order to train intersectionally fair classifiers.
Acknowledgments
This work has been partially supported by MIAI @
Grenoble Alpes (ANR-19- P3IA-0003), by the
French National Research Agency (ANR) through grant ANR-20-CE23-0007, and by TAILOR (a project funded by EU Horizon 2020 research and innovation programme under GA
No 952215).
The authors are hosted at the CREST lab (CNRS, GENES, Ecole Polytechnique, Institut Polytechnique de Paris).
A Additional elements on Measures of Fairness A.1 Generalization to other measures of fairness
Throughout the whole paper, we used a specific measure of fairness for simplicity.
Nevertheless, the same arguments apply to a broader set of fairness measures, by modifying u ∗ .
To define u ∗ , we decided to take the log of a ratio.
We note that when taking the sup over all possibles a and a ′ in A , sup A 2 u (1 , · , · ) ≤ ϵ is equivalent to the definition in [14], that is to say:
[Equation]
However if we want to define a measure of unfairness between two protected groups, it is reasonable for it to be symmetric in the groups considered.
We make it so by applying log and an absolute value function to the above middle quantity.
Other distances and pseudo distances can also be chosen, such as | Pr( ˆ Y = 1 | A = a ) − Pr
( ˆ Y = 1 | A = a ′ )
| .
It is also symmetric, but may be less useful in comparing models with many protected attributes that are not designed to be fair, as this measure will be close to 1 most of the time, making the comparison less precise between two different models.
We can also modify U and the definition of probabilistic fairness accordingly to obtain other desirable measures of unfairness.
Say that we are only interest in the unfairness related to the outcome ˆ Y = 1 .
Taking U ′
=
u (1 , A, A ′ ) and Pr( U ′ >
ϵ | ˆ Y = 1) ≤ δ the new definition of probabilistic fairness in this case, we can derive similar propositions and theorems as done in this paper.
We only need to take the expectation and variance with respect to Pr( ·| ˆ Y = 1) for L and L y .
This yields statistical quantities which are harder to interpret ( (cid:80) a ∈A Pr( A = a | ˆ Y = 1) log(Pr
( A = a ) / (cid:81) dk
=1 Pr( A k = a k ) ) but that should remain easy to estimate as they are always well defined because, considering the empirical distribution ˆ p we have ˆ p A
( A = a )
=
= ⇒ ˆ p A ( A = a | ˆ Y = 1) = 0 .
The changed definitions would be the following if we are only interested in the outcome y ∈ Y :
[Equation]
with u ( a, a ′ )=
(cid:12)(cid:12)(cid:12) log (cid:16) Pr
[Equation]
ˆ Y
[Equation]
y
[Equation]
Notably, we obtain a much nicer variant of Proposition 3.1, with u ∗ =
(cid:80) dk
=1 u ∗ k .
Similarly we can also change only the underlying probability distribution.
We can replace the underlying probability Pr by Pr Y =1 .
Using this new probability measure we see that u ∗ is a relaxed version of Equality of Opportunity for a binary predictor defined in [16] by
[Equation]
Indeed, u ∗ = 0 is now equivalent with Equality of Opportunity.
Practically, changing the underlying probability does not make much difference as showed in [21] because this amounts to measuring unfairness on the part of the dataset for which Y = 1 .
We compared the treatment faced by groups between them, such as looking at the discrimination between men and women.
Another possibility is to measure the difference in treatment faced by a group compared to a reference value.
This reference value is most of the time taken to be the population average of the decision criterion
E A
[ p ˆ Y | A ( y | A )]
= p ˆ Y ( y ) .
Therefore instead of evaluating p ˆ Y | A ( y | a ) /p
ˆ Y | A ( y | a ′ )
we evaluate p ˆ Y |
A ( y | a ) /p
ˆ Y ( y ) .
Finally we can change over which treatment criterion we want to evaluate differences.
In this paper we decided to look at the variable ˆ Y .
We can similarly define our fairness measure with Y .
We can actually use any ( X, A, Y ) -measurable random variable Z instead of ˆ Y .
For instance | ˆ Y − Y | , which tells us whether or not the prediction is correct for binary classification, can be a good candidate.
Combining all of the above comments
[Equation]
we can consider a wider array of fairness metrics for which variations of the techniques and theorems described in this paper apply
[Equation]
A.2 Some variants of Theorem 3.2 for modified fairness measures
Intersectional Fairness in terms of absolute difference: We consider the following definition of unfairness:
[Equation]
The new version of Theorem 3.2 is
Theorem.
For δ ∈ (0 , 1] , any classifier h over a distribution D is ( ϵ 1 , δ ) -probably intersectionally fair with
[Equation]
Intersectional Fairness when comparing to the population average: We consider the following definition of unfairness:
[Equation]
The new version of Theorem 3.2 is
Theorem.
For δ ∈ (0 , 1] , any classifier h over a distribution D is ( ϵ 1 , δ ) -probably intersectionally fair with
[Equation]
The proof for both of these variants is exactly the same as for 3 . 2 until we obtain an upper and lower bound on sup p ˆ Y | A and inf p ˆ Y | A .
We then use the fact that for E ⊂ R , sup (
x,y ) ∈
E 2 | x − y | ≤ sup E x − inf E y , and sup x ∈ E | log( x/y ) | ≤ max { log( y/ inf E x ) , log(sup E x/y ) } .
A.3 Comparison between other measures of fairness
In [21], a different fairness metric is used.
Indeed, instead of simply measuring unfairness as the unweighted difference | Pr( ˆ Y = 1 | A = a ) − Pr( ˆ Y = 1) | , they use the weighted difference
Pr( A = a ) | Pr( ˆ Y = 1 |
A
= a ) − Pr( ˆ Y = 1) | .
For this subsection, we will use as a definition of unfairness u ∗ = sup a ∈A u ( a ) with u ( a ) =
| Pr( ˆ Y = 1 |
A = a ) − Pr( ˆ Y = 1) | .
We define the weighted unfairness used in [21] as w ∗ = sup a ∈A w ( a ) with w ( a ) =
p A ( a ) u ( a ) the weighted version of u .
This definition yields very useful statistical properties in terms of the unfairness estimation, and [21] shows with Theorem 2.11 that the error made using the empirical estimator is less than ˜ O
( (cid:112) ((1 + V CDIM ( H )) log( n ) − log( δ ))
/n )
with high probability 1 − δ .
Unfortunately this notion of unfairness is hard to control as the meaning of w ∗ ≤ ϵ may be difficult to use for a decision maker, and can lead to the discrimination of groups of small sizes compared to using u ∗ .
This is already discussed and supported empirically in [14].
We will briefly give some inequalities relating these quantities.
We have that
[Equation]
Through these equations we see that w ∗ cannot approach u ∗ .
The advantage of the notion of probable intersectional fairness compared to w ∗ is two-fold: we can be arbitrarily close to u ∗ , and through δ we explicitly control the size of the population that faces discrimination.
Additionally we will present an example, which shows that when the number of protected groups grows large, the notion of weighted unfairness can become inadequate for certain scenarios compared to probabilistic unfairness.
We will consider that we have 991 protected groups with three different protected groups sets of sizes 1 , 495 and 495 , for which we will denote any of their elements by a 1 , a 2 , and a 3 respectively.
We will consider that Pr( A = a 1 ) = 0 .
01 , and that the remaining protected groups are distributed uniformly Pr( A = a 2 ) =
Pr( A = a 3 ) = 0 .
Suppose that Pr( ˆ Y = 1 | A = a 1 ) = 1 and Pr( ˆ Y = 1 |
A = a 2 )
=
Pr( ˆ Y = 1 |
A
=
a 3 )
= 1 / 2 .
Then Pr( ˆ Y = 1) = 1 / 100 + 99 / 200 = 101 / 200 .
Thus u ( a 1 ) = 1 − 101 / 200 = 99 / 200 , w ( a 1 ) = 99 / 20000 , u ( a 2 ) =
u ( a 3 ) = 101 / 200 − 1 / 2 = 1 / 200 , and w ( a 2 ) =
w ( a 3 ) = 1 / 200000 .
Which means that w ∗ = 99 / 20000 and the model is (1 / 200 , 0 . 99) -probabilistically fair.
Now suppose that Pr( ˆ Y = 1 | A = a 1 ) = 1 , Pr( ˆ Y = 1 | A = a 2 ) = 1 , and Pr( ˆ Y = 1 |
A
=
a 3 ) =
Then we have Pr( ˆ Y = 1) =
Thus u ( a 1 ) = 99 / 200 , w ( a 1 ) = 99 / 20000 , u ( a 2 ) = 99 / 200 , w ( a 2 ) = 99 / 200000 , u ( a 3 ) = 101 / 200 , and w ( a 3 ) = 101 / 200000 .
Which means that w ∗ = 99 / 20000 and the model is (101 / 200 , 0 . 99) -probabilistically fair.
Here we see from these two examples, that 99% population of their population saw their unfairness multiply by about a 100 times while w ∗ did not change.
But probabilistic unfairness did manage to capture this change.
What we see is that when there is a high number of protected groups, relatively bigger groups tend to determine the weighted measure of unfairness w ∗ , but they can still consist of only a very small part of the total population overall.
We present here two simple inequalities relating ( ϵ, δ ) probabilistic fairness with w ∗ and E
[ U ] .
[Equation]
Proof.
Let A ϵ = { a ∈ A | u ( a ) ≤ ϵ } and A Cϵ its complementary set.
If a ∗ = arg max w ( a ) ∈
A ϵ then w ( a ∗ ) =
p A ( a ∗ ) u ( a ∗ ) ≤ sup A p
A ϵ .
Otherwise using that
Pr( A Cϵ ) ≤ δ , we have w ( a ) ≤ δu ∗ .
Now for E [ U ] :
E [ U ] =
(cid:88) a ∈A ϵ p A ( a ) u ( a )
+ (cid:88) a ∈A
Cϵ p A ( a ) u ( a ) ≤ (cid:88)
a ∈A ϵ p A ( a ) ϵ + (cid:88) a ∈A
Cϵ p A ( a ) u ∗ =
ϵ Pr
A ϵ )
+ u ∗ Pr
( A ϵ ) ≤ ϵ + δu ∗ .
A.4 Intersectional Fairness and Continuous Protected Attributes
Here we will show that when A is continuous, even for reasonable distributions, we might end up with u ∗ = ∞ .
Whereas our definition of probabilistic unfairness still has finite values, and can therefore be used as an interpretable tool to compare unfairness across models.
Suppose that we have a random vector ( A 1 , A 2 , ..., A d , ˆ Y ) distributed according to a multivariate normal N ( µ, Σ) with µ and Σ the mean and covariance.
Because ˆ Y is continuous, we will instead use the density f ˆ Y | A in the definition of u ∗ .
Because this vector is distributed according to a multivariate normal, the conditional distribution is still normal and we can derive the exact parameters.
The conditional distribution is
[Equation]
28) with ¯ µ a linear form in a , and ¯Σ that depends only in Σ .
Basically, we can make the mean go to ∞ by making a go to ∞ .
Hence for a given y ∈ Y , we have that inf a ∈A
f ˆ Y | A ( y | a ) =
for all y , which means that the unfairness is always infinite.
Whereas our notion of probabilistic fairness, is finite and computationally tractable as we need to evaluate δ
= E [ 1 [ U > ϵ ]] .
It goes to 1 as ϵ goes to ∞ .
If we want to compare two machine learning models, and we do not want to compare for a specific point δ , then ϵ ∗ ( δ ) can be seen as a function of ϵ , and we can compare these functions.
If for two models h 1 and h 2 one function is always above the other, we could say that one is more fair than the other.
As an example, we consider for d = 10 the couples ( A, ˆ Y ) ∼ N (0 , Σ w ) with Σ w generated through a Wishart distribution, and ( A, ˆ Y ) ∼ N (0 , Σ c ) with Σ c = ( 1 + I d ) / 2 and 1 the constant matrix equal to 1 .
We then compute the probabilistic fairness on Figure 3 by computing the expectation of[ U > ϵ ] .
[Figure]
There are other fairness metrics specifically for continuous attributes, such as in [25] the HGR coefficient between A and ˆ Y , but which may be less interpretable to decision makers.
B Missing proofs and elements of partB.1 Counter example with independence of the sensitive attributes
Let us define the following probability distribution on ( A 1 , A 2 , ˆ Y ) , with A 1 , A 2 , and ˆ Y binary:
16 =
Pr( A 1 = 0 , A 2 = 0 , ˆ Y = 0) =
Pr( A 1 = 1 , A 2 = 1 , ˆ Y = 0) = Pr( A 1 = 0 , A 2 = 1 , ˆ Y = 1) =
Pr( A 1 = 1 , A 2 = 0 , ˆ Y = 1) and 16 =
Pr( A 1 = 0 , A 2 = 0 , ˆ Y = 1) = Pr( A 1 = 0 , A 2 = 1 , ˆ Y = 0) =
Pr( A 1 = 1 , A 2 = 0 , ˆ Y = 0) =
Pr( A 1 = 1 , A 2 = 1 , ˆ Y = 1)
We have p A = 1 / 4 , and p A 1 = p
A 2 = 1 / 2 .
Therefore A 1 ⊥⊥ A 2 .
We have p ˆ Y = 1 / 2 , hence p
A 1 | ˆ Y (0 | 0) =
p
A 2 | ˆ Y (0 | 0)
= 1 / 2 and p A 1 ,A 2 | ˆ Y (0 , 0 , | 0)
= 3 / 8 ̸ = p
A 2 | ˆ Y (0 | 0)
p
A 1 | ˆ Y (0 | 0) , therefore the A k are not independent conditionally on ˆ Y .
Because p A 1 | ˆ Y = p
A 2 | ˆ Y = 1 / 2 , any form of marginal unfairness is 0 , and u ∗ =
log((3 / 4) / (1 / 4))
= log(3) ̸
= 0 .
In this example we have mutual independence of the A k , independence between A k and ˆ Y , but still no meaningful relationship between intersectional and marginal fairness because we did not have independence conditionally on ˆ
Y .
B.2 Proof of Proposition 3.
Using the assumed independence, for any a in A and y in Y we can rewrite p ˆ Y |
A with marginal quantities:
[Equation]
Because the numerator is a product of independent variables (in the functional sense), taking the sup in A yields:
[Equation]
We can do the same for inf .
Hence
[Equation]
and we obtain
[Equation]
The inequality is obtained by triangle inequality and because sup y ∈Y sup ( a k ,a ′ k ) ∈A 2 k u k ( y, a k , a ′ k ) =
u ∗ k by definition.
B.3 Proof of Theorem 3.2 and Corollary 3.
Theorem.
For δ ∈ (0 , 1]
[Equation]
any classifier h over a distribution D is ( ϵ 1 , δ ) and ( ϵ 2 , δ ) -probably intersectionally fair with
[Equation]
where s ∗ = ( σ 2 / 3 + σ 2 / 3 y ) 3 /and
γ = C ( A ) − C
( A | ˆ Y )=
(cid:0) d (cid:88) k
=
I ( A k , ˆ Y ) (cid:1) − I ( A, ˆ Y ) .
Proof.
We want to show that our classifier is ( ϵ, δ ) probably fair for a given δ .
We will first bound in probability L and L y , to be able to approach the joint densities through the product of marginal densities.
We will denote by µ , µ y , σ and σ y the expectations and variances of L and L y Let us apply Chebyshev’s inequality to L .
We obtain that
[Equation]
Using the fact that {| L − µ | < α 1 } ⊂ {| L − µ | ≤ α 1 } and taking the complementary event we can write that
[Equation]
From this inequality we have
[Equation]
We can do the same for L y with a parameter α 2 > 0 .
Now we want to consider a condition on the parameters
α = ( α 1 , α 2 ) so that the probability of the conjunction of the events {| L − µ | ≤ α 1 } and {| L y − µ y | ≤ α 2 } is greater than 1 − δ .
For δ > 0 , α 1 > 0 and α 2 > 0 , a sufficient condition is that σ 2 α 2 + σ 2 z α 22 ≤ δ .
We can show this using complementary event and Boole’s inequality:
[Equation]
We define g ( α ) =
σ 2 α 21 + σ 2 z α 22 − δ .
For any α such that g ( α ) ≤ 0 we have with probability at least 1 − δ that
[Equation]
where φ ( µ y , µ ) =
e µ y − µ , ψ ( α ) =
e α 1 + α 2 , and f ( y, a ) = (cid:81) dk
=1 p ˆ Y |
A k ( y | a k ) /p
ˆ Y ( y ) d .
Hence by taking the sup over a and inf over α on the right hand-side, we obtain
[Equation]
As it is a product of functions of independent variables, sup a ∈A f ( y, a ) is just the product of the sup of each p ˆ Y | A k .
We will now solve the constrained optimization problem for ψ .
We can write inf e α 1 + α 2 = e inf( α 1 + α 2 ) , so we will just need to solve the simpler problem inf g ( α ) ≤ 0 s ( α ) , with s ( α ) =
α 1 + α 2 .
Let us compute the gradients of s and g :
[Equation]
We will now show that this is a convex problem.
The function s is linear thus convex, and we will now compute the hessian of g :
[Equation]
Clearly we have that the determinant of H g , Det ( H g ) is strictly positive.
Therefore H g is definite positive, and g is convex.
And for δ > 0 there is a feasible interior point by taking α 1 and α 2 big enough, which means that Slater’s conditions hold (e.g. a convex constraint with a feasible interior point).
We will now analyze the KKT conditions for minimization with the dual parameter c ≥ 0 :
[Equation]
We obtain that α
[Equation]
=
[Equation]
and
[Equation]
Clearly c > 0 otherwise the first two lines cannot be 1, hence using the last equation we have g ( α 1 , α 2 ) = 0 .
We now develop this last equality to obtain c :
[Equation]
Plugging c in the previous expressions we have = ⇒  
[Equation]
(cid:115)(cid:18)
[Equation]
[Equation]
(cid:115)(cid:18)
[Equation]
2 / 3 + σ 2 /
y δ
[Equation]
3 (cid:113) σ 2 y =
s ∗ 2 √ δ with  
[Equation]
=
[Equation] [Equation]
y .
Finally the minimum is inf
[Equation]
δ with s ∗ = (cid:16) σ 2 / 3 + σ 2 / 3 y (cid:17) 3 / 2 .
We will now do the same in order to lower bound p
A ( ˆ Y | A ) .
[Equation]
Here we take the sup over α and inf over a instead.
We have that sup ψ ( α ) − 1 = (inf ψ ( α )) − 1 = exp( − s ∗ / √ δ ) .
Because U involves the two variables A and A ′ , we need to bound L ′ and
L ′ y the variables L and
L y that are taken as a function of A ′ instead of A .
Because ( A ′ , ˆ Y ) ∼ ( A, ˆ Y ) , all the computations above still apply, and we have
[Equation]
Hence we only need to replace δ by δ/ 2 in the above inequalities.
Combining everything, we can conclude that when the event {| L − µ | ≤ α ∗ 1 } ∩ {| L y − µ y | ≤ α ∗ 2 } ∩ {| L ′ − µ | ≤ α ∗ 1 } ∩ {| L ′ y − µ y | ≤ α ∗ 2 } occurs, we have
[Equation]
We can conclude that
[Equation]
which means that our classifier is ( ϵ 1 , δ ) -probably intersectionally fair.
Note that ϵ 1 is a function of ( δ, σ, σ y ) .
In order to derive the proof for ϵ
[Equation]
we simply remark that p ˆ Y | A ≤ 1 which can be used to upper bound the numerator.
Therefore when the event {| L − µ | ≤ α ∗ 1 } ∩ {| L y − µ y | ≤ α ∗ 2 } ∩ {| L ′ − µ | ≤ α ∗ 1 } ∩ {| L ′ y − µ y | ≤ α ∗ 2 } occurs, we have
[Equation]
ϵ
[Equation]
We can conclude in the same way as for ϵ 1 .
Looking at the proof, it also holds true that the model will also be (min { ϵ 1 , ϵ 2 } , δ ) -probably intersectionally fair.
Now let us prove Corollary 3.3.
Recall we now suppose that we are interested in only one outcome y ∈ Y , and we redefine our notions of unfairness and probabilistic unfairness, as well as s ∗ and γ , as done in (15).
Corollary.
Denoting (Ω , T , Pr) the probability space on which ( A, A ′ ) is defined, there exists an event F so that for f ( a )
=
(cid:81) dk
=1 p ˆ Y =
y | A k ( a ) /p
d ˆ Y
=
y we have sup
[Equation]
inf
[Equation]
44) and
Pr( F | ˆ Y = y )
[Equation] [Equation]
[Equation]
45) and the same inequalities hold for sup ω ∈ F
p ˆ Y =
y | A ( A ′ )( ω ) for the same event F .
Proof.
Let F be the event defined as follows:
[Equation] [Equation]
What we have shown in the proof of the theorem, is that the event {| L − µ | ≤ α ∗ 1 } ∩ {| L y − µ y | ≤ α ∗ 2 } ∩ {| L ′ − µ | ≤ α ∗ 1 } ∩ {| L ′ y − µ y | ≤ α ∗ 2 } is included in F , and hence Pr( F | ˆ Y = y ) ≥
1 − δ .
Now we just need to take the sup and inf in ω over F for these 4 inequalities in the definition of F to directly obtain the corollary.
Note that we did not obtain a direct statement on U ( ω ) because lower bounding the sup of a ratio of two functions is not easy.
B.4 Additional Bounds on Probable Intersectional Fairness
Looking at how we proved Theorem 3 .
2 , we can derive more bounds by using other concentration inequalities.
Let κ ( t ) =
log( E [ e tL ]) and κ y ( t ) =
log( E [ e tL y ]) be the cumulant generating-function of L and L y .
We define the α -Renyi Divergence between two discrete distributions P and Q of size
S for α > 0 as
[Equation]
These moments generating functions can be expressed as functions of Renyi Divergences, indeed
[Equation]
and κ y ( t ) = log
[Equation]
While κ ( t ) can therefore be estimated using techniques of [ 15 ] for instance, the estimation of κ y is less straightforward.
For λ + and λ − in R , we define I + y ( λ + ) =
sup t ∈ R + { tλ + − κ y ( t ) }
and I − ( λ − ) =
sup t ∈ R − { tλ − − κ ( t ) } .
We apply the generic Chernoff bounds to L and L y :
[Equation]
and
[Equation]
We will apply the same reasoning used in Appendix B. 3 and will only highlight the differences.
[Equation]
We want to ensure that the right hand-side is greater than 1 − δ .
Because we need to make sure that it also holds for L ′ and
L ′ y , we need 2 δ instead of δ .
We define the constraint
[Equation]
we need to have g ( λ + , λ − ) ≤ 0 .
Hence using feasible values of λ + and λ − , the event [ L y ≥ λ + ] ∩ [ L ≤ λ − ] implies
[Equation]
We therefore have the following Theorem:
Theorem B.1.
For δ ∈ (0 , 1] , any classifier h over a distribution D is ( ϵ 3 , δ ) -probably intersectionally fair, with
[Equation]
Compared to using Chebyshev, this should be both tighter as we are using more information than the first and second moment, and this bound should also be more efficient in terms of probability, as we are using one sided concentration inequalities.
B.5 Properties of the cumulant generating-function
We now want to be able to say a bit more on the properties of this constrained minimization problem.
We will show by first recalling and developing useful properties on κ that the problem is non convex and differentiable almost everywhere.
We will list the properties about κ that will be useful to us developed in [9], with [23] being a summary containing all the information needed.
Lemma B.2.
We have that κ is strictly convex and infinitely many times differentiable.
This means that κ ′ is strictly increasing and that it can be inverted on Im( κ ′ ) .
We will write κ ′ ( − 1) =
η , and for all x so that κ ′′ ( η ( x )) ̸
=
we have η ′ = 1 /κ ′′ ( η ) .
We also have that κ (0) = 0 , κ ′ (0)
= µ , and κ ′′ (0)
=
σ 2 .
From these properties we can conclude that η ( µ ) = 0 , and that η ′ ( µ ) = 1 /κ ′′ ( η ( µ ))
= 1 /σ 2 .
We will recall the definition of the convex conjugate of a function.
Definition B.3.
Let E be a Euclidean vector space with scalar product ⟨· , ·⟩ , we define the convexconjugate of a function f : E → R for x ∈ E by
[Equation]
[Equation]
We will now list the useful properties about I = κ
⋆ also developed in [23].
Lemma B.4.
The function I is infinitely many times differentiable on Im( κ ′ ) , I ( µ ) = 0 , and for every x ∈ Im( κ ′ )
we can rewrite I as
[Equation]
Proposition B.5.
The function I + is continuously differentiable on ( −∞ , sup κ ′ ) .
Proof.
We define ˜ I + ( x )
=
I ( x ) if x ≥ µ , and ˜ I + ( x ) =
otherwise.
We will first show that ˜ I
+ =
I + .
We have µ ∈ Im( κ ′ ) because κ ′ (0)
= µ .
Let f x ( t ) =
tx − κ ( t ) .
If x < µ , then because κ ′ is increasing we have for any t ≥
[Equation]
This means that the max on R + is at 0 , and therefore I + ( x ) =
f x (0) =
x − κ (0)
=
= ˜
I + ( x ) .
If x ≥ µ , then for any t ≤ 0 we have
[Equation]
Hence for all t ≤ 0 we have f x ( t ) ≤ f x (0) therefore the sup of f x is not on R − .
Consequently when x ≥ µ we have I + ( x ) =
˜
I + ( x ) .
All in all we can conclude that ˜ I
+ =
I + on Im( κ ′ ) .
Let us analyze the potential discontinuity at µ .
We have I ( µ ) =
and I + = 0 on ( −∞ , µ ) , so the function is continuous on ( −∞ , sup κ ′ ) .
Let us compute I ′ :
[Equation]
and we know that η ( µ ) = 0 .
Hence we have that I + is continuously differentiable on ( −∞ , sup κ ′ ) .
Proposition B.6.
The function e − I + is non-convex at µ .
Proof.
We will simply look at the second derivative of h ( x ) =
e − I + ( x ) for x ≥ µ :
[Equation]
Therefore h ′′ ( µ ) =
− 1 /σ 2 < 0 for σ ̸ = 0 , which means that it is non-convex at µ .
The same proposition applies to I + y
and I − , with the relevant κ or κ y .
This means that g ′′ (( µ y , µ )) =
− (1 /σ 2 + 1 /σ 2 y ) < 0 hence the following corollary
Corollary B.7.
The constraint g is non convex at ( µ y , µ ) .
Finally we remark that when A is finite, the sup of κ ′ is bounded and therefore there are finite values of λ for which I ( λ ) = ∞ .
Hence there are feasible points for any δ ∈ [0 , 1] .
B.6 Errors bounds on Q
Let P =
( p 1 , ..., p S ) be a discrete distribution of size | P | =
S , we want to estimate the quantity
Q ( P ) =
(cid:80) S k
=1 p k log 2 ( p k ) with n i.i.d.
realizations of P .
We denote N k , the number of realizations for category k .
In order to bound the L 2 error of ˆ Q = (cid:80) Sk =1 ( N k /n ) log 2 ( N k /n ) , we will use the bias variance decomposition of ˆ Q :
[Equation] [Equation]
where b ( ˆ Q ) =
E [ ˆ Q ] − Q, Var( ˆ Q
[Equation]
=
E [
[Equation]
ˆ
[Equation]
E
[Equation]
ˆ Q ]
[Equation] [Equation]
The analysis of these error terms is completely derived from [20].
In particular, the method they use for entropy is close to this problem.
They show that the bias term can be bounded by deriving smoothness modulus for the function x
(cid:55)→ x log( x ) , and that the variance term can be bounded using an Efron-Stein inequality.
Here, we need to analyse x
(cid:55)→ x log 2 ( x ) , which is technically more difficult as some nice properties such as the convexity of x log( x ) is lost.
Still we can show the following two lemmas with the proof further down:
Lemma B.8.
[Equation]
Lemma B.9.
[Equation]
Using these two lemmas, we can directly conclude that E [( ˆ Q − Q ) 2 ]
=
O (log 4 ( n ) /n ) .
Note that we will directly use some elements already derived in [20], and will only show here the parts where special care is needed.
Proof of Lemma B.8.
Let f : x (cid:55)→ x log 2 ( x ) .
[20] analyze the statistic of the form F ( P )
=
(cid:80)
Sk =1 f ( p k ) .
We apply Lemma 13 of [20] for discrete functionals of P , which is derived from a corollary of the Efron-Stein inequality, on Q :
[Equation]
We will look for n in N ∗ at the function g : x (cid:55)→ x +1 n log 2 ( x +1 n ) − xn log 2 ( xn ) for x ∈ [0 , n ] .
We have
[Equation] [Equation]
[Equation]
We first look at the term in 1 /n · h ( x ) .
We have that h ′ ( x ) = 2(log
( x + 1) − log( n )) /
( x + 1) which is 0 for x = n − 1 .
Thus arg max x ∈ [0 ,n ] | h ( x ) | ∈ { 0 , n − 1 , n } .
We evaluate h at these points: h ( n − 1) = 0 , h ( n )
= log 2 (1 + 1 /n )
∼ 1 /n 2 , and h (0) = log( n ) 2 .
Hence the max of | h ( x ) | over [0 , n ] is log 2 ( n ) Using on (67) the inequality log(1 + 1 /x ) ≤ 1 /x , and because x (cid:55)→ log( x ( x + 1)) is increasing over R + , we obtain
[Equation]
Finally
[Equation] [Equation]
Proof of Lemma B.9.
In order to bound the bias [20] use the fact that for any f ,
E [ ˆ
[Equation]
] − Q =
[Equation]
(70) with B n ( f )( x ) = n (cid:88)
[Equation]
f
[Equation]
(cid:18) i n (cid:19) x
[Equation] [Equation]
− x
[Equation]
n − i .
The function B n ( f )( x ) is the Bernstein polynomial of f ( x ) .
Lemma 5 .
of [20] shows that for φ ( x ) =
(cid:112) x
(1 − x ) : |
[Equation] [Equation]
[Equation]
with ω 2 φ ( f, t
[Equation]
The quantity ω 2 φ is the second-order Ditzian-Totik modulus of smoothness of f .
It is shown in
Lemma 8 of [20] that for the function x (cid:55)→ x log( x ) (which corresponds the the entropy), ω 2 φ ( f, t ) =
t 2 log(4) /
(1 + t 2 ) .
We will use a proof similar to Lemma 8 to derive the modulus of smoothness for x (cid:55)→ x 2 log( x ) .
In addition, we remark using the triangle inequality that for any f and g continuous functions on [0 , 1] , then
[Equation]
Let f : x (cid:55)→ x log 2 ( x ) for x in [0 , 1] .
By expanding g ( x ) = x log 2 ( x/e ) = x log 2 ( x )+ x − 2 x log( x ) we see that we can rewrite f as f ( x ) = x log 2 ( x/e ) + 2 x log( x ) − x .
Therefore using the previous remark and because ω 2 φ ( x (cid:55)→ x, t )
=
we have
[Equation]
It remains to upper bound ω 2 φ ( g, t ) .
First we will show that g is concave on (0 , 1] .
We compute the first and second derivative of g for x in (0 , 1] :
[Equation]
thus g ′′ (
[Equation]
< 0 .
[Equation]
Hence g is strictly concave over (0 , 1] .
Now we will upper bound ω 2 φ ( g, t ) .
Let t in [0 , 1 / 2] .
To be clear, we will use the same language and logic developed in [ 20 ] for the proof of Lemma 8 to make the comparison easier .
Defining
M = ( u + v ) / 2 ∈ [0 , 1] , then the computation of the second order modulus is an optimization over the regime | u − v | ≤ 2 t (cid:112) M (1 − M ) .
Equivalently, it is in the interval [ M (1 − ∆) , M (1 +
∆)]
∩ [0 , 1] , where ∆ = t (cid:112) (1 − M ) /M .
Because g is strictly concave on [0 , 1] , the maximum of | g ( x ) + g ( y ) − 2 g (( x + y ) / 2) | is reached at the boundaries of the above feasible interval.
We have
[Equation]
Therefore the optimization problem defined by the second order modulus of smoothness is equivalent to the maximization of h ( u, v ) = | g ( u ) + g ( v ) − 2 g (( u + v ) / 2) | over three different regimes:
Regime A
[Equation]
Regime B: u
[Equation] [Equation]
C: u
[Equation]
Over the regime A :
[Equation]
The function M (cid:55)→ M reaches its max over regime
A at t 2 / (1 + t 2 ) .
The function x (cid:55)→ − x log( x ) is positive increasing until x = 1 /e .
Hence because t 2 / (1 + t 2 ) ≤ 1 /e for t ∈ [0 , 1 / 2] , M (cid:55)→ − M log( M ) reaches its max over regime
A also at t 2 / (1 + t 2 ) .
Thus over regime
A h ( u, v ) ≤
[Equation] [Equation]
Over the regime B : h
[Equation]
=
[Equation]
| = | 2 M (log 2 (2 M − 1) − log 2 ( M ))
+ 4 M (log( M
[Equation]
− log(2 M − 1))
+ 2 log(2 M − 1) − log 2 (2 M − 1) | ≤ | 2 M (log 2 (2 M − 1) − log 2 ( M ))
| + | 4 M (log( M ) − log(2 M − 1))
| + | 2 log(2 M − 1) − log 2 (2 M − 1) | = |
[Equation]
|
We will upper bound each of those three terms.
First note that as t ∈ [0 , 1 / 2] , t 2 / (1 − t 2 ) ≤ 1 / 3 and (1 + t 2 ) 2 / (1 − t 2 ) ≤ 25 / 12 Because − log is decreasing and 2 M − 1 ≤ 1 , we have that | log(2 M −
[Equation]
| = − log(2 M − 1) ≤ log(1 + t 2 1 − t 2 )
=
log(1 + 2 t 2 1 − t 2 ) ≤ 2 t
[Equation]
1 − t 2 .
Hence | log(
[Equation]
≤ t
[Equation]
[Equation]
For M ≤ 1 , we have that (2 M − 1) /M ≤ 1 .
Hence
[Equation]
The functions M (cid:55)→ 1 / ( M (2 M − 1)) and M (cid:55)→ M/ (2 M − 1) are decreasing in M over (1 / 2 , 1] and bigger than 1 .
Therefore
[Equation]
and |
[Equation] [Equation]
[Equation]
Combining everything we have over regime B using that M ≤
[Equation]
Over the regime C:
[Equation]
=
[Equation]
) log
[Equation] [Equation]
[Equation]
) log
[Equation] [Equation]
[Equation] [Equation]
[Equation] [Equation]
1+∆) log(1+∆)
|
[Equation]
∆
[Equation] [Equation]
log
[Equation]
[Equation] [Equation]
(1+∆) log
[Equation]
(1+∆) | +
[Equation]
The functions g 1 : ∆ (cid:55)→ ((1 + ∆) log(1 + ∆) + (1 − ∆) log(1 − ∆)) / ∆ 2 and g 2 : ∆ (cid:55)→ ((1 + ∆) log 2 (1 + ∆)
+ (1 − ∆) log 2 (1 − ∆)) /
∆ 2 are both continuous over [0 , 1] hence bounded with a max reached respectively (can be seen graphically, or by looking at the derivative) at 1 and 0 .
With g 1 (1) = 2 log(2) , and for ∆ → 0 :
[Equation]
Finally h (
[Equation] [Equation]
Hence using these bounds over regime A , B , and C , and remarking that it is reached for regime A on t 2 / (1 + t 2 ) , we obtain
[Equation]
By applying on Equation (75) the upper bounds we derived and taking t = n − 1 / 2 , we can conclude
[Equation]
C Using partitions of protected attributes C.1 Consistency of u ( q ∗ ) I
The main idea of this proof, is that when the number of samples n increases, the probability that min A , Y N a,y < τ goes to 0 as n → ∞ .
And when min A , Y N a,y ≥ τ then by definition u ( q ∗ ) I =
ˆ u ∗ which is consistent.
We define for a in A the modified empirical estimator ˆ p A ( A = a ) , with ˆ p A ( A = a ) =
N a /n if
N a > 0 and 1 otherwise.
Using Chebyshev’s inequality and because N a,y ∼ B ( p A, ˆ Y ( a, y ) , n ) we have for ϵ > 0 :
[Equation]
which means that ˆ p A is a consistent estimator of p A .
By Slutsky’s Theorem and because p A ( a ) > 0 , ˆ p A, ˆ Y ( a, y ) / ˆ p ( a ) is a consistent estimator of p ˆ Y | A ( y | a ) .
Hence by the Continuous Mapping
Theorem using the continuous functions max , min , log and | · | , we have that ˆ u ∗ the estimator using the modified empirical probabilities, is a consistent estimator of u ∗ .
Now for the consistency of u ( q ∗ ) I , for τ > 0 and ϵ > 0 , we have
[Equation]
The first term goes to 0 by the consistency of ˆ u .
We will show that the second term also goes to zero as n → ∞ .
For τ > 0 we apply Hoeffding’s inequality on N a,y ∼ B ( p A, ˆ Y ( a, y ) , n ) to obtain the following concentration inequality:
[Equation]
Therefore because |A||Y| is finite we have the consistency of u ( q ∗ ) I .
C
2 Some intuition on the impact of grouping protected attributes
Let q a partition and ρ a partition coarser than q , which means that every element of q is a subset of some element of ρ .
We want to somewhat relate the approximations and inequalities obtained using ρ or q .
Using the fact that ρ is coarser than q , we can define for any r ∈ ρ the partition q r = { t ∈ q | t ⊂ r } of r .
This is a partition because the t are disjoints, cover the whole set so r as well in particular, and any t ∈ q can either be a subset of r or disjoint as ρ is coarser than q .
We redefine the random variable L for these partitions.
We define L ( q ) = log( p A / (cid:81) t ∈
q p
A t ) ◦
A and L ( q,r ) = log( p A r / (cid:81) t ∈
q r p
A t ) ◦ A .
Proposition C.1.
We have
[Equation]
Therefore
[Equation]
and
[Equation]
Proof.
Using the partitions q r we can group together the p
A t terms:
[Equation]
Then by taking the log , we obtain the proposition.
Looking at (107), we have the interesting property that if ρ is a coarser partition than q , then
C ( A ( ρ ) ) ≤ C
( A ( q ) ) .
This corresponds to the intuition that taking coarser partition decreases some measure of independence, which is here the total correlation.
We define ℓ t = log( p
A t / (cid:81) k ∈ t p A k ) and L t = ℓ t ( A t ) .
By applying the previous proposition, and by remarking that any partition is coarser than the set of all singletons we obtain the following corollary.
Corollary C.2.
For any partition q ∈ Q we have
[Equation]
Which is why when using a partition q to group together the protected attributes, we may be able to reduce the original variance of L and L y , hence reduce s ∗ which is an increasing function of the variances.
The decrease in s ∗ is not guaranteed when using a coarser partition because of the covariance terms, but empirically this is often the case.
D Additional Experiments and Plots
In this section we will present additional plots from the experiments conducted in Section 5.
The experiments were conducted on a machine with a i7 7700HQ CPU, and 8gb of ram.
Running all the experiments took about 1 full day.
The main dataset used is a publicly available sample from the 1990 US census.
The US census is legally mandated, hence every citizen has to give its information to the US government.
No identifiable information is available, and the samples were randomly chosen from the original full dataset.
Full information is available at the UCI archive link.
We reproduce here Figure 1 and Figure 2 on Figure 4 and Figure 5 adding the 1 st and 10 th decile but only using α = 1 for u B and τ = 10 to make it readable.
We recall that we always take δ = 0 .
We present on Figure 6 a comparison of the relative error rate between u B , s ∗ , and inf g ( λ ) λ + − λ − where g is estimated through the empirical distribution, and the optimization problem is solved numerically.
We see that while it is easier to estimate than u B , it
[Figure] [Figure]
is harder than s ∗ .
Note that numerically solving the minimization problem may lead to numerical errors for too low number of samples.
We also compare some of the bounds presented throughout this paper.
Here we do not care about their estimation, but only their asymptotic value.
In addition, we want to evaluate the impact of using partitions on these bounds.
In order to do so we take a sample of size n = 2000 of each of our D i , and compute q ∗ .
Then we use the full dataset to compute s ∗ ( q ∗ ) for τ = 10 , γ ( q ∗ ) and inf g ( λ ) λ + − λ − .
We also compute the exact unfairness quantile ϵ ∗ ( δ ) .
We obtain Figure 7.
We see that using partitions seem to always yield tighter bounds, and that most of the time ϵ 1 ≥ ϵ 2 ≥ ϵ 3 .
Even the improved bounds are still far from ϵ ∗ (the optimal bound in probability), but it shows that these bounds can be improved.
We conjecture that if we want to find reliable information on u ∗ when d becomes very large, these bounds can be useful in practice.
Conversely, these bounds and approximations should not be used if sufficient information is available to directly use u B (for instance at least 1 sample by protected group).
[Figure] [Figure]