Fair Ranking with Noisy Protected Attributes
Anonymous Author(s)
AffiliationAddress email
Abstract
The fair-ranking problem, which asks to rank a given set of items to maximizeutility subject to group fairness constraints, has received attention in the fairness,information retrieval, and machine learning literature.
Recent works, however,observe that errors in socially-salient (including protected) attributes of items cansignificantly undermine fairness guarantees of existing fair-ranking algorithmsand raise the problem of mitigating the effect of such errors.
We study the fairranking problem under a model where socially-salient attributes of items arerandomly and independently perturbed.
We present a fair-ranking framework thatincorporates group fairness requirements along with probabilistic information aboutperturbations in socially-salient attributes.
We provide provable guarantees on thefairness and utility attainable by our framework and show that it is informationtheoretically impossible to significantly beat these guarantees.
Our frameworkworks for multiple non-disjoint attributes and a general class of fairness constraintsthat includes proportional and equal representation.
Empirically, we observe that,compared to baselines, our algorithm outputs rankings with higher fairness, andhas a similar or better fairness-utility trade-off compared to baselines............
Introduction
Given a query and a set of m items, ranking problems require one to output an ordering of a smallsubset of items in decreasing order of relevance to the query.
Such ranking problems have beenextensively studied in the information retrieval [46] and the machine learning [45] literature andalgorithms for them are used in applications such as search engines, personalized feed generators andonline recruiting platforms [44, 12, 8].
Several studies have observed that when the outputs of rankingalgorithms are consumed by end-users, e.g., image results for occupation-related queries, articles withdifferent political leanings and job applicants in online recruiting, the outputs can mislead or alter theirperceptions about socially-salient groups [38], polarize their opinions [24, 50] and affect economicopportunities available to individuals [32].
A reason is that relevance (or utilities) input to ranking algorithms may be influenced by human or societal biases, leading to output rankings that skew representations of socially-salient, and often legally-protected, groups such as women and Black people [55].
A growing number of works aim to make the output of ranking algorithms fair with respect to sociallysalient attributes [74, 58].
As for notions of fairness, in the case when each item belongs to one oftwo socially-salient groups ( G 1 or G 2 ), equal representation requires that, for every k , (roughly) k 2items from each of G 1 and G 2 appear in the first k positions of the output ranking.
Proportionalrepresentation requires that at most k · | G ℓ | m items from each G ℓ appear in the first k positions.
Fairnesscriteria that generalize proportional representation and involve p ≥ 2 groups G 1 , . .
, G p , where eachitem may belong to multiple groups, have also been considered: Given values U kℓ , they require thatat most U kℓ items from G ℓ appear in the first k positions of the output ranking [61, 18].
One setof works in the fair-ranking literature tries to improve fairness in utility-estimation [72, 62, 73, 51].
Such approaches have the benefit that no changes to the existing ranking algorithm are necessarybut they may be unable to guarantee that the output ranking satisfies the required fairness criteria
Another set of works use the given utilities as-it-is and change the ranking algorithm to outputthe ranking with the highest utility subject to satisfying the specified fairness criteria by includingthem as fairness constraints [61, 9, 18, 27, 30].
While these latter approaches can guarantee fairness,they require coming up with new algorithms to solve the arising constrained ranking problems.
Bothapproaches, however, rely on knowledge of the socially-salient attributes of the items [56].
Assuming precise access to socially-salient attributes is reasonable in some contexts and has led tosuccessful deployment of fair-ranking frameworks; see [27].
However, in several contexts, sociallysalient attributes can be erroneous, missing, or known only probabilistically.
For instance, errors canarise due to misreporting, which is a common concern with self-reported attributes [4].
Attributes canalso be missing, as is the case with images in web-search or in settings where it is illegal to collectcertain socially-salient attributes [20].
Often attributes are predicted using ML-classifiers, but suchprediction has inaccuracies [10].
In such cases, one can calibrate the confidence scores of classifiersto derive (aggregate) probabilistic information about the true attributes [35].
Moreover, probabilisticinformation about socially-salient (protected) attributes can be sometimes computed from otherattributes.
For instance, name and location of an individual, combined with aggregate census datamay be used to get a conditional distribution of their race [23, 36, 20].
Even accurate attributesmay be randomly and independently flipped to preserve user privacy, and the distribution of flippedattributes is determined by public parameters of, e.g., the randomized response mechanism [37, 69].
Several models of inaccuracies in data have been proposed [47, 26].
We consider one such model (dueto [5]) to capture inaccuracies in socially-salient attributes.
Each item i belongs to the ℓ -th group with aknown probability P iℓ .
For each item i , the distribution corresponding to P iℓ s over groups is assumedto be independent of corresponding distributions of other items.
This model can be used in cases wherethese probabilities are available or can be derived, as in some of the aforementioned examples (see Section 4 and Supplementary Material A).
In other cases, e.g., when errors are strategic or adversarial,other models are needed.
This model and its variants have also been used by works on designing fairalgorithms in the presence of inaccuracies, for problems including classification [42, 67, 66, 14], subset selection [48], and clustering [25] (Supplementary Material M briefly discusses these works).
In this noise model, while socially-salient attributes are not explicitly specified, one could still useexisting fair-ranking algorithms by first sampling groups for items from the given probabilities.
Indeed,[29] evaluate existing fair-ranking algorithms on attributes obtained from the probabilities derivedfrom ML classifiers.
They find that “errors in [socially-salient attributes] can dramatically underminefair-ranking algorithms” and can cause “[non-disadvantaged groups] to become disadvantagedafter a ‘fair’ re-ranking.”
We confirm this observation on a synthetic dataset when the goal is tofinding a ranking that satisfies equal representation (Section 4).
We assigned each item the sociallysalient group that is most likely and find that when existing fair-ranking algorithms (for equalrepresentation) are run with this group information, they output rankings that significantly violate theequal representation criteria (Figure 1).
Further, we mathematically analyze two natural methodsto sample groups from probabilities and give examples where taking such information as input,existing fair-ranking algorithms output rankings which provably violate the equal representationcriteria (Supplementary Material C).
Thus, new ideas are needed to design fair-ranking frameworksthat can guarantee given fairness criteria under this noise model.
Our contributions.
We present a fair-ranking framework that guarantees given fairness criteria whenthe socially-salient attributes are assumed to follow the probabilistic noise model mentioned above.
In particular, it finds a utility maximizing ranking subject to a class of constraints that only rely ongiven probability distributions (Program (7)).
These constraints relax the given fairness criteria by acarefully chosen factor: for equal representation, the relaxation is by roughly a 1 + 1 √ k multiplicativefactor for position k for any k .
Moreover, instead of sampling the attribute values and applyingconstraints on them, these constraints apply the relaxed-fairness criteria to the expected number ofitems from each group that appear in the first k positions.
We show that these constraints ensurethat any ranking approximately satisfying the given fairness criteria is feasible for them and anyranking feasible for them approximately satisfies the given fairness criteria (Theorem 3.1).
Ourfair-ranking framework works for the general class of fairness criteria introduced earlier, whichinvolve multiple overlapping groups G 1 , . .
, G p and upper bound U kℓ for the ℓ -th group and k -thposition (Theorem 3.1), and for their position-weighted versions (Theorem E.1).
We show that our fair-ranking framework, besides nearly satisfying the given fairness criteria, has aprovably high utility (Theorem 3.1).
Complementing Theorem 3.1, we prove near-tightness of thefairness guarantee (Theorem 3.2): for equal representation fairness criteria, this results shows that itis information theoretically impossible to output a ranking that violates this criteria by less than a multiplicative factor of 1 + (cid:101) O
( 1 √ k ) at the k -th position for any k .
Finally, we give a polynomial-timealgorithm to approximately solve Program (7) (Theorem 3.3).
Empirically, we evaluate our framework on both synthetic and real-world data against standardmetrics like weighted-risk difference (RD) that measure deviation from specific fairness criteria(Section 4).
We compare its performance to key baselines [18, 61, 27, 48] on both single and multipleattributes.
In all simulations, compared to baselines, our framework has a higher maximum fairness(2-10% for RD; Figures 1 to 3) and a similar/better fairness-utility trade-off (Figures 2, 4 and 6 to 9).
Related work.
Work on automated information retrieval dates back to 1940s [43, 21].
Since thenthe IR literature has devoted a significant effort in measuring relevance of items to specific queriesacross different tasks: including, web search [7], personalization [34], and product rating [22]; wealso refer the reader to [46] and the references therein.
In the last three decades, works in the ML literature have also made significant contributions to relevance-estimation [45], by proposingmethods that: (1) supplement traditional IR approaches, e.g., by automatically tuning their–previouslyhard to tune–parameters [65] and by improving their efficiency through clustering-based techniques[64, 3], and(2)substitute traditional IR approaches by neural-network based models to predict itemrelevance [12, 11, 68, 8].
Fair ranking.
Existing works on the fair-ranking problem take diverse approaches: Amongworks that de-bias utilities, different approaches include, post-processing the utilities so that thepost-processed utilities satisfy some fairness requirement [71], introducing a “fairness penalty” inthe objective function used to train learning-to-rank models [62, 73, 49], and modifying featurerepresentations generated by up-stream algorithms so that the utilities learned from the modifiedrepresentations satisfy some fairness requirements [72].
Works that alter the ranking algorithms canalso be further categorized into those which satisfy the constraints for each ranking [18, 70, 27, 30] andthose that satisfy the constraints in aggregate over multiple rankings [61, 9].
Among aforementionedworks, [49] uses a version of adversarial training to make (fair) learning-to-rank models robust tooutliers but, unlike this work, they require socially-salient attributes of items to be accurately knownto specify the “fairness penalty.”
All of the other aforementioned works also need access to thesocially-salient attributes of items.
When protected attributes are inaccurate, these works can fail tosatisfy their fairness and/or utility guarantees [29].
Effect of inaccuracies on fair-ranking algorithms.
Some recent works have considered assessingfairness of rankings and ranking algorithms with missing or inaccurate protected attributes.
[39]analyze the setting where all protected attributes are missing, but can be purchased at a fixed costper item.
They give statistical-techniques to estimate the fairness-value of a given ranking at a smallcost.
[29] use ML-classifiers to infer protected attributes from real-world data and study performanceof the fair-ranking algorithm by [28] when given inferred attributes as input.
While these worksunderscore the need for fair-ranking algorithms to be robust to inaccuracies in protected attributes,they only assess fairness in the presence of noisy protected attributes.
Model of fair ranking with noisy attributes
Ranking problem.
In ranking problems, given m items, one has to select a subset of n items andoutput a permutation of the selected items.
This permutation is said to be a ranking .
There is alarge body of work on estimating the relevance of items and personalizing these estimates to specificusers/queries [46, 45].
We consider a ranking problem where the relevance of items are known.
Abstracting relevance estimation, in this problem, one is given an m × n matrix W , such that placingthe i -th item at the j -th position generates utility W ij .
The utility of a ranking is the sum of utilitiesgenerated by each item in its assigned position.
The algorithmic task in the ranking problem is tooutput a ranking with the highest utility.
We denote rankings by assignment matrices R ∈ { 0 , 1 } m × n ,where R ij = 1 indicates that item i appears in position j , and R ij = 0 indicates otherwise.
In thisnotation, the utility of a ranking is ⟨ R, W ⟩ := (cid:80) mi =1 (cid:80) nj =1 R ij W ij .
Then this ranking problem is tosolve: max R ∈R ⟨ R, W ⟩ .
Where R is the set of all assignment matrices denoting a ranking:
[Equation]
Here, the constraint (cid:80) m i =1 X ij = 1 ensures position j has exactly one item and the constraint 148 (cid:80) nj =1 X ij ≤ 1 ensures that item i occupies at most one position.
While this model captures a varietyof applications, in some cases, the entries of W may be skewed by an unknown amount [40, 16] ornot known accurately [63] and the utility of the ranking may not be linear in the entries of W [2].
These are interesting directions but are not studied in this work.
Fair-ranking problem.
There are several versions of the fair-ranking problem.
We consider a version with p ≥ 2 socially-salient groups G 1 , G 2 , . . .
, G p ⊆
[ m ] (e.g., the group of all women or all Blackpeople) which are often protected by law.
Each of the m items belongs to one or more of these sociallysalient groups (henceforth referred to as just groups).
This fair-ranking problem is to output theranking with maximum utility subject to satisfying certain fairness criteria with respect to these groups.
The appropriate notion of fairness is context dependent, and to capture different fairness criteria numerous fairness constraints have been proposed.
We consider a class of general fairness constraints.160 Definition 2.1 ( Fairness constraints ) .
Given a matrix U ∈
Z n × p + , a ranking R satisfies the upperbound constraint if (cid:80) i ∈ G ℓ (cid:80) kj =1 R ij ≤ U kℓ , for all ℓ ∈ [ p ] and k ∈ [ n ] .
Existing works consider similar constraints and show that they can encapsulate a variety of fairnesscriteria [61].
For instance, when groups are disjoint, to capture equal and proportional representation,one can choose U kℓ := (cid:108) kp (cid:109) and U kℓ := (cid:108) k · | G ℓ | m (cid:109) for all k and ℓ respectively.
(That said, they do notcapture qualitative differences among groups, such as, misrepresentation of demographics in imageresults [38, 55], which could arise even when rankings has sufficient individuals from each group.)
As a running example, we consider the fair-ranking problem with equal representation with twodisjoint groups, i.e.,
[Equation]
To ease readability, we omit ceilings-operators henceforth.
Noise model.
If the socially-salient attributes of items are known accurately, then one can solve thefair-ranking problem.
However, as discussed, in many contexts, attributes are inaccurate, missing,or only probabilistically known.
Several models have been proposed to capture different errors inattributes.
Here, we consider a model (due to [5]) which has also appeared in [25, 42, 48].
Definition 2.2 ( Noise model ) .
Let P ∈ [0 , 1] m × p be a known matrix.
The groups G 1 , . . .
, G p ⊆
[ m ]are random variables, such that, for each i ∈ [ m ] and ℓ ∈ [ p ] , Pr[ G ℓ ∋ i ]
= P iℓ .
Moreover, fordifferent items i ̸
=
j the events G ℓ ∋ i and G k ∋ j are independent for all ℓ, k ∈ [ p ] .
Definition 2.2 makes two key assumptions: the matrix P is known and for each item i , the events
G ℓ ∋ i over groups ℓ are independent of the corresponding events for other items.
Both of theseassumptions hold when attributes are flipped to preserve local differential privacy (Remark A.1).
In other settings, P ’s estimate can be inaccurate and above events may be correlated.
These canadversely affect the performance of our framework.
We empirically study this in simulations where Pis estimated using confidence scores of off-the-shelf classifiers and is miscalibrated (Figures 2 and 3).
Supplementary Material A shows how Definition 2.2 captures both disjoint and overlapping groups.
Fairness constraint with noisy attributes.
Most existing fairness constraints assume that the groupsare deterministic.
Hence, it is not clear how to impose them when groups are random variables,as in Definition 2.2.
One definition is to require the constraints to be approximately satisfied withhigh probability.
Consider the instantiation of this definition for equal representation:
A ranking Rsatisfies ( ρ, δ ) -equal representation, if with probability 1 − δ , at most k 2 (1 + ρ ) items from G ℓ appearin the first k positions in R places for all k ∈ [ n ] and ℓ ∈ [ p ] .
Naturally, one would like to satisfy thisdefinition for small δ, ρ .
However, it turns out to be too stringent and is infeasible for any small δ, ρ .
Proposition 2.3.
No ranking satisfies ( ρ, δ ) -equal representation for ρ < 1 , δ ≤ 12 , and P = (cid:2) 12 (cid:3) m × p .
The proof of Proposition 2.3 shows that any ranking R violates the equal-representation constraintat the 2nd position by a multiplicative factor of 2 with probability 12 .
The issue is that the samerelaxation parameter ρ is used for each position k (whereas the information theoretically bestachievable relaxation parameter at k improves as k increases, this, e.g., follows by Theorem 3.1.)
Motivated by this observation, we consider the following alternate version of upper bound constraints.198 Definition 2.4 ( ( ε, δ ) -constraint ) .
For any ε ∈ R n ≥ 0 and δ ∈ (0 , 1] , a ranking R is said to satisfy( ε, δ ) -constraint
if with probability at least 1 − δ over the draw of G 1 , . . .
, G p
[Equation]
We would like to output a ranking that satisfies Definition 2.4 for small δ and small ε 1 , ε 2 , . . .
, ε n .
Problem 2.5 ( Ranking problem with noisy attributes ) .
Given matrices P , U , and W , find theranking R maximizing utility ⟨ R, W ⟩ subject to satisfying ( ε, δ ) -constraint for some small ε and δ .
2.1 Challenges in solving Problem 2.
In this section we discuss potential approaches for solving Problem 2.5.
In other words, solving:
[Equation]
Even for two disjoint groups, given V , it is NP -hard to decide if the value of Program (4) is at least V(Theorem F.5).
To bypass this hardness, one can consider approximation algorithms.
Program (4) isan integer program (IP) because the entries of the matrix R are required to be integers (Equation (1)).
A standard approach to (approximately) solve IPs is to: (1) consider their continuous relaxation thatdrops the integrality constraints, (2) compute the optimal solution R c of the relaxed problem, and then(3) “round” R c to satisfy integrality constraints while “retaining” its utility and fairness properties.
To take this approach, we first need an efficient algorithm to find R c .
However, not just Program (4),but even its continuous relaxation is non-convex. Hence, it is unclear how to solve it to find R c .
Due to the independence assumption in Definition 2.2, the number of items from G ℓ appearing in thefirst k positions of a ranking is concentrated around its expectation (for large k ).
This implies that if,in expectation, less that U kℓ items from G ℓ appear in the top k positions then, with high probability,the number of items from G ℓ in the top k positions is not much larger than U kℓ .
Using this one canshow that a ranking satisfying the following constraints
[Equation]
(5) also satisfies ( ε, δ ) -constraint for small ε and δ .
One idea is to find the ranking maximizing utility subject to satisfying Constraint (5).
A feature of Constraint (5) is that it is linear in R as 220 E [ (cid:80) i
[Equation]
G
[Equation]
(cid:80) kj =1 R ij ] = (cid:80) mi =1 (cid:80) kj =1 P iℓ R ij and, hence, one may hope to find the ranking with the maximum utility subject to satisfying Constraint (5).
However, the issue is that there are examples whereany ranking satisfying Constraint (5) has 0 utility and there are rankings that satisfy ( ε, δ ) -constraintand have a large positive utility (Lemma F.3).
Hence, this approach can output rankings whose utilityis significantly smaller than the utility of the solution to Problem 2.5.
To overcome this, we relax
Constraint (5) by a carefully chosen position-dependent factor, such that, any ranking satisfying the( ε, δ ) -constraint (for appropriate ε and δ ) is also feasible for our framework.
Theoretical results
In this section we present our optimization framework and its fairness and utility guarantees.
[Equation]
Parameters: Constant c > 1 , failure probability
[Equation]
1] , and k ∈ [ n ]
[Equation]
relaxation parameter
[Equation]
Our Fair-Ranking Program
[Equation]
The above program modifies the program for fair ranking with accurate groups: It has the sameobjective but different constraints.
Instead of sampling the attribute values and applying constraintson the sampled values, Constraint (8) applies upper bounds on the expected number of items in thefirst k positions from group ℓ (Section 2.1).
Further, Constraint (8) relaxes upper bounds U kℓ by asmall position-dependent factor.
Like for Constraint (5), one can show that any ranking satisfying
Constraint (8) also satisfies ( ε, δ ) -constraint (for small ε 1 , . . . , ε n and δ ).
But unlike Constraint (5),and somewhat surprisingly, any rankingthatsatisfies ( ε, δ ) -constraint(for appropriate ε 1 , . .
. , ε n andδ ) must also satisfy Constraint (8).
(In fact, γ k is chosen to be the smallest, up to logarithmic factors,value such that this is true.)
We use this to prove Theorem 3.1’s utility guarantee.
Our first result bounds the fairness and utility of the optimal solution of Program (7).
Theorem 3.1.
Let γ ∈ R n be as defined in Equation (6) .
There is an optimization program(Program (7) ), parameterized by a constant c and failure probability δ , such that for any c > 1 andδ ∈ (0 , 12 ] its optimal solution satisfies ( cγ, δ ) -constraint and has a utility at least as large as theutility of any ranking satisfying (( c − √ c ) γ, δ ) -constraint.
For equal representation, γ k is (cid:101) O (cid:16) 1 √ k (cid:17) .
Thus, Theorem 3.1 guarantees that, with high probability, theoptimal solution of Program (7) multiplicatively violates equal representation at the k -th position byat most 1+ (cid:101) O (cid:16) 1 √ k (cid:17) .
Further, this solution’s utility is higher than the utility of any ranking satisfying aslight relaxation of this fairness guarantee.
Theorem 3.1 can be extended to position-weighted versionsof fairness constraints (Theorem E.1), where the fairness constraint is (cid:80) i ∈ G ℓ (cid:80) j ∈ [ k ] v j R ij ≤ U kℓ (forall k and ℓ ) for specified discount factors v 1 ≥ · · · ≥ v n such as NDCG [33].
If we are also guaranteed
U kℓ ≥ ψk for some constant ψ > 0 and all k and ℓ , then we can improve γ k ’s dependence on δ fromlog 1 δ to (cid:113) log 1 δ (Supplementary Material D.2).
The proof of Theorem 3.1 appears in Section 5.
Lower bound on fairness guarantee.
Our next result complements Theorem 3.1’s fairness guarantee.
[Figure]
( L e ss f a i r) (cid:9)(cid:9)(cid:9) <
f un c ti on c o m pu t
e _ w e
i
gh t e d_ r i s k
d i ff a t 0x f e 96678b a
[Figure]
[Figure]
W e i gh t e d r
i s kd i ff e r e n ce ......
—————————— ϕ —————————— (more fair) (less fair)
Theorem 3.2.
There is a family of matrices U ∈
Z n × p + such that for any U in the family andany parameters δ ∈ [0 , 1) and ε 1 , . . .
, ε n ≥ 0 , if for any position k ∈ [ n ] , ε k ≤ 1 and ε k <max ℓ ∈ [ p ] (cid:113) U kℓ log 14 δ then there exists a matrix P ∈ [0 , 1] m × p , such that it is information theoretically impossible to output a ranking that satisfies ( ε, δ ) -constraint.
This family, in particular, containsthe matrices U corresponding to equal representation and proportional representation constraints.
Since γ k is O (cid:16) log( npδ ) · max ℓ (cid:113) 1 U kℓ (cid:17) , Theorem 3.2 shows that Theorem 3.1’s fairness guarantee isoptimal up to log-factors.
Supplementary Material D.3 proves Theorem 3.2.
An efficient algorithm.
As for solving our optimization program, it is NP -hard to check its feasibility(Theorem D.10).
However, because Constraint (8) is linear in R , the continuous relaxation of
Program (7) is a standard linear program and can be solved efficiently.
Our algorithm (Algorithm 1)solves the standard linear programming relaxation of Program (7) to find a solution R c and then usesa dependent-rounding algorithm by [19] to convert R c to a ranking.
(See Supplementary Material D.4for brief discussion of why straightforward rounding approaches are insufficient.)
Theorem 3.3.
There is a randomized algorithm (Algorithm 1) that given constants d > 2 , afailure probability 0 < δ ≤ 1 , and matrices P ∈ [0 , 1] m × p and W ∈ [0 , 1] m × n , outputs aranking satisfying ( O ( dγ ) , δ ) -constraint and with probability at least 1 − δ , and has a utility at least 269 (cid:0) 1 − 1 d (cid:1) · V − (cid:101) O ( √ dn ) , where V is the utility of any ranking satisfying (( d − √ d ) γ, δ ) -constraint.
The algorithm runs in polynomial time in d and the bit complexity 2 of the input.
The tension in setting d is that decreasing d improves the fairness guarantee and the utility guarantee’ssecond term, but worsens the first term in the utility guarantee.
Under the mild assumption that
V =Ω( n ) , increasing d improves the utility guarantee because the first term in the utility guaranteedominates the second term.
In this case, the utility guarantee improves to (1 − 1 d − o (1))
· V .
Finally,while Theorem 3.3 requires utilities to be between 0 and 1, it can be extended to any non-negativeand bounded utilities by scaling.
The proof of Theorem 3.3 appears in Supplementary Material D.4.
Empirical results
In this section we evaluate our framework’s performance on synthetic and real-world data.
Baselines and metrics.
The correct choice of fairness metric is context-dependent and beyond thescope of this work [60].
To illustrate our results, we arbitrarily fix the fairness metric as weightedrisk-difference (RD).
This is a position-weighted version of the standard risk-difference metric [13]and measures the extent to which a ranking violates equal representation.
The RD of a ranking R is:
[Equation]
284 Where G denotes the ground-truth protected groups and Z is a constant so that RD has range [0 , 1] .
Here, RD
= 1 is most fair and RD
= 0 is least fair.
We compare our framework, NResilient , againststate-of-the-art fair-ranking algorithms: CSV (“greedy” in [18]), SJ [61], and GAK (“DetGreedy” in[27]).
We also compare against MC , which ranks the items, in the subset output by [48]’s algorithm,to maximize utility.
Finally, we compare against the baseline, Uncons , which outputs the utilitymaximizing ranking without fairness considerations.
We present additional discussion of baselinesand results, additional plots for RD, and comparisons with weighted selection-lift (instead of RD)and different choices of U (than the ones below) in Supplementary Material B.
[Figure]
U tilit y .
[Figure] [Figure]
Setup.
We consider the DCG model of utilities [33] and a relaxation of equal representation constraints: (1) Given an intrinsic value w i ≥ 0 , for each item i , we set W ij :=
w i (log ( j + 1)) − 1 ∀ j ∈[ n ] .
(2) Given a parameter ϕ ∈ [1 , p ] , we set upper bounds U kℓ := ϕp · k for each k ∈ [ n ] and ℓ ∈ [ p ] .
Insimulations, we set m = 500 , n = 25 , and vary ϕ from p to 1.
To gain some intuition about therelevant values of ϕ , note that satisfying the 80% rule requires ϕ ≤ 5 p 5 p − 1 , i.e., ϕ ≤ 1 .
{+11 for p = 2and ϕ ≤ 1 . 
 for p = 4 .+} For each ϕ , we draw m items uniformly without replacement and computean estimate (cid:98) P of the matrix P (from Definition 2.2) using, e.g., off-the-shelf ML classifiers or public
APIs (see the paragraphs “Estimating (cid:98) P ” in simulation with image data and “Setup” in simulationon name data).
We infer socially-salient groups (cid:98) G 1 , . . .
, (cid:98) G p via (cid:98) P by assigning each item to itsmost-likely group.
Finally, we run all algorithms using (cid:98) P or (cid:98) G 1 , . . .
, (cid:98) G p as discussed next.
Implementation details.
NResilient and MC take probabilistic information about socially-salientattributes as input and are given (cid:98) P .
CSV , SJ , and GAK require access to socially-salient groups andare given (cid:98) G 1 , . . .
, (cid:98) G p .
NResilient , SJ , and CSV use fairness constraints from Definition 2.1 and aregiven: for each k and ℓ , U kℓ = ϕp · k . MC requires, for each ℓ ∈ [ p ] , an upper bound on the number ofitems from G ℓ that can appear in top- n positions.
It is given ϕp · n for each ℓ ∈ [ p ] .
GAK requires thedesired proportion α ℓ for each group G ℓ and, roughly, satisfies the constraint U kℓ = α ℓ · k for eachk ∈ [ n ] and ℓ ∈ [ p ] .
It is given α ℓ = 1 p for each ℓ ∈ [ p ] , this corresponds to ϕ = 1 (hence, figuresonly plot the GAK at ϕ = 1 ).
As a heuristic, we set γ k = 120 max ℓ ∈ [ p ] (cid:113) 1 U kℓ in all simulations.
Wefind that this suffices and expect a more refined approach to improve the performance of NResilient .
Simulation on synthetic data.
We show that on synthetic data, where error-rates of given sociallysalient attributes vary over groups, existing fair-ranking algorithms have worse RD than Uncons .
Data.
We generate w and P for two groups using code by [48] and fix (cid:98) P = P .
For all items i , w i isi.i.d.
from the uniform distribution over [0 , 1] .
(cid:98) P is constructed such that attributes inferred from (cid:98) Phave a higher false-discovery rate for the minority group compared to the majority ( 40% vs 10% ).
Results.
See Figure 1 for the observed RD averaged over 500 iterations.
We observethat NResilient achieves best RD ( ≈ 0.81), while not losing significant utility ( ≥ 98% ofmax.
; see Figure 4).
MC achieves the next best RD ( ≈ 0.79).
In contrast, CSV , SJ , and GAK , which do not account for noise in the socially-salient attributes, achieve a worse RD( ≤ 0.68) for ϕ ≤ 1 . 2 than Uncons ( ≈ 0.75).
This range of ϕ can be desired in practice: e.g., a platform must set ϕ ≤ 1 . 1 to guarantee the 80% rule is satisfied two groups.
Thus, we observe that existing fair-ranking algorithms may achieve a worse RD than Uncons .
Simulation on real-world image data.
In this simulation, given non-gender labeled images-searchresults and their utilities, our goal is to generate a high-utility and gender-balanced ranking.
Data.
We use the Occupations dataset [15] which contains the top 100 Google Image results for 96occupation-related queries.
For each image, the data has its position in search results, gender (codedas male/female) of the individual depicted in the image, collected via MTurk.
We use the (true)gender labels in the data to compute RD and to estimate (cid:98) P , but do not provide them to algorithms.
Setup.
For each image i , with rank r
i , we define w i :=
(log (1 + r i ))
− 1 .
We say an occupation isgender-stereotypical if more than 80% of images for this occupation have the same gender label(41/96 occupations).
An image is said to be stereotypical if it is in a gender-stereotypical occupationand its gender label is the majority label for its occupation.
We define the socially-salient groups asthe sets of stereotypical and non-stereotypical images in gender-stereotypical occupations. [Figure] a i [Figure]
[Figure] c [Figure] pu t e _ w e i gh t e d_ r i s k
d i ff a t 0x f 13 a b f
[Figure]
ae [Figure] o r e f i r) [Figure]
gh [Figure] —————————— ϕ —————————— (more fair) (less [Figure]
Estimating (cid:98) P .
After pre-processing, we use a CNN-based gender-classifier f [59] to predict the(apparent) gender of the person depicted in each image.
We calibrate the confidence scores outputby f by binning and use these to estimate (cid:98) P (see Supplementary Material B for more details).
We perform this calibration once and on all occupations and, then, use it for gender-stereotypicaloccupations.
Because of this (cid:98) P is miscalibrated (and hence, inaccurate).
For instance, among samplesi for which 0 .
25 ≤ (cid:98) P i 2 ≤ 0 .
5 , more than 75% are labeled as ‘man’ (instead of some percentagebetween 25% and 50%).
This violates the assumption that P is accurately known.
Results.
See Figure 2 for RD and utilities (NDCG) averaged over 1000 iterations.
We observe that
NResilient achieves the best RD ( ≈ 0.81) and has a better RD-utility trade-off than the other baselines.
In contrast, CSV , SJ , and GAK , achieve a worse RD ( ≤ 0.77).
MC achieves the worst RD ( ≤ 0.70)and a worst RD-utility trade-off.
In particular, NResilient ’s RD-utility trade-off strictly dominates allbaselines for RD ≥ 0 . 66 .
This value of RD can arise in practice: Figure 5 plots the RD vs ϕ for thissimulation and shows that if ϕ ≤ 1 . 1 (as required to, e.g., guarantee satisfaction of the 80% rule),then all baselines have RD at least 0.66.
We further evaluate the robustness of NResilient to varyinglevels of noise on the Occupations dataset in Supplementary Material B.5 and observe NResilienthas a better or similar RD than each baseline at all noise levels.
Simulation on real-world name data.
We consider gender and race (encoded as binary) as sociallysalient attributes.
Our goal is to ensure equal representation across the four disjoint groups formed bycombinations of these: non-White non-men, White non-men, non-White men, and White men.
Data.
We consider the chess ranking data [29] which has of 3,251 chess players.
For each player,among other attributes, the data has their full-name, self-identified gender (coded as male/female),
FIDE rating, and race (Asian, Black, Hispanic, White) collected via MTurk.
We use the (true) genderand race labels in the data to evaluate RD, but do not provide them to algorithms.
Setup.
We partition the races into White (81.66%) and non-White (18.34%).
For each playeri , we query Genderize and EthniColr 4 with i ’s full-name to obtain the “probabilities” p f ( i ) andp nw ( i ) that player i is labeled as a women and non-white respectively.
We assume that theseprobabilities are correct and that the gender and race of players are drawn independently.
Hence,e.g., we set the probability that i is a non-white women as (cid:98) P i,nw + f = p nw ( i ) p
f
( i ) .
Similarly, we set(cid:98)
P i,w + f =(1 − p nw ( i ))
p f
( i ) , (cid:98) P
i,nw + m =
p
nw
( i )(1 − p f
( i )) , and (cid:98) P
i,w + m = (1 − p nw ( i ))
(1 − p f
( i )) .
Notably, we do not calibrate (cid:98) P on this data.
We verify that, like the previous simulation, (cid:98) P ismiscalibrated in this simulation.
E.g., only 31% of the samples i for which (cid:98) P i,nw + m > 0 .
75 arelabeled as ‘Non-white man’ (instead of 75%).
Hence, the assumption that P is accurately known isviolated in this simulation.
We expect calibration to improve NResilient ’s performance.
Results.
See Figure 3 for RD averaged over 500 iterations.
We observe that all algorithms ( NResilient ,
CSV , GAK , SJ , and MC ) have better RD than Uncons .
Among these, NResilient achieves the best
RD ( ≈ 0.67), next CSV , GAK , and SJ obtain RD ( ≈ 0.61), and MC achieves RD ( ≤ 0 . 53 ).
Morespecifically, for all ϕ ≤ 1 . 75 , NResilient has a strictly better RD than all baselines (this range of ϕsubsumes, e.g., the range ϕ ≤ 1 . 05 –required guarantee satisfaction of the 80% rule with four groups.)
Further, in Figure 6, we observe that all algorithms have a similar fairness-utility trade-off.
Proof of Theorem 3.1
In this section we prove Theorem 3.1.
Some of the details are deferred to Supplementary Material D.2due to space constraints.
The proof is divided into two propositions:
Proposition 5.1.
For any δ ∈ (0 , 1] , any ranking feasible for Prog.
(7) satisfies ( cγ, δ ) -constraint.
Proposition 5.2.
For any δ ∈ (0 , 12 ) and c > 1 , any ranking satisfying the (( c − √ c ) γ, δ ) -constraintis feasible for Program (7) .
Proof of Theorem 3.1.
Let R ⋆ be the optimal solution of Program (7).
Since R ⋆ is feasible bydefinition, Proposition 5.1 implies that R ⋆ satisfies the ( cγ, δ ) -constraint.
Pick any R ′
that satisfiesthe (( c − √ c ) γ, δ ) -constraint.
Proposition 5.2 implies that R ′ is feasible for Program (7).
Since R ⋆is an optimal solution of Program (7), R ⋆ ’s utility is at least as large as the utility of R ′ .
Notation .
For each item i and group ℓ , let Z iℓ ∈ { 0 , 1 } be the indicator random variable
Z i :=
I [ G ℓ ∋ i ] .
By Definition 2.2, Pr[ Z iℓ ] = P iℓ and Z iℓ and Z jℓ are independent for any i ̸
= j .
Givenranking R ∈ R , group ℓ ∈ [ p ] , and position k ∈ [ n ] , let Z # ( R, ℓ, k ) be the number of items from G ℓin the top k positions of R and let P # ( R, ℓ, k ) =
E [ Z # ( R, ℓ, k )] .
From the above, we get:
[Equation]
We will use the following concentration result in the proof.
It is proved in Supplementary Material D.1.
Lemma 5.3.
For any position k ∈ [ n ] , group ℓ ∈ [ p ] , parameters ε ≥ 0 and L, U ∈ R , and ranking
R ∈ R , where R is possibly a random variable independent of { Z iℓ } i,ℓ , if P # ( R, ℓ, k ) ≤ U or
P # ( R, ℓ, k ) ≥ L then the following equations hold respectively Pr [ Z # ( R, ℓ, k ) <
(1 + ε ) U ]
≥1 − e − Uε 2 2+ ε and Pr [ Z # ( R, ℓ, k ) > (1 −
ε ) L ] ≥ 1 − e −
Lε 2 2(1 − ε ) .
Proof of Proposition 5.1.
Fix any k and ℓ .
Let
[Equation]
and ζ [Equation] Here, U ′ and ζ satisfy U ′ (1 + ζ ) =
U kℓ (1 + cγ k ) .
Fix any ranking R that is feasible for Program (7).
Since R is feasible, it satisfies that
[Equation]
Using U ′ (1 + ζ )
=
U kℓ (1 + cγ k ) , Equation (10), and Lemma 5.3, we get that
[Equation]
Fact 5.4 .
For all x, y ≥ 0 , if x ≥ y + √ y , then x 2 1+ x ≥ y .
Using Fact 5.4 and Equation (6), we can show that for each k , c 2 γ 2 k 1+ cγ k ≥ 2 (1 − ϕ ) 2 U kℓ · log 2 npδ .
(Thisuses δ < 12 and U kℓ , n ≥ 1 .)
Substituting this in Equation (11) we get:
[Equation]
12) 400 Taking the union bound over all positions k and ℓ , we get (as desired) that with probability at least1 − δ , for all k ∈ [ n ] and ℓ ∈ ,
Z #
( R, ℓ, k ) ≤ U ℓk (1 + cγ k ) .
The proof of Proposition 5.2 appears in Supplementary Material N.
Limitations and conclusion
Recent studies find that errors in socially-salient attributes can adversely affect fairness and utilityof existing fair-ranking algorithms [29].
We consider a model of random and independent errors insocially-salient attributes and present a framework that can output rankings with high fairness andutility in this model.
This framework works for a general class of fairness criteria, which involve multiple overlapping groups and upper bounds on the number of items that appear in the first k positionsfrom each group.
We also show near-tightness of the framework’s fairness guarantee.
Empirically, onboth synthetic and real-world datasets, we observe that, compared to baselines, our framework canachieve higher fairness-values and a similar or better fairness-utility trade-off for standard metrics.
Compared to existing fair-ranking frameworks, our framework does not need accurate socially-salientattributes, but assumes that errors in attributes are random and independent.
When these assumptionsdo not hold, our framework may not satisfy its guarantees and a careful assessment of this onapplication-specific data would be important to avoid any (unintended) negative social impact.
Our work only addresses one aspect of how bias may show up in rankings, and more generally, on theweb.
For instance, while we consider a large class of fairness constraints, it does not capture someimportant notions such as the qualitative representation of different groups [38, 55].
It is important totake an holistic approach to mitigate bias and incorporate our work as a part of such a broader effort.
Finally, our work adds to the line of works that develop fair decision-making algorithms robust toinaccuracies in data [42, 6, 54, 25, 67, 66, 48, 14].
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes] See the theorems in Section 3 and Figures 1 to
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Section 6discusses the importance of assessing the performance of our algorithm on applicationspecific data and using it as a part of a larger framework for mitigating discrimination.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See, e.g.,Theorems 3.1 to 3.3 and E.
(b) Did you include complete proofs of all theoretical results?
[Yes] See SupplementaryMaterials D to F
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Anonymizedcode for our simulations is available at https://github.com/NoisyRanking/FairRankingWithNoisyAttributes
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Supplementary Material B
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Please see, e.g., Figures 1 to
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Supplementary Material B
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section
(b) Did you mention the license of the assets?
[N/A] We use existing code by [48] anddata by [15, 29].
To the best of our knowledge these assets are not licensed.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]