Fair Ranking with Noisy Protected Attributes
Anonymous Author(s)
AffiliationAddress email
Abstract
The fair-ranking problem, which asks to rank a given set of items to maximizeutility subject to group fairness constraints, has received attention in the fairness,information retrieval, and machine learning literature.
Recent works, however,observe that errors in socially-salient (including protected) attributes of items cansignificantly undermine fairness guarantees of existing fair-ranking algorithmsand raise the problem of mitigating the effect of such errors.
We study the fairranking problem under a model where socially-salient attributes of items arerandomly and independently perturbed.
We present a fair-ranking framework thatincorporates group fairness requirements along with probabilistic information aboutperturbations in socially-salient attributes.
We provide provable guarantees on thefairness and utility attainable by our framework and show that it is informationtheoretically impossible to significantly beat these guarantees.
Our frameworkworks for multiple non-disjoint attributes and a general class of fairness constraintsthat includes proportional and equal representation.
Empirically, we observe that,compared to baselines, our algorithm outputs rankings with higher fairness, andhas a similar or better fairness-utility trade-off compared to baselines.
Introduction
Given a query and a set of m items, ranking problems require one to output an ordering of a smallsubset of items in decreasing order of relevance to the query.
Such ranking problems have beenextensively studied in the information retrieval [40] and the machine learning [39] literature andalgorithms for them are used in applications such as search engines, personalized feed generators andonline recruiting platforms [38, 11, 7]
Several studies have observed that when the outputs of rankingalgorithms are consumed by end-users, e g., image results for occupation-related queries, articleswith different political leanings and job applicants in online recruiting, the outputs can mislead oralter their perceptions about socially-salient groups [34], polarize their opinions [21, 43] and affecteconomic opportunities available to individuals [28].
A reason is that relevance (or utilities) inputto ranking algorithms may be influenced by human or societal biases, leading to output rankingsthat skew representations of socially-salient, and often legally-protected, groups such as women and Black people [48].
A growing number of works aim to make the output of ranking algorithms fair with respect to sociallysalient attributes [66, 51].
As for notions of fairness, in the case when each item belongs to one oftwo socially-salient groups ( G 1 or G 2 ), equal representation requires that, for every k , (roughly) k 2items from each of G 1 and G 2 appear in the first k positions of the output ranking.
Proportionalrepresentation requires that at most k · | G ℓ | m items from each G ℓ appear in the first k positions.
Fairnesscriteria that generalize proportional representation and involve p ≥ 2 groups G 1 , . .
, G p , where eachitem may belong to multiple groups, have also been considered: Given values U kℓ , they require thatat most U kℓ items from G ℓ appear in the first k positions of the output ranking [54, 15].
One setof works in the fair-ranking literature tries to improve fairness in utility-estimation [64, 55, 65, 44].
Such approaches have the benefit that no changes to the existing ranking algorithm are necessary but they may be unable to guarantee that the output ranking satisfies the required fairness criteria
Another set of works use the given utilities as-it-is and change the ranking algorithm to outputthe ranking with the highest utility subject to satisfying the specified fairness criteria by includingthem as fairness constraints [54, 8, 15, 24, 27].
While these latter approaches can guarantee fairness,they require coming up with new algorithms to solve the arising constrained ranking problems.
Bothapproaches, however, rely on knowledge of the socially-salient attributes of the items [49].
Assuming precise access to socially-salient attributes is reasonable in some contexts and has led tosuccessful deployment of fair-ranking frameworks; see [24].
However, in several contexts, sociallysalient attributes can be erroneous, missing, or known only probabilistically.
For instance, errors canarise due to misreporting, which is a common concern with self-reported attributes [3].
Attributes canalso be missing, as is the case with images in web-search or in settings where it is illegal to collectcertain socially-salient attributes [17].
Often attributes are predicted using ML-classifiers, but suchprediction has inaccuracies [9].
In such cases, one can calibrate the confidence scores of classifiers toderive (aggregate) probabilistic information about the true attributes [31].
Moreover, probabilisticinformation about socially-salient (protected) attributes can be sometimes computed from otherattributes.
For instance, name and location of an individual, combined with aggregate census datamay be used to get a conditional distribution of their race [20, 32, 17].
Even accurate attributesmay be randomly and independently flipped to preserve user privacy, and the distribution of flippedattributes is determined by public parameters of, e.g., the randomized response mechanism [33, 61].
Several models of inaccuracies in data have been proposed [41, 23].
We consider one such model(due to [4]) to capture inaccuracies in socially-salient attributes.
Each item i belongs to the ℓ -th groupwith a known probability P iℓ .
For each item i , the distribution corresponding to P iℓ s over groups isassumed to be independent of corresponding distributions of other items.
This model can be usedin cases where these probabilities are available or can be derived, as in some of the aforementionedexamples (see Section 4 and Supplementary Material A).
In other cases, e.g., when errors are strategicor adversarial, other models are needed.
This model and its variants have also been used by workson designing fair algorithms in the presence of inaccuracies, for problems including classification[36, 59, 58, 13], subset selection [42], and clustering [22].
In this noise model, while socially-salient attributes are not explicitly specified, one could still useexisting fair-ranking algorithms by first sampling groups for items from the given probabilities.
Indeed,[26] evaluate existing fair-ranking algorithms on attributes obtained from the probabilities derivedfrom ML classifiers.
They find that “errors in [socially-salient attributes] can dramatically underminefair-ranking algorithms” and can cause “[non-disadvantaged groups] to become disadvantagedafter a ‘fair’ re-ranking.”
We confirm this observation on a synthetic dataset when the goal is tofinding a ranking that satisfies equal representation (Section 4).
We assigned each item the sociallysalient group that is most likely and find that when existing fair-ranking algorithms (for equalrepresentation) are run with this group information, they output rankings that significantly violate theequal representation criteria (Figure 1).
Further, we mathematically analyze two natural methodsto sample groups from probabilities and give examples where taking such information as input,existing fair-ranking algorithms output rankings which provably violate the equal representationcriteria (Supplementary Material C).
Thus, new ideas are needed to design fair-ranking frameworksthat can guarantee given fairness criteria under this noise model.
Our contributions.
We present a fair-ranking framework that guarantees given fairness criteria whenthe socially-salient attributes are assumed to follow the probabilistic noise model mentioned above.
In particular, it finds a utility maximizing ranking subject to a class of constraints that only rely ongiven probability distributions (Program (7)).
These constraints relax the given fairness criteria by acarefully chosen factor: for equal representation, the relaxation is by roughly a 1 + 1 √ k multiplicativefactor for position k for any k .
Moreover, instead of sampling the attribute values and applyingconstraints on them, these constraints apply the relaxed-fairness criteria to the expected number ofitems from each group that appear in the first k positions.
We show that these constraints ensurethat any ranking approximately satisfying the given fairness criteria is feasible for them and anyranking feasible for them approximately satisfies the given fairness criteria (Theorem 3.1).
Ourfair-ranking framework works for the general class of fairness criteria introduced earlier, whichinvolve multiple overlapping groups G 1 , . .
, G p and upper bound U kℓ for the ℓ -th group and k -thposition (Theorem 3.1), and for their position-weighted versions (Theorem E.1).
We show that our fair-ranking framework, besides nearly satisfying the given fairness criteria, has aprovably high utility (Theorem 3.1).
Complementing Theorem 3.1, we prove near-tightness of the fairness guarantee (Theorem 3.2): For equal representation fairness criteria, this results shows thatthat it is information theoretically impossible to output a ranking that violates this criteria by less thana multiplicative factor of 1+ (cid:101)
O ( 1 √ k ) at the k -th position for any k .
Finally, we give a polynomial-timealgorithm to approximately solve Program (7) (Theorem 3.3).
Empirically, we evaluate our framework on both synthetic and real-world data against standardmetrics like weighted-risk difference (RD) that measure deviations from specific fairness criteria(Section 4).
We compare its performance to key baselines [15, 54, 24, 42] on both single and multipleattributes.
In all simulations, we observe that compared to baselines our framework has a highermaximum fairness (2 to 10% for RD; Figures 1 to 3) and a similar or better fairness-utility trade-off(Figures 2, 4 and 6 to 9).
Related work.
Work on automated information retrieval dates back to 1940s [37, 18].
Since thenthe IR literature has devoted a significant effort in measuring relevance of items to specific queriesacross different tasks: including, web search [6], personalization [30], and product rating [19]; wealso refer the reader to [40] and the references therein.
In the last three decades, works in the MLliterature have also made significant contributions to relevance-estimation [39], by proposing methodsthat: (1) supplement traditional IR approaches, e.g., by automatically tuning their–previously hard totune–parameters [57] and by improving their efficiency through clustering-based techniques [56, 2],and(2)substitute traditional IR approaches by neural-network based models to predict item relevance[11, 10, 60, 7].
Fair ranking.
Existing works on the fair-ranking problem take diverse approaches: Amongworks that de-bias utilities, different approaches include, post-processing the utilities so that thepost-processed utilities satisfy some fairness requirement [63], introducing a “fairness penalty”in the objective function used to train learning-to-rank models [55, 65], and modifying featurerepresentations generated by up-stream algorithms so that the utilities learned from the modifiedrepresentations satisfy some fairness requirements [64].
Works that alter the ranking algorithms canalso be further categorized into those which satisfy the constraints for each ranking [15, 62, 24, 27]and those that satisfy the constraints in aggregate over multiple rankings [54, 8].
Unlike this work,all aforementioned works need access to the socially-salient attributes of items.
When protectedattributes are inaccurate, these works can fail to satisfy their fairness and/or utility guarantees [26].
Effect of inaccuracies on fair-ranking algorithms.
Some recent works have considered assessingfairness of rankings and ranking algorithms with missing or inaccurate protected attributes.
[35]analyze the setting where all protected attributes are missing, but can be purchased at a fixed costper item.
They give statistical-techniques to estimate the fairness-value of a given ranking at a smallcost.
[26] use ML-classifiers to infer protected attributes from real-world data and study performanceof the fair-ranking algorithm by [25] when given inferred attributes as input.
While these worksunderscore the need for fair-ranking algorithms to be robust to inaccuracies in protected attributes,they only assess fairness in the presence of noisy protected attributes.
Model of fair ranking with noisy attributes
Ranking problem.
In ranking problems, given m items, one has to select a subset of n items andoutput a permutation of the selected items.
This permutation is said to be a ranking .
There is alarge body of work on estimating the relevance of items and personalizing these estimates to specificusers/queries [40, 39].
We consider a ranking problem where the relevance of items are known.
Abstracting relevance estimation, in this problem, one is given an m × n matrix W , such that placingthe i -th item at the j -th position generates utility W ij .
The utility of a ranking is the sum of utilitiesgenerated by each item in its assigned position.
The algorithmic task in the ranking problem is tooutput a ranking with the highest utility.
We denote rankings by assignment matrices R ∈ { 0 , 1 } m × n ,where R ij = 1 indicates that item i appears in position j , and R ij = 0 indicates otherwise.
In thisnotation, the utility of a ranking is ⟨ R, W ⟩ := (cid:80) mi =1 (cid:80) nj =1 R ij W ij .
Then this ranking problem is tosolve: max R ∈R ⟨ R, W ⟩ .
Where R is the set of all assignment matrices denoting a ranking: R :=
(cid:110) X ∈ { 0 , 1 } m × n : ∀ i ∈ [ m ] , (cid:80) nj
[Equation]
X ij [Equation] [Equation] ] , (cid:80)
[Equation] ij [Equation]
(cid:111) . [Equation] Here, the constraint (cid:80) mi =1 X ij = 1 ensures position j has exactly one item and the constraint 146 (cid:80) nj =1 X ij ≤ 1 ensures that item i occupies at most one position.
Fair-ranking problem.
There are several versions of the fair-ranking problem.
We consider a versionwith p ≥ 2 socially-salient groups G 1 , G 2 , . . .
, G p ⊆
[ m ] (e.g., the group of all women or all Blackpeople) which are often protected by law.
Each of the m items belongs to one or more of these socially- salient groups (henceforth referred to as just groups).
This fair-ranking problem is to output theranking with maximum utility subject to satisfying certain fairness criteria with respect to these groups.
The appropriate notion of fairness is context dependent, and to capture different fairness criteria numerous fairness constraints have been proposed.
We consider a class of general fairness constraints.155 Definition 2.1 ( Fairness constraints ) .
Given a matrix U ∈
Z n × p + , a ranking R satisfies the upperbound constraint if (cid:80) i ∈ G ℓ (cid:80) kj =1 R ij ≤ U kℓ , for all ℓ ∈ [ p ] and k ∈ [ n ] .
Existing works consider similar constraints and show that they can encapsulate a variety of fairnesscriteria [54].
For instance, when groups are disjoint, to capture equal and proportional representation,one can choose U kℓ := (cid:108) k · 1 p (cid:109) and U kℓ := (cid:108) k · | G ℓ | m (cid:109) for all k and ℓ respectively.
As a running example,we consider the fair-ranking problem with equal representation with two disjoint groups, i.e.,
[Equation]
[Equation]
To ease readability, we omit ceilings-operators henceforth.
Noise model.
If the socially-salient attributes of items are known accurately, then one can solve thefair-ranking problem.
However, as discussed, in many contexts, attributes are inaccurate, missing,or only probabilistically known.
Several models have been proposed to capture different errors inattributes.
Here, we consider a model (due to [4]) which has also appeared in [22, 36, 42].
Definition 2.2 ( Noise model ) .
Let P ∈ [0 , 1] m × p be a known matrix.
The groups G 1 , . . .
, G p ⊆
[ m ]are random variables, such that, for each i ∈ [ m ] and ℓ ∈ [ p ] , Pr[ G ℓ ∋ i ]
= P iℓ .
Moreover, fordifferent items i ̸
=
j the events G ℓ ∋ i and G k ∋ j are independent for all ℓ, k ∈ [ p ] .
Definition 2.2 makes two key assumptions: the matrix P is known and for each item i , the events
G ℓ ∋ i over groups ℓ are independent of the corresponding events for other items.
Both of theseassumptions hold when attributes are flipped to preserve local differential privacy (Remark A.1).
Inother settings, P ’s estimate can be inaccurate and above events may be correlated.
These can adverselyaffect the performance of our framework.
We empirically study this in simulations where P isestimated using confidence scores of off-the-shelf classifiers and is miscalibrated (Figures 2 and 3).
Fairness constraint with noisy attributes.
Most existing fairness constraints assume that the groupsare deterministic.
Hence, it is not clear how to impose them when groups are random variables,as in Definition 2.2.
One definition is to require the constraints to be approximately satisfied withhigh probability.
Consider the instantiation of this definition for equal representation:
A ranking Rsatisfies ( ρ, δ ) -equal representation, if with probability 1 − δ , at most k 2 (1 + ρ ) items from G ℓ appearin the first k positions in R places for all k ∈ [ n ] and ℓ ∈ [2] .
Naturally, one would like to satisfy thisdefinition for small δ, ρ .
However, it turns out to be too stringent and is infeasible for any small δ, ρ .
Proposition 2.3.
No ranking satisfies ( ρ, δ ) -equal representation for ρ < 1 , δ ≤ 12 , and P = (cid:2) 12 (cid:3) m × p .
The proof of Proposition 2.3 shows that any ranking R violates the equal-representation constraintat the 2nd position by a multiplicative factor of 2 with probability 12 .
The issue is that the samerelaxation parameter ρ is used for each position.
Motivated by this observation, we consider thefollowing alternate version of upper bound constraints. Definition 2.4 ( ( ε, δ ) -constraint ) .
For any ε ∈ R n ≥ 0 and δ ∈ (0 , 1] , a ranking R is said to satisfy( ε, δ ) -constraint
if with probability at least 1 − δ over the draw of G 1 , . . .
, G p
[Equation]
We would like to output a ranking that satisfies Definition 2.4 for small δ and small ε 1 , ε 2 , . . .
, ε n .
Problem 2.5 ( Ranking problem with noisy attributes ) .
Given matrices W ∈ R m × n ≥ 0 , U ∈ R n × p ≥ 0 , and P ∈ [0 , 1] m × p , find the ranking R maximizing utility ⟨ R, W ⟩ subject to satisfying ( ε, δ ) -constraintfor some small ε and δ .
2.1 Challenges in solving Problem 2.5
In this section we discuss potential approaches for solving Problem 2.5.
In other words, solving:
[Equation]
Even for two disjoint groups, given V ≥ 0 , it is NP -hard to decide if the value of Program (4)is at least V (Theorem F.5).
To bypass this hardness, one can consider approximation algorithms.
Program (4) is an integer program because the entries of the matrix R are required to be integers(Equation (1)).
A standard approach to (approximately) solve integer programs is to: (1) considertheir continuous relaxation that drops the integrality constraints, (2) compute the optimal solution R c of the relaxed problem, and then (3) “round” R c to satisfy integrality constraints whileretaining its utility and fairness “properties.”
To take this approach, we first need an efficient algo- rithm to find R c .
However, not just Program (4), but even its continuous relaxation is non-convexand, hence, it is unclear how to solve it to find R c .
Due to the independence assumption in Definition 2.2, the number of items from G ℓ appearing in thefirst k positions of a ranking is concentrated around its expectation (for large k ).
This implies that if,in expectation, less that U kℓ items from G ℓ appear in the top k positions then, with high probability,the number of items from G ℓ in the top k positions is not much larger than U kℓ .
Using this one canshow that a ranking satisfying the following constraints
[Equation]
also satisfies ( ε, δ ) -constraint for small ε and δ .
One idea is to find the ranking maximizing utility subject to satisfying Constraint (5).
A feature of Constraint (5) is that it is linear in R as 211 E [
[Equation]
(cid:80)
[Equation]
(cid:80) kj =1 P iℓ R ij and, hence, one may hope to find the ranking with the maximum utility subject to satisfying Constraint (5).
However, the issue is that there are examples whereany ranking satisfying Constraint (5) has 0 utility and there are rankings that satisfy ( ε, δ ) -constraintand have a large positive utility (Lemma F.3).
Hence, this approach can output rankings whose utilityis significantly smaller than the utility of the solution to Problem 2.5.
To overcome this, we relax
Constraint (5) by a carefully chosen position-dependent factor, such that, any ranking satisfying the( ε, δ ) -constraint (for appropriate ε and δ ) is also feasible for our framework.
Theoretical results
In this section we present our optimization framework and its fairness and utility guarantees.
[Equation]
[Table]
[Table]
[Table]
[Table]
parameter
[Equation]
The above program is a modification of the program for fair ranking with accurate groups: It hasthe same objective but different constraints.
Instead of sampling the attribute values and applyingconstraints on the sampled values, Constraint (5) apply upper bounds on the expected number of itemsin the first k positions from group ℓ (see Section 2.1).
Further, Constraint (5) relaxes upper bounds U kℓby a small position-dependent factor.
Like for Constraint (5), one can show that any ranking satisfying
Constraint (8) also satisfies ( ε, δ ) -constraint (for small ε 1 , . . . , ε n and δ ).
But unlike Constraint (5),and somewhat surprisingly, any rankingthatsatisfies ( ε, δ ) -constraint(for appropriate ε 1 , . .
. , ε n andδ ) must also satisfy Constraint (8).
We use this to prove Theorem 3.1’s utility guarantee.
Our first result bounds the fairness and utility of the optimal solution of Program (7).
Theorem 3.1.
Let γ ∈ R n be as defined in Equation (6) .
There is an optimization program(Program (7) ), parameterized by a constant c and failure probability δ , such that for any c > 1 andδ ∈ (0 , 12 ] its optimal solution satisfies ( cγ, δ ) -constraint and has a utility at least as large as theutility of any ranking satisfying (( c − √ c ) γ, δ ) -constraint.
For equal representation, γ k is (cid:101) O (cid:16) 1 √ k (cid:17) .
Thus, Theorem 3.1 guarantees that, with high probability, theoptimal solution of Program (7) multiplicatively violates equal representation at the k -th position byat most 1+ (cid:101) O (cid:16) 1 √ k (cid:17) .
Further, this solution’s utility is higher than the utility of any ranking satisfying aslight relaxation of this fairness guarantee.
Theorem 3.1 can be extended to position-weighted versionsof fairness constraints (Theorem E.1), where the fairness constraint is (cid:80) i ∈ G ℓ (cid:80) j ∈ [ k ] v j R ij ≤ U kℓ (forall k and ℓ ) for specified discount factors v 1 ≥ · · · ≥ v n such as NDGC [29].
If we are also guaranteed
U kℓ ≥ ψk for some constant ψ > 0 and all k and ℓ , then we can improve γ k ’s dependence on δ fromlog 1 δ to (cid:113) log 1 δ (Supplementary Material D.2).
The proof of Theorem 3.1 appears in Section 5.
Since ( c − √ c ) γ < cγ , Theorem 3.1 gives a pseudo-optimality guarantee on utility. Does a differentconstraint C guarantee optimal utility for the achieved fairness? Let R C be a ranking maximizingutility subject to satisfying C . Are there small ε and δ , such that R C satisfies ( ε, δ ) -constraint and hasutility at least as large as any other ranking satisfying the ( ε, δ ) -constraint?
We prove that, for anyvalue of ε and δ , the ( ε, δ ) -constraint is the unique constraint with this property (Proposition F.1). However, solving the program corresponding to ( ε, δ ) -constraint (Program (4)) seems intractable (see Section 2.1). Unless Program (4) can be efficiently solve, a pseudo-optimality guarantee is necessary.
[Figure]
( L e ss f a i r) (cid:9)(cid:9)(cid:9) <
f un c ti on c o m pu t
e _ w e
i
gh t e d_ r i s k
d i ff a t 0x f e 96678b a
[Figure]
f
[Figure]
W e i gh t e d r
i s kd i ff e r e n ce ......
—————————— ϕ —————————— (more fair) (less fair)
Lower bound on fairness guarantee.
Our next result complements Theorem 3.1’s fairness guarantee.
Theorem 3.2.
There is a family of matrices U ∈
Z n × p + such that for any U in the familyand any parameters δ ∈ [0 , 1) and ε 1 , . . .
, ε n ≥ 0 , if for any position k ∈ [ n ] ε k ≤ 1 andε k < max ℓ ∈ [ p ] (cid:113) 1 2 U kℓ log 14 δ then there exists a matrix P ∈ [0 , 1] m × p , such that it is informationtheoretically impossible to output a ranking that satisfies ( ε, δ ) -constraint.
This family contains thematrix U corresponding to equal representation constraints.
Since γ k is O (cid:16) log( npδ ) · max ℓ (cid:113) 1 U kℓ (cid:17) , Theorem 3.2 shows that Theorem 3.1’s fairness guarantee isoptimal up to log-factors.
Supplementary Material D.3 proves Theorem 3.2.
An efficient algorithm.
As for solving our optimization program, it is NP -hard to check its feasibility(Theorem D.10).
However, because Constraint (8) is linear in R , the continuous relaxation of
Program (7) is a standard linear program and can be solved efficiently.
Our algorithm (Algorithm 1)solves the standard linear programming relaxation of Program (7) to find a solution R c and then usesa dependent-rounding algorithm by [16] to convert R c to a ranking.
Theorem 3.3.
There is a randomized algorithm (Algorithm 1) that given constants d > 2 , afailure probability 0 < δ ≤ 1 , and matrices P ∈ [0 , 1] m × p and W ∈ [0 , 1] m × n , outputs aranking satisfying ( O ( dγ ) , δ ) -constraint and with probability at least 1 − δ , and has a utility at least 265 (cid:0) 1 − 1 d (cid:1) · V − (cid:101) O ( √ dn ) , where V is the utility of any ranking satisfying (( d − √ d ) γ, δ ) -constraint.
The algorithm runs in polynomial time in d and the bit complexity of the input.
The tension in setting d is that decreasing d improves the fairness guarantee and the second term inthe utility guarantee but worsens the first term in the utility guarantee.
Under the mild assumption that
V = Ω( n ) , increasing d improves the utility guarantee because the first term in the utility guaranteedominates the second term.
In this case, the utility guarantee improves to (1 − 1 d − o (1))
· V .
Finally,while Theorem 3.3 requires the utilities (entries of W ) to be between 0 and 1, it can be extended toany non-negative and bounded utilities by appropriate scaling.
The proof of Theorem 3.3 appears in Supplementary Material D.4.
Empirical results
In this section 1 we evaluate our framework’s performance synthetic and real-world data.
Baselines and metrics.
The correct choice of fairness metric is context-dependent and beyond thescope of this work [53].
To illustrate our results, we arbitrarily fix the fairness metric as weightedrisk-difference (RD).
This is a position-weighted version of the standard risk-difference metric [12]and measures the extent to which a ranking violates equal representation.
The RD of a ranking R is:
[Equation]
Where G denotes the ground-truth protected groups and Z is a constant so that RD has range [0 , 1] .
Here, RD
= 1 is most fair and RD
= 0 is least fair.
We compare our framework, NResilient , againststate-of-the-art fair-ranking algorithms: CSV (“greedy” in [15]), SJ [54], and GAK (“DetGreedy” in[24]).
We also compare against MC , which ranks the items, in the subset output by [42]’s algorithm,to maximize utility.
Finally, we compare against the baseline, Uncons , which outputs the utilitymaximizing ranking without fairness considerations.
We present additional discussion of results,additional plots for RD, and comparisons with weighted selection-lift in Supplementary Material B.
[Figure]
[Figure] [Figure]
Setup.
We consider the DCG model of utilities [29] and a relaxation of equal representation constraints: (1) Given an intrinsic value w i ≥ 0 , for each item i , we set W ij :=
w i (log ( j + 1)) − 1 ∀ j ∈[ n ] .
(2) Given a parameter ϕ ∈ [1 , p ] , we set upper bounds U kℓ := ϕp · k for each k ∈ [ n ] and ℓ ∈ [ p ] .
In simulations, we set m = 500 , n = 25 , and vary ϕ from p to 1.
[-11 for p = 2and ϕ ≤ 1 . 
 for p = 4 .-] For each ϕ , we draw m itemsuniformly without replacement and compute an estimate (cid:98) P of the matrix P from Definition 2.2; detailsare given with each simulation.
We infer socially-salient groups (cid:98) G 1 , . . .
, (cid:98) G 2 via (cid:98) P by assigning eachitem to its most-likely group.
Finally, we run all algorithms using (cid:98) P or (cid:98) G 1 , . . .
, (cid:98) G 2 as discussed next.
Implementation details.
NResilient and MC take probabilistic information about socially-salientattributes as input and are given (cid:98) P .
CSV , SJ , and GAK require access to socially-salient groups andare given (cid:98) G 1 , . . .
, (cid:98) G p .
NResilient , SJ , and CSV use fairness constraints from Definition 2.1 and aregiven: for each k ∈ [ n ] and ℓ ∈ [ p ] , U kℓ = ϕp · k . MC requires, for each ℓ ∈ [ p ] , an upper bound onthe number of items from G ℓ that can appear in top- n positions.
It is given ϕp · n for each ℓ ∈ [ p ] .
GAK requires the desired proportion α ℓ for each group G ℓ and, roughly, satisfies the constraint U kℓ = α ℓ · k for each k ∈ [ n ] and ℓ ∈ [ p ] .
It is given α ℓ = 1 p for each ℓ ∈ [ p ] , this corresponds toϕ = 1 (hence, the figures only plot the GAK at ϕ = 1 ).
As a heuristic, we set γ k = 120 · max ℓ ∈ [ p ] (cid:113) [Table] in all simulations.
We find that this parameter suffices and expect a more refined approach to improvethe performance of NResilient .
Simulation on synthetic data.
We show that on synthetic data, where error-rates of given sociallysalient attributes vary over groups, existing fair-ranking algorithms have worse RD than Uncons .
Data.
We generate w and P for two groups using code by [42] and fix (cid:98) P = P .
For all items i , w i isi.i.d.
from the uniform distribution over [0 , 1] .
(cid:98) P is constructed such that attributes inferred from (cid:98) Phave a higher false-discovery rate for the minority group compared to the majority ( 40% vs 10% ).
Results.
See Figure 1 for the observed RD averaged over 500 iterations.
We observe that NResilientachieves best RD ( ≈ 0.81), while not loosing significant utility ( ≥ 0 .
98% of maximum; see Figure 4).
MC achieves the best RD ( ≈ 0.79).
In contrast, CSV , SJ , and GAK , which do not account for noisein the socially-salient attributes, achieve a worse RD at ϕ ≈ 1 ( ≤ 0.68) than Uncons ( ≈ 0.75).
Thus,we observe that existing fair-ranking algorithms may achieve a worse RD than Uncons .
Simulation on real-world image data.
In this simulation, given non-gender labeled images-searchresults and their utilities, our goal is to generate a high-utility and gender-balanced ranking.
Data.
We use the Occupations dataset [14] which contains the top 100 Google Image results for 96occupation-related queries.
For each image, the data has its position in search results, gender (codedas male/female) of the individual depicted in the image, collected via MTurk.
We use the (true)gender labels in the data to compute RD and to estimate (cid:98) P , but do not provide them to algorithms.
Setup.
For each image i , with rank r
i , we define w i :=
(log (1 + r i ))
− 1 .
We say an occupation isgender-stereotypical if more than 80% of images for this occupation have the same gender label(41/96 occupations).
An image is said to be stereotypical if its in a gender-stereotypical occupationand its gender label is the majority label for its occupation.
We define the socially-salient groups asthe sets of stereotypical and non-stereotypical images in gender-stereotypical occupations.
Estimating (cid:98) P .
After pre-processing, we use a CNN-based gender-classifier f [52] to predict the(apparent) gender of the person depicted in each image.
We calibrate the confidence scores outputby f by binning and use these to estimate (cid:98) P (see Supplementary Material B for more details).
We perform this calibration once and on all occupations and, then, use it for gender-stereotypicaloccupations.
Because of this (cid:98) P is miscalibrated (and hence, inaccurate).
For instance, among samplesi for which 0 .
25 ≤ (cid:98) P i 2 ≤ 0 .
5 , more than 75% are labeled as ‘man’ (instead of some percentagebetween 25% and 50%).
This violates the assumption that P is accurately known.
[Figure] ( L e [Figure] f a i r) (cid:9)(cid:9)(cid:9) <
f un c ti on c o m pu t
e _ w e
i
gh t e d_ r i s k
d i ff a t 0x f 13 a b f c 7 ae
[Figure]
[Figure]
o r e f a i r)
[Figure]
W e i gh t e d r
i s kd i ff e r e n ce ......
—————————— ϕ —————————— (more fair) (less fair)
Results.
See Figure 2 for RD and utilities (NDGC) averaged over 1000 iterations.
We observe that
NResilient achieves the best RD ( ≈ 0.81) and has a better RD-utility trade-off than the other baselines.
In contrast, CSV , SJ , and GAK , achieve a worse RD ( ≤ 0.77).
MC achieves the worst RD ( ≤ 0.70)and a worst RD-utility trade-off.
We further evaluate the robustness of NResilient to varying levelsof noise on the Occupations dataset in Supplementary Material B and observe NResilient has a betteror similar RD than each baseline at all noise levels.
Simulation on real-world name data.
We consider gender and race (encoded as binary) as sociallysalient attributes.
Our goal is to ensure equal representation across the four disjoint groups formed bycombinations of these: non-White non-men, White non-men, non-White men, and White men.
Data.
We consider the chess ranking data [26] which has of 3,251 chess players.
For each player,among other attributes, the data has their full-name, self-identified gender (coded as male/female),
FIDE rating, and race (Asian, Black, Hispanic, White) collected via MTurk.
We use the (true) genderand race labels in the data to evaluate RD, but do not provide them to algorithms.
Setup.
We partition the races into White (81.66%) and non-White (18.34%).
For each playeri , we query Genderize and EthniColr 2 with i ’s full-name to obtain the “probabilities” p f ( i ) andp nw ( i ) that player i is labeled as a women and non-white respectively.
We assume that theseprobabilities are correct and that the gender and race of players are drawn independently.
Hence,e.g., we set the probability that i is a non-white women as (cid:98) P i,nw + f = p nw ( i ) p
f
( i ) .
Similarly, we set(cid:98)
P i,w + f =(1 − p nw ( i ))
p f
( i ) , (cid:98) P
i,nw + m =
p
nw
( i )(1 − p f
( i )) , and (cid:98) P
i,w + m = (1 − p nw ( i ))
(1 − p f
( i )) .
Notably, we do not calibrate (cid:98) P on this data.
We verify that, like the previous simulation, (cid:98) P ismiscalibrated in this simulation.
E.g., only 31% of the samples i for which (cid:98) P i,nw + m > 0 .
75 arelabeled as ‘Non-white man’ (instead of 75%).
Hence, the assumption that P is accurately known isviolated in this simulation.
We expect calibration to improve NResilient ’s performance.
Results.
See Figure 3 for RD averaged over 500 iterations.
We observe that all algorithms ( NResilient ,
CSV , GAK , SJ , and MC ) have better RD than Uncons .
Among these, NResilient achieves the best
RD ( ≈ 0.67), next CSV , GAK , and SJ obtain RD ( ≈ 0.61), and MC achieves RD ( ≤ 0 . 53 ).
Further,in Figure 6, we observe that all algorithms have a similar fairness-utility trade-off.
Proof of Theorem 3.1
In this section we prove Theorem 3.1.
Some of the details are deferred to Supplementary Material D.2due to space constraints.
The proof is divided into two propositions:
Proposition 5.1.
For any δ ∈ (0 , 1] , any ranking feasible for Prog.
(7) satisfies ( cγ, δ ) -constraint.
Proposition 5.2.
For any δ ∈ (0 , 12 ) and c > 1 , any ranking satisfying the (( c − √ c ) γ, δ ) -constraintis feasible for Program (7) .
Proof of Theorem 3.1.
Let R ⋆ be the optimal solution of Program (7).
Since R ⋆ is feasible bydefinition, Proposition 5.1 implies that R ⋆ satisfies the ( cγ, δ ) -constraint.
Pick any R ′
that satisfiesthe (( c − √ c ) γ, δ ) -constraint.
Proposition 5.2 implies that R ′ is feasible for Program (7).
Since R ⋆is an optimal solution of Program (7), R ⋆ ’s utility is at least as large as the utility of R ′ .
Notation .
For each item i and group ℓ , let Z iℓ ∈ { 0 , 1 } be the indicator random variable
Z i :=
I [ G ℓ ∋ i ] .
By Definition 2.2, Pr[ Z iℓ ] = P iℓ and Z iℓ and Z jℓ are independent for any i ̸
= j .
Givenranking R ∈ R , group ℓ ∈ [ p ] , and position k ∈ [ n ] , let Z # ( R, ℓ, k ) be the number of items from G ℓin the top k positions of R and let P # ( R, ℓ, k ) =
E [ Z # ( R, ℓ, k )] .
From the above, we get:
[Equation]
We will use the following concentration result in the proof.
It is proved in Supplementary Material D.1.
Lemma 5.3.
For any position k ∈ [ n ] , group ℓ ∈ [ p ] , parameters ε ≥ 0 and L, U ∈ R , and ranking
R ∈ R , where R is possibly a random variable independent of { Z iℓ } i,ℓ , if P # ( R, ℓ, k ) ≤ U or
P # ( R, ℓ, k ) ≥ L then the following equations hold respectively Pr [ Z # ( R, ℓ, k ) <
(1 + ε ) U ]
≥1 − e − Uε 2 2+ ε and Pr [ Z # ( R, ℓ, k ) > (1 −
ε ) L ] ≥ 1 − e −
Lε 2 2(1 − ε ) .
Proof of Proposition 5.1.
Fix any k and ℓ .
Let
[Equation]
Here, U ′ and ζ satisfy that U ′ (1 + ζ ) =
U kℓ (1 + cγ k ) .
Fix any ranking R that is feasible for Program (7).
Since R is feasible, it satisfies that
[Equation]
Using that U ′ (1 + ζ )
=
U kℓ (1 + cγ k ) , Equation (10), and Lemma 5.3, we get that Pr [ Z # ( R, ℓ, k ) [Equation] U ′ (
[Equation]
ζ ) ]
[Equation]
Fact 5.4 .
For all x, y ≥ 0 , if x ≥ y + √ y , then x 2 1+ x ≥ y .
Using Fact 5.4 and Equation (6), we can show that for each k , c 2 γ 2 k 1+ cγ k ≥ [Equation] · log 2 npδ .
(Thisuses δ < 12 and U kℓ , n ≥ 1 .)
Substituting this in Equation (11) we get:
[Equation]
Taking the union bound over all positions k and ℓ , we get (as desired) that with probability at least1 − δ , for all k ∈ [ n ] and ℓ ∈ , Z # ( R, ℓ, k ) ≤ U ℓk (1 + cγ k ) . Proof of Proposition 5.2. Let ϕ :=
1 − 12 √ c . Towards a contradiction, suppose that R ′ satisfies(( c − √ c ) γ, δ ) -constraint but is not feasible for Program (7). Then there exists ℓ and k such that
P # ( R ′ , k, ℓ ) > U kℓ · (1 + ϕγ k ) . Fix any k and ℓ satisfying this. Let [Equation]
[Equation] c [Equation] [Equation]
It holds that L ′ (1 − ζ ) =
U kℓ (1 + bγ k ) and, hence, we get
Pr [ Z # ( R ′ , k, ℓ ) ≤
L ′ (1 − ζ ) ] (13) , Lem .
3 ≤
[Equation]
Since γ k ≥ 8 log npδ · max ℓ (cid:113) 1 U kℓ , δ < 12 , and U ≥ 1 , we have Pr [ Z # ( R ′ , k, ℓ ) ≤ U kℓ ] ≤ δnp < 1 − δ.
Since R ′ satisfies (( c − √ c ) γ, δ ) -constraint
we have a contradiction, hence R ′ must be feasible.
Limitations and conclusion
Recent studies find that errors in socially-salient attributes can adversely affect the fairness and utilityof existing fair-ranking algorithms [26].
We consider a model of random and independent errors insocially-salient attributes and present a framework that can output rankings with high fairness andutility in this model.
This framework works a general class of fairness criteria, which involve multipleoverlapping groups and upper bounds on the number of items that appear in the first k positions fromeach group.
We also show near-tightness of the framework’s fairness guarantee.
Empirically, on bothsynthetic and real-world datasets, we observe that, compared to baselines, our framework can achievehigher fairness-values and a similar or better fairness-utility trade-off for standard metrics.
Compared to existing fair-ranking frameworks, our framework does not need accurate socially-salientattributes, but assumes that errors in attributes are random and independent.
When these assumptionsdo not hold, our framework may not satisfy its guarantees. Simulations on real-world data suggestthat, in contexts represented by this data, our framework can achieve higher fairness than baselines(Section 4). Nevertheless, a careful assessment of this on application-specific data would be importantto avoid any (unintended) negative social impact.
Our work only addresses one aspect of how bias may show up in rankings, and more generally, on theweb.
It is important to take an holistic approach to mitigate bias and incorporate our work as a part ofsuch a broader effort.
Finally, our work adds to the line of works that develop fair decision-makingalgorithms robust to inaccuracies in data [36, 5, 47, 22, 59, 58, 42, 13].
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] See the theorems in Section 3 and Figures 1 to
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] Section discusses the importance of assessing the performance of our algorithm on application- specific data and using it as a part of a larger framework for mitigating discrimination.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See, e.g., Theorems 3.1 to 3.3 and E.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Supplementary Materials D to F
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] Anonymized code for our simulations is available at https://github.com/NoisyRanking/ FairRankingWithNoisyAttributes
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Supplementary Material B
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Please see, e.g., Figures 1 to
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Supplementary Material B
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section
(b) Did you mention the license of the assets?
[N/A] We use existing code by [42] anddata by [14, 26].
To the best of our knowledge these assets are not licensed.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]