Fairness in Ranking under Uncertainty

1 Introduction . Fairness is an important consideration in decision-making , in particular when a limited resource must be allocated among multiple agents by a principal ( or decision maker ) . A widely accepted tenet of fairness is that if an agent B does not have stronger merits for the resource than A , then B should not get more of the resource than A . Depending on the context , merit could be a qualification ( e.g. , job performance ) , a need ( e.g. , disaster relief ) , or some other measure of eligibility . The motivation for our work is that uncertainty about merits is a primary reason that a principal ’ s allocations can violate this tenet and thereby lead to unfair outcomes . Were agents ’ merits fully observable , it would be both fair and in the principal ’ s best interest to rank agents by their merit . However , actual merits are practically always unobservable . Consider the following standard algorithmic decision making environments : ( 1 ) An e-commerce or recommender platform ( the principal ) displays items ( the agents ) in response to a user query . An item ’ s merit is the utility the user would derive from it , whereas the platform can only observe numerical ratings , text reviews , the user ’ s past history , and similar features . ( 2 ) A job recommendation site or employer ( the principal ) wants to recommend/hire one or more applicants ( the agents ) . The merit of an applicant is her ( future ) performance on the job over a period of time , whereas the site or employer can only observe ( past ) grades , test scores , recommendation letters , performance in interviews , and similar assessments . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In both of these examples — and essentially all others in which algorithms are called upon to make allocation decisions between agents — uncertainty about merit is unavoidable , and arises from multiple sources : ( 1 ) the training data of a machine learning algorithm is a random sample , ( 2 ) the features themselves often come from a random process , and ( 3 ) the merit itself may only be revealed in the future after a random process ( e.g. , whether an item is sold or an employee performs well ) . Given that decisions will be made in the presence of uncertainty , it is important to define the notion of fairness under uncertainty . Extending the aforementioned tenet that “ if agent B has less merit than A , then B should not be treated better than A , ” we state the following generalization to uncertainty about merit , first for just two agents : Axiom 1 . If A has merit greater than or equal to B with probability at least ρ , then a fair policy should treat A at least as well as B with probability at least ρ . This being an axiom , we can not offer a mathematical justification . It captures an inherent sense of fairness in the absence of enough information , and it converges to the conventional tenet as uncertainty is reduced . In particular , consider the following situation : two items A , B with 10 reviews each have average star ratings of 3.9 and 3.8 , respectively ; or two job applicants A , B have GPAs of 3.9 and 3.8 . While this constitutes some ( weak ) evidence that A may have more merit than B , this evidence leaves substantial uncertainty . The posterior merit distributions based on the available information should reflect this uncertainty by having non-trivial variance ; our axiom then implies that A and B must be treated similarly to achieve fairness . In particular , it would be highly unfair to deterministically rank A ahead of B ( or vice versa ) . Our main point is that this uncertainty , rather than the specific numerical values of 3.9 and 3.8 , is the reason why a mechanism should treat A and B similarly . 1.1 Our Contributions . We study fairness in the presence of uncertainty specifically for the generalization where the principal must rank n items . Our main contribution is the fairness framework , giving definitions of fairness in ranking in the presence of uncertainty . This framework , including extensions to approximate notions of fairness , is presented and discussed in depth in § 2 . We believe that uncertainty of merit is one of the most important sources of unfairness , and modeling it explicitly and axiomatically is key to addressing it . Next , in § 3 , we present algorithms for a principal to achieve ( approximately ) fair ranking distributions . A simple algorithm the principal may use to achieve approximate fairness is to mix between an optimal ( unfair ) ranking and ( perfectly fair ) Thompson sampling . We show that this policy is not optimal for the principal ’ s utility , and we present an efficient LP-based algorithm that achieves an optimal ranking distribution for the principal , subject to an approximate fairness constraint . We next explore empirically to what extent a focus on fairness towards the agents reduces the principal ’ s utility . We do so with two extensive sets of experiments : one described in § 4 on existing data , and one described in § 5 “ in the wild. ” In the first set of experiments , we consider movie recommendations based on the standard MovieLens dataset and investigate to what extent fairness towards movies would result in lower utility for users of the system . The second experiment was carried out at the 2020 ACM SIGKDD Conference on Knowledge Discovery and Data Mining , where we implemented and fielded a paper recommendation system . Half of the conference attendees using the system received rankings that were modified to ensure greater fairness towards papers , and we report on various metrics that capture the level of engagement of conference participants based on which group they were assigned to . The upshot of our experiments and theoretical analysis is that in the settings we have studied , high levels of fairness can be achieved at a small loss in utility for the principal and the system ’ s users . Due to space constraints , essentially all proofs , as well as a detailed discussion of related work , are given in the supplementary material . Alternatively , a reader may want to read the full version ( Singh et al. , 2021 ) . 2 Ranking with Uncertain Merits . We are interested in ranking policies for a principal ( the ranking system , such as an e-commerce platform or a job portal in our earlier examples ) whose goal is to rank a set X of n agents ( such as products or applicants ) . The principal observes some evidence for the merit of the agents , and must produce a distribution over rankings trading off fairness to the agents against the principal ’ s utility . For the agents , a higher rank is always more desirable than a lower rank . 2.1 Rankings and Ranking Distributions . We use Σ ( X ) to denote the set of all n ! rankings , and Π ( X ) for the set of all distributions over Σ ( X ) . We express a ranking σ ∈ Σ ( X ) in terms of the agents assigned to given positions , i.e. , σ ( k ) is the agent in position k. A ranking distribution π ∈ Π ( X ) can be represented by the n ! probabilities π ( σ ) of the rankings σ ∈ Σ ( X ) . However , all the information relevant for our purposes can be represented more compactly using the Marginal Rank Distribution : we write p ( π ) x , k = ∑ σ : σ ( k ) =x π ( σ ) for the probability under π that agent x ∈ X is in position k in the ranking . We let P ( π ) = ( p ( π ) x , k ) x , k denote the n× n matrix of all marginal rank probabilities . The matrix P ( π ) is doubly stochastic , i.e. , the sum of each row and column is 1 . While π uniquely defines P ( π ) , the converse mapping may not be unique . However , given a doubly stochastic matrix P , the Birkhoff-von Neumann decomposition ( Birkhoff , 1946 ) can be used to compute some distribution π consistent with P , i.e. , P ( π ) = P ; any consistent distribution π will suffice for our purposes . 2.2 Merit , Uncertainty , and Fairness . The principal must determine a distribution over rankings of the agents . This distribution will be based on some evidence for the agents ’ merits . This evidence could take the form of star ratings and reviews of products ( combined with the site visitor ’ s partially known preferences ) , or GPA , test scores , and recommendation letters of an applicant . Our main departure from past work on individual fairness ( following ( Dwork et al. , 2012 ) ) is that we do not view this evidence as having inherent meaning ; rather , its sole role is to induce a posterior joint distribution over the agents ’ merits . The merit of agent x is vx ∈ R , and we write v = ( vx ) x∈X for the vector of all agents ’ merits . Based on all observed evidence , the principal can infer a distribution Γ over agents ’ merits using any suitable Bayesian inference procedure . Since the particular Bayesian model depends on the application , for our purposes , we merely assume that a posterior distribution Γ was inferred using best practices and that ideally , this model is open to verification and audit . We write Γ ( v ) for the probability of merits v under Γ . We emphasize that the distribution will typically not be independent over entries of v — for example , students ’ merit conditioned on observed grades will be correlated via common grade inflation if they took the same class . To avoid awkward tie-breaking issues , we assume that vx 6= vy for all distinct x , y ∈ X and all v in the support of Γ . This side-steps having to define the notion of top-k lists with ties , and comes at little cost in expressivity , as any tie-breaking would typically be encoded in slight perturbations to the vx anyway . We writeM ( v ) x , k for the event that under v , agent x is among the top k agents with respect to merit , i.e. , that | { x′ | vx′ > vx } | < k. We now come to our key definition of approximate fairness . Definition 2.1 ( Approximately Fair Ranking Distribution ) . A ranking distribution π is φ-fair iff k∑ k′=1 p ( π ) x , k′ ≥ φ · Pv∼Γ [ M ( v ) x , k ] ( 1 ) for all agents x and positions k. That is , the ranking distribution π ranks x at position k or above with at least a φ fraction of the probability that x is actually among the top k agents according to Γ . Furthermore , π is fair iff it is 1-fair . The reason for defining φ-approximately fair ranking distributions ( rather than just fair distributions ) is that fairness typically comes at a cost to the principal ( such as lower expected clickthrough or lower expected performance of recommended employees ) . For example , if the vx are probabilities that a user will purchase products on an e-commerce site , then deterministically ranking by decreasing EΓ [ vx ] is the principal ’ s optimal ranking under common assumptions about user behavior ; yet , being deterministic , it is highly unfair . Our definition of approximate fairness allows , e.g. , a policymaker to choose a trade-off regarding how much fairness ( with resulting expected utility loss ) to require from the principal . Notice that for φ = 0 , the principal is unconstrained . We remark that the merit values vx only matter insofar as comparison is concerned ; in other words , they are used ordinally , not cardinally . This is captured by the following proposition . Proposition 2.1 . Let f : R→ R be any strictly increasing function . Let Γ′ be the distribution that draws the vector ( f ( vx ) ) x∈X with probability Γ ( v ) for all v ; that is , it replaces each entry vx with f ( vx ) . Then , a ranking distribution π is φ-fair with respect to Γ if and only if it is φ-fair with respect to Γ′ . Proof . Because f is strictly increasing , Pv∼Γ [ M ( v ) x , k ] = Pv∼Γ′ [ M ( v ) x , k ] for all x and k. This immediately implies the claim by examining Definition 2.1 . Proposition 2.1 highlights a key aspect of our fairness definition : we avoid expressing any notion of fairness when “ one agent has just ‘ a little ’ more merit than the other , ” instead arguing that fairness is only truly violated when an agent with more merit is treated worse than one with less merit . In other words , fairness is inherently ordinal in our treatment . This viewpoint has implications for a principal seeking “ high-risk high-reward ” agents , which we discuss in more depth in § B.2 .

Summary: This paper studies the fairness issue in algorithmic decision-making caused by uncertainty. Since the actual merits are hard to observe, the authors propose a new notion to capture the uncertainty, named approximate fairness in ranking. Based on the notation, they propose different algorithms to achieve approximately fair ranking distributions. In addition to the theoretical analysis, extensive empirical studies are conducted to evaluate their approach.