Fairness in Ranking under Uncertainty

1 Introduction . Fairness is an important consideration in decision-making , in particular when a limited resource must be allocated among multiple agents by a principal ( or decision maker ) . A widely accepted tenet of fairness is that if an agent B does not have stronger merits for the resource than A , then B should not get more of the resource than A . Depending on the context , merit could be a qualification ( e.g. , job performance ) , a need ( e.g. , disaster relief ) , or some other measure of eligibility . The motivation for our work is that uncertainty about merits is a primary reason that a principal ’ s allocations can violate this tenet and thereby lead to unfair outcomes . Were agents ’ merits fully observable , it would be both fair and in the principal ’ s best interest to rank agents by their merit . However , actual merits are practically always unobservable . Consider the following standard algorithmic decision making environments : ( 1 ) An e-commerce or recommender platform ( the principal ) displays items ( the agents ) in response to a user query . An item ’ s merit is the utility the user would derive from it , whereas the platform can only observe numerical ratings , text reviews , the user ’ s past history , and similar features . ( 2 ) A job recommendation site or employer ( the principal ) wants to recommend/hire one or more applicants ( the agents ) . The merit of an applicant is her ( future ) performance on the job over a period of time , whereas the site or employer can only observe ( past ) grades , test scores , recommendation letters , performance in interviews , and similar assessments . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In both of these examples — and essentially all others in which algorithms are called upon to make allocation decisions between agents — uncertainty about merit is unavoidable , and arises from multiple sources : ( 1 ) the training data of a machine learning algorithm is a random sample , ( 2 ) the features themselves often come from a random process , and ( 3 ) the merit itself may only be revealed in the future after a random process ( e.g. , whether an item is sold or an employee performs well ) . Given that decisions will be made in the presence of uncertainty , it is important to define the notion of fairness under uncertainty . Extending the aforementioned tenet that “ if agent B has less merit than A , then B should not be treated better than A , ” we state the following generalization to uncertainty about merit , first for just two agents : Axiom 1 . If A has merit greater than or equal to B with probability at least ρ , then a fair policy should treat A at least as well as B with probability at least ρ . This being an axiom , we can not offer a mathematical justification . It captures an inherent sense of fairness in the absence of enough information , and it converges to the conventional tenet as uncertainty is reduced . In particular , consider the following situation : two items A , B with 10 reviews each have average star ratings of 3.9 and 3.8 , respectively ; or two job applicants A , B have GPAs of 3.9 and 3.8 . While this constitutes some ( weak ) evidence that A may have more merit than B , this evidence leaves substantial uncertainty . The posterior merit distributions based on the available information should reflect this uncertainty by having non-trivial variance ; our axiom then implies that A and B must be treated similarly to achieve fairness . In particular , it would be highly unfair to deterministically rank A ahead of B ( or vice versa ) . Our main point is that this uncertainty , rather than the specific numerical values of 3.9 and 3.8 , is the reason why a mechanism should treat A and B similarly . 1.1 Our Contributions . We study fairness in the presence of uncertainty specifically for the generalization where the principal must rank n items . Our main contribution is the fairness framework , giving definitions of fairness in ranking in the presence of uncertainty . This framework , including extensions to approximate notions of fairness , is presented and discussed in depth in § 2 . We believe that uncertainty of merit is one of the most important sources of unfairness , and modeling it explicitly and axiomatically is key to addressing it . Next , in § 3 , we present algorithms for a principal to achieve ( approximately ) fair ranking distributions . A simple algorithm the principal may use to achieve approximate fairness is to mix between an optimal ( unfair ) ranking and ( perfectly fair ) Thompson sampling . We show that this policy is not optimal for the principal ’ s utility , and we present an efficient LP-based algorithm that achieves an optimal ranking distribution for the principal , subject to an approximate fairness constraint . We next explore empirically to what extent a focus on fairness towards the agents reduces the principal ’ s utility . We do so with two extensive sets of experiments : one described in § 4 on existing data , and one described in § 5 “ in the wild. ” In the first set of experiments , we consider movie recommendations based on the standard MovieLens dataset and investigate to what extent fairness towards movies would result in lower utility for users of the system . The second experiment was carried out at the 2020 ACM SIGKDD Conference on Knowledge Discovery and Data Mining , where we implemented and fielded a paper recommendation system . Half of the conference attendees using the system received rankings that were modified to ensure greater fairness towards papers , and we report on various metrics that capture the level of engagement of conference participants based on which group they were assigned to . The upshot of our experiments and theoretical analysis is that in the settings we have studied , high levels of fairness can be achieved at a small loss in utility for the principal and the system ’ s users . Due to space constraints , essentially all proofs , as well as a detailed discussion of related work , are given in the supplementary material . Alternatively , a reader may want to read the full version ( Singh et al. , 2021 ) . 2 Ranking with Uncertain Merits . We are interested in ranking policies for a principal ( the ranking system , such as an e-commerce platform or a job portal in our earlier examples ) whose goal is to rank a set X of n agents ( such as products or applicants ) . The principal observes some evidence for the merit of the agents , and must produce a distribution over rankings trading off fairness to the agents against the principal ’ s utility . For the agents , a higher rank is always more desirable than a lower rank . 2.1 Rankings and Ranking Distributions . We use Σ ( X ) to denote the set of all n ! rankings , and Π ( X ) for the set of all distributions over Σ ( X ) . We express a ranking σ ∈ Σ ( X ) in terms of the agents assigned to given positions , i.e. , σ ( k ) is the agent in position k. A ranking distribution π ∈ Π ( X ) can be represented by the n ! probabilities π ( σ ) of the rankings σ ∈ Σ ( X ) . However , all the information relevant for our purposes can be represented more compactly using the Marginal Rank Distribution : we write p ( π ) x , k = ∑ σ : σ ( k ) =x π ( σ ) for the probability under π that agent x ∈ X is in position k in the ranking . We let P ( π ) = ( p ( π ) x , k ) x , k denote the n× n matrix of all marginal rank probabilities . The matrix P ( π ) is doubly stochastic , i.e. , the sum of each row and column is 1 . While π uniquely defines P ( π ) , the converse mapping may not be unique . However , given a doubly stochastic matrix P , the Birkhoff-von Neumann decomposition ( Birkhoff , 1946 ) can be used to compute some distribution π consistent with P , i.e. , P ( π ) = P ; any consistent distribution π will suffice for our purposes . 2.2 Merit , Uncertainty , and Fairness . The principal must determine a distribution over rankings of the agents . This distribution will be based on some evidence for the agents ’ merits . This evidence could take the form of star ratings and reviews of products ( combined with the site visitor ’ s partially known preferences ) , or GPA , test scores , and recommendation letters of an applicant . Our main departure from past work on individual fairness ( following ( Dwork et al. , 2012 ) ) is that we do not view this evidence as having inherent meaning ; rather , its sole role is to induce a posterior joint distribution over the agents ’ merits . The merit of agent x is vx ∈ R , and we write v = ( vx ) x∈X for the vector of all agents ’ merits . Based on all observed evidence , the principal can infer a distribution Γ over agents ’ merits using any suitable Bayesian inference procedure . Since the particular Bayesian model depends on the application , for our purposes , we merely assume that a posterior distribution Γ was inferred using best practices and that ideally , this model is open to verification and audit . We write Γ ( v ) for the probability of merits v under Γ . We emphasize that the distribution will typically not be independent over entries of v — for example , students ’ merit conditioned on observed grades will be correlated via common grade inflation if they took the same class . To avoid awkward tie-breaking issues , we assume that vx 6= vy for all distinct x , y ∈ X and all v in the support of Γ . This side-steps having to define the notion of top-k lists with ties , and comes at little cost in expressivity , as any tie-breaking would typically be encoded in slight perturbations to the vx anyway . We writeM ( v ) x , k for the event that under v , agent x is among the top k agents with respect to merit , i.e. , that | { x′ | vx′ > vx } | < k. We now come to our key definition of approximate fairness . Definition 2.1 ( Approximately Fair Ranking Distribution ) . A ranking distribution π is φ-fair iff k∑ k′=1 p ( π ) x , k′ ≥ φ · Pv∼Γ [ M ( v ) x , k ] ( 1 ) for all agents x and positions k. That is , the ranking distribution π ranks x at position k or above with at least a φ fraction of the probability that x is actually among the top k agents according to Γ . Furthermore , π is fair iff it is 1-fair . The reason for defining φ-approximately fair ranking distributions ( rather than just fair distributions ) is that fairness typically comes at a cost to the principal ( such as lower expected clickthrough or lower expected performance of recommended employees ) . For example , if the vx are probabilities that a user will purchase products on an e-commerce site , then deterministically ranking by decreasing EΓ [ vx ] is the principal ’ s optimal ranking under common assumptions about user behavior ; yet , being deterministic , it is highly unfair . Our definition of approximate fairness allows , e.g. , a policymaker to choose a trade-off regarding how much fairness ( with resulting expected utility loss ) to require from the principal . Notice that for φ = 0 , the principal is unconstrained . We remark that the merit values vx only matter insofar as comparison is concerned ; in other words , they are used ordinally , not cardinally . This is captured by the following proposition . Proposition 2.1 . Let f : R→ R be any strictly increasing function . Let Γ′ be the distribution that draws the vector ( f ( vx ) ) x∈X with probability Γ ( v ) for all v ; that is , it replaces each entry vx with f ( vx ) . Then , a ranking distribution π is φ-fair with respect to Γ if and only if it is φ-fair with respect to Γ′ . Proof . Because f is strictly increasing , Pv∼Γ [ M ( v ) x , k ] = Pv∼Γ′ [ M ( v ) x , k ] for all x and k. This immediately implies the claim by examining Definition 2.1 . Proposition 2.1 highlights a key aspect of our fairness definition : we avoid expressing any notion of fairness when “ one agent has just ‘ a little ’ more merit than the other , ” instead arguing that fairness is only truly violated when an agent with more merit is treated worse than one with less merit . In other words , fairness is inherently ordinal in our treatment . This viewpoint has implications for a principal seeking “ high-risk high-reward ” agents , which we discuss in more depth in § B.2 .

Summary: The paper proposes a new definition of fairness in rankings. An LP-based algorithm is given for finding best (max utility) ranking for a given fairness level. Two empirical evaluations of the proposed frameworks are given 1) based on ranking movies from a public dataset and 2) an experiment was run during a large virtual conference in which participants were given paper recommendations according to a ranking.