Two-sided fairness in rankings via Lorenz dominance

1 Introduction . Recommender systems have a growing impact on the information we see and on our life opportunities , as they help us browse news articles , find a new job , house , or people to connect with . While the objective of recommender systems is usually defined as maximizing the quality of recommendations from the user ’ s perspective , the recommendations also have an impact on the recommended “ items ” . News outlets rely on exposure to generate revenue , finding a job depends on which recruiter gets to see our resume , and the effectiveness of a dating application also depends on who we are recommended to—and if we are being recommended , then someone else is not . Two-sided fairness in rankings is the problem of generating personalized recommendations by fairly mediating between the interests of users and items . It involves a complex multidimensional trade-off . Fairness towards item producers requires boosting the exposure of small producers ( e.g. , to avoid winner-take-all effects and popularity biases [ 1 ] ) at the expense of average user utility . Fairness towards users aims at increasing the utility of the least served users ( e.g. , so that least served users do not support the cost of item-side fairness ) , once again at the expense of average user utility . The goal of this paper is to provide an algorithmic framework to generate rankings that achieve a variety of these trade-offs , leaving the choice of a specific trade-off to the practitioner . The leading approach to fairness in rankings is to maximize user utility under constraints of equal item exposure ( or equal quality-weighted exposure ) [ 54 , 7 ] or equal user satisfaction [ 6 ] . When these constraints imply an unacceptable decrease in average user utility , so-called “ trade-offs between utility and fairness ” [ 65 , 41 ] are obtained by relaxing the fairness constraints , leading to the optimization of a trade-off between average user utility and a measure of users ’ or items ’ inequality . Thinking about fairness in terms of optimal utility/inequality trade-offs has , however , two fundamental limitations . First , the optimization of a utility/inequality trade-off is not necessarily Pareto-efficient from the point of view of users and items : it sometimes chooses solutions that decrease the utility of some individuals without making anybody else better off . We argue that reducing inequalities by decreasing the utility of the better-off is not desirable if it does not benefit anyone . The second 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . limitation is that focusing on a single measure of inequality does not address the question of how inequality is reduced , and in particular , which fraction of the population benefits or bears the cost of reducing inequalities . In this paper , we propose a new framework for two-sided fairness in rankings grounded in the analysis of generalized Lorenz curves of user and item utilities . Widely used to study efficiency and equity in cardinal welfare economics [ 53 ] , these curves plot the cumulative utility obtained by fractions of the population ordered from the worst-off to the best-off . A curve that is always above another means that all fractions of the populations are better off . We define fair rankings as those with non-dominated generalized Lorenz curves for users and items . First , this definition guarantees that fair rankings are Pareto efficient . Second , examining the entirety of the generalized Lorenz curves provides a better understanding of which fractions of the population benefit from an intervention , and which ones have to pay for it . We present our general framework based on Lorenz dominance in usual recommendation settings ( e.g. , music or movie recommendation ) , and also show how extend it to reciprocal recommendation tasks such as dating applications or friends recommendation , where users are recommended to other users . We present a new method for generating rankings based on the maximization of concave welfare functions of users ’ and items ’ utilities . The parameters of the welfare function control the relative weight of users and items , and how much focus is given to the worse-off fractions of users and items . We show that rankings generated by maximizing our welfare functions are fair for every value of the parameters . Our framework does not aim at defining what parameters are suitable in general — rather , the choice of a specific trade-off depends on the application . From an algorithmic perspective , two-sided fairness is challenging because items ’ utilities depend on the rankings of all users , requiring global inference . Previous work on item-side fairness addressed this issue with heuristic methods without guarantees or control on the achievable trade-offs . We show how the Frank-Wolfe algorithm can be leveraged to make inference tractable , addressing both our welfare maximization approach and existing item-side fairness penalties . We demonstrate that our welfare function approach enjoys stronger theoretical guarantees than existing methods . While it always generates rankings with non-dominated generalized Lorenz curves , many other approaches do not . We show that one of the main criteria of the literature , called equity of attention by Biega et al . [ 7 ] , can lead to decrease user utility , while increasing inequalities of exposure between items . Moreover , equal user satisfaction criteria in reciprocal recommendation can lead to decrease the utility of every user , even the worse-off . Our notion of fairness prevents these undesirable behaviors . We report experimental results on music and friend recommendation tasks , where we analyze the trade-offs obtained by different methods by looking at different points of their Lorenz curves . Our welfare approach generates a wide variety of trade-offs , and is , in particular , more effective at improving the utility of worse-off users than the baselines . We present our formal framework in Section 2 . We discuss the theoretical properties of previous approaches in Section 3 , and present our ranking algorithm in Section 4 . Our experiments are described in Section 5 , and the related work is discussed in Section 6 . 2 Two-sided fairness via Lorenz dominance . 2.1 Formal framework . Terminology and notation . We identify an item with its producer , so that “ item utility ” means “ item producer ’ s utility ” . The main paper focuses on fairness towards individual users and items . We describe in Appendix B the extension of our approach to sensitive groups of users or items . |X | denotes the cardinal of the set X . Given n 2 N , we denote by [ [ n ] ] = { 1 , . . . , n } . The set of users N is identified with { 1 , ... , |N | } and the set of items I is identified with { |N | + 1 , ... , n } where n = |N |+ |I| . For ( i , j ) 2 N ⇥ I , we denote by µij the value of item j to user i . A ( deterministic ) ranking : I ! [ [ |I| ] ] is a one-to-one mapping from items j to their rank ( j ) . Following [ 54 ] , we use stochastic rankings because they allow us to perform inference using convex optimization ( see Section 4 ) . The recommender system produces one stochastic ranking per user , represented by a 3-way ranking tensor P where Pijk is the probability that j is recommended to i at rank k. We denote by P the set of ranking tensors . Utilities of users and items are defined through a position-based model , as in previous work [ 54 , 7 , 63 ] . Let v 2 R|I| , where vk is the exposure weight at rank k. We assume that lower ranks receive more exposure , so that 8k 2 [ [ |I| 1 ] ] , vk vk+1 0.1 Given a user i and a ranking i , the user-side utility of i is the sum of the µijs weighted by the exposure weight of their rank i ( j ) : ui ( i ) = P j2I v i ( j ) µij . Given an item j , the item-side utility of j is the sum over users i of the exposure of j to i . These definitions extend to stochastic rankings by taking the expectation over rankings , written in matrix form:2 user-side utility : ui ( P ) = X j2I µijPijv item-side utility ( exposure ) : uj ( P ) = X i2N Pijv We denote by u ( P ) = ( ui ( P ) ) ni=1 the utility profile for P , and by U = { u ( P ) : P 2 P } the set of feasible profiles . For u 2 U , uN = ( ui ) i2N and uI = ( ui ) i2I denote the utility profiles of users and items respectively . Two-sided fairness in rankings . In practice , values of µij are not known to the recommender system . Ranking algorithms use an estimate µ̂ of µ based on historical data . We address here the problem of inference : the task is to compute the ranking tensor given µ̂ , with the goal of making fair trade-offs between ( true ) user and item utilities . Notice that the user-side utility depends only on the ranking of the user , but for every item , the exposure depends on the rankings of all users . Thus , accounting for both users ’ and items ’ utilities in the recommendations is a global inference problem . More general item utilities We consider exposure as the item-side utility to follow prior work and for simplicity . Our framework and algorithm readily applies in a more general case of two-sided preferences , where items also have preferences over users ( for instance , in hiring , job seekers have preferences over which recruiters they are recommended to ) . Denoting µji the value of user i to item j , the item side-utility is then uj ( P ) = P i2N µjiPijv . 2.2 Lorenz efficiency and the welfare function approach . Our notion of fairness aims at improving the utility of the worse-off users and items . Since this does not prescribe exactly which fraction of the worse-off users/items should be prioritized , the assessment of trade-offs requires looking at all fractions of the population . This is captured by the generalized Lorenz curve used in cardinal welfare economics [ 53 ] . Formally , given a utility profile u , let ( u ( i ) ) ni=1 be the sorted values in u from smallest to largest , i.e. , u ( 1 ) . . . u ( n ) , then the generalized Lorenz curve plots ( Ui ) ni=1 where Ui = u ( 1 ) + . . . + u ( i ) . To assess the fairness of trade-offs , we rely on the following dominance relations on utility profiles : Pareto-dominance P. u P u0 ( ) ( 8i 2 [ [ n ] ] , ui u0i and 9i 2 [ [ n ] ] , ui > u0i ) . Lorenz-dominance L. Then u L u0 ( ) U P U 0 . We write ⌫L for non-strict Lorenz dominance ( i.e. , 8i , Ui U 0i ) . Notice that Pareto-dominance implies Lorenz-dominance . Our notion of fairness , which we call Lorenz efficiency , states that a ranking is fair if the utility profiles for users and for items are not jointly Lorenz-dominated : 1We use a user-independent v for simplicity . Considering user-dependent weights is straightforward . 2We consider Pij as a row vector in the formula , so that Pijv = P|I| k=1 Pijkvk . Definition 1 ( Lorenz efficiency ) . A utility profile u 2 U is Lorenz-efficient if there is no u0 2 U such that either ( u0I ⌫L uI and u0N L uN ) or ( u0N ⌫L uN and u0I L uI ) . We consider that Lorenz-dominated profiles are undesirable ( and unfair ) because the utility of worseoff fractions of the population could have been increased at no cost for total utility . Examples of Lorenz-curves of users and items are given in Fig . 1 . The blue solid , green dotted and orange dashed curves are all non-dominated ( the blue solid ranking has higher user utility but high item inequality , the green dotted and orange dashed curves have similar item exposure profiles , but user curves that intersect ) . On the other hand , the red dot/dashed curve is an unfair ranking : compared to the green dotted and orange dashed curve , all fractions of the worse off users have lower utility , together with less exposure for worse-off items . A fundamental result from cardinal welfare economics is that concave welfare functions of utility profiles order profiles according to Lorenz dominance [ 3 , 53 ] . The choice of the welfare function specifies which ( fair ) trade-off is desirable in a specific context . This result holds when all utilities are comparable . In our case where there are users and items , we propose the following welfare function parameterized by ✓ = ( , ↵1 , ↵2 ) :3 8u 2 Rn+ : W✓ ( u ) = ( 1 ) X i2N ( ui , ↵1 ) + X j2I ( uj , ↵2 ) with ( x , ↵ ) = 8 < : x ↵ if ↵ > 0 log ( x ) if ↵ = 0 x↵ if ↵ < 0 . Inference is carried out by maximizing W✓ ( an efficient algorithm is proposed in Section 4 ) : ( ranking procedure ) P ⇤ 2 argmax P2P W✓ ( u ( P ) ) ( 1 ) In W✓ , 2 [ 0 , 1 ] controls the relative weight of users and items . The motivation for the specific choice of is that it appears in scale invariant welfare functions [ 43 ] , but other families can be used as long as the functions are increasing and concave . Monotonicity implies that maxima of W✓ are Paretoefficient . For ↵1 < 1 and ↵2 < 1 , W✓ is strictly concave . Then , W✓ exhibits diminishing returns , which is the key to Lorenz efficiency : an increment in utility for a worse-off user/item increases welfare more than the same increment for a better-off user/item . The effect of the parameters is shown in Fig . 1 ( left ) : For item fairness we obtain more item equality by using ↵1 < 1 ( here , ↵1 = 0.5 ) and incrasing ( see blue solid vs orange dashed curve ) . The parameter ↵2 controls user fairness : smaller values yield more user utility for the worse-off users at the expense of total utility , with similar item exposure curve ( green dotted vs orange dahsed curves ) . Let ⇥ = { ( , ↵1 , ↵2 ) 2 ( 0 , 1 ) ⇥ ( 1,1 ) 2 } . For every ✓ 2 ⇥ , W✓ is strictly concave , and users and items have non-zero weight . We then have ( the result is a straightforward consequence of diminishing returns , see Appendix C ) : Proposition 1 . 8✓ 2 ⇥ , 8P ⇤ 2 argmax P2P W✓ ( u ( P ) ) , P ⇤ is Lorenz-efficient . Relationship to inequality measures A well-known measure of inequality is the Gini index , defined as 1 2⇥AULC , where AULC is the area under the Lorenz curve . The difference between Lorenz and generalized Lorenz curves is that the former is normalized by the cumulative utility . This difference is fundamental : we can decrease inequalities while dragging everyone ’ s utility to 0 . However , this would lead to dominated generalized Lorenz curves . Interestingly , for item-side fairness , the cumulative exposure is a constant and thus trade-offs between user utility and item exposure inequality are not really problematic . However , for user-side fairness , the total utility is not constant and reducing inequalities might require dragging the utility of some users down for the benefit of no one . Additional theoretical results In App . C.2 , we show that as ↵1 , ↵2 ! 1 , utility profiles tend to leximin-optimal solutions [ 43 ] . Leximin optimality corresponds to increasing the utility of the worst-off users/items one a a time , similarly to a lexical order . In App . C.3 , we present an excess risk bound , which provides theoretical guarantees on the true welfare when computing rankings based on estimated preferences , depending on the quality of the estimates . 3W✓ ( u ) = 1 if ↵ 0 and 9i , ui = 0 . In practice , we use ( x+ ⌘ , ↵ ) for ⌘ > 0 to avoid this case .

Summary: The authors study fairness in recommendation lists, where fairness is considered on both the part of consumers / users, as well as producers / items. They define fairness as a combination of Pareto efficiency and equity given that total level of utility, and note that non-dominated Lorenz curves satisfy their definition of fairness. They demonstrate that their method can be efficiently computed and show that it does well on real-world datasets.