Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors of this work study fairness in two-sided platforms that utilize ranking as a mechanism for improve experience and utility of users and “items” (that can also be users). They introduce a new framework using the notion of Lorenz dominance through equity and efficiency. The work also contributes a novel algorithm (based on Frank-Wolfe) and show experimental evidence for its performance.
main_review: Originality: To the best of my knowledge, the results & contributions of the study are novel.

Quality: The work has a notable amount of contribution and all of them are supported by theoretical results (with a lot of detailed proofs in Supp. Mat.). Also, the authors provide experimental evaluation.

Clarity: On the one hand, the paper is well structured, with a good story line and introduction of all required theoretical definitions, theorems and experimental setup. On the other hand, the authors permanently consider several variants of their setup "one-sided recommendation", "reciprocal recommendation”, "one-sided preferences”, and "two-sided preferences"

For instance, Line 159 & Line 161: “In our case where there are users and items …” and “In reciprocal recommendation”. This is one among a lot of other lines of examples. Sometimes, it leads to misunderstanding of what is the considered case in this work.

Is it possible to make the work more focused on one single particular setup, while other ones localize in a dedicated place of the paper (e.g., move to discussion)?

[CLARIFIED DURING REBUTTAL PERIOD] Line 283: It is not clear about “O(K) additional bytes for each iteration” (the proof in Appendix did not help). Even \tilda P is new for each iteration t, but all previous \tilda P (from steps t-1, t-2, t-3, etc) are not required to keep in memory (Am I right?). So, we can use the same memory for each iteration. 


[CLARIFIED DURING REBUTTAL PERIOD] Significance: Overall, I appreciate the contribution of the work. However, I believe that it may significantly benefit for practice if it (at least) discusses how the results can be used in paid scenarios. Many ranking (recomm) systems are paid. So, how are these results related to such systems that are quite popular? Usually, every large system needs a monetization implying sponsored ranking / recommendations / suggestions / etc.: e.g., sponsored search, social networks, ecom, crowdsourcing platforms,…

Minor comments:
-	Line 130: I did not get “true” applied to “user”

limitations_and_societal_impact: No
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new ranking framework that concerns the fairness of all parties (i.e., users and items) participating in recommender systems. Inspired by techniques used in welfare economics, the authors formulate a convex optimization that yields Pareto-efficient and equitable (within each parties) utilities. To demonstrate the superiority over existing methods, they consider some special cases where the proposed approach is shown to achieve better utilities than the baseline methods. They provide an efficient implementation based on the Frank-Wolfe algorithm, together with theoretical results on tractability and convergence.
main_review: Strengths:
S1: The main idea (of employing Lorenz dominance to implement fair utility) is insightful.
S2: The proposed formulation is elegant: it covers both one-sided and reciprocal recommendation systems.
S3: Theoretical guarantees of the considered algorithm are provided in Theorems 1 and 2.

Weaknesses:
W1: Improvement over the considered baselines (particularly for equal_expo and equal_util) seems marginal. This blurs the superiority gained in Section 4 and hence weaken the motivation of using the proposed framework instead of a more simple item-side fairness scheme.
W2: Comparison with prior two-sided fairness algorithms (such as [41]) are missing. 
W3: "Tradeoffs" are unclear in Figure 2 and Figure 3. As described in Section 6, now the left column in Fig. 2 exhibits tradeoff between user and item utilities depending on alpha, but I am not sure this tension still remains when "alpha_1" and "alpha_2" take different values. It seems that if we take "alpha_1 = 0.95" and "alpha_2 = 0", then both user and item utilities yield the best curves with no trade-off. Figure 3 only shows aggregated two-sided utilities so tension between two utilities is impossible to see.

The technical content of this paper appears to be correct.
The paper was overall well-written yet would benefit from more proofreading. More precisely:
- Please remove "is” from "is is the value” in line 96 of page 3.

limitations_and_societal_impact: The paper discussed limitations and societal impacts adequetly in conclusion.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The authors study fairness in recommendation lists, where fairness is considered on both the part of consumers / users, as well as producers / items. They define fairness as a combination of Pareto efficiency and equity given that total level of utility, and note that non-dominated Lorenz curves satisfy their definition of fairness. They demonstrate that their method can be efficiently computed and show that it does well on real-world datasets.

main_review: The authors study a natural consideration in matching problems where you must balance fairness on the part of both the users and advertisers simultaneously.

The definition of fairness introduced by the authors makes sense in a two-sided market, and they bring up a good parallel to Pareto efficiency. However, Pareto efficiency (and Lorenz domination) can be a relatively weak property in the sense that a Pareto or Lorenz curve with one small "good part" may not ever be dominated by any other curve even though those curves are better "overall". 

I found the paper to be quite dense and notationally heavy, and some results were hard to interpret without the appendix. 

Around line 103, the analogies explaining why \mu_{ij} = \mu_{ji} in two-sided preferences don't exactly fit (at least not to me). Prospective employers and prospective employees derive different utilities from an interview; in general, people in relationships can derive different utilities as well. 

Around line 160, could you please explain the alphas a bit better? It seems like each alpha should be between 0 and 1 in order to get a strictly concave function, and the experiments show a range of alphas -- how did you choose them in asymmetric settings? 

I also found the figures in the experimental section a bit hard to interpret. Would it be helpful / informative to plot "perfectly equitable" Lorenz curves where applicable to more clearly illustrate domination? 


Minor comments:

44 and 46: either "can lead to a decrease in the utility" or "can decrease the utility"
47: worst-off
89: users
91: cardinality
92: the (i,j)-th
97: either? stress that this "either" is based on previous lines of work; it'll be clear in this paper depending on context
103: don't understand the analogies. different utilities to each side...suggest something like 
201: of the most satisfied users
224: qualitatively 
limitations_and_societal_impact: Yes.
review_title: null

Meta-review: Reviewers were very appreciative of the melding of traditional economic concepts such as Lorenz dominance with the topical problem of two-sided fairness in ranking/matching platforms.  Parallels to Pareto efficiency and other realistic desiderata and concerns were also appreciated.  Some comparisons to baselines/related work seemed marginal or weak.  In a subsequent or final version, I would ask that the authors do indeed make the changes promised during the post-rebuttal discussion (e.g., addition of discussion of sponsored search / other paid settings, all typos/method clarifications, newer results, and so on).  This is a case where the rebuttal and discussion directly and positively influenced the reviewers' (and this AC's) opinion.
Decision: Accept (Poster)