SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Fan Yao
fy4bc@virginia.edu
Chuanhao Li
chuanhao.li.c12637@yale.edu
Karthik Abinay Sankararaman
karthikabinavs@meta.com
Yiming Liao
Meta  
iao@meta.com
Google  
yanzhuyz@google.com
Meta wqfcr@meta.com
University of Virginia hw5x@virginia.edu
Haifeng Xu
University of Chicago
haifengxu@uchicago.edu
The past decade has witnessed the flourishing of a new profession as media content creators, who rely on revenue streams from online content recommendation platforms. The reward mechanism employed by these platforms creates a competitive environment among creators which affects their production choices and, consequently, content distribution and system welfare. It is thus crucial to design the platform's reward mechanism in order to steer the creators' competition towards a desirable welfare outcome in the long run. This work makes two major contributions in this regard: first, we uncover a fundamental limit about a class of widely adopted mechanisms, coined Merit-based Monotone Mechanisms, by showing that they inevitably lead to a constant fraction loss of the optimal welfare. To circumvent this limitation, we introduce Backward Rewarding Mechanisms (BRMs) and show that the competition game resultant from BRMs possesses a potential game structure. BRMs thus naturally induce strategic creators' collective behaviors towards optimizing the potential function, which can be designed to match any given welfare metric. In addition, the class of BRM can be parameterized so that it allows the platform to directly optimize welfare within the feasible mechanism space even when the welfare metric is not explicitly defined.
Online recommendation platforms, such as Instagram and YouTube, have become an integral part of our daily life [5]. Their impact extends beyond merely aligning users with the most relevant content: they are also accountable for the online ecosystem it creates and the long-term welfare it promotes, considering the complex dynamics driven by the potential strategic behaviors of content creators [17]. Typically, creators' utilities are directly tied to the visibility of their content or economic incentives they can gather from the platform, and they constantly pursue to maximize these benefits [10, 12]. This fosters a competitive environment that may inadvertently undermine the social welfare, i.e., the total utilities of all users and content creators in the system [9]. For example, consider a scenario where the user population contains a majority group and many smaller minority groups, where different groups are interested in distinct topics. The social welfare is maximized when content distribution covers the variety of topics. However, a possible equilibrium of this competition can lead
most content creators to produce homogeneous content catering only to the majority group. This is because the benefits from creating niche content cannot offset the utility loss caused by forgoing the exposure from the majority of users. Such a phenomenon could potentially dampen the engagement of minority user groups or even instigate them to leave the platform altogether. This can consequently hurt the overall social welfare and also impact the platform's long-term revenue.
To counter such effects induced by strategic content creators, the platform can design proper incentives (i.e., rewarding mechanisms) to influence the creators' perceived utilities, thereby steering the equilibrium content distribution towards enhanced social welfare. In reality, many platforms share revenue with creators via various mechanisms [15, 19, 24, 20]. These incentives are typically proportional to user satisfaction measured by various metrics, such as click-through rate and engagement time. We model such competitive environment within a general framework termed content creator competition  $(C^3)$  game that generalizes and abstracts a few established models including [23, 4, 14, 13], and frame a class of popular rewarding mechanisms as Merit-based Monotone Mechanisms  $(\mathcal{M}^3)$ . The  $\mathcal{M}^3$  are characterized by a few simple properties, intuitively meaning better content should be rewarded more (i.e., merit-based) and sum of creators' utilities increase whenever any creator increases her content relevance (i.e., monotone). These properties reflect the essence of most employed rewarding mechanisms in practice. However, we show that  $\mathcal{M}^3$  necessarily incur a constant fraction of welfare loss in natural scenarios due to failing to encourage content creators who are content with generating popular content for majority user groups to produce niche content.
This surprising negative result uncovers the intrinsic incompatibility within  $\mathcal{M}^3$  mechanisms and thus compels us to rethink the incentive design in recommender systems (RS). A key property of  $\mathcal{M}^3$  is monotonicity, which stipulates that when the matching quality of exposed creators to a specific user group exhibits a Pareto improvement, the total reward received by those creators also increases. We point out that, while seemingly plausible at the first thought, this property undesirably encourages excessive concentration of creators around the majority user groups and leaves minority groups underserved. To resolve this issue, we question the validity of this monotone property. At a high level, when creators' competition within some user group surpasses a limit that begins to harm welfare, the platform should reduce their total gain. In light of this insight, we introduce a new class of content rewarding mechanism coined the Backward Rewarding Mechanisms (BRM), which drops monotonicity but remains merit-based. The strength of BRM lies in three aspects: 1. any  $C^3$  game under any BRM mechanism forms a potential game [16]; 2. we can identify a BRM mechanism such that the induced potential function is equivalent to any given social welfare metric; consequently, the net effect of creators' competition aligns perfectly with maximizing the social welfare; and 3. BRM contains a parameterized subclass of mechanisms that allows empirical optimization of the social welfare, which is especially useful in practice when the welfare is not explicitly defined. These merits of BRM are supported by our empirical studies, in which we developed simulated environments, demonstrating the welfare induced by BRM outperforms baseline mechanisms in  $\mathcal{M}^3$ .
The studies of content creators' strategic behavior under the mediation of an RS starts from the original work of [3, 4], who proposed the Shapley mediator that guarantees the existence of pure Nash equilibrium (PNE) and several fairness-related requirements. These works only study the design of the content-user matching probability, and it was observed that user welfare could be significantly compromised. In contrast, our work considers the design of another important "knob" of contemporary platforms — i.e., the reward for each content creator. We propose a broad class of rewarding mechanisms, namely, the Backward Rewarding Mechanisms (BRM). We show that the Shapley mediator of [4] turns out to be an example of our BRM class; however, by optimizing within the class of BRM mechanisms, the RS can now achieve the goal of maximizing social welfare.
Several recent work [13, 14, 23] studied the properties of creator-side equilibrium in the  $C^3$  game, under given creator incentives. In [13, 14], creators are assumed to directly compete for user exposure without the mediation of an RS. These studies focus on characterizing the Nash Equilibrium (NE) and identifying conditions that may trigger specialization among creators' strategies. [23] demonstrate that the user welfare loss under a conventional RS using top- $K$  ranking is upper-bounded by  $O\left(\frac{1}{\log K}\right)$  when creators compete for user engagement. Our work reveals that any merit-based monotone mechanism, including but not limiting to those based on user exposure or engagement, will inevitably incur at least a  $\frac{1}{K}$  fraction of welfare loss. However, should the platform can design creators' incentive signals, then exactly optimal social welfare could be obtained.
The main goal of the present work is to design incentives for creators to steer their collective behaviors towards social optimum. Such welfare-maximizing mechanism design has been studied extensively in social choice theory as well as in recent algorithmic mechanism design literature. Two of the most fundamental findings in this space are perhaps: (1) Vickrey-Clarke-Groves (VCG) mechanism which maximizes the social welfare of multi-item allocation [22]; and (2) the Arrow's impossibility theorem which employs an axiomatic approach to show the impossibility of welfare-maximization among natural voting mechanisms [1]. While welfare maximization in resource allocation and social choice has been extensively studied, to the best of our knowledge, our work is the first study of designing optimal mechanisms for welfare maximization in recommender systems. Interestingly, both our negative and positive results are inspired by the two fundamental results mentioned above. [1] shows that there is no ranked voting system capable of transforming the ranked preferences of individuals into a communal consensus ranking, while maintaining a natural set of criteria. Drawing a parallel to this concept and using the same axiomatic approach, our Theorem 1 can be interpreted as a similar impossibility result for welfare maximization under certain axioms in recommender systems - that is, no rewarding mechanism is capable of optimizing the social welfare while adhering to both "merit-based" and "monotone" properties. On the other hand, our positive result shows that there exists a creator rewarding mechanism that can maximize the RS's welfare at the potential-function-maximizing pure Nash equilibrium. The conceptual message of this result bears similarity to VCG's welfare maximization in multi-item allocation at the dominant-strategy equilibrium, but the techniques we employed is significantly different from VCG. To the best of our knowledge, this appears the first attempt to employ potential functions for welfare-maximizing mechanism design.
3 A General Model for Content Creator Competition
In this section, we formalize the Content Creator Competition  $(C^3)$  game as well as the platform's rewarding mechanisms. Each  $C^3$  game instance  $\mathcal{G}$  can be described by a tuple  $(\{\mathcal{S}_i\}_{i=1}^n, \{c_i\}_{i=1}^n, \mathcal{F}, \sigma, M, \{r_i\}_{i=1}^n)$  illustrated as follows:
1. Basic setups: The system has a user population/distribution  $\mathcal{F}$  with (discrete or continuous) support  $\mathcal{X} \subset \mathbb{R}^d$ , and a set of content creators denoted by  $[n] = \{1, \dots, n\}$ . Each creator  $i$  can take an action  $s_i$ , often referred to as a pure strategy in game-theoretic terms, from an action set  $S_i \subset \mathbb{R}^d$ . Any  $s_i \in S_i$  can be interpreted as the embedding of a content that creator  $i$  is able to produce. Let  $c_i(s_i)$  denote the production cost for creator  $i$  to generate  $s_i$ . As an example, one may think of  $c_i(s_i) = \lambda_i \| s_i - \bar{s}_i\|_2^2$  where  $\bar{s}_i$  represents the type of content that creator  $i$  is most comfortable or confident with, though our result is general and does not depend on any assumption of  $c_i$ . In general, any creator  $i$  may also play a mixed strategy, i.e., a distribution over  $S_i$ . However, for the purpose of this study, it suffices to consider pure strategies since it always exists in all our analysis and thus is a more natural solution concept.
The connection between any user  $\pmb{x}$  (drawn from  $\mathcal{F}$ ) and content  $s_i$  is described by a matching score function  $\sigma(s; \pmb{x}) : \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R}_{\geq 0}$  which measures the matching quality between a user  $\pmb{x} \in \mathcal{X}$  and content  $s$ . Without loss of generality, we normalize  $\sigma$  to [0, 1], where 1 suggests perfect matching. This work focuses on modeling the strategic behavior of creators, thus abstracts away the estimation of  $\sigma$  and simply views it as perfectly given. With slight abuse of notation, we use  $\sigma_i(\pmb{x})$  to denote  $\sigma(s_i; \pmb{x})$  given any joint creator action profile  $s = (s_1, \dots, s_n) \in S = \cup_{i=1}^{n} S_i$ . When it is clear from the context, we often omit the reference to the generic user  $\pmb{x}$  (drawn from population  $\mathcal{F}$ ) and simply use  $\sigma_i$  to denote creator  $i$ 's score.
2. Rewarding mechanisms and resultant creator utilities. Given joint strategy  $s = (s_1, \dots, s_n) \in S$ , the platform generates a reward  $u_i \in [0, 1]$  for each user-creator pair  $(s_i, x)$ . We generally allow  $u_i$  to depend on  $s_i$ 's matching score  $\sigma_i$  and also other creators' score  $\sigma_{-i} = \{\sigma_t | 1 \leq t \leq n, t \neq i\}$ . Thus, a rewarding mechanism  $M$  is a mapping from  $(\sigma_i, \{\sigma_{-i}\})$  to  $[0, 1]$ , which is denoted by the function  $M(\sigma_i; \sigma_{-i})$ . Such rewarding mechanisms can be interpreted as the expected payoff for creator  $i$  under any user-content matching strategy and some post-matching rewarding scheme. For example, suppose the platform matches creator  $s_i$  to
<sup>1</sup>The term "welfare" in social choice is classically more concerned with fairness or stability, as opposed to utility maximization.
2We will propose mechanisms that induce a potential game structure, which guarantees the existence of PNE[18].
It is an interesting open question of studying how our results can be extended to the situation in which the estimation of  $\sigma$  is inaccurate or has bias, though this is out of the scope of the present paper.
user  $\pmb{x}$  with probability  $p(\sigma_i;\sigma_{-i})$  and then reward each matched creator-  $i$  by some  $R_{i}$ . Then by letting  $R_{i} = \mathbb{I}[s_{i}$  matched to  $\pmb{x}]\frac{M(\sigma_i;\sigma_{-i})}{p(\sigma_i;\sigma_{-i})}$  we have  $\mathbb{E}[R_i] = M(\sigma_i;\sigma_{-i})$ . Given such correspondence between expected creator payoff and user-content matching/post-matching reward, we can without loss of generality refrain from the modeling of detailed matching policy and rewarding schemes, and simply focus on the design of  $M(\cdot ;\cdot)$ .
A few remarks about the reward mechanism  $M(\cdot; \cdot)$  follow. First,  $M$  is determined only by the profile of matching scores but not directly depend on the specific user  $x$ . However, our main results can be seamlessly generalized to allow  $M$  directly depend on  $x$ .<sup>4</sup> Second, the definition of  $M(\sigma; \sigma_{-})$  above naturally implies that it is "identity-invariant". That is, it specifies the reward of a matching score  $\sigma$ , generated by whichever creator, when facing a set of competitive matching scores in  $\sigma_{-}$ . While one could have considered more general identity-dependent rewarding mechanisms, they appear less realistic due to fairness concerns. More importantly, we shall show that such identity-invariant mechanisms already suffice to achieve optimal welfare.
Under the rewarding mechanism above, creator-  $i$  's expected utility is simply the expected reward gained from the user population  $\mathcal{F}$  minus the cost for producing content  $s_i$ , i.e.,
$$
u _ {i} (\boldsymbol {s}) = \mathbb {E} _ {\boldsymbol {x} \in \mathcal {F}} [ M \left(\sigma_ {i} (\boldsymbol {x}); \sigma_ {- i} (\boldsymbol {x})\right) ] - c _ {i} (\boldsymbol {s} _ {i}), \forall i \in [ n ], \tag {1}
$$
where  $\sigma_{i}(\pmb {x}) = \sigma (\pmb{s}_{i};\pmb {x})$  is the matching score between  $s_i, x$  and  $c_{i}$  is the cost function for creator-  $i$
3. User utility and the social welfare. Before formalizing the welfare objective, we first define a generic user  $\pmb{x}$ 's utility from consuming a list of ranked content. Since the user attention usually decreases in the rank positions, we introduce discounting weights  $\{r_k \in [0,1]\}_k$  to represent his/her "attention" over the  $k$ -th ranked content. Naturally, we assume  $r_1 \geq \dots \geq r_{n-1} \geq r_n$ , i.e., higher ranked content receives more user attention. Consequently, the user's utility from consuming a list of content  $\{l(k)\}_{k=1}^n$ , which is a permutation of  $[n]$  ranked in a descending order of match scores (i.e.,  $\sigma_{l(1)} \geq \sigma_{l_j(2)} \geq \dots \geq \sigma_{l_j(n)}$ ), is defined by the following weighted sum
$$
W (\boldsymbol {s}; \boldsymbol {x}) = \sum_ {k = 1} ^ {n} r _ {k} \sigma_ {l (k)} (\boldsymbol {x}). \tag {2}
$$
We provide additional examples that account for top- $K$  ranking rules with arbitrary ad-hoc permutations in Appendix 8.1. Finally, the social welfare is defined as the sum of total user utilities and total creator utilities, minus the platform's cost:
$$
\begin{array}{l} W (\boldsymbol {s}; \{r _ {k} \}) = \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {F}} [ W (\boldsymbol {s}; \boldsymbol {x}) ] + \sum_ {i = 1} ^ {n} u _ {i} (\boldsymbol {s}) - \sum_ {i = 1} ^ {n} \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {F}} [ M (\sigma_ {i} (\boldsymbol {x}); \sigma_ {- i} (\boldsymbol {x})) ] \\ = \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {F}} [ W (\boldsymbol {s}; \boldsymbol {x}) ] - \sum_ {i = 1} ^ {n} c _ {i} (\boldsymbol {s} _ {i}). \tag {3} \\ \end{array}
$$
The set of weights  $\{r_k\}$  determines a welfare metric  $W(\cdot ;\{r_k\})$ . For ease of exposure, we assume the sequence  $\{r_k\}$  is independent of specific user  $\pmb{x}$ . However, our results also hold for the more general situation where  $\{r_k\}$  is a function of the user profile  $\pmb{x}$ . In most of our analysis, we assume  $r_k$  can be measured and is known to the platform. However, we will later discuss how to address the situations where the platform only has blackbox access to  $W(\cdot ,\{r_k\})$ , but not the individual values of  $r_k$ .
The research question: creator incentive design for welfare maximization. Unlike previous works [2, 13, 14] that primarily focus on designing user-content matching mechanisms, we consider the design of a different, and arguably more general, "knob" to improve the system's welfare, i.e., creators' rewarding schemes. Each rewarding mechanism  $M$  establishes a competitive environment among content creators, encapsulated by a  $C^3$  instance  $\mathcal{G}(\{S_i\}, \{c_i\}, \mathcal{F}, \sigma, M, \{r_i\})$ . To characterize the outcome of  $C^3$  game, we consider the solution concept called Pure Nash Equilibrium (PNE), which is a joint strategy profile  $s^* = (s_1^*, \dots, s_n^*) \in S$  such that each player  $i$  cannot increase his/her utility by unilaterally deviating from  $s_i^*$ . Our objective is thus to design mechanisms  $M$  that: 1. guarantees the existence of PNE, thereby ensuring a stable outcome, and 2. maximizes social welfare at the PNE. In the upcoming sections, we first demonstrate why many existing rewarding mechanisms can fall short of achieving these goals, and then introduce our proposed new mechanism.
This may be useful when the system wants to specifically promote a particular user group by providing higher rewards to creators for serving this group.
4 The Fundamental Limit of Merit-based Monotone Mechanisms
In this section, we employ an axiomatic approach to demonstrate the fundamental limit of many employed rewarding mechanisms in today's practice. We identify a few properties (sometimes also called axioms [1]) of rewarding mechanisms that are considered natural in many of today's RS platforms, and then show that any mechanism satisfying these properties will necessarily suffer at least  $1 / K$  fraction of welfare loss at every equilibrium of some natural RS environments. Specifically, we consider mechanisms with the following properties.
Definition 1 (Merit-based Monotone Mechanisms  $(\mathcal{M}^3)$ ). We say  $M$  is a merit-based monotone mechanism if for any relevance scores  $1 \geq \sigma_1 \geq \dots \geq \sigma_n \geq 0$ ,  $M$  satisfies the following properties:
- Merit-based:
- (Normality)  $M(0; \sigma_{-i}) = 0$ ,  $M(1; \{0, \dots, 0\}) > 0$ ,
- (Fairness)  $M(\sigma_i; \sigma_{-i}) \geq M(\sigma_j; \sigma_{-j}), \forall i > j,$
- (Negative Externality)  $\forall i$ , if  $\sigma_{-i} \precsim \sigma_{-i}'$  ( $\sigma_j \leq \sigma_j'$ ,  $\forall j \neq i$ ), then  $M(\sigma_i; \sigma_{-i}) \geq M(\sigma_i; \sigma_{-i}')$ .
- Monotonicity: the total rewards  $\sum_{i=1}^{n} M(\sigma_i; \sigma_{-i}) : [0,1]^n \to \mathbb{R}_{\geq 0}$  is non-decreasing in  $\sigma_i$  for every  $i \in [n]$ .
We use  $\mathcal{M}^3 (n)$  to denote the set of all merit-based monotone mechanisms. When the context is clear, we omit the argument  $n$  and simply use the notation  $\mathcal{M}^3$ .
The two properties underpinning  $\mathcal{M}^3$  are quite intuitive. Firstly, the merit-based property consists of three natural sub-properties: 1. zero relevance content should receive zero reward, whereas the highest relevance content deserves a non-zero reward; 2. within the given pool of content with scores  $\{\sigma_i\}_{i\in [n]}$ , the higher relevance content should receive a higher reward; 3. any individual content's reward does not increase when other creators improve their content relevance. Secondly, monotonicity means if any content creator  $i$  improves her relevance  $\sigma_{i}$ , the total rewards to all creators increase. This property is naturally satisfied by many widely adopted rewarding mechanisms because platforms in today's industry typically reward creators proportionally to user engagement or satisfaction, the total of which is expected to increase as some creator's content becomes more relevant.
Indeed, many popular rewarding mechanisms can be shown to fall into the class of  $\mathcal{M}^3$ . For instances, the following two mechanisms defined over a descending score sequence  $\{\sigma_i\}$  are widely adopted in current industry practices for rewarding creators [15, 19, 20, 24], both of which are in  $\mathcal{M}^3$ :
1. When players' utilities are set to the total content exposure [2, 13, 14], we have  $M(\sigma_i; \sigma_{-i}) = \mathbb{I}[i \leq K] \frac{\exp(\beta^{-1} \sigma_i)}{\sum_{j=1}^{K} \exp(\beta^{-1} \sigma_j)}$ , with a temperature parameter  $\beta > 0$  controlling the spread of rewards.
2. When players' utilities are set to the user engagement [23], we have  $M(\sigma_i; \sigma_{-i}) = \mathbb{I}[i \leq K] \frac{\exp(\beta^{-1} \sigma_i)}{\sum_{j=1}^{K} \exp(\beta^{-1} \sigma_j)} \pi(\sigma_1, \dots, \sigma_n)$ , where  $\pi(\sigma_1, \dots, \sigma_n) = \beta \log \left( \sum_{j=1}^{K} \exp(\beta^{-1} \sigma_j) \right)$  is shown to be the total user welfare.
We show that any mechanism in  $\mathcal{M}^3$  may result in quite suboptimal welfare, even applied to some natural  $C^3$  game environment. We consider the following representative (though idealized) subclass of  $C^3$  instances, which we coin the Trend v.s. Niche (TvN) environments. As outlined in the introduction section, TvN captures the essence of many real-world situations.
Definition 2 (TvN Games). The Trend v.s. Niche (TvN) game is specified by the following RS environments:
- The user population  $\mathcal{F}$  is a uniform distribution on  $\mathcal{X} = \{\pmb{x}_j\}_{j=1}^{2n}$  where  $\pmb{x}_j = \pmb{e}_1$ , for  $1 \leq j \leq n + 1$ ,  $\pmb{x}_{n+2} = \pmb{e}_2, \dots, \pmb{x}_{2n} = \pmb{e}_n$  and  $E = \{e_1, \dots, e_n\} \subset \mathbb{R}^n$  is the set of unit basis in  $\mathbb{R}^n$ ;
- All creators have zero costs and share the same action set  $S_{i} = E$ ; the relevance is measured by the inner product, i.e.,  $\sigma(\boldsymbol{s};\boldsymbol{x}) = \boldsymbol{s}^{\top}\boldsymbol{x}$
- The attention discounting weights  $\{r_i\}$  is induced by a top- $K$  environment, i.e.,  $r_1 \geq \dots \geq r_K \geq r_{K+1} = \dots = r_n = 0$ .
The content creation competition game induced by any mechanism  $M$  is called a TvN game, denoted as  $\mathcal{G}(\{\mathcal{S}_i\}, \{c_i = 0\}, \mathcal{F}, \sigma, M, \{r_i\})$ .