Title: Rethinking Incentives in Recommender Systems: Are Monotone Rewards Always Beneficial?
Venue: NeurIPS 2023
Accepted: yes
Keywords: Recommender system

Mechanism design

Potential function

Optimization
Abstract: The past decade has witnessed the flourishing of a new profession as media content creators, who rely on revenue streams from online content recommendation platforms. The reward mechanism employed by these platforms creates a competitive environment among creators which affects their production choices and, consequently, content distribution and system welfare. It is thus crucial to design the platform's reward mechanism in order to steer the creators' competition towards a desirable welfare outcome in the long run. This work makes two major contributions in this regard: first, we uncover a fundamental limit about a class of widely adopted mechanisms, coined \emph{Merit-based Monotone Mechanisms}, by showing that they inevitably lead to a constant fraction loss of the optimal welfare. To circumvent this limitation, we introduce \emph{Backward Rewarding Mechanisms} (BRMs) and show that the competition game resultant from BRMs possesses a potential game structure. BRMs thus naturally induce strategic creators' collective behaviors towards optimizing the potential function, which can be designed to match any given welfare metric. In addition, the class of BRM can be parameterized so that it allows the platform to directly optimize welfare within the feasible mechanism space even when the welfare metric is not explicitly defined.

Rethinking Incentives in Recommender Systems: Are Monotone Rewards Always Beneficial?

Abstract

The past decade has witnessed the flourishing of a new profession as media content creators, who rely on revenue streams from online content recommendation platforms. The reward mechanism employed by these platforms creates a competitive environment among creators which affects their production choices and, consequently, content distribution and system welfare. It is thus crucial to design the platform's reward mechanism in order to steer the creators' competition towards a desirable welfare outcome in the long run. This work makes two major contributions in this regard: first, we uncover a fundamental limit about a class of widely adopted mechanisms, coined Merit-based Monotone Mechanisms , by showing that they inevitably lead to a constant fraction loss of the optimal welfare. To circumvent this limitation, we introduce Backward Rewarding Mechanisms (BRMs) and show that the competition game resultant from BRMs possesses a potential game structure. BRMs thus naturally induce strategic creators' collective behaviors towards optimizing the potential function, which can be designed to match any given welfare metric. In addition, the class of BRM can be parameterized so that it allows the platform to directly optimize welfare within the feasible mechanism space even when the welfare metric is not explicitly defined.

1 Introduction

Online recommendation platforms, such as Instagram and YouTube, have become an integral part of our daily life [5]. Their impact extends beyond merely aligning users with the most relevant content: they are also accountable for the online ecosystem it creates and the long-term welfare it promotes, considering the complex dynamics driven by the potential strategic behaviors of content creators [17]. Typically, creators' utilities are directly tied to the visibility of their content or economic incentives they can gather from the platform, and they constantly pursue to maximize these benefits [10, 12]. This fosters a competitive environment that may inadvertently undermine the social welfare , i.e., the total utilities of all users and content creators in the system [9]. For example, consider a scenario where the user population contains a majority group and many smaller minority groups, where different groups are interested in distinct topics. The social welfare is maximized when content distribution covers the variety of topics. However, a possible equilibrium of this competition can lead

Haifeng Xu

most content creators to produce homogeneous content catering only to the majority group. This is because the benefits from creating niche content cannot offset the utility loss caused by forgoing the exposure from the majority of users. Such a phenomenon could potentially dampen the engagement of minority user groups or even instigate them to leave the platform altogether. This can consequently hurt the overall social welfare and also impact the platform's long-term revenue.

To counter such effects induced by strategic content creators, the platform can design proper incentives (i.e., rewarding mechanisms) to influence the creators' perceived utilities, thereby steering the equilibrium content distribution towards enhanced social welfare. In reality, many platforms share revenue with creators via various mechanisms [15, 19, 24, 20]. These incentives are typically proportional to user satisfaction measured by various metrics, such as click-through rate and engagement time. We model such competitive environment within a general framework termed content creator competition ( C 3 ) game that generalizes and abstracts a few established models including [23, 4, 14, 13], and frame a class of popular rewarding mechanisms as Merit-based Monotone Mechanisms ( M 3 ). The M 3 are characterized by a few simple properties, intuitively meaning better content should be rewarded more (i.e., merit-based) and sum of creators' utilities increase whenever any creator increases her content relevance (i.e., monotone). These properties reflect the essence of most employed rewarding mechanisms in practice. However, we show that M 3 necessarily incur a constant fraction of welfare loss in natural scenarios due to failing to encourage content creators who are content with generating popular content for majority user groups to produce niche content.

This surprising negative result uncovers the intrinsic incompatibility within M 3 mechanisms and thus compels us to rethink the incentive design in recommender systems (RS). A key property of M 3 is monotonicity, which stipulates that when the matching quality of exposed creators to a specific user group exhibits a Pareto improvement, the total reward received by those creators also increases. We point out that, while seemingly plausible at the first thought, this property undesirably encourages excessive concentration of creators around the majority user groups and leaves minority groups underserved. To resolve this issue, we question the validity of this monotone property. At a high level, when creators' competition within some user group surpasses a limit that begins to harm welfare, the platform should reduce their total gain. In light of this insight, we introduce a new class of content rewarding mechanism coined the Backward Rewarding Mechanisms (BRM), which drops monotonicity but remains merit-based. The strength of BRM lies in three aspects: 1. any C 3 game under any BRM mechanism forms a potential game [16]; 2. we can identify a BRM mechanism such that the induced potential function is equivalent to any given social welfare metric; consequently, the net effect of creators' competition aligns perfectly with maximizing the social welfare; and 3. BRM contains a parameterized subclass of mechanisms that allows empirical optimization of the social welfare, which is especially useful in practice when the welfare is not explicitly defined. These merits of BRM are supported by our empirical studies, in which we developed simulated environments, demonstrating the welfare induced by BRM outperforms baseline mechanisms in M 3 .

2 Related Work

The studies of content creators' strategic behavior under the mediation of an RS starts from the original work of [3, 4], who proposed the Shapley mediator that guarantees the existence of pure Nash equilibrium (PNE) and several fairness-related requirements. These works only study the design of the content-user matching probability, and it was observed that user welfare could be significantly compromised. In contrast, our work considers the design of another important 'knob' of contemporary platforms - i.e., the reward for each content creator. We propose a broad class of rewarding mechanisms, namely, the Backward Rewarding Mechanisms (BRM). We show that the Shapley mediator of [4] turns out to be an example of our BRM class; however, by optimizing within the class of BRM mechanisms, the RS can now achieve the goal of maximizing social welfare.

Several recent work [13, 14, 23] studied the properties of creator-side equilibrium in the C 3 game, under given creator incentives. In [13, 14], creators are assumed to directly compete for user exposure without the mediation of an RS. These studies focus on characterizing the Nash Equilibrium (NE) and identifying conditions that may trigger specialization among creators' strategies. [23] demonstrate that the user welfare loss under a conventional RS using topK ranking is upper-bounded by O ( 1 log K ) when creators compete for user engagement . Our work reveals that any merit-based monotone mechanism, including but not limiting to those based on user exposure or engagement, will inevitably incur at least a 1 K fraction of welfare loss. However, should the platform can design creators' incentive signals, then exactly optimal social welfare could be obtained.

The main goal of the present work is to design incentives for creators to steer their collective behaviors towards social optimum. Such welfare-maximizing mechanism design has been studied extensively in social choice theory as well as in recent algorithmic mechanism design literature. Two of the most fundamental findings in this space are perhaps: (1) Vickrey-Clarke-Groves (VCG) mechanism which maximizes the social welfare of multi-item allocation [22]; and (2) the Arrow's impossibility theorem which employs an axiomatic approach to show the impossibility of welfare-maximization among natural voting mechanisms [1]. 1 While welfare maximization in resource allocation and social choice has been extensively studied, to the best of our knowledge, our work is the first study of designing optimal mechanisms for welfare maximization in recommender systems . Interestingly, both our negative and positive results are inspired by the two fundamental results mentioned above. [1] shows that there is no ranked voting system capable of transforming the ranked preferences of individuals into a communal consensus ranking, while maintaining a natural set of criteria. Drawing a parallel to this concept and using the same axiomatic approach, our Theorem 1 can be interpreted as a similar impossibility result for welfare maximization under certain axioms in recommender systems -that is, no rewarding mechanism is capable of optimizing the social welfare while adhering to both 'merit-based" and 'monotone" properties. On the other hand, our positive result shows that there exists a creator rewarding mechanism that can maximize the RS's welfare at the potentialfunction-maximizing pure Nash equilibrium. The conceptual message of this result bears similarity to VCG's welfare maximization in multi-item allocation at the dominant-strategy equilibrium, but the techniques we employed is significantly different from VCG. To the best of our knowledge, this appear the first attempt to employ potential functions for welfare-maximizing mechanism design.

3 A General Model for Content Creator Competition

In this section, we formalize the Content Creator Competition ( C 3 ) game as well as the platform's rewarding mechanisms. Each C 3 game instance G can be described by a tuple ( {S i } n i =1 , { c i } n i =1 , F , σ, M, { r i } n i =1 ) illustrated as follows:

Basic setups: The system has a user population/distribution F with (discrete or continuous) support X ⊂ R d , and a set of content creators denoted by [ n ] = { 1 , · · · , n } . Each creator i can take an action s i , often referred to as a pure strategy in game-theoretic terms, from an action set S i ⊂ R d . Any s i ∈ S i can be interpreted as the embedding of a content that creator i is able to produce. Let c i ( s i ) denote the production cost for creator i to generate s i . As an example, one may think of c i ( s i ) = λ i ∥ s i -¯ s i ∥ 2 2 where ¯ s i represents the type of content that creator i is most comfortable or confident with, though our result is general and does not depend on any assumption of c i . In general, any creator i may also play a mixed strategy, i.e., a distribution over S i . However, for the purpose of this study, it suffices to consider pure strategies since it always exists in all our analysis 2 and thus is a more natural solution concept.

The connection between any user x (drawn from F ) and content s i is described by a matching score function σ ( s ; x ) : R d × R d → R ≥ 0 which measures the matching quality between a user x ∈ X and content s . Without loss of generality, we normalize σ to [0 , 1] , where 1 suggests perfect matching. This work focuses on modeling the strategic behavior of creators, thus abstracts away the estimation of σ and simply views it as perfectly given. 3 With slight abuse of notation, we use σ i ( x ) to denote σ ( s i ; x ) given any joint creator action profile s = ( s 1 , · · · , s n ) ∈ S = ∪ n i =1 S i . When it is clear from the context, we often omit the reference to the generic user x (drawn from population F ) and simply use σ i to denote creator i 's score.

̸

Rewarding mechanisms and resultant creator utilities. Given joint strategy s = ( s 1 , · · · , s n ) ∈ S , the platform generates a reward u i ∈ [0 , 1] for each user-creator pair ( s i , x ) . We generally allow u i to depend on s i 's matching score σ i and also other creators' score σ -i = { σ t | 1 ≤ t ≤ n, t = i } . Thus, a rewarding mechanism M is a mapping from ( σ i , { σ -i } ) to [0 , 1] , which is denoted by the function M ( σ i ; σ -i ) . Such rewarding mechanisms can be interpreted as the expected payoff for creator i under any user-content matching strategy and some post-matching rewarding scheme. For example, suppose the platform matches creator s i to

user x with probability p ( σ i ; σ -i ) and then reward each matched creatori by some R i . Then by letting R i = I [ s i matched to x ] M ( σ i ; σ -i ) p ( σ i ; σ -i ) we have E [ R i ] = M ( σ i ; σ -i ) . Given such correspondence between expected creator payoff and user-content matching/post-matching reward, we can without loss of generality refrain from the modeling of detailed matching policy and rewarding schemes, and simply focus on the design of M ( · ; · ) .

A few remarks about the reward mechanism M ( · ; · ) follow. First, M is determined only by the profile of matching scores but not directly depend on the specific user x . However, our main results can be seamlessly generalized to allow M directly depend on x . 4 Second, the definition of M ( σ ; σ -) above naturally implies that it is 'identity-invariant'. That is, it specifies the reward of a matching score σ , generated by whichever creator, when facing a set of competitive matching scores in σ -. While one could have considered more general identity-dependent rewarding mechanisms, they appear less realistic due to fairness concerns. More importantly, we shall show that such identity-invariant mechanisms already suffice to achieve optimal welfare.

Under the rewarding mechanism above, creatori 's expected utility is simply the expected reward gained from the user population F minus the cost for producing content s i , i.e.,

where σ i ( x ) = σ ( s i ; x ) is the matching score between s i , x and c i is the cost function for creatori .

User utility and the social welfare. Before formalizing the welfare objective, we first define a generic user x 's utility from consuming a list of ranked content. Since the user attention usually decreases in the rank positions, we introduce discounting weights { r k ∈ [0 , 1] } k to represent his/her 'attention' over the k -th ranked content. Naturally, we assume r 1 ≥ · · · ≥ r n -1 ≥ r n , i.e., higher ranked content receives more user attention. Consequently, the user's utility from consuming a list of content { l ( k ) } n k =1 , which is a permutation of [ n ] ranked in a descending order of match scores (i.e., σ l (1) ≥ σ l j (2) ≥ · · · ≥ σ l j ( n ) ), is defined by the following weighted sum

We provide additional examples that account for topK ranking rules with arbitrary ad-hoc permutations in Appendix 8.1. Finally, the social welfare is defined as the sum of total user utilities and total creator utilities, minus the platform's cost:

The set of weights { r k } determines a welfare metric W ( · ; { r k } ) . For ease of exposure, we assume the sequence { r k } is independent of specific user x . However, our results also hold for the more general situation where { r k } is a function of the user profile x . In most of our analysis, we assume r k can be measured and is known to the platform. However, we will later discuss how to address the situations where the platform only has blackbox access to W ( · , { r k } ) , but not the individual values of r k .

The research question: creator incentive design for welfare maximization. Unlike previous works [2, 13, 14] that primarily focus on designing user-content matching mechanisms, we consider the design of a different, and arguably more general, 'knob' to improve the system's welfare, i.e., creators' rewarding schemes. Each rewarding mechanism M establishes a competitive environment among content creators, encapsulated by a C 3 instance G ( {S i } , { c i } , F , σ, M, { r i } ) . To characterize the outcome of C 3 game, we consider the solution concept called Pure Nash Equilibrium (PNE), which is a joint strategy profile s ∗ = ( s ∗ 1 , · · · , s ∗ n ) ∈ S such that each player i cannot increase his/her utility by unilaterally deviating from s ∗ i . Our objective is thus to design mechanisms M that: 1. guarantees the existence of PNE, thereby ensuring a stable outcome, and 2. maximizes social welfare at the PNE. In the upcoming sections, we first demonstrate why many existing rewarding mechanisms can fall short of achieving these goals, and then introduce our proposed new mechanism.

4 The Fundamental Limit of Merit-based Monotone Mechanisms

In this section, we employ an axiomatic approach to demonstrate the fundamental limit of many employed rewarding mechanisms in today's practice. We identify a few properties (sometimes also called axioms [1]) of rewarding mechanisms that are considered natural in many of today's RS platforms, and then show that any mechanism satisfying these properties will necessarily suffer at least 1 /K fraction of welfare loss at every equilibrium of some natural RS environments. Specifically, we consider mechanisms with the following properties.

Definition 1 (Merit-based Monotone Mechanisms ( M 3 )) . We say M is a merit-based monotone mechanism if for any relevance scores 1 ≥ σ 1 ≥ · · · ≥ σ n ≥ 0 , M satisfies the following properties:

Merit-based:

-(Normality) M (0; σ -i ) = 0 , M (1; { 0 , · · · , 0 } ) > 0 ,

-(Fairness) M ( σ i ; σ -i ) ≥ M ( σ j ; σ -j ) , ∀ i > j ,

-(Negative Externality) ∀ i , if σ -i ≼ σ ′ -i ( σ j ≤ σ ′ j , ∀ j = i ), then M ( σ i ; σ -i ) ≥ M ( σ i ; σ ′ -i ) .

̸

Monotonicity: the total rewards ∑ n i =1 M ( σ i ; σ -i ) : [0 , 1] n → R ≥ 0 is non-decreasing in σ i for every i ∈ [ n ] .

We use M 3 ( n ) to denote the set of all merit-based monotone mechanisms. When the context is clear, we omit the argument n and simply use the notation M 3 .

The two properties underpinning M 3 are quite intuitive. Firstly, the merit-based property consists of three natural sub-properties: 1. zero relevance content should receive zero reward, whereas the highest relevance content deserves a non-zero reward; 2. within the given pool of content with scores { σ i } i ∈ [ n ] , the higher relevance content should receive a higher reward; 3. any individual content's reward does not increase when other creators improve their content relevance. Secondly, monotonicity means if any content creator i improves her relevance σ i , the total rewards to all creators increase. This property is naturally satisfied by many widely adopted rewarding mechanisms because platforms in today's industry typically reward creators proportionally to user engagement or satisfaction, the total of which is expected to increase as some creator's content becomes more relevant.

Indeed, many popular rewarding mechanisms can be shown to fall into the class of M 3 . For instances, the following two mechanisms defined over a descending score sequence { σ i } are widely adopted in current industry practices for rewarding creators [15, 19, 20, 24], both of which are in M 3 :

When players' utilities are set to the total content exposure [2, 13, 14], we have M ( σ i ; σ -i ) = I [ i ≤ K ] exp( β -1 σ i ) ∑ K j =1 exp( β -1 σ j ) , with a temperature parameter β > 0 controlling the spread of rewards.

When players' utilities are set to the user engagement [23], we have M ( σ i ; σ -i ) = I [ i ≤ K ] exp( β -1 σ i ) ∑ K j =1 exp( β -1 σ j ) π ( σ 1 , · · · , σ n ) , where π ( σ 1 , · · · , σ n ) = β log ( ∑ K j =1 exp( β -1 σ j ) ) is shown to be the total user welfare.

We show that any mechanism in M 3 may result in quite suboptimal welfare, even applied to some natural C 3 game environment. We consider the following representative (though idealized) subclass of C 3 instances, which we coin the Trend v.s. Niche (TvN) environments. As outlined in the introduction section, TvN captures the essence of many real-world situations.

Definition 2 (TvN Games) . The Trend v.s. Niche (TvN) game is specified by the following RS environments:

The user population F is a uniform distribution on X = { x j } 2 n j =1 where x j = e 1 , for 1 ≤ j ≤ n +1 , x n +2 = e 2 , · · · , x 2 n = e n and E = { e 1 , · · · , e n } ⊂ R n is the set of unit basis in R n ;

All creators have zero costs and share the same action set S i = E ; the relevance is measured by the inner product, i.e., σ ( s ; x ) = s ⊤ x ;

The attention discounting weights { r i } is induced by a topK environment, i.e., r 1 ≥ · · · ≥ r K ≥ r K +1 = · · · = r n = 0 .

The content creation competition game induced by any mechanism M is called a TvN game, denoted as G ( {S i } , { c i = 0 } , F , σ, M, { r i } ) .

The TvN game models a scenario where the user population comprises multiple interest groups, each with orthogonal preference representations. In this game, the largest group consists of nearly half the population. Each content creator has the option to cater to one-and only one-user group. While this game is simple and stylized, it captures the essence of real-world user populations and the dilemmas faced by creators. Creators often find themselves at a crossroad: they must decide whether to pursue popular trends for a broader audience population, leading to intense competition, or focus on niche topics with a smaller audience and reduced competition. Our subsequent result shows that if the platform adopts M 3 in the TvN game, this tension of content creation turns out to be a curse in the sense that a unique PNE is achieved when all players opt for the same strategy - catering to the largest user group - and we quantify the social welfare loss at this PNE in the following.

Theorem 1. For any rewarding mechanism M ∈ M 3 applied to any TvN instance, we have

the resultant game admits a unique NE s ∗ ;

the welfare of this NE is at most K K +1 fraction of the optimal welfare for large n . Formally,

The proof is in Appendix 8.4, where we explicitly characterize both s ∗ and the welfare maximizing strategy profile and calculate their difference in terms of welfare. It is worthwhile to point out that the reciprocal of left-hand side of (4) is commonly known as the Price of Anarchy (PoA). This metric gauges the welfare loss at equilibrium compared to optimal welfare. (4) suggests that the PoA of G under M 3 could be as significant as 1 / 2 for users who primarily care about the top relevant content, which is shown to be realistic given the diminishing attention spans of Internet users [6]. This theorem shows that no mechanisms in M 3 can achieve the optimal welfare at the (unique) equilibrium of any TvN game. This naturally motivates our next question about how to design welfare-maximizing rewarding mechanisms in recommender systems.

5 Welfare Maximization via Backward Rewarding Mechanisms

The aforementioned failure of the generic M 3 class for welfare maximization urges us to re-think the rewarding mechanism design in recommender systems, especially for platforms where user attention is concentrated on the top few positions. Theorem 1 demonstrates certain inherent incompatibility between merit-based conditions and group monotonicity when it comes to optimizing welfare. Thus, a compromise must be made between the two, and our choice is the latter one. On one hand, any violation to the merit-based properties is challenging to justify as it undermines creators' perceptions about the value of the matching score metric. If creators discover that highly relevant content can receive lower payoffs or no rewards despite being the most relevant, it can be detrimental to the platform's reputation. On the other hand, while an increase in a creator's matching score σ i would naturally lead to an expected increase in his/her reward, it is generally unnecessary for the total rewards to increase as required by the monotonicity property. In fact, such non-monotonicity is widely observed in free markets, e.g., monopoly vs duopoly markets. For instance, consider a monopoly market with a high-quality producer and a low-quality producer, each catering to their distinct consumer bases. Now suppose the low-quality producer dramatically elevates his/her production quality to transition the market into a duopoly. While such an action would naturally augment the producer's profits, it would concurrently establish intensified competition with the high-quality producer, typically resulting in a marked decline in the latter's profitability. This would subsequently result in a decrease in the two producers' total profit [7, 25]. As will be clear later, our designed rewarding mechanism will lead to similar situations among content creators.

To enhance welfare, it is crucial to incentivize content creators who predominantly target larger user groups to also produce content for smaller groups. However, the monotone property encourages creators to continuously increase their matching scores to a user group, even when those users already have abundant options, resulting in diminishing welfare contributions. To address this, we introduce the class of Backward Rewarding Mechanisms (BRMs). The name of BRM suggests its essential characteristic: the reward for a specific creatori depends solely on their ranking and the matching scores of creators ranked lower than i . The formal definition of BRM is provided below:

Definition 3 (BRM and BRCM) . A Backward Rewarding Mechanism (BRM) M is determined by a sequence of Riemann integrable functions { f i ( t ) : [0 , 1] → R ≥ 0 } n i =1 , satisfying f 1 ( t ) ≥ · · · ≥

(a) Reward for

σ

1

(b) Reward for

σ

2

(c) Reward for

σ

3

f2(o)

fs(o)

0

03

02

01

1

0

03

02

01

1

0

03

02

01

1

f n ( t ) ∀ t ∈ [0 , 1] , such that for any matching score sequence 1 ≥ σ 1 ≥ · · · ≥ σ n ≥ 0 , the reward to any creator i is given by

where σ n +1 = 0 and f 1 ( t ) > 0 , ∀ t ∈ [0 , 1] . We use M [ f 1 ( t ) , · · · , f n ( t )] to denote the BRM determined by ordered function sequence { f i ( t ) } n i =1 .

In addition, we identify a sub-class of mechanisms BRCM ⊂ BRM which includes those M such that { f i ( t ) ≡ f i } are a set of constant functions. Any M ∈ BRCM can be parameterized by an n -dimensional vector in the polytope F = { ( f 1 , · · · , f n ) | f 1 ≥ · · · ≥ f n ≥ 0 } .

Figure 1 illustrates an example of BRM with function { f 1 , f 2 , f 3 } . According to its definition, the blue areas represent the rewards assigned by BRM and we can easily see why BRM preserves the merit-based property. Generally, the function f i ( · ) encapsulates the significance of the matching score difference between the i -th and ( i +1) -th ranked content in contributing to the i -th ranked creator's reward. The constraint f 1 ( t ) ≥ · · · ≥ f n ( t ) is necessary to satisfy merit-based properties, as shown in the proof of Proposition 1. The broad class of BRM offers granular control over creator incentives. Meanwhile, the subclass BRCM provides opportunities for parameterized optimization over welfare, which we will discuss in Section 5.2. Additional concrete examples of BRM are illustrated in Appendix 8.3.

5.1 Properties of BRM

While the class of BRM might appear abstract at the first glance, one can confirm that it preserves all merit-based properties, making it a natural class of rewarding mechanisms. Nevertheless, in order to secure a better welfare guarantee, the monotonicity is dropped, as characterized in the following:

Proposition 1. Any M ∈ BRM is merit-based but not necessarily monotone.

The detailed proof is provided in the Appendix 8.5. Next we establish formal characterizations about the welfare guarantee of BRM. First, we show that any C 3 game under BRM possesses a PNE because it is a potential game [16]. A strategic game is called a potential game if there exists a function P : ∏ i S i → R such that for any strategy profile s = ( s 1 , · · · , s n ) , any playeri and strategy s ′ i ∈ S i , whenever playeri deviates from s i to s ′ i , the change of his/her utility function is equal to the change of P , i.e.,

This leads us to the main result of this section:

Theorem 2. Consider any C 3 game G ( {S i } , { c i } , F , σ, M, { r i } ) .

The C 3 game is a potential game under any any mechanism M ∈ BRM, and thus admits a pure Nash equilibrium (PNE);

Moreover, if the mechanism M = M [ r 1 , · · · , r n ] ∈ BRCM ( ⊂ BRM), the potential function is precisely the welfare function, i.e., W ( s ) = P ( s ; M ) . Consequently, the always exists a PNE that obtains the optimal welfare.

The proof is in Appendix 8.6, where we construct its potential function explicitly. According to [16], we also conclude: 1. the maximizers of P are the PNEs of G , and 2. if the evolution of creators' strategic behavior follows a better response dynamics (i.e., in each iteration, an arbitrary creator deviates to a strategy that increases his/her utility), their joint strategy profile converges to a PNE.

Theorem 2 suggests another appealing property of BRM: one can always select an M within BRM to align the potential function with the welfare metric, which can be simply achieved by setting each f i identical to r i . Consequently, any best response dynamic among creators not only converges to a PNE but also generates a strictly increasing sequence of W , thus ensuring at least a local maximizer of W . Denote the set of PNEs of G as PNE ( G ) . When PNE ( G ) coincides with the global maximizers of its potential function, i.e., PNE ( G ) = argmax s P ( s ; M ) , we conclude that any PNE of G also maximizes the welfare W . The following corollary indicates that such an optimistic situation occurs in TvN games, providing a stark contrast to the findings in Theorem 1.

Corollary 1. For any TvN instance G , there exists M ∈ BRCM such that any PNE s ∗ ∈ PNE ( G ) attains the optimal W , i.e.,

The proof is in Appendix 8.7. Despite the promising results presented in Corollary 1, it remains uncertain whether the strong welfare guarantee for TvN can be extended to the entire class of C 3 . This uncertainty arises because, in general, we only know that argmax s P ( s ; M ) ⊆ PNE ( G ) . However, [21] noted that the subset of PNEs corresponding to argmax s P ( s ; M ) in any potential game is robust in the following sense: in an incomplete information relaxation of G , where each creator possesses a private type and must take actions based on their beliefs about other creators' types, they will play the strategies in argmax s P ( s ; M ) at the Bayesian Nash equilibrium with a probability close to 1. This insight suggests that BRM has the potential to achieve optimal social welfare in real-world scenarios. While we lack a conclusive theoretical determination of whether BRM can attain globally optimal welfare, our empirical study in Section 6 consistently reveals that BRM outperforms baseline mechanisms in M 3 in terms of improving welfare.

5.2 Welfare Optimization within BRCM

Theorem 2 suggests that, provided the parameters { r i } are known, the platform can select a mechanism within BRCM with a better welfare guarantee. However, in many practical scenarios, the platform may not have access to the exact values of { r i } but can only evaluate the resulting welfare metric using certain aggregated statistics. This presents a challenge as it may not be analytically feasible to pinpoint the optimal M as suggested by Theorem 2. In these cases, although perfect alignment between the potential function P and social welfare W may not be feasible, we can still find a mechanism that approximates the maximizer of W in creator competition. This leads us to formulate the following bi-level optimization problem:

In problem (7), the inner optimization (8) is executed by creators: for any given M , we have justified that the creators' strategies is very likely to settle at a PNE s ∗ ( M ) that corresponds to a maximizer of P ( s ; M ) . However, the exact solution to the inner problem is neither analytically solvable by the platform (owing to the combinatorial nature of P ) nor observable from real-world feedback (due to creators' potentially long feedback cycles). Therefore, we propose to approximate its solution by simulating creators' strategic response sequences (See Appendix 8.8, Algorithm 1), on top of which we solve (7). Algorithm 1 is a variant of better response dynamics, incorporating randomness and practical considerations to more accurately emulate creator behavior, and will be employed as a subroutine in Algorithm 2. We should note that the specifics of the creator response simulator are not critical to our proposed solution: the optimizer can select any equilibrium-finding dynamic to replace our simulator 1, as long as it is believed to better represent creators' responses in reality.

Another challenge of solving (7) lies in the presence of ranking operations in W , which makes it non-differentiable in s and renders first-order optimization techniques ineffective. Consequently, we resort to coordinate update and apply finite differences to estimate the ascending direction of W with respect to each M parameterized by f = ( f 1 , · · · , f n ) ∈ F . Our proposed optimization algorithm for solving (7) is presented in Algorithm 2 in Appendix 8.9, and is structured into L 1 epochs. At the

beginning of each epoch, the optimizer randomly perturbs the current M along a direction within the feasible polytope and simulates creators' responses for L 2 steps using Algorithm 1. Welfare is re-evaluated at the end of this epoch, and the perturbation on M is adopted if it results in a welfare increase.

6 Experiments

To validate our theoretical findings and demonstrate the efficacy of Algorithm 2, we simulate the strategic behavior of content creators and compare the evolution of social welfare under various mechanisms. These include Algorithm 2 and several baselines from both the M 3 and BRCM classes.

6.1 Specification of Environments

We conduct simulations on game instances G ( {S i } , { c i } , F , σ, M, { r i } ) constructed from synthetic data. Results on MovieLens-1m dataset [11] are shown in Appendix 8.10. For the synthetic data, we consider a uniform distribution F on X as follows: we fix the embedding dimension d and randomly sample Y cluster centers, denoted as c 1 , · · · , c Y , on the unit sphere S d -1 . For each center c i , we generate users belonging to clusteri by first independently sampling from a Gaussian distribution ˜ x ∼ N ( c i , v 2 I d ) , and then normalize it to S d -1 , i.e., x = ˜ x / ∥ ˜ x ∥ 2 . The sizes of the Y user clusters are denoted by a vector z = ( z 1 , · · · , z Y ) . In this manner, we generate a population X = ∪ Y i =1 X i with size m = ∑ Y i =1 z i . The number of creators is set to n = 10 , with action sets S i = S d -1 . The relevance function σ ( x , s ) = 1 2 ( s ⊤ x +1) is the shifted inner product such that its range is exactly [0 , 1] . { r i } n i =1 is set to { 1 log 2 (2) , · · · , 1 log 2 (5) , 1 log 2 (6) , 0 , · · · , 0 } . These synthetic datasets simulate situations where content creators compete over a clustered user preference distribution. We consider two types of game instances, denoted G 1 and G 2 , distinguished by their cost functions:

In G 1 , creators have zero cost and their initial strategies are set to the center of the largest user group. This environment models the situation where the social welfare is already trapped at suboptimal due to its unbalanced content distribution. We aim to evaluate which mechanism is most effective in assisting the platform to escape from such a suboptimal state.

In G 2 , creators have non-trivial cost functions c i = 0 . 5 ∥ s i -¯ s i ∥ 2 2 , where the cost center ¯ s i is randomly sampled on S d -1 . Their initial strategies are set to the corresponding cost centers, i.e., all creators start with strategies that minimize their costs. This environment models a 'cold start" situation for creators: they do not have any preference nor knowledge about the user population and gradually learn about the environment under the platform's incentivizing mechanism.

In our experiment, we set ( d, v, Y, m ) = (10 , 0 . 3 , 8 , 52) and the cluster sizes z = (20 , 10 , 8 , 5 , 3 , 3 , 2 , 1) . The 8 clusters are devided into 3 groups ((20) , (10 , 8) , (5 , 3 , 3 , 2 , 1)) , namely group-1,2,3, corresponding to the majority, minority, and niche groups.

6.2 Algorithm and Baseline Mechanisms

We simulate the welfare curve produced by Algorithm 2 alongside five baseline mechanisms below.

BRCM opt : This refers to the dynamic mechanism realized by optimization Algorithm 2. The parameters are set to T = 1000 , L 1 = 200 , L 2 = 5 , η 1 = η 2 = 0 . 1 , f (0) = (1 , 1 , 1 , 1 , 1 , 0 , · · · , 0) .

BRCM ∗ : This denotes the theoretically optimal mechanism within BRCM, as indicated by Theorem 2. The corresponding parameters of M are derived based on the knowledge of { r i } n i =1 .

BRCM 1 : BRCM 1 = M [1 , 1 2 , 1 3 , 1 4 , 1 5 , 0 , · · · , 0] ∈ BRCM. This baseline aims to assess the impact of deviation from the theoretically optimal mechanism on the result.

M 3 (0) : This mechanism assigns each content creator a reward equal to the relevance score, i.e., M ( σ i ; σ -i ) = σ i . It is obvious that this mechanism belongs to the M 3 class and is therefore denoted as M 3 (0) . Under M 3 (0) , each creator's strategy does not affect other creators' rewards at all, and thus every creator will be inclined to match the largest user group as much as their cost allows. This mechanism acts as a reference to indicate the worst possible scenario.

M 3 ( expo. ) : The mechanism based on exposure, defined in Section 4 with K = 5 , β = 0 . 05 .

M 3 ( enga. ) : The mechanism based on engagement, defined in Section 4 with K = 5 , β = 0 . 05 .

0

200

400

600

800

1000

time step

35

40

45

50

55

60

65

Welfare

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

(a) Welfare on

G

1

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

Average User Utility

Group-1

Group-2

Group-3

(b) Avg. Group Util. on

G

1

0

200

400

600

800

1000

time step

20

30

40

50

60

Welfare

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

(c) Welfare on

G

2

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Average User Utility

Group-1

Group-2

Group-3

(d) Avg. Group Util. on

G

2

6.3 Results

We let creators play G 1 and G 2 repeatedly under mechanisms specified in Section 6.2 and record the social welfare over T = 1000 steps with Algorithm 1 in in Figure 2.

As illustrated in Figure 2a, BRCM family consistently outperformed M 3 . As anticipated, M 3 (0) does little to enhance social welfare when creators have already primarily focused on the most populous user group. The M 3 ( expo. ) and M 3 ( enga. ) mechanisms demonstrate a notable improvement over M 3 (0) as they instigate a competitive environment for creators striving to reach the topK positions. Nevertheless, they still do not perform as effectively as BRCM 1 , even though BRCM 1 's parameter deviates from the theoretically optimal one. Within the BRCMs, BRCM opt exhibits remarkable performance and even surpasses the theoretically optimal instance BRCM ∗ . One possible explanation for the empirical sub-optimality of BRCM ∗ is the stochastic nature of creators' response dynamics, which might prevent the convergence to PNE associated with the maximum welfare without sufficient optimization. This observation underscores the importance of Algorithm 2, as it empowers the platform to pinpoint an empirically optimal mechanism in more practical scenarios. As depicted in Figure 2b, the primary source of advantage stems from the increased utility among minority and niche user groups: compared to M 3 ( expo. ) and M 3 ( enga. ) , BRCM class results in higher average utility for groups 2 and 3 while preserving overall satisfaction for group-1.

Similar observations can be made for G 2 . However, it is worth noting that BRCM opt underperformed slightly in comparison to BRCM ∗ as shown in Figure 2c. Despite this, the BRCM class of mechanisms continued to significantly surpass those in M 3 . Figure 2d further highlights that BRCM mechanisms lead to a more equitable distribution of average user utility across different user groups. Nevertheless, the gap in comparison becomes less pronounced, which is probably due to the existence of costs. Creators burdened with such costs are inherently inclined towards serving specific user groups, making them less susceptible to the influence of platform's incentives.

7 Conclusion

Our work reveals an intrinsic limitation of the monotone reward principle, widely used by contemporary online content recommendation platforms to incentivize content creators, in optimizing social welfare. As a rescue, we introduce BRM, a novel class of reward mechanisms with several key advantages. First, BRM ensures a stable equilibrium in content creator competition, thereby fostering a consistent and sustainable content creation environment. Second, BRM can guide content creators' strategic responses towards optimizing social welfare, providing at least a local optimum for any given welfare metric. Finally, BRM offers a parameterized subspace that allows the platform to empirically optimize social welfare, enhancing platform performance dynamically.

For future work, we identify two potential directions. From a theoretical standpoint, it would be intriguing to ascertain whether a stronger welfare guarantee for BRM could be established when the scoring function is equipped with certain simple structures, e.g., dot product. On the empirical side, we look for developments of our suggested mechanism by addressing some practical considerations. For instance, how can we enhance the robustness of BRM to account for the estimation noise in relevance scores? And how can a platform optimize welfare subject to budget constraints? Deeper insights into these questions could significantly enhance our understanding of the rapidly evolving online content ecosystems.

Acknowledgment This work is supported in part by an NSF Award CCF-2303372, IIS-2128019, and IIS-2007492, an Army Research Office Award W911NF-23-1-0030, and an Office of Naval Research Award N00014-23-1-2802.

References

Kenneth J Arrow. A difficulty in the concept of social welfare. Journal of political economy , 58(4):328-346, 1950.

Omer Ben-Porat, Gregory Goren, Itay Rosenberg, and Moshe Tennenholtz. From recommendation systems to facility location games. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 1772-1779, 2019.

Omer Ben-Porat and Moshe Tennenholtz. Shapley facility location games. In International Conference on Web and Internet Economics , pages 58-73. Springer, 2017.

Omer Ben-Porat and Moshe Tennenholtz. A game-theoretic approach to recommendation systems with strategic content providers. Advances in Neural Information Processing Systems , 31, 2018.

Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. Recommender systems survey. Knowledge-based systems , 46:109-132, 2013.

Nicholas Carr. The shallows: What the Internet is doing to our brains . WWNorton & Company, 2020.

Bruno De Borger and Kurt Van Dender. Prices, capacities and service levels in a congestible bertrand duopoly. Journal of Urban Economics , 60(2):264-283, 2006.

Jicong Fan and Jieyu Cheng. Matrix completion by deep matrix factorization. Neural Networks , 98:34-41, 2018.

Daniel Fleder and Kartik Hosanagar. Blockbuster culture's next rise or fall: The impact of recommender systems on sales diversity. Management science , 55(5):697-712, 2009.

Angela Glotfelter. Algorithmic circulation: how content creators navigate the effects of algorithms on their work. Computers and composition , 54:102521, 2019.

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) , 5(4):1-19, 2015.

Thomas Hodgson. Spotify and the democratisation of music. Popular Music , 40(1):1-17, 2021.

Jiri Hron, Karl Krauth, Michael I Jordan, Niki Kilbertus, and Sarah Dean. Modeling content creator incentives on algorithm-curated platforms. arXiv preprint arXiv:2206.13102 , 2022.

Meena Jagadeesan, Nikhil Garg, and Jacob Steinhardt. Supply-side equilibria in recommender systems. arXiv preprint arXiv:2206.13489 , 2022.

Meta. Meta is experimenting with new monetization options for creators, 2022. https://www.digitalinformationworld.com/2022/03/meta-is-experimenting-with-new.html.

Dov Monderer and Lloyd S Shapley. Potential games. Games and economic behavior , 14(1):124143, 1996.

Kun Qian and Sanjay Jain. Digital content creation: An analysis of the impact of recommendation systems. Available at SSRN 4311562 , 2022.

Robert W Rosenthal. A class of games possessing pure-strategy nash equilibria. International Journal of Game Theory , 2:65-67, 1973.

Savy. Will the new youtube algorithm impact your content?, 2019. https://savyagency.com/newyoutube-algorithm/.

TikTok. What is the tiktok creator fund? here's how to join + start making money, 2022. https://www.backstage.com/magazine/article/tiktok-creator-fund-explained-how-to-join75090/.

Takashi Ui. Robust equilibria of potential games. Econometrica , 69(5):1373-1380, 2001.

Hal R Varian and Christopher Harris. The vcg auction in theory and practice. American Economic Review , 104(5):442-445, 2014.

Fan Yao, Chuanhao Li, Denis Nekipelov, Hongning Wang, and Haifeng Xu. How bad is topk recommendation under competing content creators? In International Conference on Machine Learning . PMLR, 2023.

Youtube. Youtube partner program overview & eligibility, 2023. https://support.google.com/youtube/answer/72851.

Piercarlo Zanchettin. Differentiated duopoly with asymmetric costs. Journal of Economics & Management Strategy , 15(4):999-1015, 2006.

8 Supplementary Material

8.1 Additional Examples of User Utility Function

Fix any user x ∈ F , let σ i = σ ( s i ; x ) and for simplicity of notations we assume σ 1 ≥ · · · ≥ σ n . As discussed in Section 3, if the platform presents the topK ranked content in terms of their relevance quality, the user utility function has the following form:

where { r k } n k =1 are the user's 'attention' over the k -th ranked content such that r k = 0 , ∀ k ≥ K +1 . We emphasize that our user utility model given in Eq.(9) is compatible with various matching strategies and here we provide additional examples that incorporate a modified version of the topK approach, taking into account considerations of advertised content. For instance, considering a scenario where K = 5 and the platform intends to promote the content originally ranked at position 6 to position 2 with probability p ∈ (0 , 1) . Consequently, the resulting utility function can be expressed as follows:

This example shows that user utility function under any position-based perturbation of topK ranking can be expressed in the form of Eq.(9), and in general the values of r k , k > K can be non-zero.

8.2 Examples of M 3

In this section we formally justify that the two examples given in Section 4 belong to the class of M 3 .

When the creators' utilities are set to the total content exposure [2, 13, 14], we have M ( σ i ; σ -i ) = I [ i ≤ K ] exp( β -1 σ i ) ∑ K j =1 exp( β -1 σ j ) , with a temperature parameter β > 0 controlling the spread of rewards. The validity of three merit-based properties are straightforward. In terms of monotonicity, we have ∑ n M ( σ i ; σ -i ) = 1 which is a constant and thus monotone.

i =1

When the creators' utilities are set to the total user engagement [23], we have M ( σ i ; σ -i ) = I [ i ≤ K ] exp( β -1 σ i ) ∑ K j =1 exp( β -1 σ j ) π ( σ 1 , · · · , σ n ) , where π ( σ 1 , · · · , σ n ) = β log ( ∑ K j =1 exp( β -1 σ j ) ) . The first two merit-based properties are obvious (Normality and Fairness). In terms of monotonicity, we have ∑ n i =1 M ( σ i ; σ -i ) = β log ( ∑ K j =1 exp( β -1 σ j ) ) which is monotone in each σ j . To verify negative externality, it suffices to show the function log ( ∑ K j =1 exp( β -1 σ j ) ) ∑ K j =1 exp( β -1 σ j ) is decreasing in σ j , ∀ j . Since exp( x ) is increasing in x , and function log( t ) t is decreasing when t > e , we conclude that M satisfies negative externality when n ≥ 3 .

8.3 Examples of BRM

To get an better intuition of how BRM works, let us consider a special case M ∈ BRCMsuchthat f 1 = · · · = f K = 1 and f k = 0 , k ≥ K +1 . By the definition, any matching score sequence σ 1 ≥ · · · ≥ σ n will be mapped to a reward sequence of ( σ 1 -σ K +1 , · · · , σ K -σ K +1 , 0 , · · · , 0) . Consequently, the topK ranked creators will experience a significant reduction in rewards if the ( K +1) -th ranked creator increases its matching score. This mechanism can deter an unnecessary concentration of creators on a specific strategy, as when the number of creators with high scores exceeds a certain threshold, even those ranked highly can receive a decreasing reward. This backward rewarding mechanism thus encourages diversity in content creation and mitigates the risk of oversaturation in any particular group of users.

Another notable special case within BRCM ∈ BRM is M SM = M [1 , 1 2 , · · · , 1 n ] , which coincides with the Shapley mediator proposed in [4]. One key feature of M SM is that for any sequence 1 ≥ σ 1 ≥ · · · ≥ σ n ≥ 0 , it holds that ∑ n i =1 M SM ( σ i ; σ -i ) = σ 1 ≤ 1 . This implies that the platform can avoid providing explicit incentives and merely implement these rewards as matching probabilities. However, to do so, it must accommodate the possibility of not matching a user with any creators, corresponding to a probability of 1 -σ 1 . Furthermore, it does not support the topK ranking strategy.

A more comprehensive understanding about the construction of BRM can be obtained through the lens of congestion games. As pointed out by [16], every finite potential game is isomorphic to a congestion game. Furthermore, the definition of M as outlined in Eq.(5) can be interpreted as the utility that creator i acquires from the following congestion game:

The set of congestible elements are given by the continuum E = X × [0 , 1] , where each element ( x , t ) ≜ e ∈ E corresponds to a user x with satisfaction level t .

The n players are n content creators.

Each creator's pure action s i ∈ S i can be mapped to a subset of E in the following way: the action s i determines the matching score σ ( s i ; x ) over each x ∈ X , and then s i is mapped to a subset { ( x , t ) | x ∈ X , t ∈ [0 , σ ( s i ; x )] } ≜ S i ⊆ E .

For each element e and a vector of strategies ( S 1 , · · · , S n ) , the load of element e is defined as x e = # { i : e ∈ S i } , i.e., the number of players who occupy e .

For each element e , there is a payoff function d e : N → R ≥ 0 that only depends on the load of e .

For any joint strategy ( S 1 , · · · , S n ) , the utility of player i is given by ∑ e ∈ S i d e ( x e ) , i.e., the sum of reward he/she collects from all occupied elements. For each occupied element e , the reward is determined by its 'congestion' level x e , which is characterized by the payoff function d e .

To better understand the constructed congestion game and the utility definition given in Eq.(5), we can consider each element in E (i.e., a user with a particular satisfaction level) as an atomic 'resource'. Each production strategy adopted by an individual creator can be thought of as occupying a subset of these resources. Given a fixed strategy profile, the load of e = ( x , t ) is determined by the number of creators who achieve a matching score exceeding t for user x , thereby linking the ranking of each creator in the matching score sequence for x . Consequently, we can reformulate the utility for a creator who is ranked in the i -th position for user x as

Eq.(10) holds because for any resource e = ( x , t ) such that t ∈ [ σ ( s k +1 ; x ) , σ ( s k ; x )] , the load of e is exactly given by k . As a result, by letting f k ( t ) = d t ( k ) , we recover the utility function defined in Eq.(5), where the value of function f i ( t ) at t = t 0 indicates the atomic reward for each creator if his/her strategy covers 'resource' ( x , t 0 ) , given that there are exactly i creators occupy ( x , t 0 ) . This relationship also rationalizes why it is natural to assume that f 1 ≥ · · · ≥ f n : as an increase in competition for the same resource from multiple creators should correspondingly reduce the return that can be accrued from that resource.

8.4 Proof of Theorem 1

Before showing the proof, we define the following notion of local maximizer:

Definition 4. We say s = ( s 1 , · · · , s n ) is a local maximizer of W ( s ) if for any i ∈ [ n ] and any s ′ i ∈ S i ,

The set of all the local maximizers of W is denoted by Loc ( W ) .

According to the definition, for any join strategy profile s ∈ Loc ( W ) , no creator can unilaterally change his/her strategy to increase the value of function W . And clearly we have arg max s ∈S W ( s ) ∈ Loc ( W ) . To simplify notation we define π ( σ 1 , · · · , σ n ) = ∑ n i =1 M ( σ i ; σ -i ) . Now we are ready to present the proof of Theorem 1. To avoid complex notations, with a slight abuse of notation we use M ( σ 1 , σ 2 , · · · , σ n ) to denote M ( σ 1 ; { σ 2 , · · · , σ n } ) in the following proof.

Proof. We start by showing that any TvN game instance with M ∈ M 3 possesses a unique NE at s ∗ = ( e 1 , · · · , e 1 ) . It suffices to show that:

For any joint strategy profile ( s 1 , · · · , s n ) in which there are k < n creators occupy e 1 , there exists a creator who can receive a strict utility gain if she change her strategy to e 1 .

At s ∗ = ( e 1 , · · · , e 1 ) , any player would suffer a utility loss when changing her strategy.

For the first claim, suppose there are k players in s who play e 1 and let i be any player who does not play e 1 . In addition, there are t ≤ n -k players who play the same strategy as s i . By the definition of M 3 , we have

If playeri changes her strategy from s i to s ′ i = e 1 , the new utility would be

̸

From Eq.(11) and Eq.(12), u i ( s ′ i ; s -i ) > u i ( s i ; s -i ) holds if and only if

And a sufficient condition for Eq.(13) to hold is

Denote ˜ π k = π (1 , · · · , 1 ︸ ︷︷ ︸ k , 0 , · · · , 0 ︸ ︷︷ ︸ n -k ) . By the monotonicity of π , we have ˜ π n ≥ ··· ≥ ˜ π 1 =

M (1 , 0 , · · · , 0) > 0 . Therefore, the RHS of Eq.(14) is a finite number. Moreover, when t ≤ k +1 , we have

and when t > k +1 , based on the negative externality principle of merit-based rewarding mechanism we have

Therefore, the RHS of Eq.(14) is strictly less than 2 n -1 .

For the second claim, we have

and if playeri changes her strategy from s ∗ i = e 1 to any s ′ i = e j , j = 1 , her new utility becomes

̸

Therefore, we conclude that s ∗ = ( e 1 , · · · , e 1 ) is the unique NE of G .

̸

Next we estimate the welfare loss of s ∗ under any sequence { r i } K i =1 . First of all, note that for any s = ( s 1 , · · · , s n ) ∈ Loc ( W ) and any 2 ≤ k ≤ n , if there exists i = j such that s i = s j = e k , then there must be k ′ ∈ [ n ] such that e k ′ / ∈ s j . In this case, W strictly increases if s j changes to e k ′ . Therefore, for any 2 ≤ k ≤ n , the number of elements in s that equal to e k is either 0 or 1. Let the number of elements in s that equal to e 1 be q . By definition,

Since q maximizes the RHS of Eq.(15), we have 1 ≤ q ≤ K and ( n +1) r q +1 ≤ r 1 ≤ ( n +1) r q . Therefore,

Since 1 ≤ q ≤ K , we conclude that max s ∈S W ( s ) W ( s ∗ ) > 1 -O ( 1 n ) + 1 K when n is sufficiently large. And therefore we conclude that

8.5 Proof of Proposition 1

Proof. To prove that any M ∈ BRM is merit-based, we need to verify the following by definition:

for any { σ j } n j =1 , { σ ′ j } n j =1 such that σ -i ≼ σ ′ -i , we can transform { σ j } n j =1 to { σ ′ j } n j =1 by taking finite steps of the following operations: 1. increase a certain value of σ j , j = i to ˜ σ j and it does not change the order of the current sequence; 2. increase a certain value of σ j , j = i to ˜ σ j , and σ i 's ranking position decreases after this change. We will show that after each operation the value of M ( σ i , · ) under the perturbed sequence does not increase.

̸

Let the perturbed sequence be ˜ σ . For the first type of operation, if j < i , we have M ( σ i ; ˜ σ -i ) = M ( σ i ; σ -i ) . If j > i , we have

For the second type of operation, with out loss of generality let's assume σ i +1 has increased to ˜ σ i +1 such that σ i ≤ ˜ σ i +1 ≤ σ i -1 . In this case we have

Therefore, M is merit-based. On the other hand, there exist instances in BRM that are not monotone. For example, if we let f 1 ( t ) = 1 and f k ( t ) = 0 , ∀ k ≥ 2 . Then we have

As a result, π (1 , 0 , 0 , · · · , 0) > 0 = π (1 , 1 , 0 , · · · , 0) , which violates monotonicity.

8.6 Proof of Theorem 2

Proof. For the first claim, consider the potential function of the following form:

where σ i ( x ) = σ ( s i ; x ) and { l ( i ) } n i =1 is a permutation such that σ l (1) ( x ) ≥ σ l (2) ( x ) ≥ · · · ≥ σ l ( n ) ( x ) .

By the definition of potential games, we need to verify that for any set of functions { f i } and a strategy pair s i , s ′ i ∈ S i for playeri , it holds that

For any user x ∈ F , let σ i = σ ( s i ; x j ) , σ ′ i = σ ( s ′ i ; x ) , ∀ i ∈ [ n ] . It suffices to show that

̸

Since the expectation of Eq.(17) over x ∈ F yields Eq.(16), we focus on the verification of Eq.(17). With out loss of generality, we also assume σ 1 ≥ · · · ≥ σ i ′ ≥ · · · ≥ σ i ≥ · · · ≥ σ n . After playeri changes her strategy from s i to s ′ i , the relevance ranking increases from i to i ′ , i.e., σ 1 ≥ · · · ≥ σ i ′ -1 ≥ σ ′ i ≥ σ i ′ ≥ · · · ≥ σ n .

Therefore, we have

Hence, Eq.(17) holds for any j which completes the proof.

For the second claim, we can verify that when f i = r i , ∀ i ,

8.7 Proof of Corollary 1

Proof. We show that any TvN game instance G with M = M [ r 1 , · · · , r K , 0 , · · · , 0] ∈ BRCM possesses a unique NE s ∗ which maximizes W ( s ) . From Theorem 2 we know that under M , G is a potential game and its potential function P is identical to its welfare function W . Therefore, any PNE of G belongs to Loc ( W ) . Next we show that all elements in Loc ( W ) yield the same value of W , thus any PNE of G maximizes social welfare W .

̸

First of all, note that for any s = ( s 1 , · · · , s n ) ∈ Loc ( W ) and any 2 ≤ k ≤ n , if there exists i = j such that s i = s j = e k , then there must exist k ′ ∈ [ n ] such that e k ′ / ∈ s j . In this case, W strictly increases if s j changes strategy to e k ′ . Therefore, for any 2 ≤ k ≤ n , the number of elements in s that equal to e k is either 0 or 1. Let the number of elements in s that equal to e 1 be q . By definition, the welfare function writes

It is clear that the q that maximizes Eq.(19) satisfies 1 ≤ q ≤ K and ( n +1) r q +1 ≤ r 1 ≤ ( n +1) r q , and all such q yields the same objective value of W . Therefore, we conclude that any PNE of G attains the optimal social welfare W .

8.8 Content Creator Response Simulator

Algorithm 1 functions as follows: at each step, a random creator i selects a random improvement direction g i . If creator i discovers that adjusting her strategy in this direction yields a higher utility,

Algorithm 1 ( simStra ) Simulate content creators' strategy evolving dynamic

Input: Time horizon T , learning rate η , utility function strategy set ( u i ( s ) , S i ) for each player, current mechanism M [ f ] parameterized by f .

she updates her strategy along g i ; otherwise, she retains her current strategy. This approach is designed to more closely mimic real-world scenarios where content creators may not have full access to their utility functions, but instead have to perceive them as black boxes. While they may aim to optimize their responses to the current incentive mechanism, identifying a new strategy that definitively increases their utilities can be challenging. Therefore, we model their strategy evolution as a trial-and-exploration process. We should note that the specifics of the simulator are not critical to our proposed solution: the optimizer can select any equilibrium-finding dynamic to replace our Algorithm 1, as long as it is believed to better represent creators' responses in reality.

8.9 Optimization Algorithm

Our proposed welfare optimization algorithm is presented in Algorithm 2, which is organized into L 1 epochs. Each epoch starts with a random perturbation of the current mechanism M within the feasible polytope and conducts simulations of creators' responses for L 2 steps using the simulator specified in Algorithm 1. Welfare is reassessed at the conclusion of each epoch, and the perturbation applied to M is adopted if it yields an increase in welfare.

Algorithm 2 Optimize W in BRCM

8.10 Additional Experiments

Experiments using MovieLens-1m We use deep matrix factorization [8] to train user and movie embeddings predicting movie ratings from 1 to 5 and use them to construct the user population X and creators' strategy set {S i } . The dataset contains 6040 users and 3883 movies in total, and the embedding dimension is set to d = 32 . To validate the quality of the trained representation, we first performed a 5-fold cross-validation and obtain an averaged RMSE = 0 . 739 on the test sets, then train the user/item embeddings with the complete dataset.

To construct a more challenging environment for creators, we avoid using movies that are excessively popular and highly rated or users who are overly active and give high ratings to most movies. This

0

100

200

300

400

500

time step

3300

3350

3400

3450

3500

Welfare

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

(a) Welfare on

G

1

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

User Util.

0

20

40

60

80

100

120

140

160

M

3 (0)

M

3 (expo.)

BRCMopt

(b) User Util. Distr. on

G

1

0

100

200

300

400

500

time step

3200

3300

3400

3500

3600

Welfare

BRCMopt

BRCM

*

BRCM

1

M

3 (0)

M

3 (expo.)

M

3 (enga.)

(c) Welfare on

G

2

0.8

1.0

1.2

1.4

1.6

1.8

2.0

User Util.

0

20

40

60

80

100

120

140

M

3 (0)

M

3 (expo.)

BRCMopt

(d) User Util. Distr. on

G

2

ensures that the strategy of 'producing popular content for the majority of active users' does not become a dominant strategy under any rewarding mechanism. Thus, we filtered out users and movies who have more than 500 predicted ratings higher than 4. After the filtering, we have a user population with size |X| = 2550 and movie set of size 1783 . The user distribution are set to uniform distribution over X , and the remaining movies become the action set {S i } for each creatori of n = 10 creators. To normalize the relevance score to [0 , 1] , we set σ ( s , x ) = clip ( ⟨ s , x ⟩ / 2 . 5 -1 , 0 , 1) . { r i } n i =1 is set to { 1 log 2 (2) , 1 log 2 (3) , 1 log 2 (4) , 1 log 2 (5) , 1 log 2 (6) , 0 , · · · , 0 } . We also consider two types of game instances, namely G 1 and G 2 , as we elaborated on in Section 6.1. Specifically, in G 1 creators' initial strategies are set to the most popular movie among all users (i.e., the movie that enjoys the highest average rating among X ) and the cost functions are set to be zero. In G 2 , we set creators' cost functions to c i = 10 ∥ s i -¯ s i ∥ 2 2 and let creator i start at the cost center ¯ s i . { ¯ s i } n i =1 are sampled at random from all the movies.

For each baseline in Section 6.2, we plot the welfare curve over T = 500 steps using Algorithm 1 and also the average user utility distribution at the end of simulations. The parameters of Algorithm 2 are set to L 1 = 100 , L 2 = 5 , η 1 = 0 . 5 , η 2 = 0 . 1 , f (0) = (1 , 1 , 1 , 1 , 1 , 0 , · · · , 0) . The results are presented in Figure 3.

The new results obtained reinforce the findings presented in Section 6. In both the G 1 and G 2 environments, the BRCM family continues to outperform M 3 overall. Specifically, BRCM opt , BRCM 1 , and BRCM ∗ consistently demonstrate strong performance in social welfare, highlighting the robustness of BRCM across different environments. When creators initially adopt the most popular strategy in G 1 , M 3 (0) does not yield any improvement since no creator would change their strategy in such a situation under M 3 (0) . In the case of G 2 , the advantage of BRCM over M 3 diminishes slightly, which aligns with our observations from the synthetic dataset. The main reason is that the cost function discourages creators to deviate from their default strategies. Additionally, Figure 3b provides further evidence that the welfare gain achieved by BRCM arises from enhanced utility for a wider range of users.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe development of online media referral platforms has provided a source of income for media content creators, and the incentive strategies of the platforms may influence the creators' creative trends. The incentive model that tends to reward may also invariably encourage creators to over-serve the majority user group, and the niche groups will be increasingly underserved. In order to solve these problems, this paper designs a backward incentive mechanism, which induces creators' behavior through the game structure to dynamically optimize the creation model and maximize social welfare. And its advantages are verified by simulation experiments.\n\nSTRENGTHS:\nTo address the current problem that rewards mechanisms in online content recommendation platforms affect creators' production choices, the platform's content distribution and social welfare. In this paper, we design a reverse reward mechanism, which can guide content creators to optimize their creation strategies and provide locally optimal results for a given welfare metric. This avoids most creators to generate a large amount of homogeneous content that caters to the majority group for the sake of rewards.\n1. originality\nIn this paper, we provide a reverse reward mechanism to address the problem that current reward mechanisms may encourage creators to concentrate on the mass range, thus leaving niche users unserved. The superiority of this mechanism, which is performance-based but discards monotonicity, has been proven in empirical studies, and the approach in this paper is well original.\n2. quality\nThe research problem of this paper is apparent, the method is introduced in detail, the logic is clear and the experiment is reliable. This paper has good quality.\n3. clarity \nThis paper clearly defines the research problem, and introduces the effectiveness of the method from the theoretical and simulation experiments in detail. The overall clarity is good.\n4. significance\nIn this paper, it designs a reverse incentive strategy to cope with the incentive drift caused by the profit-oriented incentive strategy in online content recommendation platforms, so as to reduce the undesirable incentives that cause a large number of creators to ignore the niche groups and leave them unserved. The research in this paper has significant implications.\n\nWEAKNESSES:\nThis paper’s related work on the content of the current study is weak, there are few references, and there are only a few works in the past five years, which is a bit difficult to explain the novelty of the current method.\n\nQUESTIONS:\nThis paper designs an incentive mechanism for creators in online content recommendation platforms, and its reverse incentive mechanism establishes a gaming competition environment among content creators to maximize social welfare. However, this paper has the following problems:\n1. the limitations of this paper are not declared;\n2. whether the order of content in sections 3 and 4 can be switched, which seems to be a bit unreasonable at present;\n3. in the experimental part, the data used in this paper and some experimental details are not clear descriptions. For example, synthetic data are used in this paper, but the details of the synthetic data are not shown, and for the results in Figure 1, it is not clear from the content whether synthetic data or MovieLens-1m dataset is used;\n4. some references are too old and could be replaced with newer references, research from the last five years could be added appropriately, and the paper is not standardized in its citation of references;\n\nLIMITATIONS:\nThis paper seems to have no clear statement on the limitations of the research. I have a question as to whether the creators in the current study only considered those who create for profit, and whether they considered those who create for interest. The current study seems to consider creators who create for profit, but it does not seem to be stated in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study strategic content creation in recommendation systems, focusing on the induced game's social welfare. The authors assume that the provider's rewards are entirely determined by the platform's payments, not clicks/engagements. This separation between the ranked results and the creators' incentives facilitates analyzing an expressive game in which the recommender system recommends a list of items (most prior work considers one item) with position bias. Notably, the authors assume that the rewarding mechanism is a mapping from a vector of relevance scores to reward vectors, i.e., $[0,1]^n \\rightarrow [0,1]^n$.\n\nThe paper considers two classes of merit-based mechanisms, monotone and BRM (and also the BRCM subclass). It showcases evidence against monotone mechanisms, highlighting that in a particular class of games (TvN), the POA of any monotone mechanism is at most $\\frac{K}{K+1}$, where $K$ is the length of the list, plus a small $\\frac{1}{n}$ factor. Later, for BRCM mechanisms, the authors show that the welfare function is the potential function; thus, the global optimum of the welfare is a PNE (despite the POA could still be  $<1$ for some mechanisms). \n\nFinally, the authors describe how to optimize over BRCM mechanisms in the presence of data, and conduct synthetic and semi-synthetic experiments to demonstrate their approach.\n\nSTRENGTHS:\n1.\tThe paper deals with a timely and important topic, and well-connects to previous literature.\n2.\tThe paper non-trivially extends previous literature, suggesting new theory and experimental validation.   \n3.\tThe optimization problem the paper suggests is exciting and new to this literature.\n\nWEAKNESSES:\n1.\tDue to the abundance of mathematical objects and notations, the paper is non-trivial to follow. Perhaps this is inevitable, but I see this as a weakness. \n2.\tThe case against monotonicity focuses on a relatively small class of TvN games tailored to the authors' argument. Arguing against monotonicity in (more) general games would be much more convincing. \n3.\tThe empirical evaluation lacks proper benchmarks.\n\nQUESTIONS:\n1.\tAn assumption of the model is that impressions/clicks of users and creators' rewards are separated and orthogonal. In reality, the utility function of creators is not always \"set\" by the platform (as argued in 168,170), but rather determined by the interaction the platform allows. For instance, in Medium, blog traffic means more ads, regardless of relevance. Can the platform decide that a blog is irrelevant and thus, despite its heavy traffic, not share the ad revenue with the blog's creator? As written in my summary, I see this separation as the element facilitating a much more expressive game than those considered in prior work. So my question is: Can the authors justify this modeling assumption?\n2.\tRelating to the previous point, something disturbs me in the explanation in Lines 241-250. Say the $K$ contents creators attract impressions, clicks, etc., and get high rewards. Then, the $K+1$ creators improves their content's relevance, and suddenly the $K$ selected creators suddenly get almost none. I think that this is hard to digest and probably will be deemed unfair from the creators' perspective. Could the authors justify why BRCM mechanisms benefit creators? After all, there are other unfair mechanisms that dictate the content each creator creates, which receives optimal welfare.\n3.\tThe paper advocates using BRCM mechanisms and not monotone ones. Noticeably, the intersection between these classes is not empty. But Theorem 1 suggests that the optimal BRCM mechanism that exists due to Corollary 1 is *not* monotone. This brings the question of how natural this mechanism is (the one guaranteed by Corollary 1). Do we want to sacrifice simplicity (and arguably fairness) for a proportion of $\\frac{1}{K+1} of the welfare? Since advocating for BRCM is not equivalent to advocating against monotonicity (as the intersection is non-empty), I think the reader could benefit from a more elaborated discussion. Ideally, this paper would have presented evidence beyond social welfare in TvN games, but perhaps this is saved for future work.\n4.\tThe welfare function comprises the user welfare, the creators rewards and costs, and the mechanism's payments. But the mechanism's payments are precisely the creator rewards, so they cancel. Consequently, by offering unbounded rewards the platform does not change the welfare, but obviously this will never happen in real life. What part of the modeling forbids that? This seems like a loophole in the modeling. Could the authors justify this welfare modeling? Even if we care about the welfare function defined in this paper, one would expect the platform to minimize its payments for any given welfare level. Could this be injected into the model?\n5.\tIn the experimental part, the authors compare BRCM mechanisms with monotone mechanisms. However, for the monotone mechanisms, the authors do not optimize over the hyperparameter $\\beta$. I believe an apples-to-apples comparison requires optimizing for both classes of mechanisms. Why didn't the authors do that in their analysis? Additionally, what would be other monotone baselines? \n6.\tIn Figure 1(a), the optimized BRCM scores better than the optimal BRCM. How could that be possible?\n\nMinor: \n•\t23: “benefits\\cite{}” -> add space\n\n•\tLines 139-140 say \"Our objective is to design mechanisms M that: 1. guarantee the existence of PNE, thereby ensuring a stable outcome, and 2. maximize social welfare at the PNE.\" While Line 135 says that this is not the focus of [1,12,13], I think that [1] primarily addresses the same objectives. \n\n•\t155: the parameter $n$ seems to be out of context, as it doesn't appear in the $M^3$ notation.\n\n•\t210: \"..\" -> \".\"\n\n•\t254: $BRCM \\in BRM$ -> $BRCM \\subset BRM$\n\nLIMITATIONS:\nNo limitations", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
This paper studies the incentives in recommendation systems. Specifically, it studies how to design the platform's reward mechanism to steer the creators' competition towards a desirable welfare outcome. Firstly, it shows a class of mechanisms called "Merit-based Monotone Mechanisms" lead to a constant fraction loss of the welfare. To overcome this loss, it introduces Backward Rewarding Mechanisms (BRMs) and shows that the competition games resulting from BRM induce the strategic creators’ behavior dynamics to optimize any given welfare metric.

STRENGTHS:
The paper studies an interesting question in recommendation system. It shows an interesting bad effect of a wide class of mechanisms towards social welfare and then designs another mechanism to overcome the negative result. The theoretical results within the scope of the paper are complete. And there are also empirical experiments.

WEAKNESSES:
The applied value of the model in the paper is lack of justification.

QUESTIONS:
Can you explain the applied value of the model in this paper?

LIMITATIONS:
The applied value of the model in the paper is lack of justification.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the game played by content creators in recommendation systems, which they call the content creator competition game. This game is centrally defined by a rewarding function M, decided by the platform, which rewards content creators based on how users engage with their content. The paper focuses on how to design this rewarding function in order to maximize user welfare. \n\nThey show first that a practically-motivated class of rewarding functions, “Merit-based monotone mechanisms” (M3), lead to losses in user welfare by producing an equilibrium that caters to majority-group users, and fail to cater to minority-group users. Notably, the “necessary welfare loss” is only slightly suboptimal: such mechanisms can still capture a $K/(K+1)$ factor of the optimum, where $K$ is the parameter defining the top-K recommendation policy (though they do make the point that when $K$ is effectively 1 for users who care only about the top recommendation, this ratio can be 1/2). Also notably, they prove this result in a sub-class of creator competition games with the structure of a majority group and several minority groups, all with orthogonal interests (called TvN games - “trend versus niche”).\n\nNext, they introduce a class of rewarding functions called “backward rewarding mechanisms”, which keeps the merit-based property of M3 but drops the monotonicity assumption, trading it for a a set of functions f1,…fn specified by the platform that can be tuned to encourage diversity by making it costly for too many creators to be producing the same kind of content. They show that for TvN games, there exists a backward rewarding mechanism that admits the optimal welfare. They run some simulated experiments with user preferences that constitute a TvN game.\n\nSTRENGTHS:\n- The paper aims to be general: they take almost an axiomatic approach and study an entire class of mechanisms (M3) defined by two main assumptions, which encompass multiple practical mechanisms. They also study an entire class of instances (TvN games).\n\n- They show clean equilibrium results for both classes of mechanisms\n\n- The new class of mechanisms they propose is conceptually interesting — it makes clear why the monotonicity assumption causes problems, and offers a tunable class of algorithms that can help improve user welfare, at least in theory (it remains to be seen whether there is adequate information available in practice to set the parameters of these mechanisms well).\n\n- The paper takes care to make abstract concepts understandable, giving examples and intuition to supplement the math\n\nWEAKNESSES:\n1. I do not understand the “monotonicity” property conceptually (described on Lines 43-44 as “the sum of creators’ utilities increase whenever any creator increases her content relevance”). I may be misunderstanding something here, but I interpret this to mean “when one creator benefits, all creators benefit on average”. This doesn’t seem to necessarily reflect an environment in which content creators are competing: Under competition, it could be the case that when a given creator improves her content, it greatly increases her own utility but decreases all other creators’ utilities more in total?\n\n2. Although the paper makes some attempts to justify why the K/(K+1) loss of M3 mechanisms is bad, this doesn’t seem that bad to me, especially given that it emerges from a purely theoretical model in which many abstractions have been made. I felt that the paper oversold the magnitude of this loss in multiple places. It’s also not clear to me that the proposed fixed (BRM) doesn’t have similar loss in other natural sub-classes of games outside TvN games (as their positive results applies only to TvN games). To me, the combination of these two points weakened the motivation for the results sections later in the paper.\n\nSmall points about clarity:\n- Line 34: what is a “reward signal”?\n- Line 41: “and frame a class of prevailing rewarding mechanisms… Merit-based Monotone Mechanism”. It’s not clear what “prevailing” means here.\n- In the explanation Lines 50-53, I’m missing a logical step: is “relevance quality” evaluated in terms of the *number* of users who find it relevant? Otherwise, I don’t see how this monotonicity property could cause concentration of creators around majority users’ preferences.\n\nQUESTIONS:\nQuestions are encompassed in \"weaknesses\" section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-18"}