Supply-Side Equilibria in Recommender Systems
Authors: Meena Jagadeesan
Nikhil Garg
Jacob Steinhardt
Venue: NeurIPS 2023 poster (2023)
Keywords: content creator incentives
Nash equilibria
specialization
economic aspects of recommender systems
TL;DR: We propose a game-theoretic model for competition between content producers in recommender systems, and we investigate the potential for specialization.

Abstract: Algorithmic recommender systems such as Spotify and Netflix affect not only consumer behavior but also *producer incentives*. Producers seek to create content that will be shown by the recommendation algorithm, which can impact both the diversity and quality of their content. In this work, we investigate the resulting supply-side equilibria in personalized content recommender systems. We model the decisions of producers as choosing *multi-dimensional* content vectors and users as having *heterogenous* preferences, which contrasts with classical low-dimensional models. Multi-dimensionality and heterogeneity creates the potential for *specialization*, where different producers create different types of content at equilibrium. Using a duality argument, we derive necessary and sufficient conditions for whether specialization occurs. Then, we characterize the distribution of content at equilibrium in concrete settings with two populations of users. Lastly, we show that specialization can enable producers to achieve *positive profit at equilibrium*, which means that specialization can reduce the competitiveness of the marketplace. At a conceptual level, our analysis of supply-side competition takes a step towards elucidating how personalized recommendations shape the marketplace of digital goods.

Supply-Side Equilibria in Recommender Systems
Meena Jagadeesan
Nikhil Garg
UC Berkeley
Cornell Tech
mjagadessanGberkkeeee..ee
ngarg@cornell.edd
Jacob Steinhardt
UC Berkeley
jsteinhardttbbrrklleey edu
Abstract
Algorithmic recommender systems such as Spotify and Netflix affect not only consumer behavior but also producer incentives. Producers seek to create content that will be shown by the recommendation algorithm, which can impact both the diversity and quality of their content. In this work, investigate the resulting supply-side equilibria personalized content recommender systems. We model the decisions of producers as choosing mulii-dimensional content vectors and users as having heterogenous preferences, which contrasts with classical low- dimensional models. Multi-diimessoonality and heterogeneity creates the potential for specialization, where different producers create different types content equilibrium. Using duality argument, derive necessary and sufficient conditions for whether specialization occurs. Then, we characterize the distribution of content equilibrium in concrete settings with two populations of users. Lastly, we show that specialization can enable producers to achieve positive profit at equilibrium, which means that specialization marketplace. At a conceptual level, our analysis of supply-side competition takes a step towards elucidating how personalized recommendations shape the marketplace of digital goods.
reduce the competitiveness of the
Introduction
Algorithmic recommender systems have disrupted the production of digital goods such movies, music, and news. In the music industry, artists have changed the length and structure of songs in response Spotify's algorithm and payment structure [Hodgson, 2021]. In the movie industry, personalization has led to low-budget films catering to specific audiences [McDonald, 2019], in some cases constructing data-driven "taste communities" [Adalian, 2018]. Across industries. recommender systems shape how producers decide what content to create, influencing the supply side of the digital goods market. This raises the questions: What factors drive and influence the supply-side marketplace? What content will be produced at equilibrium?
Intuitively, supply-side effects induced by the multi-sided interaction between producers, the recommendation algorithm, and users. Users tend to follow recommendations when deciding what content to consume [Ursu, digital good and impact the profit (or utility) generated by each content producer. As a result, content producers shape their content to maximize appearance in recommendations: this creates competition between the producers, which can be modeled as game. However, understanding such producer-side effects has been difficult, both empirically and theoretically. This a pressing problem, as these gaps in understanding have hindered the regulation of digital marketplaces [Stigler Committee, 2019].
recommendations inffuence how many users consume each
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

155
Support equilibrium
$a10
15
support of equilibrium
2=
15
Support equilieriu
25
15
support of equilibrium
M=10
1.0
cumulative distribution In
1.0
1.0
prbabiity
8.
profit
proft -
proft
CA
2
25
0.0
0.0
0.0
0.0
0.5
1.0
1.5
0.0
0.5
1.0
1.5
0.0
0.5
1.0
155
0.0
0.5
1.0
1.5
0.0
0.5
1.0
1.5
2.0
101
Figure 1: symmetric equilibrium for different settings [3, for users located at the standard basis vectors ex and ez, P = 2 producers, and producer cost function c(p) ||p|12. The first plots show the support of the equilibrium μa. As / increases, there is a phase transition from single-genre equilibrium to equilibrium with infinitely many genres (Theorem 2). The profit also transitions from zero to positive, demonstrating how specialization reduces the competitiveness of the marketplace (Propositions 2-3). The last plot shows the cumulative distribution function of |pl..
At a high level. there are two primary challenges that complicate theoretical analyses of these supply- side effects. (1) Digital goods such as movies have many attributes and thus must be embedded in multi-dimensional continuous space, leading to large producer action space. This multi- dimensionality is a departure from traditional economic models of price and spatial competition. (2) A core aspect of such marketplaces is the potential for specialization: that is, different producers may produce different items at equilibrium. Incentives to specialize depend on the level of heterogeneity of user preferences and the cost structure for producing goods (whether it is more expensive to produce items that are good in multiple dimensions). As a result, supply-side equilibria have the potential to exhibit rich economic phenomena, but pose a challenge to both modeling and analysis.
We introduce a simple game-theoretic model for supply-side competition in personalized recom- mender systems, that captures the mimii-dimmesiooall space of producer decisions, rich structures of production costs, and general configurations of users. Users and digital goods are represented as D-dimensional vectors, and the value that user with vector derives from digital good p is equal to the inner product (u. (). The platform offers personalized recommendations to each user, showing them the digital good with maximum value them. Each producer chooses single digital good E 2 to create. The goal of a producer to maximize their profit, which is equal to the number of users who recommended their content minus the (one-time) cost of producing the content. We study the Nash equilibria of this game between producers. We defer model specifics to Section 2.
To ground our investigation of the complex producer choices, focus phenomena- the potential for specialization. Specialization, which occurs when different producers create different genres (i.c., directions) of content at equilibrium, has several economic consequences. For example, whether specialization occurs, as well as the form that specialization takes, determines the diversity of content available the platform. Moreover, specialization influences market competitiveness by reducing the amount of competition in each genre. This raises the questions:
particular economic
Under what conditions does specialization occur at equilibrium? What is its impact on market competitiveness? What form does specialization take?
We investigate specialization and its consequences the supply-side market. We analyze how the specialization exhibited at equilibrium varies with user geometry and producer cost function parame- ters. Figure and 3 depict the equilibria in our model across several concrete instantations. Our main results provide insight into each of the questions from above: we characterize when specialization occurs (Section 3) and investigate the impact of specialization We provide an empirical analysis of supply-side equilibria using the MovieLens- 100K dataset [Harper and Konstan, 2015] (Figure 4), and we analyze the particular form that specialization takes in the case of 2 equally sized populations (deferred to Appendix D).
market competitiveness (Section 4).
Our simple model yields a nuanced picture of supply-side equilibria in recommender systems. Our results provide insight into specialization and its implications, and en route to proving these results, we develop technical toolkit to analyzing the multi-dimennional behavior of producers (see Appendix C.1 for a summary). More broadly, our model and results open the door to investigating how recommender systems shape the diversity and quality of content created by producers.

---

1.1 Related work
Our work related to research on societal effects recommender systems, models of competition in economics, and strategic effects induced by algorithmic decisions.
Supply-side effects of recommender systems. A line of work in the machine learning literature has studied supply-side effects from a theoretical perspective, but existing models do not capture the infinite, multi-dimensional decision space of producers. Ben-Porat and Tennenholtz [201 8] study supply-side effects with a focus on mitigating strategic effects by content producers; Sen-Porat et al. [2020]. building on Basat et al. [2017], also studied supply-side equilibria with a focus on convergence of learning dynamics for producers. The main difference from our work is that producers in these models choose topic from finite set of options; in contrast, our model captures the infinite, multi-dimensional producer decision space that drives the emergence of genres. Moreover, we focus on the structure of equilibria rather than the convergence of learning.
In concurrent and independent work, Hron et al. [2022] study a related model for supply-side competition in recommender systems where producers choose content embeddings in DR One main difference is that, rather than having cost on producer content, they constrain producer vectors to the & unit ball (this corresponds to our model when 3 + oc and the norm is the fg-norm, although the limit behaves differently than finite 3). Additionally, Hron al. incorporate a softmax decision rule to capture exploration and user non-determinism. whereas we focus entirely on hardmax recommendations. Thus, our model focuses on the role of producer costs while Hron et al.'s focuses on the role of the recommender environment. Technically, Hron et al. study the existence of different types of equilibria and the use of behaviour models for auditing, whereas we analyze the economic phenomena exhibited by symmetric mixed strategy Nash equilibria, with focus on specialization.
Other work has studied the emergence of filter bubbles [Flaxman et al., 2016], the ability of users to reach different content [Dean et al., 2020]. the shaping of user preferences [Adomavicius et al., 2013], and stereotyping [Guo et al., 2021].
Models of competition in microcconomies. Our model and research questions relate to classical models of competition in economics; however, particular aspects of recommender systems- trsgh-s-high- dimensionality of digital goods, rich structure of producer costs, and user geometry are not captured by these classical models. For example, in price competition, producers set a price, but do not decide what good produce (e.g. Bertrand competition, see [Baye and Kovenock, 2008] for texthook treatment). Price is one-dimensional, but producer decisions in our model are multi-dimensional.
Another line of work on product selection has investigated how producers choose goods (i.e., content) at equilibrium (see Anderson et al. [1992] for a textbook treatment). For example, in spatial location models, producers choose a direction in a low-dimensional space (e.g.. R in [Hotelling, 1929, ''Aspremont et al., 1979] and St in [Salop, 1979]), and users typically receive utility based on the negative of the Euclidean distance. In contrast, producers in our model jointly select direction and magnitude, and users receive utility based on inner product. Since some spatial location models allow producers to set prices, it may be tempting to draw an analogy between [|||] in our model and the price in these models. However, this analogy breaks down because costs can be nonlinear in ||p||-
Other related work has investigated producer chooses a function from quantity to prices, rather than what content to produce, and the pure characteristics model (e.g. [Berry, 1994]), where attributes of users and producers are also embedded in DR like in our model, but which focuses on demand estimation for a fixed set of content, rather than analyzing the content that arises at equilibrium in the marketplace. Recent work in economics has extended Berry [1994] to allow for endogenous product choice (e.g. [Wollmann. 2018]) and also studied specialization (e.g. [Vogel, 2008, Perego and Yuksel, 2022]). though with different modeling assumptions than our work.
function equilibria (e.g. [Grossman, 19. where the
Strategic classification. et al., 2016] has studied how classification can induce participants to strategically change their features to improve their outcomes. Models for participant behavior (e.g. Kleinberg and Raghavan [2019], Jagadeesan et al. [2021], Ghalme et al. [2021]) generally do not capture competition between participants. One exception is Liu et al. [2022], where participants compete to appear higher in a ranking; in contrast, the participants in our model compete for users with heterogeneons preferences.
line of work of strategic classification [Brückner al., 2012, Hardt
3

---

2 Model and Preliminaries
We introduce game- theoretic model for supply-side competition in recommender systems. Consider a platform with N 2 heterogeneous users who are offered personalized recommendations and P 2 2 producers who strategically decide what digital good to create.
Embeddings of users and digital goods. Each user is associated with a D-dimensional embed- ding u, that captures their preferences. We assume that u, E Ro {oj-ie.. ie.. the coordinates of each embedding nonnegative and each embedding is nonzero. While user vectors are fixed, producers choose what content to create, Each producer j creates a single digital good, which is associated with content vector P5 E R Z0 The value of good p to user is (u.p).
Personalized recommendations. After the producers decide what content to create, the platform offers personalized recommendations each user. We consider a stylized model where the platform has complete knowledge of the user and content vectors. The platform recommends to each user the content (and thus producer) of maximal value them. Mathematically, the platform assigns a user to the producer j", where j" (w; PI:P) arg max, j (u; P1.P) to be producer chosen uniformly at random from the argmax.
I م P (u, Py)- If there are ties, the platform sets
Producer cost function. Each producer faces a fixed (one-time) cost for producing content p, which depends on the magnitude of Since the good is digital and thus cheap to replicate, the production cost does not scale with the number of users. We assume that the cost function c(p) takes the form l|el| where || || is any norm and the exponent is least 1. The magnitude ||p|| captures the quality of the content: particular, if a producer chooses content Ap, they win at least as many users as if they choose X'p for X < X. (This relies the fact that all vectors
in the positive orthant.)
The norm and / together encode the cost of producing content vector and reflect cost tradcoff's for excelling in different dimensions (for example, producing movie that is both drama and comedy). Large 3, for instance, means that this cost grows superlinearly. In Section 3, we will see that these tradeoffs capture the extent which producers are incentivized specialize.
Producer profit. A producer receives profit equal to the number of users who are recommended their content minus the cost of producing the content. The profit of producer j is equal to:
P(p;:P-5) = E. Σ [j PI.P) = 51 - |1p,11 |1,.
where p-j = (P.....>
.ac....-Ppjlll Pj- 1. Pj+l...1........................ PP] denotes the content produced by all of the other producers; the expectation is over the randomness over platform recommendations in the case of ties.
We defer further discussion of our model to Appendix B.I.
2.1 Equilibrium concept and existence of equilibrium
We study the Nash equilibria of the game between producers. Each producer chooses a (random) strategy over content, given by a probability measure Hıy over the content embedding space RD Recall that strategies - form a Nash equilibrium if no producer-given the strategies of other producers-can can chose a different strategy where they achieve higher expected profit.'
A salient feature of our model is that there are discontinuities in the producer utility function in equation (1), since the function arg max (u, p.) changes discontinuously with the producer vectors py. Due to these discontinuities, pure strategy equilibria do not exist (Proposition 4 in Ap- pendix B.2). Nonetheless, we show that a symmetric mixed strategy equilibrium (i.e. an equilibrium where #1
(PP) is nonetheless guaranteed to exist.
Proposition 1. For any set of users and any 3 Z 1, symmetric mixed equilibrium exists.
These equilibria are interestingly atomless (Proposition 5 in Appendix B.2), so there is randomization over quality ||||| as well as randomness over genres p/ [|p|]. We defer an example of the equilibria
'Tha is, P. E arg max 2 E---~w_, [P(p::p-)]] for every j E [P] and every pj E supp(u;)-

---

in our model for homogenous users to Appendix B.3. We take the symmetric mixed equilibria of this game as the main object of our study, since they are both tractable to analyze and rich enough to capture asymmetric solution concepts such as specialization.
2.2 Specialization and the formation of genres
When users are heterogeneous, there are inherent tensions between catering to user and catering to other users. As a result, the producer make nontrivial choices not only about the quality (vector norm (|pl) of their content, but also the genre (direction p/ p/|lpl)) of their content. This can lead to specialization, which is when different producers create goods tailored to different users; alternatively, all producers might still produce the same genre of content equilibrium and thus only exhibit differentiation on the axis of quality.
We define genres the set of directions that arise symmetric mixed Nash equilibrium pr:
Genre(u) == {
p [pll
p supp(u))
where we normalize by |ppl to separate out the quality (norm) from the genre (direction). The set of genres Genre(u) captures the set of content that may arise on the platform in some realization of randomness of the producers' strategies. When an equilibrium has a single genre, all producers cater an average user, and only a single type of content appears. On the other hand, when equilibrium has multiple genres, many types of digital content are likely to appear on the platform.
We thus say that specialization occurs at equilibrium μ if and only if the support of μ has more than one genre (direction). The particular form of specialization exhibited by μ is further captured by the number and set of genres in the support of j.. See Figure for depiction of markets with single- and multi-genre equilibria, respectively.
When does specialization occur?
In order to investigate whether specialization occurs in given marketplace, distinguish between two regimes of marketplaces based on whether or not a single-gente equilibrium exists:
1. A marketplace is in the single-gens regime if there exists an equilibrium μ such that [Genre(p)] = All producers thus create content the same genre.
2. A marketplace is in the multi-genre regime if all equilibria μ satisfy [Genre(u) >> 1. Producers thus necessarily differentiate in the genre of content that they produce.
To understand these regimes. we ask: what conditions on the user vectors U un and the cost function parameters || and 3 determine which regime the marketplace is in?
In Section 3.1, we give necessary and sufficient conditions for all equilibria to have multiple genres (Theorem 1). In Section 3.2, we show several corollaries of Theorem In Section 3.3, we show that the location of the single-genre equilibrium (in cases where it exists) maximizes the Nash social welfare. Section 3.4, we provide an empirical analysis using the MovieLens- 100K dataset [Harper and Konstan, 2015] that validate our theoretical results. We defer proofs of these results to Appendix F.
3.1 Characterization of single-genre and multi-genre regimes
We first provide a tight geometric characterization of when a marketplace is in the single-genre regime versus in the multi-genre regime. More formally, let U [urs... ;ux] be the N x D matrix of user vectors, and let S denote the image of the unit ball under U:
: (Up ||e|| < 1, E
(3)
Each element of S is N-dimensional vector, which represents the user values for some unit- norm producer Additionally, let S be the image of S under coordinate-wise powers, i.e.
IIn this definition, we implicitly interpret the randomness in the producer strategies as differentiation between producers. This formalization of specialization obviates the need to reason about asymmetric equilibria, thus making the model much more tractable analyze.
5

---

8-1.0
8-2.0
8-4.0
8-8.0
1.0
Sa
0.8
0.2
1.0
0.8
0.6
0.4
0
0.25
0.50
0.75
2.30
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
y1
yn
y1
y1
Figure 2: The sets S and S for two configurations of user vectors (rows) and settings of 3 (columns). The user vectors [), [0, 1J (top, same as Figure and 0], [0.5, 0.87] (bottom). 8 transitions from convex to non-convex as / increases, though the transition point depends on the user vectors. When S convex, single vector p can more easily satisfy both users at low cost.
(21,-.., NN) E S then (=f,..., # E S We show that genres emerge when S is sufficiently different from its convex hull 8
Theorem 1. Let Let == [ui:... ; uN], let S be (Up |Iell < 1,p E ER and let S be the image of under coordinate-wise powers. Then, there is symmetric equilibrium with [Genre(μ) = 1 if and only if
N
II
max II VESA 1=1
max WESS
9:
y4-
Otherwise, all symmetric equilibria have multiple genres. Moreover, if (4) holds for some B, it also holds for every 3' B.
Theorem I relates the existence of single-genre equilibrium to the convexity of the set $ As a special case, the condition in Theorem always holds if S is convex, but strictly speaking weaker than convexity. Interestingly, the condition depends on the geometry of the user embeddings U1...., th and the cost function c(p) = !| but not on the number of producers P. Intuitively, convexity of S relates to the ease with which vector p can satisfy all users simultaneously, at low cost-each dimension of S corresponds to user's utility. In Figure 2, we display the sets S and $ for different configurations user vectors and different settings of
Theorem further shows that the boundary between the single-genre and multi-genre regimes can be represented by a threshold defined as follows
N
N
8 = sup B21
max yESE
y:
MAX
y1
where single-genre equilibria exist exactly when 3 < f". Conceptually, larger 3 make producer costs more superlinear, which eventually discentivizes producers from attempting to perform well on all dimensions at once.
The proof of Theorem draws connection minimax theory in optimization. In particular, show that the existence of a single-genre equilibrium is equivalent to strong duality holding for certain optimization program that we define (Lemma 2). This allows to leverage techniques from optimization theory to analyze when specialization occurs. We defer a full proof to Appendix F.I.
3.2 Corollaries of Theorem
To further understand the condition in equation (4), we consider a series of special cases that provide intuition for when single-genre equilibria exist (proofs deferred to Section F.2). First, let consider

---

3 ==1, 1, in which case the cost function is a norm. Then S' = S is convex, so single-genre equilibrium always exists.
Corollary 1. The threshold 3" is always at least 1. That is, if = 1, there exists a single genre equilibrium.
The economic intuition is that norms incentivize averaging rather than specialization.
We next take a closer look at how the choice of norm affects the emergence of genres. For cost functions c(p) llpllg we show that 3 2 q for any set of user vectors, with equality achieved at the standard basis vectors.
Corollary 2. Let the cost function be c(p) = llrlg- For any set user vectors, it holds that 3 2 g. If the user vectors are equal to the standard basis vectors (ei,...,eD) then 3 is equal to q.
Corollary 2 illustrates that the threshold 3 relates closely to the convexity of the cost function and whether the cost function is superlinear. In particular, the cost function must be sufficiently nonconvex for all equilibria to be multi-genre, For example, for the -ormm, where producers only pay for the highest magnitude coordinate, it is never possible to incentivize specialization: there exists single-genre equilibrium regardless of B. On the other hand, for norms where costs aggregate nontrivially across dimensions, specialization is possible.
In addition to the choice of norm, the geometry of the user vectors also influences whether multiple genres emerge. To illustrate this, we first show that in concrete market instance with 2 equally sized populations of users, the threshold depends on the cosine similarity between the two user vectors: Corollary 3. Suppose that there are N users split equally between two linearly independently vectors u1, U2 E Roo and let 6* == cos- coss- ([iitaalaa) ) If the cost function is c(p) = -Ipll then, (
2
1 - c0*
Corollary 3 demonstrates the threshold 3 increases as the angle 0 between the users decreases (i.e. the users become closer), because it is easier to simultaneously cater all users. In particular, 3r interpolates from 2 when the users are orthogonal to oo when the users point in the same direction.
Finally, we consider general configurations of users and cost functions, and we upper bound 3": Corollary 4. Let || ||+ denote the dual norm of ||. defined be |p||= [pll..EE (q. p).
Let Z == Σ n= I.. |l Then,
log(N)
3 log(N) log(Z) (Z)
(5)
In equation (5), the upper bound on the threshold 3* increases as Z increases. As an example, consider the cost function e(p) = -1rll We see that if the user vectors point in the same direction, then Z . and the right-hand side of (5) is co. On the other hand, if u1,..., ut are orthogonal, then Z V and the right-side of (5) is 2, which exactly matches the bound in Corollary In fact, for random vectors u1,-.., uy drawn from truncated gaussian distribution, we see that Z = Ō(VN) in expectation, in which case the right-hand side of (5) is close to as long as N is large. Thus, for many (but not all) choices of user vectors, even small values of | are enough induce multiple genres, In Section 3.4, we compute compute the right-hand side of (5) on user embeddings generated from the MovieLens dataset for different cost functions.
3.3 Location of single-genre equilibrium
We next study where the single-genre equilibrium is located, in cases where it exists. As a consequence of the proof of Theorem 1, we can show that the location of the single-genre equilibrium maximizes the Nash social welfare Nash [1950] of the users.
Corollary 5. If there exists with [Genre(u) = 1, then the corresponding producer direction maximizes Nash social welfare of the users:
Genre(u)=
Σ || 1 ==1
arg max
log (p, u:)).
(6)
7

---

Support of equilibrium
Support of equilibrum
support of equ equillbrium
support of equilibrium
cumuuletvve distribution fn
6'=02
6 -02
0.0
0.0
0.0
0.0
0.8
8 -155
0.6
8 0.6
8. 0.6
8. 0.0
0' m'
0.4
0.4
0.
0.2
0.
0.0
0.0
0.0
0.0
0.0
0.0
0.5
100
0.0
1.0
0.0
0.5
1.5
0.5
1.0 0.0
0.5
1.0
19
15
Figure A symmetric equilibrium for different settings of 6", for 2 users located at Uu and U2 such that
Iay [my]l||aal] for producer cost function c(p) = ||p|2 with 3 = 7, and for P = oo producers (see Thcorem 3). The first 4 plots show the support of the cquilibrium in reparamcterized space: note that the the x-axis is (u,,p) and the y-axis is (22, p), i.c., the values that the users derive from good p.
0" = cos -1 (ws,ugg)
Corollary 5 demonstrates that the single-genre equilibrium directions maximizes the Nash social welfare Nash [1950] for users. Interestingly, this measure of welfare for usery implicitly maximized by producers competing with each other in the marketplace. Properties of the Nash social welfare are thus inherited by single-genre equilibria.
3.4 Empirical analysis on the MovieLeens dataset
We provide an empirical analysis of supply-side equilibria using the MovieLens-1006 100K dataset and recommendations based on nonnegative matrix factorization (NMF). In particular, we compute the Lingle-genre equilibrium direction for different cost functions as well as an upper bound on 8 (i.e., the threshold where specialization starts to occur) for different values of the dimension D. These experiments provide qualitative insights that offer additional intuition for our theoretical results.
We focus the rich family of cost functions (p.
[pI on,..... ........ ........PP aap) [lg parameterized by weights a E Ro200 parameter 9 2 1, and cost function exponent B 1. The weights a R capture asymmetries in the costs of different dimensions (a higher value of a; means that dimension i more costly). The parameters y and 33 together capture the tradeoffs between improving along a single dimension versus simultaneously improving among many dimensions. To isolate the impact of each parameter, we either fix = 2 and vary a (and 3) or we fix a = [1,1,.... 1] and vary q (and ().
Setup. The MovieLens 100K dataset consists of 943 users, 1682 movies, and 100,000 ratings [Harper and Konstan, 2015]. For D E (2,3, 5, 10, 50), we obtain D-dimensional user embeddings by running NMF (with D factors) using the scikit-surprise library. We calculate the single genre cquilibrium genre p" arg
E log( (p, u:)) (Corollary 5) by solving the opti- mization program, using the evxpy library for q = 2 and projected gradient descent for * 2. We calculate the upper bound Ba := Nog(N)-1o log EN |- II-) 2 8 from Corollary 4.3
N
log(N)
Single-genre equilibrium direction p'. Figures 4a-4d show the direction of the single-genre cquilibrium p* across different cost functions. These plots uncover several properties of the genre p". First, the genre generally does not coincide with the arithmetic mean of the users. Moreover, the genre varies significantly with the weights a. In particular, the magnitude of the dimension p; is higher if a; is lower, which aligns with the intuition that producers invest more in cheaper dimensions. In contrast, the genre turns out to not change significantly with the norm parameter. Altogether, these insights illustrate that how the genre can be influenced by specific aspects of producer costs.
Upper bound on the threshold 3 where specialization starts occur. Figure 4e shows the value of Bg across different values of D and q. As the dimension D increases, the upper bound 8, decreases, indicating that specialization is more likely to occur. The intuition is that D amplifies the heterogeneity of user embeddings, subsequently increasing the likelihood of specialization. This insight has an interesting consequence for platform design: the platform can influence the level of specialization by tuning the number of factors D used in matrix factorization. Producer costs also impact whether specialization occurs: as the norm increases, the value of B. increases and specialization is less likely to occur.
See Appendix A and https: htmp::://gii //ggthub..tt.ttt/// www.c//mmm///cccc comm///aaaadddess////uu//p/ side--sieeeeeee htt.ppppppp-.iddddd--bbbi....... for details.
8

---

e-10.12 0.86
Q-1:
0=10.32 0.00
Q-1.5
0-10.52
[0.73.3 0:27]
14
qe6.0
5.0
0.3
65
15
20
25
60
05
1.5
23
(a) p" for and 2 (varying ()
(b) p for 2 and a [1, 1] (varying ?)
users
a=|o.88 0.1 0.11
users
q=2.0
user
a=|0.11
q=3.0
== 0.3 0.5]
a=|00.1 0.4 0.5]
c-111
q-4.0
2-2.0
600
Q-4.0
q=5.0
500
400
300
15
22
20
20 embeddding
40
(c)'² for = and q = (d) for D = 3 and a = [L, 1, (c) 9u. for a = [1....,1]
Figure 4: Empirical analysis of supply-side equilibria on the MovicLens- 100K dataset. Plots (a)-(d): Single- genre equilibrium direction p" (computed using Corollary 5) for different cost function weights E RS, and parameters 2 as well as for different dimensions D 2 1. Interestingly. the single-genre equilibrium direction is generally not aligned with the arithmetic mean and places higher weight on cheaper dimensions. Plot (e): Upper bound 3.. the threshold & where specialization starts to occur (computed using Corollary 4) for different values q. Observe that higher values of make specialization more likely to occur.
4 Impact of specialization on market competitiveness
Having studied the phenomenon of specialization, we next study its economic consequences on the resulting marketplace of digital goods. We show that producers can achieve positive profit at equilibrium, even though producers are competing with each other.
More formally, we can quantify the producer profit at equilibrium follows. At symmetric equilibria μı, all producers receive the same expected profit given by:
(n) = pe (P(rip-1)) E.
[b: PLPP) = -llx -1
where expectation in the last term taken over P....., Pp ~ as well as randomness in rec- ommendations. Intuitively, the equilibrium profit of marketplace provides insight about market competitiveness. Zero profit suggests that competition has driven producers to expend their full cost budget improving product quality. Positive profit, the other hand, suggests that the market is not yet fully saturated and new producers have incentive to enter the marketplace.
We show sufficient condition for positive profit in terms of the user geometry and the cost function, and we show this result relies on the equilibrium exhibiting specialization (Proposition 2).
To gain intuition, let us revisit two users located at the standard basis vectors and cost function c(p) = Ipllz- We can obtain the following characterization of profit.
Corollary 6. Let there be users located at the standard basis vectors 61,02 E RR, let the cost function be c(p) = |Ip| gg. For = 2 and 3 2 3 = 2, there is an equilibrium μ with Pey (p)(()) = 1-
Corollary shows that there exist equilibria that exhibit strictly positive profit for any 2. The intuition is that (after sampling the randomness in μ1), different producers often produce different genres of content. This reduces the amount of competition along any single genre. Producers are thus no longer forced to maximize quality, enabling them to generate strictly positive profit.

---

We generalize this finding to sets of many users and producers and to arbitrary norms. In particular, we provide the following sufficient condition under which the profit at equilibrium is strictly positive.
Proposition Suppose that
max min S ISISN
(p.
Tuill < N P/B/
U4
(8)
Then for any symmetric equilibrium μl, the profit (p) is strictly positive.
Proposition 2 provides insight into how the geometry of the users and structure of producer costs impact whether producers can achieve positive profit. To interpret Proposition 2, let examine the quantity Q ;= S min (P: [ that appears on the left-hand side of (8), Intuitively, Q captures how easy it is to produce content that appeals simultaneously to all users. It is larger when the users are close together and smaller when they are spread out. For any set of vectors we see that Q S 1, with strict inequality if the set of vectors is non-degenerate. The right- gand side of (8), on the other hand, goes to as - oc. Thus, for any non-degenerate set of users, if 3 is sufficiently large, the condition in Proposition 2 is met and producer profit is strictly positive.
Although Proposition does not explicitly consider specialization, we show that specialization is nonetheless central to achieving positive profit at equilibrium. To illustrate this, we show that at single-genre equilibrium, the profit zero whenever there are at least P 2 2 producers.
Proposition 3. If j is single-genre equilibrium, then the profit (u) is equal 0.
This draws a distinction between profit in the single-genre regime (where there is no specialization) and the multi- genre regime (where there is specialization,. Interestingly, this distinction parallels between the distinction markets with homogeneous goods and markets with differentiated goods (as we describe in more detail in Appendix C.2).
Our results provide insight about the number of producers needed for market to be saturated and fully competitive. Theorem 2 reveals that the marketplace of digital goods may need far more than 2 producers in order be saturated. Nonetheless, the equilibrium profit does approach 0 as the number of producers in the marketplace goes to oo: this is because the cumulative profit of all producers is at most N and producers achieve the same profit, so (p) S N/P. Perfect competition is therefore recovered in the anfinite-produce limit.
5 Discussion
We presented model for supply-side competition in recommender systems. The rich structure of production costs and the heterogeneity of users enable us to capture marketplaces that exhibit a wide range of forms of specialization. Our main results characterize when specialization occurs, analyze the form of specialization, and show that specialization can reduce market competitiveness. More broadly, we hope that our work serves as a starting point to investigate how recommendations shape the supply-side market of digital goods.
One direction for future work is to further examine the economic consequences of specialization. Several of our results take a step towards this goal: Corollary illustrates that single-genre equilibria occur the direction that maximizes the Nash user welfare, and Proposition 2 shows that specializa- tion can lead to positive producer profit. These results leave open the question of how the welfare of users and producers relate to one another. Characterizing the welfare equilibrium would elucidate whether specialization helps producers at the expense of users or helps all market participants.
Another interesting direction for future work is to extend model incorporate additional aspects of content recommender systems. For example, although we focus on perfect recommendations that match each user to their favorite content, we envision that this assumption could be relaxed in several ways: e.g., the platform may have imperfect information about users, users may not always follow platform recommendations, and producers may learn their best-responses over repeated interactions with the platform. Moreover, although we assume that producers earn fixed per-user revenue, this assumption could be relaxed to let producers set prices.
Addressing these questions would further elucidate the market effects induced by supply-side compe- tition, and inform our understanding of the socictal effeets of recommender systems.
10

---

6 Acknowledgments
We would like thank Jean-Stanislas Denain, Frances Ding, Erik Jones, Quitzé Aidenzucla--tookeee and Ruiqi Zhong for helpful comments on the paper. MJ acknowledges support from the Paul and Daisy Soros Fellowship and Open Phil Al Fellowship.
References
Josef Adalian. Inside the binge factory, 2018. URL https://ww..uultures wtw.://lltuurr.co/////////////// wtt--nttflixsssalloon www-oowddd--iinustrry .tmm..
Gediminas Adomavicius, Jesse C. Bockstedt, Shawn Curley, and Jingjing Zhang. Do recommender
systems manipulate consumer preferences? A study of anchoring effects. Information Systems Research, 24(4):956-975, 2013.
Simon P. Anderson, Andre de Palma, and Jacques-Francois Francois Thisse Discrete Choice Theory of Product Difserentiation. The MIT Press, 10 1992.
Ran Ben Basat, Moshe Tennenholtz, and Oren Kurland. A game theoretic analysis of the adversarial retrieval setting, J. Artif. Int. Res., 6071111177 2017.
Michael R. Baye and Dan Kovenock. Bertrand competition. Palgrave Macmillan UK, London, 2008.
Omer Ben-Porat and Moshe Tennenholtz. A game-theoretic approach to recommendation systems with strategic content providers. In Advances Neural Information Processing Systems (NeurIPS), pages 11 111-11 128, 2018.
Omer Ben-Porat, Itay Rosenberg, and Moshe Tennenholtz. Content provider dynamics and coordi- nation in recommendation ecosystems. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Steven T. Berry. Estimating discrete-choice models of product differentiation. The RAND Journal of Economics, 25(2):242-262, 1994.
Michael Brückner. Christian Kanzow, and Tobias Scheffer. Static prediction games for adversarial learning problems. JMLR, (3(1):2617-2654 2012.
C. d' Aspremont, J. Jaskold Gabszewicz, and J.-F. Thisse On hotelling's s "stability in competition". Econometrica, 47(5))11455 115--1100, 1979. ISSN 00129682, 14680262. URL http://www.jstor. org/stable/1911955
Sarah Dean, Sarah Rich, and Benjamin Recht. Recommendations and user agency: the reachability of collaboratively-iii filtered information. In Conference on Fairness, Accountability, and Transparency (FAT* 220,, pages 436 445. ACM, 2020.
Seth Flaxman, Sharad Goel, and Justin M. Rao. Filter Bubbles, Echo Chambers, and Online News Consumption. Public Opinion Quarterly, 80:298-323 320, 2016.
Ganesh Ghalme, Vineet Nair, Itay Eilat, Inbal Talgam- and Nir Rosenfeld. Strategic classifica- tion in the dark. In Proceedings of the 38th International Conference on Machine Learning, pages 3672-3681, 2021.
Sanford Grossman. Nash equilibrium and the industrial organization of market with large fixed costs. Econometrica, 49: 14,-72, 1981.
Wenshuo Guo, Karl Krauth, Michael I. Jordan, and Nikhil Garg, The stereotyping problem in collaboratively filtered recommender systems. In ACM Conference on Equity and Access in Algorithms (EAAMO), pages 6:1-6:10, 2021.
C. W. Ha. A non-compact minimax theorem. Pacific Journal of Mathematics, 97:115-117, 1981.
Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. Strategic classification. In Proceedings of the 7th Conference on Innovations in Theoretical Computer Science (ITCS),
pages 111-122, 2016.
11

---

F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TIIS), 5(4):1-19, 2015.
Thomas Hodgson. Spotify and the democratisation music. Popular Music, 40(1):1-17, 2021.
Harold Hotelling. Stability in competition. Economic Journal, 39(15):441-77, 1929.
Jiri Hron, Karl Krauth, Michael Jordan, Niki Kilbertus, and Sarah Dean. Modeling content creator incentives algorithm-curated platforms. Technical report, 2022.
Meena Jagadeesan, Celestine Mendler- Mendler-Dümner, and Moritz Hardt. Alternttve microfoundations for strategic classification. In Proceedings of the 38th International Conference on Machine Learning. volume 139, pages
2021.
Jon Kleinberg and Manish Raghavan. How do classifiers induce agents to invest effort strategically? In Proceedings of the 2019 ACM Conference Economics and Compulation, EC '19, pages 825-844, 2019.
Yehuda Koren, Robert M. Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, 2009.
Lydia T. Liu, Nikhil Garg, and Christian Borgs. Strategic ranking. In International Conference Artificial Intelligence and Statistics, AISTATS 2022, volume 151, pages 2489-2518, 2022.
Glenn McDonald.
On netflix and spotify, algorithms hold the power but there's way to get it back., 2019. URL https https://eeee //ep://g/xxp///////// www. 2om/2009/11/ endless-looge
John F. Nash. The bargaining problem. Econometrica, 18(2):155-166 1950.
Jacopo Perego and Sevgi Yuksel. Media competition and social disagreement. Econometrica, 90 (1):223 265, 2022. doi:
URL htttps://omiineeibrraa
wiley.com/dot www.ccccccccc/eeceoc aaoii/abb/10// 3982/ECTA16411,
Philip J. Reny. On the existence of pure and mixed strategy nash equilibria in discontinuous games.
Econometrica, 67(5)::029-1055 1999.
Steven C. Salop. Monopolistic competition with outside goods. The Bell Journal of Economics, 10 (1):141-1156, 1979.
W Sierpinski. Un théoreme sur les continus. Tohoku Mathematical Journal, First Series, 13:300-303, 1918.
Stigler Committee. able at attps://wwl www.ooooooth..cooooooooooooooo.. -dw/-//eeiiarreeeaaraa www.eeerasattteerrrrrrrrr celatforrss--comiitteeooo stit-eeee---rerr---------------e center pdf,, September
Final report: Stigler committee digital platforms. avail-
2019.
Raluca M. Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science, 37(4):530 552, 2018.
Jonathan Vogel. Spatial competition with heterogeneous firms. Journal of Political Economy, 116 (3):4234466, 2008. ISSN 00223808, 1537534X. URL http:://ww..jsstteeee org/ str//etabee/0.. 1086/589891.
Thomas G. Wollmann. Trucks without bailouts: Equilibrium product characteristics for commercial vehicles. American Economic Review, 10866):134-11066 June 2018. doi: URL https: ///ww///wwwww aeaweb. org/artiiless/dd110
20160863.
12


Reviews:
[{"review_id": "rEDfIDQucc", "reviewer": "Reviewer_ham5", "summary": "In this paper, the authors investigate the supply-side equilibria in personalized content recommender systems. They propose a game-theoretic model that captures the multi-dimensional decisions of producers and the heterogeneous preferences of users. They analyze the conditions for specialization to occur and the impact of specialization on market competitiveness. The paper provides insights into how recommender systems shape the diversity and quality of content created by producers.", "questions": "My questions are mentioned above. You can provide the feedback in the rebuttal phase.", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "(1) The paper addresses an important and timely topic – the impact of recommender systems on the supply side of the digital goods market. It sheds light on how producers make decisions to maximize their appearance in recommendations, and how this affects the diversity and competitiveness of the marketplace.\n\n(2) The proposed game-theoretic model is well-designed and captures the multi-dimensional decision space of producers and the heterogeneity of user preferences. This model allows for a nuanced analysis of specialization and its consequences in recommender systems.\n\n(3) The paper provides rigorous theoretical analysis, deriving necessary and sufficient conditions for specialization to occur. It also presents concrete settings with two populations of users to characterize the distribution of content at equilibrium.", "weaknesses": "(1) The implementation details of the proposed model are not provided. It would be helpful for readers to have a clear understanding of how the model can be replicated and reproduced.\n\n(2) The evaluation of the proposed model is limited. The paper does not compare the results with any existing baselines or alternative approaches. It would be valuable to see how the proposed model performs compared to other methods in the field.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and respond to their questions below. We present new results that address some of their concerns: in particular, a **new empirical analysis on the MovieLens-100K dataset** that validates and goes beyond our theoretical findings (see the General Response). \n\n**“The evaluation of the proposed model is limited.”** \n\nWe provide a new empirical analysis of supply-side equilibria of nonnegative matrix factorization on the MovieLens-100K dataset. These experiments beyond qualitative insights that validate our theoretical results and also provide intuition for the structure of supply-side equilibria that goes beyond our theoretical results. \n\n- We compute the direction of the single-genre equilibrium embedding (using Corollary 5) for several different cost functions (see Figures N1-N2 in the General Response pdf). (Recall that this direction is equal to the genre chosen by producers when there is no specialization at equilibrium.) Figures N1-N2 confirm our theoretical findings that the location of the genre does not seem to permit a clean closed-form characterization (Corollary 5) and depends on subtleties of the cost function. Moreover,  Figures N1-N2 go beyond our theoretical results to illustrate that the genre realized in a marketplace can be heavily influenced by some (but not other) aspects of producer costs. \n\n- We compute the boundary where specialization starts to occur, using the upper bound from Corollary 4 (see Figure N3 in the General Response pdf). As $D$ increases, specialization is more likely to occur in a given marketplace. The intuition is that as $D$ increases, the user embeddings become more heterogeneous, which increases the likelihood of specialization, as suggested by our theoretical results in Section 3. Figure N3 further indicates that the platform may influence the level of specialization by setting the embedding dimension $D$ to be higher or lower. We also show how whether specialization occurs subtly depends on the cost function structure, which also aligns with our theoretical results from Section 3. \n\nWe refer the reviewer to the General Response for details of the empirical setup and results. \n\n**“The implementation details of the proposed model are not provided. It would be helpful for readers to have a clear understanding of how the model can be replicated and reproduced.”** \n\nAs a primarily theoretical contribution (except for the new experiments on the MovieLens dataset), we believe that we have fully specified our model in Section 2, and included all the theoretical proofs in the attached supplement. As such, we believe that our results are fully replicable and verifiable by readers–if there is a particular detail that is unclear, we would appreciate if the reviewer could point it out so we can clarify it.\n\n**“The paper does not compare the results with any existing baselines or alternative approaches. It would be valuable to see how the proposed model performs compared to other methods in the field.”** \n\nIt is not clear what “baselines” would mean in our model. Our goal in this paper is not an algorithm or a method. Rather, as is typical with many papers in the economics and machine learning literature, we develop a mathematical model and analyze the model to provide economic insights about real-world marketplaces. Our specific goal in this paper is to study content creator incentives in recommender systems, and in particular, the economic phenomena of specialization. For more context about the style of our contribution, we refer the reviewer to the “Nature of contribution” section of the General Response. \n"}, {"review_id": "s2ZP4OMF3m", "reviewer": "Reviewer_2uvz", "summary": "This work studies the supply-side equilibria in content recommender platforms. The authors proposed a game-theoretic model to describe content creators' competition and derive necessary and sufficient conditions under which the specialization over genres occurs or does not occur at the equilibrium. ", "questions": "Please see the questions raised in weaknesses. \n\n1. in corollary 3, what does it mean by \"N users split equally between two linearly independent vectors?\"", "limitations": "No", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The motivation is well justified, and the proposed problem setting is interesting in general. \nThe technique used for the main result is novel and solid, and the insight behind the theoretical results is well-explained. ", "weaknesses": "The proposed problem itself and the solution are intriguing, but the specific problem setting considered seems to be oversimplified. In particular, the utility function defined in Eq.(1) is symmetric for different creators, and only the symmetric mixed NE is considered. I understand this is the only feasible solution concept with an existence guarantee in this situation; still, it is not a realistic model to characterize the outcome of creators' competition in practice. In the real world, creators should have heterogeneous preferences and costs, which should be reflected in their distinct strategy sets or cost functions. And also, it is hard to believe in any real market, all creators will eventually form a homogeneous belief about the production strategy (i.e., a symmetric mixed NE). \n\nIn terms of the theoretical results, my understanding is that the specialization phenomenon hinges on the joint property of the user population structure and parameter $\\beta$. While I appreciate the insight about how user distribution might affect the emergence of specialization, the discussion of $\\beta$ seems too restrictive to the specific form of the cost term. The current term $\\|p\\|^{\\beta}$ is too simple to capture the nature of the cost, for example, 1. why the marginal cost has to depend on an exponential factor rather than a multiplicative factor? 2. why the cost only depends on the norm of $p$? What if we generalize the cost term in the following ways:\n\n1. $c_j(p) = a \\|p\\|^{\\beta}$\n2. $c_j(p) = a_j \\|p\\|^{\\beta}$\n3. $c_j(p) = a \\|p-p_j\\|^{\\beta}$\n4. $c_j(p) = a_j \\|p-p_j\\|^{\\beta}$\n\nCan we still derive results that share similar insights using the current technique? If such an extension is promising, I would highly recommend this paper. \n\nMinor:\nL.254: should be \"linearly independent vectors\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and respond to their questions below. In response to the reviewers, we also showed **a new empirical analysis on the MovieLens-100K dataset** that both validates and go beyond our theoretical findings (see the General Response). \n\n**“What if we generalize the cost term in the following ways?”** \n\nWe describe the applicability of our cost functions to the functional forms proposed by the reviewers. \n\n- First, our model and results directly accommodate the scaled cost function $c_S(p) = a ||p||^{\\beta}$. We can define a new norm $||x||_{S} = ||x \\cdot a^{1/\\beta}||$, so that $c_S(p) =a ||p||^{\\beta} = ||p||_S^{\\beta}$. Our characterization from Theorem 1 specifies when specialization occurs for $c_S(p)$: specialization occurs for $c_S(p)$ if and only if specialization occurs for the non-scaled cost function $c(p) = ||p||^{\\beta}$. \n\n- Our model and results can also accommodate the translated cost function $c_T(p) = a ||p - q||^{\\beta}$. We can handle the scalar factor of $a$ by a similar argument to the above, so WLOG let’s assume that $a =1$. We claim that $X \\sim \\mu$ is a symmetric mixed equilibrium for $c_T(p) = ||p-q||^{\\beta}$ if and only if $(X - q)$ where $X \\sim \\mu$ is a symmetric mixed equilibrium for $c(p) = ||p||^{\\beta}$. The intuition is that we can write $\\langle u, p - q \\rangle = \\langle u, p \\rangle - \\langle u,q \\rangle$, which just shifts a user’s utility by the same constant factor for all producers so it can be disregarded. This gives us an equivalence between the equilibria in these two setups; however, to apply Theorem 1, we’d need to slightly change the definition of genre. We instead define $Genre(\\mu)$ to be set of directions $\\frac{p-q}{||p-q||}$ for $p \\in \\text{supp}(\\mu)$. That is, the genres object would instead capture the directions along which the producers *change* from the starting point of $q$, rather than the final direction of the producers. (Note: we don’t run into any technical difficulties with the nonnegative orthant constraints, because those turn out to only be required for user vectors and not for producer vectors in our results.) \n\nAs described below, we cannot directly capture heterogeneous costs in our model which precludes some of the functional forms proposed by the reviewer (see additional discussion below).\n\n**“The current term $||p||^{\\beta}$  is too simple to capture the nature of the cost. For example, why the cost only depends on the norm of p?”** \n\nWe allow for any norm (not just the $\\ell_2$-norm) within our cost function, which captures a very broad family of functions. For example, we can take the norm $||\\cdot||$ to be $\\ell_q$ norm for any $q \\ge 1$ as well as any weighted cost norm $||x||$ defined to be the $\\ell_2$ norm of $[x_1 \\cdot \\alpha_1, \\ldots, x_D \\cdot \\alpha_D]$ (we study both in our new experiments on the MovieLens-100K dataset, as described in the General Response). In more detail, weighted costs are parameterized by a $D$-dimensional vector $\\alpha \\in R_{\\ge 0}^D$ of weights such that $\\sum_{i=1}^D \\alpha_i = 1$. The cost function $c_{\\alpha}$ is defined to be $c_{\\alpha}(p) := ||[p_1 \\cdot \\alpha_1, \\ldots, p_D \\cdot \\alpha_D]||^{\\beta}$ where the norm is the $\\ell_2$ norm. This cost function captures that certain dimensions might be cheaper vs. more expensive for the producer to improve. We could also accomodate Mahalanobis distance, matrix norms, etc. within our model.\n\n**“The utility function defined in Eq.(1) is symmetric for different creators, and only the symmetric mixed NE is considered.”**\n\nOur model and results do require that the cost functions are homogeneous. One could have easily extended our model to allow for producers to have heterogeneous cost functions, but we focused on a single cost function to simplify the technical analysis. In fact, the technical analysis for homogeneous cost functions already required several novel technical innovations (see Appendix B.1). We would qualitatively expect that the tendency towards specialization would only be amplified if producers could have heterogeneous cost functions. \n\nThat being said, our model does implicitly captures heterogeneity in *producer behaviors* via the randomness in the symmetric mixed equilibrium. Under this randomness, each producer independently samples a content vector from the equilibrium distribution, so different producers create different content in any given realization. In fact, this heterogeneity in producer behavior motivated us to formalize specialization in terms of the support of the symmetric mixed equilibrium distribution (lines 179-188). \n\n**“In corollary 3, what does it mean by \"N users split equally between two linearly independent vectors?\":**\n\nWe assume that $u_1 = .. u_{N/2} = x_1$ and $u_{N/2+1} = … = u_N = x_2$, for some linearly independent vectors $x_1$ and $x_2$. "}, {"review_id": "Mfbi8b1Txl", "reviewer": "Reviewer_sbgm", "summary": "This paper aims to understand the equilibria of the digital content producer side competition in the recommender platforms. It specifically studies the potential of specialization, where different producers create different type of content. The paper proposed to model value of product as the inner product of user and item embeddings, where the personalization, product cost, as well as producer profit are derived from the embedding model. Based on this setup, the paper studies the equilibria of the producer side competition. It provides conditions under which specialization occurs and the form it takes. It claims that specialization can reduce the competitiveness of the marketplace by decreasing competition within each genre.\n\n**Note** It should be noted that I am not deeply familiar with the field of algorithmic game theory. As a result, I may lack an in-depth understanding of the conceptual links between various aspects of the theory used in the paper and the broader field. Therefore, I may not be adequately equipped to fully assess the novelty and correctness of the technical content. While effort has been made to understand and summarize the paper, the detailed insights on the specific game-theoretic models and their implications could benefit from a review by a scholar who is more specialized in this field.", "questions": "* Given the theoretical framework proposed in the paper, how can this study be practically utilized, particularly for practitioners who are actively involved in the development of new recommender system models? The practical implications and real-world applications of the theoretical findings would be of interest to the broader research community and industry practitioners. Clarification on this point would further enhance the utility and impact of the work.", "limitations": "The limitations is discussed in the paper.", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The study of the influence of recommender system on the digital content production and diversity of the content is an important topic.\n* The insights drawn from the study of  specialization could potentially provide new directions for recommender model design.\n", "weaknesses": "* The current model presented in the paper seems to be specifically tailored towards digital goods such as music, movies, and news. It is uncertain how this model might translate to other types of recommender systems such as e-commerce, dating, or job recommendation platforms, which also hold significant relevance in today's digital marketplace. Although the authors have proposed directions for future work, further investigation is required to expand this model to other sectors and recommendation scenarios, enhancing the universal applicability of the research.\n* While the model provides valuable theoretical insights, it could be strengthened by empirical validation. Observations or experiments on real-world recommender systems could further test and substantiate the authors' claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and respond to their questions below. We present new results that address some of their concerns: in particular, **a new empirical analysis on the MovieLens-100K dataset** that validates and goes beyond our theoretical findings (see the General Response). \n\n**“While the model provides valuable theoretical insights, it could be strengthened by empirical validation.”** \n\nWe provide a new empirical analysis of supply-side equilibria of nonnegative matrix factorization on the MovieLens-100K dataset. These experiments beyond qualitative insights that validate our theoretical results and also provide intuition for the structure of supply-side equilibria that goes beyond our theoretical results. \n- We compute the direction of the single-genre equilibrium embedding (using Corollary 5) for several different cost functions (see Figures N1-N2 in the General Response pdf). (Recall that this direction is equal to the genre chosen by producers when there is no specialization at equilibrium.) Figures N1-N2 confirm our theoretical findings that the location of the genre does not seem to permit a clean closed-form characterization (Corollary 5) and depends on subtleties of the cost function. Moreover,  Figures N1-N2 go beyond our theoretical results to illustrate that the genre realized in a marketplace can be heavily influenced by some (but not other) aspects of producer costs. \n- We compute the boundary where specialization starts to occur, using the upper bound from Corollary 4 (see Figure N3 in the General Response pdf). As $D$ increases, specialization is more likely to occur in a given marketplace. The intuition is that as $D$ increases, the user embeddings become more heterogeneous, which increases the likelihood of specialization, as suggested by our theoretical results in Section 3. Figure N3 further indicates that the platform may influence the level of specialization by setting the embedding dimension $D$ to be higher or lower. We also show how whether specialization occurs subtly depends on the cost function structure, which also aligns with our theoretical results from Section 3. \n\nWe refer the reviewer to the General Response for details of the empirical setup and results. \n\n**“The current model presented in the paper seems to be specifically tailored towards digital goods such as music, movies, and news. It is uncertain how this model might translate to other types of recommender systems such as e-commerce, dating, or job recommendation platforms, which also hold significant relevance in today's digital marketplace.”**\n\n As the reviewer notes, the current model presented in the paper is tailored towards digital content. This is intentional: as discussed in the introduction, the supply-side effects of digital content recommendation platforms are poorly understood, and understanding these particular supply-side effects is the focus of our paper. Other types of recommender systems, such as for dating or for jobs, have different structure, in terms of interaction between participants, incentive and strategic behavior, and the platform’s objective. We believe that grouping these two types of marketplaces into one single model would hinder the derivation of  conclusive economic insights. \n\n**“Given the theoretical framework proposed in the paper, how can this study be practically utilized, particularly for practitioners who are actively involved in the development of new recommender system models?”**\n\nOne insight from our results is that the practitioners can influence the long-run equilibrium content landscape through their choice of embedding dimension. As shown in our empirical analysis (Figure N3), the embedding dimension impacts the cost function exponent at which specialization starts to occur in a marketplace. What this means is that a practitioner can increase the embedding dimension to increase the level of specialization in the marketplace (whether every producer produces similar genres or very different ones). More generally, our work shows that platforms can influence the amount of specialization through their recommender systems.\n\nAs to whether the practitioners should opt to induce specialization or not, this depends on several factors. Our results highlight two consequences of specialization—(1) content diversity, and (2) positive producer profit—which should both impact the platform’s decision. \n- Specialization leads to content diversity, which impacts the long-term satisfaction of users and thus the long-run revenue of the platform. On the positive side, content diversity can provide users with content tailored to their interests, which may help attract and retain a wider user base. On the other hand, content diversity and the consumption of niche content may inadvertently drive filter bubbles, polarization, or other negative user experiences. \n- Specialization can also lead producers to earn a positive profit at equilibrium (Section 4). This has negative implications for content quality which might reduce the long-term satisfaction of users (and thus lower user retention). On the other hand, this has positive implications for producers, which might improve producer retention. \n\nThe platform can use knowledge of its specific marketplace to balance the positive and negative effects, and determine whether specialization improves revenue in a given marketplace. \n"}, {"review_id": "MKmTGa3o8E", "reviewer": "Reviewer_rJ3F", "summary": "The paper presents a theoretical study regarding the equilibrium of supply-side competition in recommender systems. In particular, the analysis examined when and how the recommender system influence producers' creation of online contents and will specialization occur under the effect of recommender systems.", "questions": "1. To what extent the theoretical analysis reflects real-world recommender systems that are driven by deep learning and LLMs.\n\n2. Is there any experimental analysis to show the predicted effects, at least based on simulation.", "limitations": "The paper does not discuss limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Examining the supply-side competition under the influence of recommender system is an interesting topic to explore.\n\n2. The paper presents theoretical analysis for the equilibrium of supply-side competition and specialization.", "weaknesses": "1. The theoretical analysis is based on a very simplified recommendation model, i.e, inner-product between the user and item vectors which is basically from the idea of matrix factorization. However, modern recommender systems have advanced much more beyond MF, and real-world systems are driven by deep neural networks and large language models. As a result, it's not clear if the theoretical analysis really sheds light on real-world recommender systems.\n\n2. There is no experimental analysis, even simulated analysis is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and respond to their questions below. We present new results that address some of their concerns: in particular, **a new empirical analysis on the MovieLens-100K dataset** that validates and goes beyond our theoretical findings (see the General Response). \n\n**“Is there any experimental analysis to show the predicted effects”** \n\nWe provide a new empirical analysis of supply-side equilibria of nonnegative matrix factorization on the MovieLens-100K dataset. These experiments beyond qualitative insights that validate our theoretical results and also provide intuition for the structure of supply-side equilibria that goes beyond our theoretical results. \n\n- We compute the direction of the single-genre equilibrium embedding (using Corollary 5) for several different cost functions (see Figures N1-N2 in the General Response pdf). (Recall that this direction is equal to the genre chosen by producers when there is no specialization at equilibrium.) Figures N1-N2 confirm our theoretical findings that the location of the genre does not seem to permit a clean closed-form characterization (Corollary 5) and depends on subtleties of the cost function. Moreover,  Figures N1-N2 go beyond our theoretical results to illustrate that the genre realized in a marketplace can be heavily influenced by some (but not other) aspects of producer costs. \n- We compute the boundary where specialization starts to occur, using the upper bound from Corollary 4 (see Figure N3 in the General Response pdf). As $D$ increases, specialization is more likely to occur in a given marketplace. The intuition is that as $D$ increases, the user embeddings become more heterogeneous, which increases the likelihood of specialization, as suggested by our theoretical results in Section 3. Figure N3 further indicates that the platform may influence the level of specialization by setting the embedding dimension $D$ to be higher or lower. We also show how whether specialization occurs subtly depends on the cost function structure, which also aligns with our theoretical results from Section 3. \n\nWe refer the reviewer to the General Response for details of the empirical setup and results.\n\n**“The theoretical analysis is based on a very simplified recommendation model, i.e, inner-product between the user and item vectors which is basically from the idea of matrix factorization…To what extent the theoretical analysis reflects real-world recommender systems that are driven by deep learning and LLMs?”**\n\nOur formalism is not limited to simple matrix factorization algorithms. If we interpret user and movie vectors as “embeddings” *learned* by the algorithm, our formalism can also capture any deep learning-based recommendation systems that learn embeddings for the users and movies. In particular, the process by which these embeddings are learned can be arbitrarily complex, as long as the recommendation system ultimately evaluates user value based on the inner product of the learned embeddings. \n\nAs to language-model based recommendation systems, it is not clear if our model is a perfect fit for these newer systems. Understanding the supply-side equilibria of language model-based recommendation systems would be a very interesting direction for future work. We hope that our analysis of specialization inspires future work on this topic. \n\nAs a broader point about the stylized nature of our model, we refer the reviewer to “Nature of our contribution” in the General response. We don’t think that incorporating all of the complexities of real-world recommender systems into our model would have improved our analysis of specialization. If a model has too many changing components and degrees of freedom, it is hard to pinpoint exactly which component is responsible for a given phenomena. In general, complex models are often less robust, whereas results from simple models tend to at least qualitatively generalize beyond their assumptions. Our stylized model is one of the simplest that enables us to study specialization (see Appendix A.1). "}]

Author rebuttal:
Thanks to the reviewers for their feedback. We provide a **new empirical analysis of our theoretical findings on the MovieLens-100K dataset**. We then clarify the nature of our contribution of proposing and analyzing a mathematical model to study an economic phenomenon. (We respond individually to the reviewers below.) 

## Empirical analysis on MovieLens dataset

Several reviewers asked about an empirical analysis of our theoretical findings on real-world datasets. We provide an empirical analysis of supply-side equilibria using the MovieLens-100K dataset and recommendations based on nonnegative matrix factorization (NMF). These experiments provide qualitative insights that validate our theoretical results and provide intuition going beyond our theoretical results. 

We construct user embeddings of dimension $D = 2, 3, 5, 10, 50$ by running NMF with $D$ factors. We consider two families of producer cost functions:

- (C1) Let $\alpha \in R_{\ge 0}^D$ be a weight vector such that $\sum_{i=1}^D \alpha_i = 1$. The cost function $c_{\alpha}(p) := ||[p_1 \cdot \alpha_1, \ldots, p_D \cdot \alpha_D]||^{\beta}$ captures that certain dimensions might be cheaper vs. more expensive to improve along.   

- (C2) Let $q \ge 1$ and let $c_q(p) := ||p||_q^{\beta}$. 

**Single-genre equilibrium direction**: We compute the direction of the single-genre equilibrium embedding (using Corollary 5) for cost functions in (C1) (see Figure N1 in pdf) and in (C2) (Figure N2 in pdf). (Recall that this direction is equal to the genre chosen by producers when there is no specialization at equilibrium.) We observe the following:  
- For (C1), the genre varies significantly with the weights $\alpha$ (see Figure N1). The magnitude of the genre coordinate is higher along the cheaper dimension. 
- For (C2), the genre does not change significantly with the norm parameter $q$ (see Figure N2). 
- In both cases, the genre typically does not coincide with the arithmetic mean of the users. 

Figures N1-N2 align with our theoretical findings that the genre location doesn’t permit a clean closed-form characterization (Corollary 5) and depends on subtleties of the cost function. Figures N1-N2 also go beyond our theoretical results to show that the genre can be heavily influenced by some (but not other) aspects of producer costs. 

**Boundary where specialization starts to occur**: We investigate the cost function exponent $\beta^*$ where specialization starts to occur (line 224, Theorem 1). We compute an upper bound $\beta^u$ on $\beta^*$ (using Corollary 4) for cost functions in (C1) and different embedding dimensions $D$ (Figure N3). We observe the following: 
- As $D$ increases, the value of $\beta^u$ *decreases*, and specialization is more likely to occur (see Figure N3). The intuition is that increasing $D$ increases the heterogeneity of user embeddings, which increases the likelihood of specialization as suggested by our theoretical results in Section 3. Figure N3 suggests that the platform may influence the level of specialization by tuning $D$. 
- As the cost function parameter $q$ increases, the value of $\beta^u$ *increases* and specialization is less likely to occur (see Figure N3). This confirms our theoretical insights about the subtle role of the cost function in whether specialization occurs. 

**Details of empirical setup**: We use the MovieLens 100K dataset which consists of 943 users, 1682 movies, and 100,000 ratings. To obtain $D$-dimensional user embeddings, we ran NMF (with $D$ factors) using the scikit-surprise library on the full dataset. For Figure N3, we directly calculate $\beta^u$ using Corollary 4. For Figures N1-N2, we numerically solve the optimization program in Corollary 5: we directly solve this using CVXPY for (C1), and we use projected gradient descent with step size 1.0 over 100 iterations with projection done with CVXPY for (C2). 

We will use the extra page in the final version to include the details of this empirical analysis. 

## Nature of contribution: Proposing and analyzing a stylized model 

Some of the reviewers asked for comparison against baselines or asked questions about the stylized nature of our model. We would like to clarify the nature and style of our contribution. 

**Proposing and analyzing a mathematical model:** Our contribution is to develop a mathematical model and analyze the model to provide economic insights about real-world marketplaces.  As a result, we do not design an algorithm or a method, but rather analyze *behavior* within the mathematical model that we propose (i.e., *specialization* within a model for content creator competition). 

Proposing and analyzing behavior within a mathematical model is standard in economics and machine learning, machine learning theory, and the societal aspects of machine learning more broadly (e.g. [A], [B], [C]). These subfields are listed in the NeurIPS call for papers: Theory (algorithmic game theory) and Social and economic aspects of machine learning (strategic behavior). 

[A] Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. “Delayed Impact of Fair Machine Learning”. ICML 2018 Best Paper.
 
[B] Simon Zhuang and Dylan Hadfield-Menell. “Consequences of Misaligned AI.” NeurIPS 2020. 

[C] Kate Donahue and Jon Kleinberg. “Optimality and Stability in Federated Learning: A Game-Theoretic Approach.” NeurIPS 2021. 

**Stylized nature of our model:** We don’t think that incorporating all of the complexities of real-world recommender systems into our model would have improved our analysis of specialization. If a model has too many changing components and degrees of freedom, it is hard to pinpoint exactly which component is responsible for a given phenomena. In general, complex models are often less robust, whereas results from simple models tend to at least qualitatively generalize beyond their assumptions. Our stylized model is one of the simplest that enables us to study specialization (see Appendix A.1). 

Meta-review: The paper studies the supply-side equilibria in personalized content recommender systems balancing producer incentives along the heterogeneous preferences of users. All the reviewers appreciate the timely topic studied in the paper, the rigorous theoretical analysis under the game-theoretic model framework, and the insights well explained. Multiple reviewers did ask for more discussions on the practicality of the proposed model, e.g., theories are built upon stylized models, and empirical validation. Please consider address them in your manuscript.
Decision: Accept (poster)