Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition
Authors: Meena Jagadeesan
Michael Jordan
Jacob Steinhardt
Nika Haghtalab
Venue: NeurIPS 2023 poster (2023)
Keywords: competition
equilibria
inverse scaling
digital marketplaces
TL;DR: Competition between model-providers can alter the behavior of scaling laws: increasing scale can decrease the equilibrium social welfare for users, even when the Bayes risk of a single model-provider improves

Abstract: As the scale of machine learning models increases, trends such as scaling laws anticipate consistent downstream improvements in predictive accuracy. However, these trends take the perspective of a single model-provider in isolation, while in reality providers often compete with each other for users. In this work, we demonstrate that competition can fundamentally alter the behavior of these scaling trends, even causing overall predictive accuracy across users to be non-monotonic or decreasing with scale. We define a model of competition for classification tasks, and use data representations as a lens for studying the impact of increases in scale. We find many settings where improving data representation quality (as measured by Bayes risk) decreases the overall predictive accuracy across users (i.e., social welfare) for a marketplace of competing model-providers. Our examples range from closed-form formulas in simple settings to simulations with pretrained representations on CIFAR-10. At a conceptual level, our work suggests that favorable scaling trends for individual model-providers need not translate to downstream improvements in  social welfare in marketplaces with  multiple model providers.

Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition
Meena Jagadeesan UC Berkeley mj mjaaaddesannbeereleey edu
Michael Jordan UC Berkeley j jordanecs bew.eley.eeeec......... edu
Jacob Steinhardt" UC Berkeley o jsteinhaarct@@berkeeeeyyee
Nika Haghtalab"
UC Berkeley nika@berkeley.cm edu
Abstract
As the scale of machine learning models increases, trends such as scaling laws anticipate consistent downstream improvements in predictive accuracy. However, these trends take the perspective a single pooeldprovider in isolation, while in reality providers often compete with each other for users. In this work, we demonstrate that competition can fundamentally alter the behavior of these scaling trends, causing overall predictive accuracy across users be non-monotonic or decreasing with scale. We define a model of competition for classification tasks, and use data representations in scale. We find many settings where improving data representation quality (as measured by Bayes risk) decreases the overall predictive accuracy across users (i.e., social welfare) for a marketplace examples range from closed-form formulas in simple settings simulations with pretrained representations on CIFAR-I0. At conceptual level, our work suggests that favorable scaling trends for individual model-providers need translate downstream improvements in social welfare in marketplaces with multiple model providers.
lens for studying the impact of increases
competing model-providers. Our
Introduction
Scaling trends in machine learning suggest that increasing the scale of a system consistently improves predictive accuracy. For example, scaling laws illustrate that increasing the number of model parameters [Kaplan et al., 2020, Sharma and Kaplan 2020, Bahri et al., [021] and amount of data [Hoffmann [022] can reliably improve model performance, leading better representations and thus better predictions for downstream tasks [Hermandez et al. 2021
However, these scaling laws typically take the perspective of single model-provider in isolation, when reality, modeidproviders often compete with each other for users. For example, in digital marketplaces, multiple online platforms may provide similar services (e.g., Google search Bing, Spotify vs. Pandora, Apple Maps vs. Google) and thus compete for users on the basis of prediction quality. A distinguishing feature of competing platforms is that users switch between platforms and select platform that offers them the highest predictive accuracy for their specific requests. This breaks the direct connection between predictive accuracy of single platform isolation and social welfare across competing platforms, and raises the question: what happens to scaling laws when model-preet -roevd-rrviders compete with each other?
"Equal contribution
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Model provicer
Model provider Bayes aptima
correct preciction
all incerrect all incorrect
Model provider
correct preciction
3
Bayes risk 0.21
Bayes risk 0.07
Equiliarium 0.03
Fquillarium - n4
2
1-
0-
-1-
-2
-2
-3
-3
-2
(a)
(b)
Figure I: Comparison cquilibrium loss on two data distributions, one with high Bayes risk (left) and one with lower Bayes risk (right). Each plot shows the linear predictors chosen at equilibrium under competition between three providsss (solid lines), along with two approximately Bayes-optimal predictors (dashed lines). The equilibrium social loss is lower in the left plot than the right plot, even though the Bayes risk is much higher. The intuition thus, users have a greater likelihood the overall predictive accuracy for users (i.e., the social welfare).
approximate Bayes optima disagree on more data points in left plot than the right plot; least one predictor offering them a correct prediction, which increases
We show that the typical intuition about scaling laws can fundamentally break down under competition. Surprisingly, even monotonicity can be violated: increasing scale can decrease the overall predictive accuracy (social welfare) for users. More specifically, we study increases scale through the lens of data representations (i.e.. learned features). motivated by how increasing scale generally improves representation quality [Bengio al. 2013 We exhibit several multi-class classification tasks where better data representations (as measured by Bayes risk) decrease the overall predictive accuracy (social welfare) for users, when varying data representations along several different axes.
The basic intuition for this non-monotonicity is illustrated in Figure M When data representations are large fraction of users, and near-optimal predictors
low quality, any predictor will be incorrect may disagree on large subpopulations of users. Model providers are thus incentivized to choose complementary predictors that cater to different subpopulations (market segments), thus improving the overall predictive accuracy for users. In contrast, when representations are high quality, each optimal predictor is incorrect on only a small fraction of users, and near-optimal predictors likely agree with each other on most data points. As result, providesss are incentivized to select similar predictors, which decreases the overall predictive accuracy for users.
To study when non-monotonicity can occur, we first focus on stylized setup that permits closed- form calculations of the social welfare equilibrium (Section 3). Using this characterization, in three concrete binary classification setups, we show that the equilibrium social welfare can be non-monotonic in Bayes risk. particular, we vary representations along three axes the per-representation Bayes risks, the noise level of representations, and the dimension of the data representations and exhibit non-monotonicity in each case (Figure
Going beyond the stylized setup of Section 3 in Section we consider linear function classes and demonstrate empirically that the social welfare be non-monotonic in the data representation
"e motivated by emerging marketplaces where different proeieproviders utilize the same pretrained model, but finetune model different ways. To simplify this complex training process, we conceptualize pretraining as learning data representations (e.g. features) and fine- mie-tuning as learning a predictor from these representations. In this formalization, increasing the scale of the pretrained model leads to improvements in data representations accessible the model- provider during "fine-tuning".

---

quality. We consider binary and 10-class image classification tasks on CIFAR-10 where data representations are obtained from the last-layer representations of AlexNet, VGGI6, ResNet18. ResNet34, and ResNet50, pretrained on ImageNet. Better representations (as measured by Bayes risk) can again perform worse under competition (Figure S We also consider synthetic data where we can vary representation quality more systematically, again finding ubiquitous non-monotonicities.
Altogether, our results demonstrate that the classical setting of a single model-provider can be a poor proxy for understanding multiple competing model-providers. This suggest that caution is needed when inferring that increased social welfare necessarily follows from the continuing trend towards improvements in predictive accuracy in machine learning models. Machine learning researchers and regulators should evaluate methods in environments with competing model- providerosider in order to reasonably assess the implications of raw performance improvements for social welfare.
1.1 Related work
Our work connects to research threads on the welfare implications of algorithmic decisions and competition between data-driven platforms.
Welfare implications of algorithmic decisions. Recent work investigates algorithmic monoculuure
[Kliiberg and Raghavan 202T Bommasani et al. 2022], setting in which multiple model-providers use the same predictor. In these works, monoculture ture is intrinsic to the decision-makinn pipeline: model-provee pooidde-roviderss given access to shared algorithmic ranking Kleinberg and Raghavan 2021 or shared components in the training pipeline [Bommasani et al., 2022]. In contrast, in our work, monoculture may arise endogenously from competition, as result of scaling trends. Model-providers are al always given access to the same function classes and data, but whether or not monoculture arises depends the quality of data representations and its impact the incentives of model-provee providers. Our work thus offers a new perspective on algorithmic monoculture, suggesting that it may arise naturally in competitive settings as a side effect of improvements in data representation quality.
More broadly, researchers have identified several sources of mismatch between predictive accuracy and downstream welfare metrics. This includes nurrowing of a classifier under repeated interactions with users HHsshimoto et al. 2018). preference shaping of users induced by a recommendation algorithm [Carool et al. 2022 Dean and Morgenstern 2022 Curmei et al. 2022 strategic adaptation by users under a classifier Brückker et al. 2012 Hardt et al. 2016). and the long-term impact of
algorithmic decisions [Liu al. 2018 2020
Competition between data-driven platforms. Our work is also related to the literature on competing predictors. The model in our paper shares similarities with Ben-Porat and Tennenholtz [2017] 2019].
who studied equílibria between competing predictors. [Ben- and Tennenholtz [2017] 2019 show that empirical risk minimization is not optimal strategy for a model-provider under competition
and design algorithms that compute the best-responses; contrast, our focus on the equilibrium social welfare and how it changes with data representation quality. The specifics of our model also slightly differ from the specifics of Ben- and Tennenholtz 2019]. In their model, each user has an accuracy target that they wish to achieve and randomly chooses between model-providen meeting that target; in contrast, in our model, each user noisily chooses the model-provinn that minimizes their loss and
can have asymmetric market reputations.
Our work also relates to viasivariance games (Feng al. 2019 between competing model-providers. but do not consider equilibrium social welfare for users; in contrast, our work focuses on the equilibrium social welfare. The model of Feng et al. [019) also differs from the model in our work. In Feng et al. 2019). a providerrovider action is modeled as choosing an error distribution for each user, and the action set includes error distributions with range of different variances. In contrast, in our setup, the error distribution for every user is always a point mass (variance 0). Thus, the equilibrium characterization utility in
However, [Fng al. [2019] focus on the the equilibrium strategies for the
of Feng al. [219]] does not translate to our model. The specifics of the the work of Feng et al. 2019 differs slightly from our model as well.
Other aspects studied in this research thread include competition between model providers using out-of-box learning algorithms that do not directly optimize for market share [Ginart et al. 2021 Kwon et al. 2022 Dean et al. 2002], competition between model-providers selecting regularization
parameters that tune model complexity [Tyer and Ke 2022], competition between bandit algorithms
where data directly comes from users [Arddrr et al. 2020 Jagadeesan et al. 2022 and competition
3

---

between algorithms dueling for user (Immorlica et al. 2011 Our work also relates to classical economic models of product differentiation such Hotelling's 's model [Hotelling 1981 dssppmmontt et al. 1979] (see Anderson et al. 1992] for textbook treatment), as well as the emerging area of
platform competition [see, e.g., Jullien and Sand-Zantman
202T Calvano and Polo 2021
2 Model
We focus on a multi-class classification setup with input space X C Rd and output space Y Each user has an input x and corresponding true output y, drawn from
1}. 1}. a distribution D over X Y. Model providers choose predictors F S (A(Y)) where A(Y) is the set of distributions over Y. A user's s loss given predictor f is ((f(z). = Ply & f(x)]. In Section 3, we take JF {0, 1, 2, ...... K - X be all deterministic functions mapping inputs to classes, while in Section 4 we consider linear predictors of the form (() = softmax Wxx b).
{0, 1, 1.2...., K -K-1
from some model family
We study competition between m. 2 2 model-providers for users, building on the model of B Benn-Porat
and Tennenholiz 2017 2019 We index the model-providers by [m]] == {1, 2...,.... m., and let denote the predictor chosen by model provider j. After the model-providers choose predictors fi....,,m fms each user then chooses one of the m -povvd-prvvserrs to link to, based on prediction accuracy. Model-provider aim to optimize the number of users that they win. We describe the model in detail below. (We note that this model stylized and will make several simplifying assumptions; we defer a detailed discussion of the implications of these assumptions to Appendix A
User decisions. Users noisily pick the model-provider offering the best predictions for them. That is, a user with representation & and true label y chooses a model-provider, j (xt y) such that the loss ((f))( (zr,,) is the smallest across all model-providers j [m]. subject noise in user decisions. More formally, we model user noise with the logit model [Trin 2002
also known as the Boltzmann
rationality model:
(()(())))
(( =
where e > denotes a noise parameter. We extend this model to account for uneven market reputations across decisions in Section B.1
Model provider incentives. model providers utility captured by the market share that they win. That is, model-provider j's utility is
E v( ((-)))
(b)
= =]],,
where (-j denotes the predictors chosen by the other model-providers and where the expectation is over (z. 3) drawn from D. Since the market shares always sum to one, this a constant-sum game.
Each model-provider chooses a best response to the predictors of other model-providers. That is, model-provider j chooses predictor fy such that
fj E arg max u(fz;f=z)- JeF
The best-response captures that model-providers optimize for market share. In practice, model- providers may do so via A/B testing to steer towards predictors that maximize profit, or by actively collecting data on market segments where competitors are performing poorly.
We study market outcomes f = f2........ fm) that form a Nash equilibrium. Recall that f is a pure strategy Nash equilibrium if for every E [m). model-provider j's predictor is best-response f-y: that is, J3 E arg j w(fz; f.). In well-hehaved instances, pure-strategy equilibria exist (see Proposition and simulation results Section However, for uneven market reputations (Appendix B.I. we must turn to mixed strategy equilibria where model providess choose distributions over F.
Quality of market outcome for users. We are interested in studying the quality of market outcome f= f2......., fmm in terms of user utility. The quality of is determined by the overall social loss that it induces on the user population, after users choose between model- providero
SL(f fm) = (x))
(2)

---

When fm is Nash equilibrium, we refer to SL(f1.... fm) as the equilibrium social loss.
Our goal is study how the equilibrium social loss changes when the representation quality (i.e., the quality of the input representations X) improves. We formalize representation quality as the minimum risk single that single model-provider could have achieved on the distribution 'D with the model family F. This means that single is equal to the Bayes risk:
siglle == min E [e(f()), JEF
We show that the equilibrium social loss sL(fi,... fm) can be non- non-monotonic in the representation quality (as measured by SPTsingle), when representations are varied along a variety of axes.
3 Non-monotonicity of Equilibrium Social Loss in a Stylized Setup
To understand when non-monotonicity occur, first consider stylized setup (described below) that permits closed-form calculations the social loss. We characterize the equilibrium social loss in this setup for binary classification (Proposition 2). and apply this characterization to three concrete setups that vary representation quality along different axes (Section 5.2): we show that the equilibrium social loss can be non-monotonic in Bayes risk in all of these setups (Figures 2b 2ch. Finally, we
extend our theoretical characterization in Proposition to setups with more than 2 classes (Section and extend to modee-providers with unequal market reputations (Appendix B.T.
Specification of stylized setup. Assume the input space X is finite and let F= Fall Fuuutii class contain all deterministic functions from X to {0, 1, K 1}. For simplicity, assume that users make noiseless decisions (i.c., C + 0), so a user's choice of model-provider j (xx y) is specified as follows:
if j E arg [in)) 1/y * [x)(
(x,
if j E arg 1 + fy (x)).
(3)
arg minlyeem) [[
In other words, users pick the model-provider with minimum loss, choosing randomly in case of ties. We show that pure strategy equilibria are guaranteed to exist in this setup.
Proposition 1. Let X be a finite set of representations, let there K 2 2 classes, let F =F F-muiti-cllas and lei D be the distribution over (X,Y). Suppose that user decisions noiseless (i.e., user decisions are given by 3). For any m 2 2. there exists a pure strategy equilibrium.
3.1 Characterization of the equilibrium social loss for binary classification
We first focus on binary classification. Let Let biinary denote the function class F_multi-cluss the special case of K = 2 classes. Since F lets model-providers make independent predictions about each representation x, the only source of error is noise in individual data points. To capture this, we define the per-representation Bayes risk a<(z) to be (xx == min (P = 1 z). = 0 x)). The value axx) measures how random the label y is for a given representation z. As a result, a(z) the minimum error that a model-provider can hope to achieve on the given representation
Increasing (<r) increases the Bayes risk sigsne in particular, OPT is equal to the average value [[()(())] across the population. The equilibrium social loss, however, depends other aspects of a(z).
We characterize the equilibrium social loss in terms of the per-representation Bayes risks in the following proposition. Our characterization focuses on pure-strategy equilibria, which are guaranteed exist in this setup (see Proposition D.
Proposition 2. Let X be a finite set, let K 2, and let == Fhinnary Suppose that user decisions are
noiseless (i.e., user decisions are given by B Suppose also that a(x) # 1/m for all E X Ar
any pure strategy Nash equilibrium fr fi..... fnm the social loss SL(ff,-..,. ftm is equal to:
SL
fth) E __ la(x) I[((())( < 1/m]].
(a.a)>
The primary driver of Proposition is that as the per-representation Bayes risk (xx decreases, the equilibrium predictions for x go from heterogeneous (different model model-providers offer different
When a(z) 1/m, there turn out be multiple pure-strategy equilibria with different social losses.
5

---

2.0 Equilibruum social lo58
Equilbrrium social lose
Equiitbrium social loss
Royes
0.4
m-4
n--13
0.006
n
n=100
0.004
0.010
m=500
0.002
m-03
0.005
mn:20
0.0
0.000
0.0
0.5
1.5
2.0
Per rerreseprsenttionn Bayes
Representation
Representation dimension
(a)
(b)
(c)
Equilibruum social loss
0.4
0.20
Equilibrium social
0.4
0.3
0.15
0.2
Equilibrium
0.10
0.2
0.1
0.05
0.1
0.2
0.3
0.4
0.5
2 Representation dimension D
1
Per-representation Bayes risk a
Representation noise o²
Figure 2: Equilibrium social loss (y-axis) versus data representation quality (X-axis) given mn model-providers, for different function classes FF (rows) and when representations are varied along different aspects (columns). Top row: F Johinary with closed- form formula from Proposition 2 Bottom row: linear functions, computed via simulation (Section F. We vary representations with respect to per-representation Bayes risk (a.d), noise level (b,e), and dimension (c.f). The dashed line indicates the Bayes risk (omitted if it is too high to fit the axis). The Bayes risk is monotone, but the equilibrium social loss is non-monotone.
predictions for x)) to homogenous (all model- offer the same prediction for 2²). In particular, if (x())) is helow 1 //m, then all model-providers choose the Bayes optimal label y" = arg max, Pfy/ a), so predictions are homogeneous; on the other hand, if (<xx) is above 1/m, then at least one model- y so predictions are heterogeneous. When predictions are heterogeneous, each user is offered perfect predictive accuracy by some model model-provider,, which results in zero social loss. On the other hand, predictions are homogeneous and all model-providers choose the Bayes optimal label, the social loss on is the per-representation Bayes risk a(x).. Putting this all together, the equilibrium social loss takes the value in We defer a proof of Proposition to Appendix
provider will choose
3.2 Non- Non-monotonicity along several axes of varying representations
Using Proposition 2 we next vary representations along several axes and compute the equilibrium social loss, observing non-monotonicity in each case.
Varying the per-representation Bayes risks. Consider a population with single value of x with Bayes risk a(x) a. We vary representation quality by varying a from 0 to 0.5. Figure depictss the result: by Proposition 2 the equilibrium social loss zero if a > 1/m and a model-providers, the equilibrium social loss is thus non-monotonic in a i As m increases, the non-monotonicity occurs at higher data representation quality (a lower Bayes risk).
Setting 1:
1/m, leading non-monotonicity at a 1/m. When there
Varying the representation noise. Consider a one-dimensional population given by a mixture of two Gaussians (one for each class), where each Gaussian has variance o² (see Appendix C for the details of the setup). We vary the parameter a to change the representation quality. Intuitively, well-separated, which improves representation quality
Setting 2:
lower value of makes the Gaussians (Bayes risk). By Proposition 2 the equilibrium social loss is E [a((z) [[(()) < 1/m]]- For each value of estimate the equilibrium social loss by sampling representations from the population and taking an average 5 Figure depicts the result: the equilibrium social loss non-monotonic in o
'For = 2. where 1/2 the maximum possible perrrepresentation Bayes risk. the equilibrium social loss monotone a.
'Strictly speaking, we can't directly apply Proposition 2 to this setup since X is infinite. We circumvent this issue applying Proposition 2 on sample of the representations.

---

(and thus the Bayes risk). Again, as the number of model provid-po increases, the non monoooootyyyctty occurs at higher representation quality (a lower Bayes risk).
Setting 3: Varying the representation dimension. We consider a four-dimensional population
(Xall, Y), and let the representation X consist of the first D coordinates of X for D varying from 0 to 4 (see Appendix for full details). Intuitively, a higher dimension 1) makes the representations more informative, thus improving representation quality (Bayes risk). As before, for each value of D, we estimate the equilibrium social loss by sampling representations æ from the population and taking an average. Figure 2c depicts the result: the equilibrium social loss is once again nn-monotonic in the representation dimension D (and thus the Bayes risk).
Discussion. Settings 1-3 illustrate that equilibrium social loss can be non-monotonic in Bayes risk when representations are improved along many different axes. The intuition is that varying representations along these (x) can lead to non-monotonicity in the equilibrium social loss. We revisit Settings 1-3 for richer market structures (Appendix B.I and for linear predictors and noisy user decisions (Section
increase a(x:) for inputs ær; by Proposition 2 these changes to
3.3 Generalization to more than 2 classes
While our analysis has thus far focused on classification with K = 2 classes, the number of classes K" can be much larger in practice. As motivating example, consider content recommendation tasks where each class represents a different genre of content; since the content landscape can be quite diverse, we would expect K to be fairly large. This motivates to extend our theoretical characterization in Proposition 2 to classification with K 2 2 classes.
For the case of K > 2 classes, the appropriate analogue of the per-representation Bayes risk is the perelaas-perrrepessenattiiin Bayes risk, defined to be a² (z) == Pyy = i x) for each x E X and i E 10. K - 1}. As result, maxosig a (x) is the minimum error that model- provider can achieve on and OPT is equal to the average value E[1 maxosisK- o (æ)] across the population. The equilibrium social loss, however, depends on other aspects of the o (x) values.
We characterize the equilibrium social loss in terms of the per-class-per-represss per-reprssenntaioon Bayes risks in the following proposition. Our characterization again focuses on pure- stratey equilibria, which are guaranteed to exist in this setup by Proposition
Proposition 3. Let X be a finite set, let K let J Fall cluuti- Suppose that user decisions are noiseless (i.e., user decisions are given by G.. c mingex maxosick- a (z). Then, any pure strategy Nash equilibrium "I,...,Jiv the social loss sL(Ji,,... Im) is bounded as
K
E (w.
-a'(z)-
d((x)
= m
sL(fi ..... ft)
E (a.y)~
aa'(2)
a
m
(5)
The high-level intuition for Proposition is similar to the intuition for Proposition except that each class needs to considered separately. In particular, when class occurs sufficienlly frequently for the representation (i.e., when a (xx is not too small), then some model-prove model-provider will label é; on the other hand, if the class i occurs very infrequently for æ, then no model-provider will label ar as We defer proof of Proposition
Appendix [
While Proposition 3 is conceptually a generalization of Proposition 2 the details of Proposition 3 slightlly differ. In particular, Proposition does not completely pin down the equilibrium social loss, and there a factor of e slack in the constraint on each a (xx in ( between the upper and lower bounds. Nonetheless, since the value minimum accuracy of the Bayes optimal predictor across all inputs x. we expect that "reasonable" representations (i.e., sufficiently informative representations) would have eqqul constant. When c is a constant, there at most a constant factor slack in the a² ax(x) constraints in 5 between upper and lower bound.
mingg X max a ('(2) measures the
For similar reasons to Proposition Proposition 3 implies that the equilibrium social loss can be non-monotonic in representation quality (i.c., Bayes risk). We defer discussion to Appendix B.2
7

---

Rayos risk
0.03
m-4
m-5
Bayes risk
0.02
m=8
0.065
Equilbbium
0.060
0.01
0.055
0.050
resnet18 resnet34 vgg16 alleneet resnet50
resnet18 resnet34 vgg16
alexnet resnet50
representations with varying Bayes risks
representations with varying Bayes risks
(b)
Bayes risk
m-4
0.095
0.04
m=5
Bayes risk
0.090
Equillerium
0.03
m=8
0.02
0.01
0.075
resnet18 vgg16
alexnet resnet34 resnet50
resnet18 vggl6 alexnet resnet34 resnet50 representations with varying Bayes risks
representations with varying Bayes risks
Figure Equilibrium social loss (left) and Bayes risk (right) binary (top) and multi-class (hottom) classification tasks on CIFAR-10 (Section 4.3). Representations are generated from different networks pre- trained on ImageNet. The points show the equilibrium social loss when m model-providers compete with each other (left) and the Bayes risk of single model-provider in isolation (right). While Bayes risk is decreasing in this representation ordering, the equilibrium social loss is non-decreasing in this ordering. The equilibrium social loss thus non-monotonic representation quality as measured by Bayes risk. Error bars are standard error.
4 Empirical Analysis of Non-monotonicity for Linear Predictors
We next turn to linear predictors and demonstrate empirically that the social welfare can be non- monotonic in data representation quality this setup as well. take
RRD and we let the
model parameters be Φ. For binary classification, we let Jbinary the family of linear predictors fw.b = sigmoid (w, z) + b) where w E IR, E R, and o = [w, b). Similarly, for classification with more than classes, we let Ji class be the family of linear predictors fw. = softmax [ + 6) where w E R E R/YI, and & = [W. b). Since this setting no longer admits closed-fome formulae, we numerically estimate the equilibria using variant of best-response dynamics, where model-providers repeatedly best-respond to the other predictors.
We first show on dow-dimensional synthetic data binary classification task that the insights from Section B.2 generalize to linear predictors (Figures 2d 21 We then turn to natural data, considering binary and 10-class image classification tasks for CIFAR- 10 and using pretrained networks-Alexte VGG16, and various ResNets-te to generate high-dimensional representations (ranging from 512 to 4096). We again find that the equilibrium social loss be non-monotonic in Bayes risk (Figure B
4.1 Best-response dynamics implementation
To enable efficient computation, assume the distribution D corresponds to finite dataset with N data points. We calculate equilibria using an approximation of best-response dynamics. Model- providers (players) iteratively (and approximately) best-respond to the other players' actions. We implement the approximate best-response as running several steps of gradient descent.
In more detail, for each j E [m]. we initialize the model parameters standard deviation a. Our algorithm then proceeds in stages. At given stage, we iterate through the
mean zero Gaussians with
in the order 1,...,m. When j is chosen, first we decide whether to reinitialize:

---

if the risk E ((aa))=D(((fa [f (2))).(()) (2), y)] exceeds threshold p. we re-initialize wy and by (sampling from mean zero Gaussians as before); otherwise, we do not reinitialize. Then we run gradient descent on uf:: )-) (computing the gradient on the full dataset of N points) with learning rate for T iterations, updating the parameters Φ. We run this gradient descent step up to more times if the risk (( ( (x)), v'] exceeds a threshold p'. At the end of stage, the stopping condition is that for every j E [m²]
j's utility u(f,, f has changed by at most relative to the previous stage. If the stopping condition is not met, we proceed to the next stage
4.2 Simulations on synthetic data
We first revisit Settings 1-3 from Section 5.2 considering the same axes of varying representations and
distributions over (I. y). In contrast to Section 3.2
we restrict the model family to linear predictors
Foimary instead of allowing all predictors J.hinary We also set the noise parameter in user decisions to 0.3. Our goal is to examine if the findings from Section generalize to this new setting.
We compute the equilibria for each of the following (continuous) distributions follows. First, we let D be the empirical distribution over N = 10, 000 samples from the continuous distribution. Then we run the best-response dynamics described in Section 4.1 with p p' 1, 5000, 17 0.1, = 0.001, and a = 1.0. We then compute the equilibrium social loss according to Q We also compute the Bayes optimal predictor with gradient descent. See Appendix for full details.
Our results, described below, are depicted in Figures 2d 2r (row 2). We compare these results with Figures 2a 2c
(row 1), which shows the analogous results for J-binary from Section 3.2
Setting 1: Varying the per-representation Bayes risks. Consider the same single setup in
Setting in Section 3.2 The only parameter of the predictor is the bias E R (i.e., we treat æ as
zero-dimensional), Figure 2d shows that the equilibrium social loss is non-monotonic in a, which mirrors the non-monotonicity in Figure 2a
Setting 2: Varying the representation noise. Consider the same one-dimensional mixture-of- Gaussians distribution as in Setting in Section 3.2 (The weight we is one-dimensional. We again vary the noise to change the representation quality. Figure 2e shows that the equilibrium social loss is non-monotonic in the noise o, which again mirrors the non non-monotonicity in Figure 2b
Setting 3: Varying the representation dimension. Consider the same four-dimensional population as in Setting in Section 3.2 We vary the representation dimension D from 0 to 4 to change the representation quality. Figure A shows that the equilibrium social loss is non monooonoo in the
dimension D, which once again mirrors the non-monotonicity in Figure 2c
Discussion. In summary, in Figure rows and 2 exhibit similar non-monotonicities. This illustrates that the insights from Section translate to linear predictors and noisy user decisions.
4.3 Simulations on CIFAR-10 for binary classification
We next turn experiments with natural data. While we have directly varied the informativeness of data representations thus far, representations in practice are frequently generated by pretrained models. The choice of the pretrained model implicitly influences representation quality, as measured by Bayes risk on the downstream task. In this section, we consider how the equilibrium social loss changes with representations generated from pretrained models of varying quality. We restrict the model family to linear predictors J bhinary and set the noise parameter in user decisions to 0.1.
We consider a binary image classification task on CIFAR-10 10 Krizhevsky 2009] with 50,000 images. Class 0 is defined to be (airplane, bird, automobile, ship, horse, truck) and the class is defined to be (cat, deer, dog, frog). We treat the set of 50,000 images and labels as the population of users, meaning that it is both the training set and the validation set Representations are generated from five
models-Alex [Krihevsky et al. 2012), VGG16 (Simonyan and Zisserman, 2015 ResNet18. 8, ResNet34, and ResNet50 [He et al.
2016 pretrained on ImageNet [Deng et al. 2009 The
'The code can be found at //thttts:///giii htt.tttttthttteegggg www.commmmmm.. attaaaadddeeeeooopeaaaaaeeeeeeee wwwtioonnooooootooooo nonmonotonicitt city We make this choice to be consistent with the rest of the paper, where we focus on population- level behavior and thus do not consider generalization error.

---

representation dimension is 4096 for AlexNet and VGG16, 512 for ResNet18 8 and ResNet34, and 2048 for ResNet50.
We compute the equilibria as follows. First, let D be the distribution described above with N = 50, 000 data points. Then we run the best-response dynamics described in Section 4.1 for {3, 4.5, 5, 6, } mooeedproviders with p = o' = 0.3, I 2000, = 0.001, σ = 0.5, and a learning rate schedule that starts at 17- 1.0. We then compute the equilibrium social loss according to We also compute the Bayes risk using gradient descent. For experimental details, see Appendix
Figures Ba 3b show that the equilibrium social loss can be mon-monteoee in the Bayes risk. For example, for m = 3, VGG16 outperforms AlexNet, even though the Bayes risk of VGG16 is much higher than the Bayes risk of AlexNet. Interestingly. the location of the non- non-monotonicity differs across different values of m. For example, for m = 5 and m = 8, Aetx outperforms ResNe150 despite having higher Bayes risk, but ResNet50 outperforms Alex for m 3 and m 4.
4.4 Simulations on CIFAR-10 for 10-class classification
While our empirical analysis has thus far focused on binary classification, we now turn to classification with more than 2 classes. In partieular, we consider with 50,000 images. The labels are specified by the CIFAR-I0 classes in the original dataset. We treat the set of 50,000 images and labels as the population of users, meaning that it is both the training set and the validation set. Representations are generated from the same five models as in Section 4.3 We restrict linear predictors Jauutt-class and again the noise parameter in user decisions D to 0.1.
class CIFAR-10 (Krizhvssyy 2009) task
We compute the equilibria. We again let D be the distribution described above with NN = 50, 000 data points. Then, we run the best-response dynamics described in Section 4.1 for E {3, 4, 6, 8). and we compute the equilibria with the same hyperparameter settings as in Section 4.3 (except that
P 0.7 and p' 1). We then compute the equilibrium social loss according to the Bayes risk using gradient descent. For full experimental details, see Appendix
2 We also compute
Figures 3c 3d show that the equilibrium social loss can be non-monotone in the Bayes risk. For example, across all five values of m, ResNet18 outperforms VGG16, even though the Bayes risk of ResNet is substantially higher than the Bayes risk of VGGI6. Furthermore, for m = 3, VGG16 outperforms AlexNet despite having a larger Bayes risk. Interestingly, the shape of the equilibrium social loss curve for each value of m (Figure BC appears qualitatively different than the analogous cquilibrium social loss curve for binary classification (Figure Ba).
Discussion
We showed that the monotonicity of scaling trends can be violated under competition. We demon- strated that when multiple model-providers compete for users, improving data representation quality (as measured by Bayes risk) can increase overall loss at equilibrium. We exhibited non non-monotonicity of the equilibrium social loss in Bayes risk when representations are varied along several axes (per- representation Bayes risk, noise, dimension, and pre-trained model which generates representations).
An interesting direction for future work is to further characterize the regimes when the equilibrium social loss is monotonic versus mon-monotoiic in data representation quality as measured by Bayes risk. For example, it would be interesting to generalize our theoretical results from Section D to general function classes and market reputations as well as to generalize our empirical findings from Section to other axes of varying data representations and to non-linear functions. Finally, while we focused on classification, an interesting direction would be to generalize our findings to regression tasks with continuous outputs or to generative AI tasks with text-based or image-based outputs.
More broadly, the non-monotonicity of equilibrium social welfare scale under competition estab- lishes a disconnect between scaling trends in the single model- model-provider setting and in the competitive setting. In particular, typical scaling trends (e.g. (Kaplan et al., 2020 (Sharma and Kaplan 2020 Bahri et al. 2021 Hoffmann et al,, 2022 Hernandez et al. 2021)) may not translate to competitive settings such as digital marketplaces. Thus, understanding the downstream impact of scale on user welfare in digital marketplaces will likely require understanding how scaling trends behave under competition. We hope that our work serves as starting point for analyzing and eventually characterizing the scaling trends of learning systems in competitive settings.
10

---

Acknowledgments
We thank Yiding Feng, Xinyan Hu, and Alex Wei for useful comments on the paper. This work was partially supported by the Open Phil AI Fellowship, the Berkeley Fellowship, the European Research Council (ERC) Synergy Grant program, and the National Science Foundation under grant numbers 2031899 and 1804794.
References
Simon P. Anderson, Andre de Palma, and Jacques-Fraiee Francois Thisse Discrete Choice Theory of Product Differentiation. The MIT Press, 10 1992.
Guy Aridor, Yishay Mansour, Aleksandrs Slivkins, and Zhiwei Steven Wu. Competing bandits: The perils of exploration under competition. CoRR, abs/2007.10144, 2020.
Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. Explaining neural scaling laws. CoRR, abs/2102.06701, 2021.
Omer Ben-Porat and Moshe Tennenholtz. Best response regression, In Advances in Neural Infor- mation Processing Systems 30: Annual Conference Neural Information Processing Systems (NIPS), pages 1499-1508, 2017.
Omer Ben-Porat and Moshe Tennenholtz. Regression equilibrium. In Proceedings of the 2019 ACM Conference on Economics and Computation (EC), pages 173-191. ACM, 2019.
Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 3588) 2013.
Rishi Bommasani, Kathleen A. Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang. Picking on the same person: Does algorithmic monoculture lead outcome homogenization? In NeurlPS, 2022.
Michael Brückner, Christian Kanzow, and Tobias Scheffer. Static prediction games for adversarial learning problems. JMLR, 13(1):2617-2654, 2012.
Emilio Calvano and Michele Polo. Market power, competition and innovation in digital markets: A survey, Information Economics and Policy, 54:10085; 2021.
Micah D. Carroll, Anca D. Dragan, Stuart Russell, and Dylan Madfield-Menede Estimating and penalizing induced preference shifts recommender systems. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2686-2700 PMLR, 2022.
Shih-fen Cheng, Daniel M. Reeves, Yevgeniy Vorobeychik, and Michael P. Wellman. Notes on equilibria in symmetric games. In Proceedings the 6th International Workshop Game Theoretic and Decision Theoretic Agents (GTDT), pages 71 71-78, 2004.
Mihacla Curmei, Andreas A. Haupt, Benjamin Recht, and Dylan Hadfield-Menell. Towards psycholegically-gronndd dynamic preference models. In Proceedings of the 16th ACM Conference on Recommender Systems, RecSys '22, page 35-48. 2022.
C. d' Aspremont, J. Jaskold Gabszewicz, and J.-F. Thisse On hotelling's "stability in competition". Econometrica, 47(5): 1150, 1979. ISSN 00129682, 14680262.
Sarah Dean and Jamie Morgenstern. Preference dynamics under personalized recommendations. In
David M. Pennock, Ilya Segal, and Sven Seuken, editors, EC '22: The 23rd ACM Conference on Economics and Computation, Boulder, CO, USA, July - 15, 2022, pages 795-816. ACM, 2022.
Sarah Dean, Mihaela Curmei, Lillian J. Ratliff, Jamie Morgenstern, and Maryam Fazel. Multi-learner risk reduction under endogenous participation dynamics. CoRR, abs/2206.0266 2022.

---

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pages 248- 248-255. IEFF Computer Society, 2009.
Yiding Feng, Ronen Gradwohl, Jason D. Hartline, Aleck C. Johnsen, and Denis Nekipelov. Bias- variance games. CoRR, abs//199,,,318, 2019.
Tony Ginart, Eva Zhang, Yongchan Kwon, and James Zou. Competing Al: how does competition feedback affect machine learning? In Arindam Banerjee and Kenji Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics (AISTATS), volume 130 of Proceedings of Machine Learning Research, pages 1693-1701, 2021.
Moritz Hardt, Nimrod Megiddo, Christos Papadimitriou, and Mary Wootters. Strategic classification. In Proceedings of the 7th Conference on Innovations in Theoretical Computer Science (ITCS), pages 11 11-122, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In Jennifer Dy and Andreas Krause, editors, Pro- ceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1929-1938. PMLR, 10-15 Jul 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770-778. IEEE Computer Society, 2016.
Danny Hernandez, Jared Kaplan, Tom Henighan, and Sam McCandlish. Scaling laws for transfer. arXiv preprint arXiv:2102.0129)) 2021.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models. CoRR, abs/2203. 15556, 2022.
Harold Hotelling. Stability in competition. Economic Journal, 39(1 53):41-57, 1981.
Nicole Immorlica, Adam Tauman Kalai, Brendan Lucier, Ankur Moitra, Andrew Postlewaite, and Moshe Tennenholtz. Dueling algorithms. In Proceedings of the 43rd ACM Symposium on Theory of Computing (STOC), pages 215-224, 2011.
Ganesh Iyer and T. Tony Ke. Competitive algorithmic targeting and model selection. Available SSRN: https://ssm.cont https://ssm..co/aastraacaat https//mmmoommmastat=44111111111 4973, 2022.
Meena Jagadeesan, Michael I. Jordan, and Nika Haghtalab. Competition, alignment, and equilibria in digital marketplaces. CoRR, abs/2208. 14423, 2022.
Bruno Jullien and Wilfried Sand-7antman. The economics of platforms: A theory guide for competi- tion policy. Information Economics and Policy, 54:100880, 2021.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford. Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.08361. 2020.
Jon M. Kleinberg and Manish Raghavan. Algorithmic monoculture and social welfare. Proc. Natl. Acad. Sci. USA, 1 118(2))e2018340018 2021.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolu- tional neural networks. In Peter L. Bartlett, Fernando C. Pereira, Christopher J. C. Burges, Léon Bottou, and Kilian Q Weinberger, editors, Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012, Proceedings of meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States, pages 1106-1114, 114, 2012.
12

---

Yongchan Kwon, Tony Ginart, and James Zou, Competition over data; how does data purchase affect users? Trans. Mach. Learn. Res., 2022.
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässann Stockholm, Sweden, July 10-15, 2018, volume 80 Proceedings of Machine Learning Research, pages 3156-3164. PMLR, 2018.
Lydia T. Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer T. Chayes. The disparate equilibria of algorithmic decision making when individuals invest ratio- nally. In Mireille Hildebrandt, Carlos Castillo, L. Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna, editors, FAT '00: Conference on Fairness, Accountubility; and Transparency, Barcelona, Spain, Junuary 27-30, 2020, pages 381-391. ACM, 2020.
Dov Monderer and Lloyd S. Shapley. Potential games. Games and Economic Behavior, 14(1): 124 223,143,7 1996.
Robert W. Rosenthal. A class of games possessing pure-strategy nash equilibria. Int. J. Game Theory, 2(1):65-67, dec 1973.
Utkarsh Sharma and Jared Kaplan. neural scaling law from the dimension of the data manifold. CoRR, abs/2004. 10802, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. 2015.
Kenneth Train. Discrete Choice Methods with Simulation. Cambridge University Press, New York, 2002.
13


Reviews:
[{"review_id": "nov0iNN6j1", "reviewer": "Reviewer_4Cpq", "summary": "The authors consider a setting where there exist multiple provides of a machine learning model, and all together choose to increase their representation quality. The authors show, via both theory and experiment, that this improvement in data representation does not necessarily translate to improved social welfare. In particular, the authors shows that, due to their model of competition,  when data representation qualities increases, there is less specialization in that the models are unlikely to differ significantly in providing strong performance for a certain group of users, and the equilibrium achieved may be worse off in terms of overall loss each user received. The authors present empirical results on CIFAR-10 using varying pretrained representations, as well as a generalization of their theory to situations where users break tied based on model-provider reputation. ", "questions": "\n- What are the bars on Figure 3? Standard error or CIs? \n\n- Did the authors try more values of m? Is there a trend to observe over m? Currently, it is hard to make an overall statement just comparing m=8 and 12. \n\n- Line 263: Should this say initialize randomly? How important is this choice?", "limitations": "There is essentially no limitations section in the paper, which I feel is important given the many assumptions used for overall toy setting that aren't reflected in the real world the authors motivate the work with. While the authors did address prior work limitations by generalizing to a setting where users are influenced by model provider reputation, even this is only accounted for in the tie-breaking scheme and I think the overall user choice setting is quite idealized. \nFurthermore, in practice users would likely have several different inputs, and pick the model that provides low loss on the first few inputs they try with - this suggests consider the distribution of inputs different users care about, which the authors' model doesn't study. \n\nI think the authors should state their assumptions very clearly, and explain why running additional experiments / analyzing settings without these assumptions are out of scope for this work. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, I think the paper studies a very interesting setting (multiple machine learning providers in competition), that is increasingly relevant these days (though I think any model of this should at least strive to capture the more realistic setting in that some players, like Google/OpenAI/Apple are clearly very dominant in their respective spaces, and at times might be seen more as monopolies,  and not just from a market-reputation `tie-breaking' angle). \n\nThe paper is also well-written and easy to follow, and has a good mix of theoretical and empirical results. \n\n", "weaknesses": "My main concerns with the paper are lack of discussion on the assumptions needed, and confusion about the central takeaway of the work re: its relationship to the scaling law literature. \n\nW1 Scaling Laws / Takeaway: Even after a few reads, the framing of the work as a contrast to current works on scaling laws that only focus on one model feels a bit arbitrary, and it seems a bit unfair to say the focus on one-model is a \"limitation\" of work on scaling laws. I don't agree that the purpose of scaling law research is to make a statement on social welfare, and rather to precisely characterize phenomena that change as you increase model size / parameters. I think focusing on clean measures like loss as # parameters / depth / etc. are important to rigorously do, as for a lot of research in the field its purpose is to shed some empirical insights on the model and training processes themselves.  Whereas this work opens up so many questions about the right way to model social welfare, user choice, model provider decisions, etc., I feel like it is a big departure from the intention of most research studying scaling trends. \n\nI think at minimum, the authors should try to soften claims where they draw this contrast, such as the first two sentences of the abstract, that almost seem to imply that the finding that accuracy improves with scale is only due to the one-model, when what the authors measure (social welfare) is anyways different from accuracy. I am mainly concerned that this paper runs the risk of misleading general readers who might just get the takeaway \"we shouldn't try to increase model size\". That might be an attention-grabbing claim, but I don't think the small-scale setting the authors study warrants such a claim - at best, it opens up the need for more research on analyzing competition in the \"large-scale machine learning models\" space. \n\nTherefore, I think the authors should also think carefully what the central takeaway of the paper really is. Even if the authors stick with the contrast with scaling trends as a broad framing, I'm still not sure its too strong/surprising of a finding.  \n\nW2: Why is keeping the data constant a realistic assumption to make? None of the cited examples to motivate the paper (chatGPT vs. Claude, Spotify v. Pandora) have this property, particularly when you consider that their data is reliant on the users who use their serve, as well as methods like instruction-tuning and RLHF that are more proprietary and key to the mass deployment of these models. in the LM space, we can see this with the development of products like CharacterAI. In a competitive setting, if model providers could change the data they use, wouldn't that be an action they could take to provide unique services that would break the setting studied by the authors? \nIt seems like the purpose of the assumption is only to be able to focus on the scaling trends, which goes back to W1. But at the least, this assumption should be discussed. \n\nW3: I think the reasons behind why increasing data quality representations leads to models that are \"likely to agree with each other on most data points\" should be clearly stated. Isn't one aspect important for social welfare whether it is a particular kind of user subgroup that is hurt by all models (related to the cited work on homogenization)? The data representation methods studied (e.g. varying the representation dimension, representation noise)  are all pretty synthetic that don't result in semantically meaningful improvements (e.g. over a subgroup) in a way more realistic improvements (increase deep NNs model parameters) might. \n\nW4. While I think the generalization to asymmetry in the market where model-provider reputation matters is a great addition, this still feels somewhat limited (only executed through the form of weighting when breaking ties). People choose models based on a variety factors including ease of service, its performance over the generally distribution of inputs the user expects to query with, etc. -- why pick just reputation in a limited way to study? Again, I think these limitations should at least be stated. \n\nMinor\n\n- I think more experiment details should be included in the main paper, such as calculating the Bayes risk using gradient descent from pretrained representations for the CIFAR10 experiments. Since the paper's message seems intended to capture a setting that is only really arising due to large, pre-trained models that afford high-quality data representations, I think these experimental results are important to be clear to the community about, and I might suggest even including another experiment in the language domain. \n\n- The abstract could hint at the punchline \"why\" for the decrease in social welfare (the fact with low quality representations there is a more  heteregenous set of choices)\n\n- typo in line 162", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We present **new results** that address some concerns and also respond to other concerns.  \n\n**“Additional experiments / analyzing settings without these assumptions.”** \n\nWe provide new CIFAR-10 experiments on 10 classes (Figure N1, General Response) and for local improvements to data representations (Figure N2, General Response). We consistently observe non-monotonicity, which provides evidence that our conclusions are robust to different assumptions/settings. \n\n**“Did the authors try more values of m? Is there a trend to observe over m?”** \n\nWe computed additional values of $m$ for the original binary classification task (see Figure N3 in General Response pdf). Non-monotonicity is consistently observed. Interestingly, the curve shape changes with $m$, which aligns with our theoretical findings (Proposition 1). \n\n**“What the central takeaway of the paper really is.”** \n\nThe primary takeaway (lines 35-36) is that improvements to data representation quality can decrease the *overall predictive accuracy* for users (i.e., social welfare). A consequence for scaling laws is that scaling trends for predictive accuracy of a single model-provider may not capture scaling behavior for *overall predictive accuracy* for competing model-providers, which is important when considering the impact of large-scale ML systems on society. \n\n**“I feel like it is a big departure from the intention of most research studying scaling trends.”** \n\nOur work aims to bring new perspectives from the emerging discourse on platform competition and policy in digital marketplaces (e.g. Jullien and Sand-Zantman, 2020) into the analysis of scaling trends. The prevalence of competition in today’s multi-platform settings motivates studying scaling trends under competing model-providers, which indeed goes beyond the scope of most research on scaling laws. \n\nAs to why we focus on social welfare (which differs from typical metrics in the scaling laws literature), we believe that social welfare (i.e. “overall predictive accuracy”) is a more natural metric for the multiple model-providers setting. “Overall predictive accuracy” is studied in prior work on competing predictors (e.g. Ben-Porat and Tenneholtz [2017, 2019]) and captures how well the marketplace serves users, which matters for society (e.g., Kleinberg and Raghavan 2021). \n\nOur results provide some evidence that the impact of increased scale can significantly differ in multi-platform and in single-platform settings, making the study of how competition impacts scaling trends an important avenue of future research. \n\n**“I don't agree that the purpose of scaling law research is to make a statement on social welfare. It seems a bit unfair to say the focus on one-model is a \"limitation\" of work on scaling laws.”** \n\nWe agree with the reviewer that the focus on a single model is not a “limitation” of scaling laws and we do not intend to claim that anywhere in the paper. We will make this clear in the final version and clarify that we study a different setting (competing model-providers) and a different metric (social welfare) than the typical scaling laws literature. If the reviewer has a specific sentence in mind, please let us know which one. \n\n**“This work opens up so many questions about the right way to model social welfare, user choice, model provider decisions. Many assumptions used for overall toy setting that aren't reflected in the real world.”** \n\nWe agree that the multi-platform setting is inherently messier to model than the single platform setting. Our work is the first to study how competition between model-providers impacts social welfare, and introducing simplifying assumptions was necessary to make progress on analyzing the equilibria. Despite these assumptions, we expect the primary takeaway of our work (that social welfare can be non-monotonic in improvements to data representations) to be robust. \n\n**“There is essentially no limitations section in the paper.”** \n\nWe will include a more thorough discussion of our assumptions in the final version. We hope that our results inspire future work that incorporates additional complexities (e.g., richer user choice models) and that carries out additional empirical evaluation (e.g., for language models). \n\n**“I think the reasons behind why increasing data quality representations leads to models that are \"likely to agree with each other on most data points\" should be clearly stated.”**\n\n For high-quality representations, a Bayes optimal predictor is incorrect only on a small set of users (call this set $S$). Intuitively, any predictor that correctly labels users in $S$  would mislabel many other users (e.g., if the Bayes optimal label for $x$ is $1$, correctly labeling users with $(x, 0)$ requires mislabelling the many other users with $(x, 1)$). This could happen in practice when representations fail to encode a predictive feature. If representation quality is high, a model-provider optimizing for market share would choose the Bayes optimal label, regardless of other models’ providers actions (e.g., if $p(y = 1|x) = 0.9$ and $m=3$, it’s always better to choose $f(x)=1$ and get $\\ge 0.33$ market share than choose $f(x) = 0$ and get $\\le 0.1$ market share). This suggests model-providers will \"agree\" on many data points at equilibrium. With lower quality representations (e.g., $p(y = 1|x) = 0.6$), the tradeoff would change, which could lead to “disagreement”. Proposition 1 mathematically formalizes this intuition. \n\n**“I think more experiment details should be included in the main paper.”** \n\nWe will add more experiment details to the final version, by expanding upon and moving some of Appendix B into the main body. \n\n**“Line 263: Should this say initialize randomly?”** \n\nWe initialized to a $0$-mean gaussian with stdev 0.1. The stdev needs to be sufficiently high for convergence of best-response dynamics. \n\n**Bars on Figure 3:** \n\nBars are based on standard errors. "}, {"review_id": "byKN5GLBOZ", "reviewer": "Reviewer_5U8U", "summary": "This paper shows that \"social welfare\" might (under some settings) be worse-off as the representation quality of machine learning models increase when there are multiple strategic machine-learning-model providers. \n\nThe paper focuses the analysis on binary classification tasks. The paper assumes that users choose the model provider which gives the most accurate prediction. And multiple strategic machine-learning-model providers compete for the number of users choosing them. The social welfare is defined as the difference between the most accurate model prediction and the true label in expectation. The representation quality is defined as the Bayes risk. The paper shows that under a stylized setting, increasing the representation quality might decrease the social welfare. \n\nThen many empirical evaluations are conducted to show that this also happens in several relaxed settings. \n\n\n\n\n", "questions": "See weaknesses. ", "limitations": "To me, the authors adequately addressed the limitations. ", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and easy to follow. \n\nThe paper tries to answer an interesting question of what is the effect of representation quality on social welfare when there are competing machine-learning-model providers. \n\nThe paper provides theoretical analysis on a stylized setting which shows an intriguing phenomenon: when the representation quality increases, the defined social welfare can decrease. And it also provides empirical evaluations to show this phenomenon beyond the restricted setting. This calls for further discussion on whether this happens more broadly and how to address this problem. ", "weaknesses": "My major concern about this paper is that all the theoretical analysis and empirical evaluation are conducted on settings where all model providers have the same representation. In reality, competing providers do not share their models and they rarely have the same representations. So the conclusion of this paper is based on evidence on a very restricted setting. \n\nI am not sure whether binary classification is the right problem to analyze the concerned question. First, if there are only two options, the users can simply look at the two options and make a decision. They do not even need to hop between model providers. Second, in the examples given in the paper, there is no single right choice of prediction (e.g. there might be multiple relevant webpages and songs, routes, and chat messages). \n\nMinor:\n\nIn the description of Figure 1, \"in the left plot than the plot plot\" -> \"in the left plot than that in the right plot\". \n\n\n\n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We present **new results** that address some of their concerns and also respond to other concerns below.  \n\n**“I am not sure whether binary classification is the right problem to analyze the concerned question.”** \n\nWe provide new results showing that our empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes, and our theoretical results in Section 3.1 directly extend to multi-class classification with K classes. (Our motivation for studying binary classification was to simplify the exposition and analysis.) \n- In new experiments on CIFAR-10, we consider a data distribution over 50,000 samples and the original 10 classes. We show that the equilibrium social loss is non-monotonic in the data representation quality as measured by Bayes risk (see Figure N1 in the General Response pdf). Figure N1 shows that the empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes. \n- In a new theoretical result, we show that Proposition 1 can be directly extended to multi-class classification, where the function class is all functions $f:X \\rightarrow Y$ where $Y = [K]$. The same insights from Section 3.1 thus carry over to multi-class classification. \n\nFor details of the new experiments and new theoretical result, we refer the reviewer to the \"Extension to classification with more than 2 classes\" section of the General Response. \n\n**\"My major concern about this paper is that all the theoretical analysis and empirical evaluation are conducted on settings where all model providers have the same representation.\"** \n\nWe provide an **additional experiment to study “local improvements”** where each model-provider uses different data representations and only one model-provider changes their data representations. We continue to observe non-monotonicity of the social loss (see Figure N2 in the attached pdf) in a CIFAR-10 setup. In these setups, data diversity also impacts the equilibrium social loss: that is, changing a single model-provider’s data representations can increase (or decrease) the overall diversity in the data representations across model-providers, which we expect would decrease (or increase) the level of monoculture/outcome homogeneity. Understanding exactly how varying data representations impacts data diversity (and equilibrium social loss) is an interesting direction for future work. \n\n**“In reality, competing providers do not share their models and they rarely have the same representations.”**\n\nAs described above, we provide an additional experiment on CIFAR-10 to study “local improvements” where only one model-provider changes their representations. We continue to observe non-monotonicity of the social loss (see Figure N2 in the attached pdf).\n  \nThe reason that we focused on global data representations is to isolate the role of competition on the social loss. In particular, we intentionally designed our model to be as close as possible to the classical single-decision-maker setting (where there is a single data distribution), so that we could directly compare scaling trends in the single decision-maker setup and the competitive setup. Our results show that competition leads to non-monotonicity in the social loss, even if the model-providers are given the exact same data representations and have access to the same function class.\n\nOne real-world motivation for global data representations is that the model-providers all rely on the same pretrained model for representations, but each model-provider fine-tunes their model separately. This also motivates our consideration of model-providers choosing linear functions over the shared representations in our experiments.\n\n"}, {"review_id": "ZFzb8d08z7", "reviewer": "Reviewer_Gqjx", "summary": "This paper looks and trends that arise in predictive utility across a population as model providers adapt to improving representations. At the core of the paper is an analysis that characterizes the population utility of equilibria as the representation varies in quality, noise, and dimension, which shows that overall utility may not be monotone along these axes. The paper additional contains an empirical study, over synthetic data and CIFAR, which addresses some of the simplifications made in the analysis while showing qualitatively similar trends in overall utility.", "questions": "1. If you are able, please explain the rationale behind the global representation used in your work. To what extent do you believe that your results depend on this assumption?\n2. Do you have any insight into the likely outcomes if your experiments were to relax some of the conditions that I mentioned above?", "limitations": "The section 5 at the end of the paper would be a good place to add discussion of the fact that there is certain risk that the results are overly-sensitive to choices made in modeling and experiment design. It currently does not, and limitations are not discussed in much detail elsewhere.", "rating": 7, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Topically, this paper addresses an interesting and potentially important question: what is the right way to model the effect that continued improvements to predictive accuracy have on social welfare? Some examples of prior work have pointed to a misalignment between predictive accuracy and social welfare, but the results in this paper are original and distinct, the the best of my knowledge. Namely, \"model monoculture\" can emerge when competing providers behave rationally, and representations are sufficiently good, leading to a subpopulation that isn't served by any provider.\n\nMany of the concerns that I had about the simplified setting when going through the theoretical analysis were addressed to some extent in sections 3.3 and 4, and it seems like the results may be fairly robust to these choices.\n\nThe writing is polished, clear, and concise.", "weaknesses": "A central assumption of the work is that all model providers have access to the same global representation. This makes some sense, but is not all that realistic: despite using on the same underlying techniques and relying on the same public-knowledge advances in learning, different model providers show very different performance metrics. The market reputation factor incorporated in 3.3 does not quite address this, as it is independent of model performance.\n\nWhile the experimental analysis is a nice demonstration that the theoretical results hold in some more realistic settings, there are still some simplifications that weren't explicitly motivated (only linear classifiers, binarized CIFAR). These choices make sense in that they are consistent with the rest of the paper, but the point of the experimental analysis is to show that the results are robust, so they do limit the significance of the work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We present **new results** that address some concerns and also respond to other concerns.\n\n**“There are still some simplifications that weren't explicitly motivated (binarized CIFAR).”**\n\nWe provide new results showing that our empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes, and our theoretical results in Section 3.1 directly extend to multi-class classification with K classes. (Our motivation for studying binary classification was to simplify the exposition and analysis.) \n\nIn **new multi-class experiments on CIFAR-10**, we consider a data distribution over 50,000 samples and the original 10 classes. We show that the equilibrium social loss is non-monotonic in the data representation quality as measured by Bayes risk (see Figure N1 in the General Response pdf). Figure N1 shows that the empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes. \n\nIn a new theoretical result, we show that Proposition 1 can be directly extended to multi-class classification, where the function class is all functions $f:X \\rightarrow Y$ where $Y = [K]$. The same insights from Section 3.1 thus carry over to multi-class classification. \n\nFor details of the new experiments and new theoretical result, we refer the reviewer to the \"Extension to classification with more than 2 classes\" section of the General Response. \n\nWe also extend our theoretical result (Proposition 1) to classification with K classes. We refer the reviewer to the General Response for details. \n\n**“To what extent do you believe that your results depend on this assumption [the global representations assumption]?”**\n\nWe provide **an additional CIFAR-10 experiment to study “local improvements”** each model-provider uses different data representations and where only one model-provider changes their data representations. We continue to observe non-monotonicity of the social loss (see Figure N2 in the attached pdf). In these setups, data diversity also impacts the equilibrium social loss: that is, changing a single model-provider’s data representations can increase (or decrease) the overall diversity in the data representations across model-providers, which we expect would decrease (or increase) the level of monoculture/outcome homogeneity. Understanding exactly how varying data representations impacts data diversity (and equilibrium social loss) is an interesting direction for future work.\n\n**“If you are able, please explain the rationale behind the global representation used in your work.”**\n\nAs described above, we provide an additional experiment on CIFAR-10 to study “local improvements” where model-provider use different data representations and where only one model-provider changes their data representations. We continue to observe non-monotonicity of the social loss (see Figure N2 in the attached pdf).  \n\nThe reason that we focused on global data representations is to isolate the role of competition on the social loss. In particular, we intentionally designed our model to be as close as possible to the classical single-decision-maker setting (where there is a single data distribution), so that we could directly compare scaling trends in the single decision-maker setup and the competitive setup. Our results show that competition leads to non-monotonicity in the social loss, even if the model-providers are given the exact same data representations and have access to the same function class.\n\nOne real-world motivation for global data representations is that the model-providers all rely on the same pretrained model for representations, but each model-provider fine-tunes their model separately. \n\n**“There are still some simplifications that weren't explicitly motivated (only linear classifiers).”**\n\nLet’s return to the real-world motivation described above, where all model-providers all rely on the same pretrained model for representations, but each model-provider fine-tunes their model separately. We view the restriction to linear functions as a stylized way of capturing finetuning. \nMore broadly, considering nonlinear predictors is interesting, but makes it more challenging to compute the Nash equilibria via gradient-descent-based best response dynamics. \n\n\n**“The section 5 at the end of the paper would be a good place to add discussion of the fact that there is certain risk that the results are overly-sensitive to choices made in modeling and experiment design.”**\n\nUsing the extra space in the final version of the paper, we will add a qualitative discussion of our assumptions that highlights the simplifications in the user choice formalization, market reputations formalization, and model-provider actions formalization. We hope that our results inspire future work that extends our model to relax these assumptions and incorporate additional complexities. "}, {"review_id": "T3lbivt2Vm", "reviewer": "Reviewer_YGiJ", "summary": "The paper explores the impact of competition on machine learning models and social welfare. It investigates how scaling trends can be altered by competition and how improving data representation quality may not always lead to better predictive accuracy. The authors provide a theoretical framework for analyzing the impact of competition on machine learning models and demonstrate their findings through experiments on real-world datasets. The paper makes contributions to the understanding of the relationship between competition, data representation quality, and predictive accuracy in machine learning models.", "questions": "- How does the quality of data representation affect the equilibrium social welfare in the authors' model, and how does this compare to existing models?", "limitations": "Yes.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper provides a comprehensive review of related work and connects to multiple research threads on the welfare implications of algorithmic decisions and competition between data-driven platforms.\n- The authors apply their framework to three concrete binary classification setups that vary representation quality along different axes, demonstrating the usefulness of their approach.\n- The paper is well-written and organized, making it easy to follow the authors' arguments and contributions.", "weaknesses": "- The paper would benefit from a more thorough evaluation of the proposed framework, including experiments on real-world datasets and comparison to existing methods.\n\n- The paper could benefit from a more detailed discussion of the implications of their findings for real-world decision-making scenarios.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. We present **new results** that address some concerns and also respond to other concerns.\n\n**“The paper would benefit from a more thorough evaluation of the proposed framework, including experiments on real-world datasets.**\n\nBeyond the experiments in the paper on binary classification on CIFAR-10 (Section 4.1) and synthetic data (Sections 3.2 and 4.2), we provide **new experiments with CIFAR-10 on 10 classes** (see Figure N1, General Response) and **new experiments for local improvements to data representations** (see Figure N2, General Response). We also have included some **more values of m in the original binary classification task** (see Figure N3 in pdf attached in the General Response). We observe non-monotonicity of the equilibrium social loss in these additional settings. \n\n**“The paper could benefit from a more detailed discussion of the implications of their findings for real-world decision-making scenarios.”**\n\nOur work suggests  that the consistent trend towards increases in scale might not lead to improvements in social welfare (the overall predictive accuracy for users) in today’s multi-platform settings. “Overall predictive accuracy”, which is the typical metric studied in the existing work on competing predictors (e.g. Ben-Porat and Tenneholtz [2017, 2019]), captures how well the overall marketplace of model-providers actually serves users, which is ultimately what is important from a societal perspective (e.g., Kleinberg and Raghavan 2021). The fact that scale may not reliably improve social welfare under competition should inform the emerging discourse on competition policy in digital marketplaces (e.g. Jullien and Sand-Zantman, 2020) and regulation/deployment of large models more broadly. \n\nMoreover, our results highlight the need for the research community to analyze scaling laws in competitive settings such as today’s multi-platform marketplaces. \n\n**“How does the quality of data representation affect the equilibrium social welfare in the authors' model, and how does this compare to existing models?”**\n\nMost of our results—Proposition 1, the results in Section 3.2, the results in Section 4.1, and the results in Section 4.2—aim to study the impact of data representation quality (as measured by Bayes risk) on the equilibrium social welfare. These results illustrate that improving data representation quality as measured by Bayes risk (e.g., from increases to scale) can decrease the *overall predictive accuracy* for users (a measure of social welfare). \n\nAs to how our results compare to existing models, our paper is the first to study the impact of improving data representation quality (Bayes risk) on the equilibrium social welfare. Thus, there are no other clear models to compare our results with. \n\nPlease let us know if this answers your question, we would be happy to clarify this further during the discussion period. \n"}, {"review_id": "C8Tj8Hr6xh", "reviewer": "Reviewer_tyS5", "summary": "This study highlights the inconsistencies of scaling laws in machine learning when model providers compete with each other in the market. The researchers conducted experiments with varying data representations and found that improving data representation quality may actually decrease the overall predictive accuracy, which is modeled as social welfare, across users in the competitive market. They also generalized experiment to unfair markets and study the market with 2 model providers only. The study raises awareness about the importance of evaluating machine learning models in a market where organizations are competing.", "questions": "If the market is competitive, when and why do people care about the social welfare? Organizations are selfish as they only care about earning more share in the market as much as possible. Is there any scenario in practice in machine learning that people care about social welfare more than their own preference or happiness.", "limitations": "This paper presents a novel finding that challenges the conventional understanding of scaling laws in machine learning in the context of organizational competition. The framework proposed is foundational and can be applied to various scenarios.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper presents several strengths, including a clear and well-written introduction that guides the reader to understand their interesting findings and the awareness behind it. Approaches to vary data representation are comprehensive and practical. They provide basic setups to study and offer results empirically and theoretically. Notably, the paper's conclusion offers an interesting insight into why enhancing model accuracy may not necessarily lead to significant social benefits in real-world applications.", "weaknesses": "Here are some potential improvements to the original content:\n\n1. The available training resources of model providers in the market are unclear, as well as the basis on which they build their models. If models are trained solely on user-provided datasets, their performance may not vary significantly, which may make the best response trivial.\n\n2. The experiment only covers CIFAR-10 and reduces it to a binary classification task, which warrants further exploration with datasets of higher cardinality.\n\n3. The finding about inefficient translation to social welfare is interesting, but it lacks proper and thorough explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback.  We present **new results** that address some concerns and also respond to other concerns. \n\n**“The experiment only covers CIFAR-10 and reduces it to a binary classification task, which warrants further exploration with datasets of higher cardinality.”**\n\nWe provide new results showing that our empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes, and our theoretical results in Section 3.1 directly extend to multi-class classification with K classes. (Our motivation for studying binary classification was to simplify the exposition and analysis.) \n\nIn **new multi-class experiments on CIFAR-10**, we consider a data distribution over 50,000 samples and the original 10 classes. We show that the equilibrium social loss is non-monotonic in the data representation quality as measured by Bayes risk (see Figure N1 in the General Response pdf). Figure N1 shows that the empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes. \n\nIn a new theoretical result, we show that Proposition 1 can be directly extended to multi-class classification, where the function class is all functions $f:X \\rightarrow Y$ where $Y = [K]$. The same insights from Section 3.1 thus carry over to multi-class classification. \n\nFor details of the new experiments and new theoretical result, we refer the reviewer to the \"Extension to classification with more than 2 classes\" section of the General Response. \n\n**“If the market is competitive, when and why do people care about social welfare? Is there any scenario in practice in machine learning that people care about social welfare more than their own preference or happiness?”**\n\nThe reviewer is correct that every *individual user* cares only about their own preferences and happiness, which is why the user chooses the model-provider that maximizes that user’s happiness (see discussion below for more details). However, when model-providers compete to attract the most users (line 124), our results show that *society* (i.e., the user population as a whole) becomes worse off. Our measure of social welfare exactly captures the *overall happiness of the user population as a whole*. \n\nThis measure of social welfare is the typical metric studied in the existing work on competing predictors (e.g. Ben-Porat and Tenneholtz [2017, 2019]) and is ultimately what is important from a societal perspective (e.g., Kleinberg and Raghavan 2021). We envision that policymakers and regulators would care about social welfare of the user population and may develop interventions to ensure that social welfare is high. \n\nIn more detail, to see why our social welfare metric exactly captures the overall happiness of the user population, we refer the reviewer to Section 2 and summarize the main points below. \n- The  user utility function implicit in this definition is $-\\ell(y, f(x))$ where $y$ is the user’s true label and $f(x)$ is the prediction that they receive. In other words, user utility is a negative of the loss that they receive for their prediction. \n- Each user chooses the model-provider $j^*(x,y)$ (see line 120) with predictor $f_{j^*(x,y)}$ that maximizes that user’s utility. \n- Recall that we measure social welfare by the “overall predictive accuracy” for users (see definition of social loss in line 141). Social welfare thus measures the overall happiness (total user utility) of all users in the population. \n\n**“If models are trained solely on user-provided datasets, their performance may not vary significantly, which may make the best response trivial.”**\n\n This intuition actually turns out to be incorrect: even if the models are trained on the same data distribution and dataset, the model-providers may select different predictors. The reason is that optimizing for market share is  fundamentally different from optimizing for empirical risk. As a result, model-providers may not choose the same predictor, or even choose Bayes optimal predictors, at equilibrium ((Proposition 1, Section 3.2, Section 4, Figure 1). \n\n**“The finding about inefficient translation to social welfare is interesting, but it lacks proper and thorough explanation.”**\n\nWe are unsure what form of explanation the reviewer has in mind. The paper includes formal details of the model and assumptions (Section 2), theoretical justification of our results (Section 3), and empirical evaluation of our results (Section 4). Furthermore, we have included additional experiments for multi-class classification (see above and the General Response) and local improvements to representations (see the General Response). We would be happy to provide further explanations during the discussion period and to the final version of the paper if the reviewer could provide more details about what they find lacking.\n"}]

Author rebuttal:
Thanks to the reviewers for their detailed feedback. In response to reviewer feedback, we provide the following new results: (1) we provide **additional experiments on CIFAR-10 and theoretical results for multi-class classification** and (2) we provide **additional experiments for non-global representations** across model-providers. (We respond to other points raised by the reviewers in the individual responses to each reviewer.)  

## Extension to classification with more than 2 classes 
Several reviewers asked if our results extend beyond binary classification. We provide new experiments showing that our empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes, and new theoretical results showing our results in Section 3.1 directly extend to multi-class classification with $K$ classes. 

**Extension of empirical results on CIFAR-10:** In *new CIFAR-10 experiments for multi-class classification*, we consider a data distribution over 50,000 samples and the original 10 classes. Each function maps each x to a probability distribution $p$ over the 10 classes, according to a linear function followed by a softmax. The loss function $\ell(y, p)$ is equal to $|1 - p(y)|$, where $p(y)$ is the probability of choosing the correct class $y$. We show that the equilibrium social loss is non-monotonic in the data representation quality as measured by Bayes risk (see Figure N1 in the General Response pdf). Figure N1 shows that the empirical results in Section 4.3 directly extend to CIFAR-10 with 10 classes. 

**Extension of theoretical results (Proposition 1):** In a *new theoretical result for multi-class classification*, we show that Proposition 1 can be directly extended to $K$-class classification. The function class is all functions $f:X \rightarrow Y$ where $Y = [K]$. 

*Result*: Let $X$ be a finite set. For each class $i \in [K]$, let $\alpha_i(x) := \mathbb{P}[Y = i | X = x]$ be the probability of each class for the representation $x$. At any pure strategy equilibrium, the equilibrium social loss is in between $E[\sum_{i \in [K]} \alpha_i(x) \cdot  I[\alpha_i(x) < 1/(2m)]]$ and $E[\sum_{i \in [K]} \alpha_i(x) \cdot I[\alpha_i(x) \le 1/m]]$. 

*Comparison with Proposition 1*: Note that $\alpha(x)$ in the paper is equal to $\min(\alpha_0(x), \alpha_1(x))$. In the special case of 2 classes, this result exactly corresponds to Theorem 1 with a factor of 2 slack in the indicator random variable. 

*Takeaway*: When representations improve—and thus the $\alpha_i(x)$ values approach either $0$ or $1$— we see that the equilibrium social loss can increase. The same insights from Section 3.1 thus carry over to multi-class classification. 

*Proof sketch*: The proof of this result follows a very similar argument to the proof of Proposition 1. It suffices to show (a) if $\alpha_i(x) > 1/m$, then some model-provider will choose $f(x) = i$, and (b) if $\alpha_i(x) < 1/(2m)$, then no model-provider will choose $f(x) = i$. To see this, like in the proof of Proposition 1, we can restrict to the case where $X$ only contains a single data representation $x$ (apply a generalization of Lemma 3 in Appendix C). The first part follows from the fact that some model-provider must get market share $\le 1/m$ so that model-provider would want to switch to label $i$ if no other model-provider chooses $i$. The second part follows from the fact that some model-provider gets market share $\ge 1/m$, and any model-provider can get market share $\ge 1/(2m)$ by copying the label of that model-provider. 

## Beyond global representations across model-providers 

**Extension of empirical results on CIFAR-10:** To address the reviewers’ questions about differing representations, we also provide *a new experiment on CIFAR-10 to study “local improvements”* where each model-provider uses different data representations and only one model-provider changes their data representations. We continue to observe non-monotonicity of the social loss (see Figure N2 in the attached pdf) in a CIFAR-10 setup. In these setups, data diversity also impacts the equilibrium social loss: that is, changing a single model-provider’s data representations can increase (or decrease) the overall diversity in the data representations across model-providers, which we expect would respectively decrease (or increase) the level of monoculture/outcome homogeneity. Understanding exactly how varying data representations impacts data diversity (and equilibrium social loss) is an interesting direction for future work. 

**Discussion of global representations assumption:** The reason that we focused on global data representations is to isolate the role of competition on the equilibrium social loss. In particular, we intentionally designed our model to be as close as possible to the classical single-decision-maker setting (where there is a single data distribution), so that we could directly compare scaling trends in the single decision-maker setup and the competitive setup. Our results show that competition leads to non-monotonicity in the social loss, even if the model-providers are given the exact same data representations and have access to the same function class.


In addition, in terms of real-world motivation, global data representations capture a marketplace where all model-providers all rely on the same pretrained model for representations, but each model-provider separately fine-tunes their own model. In fact, the linear function class in our experiments can be viewed as a stylized model for fine-tuning. 


Meta-review: This paper studies an interesting problem in which improved representations (in terms of Bayes risk) result in reduced social welfare under competition. All reviewers found the paper to be clear, organized, and well-written, and appreciated the paper's complementing mix of theory and empirics. The general message of this paper is interesting and relevant, showing how competition can challenge some of our fundamental beliefs on scaling processes. The authors' efforts in adding results (multi class and "local" representations) in response to reviewer requests is commendable.

Reviewers were overall positive, but nonetheless pointed our several issues that we hope the authors would take into account in their final version:
1. Reviewers felt that the limitations of the approach and the implications of its assumptions could have been better discussed. For this the authors provided list of additions, which we hope they will implement.
2. Several reviewers found the assumption of global representation to be insufficiently justified. The additional experiment on local representations helped, but this should not be considered a substitute for a proper justification of the main assumption.
3. Some reviewers felt that the market setting should be better justified, and have asked several "what if" questions that challenge the paper's main results. Authors are encourage to address these.
4. One reviewer found the paper's framing around "scaling laws" to be incompatible with how the actual scaling laws community views these. Authors are encouraged to make the necessary adaptations so that their message will be relevant to a broader audience, including this community.

One final point regards the paper's title - which is perhaps somewhat misleading, since it implies that what causes a reduction in welfare is the improvement of Bayes risk, where the actual control variable is representation quality (there are other ways to improve risk). If the authors agree with this, we hope they would be willing to reconsider the message they would like to convey through their title.
Decision: Accept (poster)