SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Setting 2: Varying the representation noise. Consider a one-dimensional population given by a mixture of two Gaussians (one for each class), where each Gaussian has variance  $\sigma^2$  (see Appendix C for the details of the setup). We vary the parameter  $\sigma$  to change the representation quality. Intuitively, a lower value of  $\sigma$  makes the Gaussians more well-separated, which improves representation quality (Bayes risk). By Proposition 2, the equilibrium social loss is  $\mathbb{E}[\alpha(x) \cdot \mathbb{1}[\alpha(x) < 1/m]]$ . For each value of  $\sigma$ , we estimate the equilibrium social loss by sampling representations  $x$  from the population and taking an average. Figure 2b depicts the result: the equilibrium social loss is non-monotonic in  $\sigma$
For  $m = 2$ , where  $\alpha = 1/2$  is the maximum possible per-representation Bayes risk, the equilibrium social loss is monotone in  $\alpha$ .
Strictly speaking, we can't directly apply Proposition 2 to this setup since  $X$  is infinite. We circumvent this issue by applying Proposition 2 on a sample of the representations.
(and thus the Bayes risk). Again, as the number of model-providers increases, the non-monotonicity occurs at a higher representation quality (a lower Bayes risk).
Setting 3: Varying the representation dimension. We consider a four-dimensional population  $(X^{\mathrm{all}}, Y)$ , and let the representation  $X$  consist of the first  $D$  coordinates of  $X^{\mathrm{all}}$ , for  $D$  varying from 0 to 4 (see Appendix C for full details). Intuitively, a higher dimension  $D$  makes the representations more informative, thus improving representation quality (Bayes risk). As before, for each value of  $D$ , we estimate the equilibrium social loss by sampling representations  $x$  from the population and taking an average. Figure 2c depicts the result: the equilibrium social loss is once again non-monotonic in the representation dimension  $D$  (and thus the Bayes risk).
Discussion. Settings 1-3 illustrate that equilibrium social loss can be non-monotonic in Bayes risk when representations are improved along many different axes. The intuition is that varying representations along these axes can increase  $\alpha(x)$  for inputs  $x$ ; by Proposition 2, these changes to  $\alpha(x)$  can lead to non-monotonicity in the equilibrium social loss. We revisit Settings 1-3 for richer market structures (Appendix B.1) and for linear predictors and noisy user decisions (Section 4.2).
3.3 Generalization to more than 2 classes
While our analysis has thus far focused on classification with  $K = 2$  classes, the number of classes  $K$  can be much larger in practice. As a motivating example, consider content recommendation tasks where each class represents a different genre of content; since the content landscape can be quite diverse, we would expect  $K$  to be fairly large. This motivates us to extend our theoretical characterization in Proposition 2 to classification with  $K \geq 2$  classes.
For the case of  $K \geq 2$  classes, the appropriate analogue of the per-representation Bayes risk is the per-class-per-representation Bayes risk, defined to be  $\alpha^i(x) \coloneqq \mathbb{P}(y = i \mid x)$  for each  $x \in X$  and  $i \in \{0,1,\dots,K-1\}$ . As a result,  $1 - \max_{0 \leq i \leq K-1} \alpha^i(x)$  is the minimum error that a model-provider can achieve on  $x$ , and  $\mathrm{OPT}_{\mathrm{single}}$  is equal to the average value  $\mathbb{E}[1 - \max_{0 \leq i \leq K-1} \alpha^i(x)]$  across the population. The equilibrium social loss, however, depends on other aspects of the  $\alpha^i(x)$  values.
We characterize the equilibrium social loss in terms of the per-class-per-representation Bayes risks in the following proposition. Our characterization again focuses on pure-strategy equilibria, which are guaranteed to exist in this setup by Proposition [1].
Proposition 3. Let  $X$  be a finite set, let  $K \geq 2$ , let  $\mathcal{F} = \mathcal{F}_{all}^{multi-class}$ . Suppose that user decisions are noiseless (i.e., user decisions are given by (3)). Let  $c = \min_{x \in X} \max_{0 \leq i \leq K-1} \alpha^i(x)$ . Then, at any pure strategy Nash equilibrium  $f_1^*, \ldots, f_m^*$ , the social loss  $SL(f_1^*, \ldots, f_m^*)$  is bounded as
$$
\underset {(x, y) \sim \mathcal {D}} {\mathbb {E}} \left[ \sum_ {i = 1} ^ {K} \alpha^ {i} (x) \cdot \mathbb {1} \left[ \alpha^ {i} (x) <   \frac {c}{m} \right] \right] \leq S L \left(f _ {1} ^ {*}, \dots , f _ {m} ^ {*}\right) \leq \underset {(x, y) \sim \mathcal {D}} {\mathbb {E}} \left[ \sum_ {i = 1} ^ {K} \alpha^ {i} (x) \cdot \mathbb {1} \left[ \alpha^ {i} (x) \leq \frac {1}{m} \right] \right]. \tag {5}
$$
The high-level intuition for Proposition 3 is similar to the intuition for Proposition 2 except that each class needs to be considered separately. In particular, when class  $i$  occurs sufficiently frequently for the representation  $x$  (i.e., when  $\alpha^i(x)$  is not too small), then some model-provider will label  $x$  as  $i$ ; on the other hand, if the class  $i$  occurs very infrequently for  $x$ , then no model-provider will label  $x$  as  $i$ . We defer a proof of Proposition 3 to Appendix D.
While Proposition 3 is conceptually a generalization of Proposition 2, the details of Proposition 3 slightly differ. In particular, Proposition 3 does not completely pin down the equilibrium social loss, and there is a factor of  $c$  slack in the constraint on each  $\alpha^i(x)$  in (5) between the upper and lower bounds. Nonetheless, since the value  $c = \min_{x \in X} \max_{0 \leq i \leq K-1} \alpha^i(x)$  measures the minimum accuracy of the Bayes optimal predictor across all inputs  $x$ , we expect that "reasonable" representations (i.e., sufficiently informative representations) would have  $c$  equal to a constant. When  $c$  is a constant, there is at most a constant factor slack in the  $\alpha^i(x)$  constraints in (5) between upper and lower bound.
For similar reasons to Proposition 2, Proposition 3 implies that the equilibrium social loss can be non-monotonic in representation quality (i.e., Bayes risk). We defer a discussion to Appendix B.2
(d)
Figure 3: Equilibrium social loss (left) and Bayes risk (right) on a binary (top) and multi-class (bottom) classification tasks on CIFAR-10 (Section 4.3). Representations are generated from different networks pretrained on ImageNet. The points show the equilibrium social loss when  $m$  model-providers compete with each other (left) and the Bayes risk of a single model-provider in isolation (right). While Bayes risk is decreasing in this representation ordering, the equilibrium social loss is non-decreasing in this ordering. The equilibrium social loss is thus non-monotonic in representation quality as measured by Bayes risk. Error bars are 1 standard error.
4 Empirical Analysis of Non-monotonicity for Linear Predictors
We next turn to linear predictors and demonstrate empirically that the social welfare can be non-monotonic in data representation quality in this setup as well. We take  $X = \mathbb{R}^{D}$  and we let the model parameters be  $\phi$ . For binary classification, we let  $\mathcal{F}_{\mathrm{linear}}^{\mathrm{binary}}$  be the family of linear predictors  $f_{w,b} = \mathrm{sigmoid}(\langle w,x\rangle +b)$  where  $w\in \mathbb{R}^D$ ,  $b\in \mathbb{R}$ , and  $\phi = [w,b]$ . Similarly, for classification with more than 2 classes, we let  $\mathcal{F}_{\mathrm{linear}}^{\mathrm{multi-class}}$  be the family of linear predictors  $f_{W,b} = \mathrm{softmax}(Wx + b)$  where  $w\in \mathbb{R}^{|Y|\times D}$ ,  $b\in \mathbb{R}^{|Y|}$ , and  $\phi = [W,b]$ . Since this setting no longer admits closed-form formulae, we numerically estimate the equilibria using a variant of best-response dynamics, where model-providers repeatedly best-respond to the other predictors.
We first show on low-dimensional synthetic data on a binary classification task that the insights from Section 3.2 generalize to linear predictors (Figures 2d-2f). We then turn to natural data, considering binary and 10-class image classification tasks for CIFAR-10 and using pretrained networks—AlexNet, VGG16, and various ResNets—to generate high-dimensional representations (ranging from 512 to 4096). We again find that the equilibrium social loss can be non-monotonic in Bayes risk (Figure 3).
4.1 Best-response dynamics implementation
To enable efficient computation, we assume the distribution  $\mathcal{D}$  corresponds to a finite dataset with  $N$  data points. We calculate equilibria using an approximation of best-response dynamics. Model-providers (players) iteratively (and approximately) best-respond to the other players' actions. We implement the approximate best-response as running several steps of gradient descent.
In more detail, for each  $j \in [m]$ , we initialize the model parameters  $\phi$  as mean zero Gaussians with standard deviation  $\sigma$ . Our algorithm then proceeds in stages. At a given stage, we iterate through the model-providers in the order  $1, \ldots, m$ . When  $j$  is chosen, first we decide whether to reinitialize:
if the risk  $\mathbb{E}_{(x,y)\sim \mathcal{D}}[\ell (f_{\phi}(x),y)]$  exceeds a threshold  $\rho$ , we re-initialize  $w_{j}$  and  $b_{j}$  (sampling from mean zero Gaussians as before); otherwise, we do not reinitialize. Then we run gradient descent on  $u(\cdot ;\mathbf{f}_{-j})$  (computing the gradient on the full dataset of  $N$  points) with learning rate  $\eta$  for  $I$  iterations, updating the parameters  $\phi$ . We run this gradient descent step up to 2 more times if the risk  $\mathbb{E}_{(x,y)\sim \mathcal{D}}[\ell (f_{\phi}(x),y)]$  exceeds a threshold  $\rho^{\prime}$ . At the end of a stage, the stopping condition is that for every  $j\in [m]$ , model-provider  $j$ 's utility  $u(f_j,\mathbf{f}_{-j})$  has changed by at most  $\epsilon$  relative to the previous stage. If the stopping condition is not met, we proceed to the next stage
4.2 Simulations on synthetic data
We first revisit Settings 1-3 from Section 3.2, considering the same axes of varying representations and distributions over  $(x,y)$ . In contrast to Section 3.2, we restrict the model family to linear predictors  $\mathcal{F}_{\mathrm{linear}}^{\mathrm{binary}}$  instead of allowing all predictors  $\mathcal{F}_{\mathrm{all}}^{\mathrm{binary}}$ . We also set the noise parameter  $c$  in user decisions (I) to 0.3. Our goal is to examine if the findings from Section 3 generalize to this new setting.
We compute the equilibria for each of the following (continuous) distributions as follows. First, we let  $\mathcal{D}$  be the empirical distribution over  $N = 10,000$  samples from the continuous distribution. Then we run the best-response dynamics described in Section 4.1 with  $\rho = \rho' = 1$ ,  $I = 5000$ ,  $\eta = 0.1$ ,  $\epsilon = 0.001$ , and  $\sigma = 1.0$ . We then compute the equilibrium social loss according to (2). We also compute the Bayes optimal predictor with gradient descent. See Appendix C for full details.
Our results, described below, are depicted in Figures 2d-2f (row 2). We compare these results with Figures 2a-2c (row 1), which shows the analogous results for  $\mathcal{F}_{\mathrm{all}}^{\mathrm{binary}}$  from Section 3.2.
Setting 1: Varying the per-representation Bayes risks. Consider the same single  $x$  setup as in Setting 1 in Section 3.2. The only parameter of the predictor is the bias  $b \in \mathbb{R}$  (i.e., we treat  $x$  as zero-dimensional). Figure 2d shows that the equilibrium social loss is non-monotonic in  $\alpha$ , which mirrors the non-monotonicity in Figure 2a.
Setting 2: Varying the representation noise. Consider the same one-dimensional mixture-of-Gaussians distribution as in Setting 2 in Section 3.2 (The weight  $w$  is one-dimensional.) We again vary the noise  $\sigma$  to change the representation quality. Figure 2e shows that the equilibrium social loss is non-monotonic in the noise  $\sigma$ , which again mirrors the non-monotonicity in Figure 2b.
Setting 3: Varying the representation dimension. Consider the same four-dimensional population as in Setting 3 in Section 3.2. We vary the representation dimension  $D$  from 0 to 4 to change the representation quality. Figure 2f shows that the equilibrium social loss is non-monotonic in the dimension  $D$ , which once again mirrors the non-monotonicity in Figure 2c
Discussion. In summary, in Figure 2 rows 1 and 2 exhibit similar non-monotonicities. This illustrates that the insights from Section 3.1 translate to linear predictors and noisy user decisions.
4.3 Simulations on CIFAR-10 for binary classification
We next turn to experiments with natural data. While we have directly varied the informativeness of data representations thus far, representations in practice are frequently generated by pretrained models. The choice of the pretrained model implicitly influences representation quality, as measured by Bayes risk on the downstream task. In this section, we consider how the equilibrium social loss changes with representations generated from pretrained models of varying quality. We restrict the model family to linear predictors  $\mathcal{F}_{\mathrm{linear}}^{\mathrm{binary}}$  and set the noise parameter  $c$  in user decisions (1) to 0.1.
We consider a binary image classification task on CIFAR-10 [Krizhevsky, 2009] with 50,000 images. Class 0 is defined to be {airplane, bird, automobile, ship, horse, truck} and the class 1 is defined to be {cat, deer, dog, frog}. We treat the set of 50,000 images and labels as the population of users, meaning that it is both the training set and the validation set. Representations are generated from five models—AlexNet [Krizhevsky et al., 2012], VGG16 [Simonyan and Zisserman, 2015], ResNet18, ResNet34, and ResNet50 [He et al., 2016]—pretrained on ImageNet [Deng et al., 2009]. The
The code can be found at https://github.com/mjagadeesan/competition-nonmonotonicity.
We make this choice to be consistent with the rest of the paper, where we focus on population-level behavior and thus do not consider generalization error.
representation dimension is 4096 for AlexNet and VGG16, 512 for ResNet18 and ResNet34, and 2048 for ResNet50.
We compute the equilibria as follows. First, let  $\mathcal{D}$  be the distribution described above with  $N = 50,000$  data points. Then we run the best-response dynamics described in Section 4.1 for  $m \in \{3,4,5,6,8\}$  model-providers with  $\rho = \rho' = 0.3$ ,  $I = 2000$ ,  $\epsilon = 0.001$ ,  $\sigma = 0.5$ , and a learning rate schedule that starts at  $\eta = 1.0$ . We then compute the equilibrium social loss according to (2). We also compute the Bayes risk using gradient descent. For experimental details, see Appendix C
Figures 3a-3b show that the equilibrium social loss can be non-monotone in the Bayes risk. For example, for  $m = 3$ , VGG16 outperforms AlexNet, even though the Bayes risk of VGG16 is much higher than the Bayes risk of AlexNet. Interestingly, the location of the non-monotonicity differs across different values of  $m$ . For example, for  $m = 5$  and  $m = 8$ , AlexNet outperforms ResNet50 despite having a higher Bayes risk, but ResNet50 outperforms AlexNet for  $m = 3$  and  $m = 4$ .
4.4 Simulations on CIFAR-10 for 10-class classification
While our empirical analysis has thus far focused on binary classification, we now turn to classification with more than 2 classes. In particular, we consider a ten class CIFAR-10 [Krizhevsky, 2009] task with 50,000 images. The labels are specified by the CIFAR-10 classes in the original dataset. We treat the set of 50,000 images and labels as the population of users, meaning that it is both the training set and the validation set. Representations are generated from the same five models as in Section 4.3. We restrict to linear predictors  $\mathcal{F}_{\mathrm{linear}}^{\mathrm{multi-class}}$  and again set the noise parameter  $c$  in user decisions (1) to 0.1.
We compute the equilibria. We again let  $\mathcal{D}$  be the distribution described above with  $N = 50,000$  data points. Then, we run the best-response dynamics described in Section 4.1 for  $m\in \{3,4,5,6,8\}$ , and we compute the equilibria with the same hyperparameter settings as in Section 4.3 (except that  $\rho = 0.7$  and  $\rho^{\prime} = 1$ ). We then compute the equilibrium social loss according to (2). We also compute the Bayes risk using gradient descent. For full experimental details, see Appendix C
Figures 3c-3d show that the equilibrium social loss can be non-monotone in the Bayes risk. For example, across all five values of  $m$ , ResNet18 outperforms VGG16, even though the Bayes risk of ResNet is substantially higher than the Bayes risk of VGG16. Furthermore, for  $m = 3$ , VGG16 outperforms AlexNet despite having a larger Bayes risk. Interestingly, the shape of the equilibrium social loss curve for each value of  $m$  (Figure 3c) appears qualitatively different than the analogous equilibrium social loss curve for binary classification (Figure 3a).
We showed that the monotonicity of scaling trends can be violated under competition. We demonstrated that when multiple model-providers compete for users, improving data representation quality (as measured by Bayes risk) can increase overall loss at equilibrium. We exhibited non-monotonicity of the equilibrium social loss in Bayes risk when representations are varied along several axes (per-representation Bayes risk, noise, dimension, and pre-trained model which generates representations).
An interesting direction for future work is to further characterize the regimes when the equilibrium social loss is monotonic versus non-monotonic in data representation quality as measured by Bayes risk. For example, it would be interesting to generalize our theoretical results from Section 3 to general function classes and market reputations as well as to generalize our empirical findings from Section 4 to other axes of varying data representations and to non-linear functions. Finally, while we focused on classification, an interesting direction would be to generalize our findings to regression tasks with continuous outputs or to generative AI tasks with text-based or image-based outputs.
More broadly, the non-monotonicity of equilibrium social welfare in scale under competition establishes a disconnect between scaling trends in the single model-provider setting and in the competitive setting. In particular, typical scaling trends (e.g. Kaplan et al., 2020, Sharma and Kaplan, 2020, Bahri et al., 2021, Hoffmann et al., 2022, Hernandez et al., 2021)) may not translate to competitive settings such as digital marketplaces. Thus, understanding the downstream impact of scale on user welfare in digital marketplaces will likely require understanding how scaling trends behave under competition. We hope that our work serves as a starting point for analyzing and eventually characterizing the scaling trends of learning systems in competitive settings.
We thank Yiding Feng, Xinyan Hu, and Alex Wei for useful comments on the paper. This work was partially supported by the Open Phil AI Fellowship, the Berkeley Fellowship, the European Research Council (ERC) Synergy Grant program, and the National Science Foundation under grant numbers 2031899 and 1804794.
Simon P. Anderson, Andre de Palma, and Jacques-Francois Thisse. Discrete Choice Theory of Product Differentiation. The MIT Press, 10 1992.
Guy Aridor, Yishay Mansour, Aleksandrs Slivkins, and Zhiwei Steven Wu. Competing bandits: The perils of exploration under competition. CoRR, abs/2007.10144, 2020.
Yasaman Bahri, Ethan Dyer, Jared Kaplan, Jaehoon Lee, and Utkarsh Sharma. Explaining neural scaling laws. CoRR, abs/2102.06701, 2021.
Omer Ben-Porat and Moshe Tennenholtz. Best response regression. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems (NIPS), pages 1499-1508, 2017.
Omer Ben-Porat and Moshe Tennenholtz. Regression equilibrium. In Proceedings of the 2019 ACM Conference on Economics and Computation (EC), pages 173-191. ACM, 2019.
Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-1828, 2013.
Rishi Bommasani, Kathleen A. Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang. Picking on the same person: Does algorithmic monoculture lead to outcome homogenization? In NeurIPS, 2022.
Michael Brückner, Christian Kanzow, and Tobias Scheffer. Static prediction games for adversarial learning problems. JMLR, 13(1):2617-2654, 2012.
Emilio Calvano and Michele Polo. Market power, competition and innovation in digital markets: A survey. Information Economics and Policy, 54:100853, 2021.
Micah D. Carroll, Anca D. Dragan, Stuart Russell, and Dylan Hadfield-Menell. Estimating and penalizing induced preference shifts in recommender systems. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 2686-2708. PMLR, 2022.
Shih-fen Cheng, Daniel M. Reeves, Yevgeniy Vorobeychik, and Michael P. Wellman. Notes on equilibria in symmetric games. In Proceedings of the 6th International Workshop on Game Theoretic and Decision Theoretic Agents (GTDT), pages 71-78, 2004.
Mihaela Curmei, Andreas A. Haupt, Benjamin Recht, and Dylan Hadfield-Menell. Towards psychologically-grounded dynamic preference models. In Proceedings of the 16th ACM Conference on Recommender Systems, RecSys '22, page 35-48, 2022.
C. d'Aspremont, J. Jaskold Gabszewicz, and J.-F. Thisse. On hotelling's "stability in competition". Econometrica, 47(5):1145-1150, 1979. ISSN 00129682, 14680262.
Sarah Dean and Jamie Morgenstern. Preference dynamics under personalized recommendations. In David M. Pennock, Ilya Segal, and Sven Seuken, editors, EC '22: The 23rd ACM Conference on Economics and Computation, Boulder, CO, USA, July 11 - 15, 2022, pages 795-816. ACM, 2022.
Sarah Dean, Mihaela Curmei, Lillian J. Ratliff, Jamie Morgenstern, and Maryam Fazel. Multi-learner risk reduction under endogenous participation dynamics. CoRR, abs/2206.02667, 2022.
Yongchan Kwon, Tony Ginart, and James Zou. Competition over data: how does data purchase affect users? Trans. Mach. Learn. Res., 2022.
Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 3156-3164. PMLR, 2018.
Lydia T. Liu, Ashia Wilson, Nika Haghtalab, Adam Tauman Kalai, Christian Borgs, and Jennifer T. Chayes. The disparate equilibria of algorithmic decision making when individuals invest rationally. In Mireille Hildebrandt, Carlos Castillo, L. Elisa Celis, Salvatore Ruggieri, Linnet Taylor, and Gabriela Zanfir-Fortuna, editors, FAT* '20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, pages 381-391. ACM, 2020.
Dov Monderer and Lloyd S. Shapley. Potential games. Games and Economic Behavior, 14(1): 124-143, 1996.
Robert W. Rosenthal. A class of games possessing pure-strategy nash equilibria. Int. J. Game Theory, 2(1):65-67, dec 1973.
Utkarsh Sharma and Jared Kaplan. A neural scaling law from the dimension of the data manifold. CoRR, abs/2004.10802, 2020.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In *Joshua Bengio and Yann LeCun*, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Kenneth Train. *Discrete Choice Methods with Simulation*. Cambridge University Press, New York, 2002.