Competition over data: how does data purchase affect users?
Authors: Yongchan Kwon
Tony A Ginart
James Zou
Venue: Accepted by TMLR ()

Abstract: As the competition among machine learning (ML) predictors is widespread in practice, it becomes increasingly important to understand the impact and biases arising from such competition. One critical aspect of ML competition is that ML predictors are constantly updated by acquiring additional data during the competition. Although this active data acquisition can largely affect the overall competition environment, it has not been well-studied before. In this paper, we study what happens when ML predictors can purchase additional data during the competition. We introduce a new environment in which ML predictors use active learning algorithms to effectively acquire labeled data within their budgets while competing against each other. We empirically show that the overall performance of an ML predictor improves when predictors can purchase additional labeled data. Surprisingly, however, the quality that users experience---i.e., the accuracy of the predictor selected by each user---can decrease even as the individual predictors get better. We demonstrate that this phenomenon naturally arises due to a trade-off whereby competition pushes each predictor to specialize in a subset of the population while data purchase has the effect of making predictors more uniform. With comprehensive experiments, we show that our findings are robust against different modeling assumptions.

Published in Transactions on Machine Learning Research (11/2022)
Competition over data: how does data purchase affect users?
Yongchan Kwon
yk30112@coummiic.com edu
Deportment of Statistics, Columbia University
Tony Ginart
tginart@sttnford edu
Deportment of Electricol Engineering, Stanford University
James Zou
jameszSisaanfore edu
Deportment of Biomedical Data Science, Stanford University
Reviewed on OpenReview: https: openreview. net/ forun? id= 63sJaCmqol
Abstract
As the competition among machine learning (ML) predictors is widespread in practice, it becomes increasingly important to understand the impact and biases arising from such competition. One critical aspect of ML competition is that ML predictors updated by acquiring additional data during the competition. Although this active data acquisition can largely affect the overall competition environment, it has not been well-studied before. In this paper, we study what happens when ML predictors can purchase additional data during the competition. We introduce a new enviromment which ML predictors use active learning algorithms effectively acquire labeled data within their budgets while competing against each other. We empirically show that the overall performance of an ML predictor improves when predictors can purchase additional labeled data. Surprisingly, however, the quality that users experience-iee. the accuracy of the predictor selected by each user-can decrease even as the individual predictors get better. We demonstrate that this phenomenon naturally arises due to a trade-off whereby iompetition pushes each predictor to specialize in a subset of the population while data purchase has the effect making predictors more uniform. With comprchensive experiments, we show that our findings are robust against different modeling assumptions.
constantly
1 Introduction
When there several companics marketplace offering similar services, customer usually chooses the one that offers the best options functionalities. It naturally causes competition among the companies, and they are motivated to offer high-quality services, as their ultimate goal is to attract more customers and make more profits. When it comes to machine learning (ML)-based services, high-quality services are often achieved by re-training their models after acquiring more data from customers or data vendors (Meierhofer et al. 2019). Tn this paper, we consider a competition situation where multiple companies offer ML.-baedd services while constantly improving their predictions by acquiring labeled data.
For instance, consider the U.S. auto insurance market (Jin & Vasserman 2019 Sennaar 2019) The auto insurance companies including State Farm, Progressive, and AllState use ML models to analyze customer data, assess risk, and adjust actual premiums. These companics also offer insurance called the How-You-Drivee which is usually cheaper than regular auto insurances on the condition that the insurer monitors driving patterns, such as rapid acceleration or oscillations in speed (Arumugam & Bhargavi 2019 Jin Vasserman, 2019). That is, the companies essentially provide financial benefits customers, collecting customers' driving pattern data. With these user data, they can regularly update their ML models, improving model performance while competing with each other.

---

Published in Transactions on Machine Learning Research (11/2022)
Case 1. There is a buyer
Case No one shows purchase intent
Model
Model
X
Model
Buy
X
Model 2
Poor
obtains
Yt
obtains Yt
New user
New user
Model
Pass
Model
Good
Each predictor decides whether purchase
Predictor acquires
Each model makes prediction
User selects predictor
user data
Figure 1: Illustrations of our competition environment, (left) when there is a company showing purchase intent and (right) when company shows purchase intent. In step 1, described in the first arrow, each predictor receives a user query and decides whether to buy user data. In step 2, described in the second arrow, (left) if there is a company that thinks the data is worth buying, the company shows purchase intent. The user X, then selects the buyer for financial benefits. (Right) If no one thinks the user data worth buying, a user selects one company based on received ML predictions. In step 3, the only selected predictor gets the user label Y, and updates its model. We provide details on the environment Section
Analyzing the effects of data purchase in competitions could have practical implications, but it has not been studied much in the ML literature. The effects of data acquisition have been investigated in active learning (AL) literature (Settles 2009 Ren et al., 2020 but is not straightforward to establish competition in AL settings because considers the single-agent situation. Recently, Ginart al. (2021) studied implications of competitions by modeling an environment where ML predictors compete against each other for user data. They showed that competition pushes competing predictors to focus and helps users find high-quality predictions. Although this work describes limited to describe the data purchase system duc to the simplicity of its model. The impact of data purchases on competition has not been studied much in the literal which is the main focus of our work. Our environment is able to model situations where competing companies actively acquire user data by providing a financial benefit to users, and influence the way users choose service providers (See Figure 1 Related works are further discussed in Section L.
small subset of the population interesting phenomenon, it is
Contributions In this paper, propose general competition environment and study what happens when competing MI. predictors can actively acquire user data. Our main contributions are as follows.
We propose novel environment that can simulate various real-world competitions. Our environment allows ML predictors to use AL algorithms to purchase labeled data within a finite budget. while competing against each other (Section
Surprisingly, our results show that when competing ML predictors purchase data, the quality of the predictions selected by each user can decrease even as competing ML predictors get better (Section 3.1
We demonstrate that data purchase makes competing predictors similar to each other, leading to this countorintuitive finding (Section 3.2 Our finding is robust and is consistently observed across various competition situations (Sect 3.3
We thcoretically analyze how the diversity user's available options can affect the user experience lo support our empirical findings. (Section 4p.
1.1
Related works
This work builds off and extends the recent paper, Ginart al. (2021), which studied the impacts of the competition. We extend this setting by incorporating the data purchase system into competition systems.
2

---

Published in Transactions on Machine Learning Research (11/2022)
Note that the setting by Ginart et al. (2021) is special case of ours when no competitor buys user data, i.c., ( = 0 for all i E [M] with the notation in Section 2 Our environment enables us to study the impacts of data acquisition in competition, which is not considered in the previous work. Compared to the previous work, which showed competing predictors become too focused on sub-populations, our work suggests that this can be good thing in the sense that it provides variety of different options and better quality of the predictors selected by users.
A related field of our work is the stream-based based AL, the problem of learning an algorithm that effectively finds data points to label from a stream of data points. (Scttles 2009; Ren et al. 2020 AL has been shown to have belter sample complexity than passive learning algorithms [Kanamori] 2007 Hanneke et al. 2011
& Wiener, 2012), and it is practically effective when the training sample size is small (Konyushkova et al. 2017 Gal al. 2017 Sener & Savaresel 2018). However, our competition environment is significantly different from AL. In AL, since there is only agent, competition cannot be established. In addition, while an agent in AL collects data only from label queries, competing predictors in our environment can obtain data from data purchase as well as regular competition. These differences create a unique competition environment, and this work studics the impacts of data purchase in competitive systems.
Competition has been studied in multi-agent reinforcement learning (MARL), which is a problem of optimizing goals in setting where a group of agents in common environment interact with each other and with the environment (Lowe et al., 2017 Foerster et al.) 2017). Competing agents in MARL maximize their own objective goals that could conflict with others. This setting is often characterized by zero-sum Markov games and is applied to video games such as Pong or Starcraft II (Littman 1994 Tampuu et al. 2017: Vinyals let al. 2019). As for the theoretical works, on zero-sum Markov games, Bai & Jin (020) studies sublinear regret a algorithm, and Loftin et al. 2002)) studies the efficient exploration in competitive multi-agent. settings. We refer to Busoniu et al. (2008) and Zhang et al. (2019) for a complementary literature survey of MARL.
Although MARL and our environment have some similarities, the user selection and the data purchase in our environment uniquely define the interactions between users and MT. predictors. In MARL, it is assumed that all agents observe information drawn from the shared environment. Different agents may observe different statuses and rewards, but all agents receive information and use them to update the policy function. In contrast, in our environment, the only selected predictor oblains the label and updates the predictor, which might be more realistic. In addition, ML predictors can collect data points through the data purchase. These settings have not been considered in the field of MARL.
2 A general environment for competition and data purchase
This section formally introduces a new and general competition environment. In our environment, competition is represented by a series of interactions between sequence of users and fixed competing ML predictors. Here the interaction is modeled by supervised learning tasks.
Notations At each round t E [T] {1...., T), we denote a user query by X, E X and its associated user label by Y, E V. We focus on classification problems, i.e., |}] is finite, while our environment can easily extend to regression scttings. We denote a sequence of users by {(X,, Y))) and assume users are independent and identically distributed (i..d.) by some distribution Px,. We call Px, the user distribution.
As for the ML predictor side, we suppose there are M competing predictors in market. For i E [M], each ML predictor is described as a tuple CD0 == (i) mb ((0)))()))) where n. E N is the number of i.i.d. seed data points from Px,y, m N is budget, f()) X + y is an ML model, and x(1) X - (0.1) is buying strategy. We consider the following setting. A predictor ((i) initially owns n (5) data points and can additionally purchase user data within (2) budgets. We set the price of one data point is one, í.e., a predictor (8) data points from a sequence of users. A predictor c( produces a prediction using the ML model f(6) and determines whether to buy the user data with the buying strategy x(1), We consider the utility function for ((i) is the classification accuracy of f(i with respect to the user distribution Px,y- Lastly, f(9 and 7( are allowed to be updated throughout the T competition rounds. That is, companies keep improving their MT. models with newly collected data points.
(lo) can purchase up to nb
3

---

Published in Transactions on Machine Learning Research (11/2022)
Competition dynamics Before the first competition round, all the M competing predictors independently train their model f(t) using the m seed data. points. After this initialization, at each round t E [7], a user sends a query X to all the predictors {CUn)}E and each predictor CD determines whether to buy the user data. We describe this decision by using the buying strategy xe x(1) If the predictor C( thinks that the labeled data would be worth one unit of budget, we denote this by (X,) = 1. Otherwise, if cD)) thinks that it is not worth one unit of budget, then x (X,) = 0. As for the x((), ML. predictors can use any stream- based AL algorithm (Freund et al., 1997 Zliobaité et al., 2013). For instance, a predictor Cti) CC) attempts to purchase user data if
can use the uncertainty- the current prediction (X.) is not conlident (e.gg, the Shanmon's 's entropy of pl is higher than some predefined threshold value where (X,) is the probability estimate at the t-th round). In brief, we suppose a predictor c(+) shows purchase intent if the remaining budget is greater than zero and (X,) = 1. If the remaining budget is zero or (X,) = 0, then C() does not show purchase intent and suggests a prediction (() to the user.
AL rule (Sottles & Craven
We now elaborate on how a user selects one predictor. At every round E [T], the user selects only one predictor based on both purchase intents and prediction information received from | If there is buyer, then we assume that a user selects one of the companies with purchase intent and receives the prediction he or she selects. We can think of this as a bargain in that the company offers a financial advantage (e.g,, discounts or coupons) and the user selects it even if the quality might not be the best. When there is more than one buyer, we assume a user selects one of them uniformly at random. Once selected, the only selected predictor's budget is reduced by one; all other predictor's budget stays the same because they are not selected and do not have to provide financial benefits. If no predictor shows purchase intent, then user receives prediction information ()()((X)) and chooses the predictor ((o) with the following probability.
exp ag (Y, (t) (x)))
P P(W.= = Yo, ( (1))
M 2-=1 Σj exp og (Yr, (60 (x.))
where a 2 0 denotes temperature parameter and Wt E [M] denotes the index of selected predictor. Here, q 3 > RR := (= E = 2 0} is predefined quality function that measures similarity between the user label Yi and the prediction (e.g., 1((2 = Y21)). With the softmax function in Equation ( users are more likely to select high-quality predictions, describing the rationality of the user selection. Here, the temperature parameter a indicates how selective users are. For instance, a is close to xo, users are very confident in their selection and choose the best. company. Afterwards, the selected predictor C(W)) gets the user label Y, and updates the model (WW) by training on the new datum (X,, Y2). The other predictors f(i) stay the same for + Wt. We describe our competition system in Environment
Characteristics of our environment Our environment simplifies real-world competition and data purchases, which usually exist in much more complicated forms, yet it captures key characteristics. First, our environment reflects the rationality of customers. Customers are likely to choose the best service within their budget, but they can select a company that is not necessarily the best offers financial benefits, such as promotional coupons, discounts, or free services (Rowley) 1998 Familmaleki et al.) 2015 Reimers & Shiller 2019). Such a user selection represents that a user can prioritize financial advantages and change her selection, which has not been considered in the ML literature. Second, our environment realistically models a company's data acquisition. Competing companies strive to attract more users, constantly purchasing user data to improve their ML predictions. Since the data buying process could be costly for the companies, data should be carefully chosen, and this is why we incorporate AL algorithms. Our environment allows companies to use AL algorithms within finite budgets and to selectively acquire user data, Third, our environment is flexible and takes into account various competition situations in practice. Note that we make no assumptions about the number of competing predictors M or budgets nb algorithms for predictors f(i or buying strategies 7(0, and the user distribution px,.
(6)
Example (Auto insurance in Section D. X, includes the t-th driver's demographic information, driving or insurance claim history, and Y, is the driver's preferred insurance plan within the user's budget constraints. Each predictor (+)) is one insurance company (e.g., State Farm, Progressive, or AllState), offering an anto
4

---

Published in Transactions on Machine Learning Research (11/2022)
Environment 1 A competition environment with data purchase
Input: Number of competition rounds T'; user distribution Px,:; number of predictors M; competing (()) for i E [M]]
predictors c( = (n=
(8) b f())=(1)))
Procedure:
For all i E [M]] a model f() is trained using the ( seed data points
for E [T] do
(Xe: Yi) from Px, is drawn and of buyers 0 is initialized
for i [M] do
if (n) 2 1) and (x,) = 1) then
B {
else
Predict f (Xt)
end if end for
if
[B] 2 then
A user selects one predictor W, from B uniformly at random (Wr)
(Wa) -1
n + nb
clse
A user selects one predictor W, based on
end if
((W) receives a user label Y, and updates ((W))
end for
insurance plan (X,) based on what it predicts to be most suitable for this driver. The driver chooses one company whose offered plan is the closest to Y, Tf a company believes that in ils database there are infrequent data from a particular group of drivers t-th driver belongs to (e.g., new drivers in their 30s), can attempt to collect more data. Accordingly, the company offers discounts to altract her, and the acquired data used to improve the company's future ML predictions.
3 Experiments
Using the proposed Environment we investigate the impacts of the data purchase in ML competition. Our experiments show an interesting phenomenon that data purchase can decrease the quality of the predictor selected by user, even when the quality of the predictors gets improved on average (Section 3.1 We demonstrate that data purchase makes ML predictors similar to each other. Data purchase reduces the effective variety of options, and predictors can avoid specializing to a small subset of the population (Section 3.2 Lastly, we show the robustness of our findings against different modeling assumptions (Section 3.3
Metrics To quantitatively measure the effects of data purchase, we introduce three population-level evaluation metrics. First, define the overall quality follows.
M (x
1 M
E
(Overall quality)
where the expectation taken over the user distribution Px,.- The overall quality represents the average quality that competing predictors provide in the market. Second is the quality of experience (QoE), the quality of the predictor selected by a user. The QoE is defined as
(X))))))
E q
(QoE)
Here, the expectation is over the random variables (X, Y. W) and a conditional distribution of a selected index P(W] X,Y) considered Equation ①1. Given that user selects one predictor based Equation

---

Published in Transactions on Machine Learning Research (11/2022)
when there is no buyer, QoE can be considered as the key utility of users. We exchude the data purchase procedure from the definition of QoF. The main reason for this exclusion is to clearly capture the user's expected satisfaction driven only by user selections after T competition rounds.
Next, we define the diversity to quantify how variable the ML predictions are. To be more specific, for i E V we define the proportion of predictors whose prediction is as px()) == bo Σラ (x))) = i). Then the diversity is defined as
[x(x)))o(xxx)) iEY
E
-
(Diversity)
where the expectation is taken over the marginal distribution Px and we use the convention 0 Olog(0) = 0 when (x) = 0. Note that the diversity is defined as the expected Shannon's entropy of competing ML predictions. When there are various different options that user can choose from, the diversity is more likely to be large.
Implementation protocol Our experiments consider the seven real datasets to describe various user distributions Px,y, namely Insurance (Van Der Putten & van Someren 2000 Adult (Dua & Grafi
2017), Postures (Gardner et al. 2014), Skin-nonskin (Chang Lin 2011 MNIST (LeCun et al. 2010
Mashion-MMIST (Xiao al. 2017]. and CIFAR10 (Krizhevsky et al. 2009) datasets. To minimize the variance caused by other factors, first consider homogeneous setting in Sections 3.1 and 3.2 for each competition, all predictors have the same number of seed data n² and budgets (,) the same classification algorithm
for f(), and the same AL algorithm for =(0, As for heterogeneous settings in Section 3.3 allowed to have different configurations of parameters.
competitors are
Throughout the experiments, the total number of competition rounds is T = 10 the number of predictors is M = 18, and a quality function is the correctness function, i.c ((1 Y2) = 1(Y = Y22) for all Yi, Y2 E y. We set the number for seed data points n² between 50 and 200 depending on the user dataset. We use either logistic model or neural network model with one hidden layer for f() As for the buying policy, we use standard catropy-based AL rule for π(1) (Settles & Craven 2008). We consider various competition situations by varying the budget nh E (0, 100, 200, 400 0 and the temperature parameter a E {0, 1,,1,2, 2, 4]. Note that a pair (mb, a)) generates one competition environment. We repeat experiments 30 times to obtain stable estimates for each pair (mb, a... We provide the full implementation details in Appendix A
To clearly capture the effect of the data purchase at a certain competition round, the data purchase procedures are not performed when we evaluate metrics. Since the evaluation metries are defined as the population-level quantity, it is difficult to compute the expectation exactly. To handle this problem, consider the sample averages using the i.i.d. held-out test data that are not used during the competition rounds.
3.1 Effects of data purchase quality
We first study how data purchase affects the overall quality and the QoE in various competition settings. As Figure illustrates, data purchase increases the overall quality as nh increases across all datascts. For instance, when a = 4 and the dataset. is Postures, the overall quality is 0.405 on average when 7hb = 0, but it increases to 0.440 and 0.464 when nb = 200 and n, = 400, which correspond to 9% and 14% increases, respectively. As for the QoE, however, data purchase mostly docrcases QoE as ny increases. For example, when the user distribution is Insurance and a 2, QoE is 0.875 when mi = 0, but it reduces to 0.867 and 0.814 when n4 200 and n, 400, which correspond to 1% and 7% reduction, respectively. For MNIST Fashion-MMIST although there are small increases when a 1, QoE decreases when = 4.
This can be explained as follows. Given that an ML predictor attempts to collect user data when its prediction is highly uncertain, this active data acquisition increases the predictability of the individual model and reduces the model's uncertainty. Similar to AL, data purchase effectively increases a model's predictability, and so does the overall quality.
II Section 3 for notational convenience, we often suppress the predictor index in the superscript if the context is clear. For example, we use ny instead of (1)
6

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
MNIST
0.55
0.3
Q00
0.8
Q0E
0.70
fly - 0
a-4
0.7
0.75
ng = 100
d=2
0.6
ng = 200
a=1
0.70
0.54
0.05
35r a.s 677 0.72
0.40 0.45
035
fp 400
a=0
Overall quality
Overall quality
Overall quality
CIFAR10
0.05
Postures
Skin-nonskin
Fashion-MNIST
0.9
0.15
0.000
0.75-
0.70
0.60
0.65
0.55
0.000
0.50
0.25
0.3
0800
0.2
044 042 843 0.4 0.47
0.26 0.70
0.30 0.40 3.45 5.50 0.55
6.10 0.1 nit 6.6 ain náo 622
Overall quality
Overall quality
Overall quality
Overall quality
Figure 2: Illustrations of Qol function of the overall quality in various levels of m. {0, 100, 200, 400} and a E {0, 2, on the seven datasets. Different color indicates different a, and the size of point indicates budgets ny. The larger budget is, the larger the point size is. In several settings, the overall quality increases as more budgets are used, but QoE decreases.
Insurance
Adult
MNIST
0.39
Diversity
Diversity
Diversity
0.5
0.25
1.0
4
03
Q=
0.4
Budget П6
Budget fb
Budget Ω6
Postures
Skin-nonskin
Fashion-MNIST
CIFAR10
3.40
2.2
Diversity
0.30
Diversity
1.2
Diversity
2.0
2.25
0.20
3.8
0:15
0.8
1.6
0.05
1.4
150
200
400
150
250
450
120
203
400
150
250
400
Budget
Budget Db
Budget
Ghb
Budget fb
Figure 3: Illustrations of the diversity as function of the budget ny for various a E 1, 4) on the seven datasets. Each color indicates different a. We denote 99% confidence band based 30 independent runs. Competing ML predictors become similar in the sense that the diversity decreases the budget increases.
In most cases, surprisingly, QoE decreases even when the overall quality increases. In other words, the quality that competing predictors provide is generally improved, but it does not necessarily mean that users will be more satisfied with the ML predictions. Although this result might sound counterintuitvve we demonstrate that it happens when the data purchase makes users experience fewer options, incrcasing the probability of finding low-quality predictions. To verify our hypothesis, we examine how data purchase affects diversity in the next section.
7

---

Published in Transactions on Machine Learning Research (11/2022)
(ng. diveeiiityy===i = 0.682)
(mg. diversity) (0, 0.384)
Class ID (Adult income level)
Class (Insurance type)
9.6
0.4
(np.diveriity) (100, 0.615)
(Np, diversity) (100, 0.312)
8.
(Nb, diversity) (200, 0.613)
0.0
(gg. diversity) (200, 0.258)
0.0
-0.2
(nb, diversity) = (400, 0.506)
(Np, diversity) (400, 0.203)
-0.6
-0.6
1011111110111007111
1011 101122331111111
Predictor ID
Predictor ID
Figure 4: Heatmaps of Q(iv) v) Q2vg(z) for (left) Insurance and (right) Adult datasets. We consider E f0, 100, 200, 400} and a = 4. The heatmaps in each row represent different n, but share the same color scale. For each heatmap, a horizontal axis indicates a predictor ID in {1...., 18} and vertical axis indicates a class in y = 21, The grid colored red (resp. blue) indicates a class-specific quality is higher (resp. lower) than average, and the white grid indicates the average. As the budget increases, the diversity decreases, and predictors produce similar class-specific quality.
Insurance
Adult
MNIST
1.2
1.4.
1.2:
2.0
Density
0.8
1.0-
Density
0.6
0.5
0.6-
nb 100
0.4
0.2
0.4
0.5
п. 200
0.00
0,0000
nb 400
Average quality Postures
Average quality
Average quality
Skin-nonskin
Fashion-MNIST
CIFAR10
2.5
3.0
2.5
Density
Density
1.25
Density
Density
1.5
3.5
2.00
2.5
3.5
55-
25
0,00
8.0.00
Average quality
Average quality
Average quality
Average quality
Figure Probability density plots of the average quality 1 E (( at near zero when ni E {0, 100, 200, 400} and a 4. Different color indicates different 7b. As n. increases, the average quality is more likely be close to zero. That is, the probability that all ML predictors produce low-quality prediction at the same time increases, and users might not be satisfied with the ML predictions after the competing predictors purchase data.
3.2 Effects of data purchase on diversity
We investigate the effect of data purchase on the diversity. Figure 3 illustrates the diversity as function of the budget nh, in various competition settings. In general, the diversity monotonically decreases as 7i increases across all datasets. That is, the competing predictors get similar as more budgets are allowed, and users get essentially fewer options when n, increascs. In particular, when a and the dataset is Adult, the diversity is 0.371 on average when n, 0, but reduces 0.270 and 0.187, which correspond 27% and 50% reduction, when n4 200 and na 400, respectively.
8

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
Fashion-MNIST
1.0
0.55
0.9
0.75
000
0.55
W
0.65
f6 0
0.75
fg 100
d=2
0.6
0.55
ng 200
d=1
0.52
0.54
0.56
0.58
0.60
0.67 u.5a c.aa 0.5o 0.71 0.72 0.75
0.30
0.35
0.00
045 0.50 9.53
400
Q=0
Overall quality
Overall quality
Overall quality
Postures
Skin-nonskin
MNIST
CIFAR10
0.55
0.975
0.85
0.50
0.953
0.8
0.75
0.925
0300
QoE
0.60
0.65
0875
9.70
0.55
0850
0.65
0.30
0823
2.00
0.3
0.00
0.2
0430422448 0.44
0.50
0.30 0.12 0:14 0.1 6.18 0.20 0.2
Overall quality
Overall quality
Overall quality
Overall quality
Figure Heterogeneous predictors. Illustrations of QoE as function of the overall quality when ML predictors have different buying strategies Different color indicates different. a, and the size of point indicates budgets Tb. The larger budget is, the larger the point size is. In several scttings, the overall quality increases more budgets are used, but QoE decreases.
We also compare the class-specific qualities of competing predictors. In Figure 4 we illustrate healmaps of the difference Q(j v) Quvg(v) where Q(j.v) is the class-specific quality defined as Q v) = E q (x)) Y = for j E [M] and E V, and Quvg(v) for E y is defined as 37 Σイ Q(i,))
This difference measures the class-specific quality of a company, showing how specialized its ML model is. We use the Insurance and Adult datasets. When Th, 0, the Adult heatmap shows that predictor 1 and predictor 5 specialize to class 2 prediction that they sacrifice their prediction power for class 1 compared to other predictors. In other words, competition rounds encourage each MT, model to be very specialized in a small subgroup of the population. However, when п₁ = 400, all predictors have similar levels of class-specific quality. The data purchase makes competing ML predictors similar and helps predictors not too much focus on a subgroup of the population. Similar results are shown in Ginart et al. (2021). but one finding that the previous works have not shown that this specialization can be alleviated when predictors purchase user data.
Implications of decreases in diversity We now examine the connection between the diversity and the quality of the predictor selected by user. We demonstrate that the probability of finding low-quality predictions can increase due to the reduction in diversity, explaining how diversity affects QoE. Figure 5 illustrates the probability density functions of the average quality near zero. It clearly shows that the probability that the average quality is near zero increases as more budgets 7i, are used: the areas for 74 400 (colored in yellow) are clearly larger than those for n, = (colored in red). That is, as predictions become similar, is more likely that all MT predictions have low quality simultancously. Hence, the probability that users are not satisfied with the predictions increases, and lcading docreases QoE even when the overall quality increases.
3.3 Robustness to modeling assumptions
We further demonstrate that our findings are robust against different modeling assumptions. We consider the same situation used in the previous soctions but ML predictors now can have different buying strategies n. To be more specific, we consider the three different types of buying strategies by varying the threshold of the uncertainty-based AL method. For Crnt E {0,0.3, 0.6), we consider the following buying strategy models
9

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
Fashion-MNIST
0.50
0.55
Diversity
Diversity
1.2
045
0.25
Q
4
2
0:35
0.20
1
630
0.15
150
400
100
200
100
200
450
0
Budget nh
Budget no
Budget
Postures
Skin-nonskin
MNIST
CIFAR10
0.9
0.35
1.4
Diversity
Diversity (stt
0.330
Diversity.
1.2
Diversity.
2.0
0.7
0.25
0.20
L.
0.6
0.35
0.8
0.30
Budget Fl
Budget Пb
Budget 0g
Budget ng
Figure 7: Heterogencous predictors. Illustrations of the diversity function of the budget ny when ML predictors have different buying strategies æ Different color indicates different We denote a 99% confidence band based on 30 independent runs. As the budget increases, the diversity decreases.
Tora (X2) 1((Ent(p))(X))))) (X)) 2 CEnt log({))))), where Ent is the Shannon's entropy function and (x(())) is the probability estimate given X,- We assume there are 6 predictors for each buying strategy To, To.3, and Wo.6. This modeling assumption considers the situation where there are three groups with different levels of sensitivity to data purchases. For instance, in our setting, mo.6 is the most conservative data buyer and is less likely to buy new data.
Figure shows the relationship between the QoE and the overall quality when there are heterogeneous ML predictors with different buying strategies. Similar to Figure 2 the overall quality increases but QoF generally decreases the budget increases across all datasets. As for the diversity, anticipated, Figure shows that the diversity decreases as the budget increases. It demonstrates that our findings are robustly obscrved against different environment settings. We also conduct more experiments (i) when budgets ny are different across companies or (ii) when there are different number of predictors M. All these additional results are provided in Appendix
4 Theoretical analysis on competition
In this section, we establish simple representation for QoE when quality function is the correctness function. Based on this finding, we theoretically analyze how the diversity-like quantity can affect QoE. Proofs are provided in Appendix
Lemma 1 (A simple representation for QoE). Suppose there is set of A 2 2 predictors {f6) J and a quality function is the correctness function, i.e., q(Y1, Y2) = 1({2) = Y23) for all Yi,Y2 E y. Let
Z = 1 N E1T (x, be the average quality for a user (X,Y). For any a 2 0, we have
Zen
E E = (Overall quality) S (QoE) = E Zea
(2)
where the inequalily holds when 0 and the expectation is considered over Рx,.
Lemma D presents a relationship between QoE and the overall quality QoE is always greater than the overall quality a O. In addition, it shows that QoE can be simplified as a function of the average quality Z over competitors when quality function g is the correctness function. Using the relationship shown in Lemma [ we provide a sufficient condition for the overall quality to be greater but the QoE be less in the following theorem.
10

---

Published in Transactions on Machine Learning Research (11/2022)
Theorem 1 (Comparison of two competition dynamics). Suppose there are two sets of M 2 predictors, Fi == ( and F2 := ر Without loss of generaliyy, the overall quality for F2 is larger than that of Fi- For the correctness function 4, we define Z1 := M g and Zz := Ni "== -
as Lemmal 1 If a 2 Ca and [ar[Zz] 2 C1 Var[Zi] for some explicit constants Ca and C1 S 1, then QoE for F2 is smaller than that for F1-
Theorem 1 compares two competition dynamics, F1 and F2: providing sufficient condition for when QoE for F2 is smaller than that for Fi whereas the associated overall quality is larger. Theorem | implies that QoE can decrease when Var is large enough compared to Var Var Considering
results in Figures that data purchase makes competing predictors similar when a is large enough, the average quality is more likely to become zero or one. As a result, it increases variance Var Z2] because the variance is maximized when random variables spread over [0, 1]. To be more specific, on Insurance, we compare two environments where one consists of competitors with ni = O, and the other has competitors with ni = 400 when a = 4. We denote a set of competing predictors by F1 when m. 0 and F2 when nb 400. In this setting, the constants are (C,: C2) (0.03, 0.08) and the variances are (Var(Z)), Var Var(Zz) (0.01, 0.11), making the inequality assumptions in Theorem 1 feasible.
4 and
Ginart et al. (021) theoretically examines how the number of competitors affects QoE, showing having too few or too many competitors may decrease the QoE. Our result in Theorem I focuses on the relationship between the overall accuracy and QoE, explaining why QoE decreases when ML predictors can actively acquire user data through the data purchase system, supporting our main findings experiments.
4.1 QoE for a general quality function
When the quality function q is not the correctness function, QoE does not have a comprehensible representation as Lemma function.
The following theorem shows the upper and lower bounds of QoE for a general quality [
Theorem 2. Suppose there is a set of M 2 prediction models {f
13-1- For any non-negative function q: y: x R and 2 0, we have the sollowing upper and lower bounds.
max a ( (x.10()))
(Overall quality) S QoE SE
(E[MS]
The equality for the left and right equations holds when a and a 00, respectively.
Theorem 2 shows that QoE is lower bounded by the overall quality as in Lemma I and is upper bounded by the expectation of maximum quality when a co. In other words, QoE is the highest quality of predictions available in the market when users have strong confidence in which predictions are best. The impact of different choices of quality functions on MQ, QoE, or the relation between them is an interesting research problem, but we leave it to a future topic.
5 Conclusion
In this paper, characterizing the nature of competition and data purchase, we propose a new competition environment in which MT. predictors are allowed to actively acquire labeled user data and improve their models. Our results show that even though the data purchase improves the quality that predictors provide, it can decrease the quality that users experience. We explain this counterintuitive finding by demonstrating that data purchase makes competing predictors similar to each other in various situations.
In order to derive tractable analysis and experiments, we have to make some modeling simplifications. Similar simplifications are commonly used in ML and multi-agent literature, and these are necessary here, especially because there lacks systematic analysis of data purchase in competition. For example, one assumption in our environment for simplicity is that the user distribution does not change over time. In practice, customer behavior can change or evolve over time (Jin & Vasserman 2019 Reimers & Shiller 2019). Another assumption we used the competitor side is that the MT. competitor's purchase intent is dichotomous:
11

---

Published in Transactions on Machine Learning Research (11/2022)
either buy not. In practice, willingness is often described with continuous values, as ML competitors pay the price much as they want, even affecting how users select service provider. Hence, it is important to expand various directions of our research in future works.
Broader impact statement
Our findings can broadly benefit the ML communities by providing insights into how competition over datasets and data acquisitions can affect a user's experiences. As more companies shift their focus to data science and AT, there will be increasing competition in acquiring valuable data. We believe it is important investigate the impact and potential biases that such competition induce in the machine Iearning training and predictions.
Acknowledgments
The authors would like thank all anonymous reviewers for their constructive comments.
References
Subramanian Arumugam and R Bhargavi. A survey on driving behavior analysis in usage based insurance using big data. Journal of Big Data, 6(1):1-21, 2019.
Yu Bai and Chi Jin. Provable self-play algorithms for competitive reinforcement learning. In International conference on machine learning, pp. 551 560. PMLR, 2020.
Lucian Busoniu, Robert Babuska, and Bart De Schutter. A comprehensive survey multiagent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part (Applications and Reviews), 38 (2):156-172) 2008.
Chih-Chung Chang and Chih-Jen Lin. Libsvm: A library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):1227, 2011.
Dheeru Dua and Cascy Graff. Uci machine learning repository, 2017. Uttphttp://rccciee.. uci. edd..ddu/ie ml
Ran El-Yaniv and Yair Wiener. Active learning via perfect selective classification. The Journal of Machine Learning Research, 13(1):255-279, 2012.
Mahsa Familmalcki, Alireza Aghighi, and Kambiz Hamidi. Analyzing the influence of sales promotion on customer purchasing behavior. International Journal of Economics & management sciences, 4(4):166, 2015.
Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras, Philip HS Torr, Pushmeet Kohli, and Shimon Whileson. Stabilising experience replay for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pp. 1146 1155. PMLR, 2017.
Yoav Freund, Sebastian Seung, Eli Shamir, and Naftali Tishby. Selective sampling using the query by committee algorithm. Machine learning, 28(2):133 168, 1997.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, pp. 1183 1192, 2017.
Andrew Gardner, Christian A Duncan, Jinko Kanno, and Rastko Selmic. 3d hand posture recognition from small unlabeled point sets. In 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 164-169. IEEE, 2014.
Tony Ginart, Eva Zhang, Yongchan Kwon, and James Zou. Competing ai: How does competition feedback affect machine learning? In International Conference Artificial Intelligence and Statistics, pp. 1693 1701. PMLR, 2021.
Steve Hanneke et al. Rates of convergence in active learning. The Annals of Statistics, 39(1):333 361, 2011.
12

---

Published in Transactions on Machine Learning Research (11/2022)
Yizhou Jin and Shoshana Vasserman. Buying data from consumers: The impact of monitoring programs in auto insurance. Unpublished manuscripl. Harvard Universily, Department of Economics, Cambridge, MA, 2019.
Takafumi Kanamori. Pool-based active learning with optimal sampling distribution and its information geometrical intorpretation. Neurocomputing, 71(1-3):353-300 2007.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXio:1412.6988, 2014.
Ksenia Konyushkova, Raphael Sznitman, and Pascal Fua. Learning active learning from data. In Advances in Neural Information Processing Systems, pp. 4225 4235, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Lobs [Online]. Available: http://yann..eeeee hom/erdb/eee /erdb/mmiitt 2, 2010.
Michael L Littman. Markov games framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, pp. 157-163. Elsevier, 1994.
Robert Loftin, Aadirupa Saha, Sam Devlin, and Katja Hofmann. Strategically efficient exploration in competitive multi-agent reinforcement learning. In Uncertainty in Artificial Intelligence, pp. 1587-1596. PMLR, 2021.
Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAl Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-oompetiiiee environments. In Advances in Neural Information Processing Systems, pp. 6379-6390, 2017.
Jürg Meierhofer, Thilo Stadelmann, and Mark Cieliebak. Data products. In Applied Data Science, pp. 47-61. Springer, 2019.
Imke Reimers and Benjamin R Shiller. The impacts of telematics on competition and consumer behavior insurance. The Journal of Law and Economics, 62(4):613 632, 2019.
Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po- Yao Huang, Zhihui Li, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. arXiv preprint arxiv:2009.000 00236, 2020.
Jennifer Rowley. Promotion and marketing communications in the information marketplace. Library review, 1998.
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations, 2018.
Kumba Sennaar. How america's top 4 insurance companies are using machine learning. https://emerj come ltt.rrrreeeeeeeccaaeeeeaeaeaaaaa
2019. Posted February 26, 2020;
Retrieved May 19, 2021.
Burr Settles. Active learning literature survey. Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.
Burr Settles and Mark Craven. An analysis of active learning strategies for sequence labeling tasks. In Proceedings of the 2008 Conference on Empirical Methods én Natural Language Processing, pp. 1070-1079, 2008.
Ardi Tampuu, Tambet Matiisen, Dorian Kodelja, 1lya Kuzovkin, Kristjan Korjus, Juhan Aru, Jaan Aru, and Raul Vicente. Multiagent cooperation and competition with deep reinforcement learning. PloS onc, 12(4): e0172395, 2017.
Peter Van Der Putten and Maarten van Someren. Coil challenge 2000: The insurance company case. Technical report, Technical Report 2000-06 09, Leiden Institute of Advanced Computer Science, 2000.
13

---

Published in Transactions on Machine Learning Research (11/2022)
Oriol Vinyals, Igor Babuschkin, Wojeiech M Czarnecki, Michaël Mathicu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575(7782):350 354, 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.0711 2017.
Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview theories and algorithms. arXiv preprint orXiv:1911.1063: 2019.
Indré Zliobaité, Albert Bifet, Bernhard Pfahringer, and Geoftrey Holmes. Active learning with drifting streaming data. JEEE trunsactions newral nelworks and learning systems, 25(1):27-30 -9, 2013.
Appendix
In this appendix, provide implementation details in Appendix A additional numerical experiments in Appendix B and proofs and additional theoretical results in Appendix D
A Implementation details
In this section, provide implementation details. We explain the user distribution, ML predictors, and the proposed environment. Our Python-based implementations are available at htttps://github.com (xwon0007//data_purchhee lectiaasseiii______e_
Datasets (user distribution) As for the datasets (user distribution Px,y), we used the following seven
datasets for our experiments: Insurance (Van Der Putten van Someren 2000), Adult (Dua & Graff 2017)
Postures (Gardner et al.] 2014), Skin-nonskin (Chang & Lin 2011). FashionMNIST (Xiao al., 2017):
MNIST [Leunn al. 2010), and CIFAR10 (Krizhevsky et al.) 2009). For all datasets, we first split a dataset into competition and evaluation datasets: the competition dataset is used during the T = 10a competition rounds and the evaluation dataset is used for evaluate metrics after the competition. For Fashi OMMIIST, MNIST, and CIFAR10, we use the original training and test datasets for competition and evaluation datasets, respectively. For Insurance, Adult, Postures, and Mkinnnnnskin, we randomly sample 5000 data points from the original dataset to make the evaluation dataset and use the remaining data points as the competition dataset. To account for uncertainty user selection, user labels are randomly flipped for 30% of user data points. At each round of competition, we randomly sample one data point from the competition dataset. After the T competition rounds, we randomly sample 3000 points from the evaluation dataset and evaluate the metrics (the overall quality, QoE, and diversity). Note that all of experiment results based the evaluation dataset. Table 1 shows summary of the seven datasets used in our experiments.
Table 1: A summary of datasets used in our experiments.
The size of
The size of
Dataset
Input dimension
# of classes |
competition dataset evaluation dataset
Insurance
13823
5000
16
2
Adult
43842
5000
108
2
Postures
69975
5000
15
5
Skin-nonskin
239057
5000
2
Fashion-MNIST
60000
10000
784
10
MNIST
60000
10000
784
10
CIFAR10
50000
10000
3072
10
As for the preprocessing, we apply the standardization to have zero mean and one standard deviation for Skin-nonskin. For the two image datasets, MNIST and CIFAR10 we apply the channel-wise standardization.
14

---

Published in Transactions on Machine Learning Research (11/2022)
Other than the three datasets, we do not apply any other preprocessing. To reflect the customers' randomness in their selection, apply random noise the original label. We assign random label with 30% for every dataset. This random perturbation is applied to both the competition and evaluation datasets.
ML predictors We fix the number of predictors to M = 18 throughout our experiments. For each dataset, which makes one competition environment, we consider a homogeneous setting, i.e,, all predictors have the number of seed data ng budget nh
(6) model f(), and buying strategy x(), As for the buying
strategy, we fix x00()))) = ((EEn((p()) (X+)) 2 0.3 log({>"))), where Ent(p()) (X.)) is the Shannon's entropy of (X),, and p()) (X,) is the corresponding probability estimate for P(Y = -)). That is, if the entropy is higher than the pre-defined threshold 0.3 log(()))), a predictor decides to buy the user data. Note that log( is the Shannon's 's entropy of the uniform distribution on V
Table shows a summary information for the seed data n, and the model f for each dataset. Every ML predictor initially trains with the n, seed data points. For all experiments, Kingma & Ba 2014) with the specified learning rate and epochs. The batch size is fixed to 64. If an predictor is selected, then its ML model is updated with one iteration with the newly obtained data point, and we retrain the model whenever the 'retrain period' new samples are obtained.
the Adam optimization
Table 2: A summary of hyperparameters by datasets. Logistic denotes logistic model and NN denotes neural network one hidden layer.
ML prodictor f
Dataset
Sced data ng
Model
# of hidden nodes
Epoch
Learning rate
Retrain period
Insurance
100
Logistic
5 x 10-3 3
50
Adult
100
Logistic
10-2
50
Postures
200
Logistic
10
3 10-2
50
Skin-nonskin
50
Logistic
10
3 x 10
50
FaSHion-MNIST
50
NN
400
30
10-4
150
MNIST
50
NN
400
30
10-4
150
CIFAR10
100
NN
400
30
10-4
150
B Additional numerical experiments
In this section, provide additional experimental results to demonstrate the robustness of our findings against different modeling assumptions in the heterogeneous setting. As for the helerogeneous settings, we consider different budgets in the subsection B.1] and different number of competing predictors in the subsection B.2 All additional results again show the robustness of our experimental findings against different modeling assumptions.
B.1 Different budgets
We the setting used in the homogeneous setting but with different budgets. We the Insurance, Adult, and Skin-nonskin datasets. For nip {0, 100, 200, 400). we assume that the first predictors have (8) budgets, but the last predictors have (2) /2 budgets. That is, half of the predictors have half the budget compared to the other group. This situation can be considered as some groups of companies have a larger amount of capital than others. Figure S shows that the main findings appear again even when different budgets are used (QoE generally decreases, overall quality increases, and diversity decreases).
B.2 Different number of competing predictors
We also show that our findings are consistent for the different number of competing predictors in the market. All the experiments in Section of the manuscript consider M 18. Here, consider the homogeneous
15

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
Skin-nonskin
13
0:35
-p=
100
g=2
0.322
64
6.00
Fb= 200
m11
Overall quality Insurance
Overall quality Adult
0.16
6TA a74 330
quality
Py = 400
a=0
Skin-nonskin
0.4
0.25
0.29
0.40
Diversi
0.35
6.53
Diversity
0:30
Diversity
0.0
0.15
2.
0.35
Budget
Budget
Budget
Insurance
Adult
Skin-nonskin
1.0
0.2
Density
Density
3.6
Donsity
п =
Ωi = 100
0.3
nn 200
5.
63 Average quality
0.00
0.23
6.16
п. 400
Average quality
Average quality
Figure 8: Heterogeneous predictors. Main figures when competing predictors use different budgets. The results are similar to the homogeneous setting, showing the robustness our main findings.
setting but different number of competing predictors = M 12. As Figures and 10 show, the main findings are captured again when the number of predictors are used.
B.3 Different active learning algorithms
We conduct additional experiments with different active learning algorithms. Specifically, we consider confidence-based active learning algorithms, i.e., competitors decide to buy user datum based on the confidence of their model prediction. Specifically, the confidence we considered is the maximum predicted probabilities over different classes:
(X,) = ((mmax(p,)) (X,)) & 122/1))).
(3)
jEY
Given that maxjey(p, (X+)) is always bigger than 1/13%, our confidence-based active learning algorithm in will try to purchase user data predicted probability vector close to the random guess. As Figure B.3 shows on two datasets (Skin-nonskin and Adult), our main findings are consistently observed even when compoting predictors use different active learning algorithms for data purchase.
C Consistent results from relaxing model assumptions
In this section, we relax the model assumptions make proposed competition environments more realistic. We consider the following two extensions: (i) each user datum has different value and (ii) chooses one competitor based on their prediction quality even when there is more than one buyer.
The setting (i) is to consider the case where some user data can be cleaner or noisier than others, and accordingly, some data values are more expensive or cheaper than others. Specifically, we divide users into two groups: clean group and noisy group. We assign a user the noisy groups its label is changed from the random flip, and the rest are in the clean group. Note that labels have been randomly flipped 30% of users in the homogeneous setting. We set the data prices for the clean and noisy groups 8/7 and 2/3,
16

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
Skin-nonskin
1.
0.4
0.0
040
6.4
0.41
-p=
0.75
388
100
g=2
Fb= 200
m11
6.001
Overall quality Insurance
Overall quality Adult
6.6 quality Skin-nonskin
6.76
Py = 400
a=0
Diversity
0.6
Diversity
2.35
Diversity
0.0
0.5
0.2-
0.0
4
222
0.35
0.00
Budget fi
Budget
Budget
Insurance
Adult
Skin-nonskin
1.75
1.75
2.5
Density
1:2
Density
1.25
Density
n. =
1.0
n: 100
335
25
nb = 200
022
126
01
3.0
0 Average quality
805 0.05 0s15 360 Average quality
n6 400
Average quality
Figure 9: Heterogeneous predictors. Main figures when there are M 9 competing predictors. The results are similar to the M = 18, showing the robustness of our main results.
respectively. This choice of data price does not change the total amount of the price from the homogeneous setting.
The sctting (ii) is to consider the sctting where a user prefers a buyer with a high-quality model prediction. Specifically, we consider the Equation but with buyers instead of selecting one uniformly at random. At t-th round, for a set of buyers Be S [M] and e Br,
exp aq (Y, (
P P(W, = Ye (x)(be)))))
EjeB, exp ag (Ye, ff f(² (())
Figures 12 and 13 show the main figures for the settings (i) and (ii), respectively. We consider Skin-nonskin and Adult datasets for user data distributions. Experimental results show that main findings are consistent under these relaxations-i.e. the prediction quality users experience can decrease even when the overall quality increases.
D Proofs and additional theoretical results
We provide proofs for Lemma and Theorem in the subsection D.I
D.1 Proofs
Proof of Lemma
For notational convenience, set g(t) = f() (X)) for E [M].
E q (Y, (W())())))
E (Y, (w(())))) 24 (
= E
Σ 2 (a)g
(4)
17

---

Published in Transactions on Machine Learning Research (11/2022)
Insurance
Adult
Skin-nonskin
1.
5.8
1.00
mp=
082
100
g=2
6.
Fy= 200
m11
Overall quality
Overall quality Adult
033 Overall quality
Py = 400
a=0
Insurance
Skin-nonske
0.35
0.25
Diversity
Diversity
Diversity
0.
0.2
0.20
4
0.
22
0.45
03
03
a=
Budget fi.
Budget
Budget
Insurance
Adult
Skin-nonskin
Density
3-
1.25
Donsity
0.6
1.00
п =
0.13
Ωi = 100
3.50
0.2
nn 200
56
63
П. 400
Average quality
Average quality
Average quality
Figure 10: Heterogeneous predictors. Main figures when there are M = 12 competing predictors. The results are similar to the M = 18, showing the robustness our main results.
Skin-nonskin
Adult
0.95
3.94
0.02
3.90
3.0
0.05
Q=4 fl
fby =
3.44
0.75
fg -6000
g=2
3.0
3.90
8.735 FCB
753786 773 R6:
167 eés aás e am
ng 400
g=0
Overall quality
Overall quality
Skin-nonskin
Adult
1.40
0.00
0.30
Diversity
Diversity
6.30
3.20
0:20
2114
6:00
0 2
230
3:0
6.15
180
Budget fb
Budget n
Figure 11: Different active learning algorithms. Main figures when competing predictors use the confidence-basee active learning algorithm for data purchase. The results are similar to the uncertainty-based active learning algorithm cases, showing the robustness of our main results.
where for E [M],
exp (gq(t)
(a)
j= exp (ag(i))
Since /¹ = f()())) {0, 1], for Neor EN q(Y, f(1) (X)) 1((Y (X)))) we have
exp(a1(1Y = (((())))
(() = Noor exp(a) + (M Noor
18

---

Published in Transactions on Machine Learning Research (11/2022)
Skin-nonskin
Adult
2.6
1.05
1.40
3.6-
8
7.00
398-
fly
2.84
100
o=2
Fb. 200
g=1
oss abs quality Adult
Fby 400
a=0
Overall quality
Skin-nonskin
2.00
0.40
0.35
Diversity
Diversity
3.24
5:30
3.20
0:30
4
1.5
Budget
Budget
Figure 12: Non-constant data price: We consider the setting where the user data price is fixed to constant.
Skin-nonskin
Adult
3.94
3.90
3.0
0
g=1
3.6
0.75
nu =100
a=2
3.2
fb 200
g=1
fly
Overall quality
Overall quality
Skin-nonskin
Adult
Diversity
3.24
Diversity
0.30
4
0.25
104
8:20
160
400
300
Budget
Budget Do
Figure 13: Non-random selection: In this setting, user selects one of buyers based their prediction quality.
19

---

Published in Transactions on Machine Learning Research (11/2022)
and
1 M
(Neor/ (Mee (Noot/ M e + (1 - Noor/M) (Noos Moor + (1 - Noor/M)
e (1Y = (xi))
(a)g())
M
Since Z Neor/M M and k(z, a) se" ze- (1-s) is an increasing function, it concludes proof.
Proof of Theorem 1 For S 2 1 and a 2 0, let k(z,a)
26 (1-+(1=-)² M1 E [21], and μ2 13 [Z2]. Note
that
E
q Y, ((
= i4 +1 (A Z1|
E
g (w)
Y,
= 42 + +2
(((2, Z2].
Thus, we have
E q (Y, (wwoooo))))
2 E ( (x)x)((())))x
E (z)1 - _1] - (k( -Z2] 12 pl.
For Z E { 11 .......
M-1) M
}, we have
M(e -1) 1)
ea -
M(e--1)
(M- - 1)e" +1 S
Ze + (1 - Z)
et + (M 1)
Therefore, since k(z) - 2(1 2(--2)e--11
we have
Z0 +01+(1-)
(M - 1)ca + 7(1 - 2) S k(Z) - 2 S M(e 1)
M(e" - 1)
-(11-22
2(1 2).
(5)
Ea + (M 1)
Let Cinw Mi(e"
[ and Cupp M(e--1) 3-+(M-1)
From the inequalities 5 we have
E (M( - - E ()) a)) - [ 2 Clow E [Z| - Z Cupp E Zz 22)]
Clow (M) (1 - m1) Var(Z Cupp (uz p2) - Var (2)).
The last equality is due to E [2(1 Z |Z| E [2]) Var(Z). Therefore, QoE is decrcased if
(Mi (1
m1)= Var Z1) (up w2) Var[Zz]) 2 p22 j41
Clow (Var [z] (1 -m1) Cupp
1
Var[Z2] 2
(w2 m1) + #2(1 k22
Cupp
Var[Zz] 2 C Var [11 C2(M1.12);
where
Clow
+ (M - 1)
C1 =
< 1
Cupp (M 1)ea + 1
C2(p1;12) == -1114(11 - -111) + Cupp
(12 m1) + 12 - 122)
Therefore, if there is a constant Co such that C2 2 0, then it concludes a proof.
By definition of C2, it is positive when p02(1 n2) Cinn (1 - m1).
(2 (2)) Cifm (1 -1) > 0
20

---

Published in Transactions on Machine Learning Research (11/2022)
C H2 u2)
H1 (4)
c
(M- - 1)
P22 112)
(M- 1) + <
M1 (1(1- - 11)
(M 1) (1 p1) - H2 U2)
e
(M 1) U22 M1 (1 141
By setting Ca log (M-1w) (1
-
19
M2 - m2) m1)
it concludes proof.
[M w 20)
11
We use the same notations in the proof of Lemma 1 function as a. From the representation we have
Proof of Theorem
We first show QoE is an increasing
bE
(a)(())))
da
(a) Oox
g())
E
IM exp (ag(ti))
(expp exp (aql") Exe exp (ag 90(3)
exp (aql)))())
M
E
g0)
Σ E/ exp (aglii)
j=1
M
Σ
())((())) -
=
where q DM (b)) (a)g
From the last equality, have
M
M -79 (1
Σ
(a)
Σ
(1)
-
>00 -
Note the non-negativity is from Cauchy-Schward inequality. We now prove an upper bound. Note that
M
b (a)g
max 9()) JE[M]
and the equality holds when oc. Therefore, taking expectations on both sides provides an upper bound. As for the lower bound. Due the representation it is enough to show that
M
Σ
(a)gg)) > Σ S
Since QoE an increasing function, by plugging a = 0, it gives (a) 1/M. concludes a proof.
21


Reviews:
[{"review_id": "HsWs0piF8I", "reviewer": "Reviewer_WG6Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. The paper explores an interesting research topic: the impacts of data purchase on ML competition. To the best of my knowledge, this is a research area that is less explored in the literature. \n2. The paper establishes a formulation to model the competition environment, and conducts extensive empirical experiments on multiple real datasets to examine the impacts of data purchase on the predictor quality, the quality of user experiences, and diversity. Importantly, the theoretical results are consistent with the empirical findings. \n\nWeaknesses: \n1. My biggest concern is that the main conclusion of the paper (Theorem 1), i.e., the quality of user experience may decrease even as the quality of each individual predictor improves, heavily depends on the modeling assumption. Specifically, the paper implicitly assumes that the users do not receive predictions if there is a buyer who shows the purchase intent and the quality of user experiences is only calculated based on the settings with no buyer. However, this assumption may not be realistic. If we change the definitions of predictor overall quality and quality of user experience, the conclusion may no longer hold. \n2. The sufficient conditions of the theoretical results are quite strong, i.e., the conclusion holds only when users are sufficiently confident in their selections ($\\alpha\\geq C_{\\alpha}$), and when the variance of the set of predictors with the higher overall predictor quality is smaller. How about other settings? More results on other problem settings would make the paper a much stronger submission.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the impacts of data purchase in ML competition; it first constructs a competition environment where a set of predictors interact with data owners over multiple rounds: at each round, a data owner reveals the unlabeled data to predictors who then decide whether to purchase the data with their budgets or not. If at least one predictor shows purchase intent, then the data owner will select one uniformly; otherwise, the data owner will receive the prediction from predictors with probability proportional to the prediction accuracy. For both cases, only the predictor that got selected receives the true label. Based on this environment, the paper empirically examines the impacts of data purchase on the predictor quality, the quality of user experiences, and diversity; and it theoretically identifies conditions under which the quality of user experience diminishes as individual predictor quality improves.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. The user quality of experience (QoE) is only calculated based on the settings when there is no buyer, and it seems that the users with buyer(s) will NOT receive the predictions. More justification on this is needed (i.e., why does QoE not depend on the settings with buyers). In example 1 (auto insurance) given by the authors, if the company attracts a user by offering a discount, doesn’t the user also receive the insurance plan from this company?   \n2. I am wondering whether the model can be generalized to a setting where different predictors compete for a potential user by offering different prices, and the user selects the buyer offering the highest price (in contrast to the present model where the price for purchasing a data point is one for every predictor and the user selects the buyer uniformly). If so, how may the conclusions be affected; if not, what are the challenges?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Z71hcYTIlI", "reviewer": "Reviewer_j99t", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is clear but would benefit from the use of a more detailed formalization of some quantities used in the paper, especially in the experimental part. Even if the model used is clearly described, the modeling choices are not always clear and require either an empirical motivation or reference to motivate their use.\n\nI think this work has a valuable interest from an application point of view. The outcomes provided by the experiments are an interesting starting point for the design of algorithms reaching some kind of equilibrium under specific budget constraints.\nConversely, this work would greatly increase its interest by the scientific community if some theoretical analysis of the dynamics of the system is added. I think this lack reduces the novelty of the paper by a great margin.\nFinally, the authors provided some discussion about the similarities with the MARL field. I suggest expanding such a discussion and clarifying more the difference between what is presented here and the existing MARL settings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studies the dynamics of the process of data acquisition in an environment in which multiple agents are providing an ML service and want to improve their service thanks to new data. The paper formalizes this multi-agent setting and provides simulations to show the effect of the interaction among the agents. Finally, they also provide some theoretical results on the evolution of the system in a specific case in which two distinct sets of ML services are present in the market.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "\"To analyze ... predictions.\" This is not clear to me. From the description you provide in the following paragraph, you are selecting at random if anyone is interested in buying the data. Why not choose, even in this case, based on the performance of the classifier?\n\nIs there empirical evidence that the user chooses the service according to Equation (1)? Otherwise, it seems like you made an arbitrary choice of this function.\n\nPlease define some symbols for the metrics in the experimental part, so they can also be used in the figures as labels.\n\nThe difference between this work and the one by Ginart et al. 2021 is stated in the related works, which are placed at the end of the paper. On the one hand, since you did not provide any technical comparison in this section, I would suggest moving it to the first part of the paper to give the reader a clear idea of the novelty of the proposed work. On the other hand, I think a comparison in terms of theoretical results with the ones of Ginart et al. 2021 would be a nice addition to the paper.\n\nThe choices of the parameters in the experimental part are not justified or motivated. This reduces the value of your findings since it may happen that in real-world situations, the behaviour might be different. I suggest justifying how you choose these parameters.\n\n\"As the evaluation time ... rounds.\" This concept is not clear, please rephrase.\n\n\"In other words, ... subgroup.\" It is not clear to me what you mean by subgroup.\n\nIt is not clear to me why you decided to move the lower-upper bounds on the QoE in the appendix. I suggest you move them to the main paper.\nMoreover, I suggest providing more insights and correspondence with the empirical findings you presented. The same holds for Theorem 1. I think it would be better to refer to the specific result to compare this result and the one obtained in the previous section.\n\nIs the experimental setting fulfilling all the assumptions needed for Theorem 1? How do you define the two sets F_1 and F_2?", "broader_impact_concerns": "I think that the discussion about the broader impact is correct and does not require substantial modifications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SrVejptntN", "reviewer": "Reviewer_WCny", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, I think the paper studies an interesting problem, but I have following comments that I hope authors can address in the revised manuscript.\n\nStrengths:\n\n1. The problem setup is interesting;\n2. The paper is well-written;\n3. Numerical findings are interesting.\n\nWeakness:\n\n1. The modeling part, especially the modeling of the predictor's decision is weak;\n2. The paper does not have very strong theoretical support.\n\nPlease see my major comments below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies an interesting environment in which ML predictors use active learning algorithms to effectively acquire labeled data within their budgets while competing against each other. It models the problem from two sides. On the one side, it models that M competing predictors independently train their model using some seed data points.  After initialization, each predictor determines whether to buy the user data. On the other side, the paper also models how a user selects one predictor. At each round, the user selects only one predictor based on both purchase intents and prediction information. The paper empirically shows that the overall performance of an ML predictor improves when predictors can purchase additional labeled data. It also shows a surprising result that the accuracy of the predictor selected by each user can decrease even as the individual predictors get better.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. Problem setup: \n\n- It assumes that all labeled data would be worth one unit of budget. I think this assumption is a bit restrictive. Different data may have different values, especially when some data is much more difficult to obtain. For example, some products are much more popular than others so the data should be cheaper to purchase.\n\n- The modeling part on the predictor side is a little bit weak.\n   To model the problem more realistically, I think it is natural to let the predictors decide how much they are willing to pay for the data. Then the willingness of the user selecting a particular predictor not only depends on how good the predictor is, but also the budget that they pay for. This point is reflected in Example 1. In that example, each predictor is one insurance company, offering an auto insurance plan based on what it predicts to be the most suitable for the driver. The company can offer discounts to attract the customer. Then the decision of the customer not only depends on the suitableness of the plan, but also how much the company is willing to pay/offer.\n   I think it is important to adjust this part to a more realistic setting.\n\n2. Predictor's decision: the paper does not discuss much about how the predictor makes the decision. On page 2, it states that \"ML predictors can use any stream-based AL algorithm. For instance, a predictor can use the uncertainty-based AL rule.\" The predictor's decision may play a very important rule in this problem. I encourage authors to include more discussions as well as numerical experiments on how different active learning algorithm would lead to different/similar results.\n\nThe data distribution P_{X,Y}: Is the data distribution known to all predictors and how do they utilize this piece of information in their budget allocation? How do different distributions influence the final result? This distribution can also change over the time. It would help readers to get some interesting intuitions if authors can add some numerical experiments on non-stationary data distributions.\n\n\n3. Numerical experiments: The paper studies effects of data purchase on quality in Section 3.1 and effects of data purchase on diversity in Section 3.2. Figure 2 shows QoE as a function of the overall quality in various levels of budget and \\alpha.  It is interesting to observe that in Fashion-MNIST in Figure 2, the yellow curve decreases with overall quality while blue, green curves increase with overall quality. Could you provide more intuition of that? In Skin-nonskin in Figure 3, red curve increases with the budget. Why does that happen?\n\nAs mentioned in point 2, can you implement different AL algorithm on these datasets to verify the effectiveness and performance of other AL algorithms?\n\nMinor Comments\n\n- To help readers to understand easily about the problem setup etc., the paper should provide more intuitive examples.\n- Probably it is better to move Section 5 Related works right after Section 1 as Section 2.\n\n", "broader_impact_concerns": "I hope authors can also discuss the fairness issue that might be involved in this problem from both the predictor and customer's side.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers are in agreement that this paper studies an interesting setting that models the impacts of data purchase on ML competitions and has practical values, providing both theoretical results and empirical simulations. They also agree that the revision and the newly added experiments in relaxed environments addressed some of their their initial concerns. 

The reviewers would have appreciated a more detailed and deep discussion on some of the issues raised by them (e.g. the second bullet point of Reviewer WCny's comment 1).
Decision: Accept (poster)