Fairshare Data Pricing via Data Valuation for Large Language Models
Authors: Luyang Zhang
Cathy Jiao
Beibei Li
Chenyan Xiong
Venue: NeurIPS 2025 poster ()
Keywords: data pricing
data market
large language model
data valuation
data attribution
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: We present a fairshare training data pricing framework for large language models  where data pricing depends on the data valuation (which estimates data contribution towards model learning), and benefits both sellers and buyers.

Abstract: Training data is the backbone of large language models (LLMs), yet today’s data markets often operate under exploitative pricing -- sourcing data from marginalized groups with little pay or recognition. This paper introduces a theoretical framework for LLM data markets, modeling the strategic interactions between buyers (LLM builders) and sellers (human annotators). 
We begin with theoretical and empirical analysis showing how exploitative pricing drives high-quality sellers out of the market, degrading data quality and long-term model performance. 
Then we introduce fairshare, a pricing mechanism grounded in data valuation that quantifies each data’s contribution. It aligns incentives by sustaining seller participation and optimizing utility for both buyers and sellers. 
Theoretically, we show that fairshare yields mutually optimal outcomes: maximizing long-term buyer utility and seller profit while sustaining market participation. 
Empirically when training open-source LLMs on complex NLP tasks, including math problems, medical diagnosis, and physical reasoning,
fairshare boosts seller earnings and ensures a stable supply of high-quality data, while improving buyers’ performance-per-dollar and long-term welfare. Our findings offer a concrete path toward fair, transparent, and economically sustainable data markets for LLM. Our code will be open sourced.

Fairshare Data Pricing via Data Valuation for Large Language Models
Luyang Zhang
Cathy Jiao Carnegie Mellon University cljiaoBcs. cmu. edu
Carnegie Mellon University luyangz0andrew. cmu. edu
Beibei Li
Chenyan Xiong Carnegie Mellon University cx9cs. cmu edu
Carnegie Mellon University beibeili Seeeili@nndee.... cmu edu
Abstract
Training data is the backbone large language models (LLMs), yet today's ' data markets often operate under exploitative pricing sourcing data from marginalized groups with little pay or recognition. This paper introduces theoretical framework for LLM data markets, modeling the strategic interactions between buyers (LLLM builders) and sellers (human annotators). We begin with theoretical and empirical analysis showing how exploitative pricing drives high-quality sellers out of the market, degrading data quality and long-term model performance. Then we intro- duce fairshare, pricing mechanism grounded in data valuation that quantifies each data's contribution. aligns incentives by sustaining seller participation and optimizing utility for both buyers and sellers. Theoretically, fairshare yields mutually optimal outcomes: maximizing long-teri buyer utility and seller profit while sustaining market participation. Empirically when training open-source LLMS on complex NLP tasks, including math problems, medical diagnosis, and physical reasoning, fairshare fairshae boosts seller earnings and ensures stable supply of high-quality data, while improving buyers' iefformanne-perddoolee and long-term welfare. Our findings offer concrete path toward fair, transparent, and economically sustainable data markets for LLM.
show that
Introduction
High-quality training data is foundational to building effective and reliable large language models (LLMs). As LLMs take on increasingly complex tasks today such as coding [[], reasoning (2), and A14Science 3 they rely heavily on carefully curated, human-annotated data. This growing demand has triggered "generative data gold rush". with major tech companies racing to acquire training data, fueling the rise of a nascent AI data market [4. In this market, Al firms create networks of short-term contract workers to generate data labels, resembling an Uber-like gig economy for data [4].
However, the current Al data market operates with limited oversight and is widely criticized for a lack of transparency and faimess in pricing (5/8. Data prices largely low and fail reflect the quality or effort involved, threatening the sustainability and quality of data supply 1111 In particular, for data sellers, such human annotators content creators, the prevailing market routinely undervalues their labor, offering compensation that neglects the skill, effort, and downstream value of their contributions. These harms are especially concentrated in low-wage labor markets, where annotators often face overwork, underpayment, and exclusion from decision-making [12]. This reflects broader ethical concern known "AI parachuting". where developers extract data from marginalized
* Equal Contribution. +Co-Advisors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

communities [13]55], tieing to wider debates on epistemic injustice and data colonialism 18) As one civil rights advocate noted on 60 Minutes, "They don't t pay well. they could pay whatever, and have whatever working conditions."!
Motivated by these issues, we present fair pricing framework for the LLM training data market to promote equitable and sustainable generative AI ecosystems. Economic theory suggests that prices should reflect the value delivered the buyer - signaling quality and alignment [19]. Guided by this, fairshher our framework introduces fairshare pricing based on established data valuation techniques for LLMs which quantify each dataset's contribution to model performance. Buyers and sellers both have access to data valuation scores, which guide decision maakig on both sides: buyers use them to select datasets under budget constraints to maximize utility (i.e., a standard measure of welfare or satisfaction [4 (25)), while sellers use them to set prices based on the anticipated demand from buyers. This shared access - enabled by our assumption of information transparency supports procedural fairness [26). improves seller participation, and enhances overall data quality.
20 23
Our theoretical and empirical findings show that fairshare pricing offers clear advantages compared to existing methods. First, we show that existing exploitative pricing leads to a lose-lose outcome for the data market. For data buyers, underpaying data sellers might cut costs in the short term but it comes at a cost of drive sellers away, shrinking the supply of high-quality training data. This weakens the data pipeline and limits model improvement, even as investments grow. In contrast, we show theoretically that fairshare pricing leads to a win-win outcome: sellers maximize profit while remaining engaged, and buyers secure long-tenn -erm-termmm utility by maintaining access to high-quality data.
Second, we empirically validate our approach through simulations of buyer-seller interactions in data markets. We focus on training open-source LLMs complex NLP tasks, including math problems (27]. medical diagnosis (28). and physical reasoning (29]. Analyzing both pricing and valuation outcomes, we find that under fairshare pricing, buyers achieve higher model performance per dollar spent, making it particularly beneficial for those with limited budgets. In addition, our simulations of long-term market dynamics demonstrate that fairshare pricing encourages sustained seller participation, resulting in a stable and sufficient supply of training data over time compared to pricing not only
exploitative pricing. These findings show that our framework's 's iittaavaluation-baa improves short-term training efficiency, but also ensures the long-term viability of the data market.
Finally, to evaluate the robustness and method-agnoste applicability of our framework, conduct an ablation study using a diverse set of data valuation methods (including BM25 [30], Inflp (22], and Datainf[21) our fairshare pricing framework consistently delivers mutually beneficial results for both buyers and sellers in the LLM data market, confirming that its performance is not tied any specific data valuation technique. This analysis underscores key strength of our approach: buyers and sellers can flexibly choose valuation methods tailored to their downstream needs without compromising the preetive-alignee structure of the market, and demonstrates the broad applicability of our solution.
selected for their scalability and efficiency in LLM tasks. Across all variants,
To summarize contributions are follows:
1. We present a novel theoretical framework to model the LLM training data market utilizing data valuation, economics of market, and game theory.
2. We propose fairshare pricing mechanism that captures the strategic interplay between buyers and sellers. Both theory and empirical analyses show that exploitative pricing results in lose-lose outcomes, while fairshare promotes long-term stability and mutual benefits.
3. Our proposed pricing framework is highly generalizable across LLM models and tasks, and robust to the data valuation methods and utility measures used by market participants.
Our results demonstrate the versatility of our approach in data markets by aligning incentives, sustaining data quality, and maximizing long-term value. This highlights our fairshare mechanism as guide for designing sustainable data procurement practices in public and private sectors.
CBS, 60 Minutes
2

---

2
Related Work
Data Market Design and Exploitative Pricing. Recent studies on ML data markets focus on platform-based pricing and allocation mechanisms such as auctions and personalized pricing 33 These frameworks typically aim to maximize profit or efficiency, assuming myopic buyers and fixed seller participation. They also consider information transparency ensures that sellers have access to the same data valuation scores used by buyers [36] 33). The study was mainly conducted on classic machine learning models rather than large language models.
Concurrently, research has highlighted ethical concerns in LLM data acquisition, particularly around exploitative pricing. Annotation work central to LLM training - is often outsourced to low-wage workers with minimal labor protections (9 10 37 38), who are frequently exposed harmful content 12 [9]. Several studies document low pay, precarious labor, and lack of protections in global data supply chains [6H18]. Although these studies highlight systemic exploitation in data labor, their findings have yet to be integrated into formal models of LLM data pricing, Economic theory suggests that misaligned compensation undermines efficiency. discourages participation, and lowers data quality. In LLM contexts, this reduces data availability and degrades model performance 40 underscoring the need for value-aligned pricing mechanisms.
Game- Theoretic Models of Pricing and Participation. In many domains, pricing problems are modeled thorugh game theory. Particularly, those involving decision-making are frequently modeled using Stackelberg games and bilevel optimization [3]. Although these frameworks have not yet been applied to LLM data market, they have been extensively used in contexts such as resource allocations [4]. energy market 45 46). supply chains [7). distributed systems [48]. In these settings, agents typically optimize revenue, efficiency, or utility while anticipating the strategic responses of others. In dynamic settings, these models extend to repeated interactions, where players balance immediate gains against long-term benefits [4] 43). In repeated interactions, fairness perceptions are critical: studies show that perceived pricing unfairness can erode trust and reduce participation 53).3
Data Valuation Methods: Data valuation estimates the contribution of training examples model performance [54]. Examples include fiftuence-functoon--
methods (55). which estimate data utility via model gradient computations. Variations such of this method enhance efficiency by approximatiig//yyassinng the inverse-Hessiad [56), or utilizing lower dimension model gradients (e.g., Datainf (21)). which balance efficiency and accuracy, and make them suitable for the LLM realm. Beyond the influence of loss functions, previous research (57 has examined data valuation with respect to fractional utility functions.
Data valuation approaches are effective for LLM training data selection
23
58
and other
applications including toxicity detection (59] memorization analysis [60]. training optimization
[61] and and active sampling (62]. Shapley-based method is another direction 63
65
36 such as
CS CS-Shaply(66) In-run Shapley (67. DU-Shapley[(SS [68) etc. Simpler methods like BM25 30 offer
a model-agnostic baseline based on lexical similarity [69] 770.. While exact data value estimation
computationally costly at LLM scale, recent studies show that influence-based approximations correlate meaningfully with actual performance outcomes [71] (2).. For example, Jiao et al. [7l]
shows that influence-based methods have high correlation between their estimated data valuation scores and the oracle value, which is obtained through model re- re-training.
3 A Theoretical Framework of LLM Data Market
This section formalizes the LLM data market. We model buyer buyer-seller interactions using economic utility theory and game theory, show that exploitative pricing leads to long-term market failure, and introduce the fairshare pricing strategy.
3.1 Data Market Definition
We formalize the LLM data market by modeling the ceciision-making processes of sellers and buyers in non-cooperative Stackelberg game [35 45 48) In this setup, sellers first set prices by anticipating buyer demand. Buyers then respond by selecting datasets to maximize their utility ? Each player
Unility is detined as general economic concept that measures an individual's welfare, benefit, or satisfaction 24 25 for example, monetary value, model fairness, other domain-specifte benefits.
3

---

LLM builders (Buyers)
Price set based on Gamo theory and Date valuation
Data annotators (sellors)
Annotated Cattaatts
cen
arrotate
Modal
fi
Sel.
FL
picsp:
D
Usity
mathoul
Dwn
B&
anscitate
Figure I: Illustration of the LLM data market, showing buyers (LLM builders) and sellers (annotators) interacting via pricing mechanism based on game theory and data valuation.
makes decisions based solely the current state of the market, without considering future updates interactions.
Notably, our choice of a conperaree Stackelberg game fits the LLM data market's reality: many independent, reff-interested transactions where binding grand coalitions are infeasible, such as gig-work platforms [18]. In contrast, cooperative bargaining targets few-party, contractible surplus splits, which is a different regime and question. this non-cooperative game, we model fairness behaviorally via seller participation/exit that keeps the model tractable, and the leader-follower timing mirrors real price setting and demand response.
Following prior work 773. we assume information transparency: all participants observe each dataset's value. Practically, this can be estimated using data valuation methods that quantify its contribution to LLM performance. These valuation scores potentially generated by the platform or a trusted third party serve as common signals informing both pricing and purchasing decisions.
Next, we define key factors determining each player's decision makiin... In practice, this can be estimated using data valuation methods that quantify its contribution L.M performance. These scores potentially generated by the platform or a trusted third party serve as common signals guiding both pricing and purchasing decisions.
In the rest of this section, we formally define key factors determining each player's decision becksion-making.
Data Market Setup. We begin by defining the players and the notion of utility. The market comprises set of buyers { (Bx J 1 each with an LLM M&, and a of sellers { each with a dataset D,. Each buyer gains a certain level of economic utility from acquiring new dataset based on data value, which is captured by data valuation function, 2k D - IR R2,, measuring the marginal contribution of each dataset D to the performance of buyer B 's LLM M,,
The value Up can be estimated using data valuation method (e.g., Influence Function 55 DataInf 21), etc.). Once the data valuation score U/2 is estimated, the corresponding utility gain ug can be derived function of Uk. We use task-specific mapping between data value eù and utility uk Uk + Rgo- Appendix CCinnisssCCssssss common mappings between ug and U& in downstream tasks.
Buyer's Decision-Making Each buyer By selects subset of datasets to acquire, maximizing the net utility, defined as the utility gain minus total acquisition cost.
B₁ 's decision, represented by binary vector æ (where each entry is 1 if the corresponding dataset is selected, and 0 otherwise), depends on three components: (i) the current dataset prices, (ii) buyer's budget by. and (iii) the utility gain ux(2x) from acquiring the selected datasets.
This utility gain reflects the downstream value, resulting from performance improvements in the buyer's ' model after training on the dataset bundle x The net utility is defined as:
Jk.N (x:) == (x:) x² TTp. p.
where p= [pi,.... "pi,-..,PN' PN]] is the price vector for current avaliable datasets.
Datasets may exhibit dependencies. The set utility ux (æ) is necessarily additive over individuals.

---

Finally, By's purchasing problem is formulated as selecting an optimal collection of datasets to maximize its net utility:
+ N.NN
arg max Ik.N (x), s.t. ME,N == for Ik,N 2 O, x² s bk).
(2)
NEAL,N
where KK.N is the optimal solution, and NN,N includes all feasible solutions, x²" S by ensures that the total acquisition cost within the budget bj, and 9k.N (a:) ensures a non-negative net utility.
Seller's Decision-Making. When the seller Sj offers dataset D,, it sets a price to maximize its net profit, defined (i) the anticipated sales from all buyers, minus (ii) a fixed cost to create the dataset. Anticipated sales from buyers are estimated by solving the previous buyer's purchasing problem (Equation (2) known in the full transparent market. Formally, the seller's net profit function is:
T(p)) = K.N (p)p; Cj.
N where at N.V. (p) is j-th entry of buyer Bg decision vector, indicating B& purchases Dj (j-th entry is 1) or not (j-th entry is 0), given the prices of all datasets in the market; cy denotes the cost. The seller Sj solves the following pricing problem:
Pj
== arg max r(p;), s.t. P=M E R+ r(p)) 0},
P
where Pj is the optimal price and r(p;) 2 O ensures that seller Sj 's net profit must be non-negative. is noted that we assume that selling data for annotator is profitable, i.c., Pj 2 Cj-
In the above formulation, buyer purchase decisions depend on prices and data value; seller pricing decisions depend on anticipated sales from buyers. Together, these decisions define the equilibrium dynamics of transparent and incentive-alignat LLM data market. This formulation can be extended royalty-basee -ased scheme as presented Appendix
3.2 Exploitative Pricing: A Lose-Lose Outcome
We now present a theoretical analysis showing that exploitative pricing in data markets is ultimately unsustainable and detrimental to all participants.
Theoretical Setup. We analyze simplified dynamic setting involving single buyer and a single seller over infinite time horizon. At each time step t, both players decide whether to transact based on the proposed price. Importantly, to align with real-world market behavior, the seller's 's future participation depends the history of past transaction prices.
We first introduce some assumptions on the seller behavior.
Assumption 1 (Declining Participation Probability). When offered price pt below the seller's ideal price p the probability of seller's continued participation declines. This decline is captured by strictly increasing function
π (p:, p.) j0, 1], satisfying p.) =0 and क (P),pi) =
In addition, the probability that the seller S remains active at time T is Pr : IIF-01 ग P().
Assumption D models how the seller respond exploitative pricing over time. states that when offered price below its ideal level, the seller become less likely to stay in the market, consistent with prior studies 74 75.. For any exploitative pricing, the participation probability Pr declines multiplicatively, leading to sustained and irreversible drop in engagement.
To capture the degree of participation decline, assume sellers respond sensitively to underpayment:
Assumption 1.1 (Sensitivity of Participation Is Lower-Bounded), The Lipschitz continuity of it over exploitative pricing is dower-bounded
(p(L.,,P)) 2Pr... L |Pc,1 Pr.2l
(6)
for some constant L 0 and all exploitative pricing Pt.1, Pt,2 [0, p() for all t.
5

---

Buyat Cummllaiive Utfiity
Idml Explottative
Seller's Purticipation Pro5
4.0
pe Beeet 20N7
4.0
San Royar
1.5
08-
Beecc 3007
Repach
1.2
Profit
Pro r
Cty prinps
0.6
2.0
2.0
0.9
Explottative
0.6
0.-
0.3
0.0
0.0
0.0
4
8
10
10
2.0
4.0
2.0
4.0
Time Step
Time Step
Price P1
Price P2
(a) Buyer's utility.
(b) Seller's 's participation.
(c) Seller Si's profit.
(d) Seller S2 profit.
Figure 2: Buyer's cumulative utility and seller participation under ideal and exploitative pricing
2a 2b Profits of sellers S1 and S2 with buyer B1 s and Bz's MWP over price (2c 2d
We also introduce the following assumptions on buyers:
Assumption (Discount Factor Is Lower-Bool Boner-Bounddd. The buyer evaluates long-term utility using a discount factor 8, satisfying:
8 min Blue
Vt.
př]
A higher discount factor ó indicates greater emphasis on future utility. Assumption Plaaces a lower bound on ensuring that the buyer sufficiently values future gains when making decisions.
Assumption 2 assumes the buyer values both immediate and future utility gains from acquiring training data. Major LLM developers invest in large-scale data acquisition and model training in expectation of long-term gains in performance, deployment value, and commercial returns 776 79).
Buyer's Objective. With the previous setup, the buyer aims to maximize expected cumulative utility over an infinite horizon. Let G(u,, bt) denote this value function, representing the buyer's expected total cumulative utility at time I, conditioned on current utility u, and budget by Then this objective satisfies the following Bellman equation[80 [80]:
G(u,, be)
max
[ [w: pe] SE (T(P:.))) G( (,44)) 1) we, be]]
(8)
P.E (0.(0.
This Bellman equation captures the buyer's central trade-off: offering an exploitative price pt increases immediate surplus E[w,- - but reduces future seller participation via a lower (Pt,Pī); conversely, setting a fairer price decreases short-term gain but sustains future transactions by increasing (Pt. p.). With this insight, we then obtain the following result:
Lemma 1 (Inevitable Failure of Exploitative Pricing). With Assumptions 1 2 any exploitative pricing (i.e., Pt < Pt, Vt) will only maximize cumulative utility within a finite horizon after which is strictly suboptimal.
Lemma I reveals fundamental limitation: any exploitative pricing strategy is only optimal for finite time. Over time, it becomes strictly suboptimal, due to the declining seller participation. Thus, no exploitative strategy can maximize cumulative utility in the long run. (See Appendix for the proof, shows that exploitative pricing yields a suboptimal value function.)
Exploitative Pricing Leads to I Lose-lose: While exploitative pricing clearly reduces seller welfare, our results show it also harms long-term cumulative utility for buyers. Although buyers may benefit initially from lower costs, reduced seller participation quickly leads to decline in data quality and availability, Over time, even well-resoureed buyers face data shortages. This sets off a self-defcating cycle where short-term savings come at the expense of long-term model performance.
In addition to our theoretical findings, we run simplified dynamic market simulation with one buyer and one seller, making sequential decisions. The utility of the dataset and buyer's budget varies randomly over time, and seller participation follows T Tpt, ) = Pt/pt p Figures 2a and 2b 2b provides consistent evidence that exploitative pricing causes rapid seller exit and an immediate shortage of
training data. This dynamic closely resembles the classic "market for lemons" problem, where underpricing drives out high-quality supply, ultimately leading to market collapse (8).

---

Cammlarive Net (hille
Cumulative Net Untilty
8.0
8.0
Cumulative Net Unitty
Cumulative Net
Scller's optmal peke P
4.0
4.0
4.0
4.0-
0.0
0.0
0.0
0.0
0.0
0.5
1.0
1.3
2.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.3
2.0
Price p
Price P
p
Price P
(a) T=
(b) 2
(c) 5
(d) T= 10
Figure 3: Analysis of the buyer's cumulative net utility as function of the acquisition prices over
time (T' 1,2, 5, 10). Note: the market has one buyer and seller.
3.3 Fairshare Pricing: A Win-Win for Sellers and Buyers
The rest of this section presents the fairshare pricing in our LLM data market framework and show it yields a stable, mutually optimal outcome for both buyers and sellers
Pricing Based on Buyers' Maximum Willingness to Pay. When the seller Sj prices its dataset D,, evaluates how each buyer's decision changes depending on whether Dj is available at given price. For each buyer Bg, the seller compares two scenarios: (1) the buyer's optimal dataset selection when Dj not part of the market, and (2) the buyer's s new optimal selection if Dj is included at price Py. The buyer will purchase Dj only if the dataset D, increases its net utility and remains within budget. Formally, let *.N be buyer Bg 's optimal decision without the presence of Dj- For any feasible decision a E N we define:
Seller-Side:
where ej (2 - 1. Marginal utility from adding Dy: (u)( := 9k.N + ej) Gk. N1 is the unit vector indicating D, is selected and pj is set as zero, and
2. Budgel surplus based on the prior decision: Alr (N1) by k. (2
buyer's maximum willingness to pay (MWP) is defined as the highest price buyer Bg is willing to pay based on marginal utility and able pay based on budget surplus:
MWP
max (min(Aux(a)) (bu ( }}} N-1
KNN
where (=x(=)) (Aug(a)) (z), 0} denotes the positive part of marginal utility, ensuring buyer B₁ 's MWP; is non-negative if the marginal utility of Dj is negative, the buyer will not purchase it.
Then the seller's optimal price p) is the MWP that results the largest profit across buyers:
Lemma 2 (Characterization of Optimal Price p3). Seller Sy optimal price for Dj is characterized as one of buyers' MWP (see Appendix D for proof):
p E M MWP
(10)
We see that optimal price p is fairshare for the seller: maximizes seller's profit while aligning with dataset's utility and buyer's budget.
As shown in Figures 2c and 2d] we simulate a data market with sellers and 2 buyers, where players make one-shot decisions. Seller S sets its price first, followed by seller Sz. Each plot shows each seller's 's profit function rpp)). ), with brcakpoints at each buyer's 's MWP. Pricing above a buyer's 's MWP leads to a drop in sales. Thus, the seller's optimal price aligns with the MWP that yields maximum revenue.
Buyer-Side: Fairshare Is Overall Optimal. We now show that the fairshare price p", derived from seller optimization, also optimal from the buyer's 's perspective. As established Section [3.2] ex- ploitative pricing leads to persistent seller exit. In contrast, fairshare pricing ensures full participation and maximizing the buyer's overall utility. To formalize this, we consider the same infinite-horizon setting in Section 3.2 with single buyer and seller. Notably, the fairshare price is the ideal price for sellers as it maximizes profit. Therefore, under fairshare pricing, the sellers will maintain sustained participation. Then, obtain:
7

---

Commulariee Uiblittes
Conulativve Profit
10-
240
240
75
160
Cammularive
160
Sellers
25
0
0-
0-
40
100
40
100
60 100
20 60 100
Time Steps
Time Steps
Time Steps
Time Steps
(a) High-budget buyer.
(b) Low-budget buyer.
(c) Sellers' profits.
(d) Sellers' participation.
Figure 4: (1) buyer's cumulative utilities with high- (Figure 4aj and low-budget buyer (Figure 4b). and (2) sellers' average cumulative profits (Figure 4c) and active seller numbers (Figure 4d over 100
time periods. Pythia- MedQA; Groups: (1) fairshare, (2) reduced, (3) random, and (4) exploitative.
Lemma (The Optimal Price for Buyer Is Also p.). The fairshare price for the seller S under LLM p is the optimal pricing strategy
data framework is pテ = min (u, b Vt. With Assumptions I to 2
that maximizes buyer's expected cumulative utility (Equation
In Figure 3 we develop a synthetic simulation illustrating that a buyer's optimal pricing strategy rapidly converges the fairshare price. Consider simulation with players receive constant utility 2 2 and budget b, 1, Vt, with the discount factor 0.95, so the seller's optimal price remains P L. At each time step. the buyer selects a price that maximizes cumulative net utility. The buyer begins with low, exploitative price at T 1, but by T = 5, converges to the fairshare price and maintains thereafter. This illustrates our core theoretical insight: fair pricing emerges as the optimal long-run strategy when buyers account for overall market sustainability.
Next, we also explore the role of the discount factor ō in our framework:
Lemma 4 (The Trade- Off Threshold Is Increasing as ē Decreases). The threshold time period where the fairshare pricing obtains higher cumulative utility than any class of exploitative pricing is:
sup Ps VE
T Σ
(ur pt)
II T p.) (4+ p.)
z0).
(1I)
1=
2=
And 1" is increasing as 5 decreases.
We run detailed robustness check with different values of for our experiments Appendix E.2
Empirical Analyses: Benefits of Fairshare Pricing
This section evaluates the LLM data market and the proposed fairshare pricing framework.
4.1 Experimental Setup
In our experiments, buyer is equipped with single LLM, and each seller owns one data sample. Buyers seek buy training data to improves task-specific model performance (e.g..math problem solving, medical diagnosis, or physical reasoning), which in turn increases their utility. (We assume an affine mapping between performance and utility; see Appendix C.I)
Buyers and Models. We consider three buyers, each using a standard open-source LLMs: Llama- 2-Instruct-11 lb [81, Pythia-Ib. lb, and Pythia-410m [82.. These models are pre- mre-trained different corpora and exhibit varying preferences for downstream post-training data 83
Sellers and Datasets. We focus on challenging, human-annotatee tasks: MathQA and GSM8K 27 84 for math, MedQA (28 for medical diagnosis, and PIQA (29 for physical reasoning (85)87). Table I in Appendix [ shows dataset splits and examples. We use the training splits as seller data and simulate the market dynamics from Section 3 treating each task as a market scenario.
4.2
LLM Data Market Experiments
We first evaluate pricing framework in terms of buyer and seller welfare.

---

Market Design: Following the general setups in Section 4.1 and 10 sellers over multiple time steps. To examine buyers with varying resources, we include a high-budget buyer (well-funded LLM builder) and a low-budget buyer (under-resourced one). Each buyer's budget is randomly drawn from distributions with different mean. At each time step, (1) sellers arrive sequentially with new dataset 300 data samples) at a fixed price, and then (2) once all sellers arrive, buyers make purchases by solving Equation (2). Full details are in Appendix E.2
we simulate a market with 2 buyers
Participation Function: Following Assumptions too 2 we simulate 100 time steps with a discount factor 5 = 0.98. We also run experiments with different value of 8 showing consistent and robust results. (See Appendix E). Seller participation probability is defined as aP Pa))) = P,,/P5, for its simplicity and compliance with Assumptions I and I.I Sellers receiving exploitative pricing (
Pjá < P)),)) are less likely to participate.
Pricing Methods: We consider four pricing methods: (1) Fairshare - Pj.a = Pj. (our fairshare
pricing framework); (2) Reduced - a fixed discount of fairshare price, Pj,+ = c* Pi, with e = 0.5; (3)
Random - random drawn from p),) (4) Exploitative fixed low price (10% of the avg. utility).
Figure 4 compares the overall welfare outcomes of different pricing methods for buyers and sellers using Pythia- lb on the MedQA task. Results on MathQA and PIQA (with Pythia- 110ma and Llama- 3.2- 12-Instruct-Ie 1b) in Appendix F show similar patters..
Exploitative Pricing Leads Lose-Lose Out- comes: The exploitative pricing method, which sets uniformly low, fixed prices to reflect real- world practices (III,, offers LLM developers short-term utility gains (see Figures 4a and (4b). However, this approach systematically under- values datasets and unfairly compensates an- notators. As result, sellers exit the market over time, triggering a collapse in data supply. Even well-funded LLM developers are unable to source sufficient data hindering the advance- ment of LLMs.
PurPeed Daluvets
10-
of Purchnsed Datnsets
10-
Fandan
Rodeed
Faindure Redixed [uenienn
Rasieg
Explaiiaive
5
5
Explekotive
20
40
60
80 100
20 40 60 80 100
Time Steps
Time Steps
(a) High-budget buyer.
(b) Low-budget buyer.
Figure 5: Purchased datasets for the buyer with high budget (Figure and low budget (Figure (5b) over 100 time periods. Pythia- 1b; MedQA.
Fairshare Pricing Leads to Win-Win Out-
comes: For sellers, fairshare pricing consis- tently yields the highest profit over time (Fig- ure 4c). aligning with predictions from our the- oretical model (Section 3.T). For fairshaer pricing framework particularly effective for the high-budget buyer (Figure 4aj, maximizing long-ten term utility. However, the low-budget buyer (Figure 4b) experiences reduced short-term utility in exchange for long-term gains. Its limited budget prevents them from fully leveraging the increased dataset supply ensured by fairshare pricing, making other low low-pricing methods initially more appealing. Yet, in the long fuinsshaaar pricing sustains seller participation, ensuring data supply.
fairshare
4.3 Ablation Study: Effect of Different Data Valuation Methods
This experiment assesses the impact of different data valuation strategies in the fairshare framework.
Setup. Using the models and datasets introduced earlier Section 4.1 we run separate simulations for each market (math, medical, physical reasoning), testing different data valuation methods for the buyer. Each buyer is randomly assigned to use one of four valuation methods to select training data and fine-tune their model. The seller receives payments according to the data's 'ass assigned value. Valuation scores are normalized to [o, 1] for pricing compatibility.
We consider the following methods, where each assigns a value to every data sample: (1) Constant baseline assigns the same value, mimicking flat-rate pricing on platforms; (2) Random baseline- - values drawn uniformly from [0, 1]; (3) Semantic - uses BM25 [30] to compute average similarity to the representative set; (4) refluence-based learning gradients to estimate data sample's avg. contribution to model learning of a representative dataset. Specifically, we use Infl [56] 22 and DataInf (2I. which are influence- methods
Data Valuation Methods.
returns score which leverages
adapted for the LLM realm. See Appendix A for additional details.
9

---

Acc.=34.2
BP ncc.
BP Acc,=74,4
Price per Sample
0.544
40
31.3
76.25
0.328
Accuracy
30
Accuracy
31.0
Accuracy
76.00
30.5
75.75
36
30.0
0550
75.50
34
29.5
Aeston
75.25
1K
1.5K 2K
2.5K
0.5K
1K
1.5K
2K
1K
1.5K
2K
2.5K
Method
Price
Price
Price
(a) Avg Price (Llama)
(b) MathQA (Llama)
(c) MedQA (Llama)
(d) PIQA (Llama)
Figure 6: Left column: Average price-per-sample cost purchased data across math, medical, and physical reasoning data markets using different data valuation methods. Right, middle-colume
Buyers' model performance versus cost, before purchasing (BP) data, and after purchasing 2K and 4K data samples. Additional analysis on the Mithia-1//100 models are in Appendix F
Furthermore, we adopt a one-step training approach for data valuation, where the value of each data sample is estimated by performing single training step it and measuring the resulting change in model performance relative to the original model. Results are presented in in Appendix [-2 This approach serves value through model training, although it is more computationally expensive than the previous listed methods.
"oracle" for influence-basee methods [6),, it directly quantifies data
We reserve 1% the samples from each dataset's training split to represent the existing data in their respective markets. Each data sample was randomly priced between (0, 1]. Next, for each remaining data sample in the training set, we determine whether each buyer will purchase the data sample potential price points [0.5, 0.625, 0.75, 0.875, 1.0] by solving Equation (2). The seller then sets their prices according to Equation (4). We price data separately for each data valuation method. This assesses the method's ability to discern whether new data sample is worth purchasing for each buyer given the existing market data, as noted in our analysis in Section Further details on all experiments are described in Appendix I
Market/Pricnng Setup:
results across data valuation methods. Buyers using valuation method (ie., BM25, Inflpp Datalnf) in general achieved higher model performance across tasks. When considering the trade-off between cost and performance, Inflpp offered the best balance, delivering strong model improvements at lower cost than constant, random, and BM25 (Figures 6a and 9ap. Our results highlight the benefits of learning-awer aware data valuation methods. By prioritizing high-impact data, they offer better alternative for buyers, particularly those with limited budgets.
Results. F
We also run the error analysis of data valuation methods. Following previous studies 71 we compute the correlation between Oracle and Inflyr reporting spearman correlation of 0.54, 0.42 for MathQA and PIQA (see Figure advances in data valuation accuracy and scalability, which can be seamlessly integrated into our flexible fairshare framework.
X in Appendix(F) This underscores the opportunity for future
Conclusion
We introduced a fairshare pricing framework that leverages data valuation methods to enable transparent and equitable training data pricing for LLMs. Our results show that buyers achieve higher gains at lower cost. At the same time, sellers earn optimal prices for their data, fostering a win-win outcome that enhances long-term market sustainability and social welfare. To knowledge. this is the first work to integrate theerethc pricing models with LLM-specific data valuation techniques, addressing the real- world dynamics of the emerging data market. Our framework offers actionable insights for policymakers and regulators aiming to ensure fairness and transparency in LLM training data markets. By fostering fair market access, our framework also empowers small businesses and startups, leading to more equitable technological advancements. Future work could extend our framework to incorporate additional data valuation methods, incomplete information settings (e.g., Bayesian games), and diverse data domains (e.g., pretraining vs. fine-tuning). We hope this work paves a fruitful way for future research in equitable markets for Al and emerging technologies.
10

---

References
[1] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss. William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei. Sam McCandlish, Ilya Sutskever, and Wojeiech Zaremba. Evaluating large language models trained on code, 2021.
[2] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Lc, Denny Zhou, et al. Chain-of-than of-thuught prompting elicits reasoning in large language models. Advances in neural information processing systems, 15:24422--44311 2022.
[3] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint Stfi:22211.900S 2022.
[4] Reuters. Big tech companies in race to buy training data for artificial intelligence. Business Standard, April 2024.
[5] Katie Paul and Anna Tong. Inside big tech's underground race buy ai training data. Reuters, April 5 2024.
[6] Jiayao Zhang, Yuran Bi, Mengye Cheng, Jinfei Liu, Kui Ren, Qiheng Sun, Yihang Wu, Yang Cao, Raul Castro Fernandez, Haifeng Xu, Ruoxi Jia, Yongchan Kwon, Jian Pei, Jiachen Wang, Haocheng Xia, Li Xiong, Xiaohui Yu, and James Zou. A survey on data markets, 2024.
[7] Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. Algorithmic impact assessments and accountability: The co-construction of impacts, In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 735 735-746,22 2021.
[8] George A Akerlof. The market for "lemons"; Quality uncertainty and the market mechanism. In Uncertainty in economics, pages 235-251, Elsevier, 1978.
[9] Winter Mason and Duncan J. Watts. Financial incentives and the "performance of crowds". In Association for Computing Machinery, HCOMP '09, page 77-85, New York, NY, USA, 2009.
[10] Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jef- frey P. Bigham. A data-driven analysis of workers' earnings on amazon mechanical turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 18, page 1-14, New York, NY, USA, 2018. Association for Computing Machinery.
[11] CBS News. Labelers Training AI Say They're Overworked, Underpaid, and Exploited: 60 Minutes Transcript, November 2024. Accessed: 2024-11-28.
[12] Time Staff. Openai used kenyan workers on less than $2 per hour to make chatgpt less toxic. https::// :ttppss://mmmm..//ee com/ 2025-04-18.
chatgpt cettippt--eny--eee kenya-workers 2023. Accessed:
[13] Eun Seo Jo and Timnit Gebru. Lessons from archives: Strategies for collecting sociocultural data in machine learning. Proceedings of the 2020 Conference on Fairness, Accountability and Transparency (FAT* 2020), pages 306-316, 2020.
[14] Abeba Birhane, Vinay Uday Prabhu, Emmanuel Kahembwe, Olivia Guest, and Kirstie Whitaker Jamieson. The impossibility of automating ambiguity. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT), pages 369-381. ACM, 2021.

---

[15] Lynsey Chutel. African scientists are being sidelined by "parachute" research teams. Quartz Africa, 2019.
[16] Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, Emily Denton, and Alex Hanna. Data and its (dis)contents: A survey of dataset development and use in machine learning research. Patterns, 20:1000,,,,,, 2021.
[17] Nithya Sambasivan, Anirudh Kasyap, David Thomas, Mrinmaya Natarajan, Nithin Balu, and Samir Passi Ahuja. "everyone wants do the model work, not the data work": Data cascades in high-stakes ai. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CIII), pages 1-15, ACM, 2021.
[18] Mary L Gray and Siddharth Suri. Ghost work: How to stop Silicon Valley from building a new global underclass. Harper Business, 2019.
[19] Michael Spence. Job market signaling, In Uncertainty in economics, pages 281-306. Elsevier, 1978.
[20] Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. Advances in Neural Information Processing Systems, 33: 13:19920-19930, 2020.
[21] Yongchan Kwon, Eric Wu, Kevin Wu, and James Zou. Datainf: Efficiently estimating data influence in lora-tuned Ilms and diffusion models. arXiv preprint arXiv:2310.0090,, 2023.
[22] Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning, 2024.
[23] Zichun Yu, Spandan Das, and Chenyan Xiong. Mates: Model-aware data selection for efficient pretraining with data influence models, 2024.
[24] Andreu Mas-Colell, Michael Dennis Whinston, Jerry R Green, et al. Microeconomic theory, volume 1. Oxford university press New York, 1995.
[25] John Von Neumann and Oskar Morgenstem. Theory of games and economic behavior: 60th anniversary commemorative edition. In Theory of games and economic behavior. Princeton university press, 2007.
[26] Mary A Konovsky. Understanding procedural justice and its impact on business organizations. Journal of management, 26(3):489-511, 2000.
[27] Aida Amini, Saadia Gabriel, Peter Lin, Rik Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation- based formalisms, 2019.
Yejin Choi, and Hannaneh
[28] Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021.
[29] Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about phys- ical commonsense in natural language. In Proceedings of the AAAI conference artificial intelligence, volume 34, pages 7432-7439, 2020.
[30] Andrew Trotman, Antti Puurula, and Blake Burgess. Improvements to bm25 and language models examined. In Proceedings of the 19th Australasian Document Computing Symposium, pages 58-65, 2014.
[31] Anish Agarwal, Munther Dahleh, and Tuhin Sarkar. A marketplace for data: An algorithmic solution. In Proceedings of the 2019 ACM Conference on Economics and Computation, pages 701-726, 2019.
[32] Lingjiao Chen, Paraschos Koutris, and Arun Kumar. Towards model-based pricing for ma- chine learning in a data marketplace. In Proceedings of the 2019 international conference on management of data, pages 1535-1552, 2019.
12

---

[33] Junjie Chen, Minming Li, and Haifeng Xu. Selling data to machine learner: Pricing via costly signaling. International Conference on Machine Learning, pages 5336-3359. PMLR, 2022.
[34] Kai Hao Yang. Selling consumer data for profit: Optimal market-segmmentttio design and its consequences. American Economic Review, 112(4):1364-13)) 2022.
[35] Jiayao Zhang, Yuran Bi, Mengye Cheng, Jinfei Liu, Kui Ren, Qiheng Sun, Yihang Wu, Yang Cao, Raul Castro Fernandez, Haifeng Xu, et al. A survey on data markets. arXiv preprint arXiv:2411.07267 2024.
[36] Amirata Ghorbani, Michael Kim, and James Zou. A distributional framework for data valuation. In International Conference Machine Learning, pages 3535-3544. PMLR, 2020.
[37] Carlos Toxtli, Siddharth Suri, and Saiph Savage. Quantifying the invisible labor in crowd work. Proceedings of the ACM on human-computer interaction, 5(CSCW2):1-26 2021.
[38] Drew Harwell and Regine Cabato. Ai boom relies on underpaid workers in the philippines. The Washington Post, 2023. Accessed: 2025-04-20.
[39] Laura Weidinger, John Mellor, Mariheth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean Legassick, Geoffrey Irving, and lason Gabriel. Ethical and social risks of harm from language models, 2021.
[40] TechCrunch. AI Training Data Has a Price Tag That Only Big Tech Can Afford, June 2024. Accessed: 2024-11-28.
[41] Deepa Seetharaman. The next great leap in ai is behind schedule and crazy expensive. The Wall Street Journal, December 2024.
[42] Ayesha Javed. Why amazon web services ceo matt garman is playing the long game on ai. TIME, February 2025.
[43] James A Brander and Barbara Spencer. Export subsidies and international market share rivalry. Journal of international Economics, 18(1-2):83-100, 1985.
[44] Jonathan F Bard. Practical bilevel optimization: algorithms and applications, volume 30. Springer Science Business Media, 2013.
[45] Heinrich Von Stackelberg. Market structure and equilibrium. Springer Science & Business Media, 2010.
[46] Jian Pei. A survey data pricing: from economics to data science. IEEE Transactions knowledge and Data Engineering. 34(10):4586 4608, 2020.
[47] Maryam Esmaeili, approach in seller-buyer supply chain. European Journal of Operational Research, 195(2):42- 448, 2009.
Aryanezhad, and Panlop Zeephongsekul. A game theory
[48] Sabita Maharjan, Quanyan Zhu, Yan Zhang, Stein Gjessing, and Tamer Basar. Dependable de- mand response management in the smart grid: A stackelberg game approach. IEEE Transactions on Smart Grid,
2013.
[49] Nancy L Stokey and Robert E Lucas Jr. Recursive methods in economic dynamics. Harvard University Press. 1989.
[50] Robert G Folger, Robert Folger, and Russell Cropanzano. Organizational justice and human resource management, volume 7. Sage, 1998.
[51] Robert Folger and Russell Cropanzano. Fairness theory: Justice as accountability. Advances in organizational justice, 1(1-55):12, 2001.
13

---

[52] Jason A Colquitt, Jerald Greenberg, and Cindy P Zapata-Phelas Phela.. What is organizational justice? a historical overview. In Handbook of organizational justice, pages 3-56. Psychology Press, 2013.
[53] Mladen Adamovic. Organizational justice research: A review, synthesis, and research agenda. European Management Review, 20(4):762-78) 2023.
[54] Zayd Hammoudeh and Daniel Lowd. Training data influence analysis and estimation: A survey, Machine Learning, 113(5):3351-240) 2024.
[55] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions.
In International conference on machine learning, pages 1885-1894. PMLR, 2017.
[56] Garima Pruthi, Frederick Liu, Mukund Sundararajan, and Satyen Kale. Estimating training data influence by tracing gradient descent, 2020.
[57] Jialu Wang, Xin Eric Wang, and Yang Liu. Understanding instance-level impact of fairness constraints. In International Conference on Machine Learning, pages 23114-23130. PMLR. 2022.
[58] Cathy Jiao, Yijun Pan, Emily Xiao, Daisy Sheng, Niket Jain, Hanzhang Zhao, Ishita Dasgupta, Jiaqi W. Ma, and Chenyan Xiong. Date-Im: Benchmarking data attribution evaluation for large language models, 2025.
[59] Masaru Isonuma and Ivan Titov. Unlearning traces the influential training data of language models. arXiv preprint arXiv:2401.1244 2024.
[60] Michael Kounavis, Ousmane Dia, and Ilqar Ramazanli. On influence functions, classification influence, relative influence, memorization and generalization. arXiv preprint arXiv:2305. 16094, 2023.
[61] Van, Alycia N Carey, and Xintao Wu. Hint: Healthy influential-nois based training to defend against data poisoning attacks. In 2023 IEEE International Conference on Data Mining (ICDM), pages 617- IEEE, 2023.
[62] Jinlong Pang, Jialu Wang, Zhaowei Zhu, Yuanshun Yao, Chen Qian, and Yang Liu. Fairness without harm: An ceuueenne-guide active sampling approach. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[63] Lloyd S Shapley. A value for n-person games. Contribution to the Theory of Games, 2. 1953.
[64] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1167-1176. PMLR, 2019.
[65] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, pages 2242-2251. PMLR, 2019.
[66] Stephanie Schoch, Haifeng Xu, and Yangfeng Ji. Cs-shapley: class-wise shapley values for data valuation in classification. Advances in Neural Information Processing Systems, 35:34574- 34585, 2022.
[67] Jiachen T Wang, Prateek Mittal, Dawn Song, and Ruoxi Jia. Data shapley in one training run. arXiv preprint arXiv:2406.11011 2024.
[68] Felipe Garrido Lucero, Benjamin Heymann, Maxime Vono, Patrick Loiseau, and Vianney Perchet. Du-shapley: shapley value proxy for efficient dataset valuation. Advances Neural Information Processing Systems, 37:1 10000000,,000.. 2024.
[69] Ekin Akyurek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Guu. Towards tracing knowledge language models back to the training data. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2429-2446. Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.
14

---

[70] Kangxi Wu, Liang Pang, Huawei Shen, and Xueqi Cheng. Enhancing training data attribution for large language models with fitting error consideration. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14131-14143, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[71] Cathy Jiao, Gary Gao, and Chenyan Xiong. In-context probing approximates influence function for data valuation. arXiv preprint 1rXV:22407.12259, 2024.
[72] Jiachen Tianhao Wang, Tong Wu, Dawn Song, Prateek Mittal, and Ruoxi Jia. Greats: Online selection of high-quality data for Ilm training in every iteration. Advances in Neural Information Processing Systems, 177:11197--11111
2024.
[73] Steven Tadelis. Reputation and feedback systems in online platform markets. Annual review of economics, 8(1):321-340, 2016.
[74] Ian L Gale and Donald B Hausch. Bottom fiishnn and declining prices in sequential auctions. Games and Economic Behavior, 7(3):318-33) 1994.
[75] Matthew Backus, Thomas Blake, Brad Larsen, and Steven Tadelis. Sequential bargaining in the field: Evidence from millions of online bargaining interactions. The Quarterly Journal of Economics, 135(3):1319-136) 2020.
[76] Andrew R. Chow. Why amazon web services ceo matt garman is playing the long game on ai. Time, 2024.
[77] Tom Dotan, Openai's 's next big ai effort, gpt-5, behind schedule and crazy expensive. The Wall Street Journal, 2024.
[78]
Jane Li. Sundar pichai: The 100 most influential people in ai 2024. Time, 2024.
[79] Hannah Mayer, Lareina Yee, Michael Chui, and Roger Roberts. Superagency in the workplace: Empowering people to unlock ai's full potential. hattps::/www..mkkinsss cttp://iaieeccccoo/aaaaacaaaa/aa empower -ow.full--pptttiiaa--t------- 2025. McKinsey & Company.
ing peoplee trrpplle-o-unloc--aiie
[80] Richard Bellman and Robert Kalaba. Dynamic programming and statistical communication theory. Proceedings of the National Academy of Sciences, 43(8):749 1957.
[81] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and etc. The llama herd of models,
2024.
[82] Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O' O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward RafI, et al. Pythia: A suite for analyzing large language models across training and scaling, In International Conference on Machine Learning, pages 2397-2430. PMLR, 2023.
[83] Zheda Mai, Arpita Chowdhury, Ping Zhang, Cheng-Har Tu, Hong- You Chen, Vardaan Pahuja, Tanya Berger- Wolf, Song Gao, Charles Stewart, Yu Su, et al. Fine-tuning is fine, if calibrated. arXiv preprint arXiv:2409. 2024.
[84] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems, 2021. URL https://arxiv. org/abs/2110 14168, 2021.
[85] Pan Lu, Liang Qiu, Wenhao Yu, Sean Welleck, and Kai- Chang. A survey of deep learning for mathematical reasoning. In Anna Rogers. Jordan Boyd-Graben and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14605-14631 Toronto, Canada, July 2023. Association for Computational Linguistics.
[86] Wentao Liu, Hanglei Hu, Jie Zhou, Yuyang Ding, Junsong Li, Jiayi Zeng, Mengliang He, Qin Chen, Bo Jiang, Aimin Zhou, et al. Mathematical language models: survey. arXiv preprint arXiv:2312.07622. 2023.
15

---

[87] Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. In Neele Falk, Sara Papi. and Mike Zhang, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pages 225-237, St. Julian's, s, Malta, March 2024, Association for Computational Linguistics.
[88] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, International Conference Learning Representations, 2021.
[89] Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jinge Wu, Yiru Li, Sam Chen, Peilin Zhou, Junling Liu, et al. A survey of large language models in medicine: Progress, application, and challenge. arXiv preprint arXiv:231 1221311011111 1.05 2023.
[90] Liwen Sun, Abhineet Agarwal, Aaron Kornblith, Bin Yu, and Chenyan Xiong. Ed-copilot: Reduce emergency department wait time with language model diagnostic assistance, 2024.
[91] Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models, In International Conference on Learning Representations, 2021.
16

---

A Influence-based Data Valuation
In Section[4 4 we introduced gradient-based data attribution method, denoted as Inflp- In this section, we provide additional information on Inflp, which has been shown to be effective in training data selection in previous works (56] 22]. Suppose we have LLM parameterized by 0, and train set D and test set D'' For a training sample d E D, we wish to estimate its training impact on a test sample d' E D. That is, we want to measure the impact of d on the model's loss d' (i.e., L(d'; 8)). As simple method of achieving this is to take training step that is, gradient descent step on d' and obtain:
-= 0- ()
(12)
where 17 is the learning rate. Then, in order to measure the influence of d towards d', we wish to find the change in loss on d':
[(d';) L(d';
(13)
Instead of taking single training step to measure the influence of d e D on d', we can instead approximate Equation [3) with using the following:
Lemma 5. Suppose we have L.M with parameters 0. We perform gradient descent step with training sample d with learning rate v such that f = 0 - v (d; 8). Then,
L(d'; 0) L(d'; = V (d;;;))
See Appendix D for the proof.
Then, we set Inflp to be:
Inflp = VL(d:)) VL(d;
(14)
which is the dot-product between the learning gradients of d' and d.
B Royalty model
So far, we have shown the case of flat rate (see Section 3.1), which is well-suited resource-rich buyers, such as leading tech companies whose L.MS generate significant economic value due to their wide-ranging impact and scalability. In this section, we introduce the royalty model, contract framework that differs from the flat rate by offering a subscription-like structure. Under the royalty model, the price paid for training data is proportional to the future economic value generated by the LLM, providing a flexible and performance- based approach to data valuation. This scenario incorporates buyers in less dominant position those who (1) uncertain about the prospective model outcome or (2) do not own a sufficient cash flow for purchasing data with full prices. We present updated decision decision-making models for buyers and sellers follows.
Buyers. Unlike the flat pricing setting, the buyer Bg would alternatively pay with a fractional price. Suppose each dataset Dj is priced with an individual rate 0j E [0. 1) (as we denote = (a1, ... )) then the price of an arbitrary data collection uy (x) is a fraction of its future marginal gain, i.e., x²' p = f(x,, æ' (z)) where the overall rate function f [0, l) {0, + [0, 1) depends on specific contexts. We assume that I is monotonically non-decreasing function of a. In this sense, the buyer B& reduces the risk of losing by from its cash flow while the seller is betting on the potential value of the LLM Mg. Then obtain updated objective function for By:
Ge, frac (x) = (1 - f(a, )))u (æ),
(15)
On the other hand, similar to the budget constraint (see Equation (2), here each buyer Bg has maximum rate og that it is willing to pay. Then the buyer's purchasing problem is given as
J. Nffre = arg max I (x), s.t.
(16)
N.
N2, No frac
:= fØ Ir.N (x) 2 0, f(a k
(17)
And N. is the optimal solution to maXxe Xs. Ne 9k. N frae frae (x) with given rate vector
17

---

Sellers. In the fractional pricing setting, since the buyer B5, pays for the entire data collection, there should exists a fair and transparent allocation mechanism that distributes portion of the total price charged to each individual dataset D,. That is, ,, = >h= fj x N yuk, (a (N,.,, ) where
f(+) = وز= (-). And we assume that for all j E f() is monotonically non-decreasing over a. Therefore, we have an updated profit function for Sey:
M Pirac (ay) = Σ fj 1² H., M.N..freuu (o (Eeh,N Cj.
(18)
km
which gives the following problem:
03 = arg max Tirac (ay)) s.t. agC A.
(19)
AI.M == {az E (0, Pima(aa) 2 0}.
(20)
From this point onward, the market dynamics stays the same as in the previous section. It is noted that, compared to to/maxp, maxp, r(p;) where optimal flat rate is indirectly connected to the utility, the
optimal rate of maxa, E.,,ar Frrac (a)) offers a more direct representation of the utility.
B.1 Solving for optimal price of the royalty model
Similar flat rate. in the case of royalty model, we need to solve buyer's problem
m, N 1-11tm toe Nk.N 1. ax) (before the arrival of S;) and max N Irac (after the
arrival of Sj) for all k E [M] and seller's problem max Tfrac (aj)
Solve buyer's problems. For each feasible collection of datasets x E NK.N (before the arrival of S)), denotes it union with dataset
Dj as xeew Then we run the check: (1) if the net utility of
vew is larger than the one of 1., i.e., IN.NN (ee"ew) >> N..NN N.
and (2) if the
rate for purchasing neew is still under the budget ūik i.e., I[[a"" aj] xx0ew) ) S ag where [a" ag]
denotes concatenating Or, o. If the answer is positive to bother tests, then we can determine that the buyer B₁ will change its decision and purchase S, under the rate aj.
Solve seller's problem. First, we consider when aj 0. We could first find all E 1 (ar new
such that Ik,N Irc
(Rt.Nttre (r NN, And we denote the set that contains such as X% 1,, If X -... is empty, then (B,.D; 0 (indicating whether B& will purchase D, at price Py or not), as S, cannot bring positive value to BN; else, then for all æ E X R thanks to the monotonicity of f, over a.j, we could gradually increase aj until the either of the two criterion are met first: (1) we find the largest aj such that N.taee (x="x) > Js.N frac (æ K.N k.g... -1.fra).. and (2) [ ([aT an) wewww) S Tg.. Then we have the following property about the optimal rate oj
for Equation (19]
Lemma 6 (Characterization of az under royalty model). Define az as
ag: fz([aT as] ) < - (1 fy(ar, KN -1 us 0x (x R. N 1, 1,fra
min sup (a)E(0.1)
uy (wenow)
sup
{az: fs([a as] , " S az
(21)
a, a,E(0.1)
For every & E A - and all l E [M] we obtain oz and their union UM UxE x-
{ag)
Then we have az E M U {az).
Remark B.1 (Similarities between flar rate and royalty model). Observing from Lemmas 2 and 6 we see that the both the optimal price p3 and the optimal rate aj are closely tied to BN 's maximum willingness to pay. That is, compared to the market prior to the arrival of S,, the optimal values are characterized by the minimum of two factors: (1) marginal utility that Sj provides to Bg and (2) Bg 's budget surplus. It is also noted that, under royalty model, the rate function f also plays an important role as it determines the how the single rate aj affects the total rate that B₁ pays.
18

---

C Applications for Real-Life Scenarios
In real-life settings, the relationship between the data valuation of training sample and the buyer's utility up (i.e., the economical value, which may be expressed in dollar amounts) can have different mappings, mentioned in Section 3.11 Suppose the data valuation function is denoted 2a D IR for a dataset D. Then, a buyer may expect a linear relationship between t and 1kk, where the utility increases the data valuation score increases, Alternatively, a buyer may prefer to only purchase data beyond a certain threshold for Uk. In this section, we present three types mappings between en and ug reflect these scenarios: linear, discrete, and zero-one mappings. We show that these mappings can be easily adapted to our proposed framework in Section 3 We only present the updated buyer's purchasing problem (Equation (2)) since the seller's pricing problem (Equation stays the same.
C.1 Linear Outcome
In practice, there are many applications where us is an affine function of U/. As previously mentioned, training LLMs on data with higher valuation scores Ug can result better economic value towards downstream model performance, as shown in previous works [22 23 In this outcome setting, in addition to considering dg to be a affine function ut, we also include a bias variable 33 to account for other potential other factors that are independent of uk- Therefore, we can set ug = =Yee + into Equation ( where y E R., is a known coefficient, and obtain buyer BB net utility function for the lincar outcome:
(x) = (vx(2) + 3 -3x'r x=p.
(22)
To obtain optimal price p", we can directly refer to same procedure described in Section 3 Jusing set values for y and 3.
C.2 Discrete Outcome
There are also many applications where ug is discrete. For instance, if the data buyers are participating in an LLM benchmark challenge, such MMLU [88], then training on data that falls within various ranges 222 may lead to drastically different model performance, and hence leaderboard rankings.
To mirror this, consider uą to be category variable. We denote fca ) as strictly increasing set of numbers such that when Us E [cs, C1+1), the buyer will receive reward uk. We also assume that
uk,b+1 uk.h since higher data valuation scores may lead to larger reward. Therefore, we could H
set ug Σim (vx cx41) (x)) ",, and rewrite buyer Bg net utility function as
Ik.Nv (x)) =
Σ (.) Uk.
- x.=T
(23)
We again apply the same procedure in Section 3 to solve for the optimal pricing.
C.3 Zero-One Outcome
There scenarios where the data buyers risk-adverse and focus the effects of rare events. In these cases, suppose that Uk is normalized between [0, 1]. Then buyers may wish to purchase training data with higher values of Ug. assuming that purchasing data with lower ty may result in severe adverse effects. For instance, data buyers who are building AI for healthcare should not purchase data with incorrect medical information, and even small amount of contaminated data can result in severe real-life consequences such as mis mis-diagnosis [28, 89 or unsuitable medical protocols in emergency situations [90]. Therefore, in this context, we consider Uy as a generalized Bernoulli distribution. The downstream outcome has small positive reward u with probability (normal events) and a massive negative reward u with probability assume that E(u))= > 0. Therefore, we can plug in and obtain buyer B& 's net utility function:
2 (undesirable rare events). And we
N E(u((x)) - xTp T
(24)
(x( T) + TTp,
(25)
which is an affine function of ug. Therefore, we again apply same procedure in Section to solve for the optimal pricing.
19

---

C.4 Multiple tasks
In practice, many LLMs are evaluated over multiple tasks [8 To this end, we consider the context where buyer B₃ wishes their model Mr to perform well across multiple tasks, denoted as Q. Each data valuation score for a task is denoted by of, "'. zQ and the vector of all task valuations denoted as 0y = (0K,1 ... v).Q).. U..Q)) Then we consider that the utility ug is an affine function of the urility in each task, denoted by u4 [ug,1--. (x.Q) that is, us = 0 un + e, where 0 E RQ is a coefficient vector and E R denotes other factors independent from ug. We also assume that the each task is one of three categories mentioned in the last section. Therefore, we can rewrite un as function of Uk. which gives up = 0" ux((x (vk) + Therefore, the buyer's net utility function becomes
Nk.N (x) 0" (v)(w())) + + x"p
(26)
whose solution could adopt the same procedure as described in Section 3 to solve for the optimal pricing.
D Lemmas and Proofs
Lemma D With Assumptions to 2 any exploitative pricing (i.e., < pt, Vt) will only maximize cumulative utility within finite horizon after which it is strictly suboptimal.
Proof. Lemma Commma@eqivalenttee states that with Assumptions T the optimal pricing strategy for the buyer is also the ideal price p/. Thus, we show that when the buyer pays the ideal price pi. its total value is the largest, i.e.,
[l pt + 5E (r(t..P)) ue, be]] 2 E [u - Pt + 6R (pt. we be]l
(27)
for all P. E (0.
The seller will not set a price above its ideal price p as will decreases its profit, since the ideal price Př should be its profit-maxiizing price. Any pt. př results lower profits.
Also, the buyer will not accept a price pt > př. since decreases the net utility gain us - pr without increasing seller's participation probability I7 (P),
Through some linear transformation, this is equivalent to show that
E [p: P + 8IE [G(T(p) pt) π T(p.. p()) ue, be]] 2 0.
(28)
We first find the lower bound of G. We see that vē gives a payoff of
minee [0. oz) Blu p.1
E
5t (u+-
pi)
(29)
1
Therefore, we must have G 2 min.e(o.oo) a given un and b,
Elul-pil 1-8
Along with Assumptions to 2 this gives us, for
8G (T(P př) pi).
2 8GL 2 1,
(30)
pt - Pt
implying that
E [p: - P. + SE (G pi) T (P) ue, be]l 0.
(31)
Lemma 7 (Participation Loss Is Lower-Bounded), With Assumption 1 let P limm Pr and
S : Ei (1 ग (P., pt Then for the class of all exploitative pricing strategies where Pt S př Vt, we have (1) Pr is strictly decreasing and (2) The limit of reduced participation is lower bounded, i.c. 0.
1 P
Proof. The first part is trivial see as we assume that π (pc.PP) < for ps < P. from Assump- tion
20

---

For the second part, we first denote ::= lim Pt. Then we take the following transformation:
log P = =log
πI 1=0
n (pt p.)
Σ 1=0
log (p).
(32)
Since have 0 (pi, p.) 1, then log (T(p,, pf)) S (ps, (p) - 1, indicating that
log P =
log (mt p()) - (m(2) -1) = SS
(33)
i=0
which gives P Se S by exponentiating both sides.
Lemma 2
Seller Sj 's optimal price for Dj is characterized one of buyers' MWP:
pj E U
(34)
Proof. Recall that without dataset Dj, each buyer B₁ has already solved maX x-1 Ik.N -1. according to our market definition in Section 3 where Nx,N-1 is the set of all feasible purchase decisions. Next, after seller Sj (with D,) has arrived on the market, in which B, will purchase Dj at a potential price Pj. For each feasible purchase decision (i.c., a collection of datasets), represented by the unit vector indicating D2 is selected. For buyer B₃ to change their previous decision to purchase requirements that need to be satisfied. First, we must have:
analyze the conditions
N let + eg denote its union with Dj. where ej is
D,, there
9k. N ( + +ey) > 9
(35)
That is, the net ulility Nk,N (æ + ee) of purchasing decisions æ + ej. must be larger than the net of a previous optimal purchasing decision A It is also noted that Ik.N (a.,N(" (2*,N-1) N 15 (a I.). Second, for buyer B₁ to purchase ey price Pj. must
utility Ok.N (
fulfill the budget constraint:
Pj S by - (Bhh--1²² p Aby (bk.
(36)
which ensures that purchasing D, does not exceed the buyer's budget by. If both requirements are satisfied, then the buyer B& will change their previous purchasing decision in order to purchase D, under the price Py. This procedure is presented in detail in Algorithm iin Appendix F.
Next, given the conditions for the buyer B3 to purchase Dj. the seller must solve maxp, PM r(p)) to find the optimal price pj- First, we consider an edge case where the price of dataset Dj is set as Pj = For a buyer BN, we denote X as the set of all purchasing decisions where including D, in the purchase improves the buyer's previous net utility Jk.N
+ 6j N we have N + e,) > 9k.N (i... Na. (....N11). That for every X is empty, then Bg will not purchase D, at any price, since D, cannot bring positive improved net utility BR. Then, when pj gradually increases and exceeds max {min ( [Aug (ær + ez), Abg h then B& will decide not purchase Dj. causing the value of k= x K.N (p) to drop by one. Since the profit function r(2)) is piecewise linear function, the its optimal point must be one of its breakpoints. D
Lemma The optimal price for the seller S under our framework is
p min(ue, bc), Vt.
(37)
With assumptions
1 to P gives the buyer the maximum cumulative net utility over infinite horizon.
Proof. In a single buyer and seller setting, we could trivially see that the optimal price for the seller pī min{ u. be): ): if p. pi then the buyer would not purchase this dataset since the net utility would be negative; if pt < Př, then the seller's profit is not maximized.
Further, refer to the proof of Lemma for the second part.
21

---

Lemma 4 The threshold time period where the fairshare pricing obtains higher cumulative utility than any class of exploitative pricing is:
[-1 (wt - pi)
1" = sup TEN:E
Σ
8f
II (p.. (4: pi) 50
(38)
Pr pe WE
And 1" is increasing increases.
Proof. For any given class of exploitative pricing strategy, when increases, the part: E 8' (iu - pi) I T(p,, pf) (u, - p:)) increases. Therefore, for all class of exploitative pricing strategy, i.e., Pt. pi, Vt, then t" also decreases.
Lemma Suppose we have a L.MM with parameters 0. We perform a gradient descent step with training sample d with learning rate v such that 0 = 0 v [(d: 0). Then,
[id'; 8) [(d'; 0) V [d': 0) ((d;))
Proof. First, we consider the change in loss of using a first-order approximation: L(d':) C(d';) 0) + vL(d' 0) 0) 0
(39)
[(d'; 0) (d'; - vL(d'; 0) + 0010-013 011²)
(40)
Next, suppose gradient descent step taken on training sample d, and the model parameters updated as: θ v). Thus, we have f V 0), and the change in loss can be written
[(d'; 0) L(d'; = () () a VL(d'; (d
(41)
Given that 17 is a constant.
Lemma Define a as
ag fs([aT as] pew) < (1 f3(a,, -1, ux va R. (t
Fra
min
sup ay E(U,1)
Ug (xx
sup 0,
{az : [i([ as] www S an
(42)
For every æ x11 - and &s E [M], we obtain of and their union U UnE x
{ag
Then have a3 U Unext N 1.6 {a.).
Proof. We show that, for every æ E X -ffac a" gives the largest revenue of + ej for Sj (note that æ + ee = "e"), Recall that in the main text, we need to increase aj from zero until we find the largest ay such that either of:
1. Il, (x + +e²) Nk,N.am www.N.meekkkkke. K.
2. [[( a;} x +ee) Okk.
we rewrite the first condition, we are essentially looking for o, such that
&. 11 < (1 (1 -1- ff((, x N. (.rrc)) 1,fra)) us
sup (01)
a, fa([aT
(43)
ux + E2)
Then we see that the revenue that for each X1 N seller Sy can make from buyer By is f f a,} x + (x)
(44)
where [a( ayj + e5) is non-decreasing function over aj while other terms stays fixed. It indicates that a. is the largest a, that the seller S, could set for buyer B& to purchase Sj. Therefore, the optimal rate az is one of the rates U.M. U0EX2 1.ime {a
22

---

E
Additional Experimental Details
E.1
Data Valuation Experiments
For each dataset, we randomly sample 200 demonstrations from the validation set to form a rep- resentative dataset Each data sample in the market is then scored based on its similarity this representative set.
After obtaining purchasing decisions for all data samples, the buyers train their models using the purchased data. In order to conduct a fair comparison across buyers, we sample a number of data from the buyers' purchases (shown in 6j. We train each model (i.e., buyer) on these samples separately using LoRA A [91] for 3 epochs, with a learning rate of 2e-5 and batch size 32. All models trained A6000 GPUs single GPU settings and take less than hour.
Model Training:
For evaluation, we use the test splits of the previously mentioned datasets. In particular, we use 5-shot evaluation on the MathQA test set, and 4-shot evaluation in on the MedQA test. Table 2 in Appendix I shows the demonstrations used for 5-shot and 4-shot evaluation.
Model Evaluation:
F.2 Data Pricing Experiments
We simulate two buyer budgets at each time step L. The first buyer (high budget) has a budget uniformly randomly generated between 95% and 100% of the total utilities of all 10 datasets listed in the market. The second buyer (low budget) has budget uniformly randomly generated between 90% and 95% the total utilities of all datasets listed in the market. At each time period, seller's arriving orders randomly shuffled. And they prices their datasets sequentially.
Experiment Setups:
Robustness Check. As discussed in Lemma 4 the threshold t" when fairshare becomes optimal for the buyer) increases as the discount factor decreases. To run robustness check. for the high- budget buyer in Figure 4aj setting 8 = 0.999, 0.99, 0.98 yields t* = 32, 38, 44 respectively, which is consistent with our theoretical analysis. In below, we compare the change of buyer's accumulative utilities over different values of 6.
200
200
Fairdhun:
Cunulative Uniiiies
Redaced
150
Randum
Exploitative
100
100
0-
0-
40
80
120
1611
200
40
80
1201
160
2001
120
160
200
(
0.980
(b) 5 = 0.990
(6)
0.999
Figure 7: Buyer's accumulative utilities over time periods for 5 = 0.980, 0.990, 0.999.
'Note: For PIQA we take 200 samples from the training set since the validation set is commonly reserved for testing.
23

---

Additional Experiments and Figures
F1
Algorithms
Algorithm Determine if buyer B₃ will purchase dataset Dj at price Pj
1: Inputs: prices previous optimal RN,N- feasible set N price Pj. budget by.
2: Output: indicator (Bc.Ds P53
3: Initialize 11 (D,5D <-
4: Pj +-
5: for x NI do
6: weew f xt tj
7:
if Ih.N N2:,N(
> Jk.N (0. - [BE 1
and T p + S by then
8:
9:
break end if
11: end for
return 1 (Bc.
Algorithm 2 Market Dynamic Procedure
1: Inputs: Buyers 1== Sellers {S
2: Initialization: Buyers {B& enter the market
3: for to N do
4: Seller Sj enters with potential prices PJMM for dataset D
5: for all pj P do
6: for to M do
7:
Buyer B₃ solves:
5 khi-1 arg max Ik.j (x)
meNt.,
8:
decide whether purchase Dj price Pj
See Eqn.
9:
end for
Seller computes net profit r(p)) assuming price Pj
See Eqn.
11: end for
Seller selects:
p3
arg max r(p,) P
13:
and sets p) as the fixed price for Dj
> See Eqn.
14:
end for
F22 Data Valuation Oracle Experiments
Spearman0..42
Speaman-0.544
BP Acc.=26.13
20.5
One Stop
One-Spep
Accuracy
76.00
Accuracy
75.75
75.50
JK:
1.55
26
255
15:
1.5K 2K 2.5K Price
Infie
Infi,
Price
(a) PIQA (blama-1b) (b) MathQA (Pythia-
(c) PIQA (Llama-1b)
(d) MathQA (Pythia-
Figure 8: Left, middle-left columns: Correlation analysis between oracle and Inflyp valuation. Right, middle-right columns: Performance analysis between between oracle and Infhp valuation.
Influence-based methods, such as Inflip approximate the influence of sample d on d' for model parameterized by 0 by estimating the effects of training or ''upweighting" (alternatively, dowwweiggnwwn d (see Appendix(A) A). In past literature, Inflpp is validation through an

---

"Oracle" (One-Step Training) score, which we denote as Oracle(d, d') L(d'; L(d'; 0). where i 0 LyV 0) and is the learning rate [56
23
To compare the difference between Infli valuation versus its oracle valuation, we conduct the same experiments described in Section 4.3] Figure that Infl and Oracle have decent correlation in their agreement in their valuation of the sellers' data, which supports findings previous works [7]]. We note that in the case where correlation is decent, such as in Figure [b) the final model performance between these methods is close, as seen Figure &d In the case where correlation lower, such as in Figure Sal the final model performance between these methods initially have gap, but become more similar as the amount of data purchased increases, as seen in Figure Sc This suggests that in practice, even in cases when the agreement between Inflpp and Oracle may not be very high, final model performance resulting from these two methods can still be similar.
F.3 Additional Experimental Results
BP Acc.=26.13
BP Acc. #27.0
BP Acc.=70.4
Price per Semple
27.25
71.75
Accuracy
28
27.00
Accuracy
71.25
26.75
27
71.00
0,50
26.50
26
70.75
CMAA NE. M6L mas. Go
1K
1.5K 2K
26.25
1K
2.5K
Method
Price
Price
Price
(a) Avg Price (lb)
(b) MathQA (Ib)
(c) MedQA (Ib)
(d) PIQA (1b)
BP Acc.=27.2
BP Acc. 67.41
Price per Sample
2,509
0.334
Accuracy
29
28
Accuracy
28.5
Accuracy
69.0
27
27.0
68.5
Dead
26.5
Contert
1K
1.5K 2K 2.5K
0.5K
1K
1.5K
Method
Price
Price
Price
(e) Avg Price (410m) (f) MathQA (410m)
(g) MedQA (410m)
(h) PIQA (410m)
Figure 9: Buyers' model (top row: Pythia-lb, bottom row: Pythia-410m) performance and costs from their purchased data from math, medical, and physical reasoning data markets. Purchasing decisions were using the constant, random, BM25, Inflp data valuation methods (see Section 4.3 for details).
25

---

Cunmalltve Utblines
Fundur
Commanattv Utitice
Comulatee Prolit
HolSelen Market
10
240
Hecees
2440
160
160
50
80
80
20 40 60 100
20 40 60 so 100
20
40
60
80
100
20 40 60 80 100 Time Steps
Time Steps
Time Steps
Time Steps
(a) High-budget buyers'
(b) Low-budget buyers' utilities (Pythia-Ib).
(c) Sellers'
profits (Pythia-
(d) Sellers' (Pythia-Ib).
participation
utilities (Pythia-Ib).
lb).
Cummulativ Utilities
o Selters Market
of Sollers on Market
Fandhre
Comulttive Phefftt
10
240
75
160
50
80
25
20 40 60 100
20 80 100
20
80 100
20
80 100 Time Steps
Time Stops
Time Steps
Time Steps
(e) High-budger buyers' utilities (Pythia-410m).
(f) Low-budget buyers' utilities (Pythia-410m).
(g) Sellers' 410m).
profits (Pythia-
(h) Sellers' (Pythia-410m).
participation
Cummulative Utlities
240
Farsiie
Commutativ Uniiites
240
Comulative Proftt
80
Selter Market
10
180
Beluel
180
60-
40
60
60
20
100
Time Steps
Time Steps
Time Steps
Time Steps
(i) High-budget buyers'
( Low-budget buyers' (k) Sellers' profits (Llama-
(I) Sellers' participation (..lama-..2Inss--11
utilities 3I. utilities (I.lama-3.2-Inst. 3.2-Inst.-lb).
Ib).
Ib).
Figure 10: Analysis of (1) buyer's cumulative utilities with high-budget buyer (Figures 10a
10e
and and low-budget buyer (Figures [0b Tor and [0j.. and (2) sellers' average cumulative profits (Figures T0c 10g and 1OK) and number of sellers in the market (Figures [Tdi 10h and TOI) over time Task: medqaQA. Experimental groups: (1) fairshare, (2) reduced, (3) random, and (4) current pricing.
(T = 100). Model: Pythia- Pythia-410m 40m, and 1.ma--.2-Inntt...
26

---

Chnsaulatvv Ltaitiies
1000
Racaa
Curmuuliivve Utinke
1000
Camuluive Proftt
320
#oe Sellers Morket
10
500
500
160
250
250
80
0-
0-
so 10000
20 40 80100
20 40 60 80 100 Time Steps
20
6a so 100 Time Steps
Time Steps
Time Steps
(a) High-budget buyers'
(b) Low-budget buyers' utilities (Pythia-Ib).
(c) Sellers' lb).
profits (Pythia-
(d) Sellers' (Pythia-Ib).
participation
utilities (Pythia-Ib).
Cuntmolativ Utiiliiss
Tanhar
o Sollemeo on Market
10
Comeslatve Profit
320
of Sollexxon on Market
10
2.40
600
300
80
Time Stops
Time Stops
Time Stops
Time Stops
(e) High-budes buyers'
(f) Low-budget buyers' utilities (Pythia-410m).
(g) Sellers' 410m).
profits (Pythia-
(h) Sellers' participation (Pythia-410m).
utilities (Pythia-410m).
Cunmmlativ Uillities
1000
Radaa
Coreeallbbv Uelitkes
1000
Comuiativ Polit
320
Seiters Mieket
10
750
750
240
500
250
250
80
0-
0-
so
10100
Time Steps
Time Steps
Time Steps
Time Stops
(i) High-budget buyers'
( Low-budget buyers' (k) Sellers' profits (Llama-
(I) Sellers' participation
utilities 3.lama-3.2-Inst- utilities (Llama-3.2-Inns) 3.2-Inst.-Ib).
3,2-Inst.-Ilb..
1b).
lb).
Figure 11: Analysis of (1) buyer's cumulative utilities with high-budget buyer (Figures 1la
lle
Tii) and low-budget buyer (Figures [Tb]] IIf and IT] and (2) sellers' average cumulative profits
(Figures TTc T1g and TTk) and number of sellers in the market (Figures [id Ih and III) over time Task: MathQA. Experimental groups: (1) fairshare, (2) reduced, (3) random, and (4) current pricing.
(T 100). Model: Pythia-Ib, Pythia-410m. and Llama-3.22
27

---

Cummulative Uttiites
320
Commocative Utzites
320
Camolative Polftt
2,00
#o Sellers Morket
240
240
1.00
80
80
50
20 40 60 100
20 40 60 so 100 Time Steps
20 40 60 80 100 Time Steps
20
80 100 Time Steps
60
Time Steps
(a) High-budger buyers'
(b) Low-budget buyers' utilities (Pythia-Ib).
(c) Sellers' profits (Pythia-
(d) Sellers' (Pythia-lb).
participation
utilities (Pythia-Ib).
lb).
Cuncmolative Utiliiies
450
Tanhare
SallSlll on Market
10
Conuuaatve Profit
400
Solleen on Market
10
300
300
150
100
Time Stops
Time Stops
Time Stops
Time Stops
(e) High-budget buyers'
(f) Low-budget buyers'
(g) Sellers' 410m).
profits (Pythia-
(h) Sellers' participation (Pythia-410m).
utilities (Pythia-410m).
utilities (Pythia-410m).
Cummulatii atve Uitities
300
Betavod
Fandire
CimmruSaatvv Utcktees
300
Cumulaiie Proft
120
Rol on Market
10
Rodle
60
100-
100
100
20 40
Time Steps
Time Steps
Time Steps
Time Steps
(i) High-budget buyers'
( Low-budget buyers' (k) Sellers' profits (Ilamaa-
(I) Sellers' participation
utilities 3I.ta-a-3.2-Inst. utilities -I-ss 3.2-Inst.-lb).
1b).
lb).
Figure 12: Analysis of (1) buyer's cumulative utilities with high-budget buyer (Figures 12a 12e
and [2i) and low-budget buyer (Figures [2b 12n and [2j.. and (2) sellers' average cumulative profits (Figures T2c 12g and 12K) and number of sellers in the market (Figures [2d [2hand [2]] over time (T = 100). Model: Pythia-lb, bb, Pythia-410m and Llama-3.22 3.2ns-,,- Task: PIQA. Experimental
groups: (1) fairshare, (2) reduced, (3) random, and (4) current pricing.
28

---

of Purchased Datasets
10-
# of Purchased Datascis
10
Purchased Datasets
10
Fairshure Reducad
Fairshure Rediced Random Exploitative
Fainbato
Reduced
5-
5
Randomn
Random
Exploinaiie
Exploitative
0
0
20
40
60
80 100
0 20
40
60
80 100
0
20
40
60
80 100
Time Steps
Time Steps
Time Steps
(a) High-budger buyer (MedAQ. (b) High-budget buyer (MedAQ. Pythia-Ih).
(c)
High-budget buyer (MedAQ.
Pythia-410m).
1.
Purchsed Datasets
10
Purchasee Datasets
10
of Datasets
10
Fainsare
Fairshure
Fairslure
Reduced Random
Reduced Random Exploitative
Reduced
5
Random
Current
Current
0
0
0
20
40
60
80 100
20
40
60
80 100
20 40 60
80 100
Time Steps
Time Steps
Time Steps
(d) High-budget buyer (MathAQ, Pythia-Ib).
High-budget buyer (MathAQ.
(f)
High-budget buyer (MathAQ.
Pythia-410m).
Llama-3.2Innst-IIbb)
Purchased Datasets
10
# of Purchased Datasets
10
# Pfrchasee Datasets
10
Fairshare
Fairshare
Fainhare
Reducal Random Exploitatie
Reduced Random
Reduced
5
5
5
Random
Cureett
Exploitative
0
0
0
20
40
60
80 100
0
20 40
60 80 100
20 40
60
80 100
Time Steps
Time Steps
Time Steps
(g) High-budget buyer (PiQA, (h) High-budget buyer (PiQA, (i) High-budget buyer (PIQA, Pythia-Ib).
Pythia-410m).
1.ama-3.2-Inst-11b
Figure 13: Number of purchased datasets for the buyer with high budget over time periods (T = 100). Model: Pythia-lb, Pythia-410m, and 1lama-3.2-Instt-- Task: MedQA, MathQA, and PiQA. Experimental groups: (1) fairshare, (2) reduced, (3) random, and (4) current pricing.
29

---

of Purchased Datasets
10
Fairshare Reducad Randogn Exploitaive
# of Purchased Datascis
10
Purchased Datasets
10
Faintare
Reduced
Fairslane Rediced Random Exploitativ
Random
Exploitative
5
0
0
20
40
60 80 100
0
20
40
60
80 100
0
20
40
60
80 100
Time Steps
Time Steps
Time Steps
(a) Low-budget buyer (MedAQ. Pythia-Ih).
(b) Low-budget buyer (MedAQ. (c) Low-budget buyer (MedAQ. Pythia-Ib).
Llama-3.-Imnst-Ibb,
Purchsed Datasets
10
Pfrchase Datasets
10
Fairslure
Pfrchasde Datasets
10
Fainbare
Reduced
Reduced
Randem
Randog
Expleitative
Explomitiie
5
0
0
0
20
40
60
80 100
20
40
60
80 100
20 40
60
80 100
Time Steps
Time Steps
Time Steps
(d) Low-budget buyer (MathQA, Low-budget buyer (MathQA, Pythia-Ih).
(f)
Low-hudget huyer (MathQA, Llama-3.2IInst-IIb)
Pythia-410m).
of Purchased Datasets
10
Fairshare
# of Purchased Datasets
10
# Purchased Datasets
10
Fainhhre
Reduced
Reduced
Random Exploitative
Random
Exploitative
5
5
0
0
0
20
40
60 80 100
0
20 40 60 80 100
20 40 60 80 100
Time Steps
Time Steps
Time Steps
(g) Low-budget buyer (PiQA, (h) Low-budget buyer (PiQA, (i) Low-budget buyer (PiQA, Llama- 3.2-Inst.-lb).
Pythia-Ib).
Pythia-410m).
Figure 14: Number of purchased datasets for the buyer with low budget over time periods (T = 100). Model: Pythia-lb, lb, Pythia-410m, and 3lamma3.2-Innst-111 Task: MedQA, MathQA. and PIQA. Experimental groups: (1) fairshare, (2) reduced, (3) random, and (4) current pricing.
30

---

F4
Datasets
Dutasel
# oo Tralin/Vaiiid
Example
MatQQA
29837/4475/2985
Question: nunning speed of crosses a pole seconds. length of the train? a) 140, b) 130, 120, 170, 160
Answer: C
GSMSK
7473/1319
Question: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips May. many clips did Natalia altogether April May?
Answer: 72
MedQA
10178/1272/1273
Question: 27-ycar-old presents the emergency room with persistent fover, nausea, and vomiting for the past days. While watting to be seen. he quickly becomes discriented visible signs difliculty bresathing with copious oral secretions and generalized muscle twitching. The patient's temperanure is 104"F (40'C), blood pressure 9064 mmHg. pulse 88/mnn, and respirations are 18/min with oxygen air. When the
agitated. Upon examination,
nasal cannula, patient becomes fcarful and combative. The patient sedated and placed on mechanical ventilation. Which of following beef b) Epiglottic cyst c) Mosquito bite d) Spelunking
saturation 90%
tries
factor for the paticut's likely diagnosis? a) Contaminated
PIQA
16000/2000
Question:
How do ready guinea pig cage for it's new occupants? a) Provide the guinen pig with eage full fow inches bodding made ripped paper strips, you will also nood supply it with water bottle and food dish. b) Provide the guinea pig with a cage full of inches bedding made of ripped jeans materrial, you also supply water bottle and a food dish. Answer:
Table Dataset splits and demonstrations from the MathQA, GSM8K, MedQA, and PIQA datasets
31

---

Dataset
Prompts
MatiQA
Question: the banker' is gain of certain sum due 3 years bence at 10 % per annum is 36 what the present worth? as 400, rs. 300, rs. 500, d rs. 350.e e none of these
Answer:
Question: average age of students of an adult school is 40 years 120 new students whose average age is 32 years joined school. as
the average age is decrensed by 4 years. the number students of the school after joining of the new students a) 1200, b) 120, c) 360, d 240, e) none of these
Auswer:
Question: sophia linished 2/3 of book. she calculated that she finished 90 more pages than she has yet read. how long is her book ? a 229, b) 270. ) 877 266 281
Answer:
Question: 120 is what percent of 50 ? no 5 %. b)) 240 %, c 50% .d% .d) _. 5,000
Answer:
Question: there are 10 girls and 20 boys in classroom. what the ratio of girls 10/ 30. e2/5
Answer:
MedQA
pediatrician's office because she is concerned about his feeding habits. He was bomn without complications and has not had any medical problems until this time. However, for the past days, be has been fussy, regurgitating all of his foods, and vomit is yellow in color. On physical exam, the child's s abdomen minimally distended but other abnormalities are appreciated. Which of the following embryologic errors could account for this presentation? a) Abnormal migration of ventral pancreatic bud h) Complete failure of proximal duodenum recunalize Abnormal hypertophy of the pylors Failure of Interal body folds to move ventrally and fuse in the midline
Question: A mother brings her w-wee old
Answer:
Question: A 53-year-old man comes the emergency department because of severe right-sided flank pain for hours. The pain is colicky, radiates towards right groin, and he deseribes as 8/10 in intensity. He has vomited once. He has history of similar episodes the pasi. Last year. he treated with naprosen for swelling puin of his right loe. He has a history of hypertension. He drinks one to two beers on the weekends. Current medications include amlodipine. He appears uncomfortable. His temperature is 100b0C (99. 30060F), pulse 101/min, and bload pressure is 130/90 Hg. Examination shows soft, nontender abdomen and right costovertebral angle tendemess. An upright x-ray of the ahdomen shows no abnormalities. CT scan grade I tnyrroncphrosis on the right. Which of the following is most likely to be soca on urinalysis? Urinary pH: 7.3 b) Urinary pH: 4.7 e) Positive nitriles test d) Largely positive urinary protein Answer:
ahdomen and polvis shows a 7-mm stone in the proximal ureter and
Question: A 48-year-old woman comes to the emergency department because of a photosensitive blistering rash on her hands, forearms, and face for weeks. The lesions not itchy, She has also noticed that her urine been dark brown in color recently. Twenty years ago, she was successfully treated for Coats discase of the retina via retinal sclerotherapy. She currenily on hornomal replacement therapy for perimenopuusal symptoms. similar skin lesions. Examination shows multiplc fluid- dilled blisters and oozing cresions on the forearms, dorsal side of both hands, and forehead. There is hyperpigmented scarring and patches of bald skin along the sides of the blisters. Laboratory studies show normal management to induce remission in this patient? a) Pursue liver transplantation b) Begin oral thalidomide therapy c) Begin phlebotomy therapy Begin oral hy lydroxychlorqquin therapy Answer: C
and sister have a histury of
ferritin concentration. Which of the following the most appropriate step in
Question: 23-year-old pregnant woman at weeks gestation presents burning upon urination. states started day ago and has been wersening despite drinking water and taking cranberry extract. She otherwise foels well and is followed by a docter for her preynancy. Her temperature is 97. 700000 (36.5000M0)) blood pressure is 122/77 mmllg. air. Physical exam is notable for ahsence
pulse is 80/min, respirations of costovertebral angle tenderness and gravid uterus. Which of the following is the best treatment for this patient? a) Ampicillin Coftriaxone e) Doxycycline d) Nirrofuantoin Answer: D
19/min, and oxygen saturation 98%
Table 2: Demonstrations included for 5-shot evaluation on the MathQA dataset and for 4-shot evaluation on the MedQA dataset. Demostrations were randomly selected from their respective dataset's 's training sets.
32

---

G Limitations and Impact
This paper addresses the critical issue of fairshare pricing in the data market for large language models (LLMs) by proposing framework and methodologies for fair compensation of datasets from LLM developers data annotators. Our work directly tackles the ethical and societal challenges in the current data market, where many data annotators are underpaid and receive compensation significantly disconnected from the true economic value their contributions bring to LLMs.
G.1 Limitations
Since our work proposes a novel fairshare framework, there are several lines of future research that can investigate future adjustments this framework, which lie beyond the scope of our paper. For instance, sarge-scale simulation of this market with wider range of datasets and models is one possibility. addition, running the simulation with human buyers/sellers is another avenue. Finally, there are several other diverse market dynamics (e.g., incomplete information between buyers/sellers) that can be explored with our proposed framework.
G.2 Impact Statement
From ethical and societal perspectives, our framework prioritizes the welfare of both data annotators and LLM developers. Our methodology ensures that data annotators fairly compensated for their labor, promoting equity and fairness in the data ecosystem. This contributes to mitigating the exploitation of vulnerable annotators in the data market and aligns the incentives of stakeholders toward more ethical and sustainable practice. In addition, our framework also benefits LLM developers, by demonstrating that our framework maximizes their utilities and welfare the long term. Fair compensation encourages ongoing participation of data annotators in the market, ensuring a steady supply of diverse, high-quality datasets essential for LLM development. By addressing existing inequities, our work lays the foundation for more sustainable, equitable, and mutually beneficial ecosystem for all stakcholders the LLM data market.
33

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We base abstract/introduuciin on the experiments, analysis, and main findings in our paper. Our theoretical and empirical results support the framework we propose.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss our limitations Appendix G
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiaatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
34

---

Answer: [Yes]
Justification:
All assumptions and proofs are discussed the main paper and in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide details on datasets, dataset splits, models, and training proce- dure/hyperparameters in the paper. In addition, all resources use for our experiments are open-sourced.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
35

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
Our code/data will be openly availiable on GitHub
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.col
plic/guidea//oe Sewiiddde//ooeeSuumiisaeese wwwpbmmmsssonniii ssioiiiicyyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ips.cc/ppile wttpcccpppppppppeeeeeeeeeeeeeeee http:////ggeeee/edeSSubiiiissss /Codeeooodiisssseeeeeeee siiliiicccyyiiiiiccococcccciiipi for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We describe all these details in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The experiments in paper focus qualitative directional insights, not formal statistical claims.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
36

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We describe this in Appendix E.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and confirms that our research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss this in Appendix G
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
37

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: All data/models used in our research has been obtained from existing open- sourced resources that researchers already use.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We provide citations and acknowledgments for all materials (datasets, models, code) used in our research.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
38

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thoode. canddataeet has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will provide an official open source repo for this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not conduct any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do not conduct any research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
39

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
40


Reviews:
[{"review_id": "3xjGFi5Fr2", "reviewer": "Reviewer_9cVB", "summary": "Setting:  \nThe paper studies data markets over repeated interactions between persistent sellers (data providers who create and sell data) and a buyer (ML model trainer) in a full information setting i.e. the buyer data valuation is known to all parties. They also define ${p^\\ast}_i$ be the maximum price the buyer is willing to pay seller $i$ for their data, taking their budget and marginal utility into account. This price is clearly the best price for the seller - any more and the buyer drops out and any less the seller leaves some money on the table.\n\nResults:  \nThe authors introduce a new assumption about seller behavior - that if their data is priced under the maximum price ${p^\\ast}_i$, the seller $i$ is significantly less likely to take part in future markets. The authors show that this implies that consistently pricing under this threshold means that sellers dropout, leading to market collapse. Thus, under the assumptions introduced, it is optimal for buyers as well to price at ${p^\\ast}_i$.", "questions": "Would similar results hold under a more justifiable seller behavior model as described in weaknesses? Can the authors provide justification for their choice?", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- Data valuations are a very important and fast-growing topic. However, past work has mostly ignored the game theoretic aspect. I really like how this work attempts to combine the ML / empirical data valuations (such as influence estimation or attribution techniques) with more classical game theoretic investigations. I see a scope for a lot more such follow up research.   \n- The repeated interactions model has also been understudied in data markets and, as this paper shows, yields interesting mechanisms and behaviors.  \n\nWeaknesses\n\n- The biggest flaw of the work is the assumption about seller behavior - that pricing less than ${p^\\ast}_i$ leads to increase in dropout likelihood. This usage of ${p^\\ast}_i$ as the threshold used is extremely artificial since ${p^\\ast}_i$ has to do with buyer purchase-power and has nothing to do with the seller. In particular, the seller behavior changes given a different set of buyers. A more realistic model would use a buyer-independent threshold based on cost of data creation to the seller. However, in such a model the conclusions shown here would not work.   \n- Similarly, I contest the naming of ${p^\\ast}_i$ to be the \"Fairshare\" data pricing -- by definition it is the seller-optimal prices. It is unclear how this price is \"fair\".  \n- There is also insufficient literature review. In particular, a lot of the work in incentivizing data sharing in federated learning might be highly relevant here. See for e.g. citations in this very recent (parallel) work [[Pang et al. 25](https://arxiv.org/abs/2506.16731)] or [this recent survey](https://www.sciencedirect.com/science/article/abs/pii/B9780443190377000260).  \n- Finally, the existing work assumes a full information setting where all the participants know the true valuation and budgets of everyone. While this is unlikely in practice, I expect this aspect can be improved upon in future research.  \n\n\nMy overall impression is that this is an intriguing but flawed first step along a new research direction.", "was_revised": true, "final_justification": "The considered seller behavior model is deeply artificial and designed to obtain the results obtained. Here, the sellers essentially pose an ultimatum - either the average price over the runs is specifically $p_j^{\\star}$ (which is the max price the buyer would be willing to pay), or they drop off from the market. This was even admitted to as such by the authors in the rebuttal discussion (though they wrongly brush off the seriousness of this flaw). With even a slight perturbation of this assumption, the results and conclusions made collapse.\n\nFurther, this is a deeply extractive pricing model, purely benefitting the sellers. It baffles me that the authors chose to call this a \"fairshare\" pricing. \n\nWhile I do indeed think this is a very cool problem, and agree with the rest of the reviewers that the paper is well presented, I think the flaw with the seller behavior model is hard to overlook. I suspect the other reviewers who find the results of the paper compelling may not have interrogated the seller response model. I urge them to read the review and discussion posted above.", "rebuttal": "We sincerely thank Reviewer 9cVB for the thoughtful feedback. We appreciate the opportunity to clarify our core assumptions and contributions.\n\n**1. Seller Behavior Assumption and the Role of Cost:**\n \nWe thank the reviewer for this crucial question, which allows us to elaborate on our model's core mechanics. We would like to clarify that our choice to model seller behavior around the optimal price ($p^{∗}_{j}$) is a justified one for capturing perceived fairness in a transparent market.\n\n**$p^{∗}_{j}$​ as a Fairness Anchor:** Our framework assumes a transparent market with publicly known data valuation scores (lines 90, 123-125). Here, Maximum Willingness to Pay represents a market-defined anchor reflecting the highest possible value a seller’s dataset $D_j$ can provide to each buyer. Thus, this price also informs **sellers** about the utility their data contributes, not just a buyer-side metric. A rational seller, aware of this maximum, chooses between accepting \"satisficing\" [10] on any profitable offer (  $\\geq c_j$) and optimizing for a fair share of the value created. Foundational literature in behavioral economics, particularly results from the Ultimatum Game [11], demonstrates that agents frequently reject profitable but deeply unfair offers, choosing to punish unfair behavior even at a cost to themselves. This provides strong empirical support for our core assumption: a seller, perceiving a price well below the $p^{∗}_{j}$ anchor as exploitative, may exit the market. Behavioral models of inequity aversion and reciprocity [12,13] further formalize why fairness significantly influences agents' utility and market participation.\n\nThis seller reaction to perceived unfairness is the core mechanism that drives the market to a fair, strategic equilibrium. In dynamic settings, sellers exiting the market serve as credible threats of punishment, aligning with the \"Folk Theorem\" [7] of repeated games. This theorem suggests cooperative outcomes can be sustained through retaliation. Consequently, it becomes the buyer's own optimal long-term strategy to pay the fairshare price to ensure a stable supply of high-quality data. Our theoretical results in Lemma 3 and empirical findings in Section 4.2 confirm this, showing that the buyer's long-term utility is maximized at the same price that maximizes the seller's profit, creating the mutually beneficial \"win-win\" outcome that we term \"Fairshare\".\n\n**Integrating Cost as a Foundational Constraint:** We thank the reviewer for highlighting the importance of cost. Our model already incorporates it: as shown in Eqns. (3) and (4) (lines 160–163), sellers must achieve non-negative net profit (revenue minus cost).\n\nIn particular, under our current model, the conclusion (Lemmas 1-3) hold as long the optimal price $p^{*}_{j}$ is at least as large as the cost $c_j$. \n\nThe reason is that the sellers will still lower their willingness or even refuse to participate in the market when receiving compensation below $p^{*}_{j}$, according to our existing assumption.  Within this viable market, our framework is built on the foundational assumption that sellers behave rationally, seeking to optimize their own welfare (e.g., profit). This principle of the rational, welfare-maximizing agent is the standard and widely adopted axiom that underpins the majority of modern microeconomic and game-theoretic literature [14,15].\n\nOn the other hand, if the optimal price $p^{*}_{j}$ is lower than the cost $c_j$, then it means that either the investment of LLM companies or the marginal utility that the data could contribute is lower than the cost to produce these datasets. And this will lead to a market collapse, meaning that there will be no transactions. \n\nHowever, in the current norm of LLM data market, we believe that in most cases, the companies are still devoting enormous amount of investment to acquire the data as data is still a very valuable asset, which greatly exceeds the cost (e.g., labor cost) to annotate these datasets [17,18]. And this justifies the implicit assumption such that the optimal price $p^{*}_{j}$ is always larger or equal to the cost $c_j$. We thank the reviewer for pointing that out and will update our writing more rigorously in the next version of the manuscript by explicitly adding the assumption regarding the cost and the optimal price. \n\n**Cost-Based Models:** Such a model addresses important questions around production viability, specifically, \"Is it profitable to produce this data?\" Our current framework, on the other hand, focuses on a distinct, yet equally important, economic phenomenon relevant to ongoing debates about fair compensation and ethical data work: \"Am I being compensated fairly relative to the value I create?\" Recognizing the significance of both approaches, our work serves as a foundational first step, emphasizing value-based compensation as a means to enhance annotator morale and sustain long-term market participation in the LLM industry. We will clarify this perspective explicitly in our revision and highlight the potential for future work to explore cost-based models in parallel with our approach. \n\n**2. On the Naming of \"Fairshare\" Pricing:**\n \nWe appreciate the feedback regarding the term \"Fairshare\" pricing. The name was chosen to reflect that our price is not merely seller-optimal, but is a fair equilibrium outcome that benefits both parties in the long run. The fairness of this equilibrium stems directly from our mechanism's core principle: aligning a seller's compensation with the value their data contributes to the buyer's utility, as quantified by data valuation.\n\nAs we detailed in our response to the first point, this notion of a sustainable, fair equilibrium is rigorously grounded in both dynamic game theory (e.g., the Folk Theorem [7]) and behavioral economics (e.g., inequity aversion [8,9]). Our theoretical results (Lemma 3) and empirical results (Section 4.2) confirm that this price creates a mutually beneficial \"win-win\" outcome. The \"Fairshare\" label captures this crucial alignment of incentives, in contrast to the lose-lose dynamic of exploitative pricing. We will clarify this rationale and its connection to game-theoretic principles in our revision.\n\n**3. Federated Learning (FL) Literature:**\n \nWe thank the reviewer for pointing us to this relevant body of work. We are happy to expand our Related Work section to include more references to this parallel research in our revision, including the ones that the reviewer listed. While both our work and the  [Pang et al. 25] tackle data incentivization, they address fundamentally different domains:  [Pang et al. 25] focuses on preventing free-riding in decentralized, collaborative settings, whereas our work pioneers a model for a centralized, transactional data marketplace.\n\n**4. On the Full Information Assumption:**\n \nWe thank the reviewer for this critical point. Our choice to begin with a full-information model was a deliberate one, following a long-standing tradition in economic research.\n\n**An Essential Benchmark:**  The full information model serves as an essential benchmark to isolate core strategic interactions and establish a theoretical “best-case” equilibrium. This is crucial for systematically measuring the inefficiencies that arise when more complex forms of information asymmetry are introduced in future work.\n\n**Precedent in Economic Research:** This approach of starting with a complete information model is a hallmark of seminal economic research, like Akerlof's \"markets for lemons\" [1] and Spence's \"job market signaling\" [2]. They both used this methodology to establish groundbreaking insights before the field expanded to tackle various forms of asymmetric information. As one of the first papers to formalize the LLM data market with a game-theoretic framework, we believe providing a tractable model is a valuable starting point.\n\n**Scope and Enabling Future Work:** Moving to an incomplete information setting is a vast and important future research direction, not a single extension. It requires extensive assumptions about which party holds private information (cost, budget, utility), their prior beliefs, and the signaling mechanisms available. This opens up a vast research program involving different assumptions about private information and agent beliefs. By establishing the core model, we hope our work enables and encourages future research in this area, as we note in our conclusion (line 357-359). \n\nWe will revise our limitations and conclusion sections to better frame our contribution as a tractable starting point for this research area and highlight exciting avenues for future work, such as using Bayesian games to model incomplete information. \n***\n[1] Akerlof. The market for “lemons”: Quality uncertainty and the market mechanism. \n\n[2] Spence. Job market signaling. \n\n[3] Folger et al. Organizational justice and human resource management. \n\n[4] Folger et al. Fairness theory: Justice as accountability. \n\n[5] Colquitt, et al. What is organizational justice? a historical overview. \n\n[6] Adamovic. Organizational justice research: A review, synthesis, and research agenda. \n\n[7] Fudenberg et al. The Folk Theorem in Repeated Games with Discounting or with Incomplete Information. \n\n[8] Fehr et al. A Theory of Fairness, Competition, and Cooperation. \n\n[9] Rabin. Incorporating Fairness into Game Theory and Economics.\n\n[10] Simon. Rational choice and the structure of the environment. \n\n[11] Güth et al. An experimental analysis of ultimatum bargaining. \n\n[12] Bolton et al. ERC: A Theory of Equity, Reciprocity, and Competition.\n\n[13] Dufwenberg et al. A theory of sequential reciprocity. \n\n[14] Mas-Colell et al. Microeconomic theory. \n\n[15] Von Neumann et al. Theory of games and economic behavior.\n\n[16] Brandom. Scale AI’s Remotasks platform is dropping whole countries without explanation.\n\n[17] Perrigo. Meta’s $15 Billion Scale AI Deal Could Leave Gig Workers Behind."}, {"review_id": "49neMjsuR1", "reviewer": "Reviewer_7FBq", "summary": "This work addresses the challenges posed by exploitative pricing (i.e. low pricing from buyers to data annotators where compensation does not reflect skill, effort, and downstream value of contributions) in LLM data markets. Specifically, the authors demonstrate theoretically and empirically that exploitative pricing results in a lose-lose scenario with regards to data sellers being underpaid and data buyers driving sellers away and thereby weakening the data pipeline and limiting model improvements. Based on this, the authors propose a fairshare pricing mechanism that they theoretically show results in win-win scenarios, where sellers maximize profits and buyers achieve higher model performance per dollar spent. Simulation experiments. are used to validate fairshare, and ablations demonstrate the generalizability across different models, tasks, and data valuation methods.", "questions": "- The Related Work section (beginning line 106) should include a discussion of Shapley-based data valuation methods.\n\n- 17: LLM --> LLMs\n\n- 49: drive --> driving\n\n-  98: thorugh --> through", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The work is well-written, theoretically sound, and addresses an important, timely topic.\n- As the proposed fairshare pricing mechanism is model and method agnostic, it is flexible as new models and data valuation methods are developed, and flexible to real-world applications desired by potential data-market participants. \n\nWeaknesses:\n- Shapley-based data valuation methods are not discussed in the work, nor compared against in the ablation study. Methods such as In-Run Data Shapley (https://openreview.net/pdf?id=HD6bWcj87Y) should make comparison computationally feasible.", "was_revised": true, "final_justification": "I have read the other reviews and rebuttals. I agree with Reviewer AWaS that the idea in the paper is novel and is sufficiently supported, both theoretically and empirically. I will be maintaining my \"Accept\" rating.", "rebuttal": "We thank 7FBq for the thoughtful review and insightful suggestions. We will address your concerns below.\n\n**1. On Shapley-Based Data Valuation Methods:**\n\nWe thank the reviewer for this valuable suggestion. This provides a welcome opportunity to demonstrate the generality and modularity of our proposed framework. Our \"Fairshare\" framework is designed to be agnostic (line 62, 347) to the specific data valuation method used. Its core contribution is to model the market dynamics that result from a set of value scores. To empirically demonstrate this, we ran new experiments during the rebuttal period using the Run-in Shapley-based method and following the same setup as Figure 6 in our paper.\n\nFor the MedQA on LLama-3.2-1b-Instruct:\n| Purchased Data Price | BM25 | INFL_IP |DataInf | Run-In Shapley |\n| --- | --- | --- | --- | --- |\n|$500 | 30.2 | 30.6 | 29.4 | 28.3 |\n| $1500 | 30.9 | 31.6 | 29.7 | 31.0 |\n\nAs highlighted in these results, the Run-In Shapley method performs similarly with the other data valuation methods. We will include the full suite of results using Run-In Shapley in the next version of our paper.\n\nIn addition to this, we will update our Related Work section (Section 2, line 76) to include a thorough discussion of Shapley-based data valuation methods. We will contextualize our chosen methods (e.g., Infl_IP, DataInf) by comparing their scalability and theoretical properties with those of prominent Shapley-based approaches. We will also add this to our Ablation Study discussion (Section 4.3) as a key area for future comparative analysis as these methods become more scalable.\n\n**2. On Minor Typos:**\n\nThank you for your careful reading of our paper and for catching these errors. We have identified and will correct all the typos identified by the reviewer (line 17: LLM -> LLMs, line 49: drive -> driving, line 98: thorugh -> through) in the revision. We will perform another thorough proofread of the entire manuscript to ensure correctness.\n\nWe thank the reviewer once again for their valuable feedback and strong support for our work. We believe the requested changes will make our paper more comprehensive and impactful."}, {"review_id": "nnUmvD4jnj", "reviewer": "Reviewer_AWaS", "summary": "Authors propose a theoretical fairshare data pricing mechanism that maximizes the seller's profits while aligning with the dataset's utility and the buyer's budget. \nThe mechanism relies on data valuation methods to determine the value of data, and the authors use data for training LLMs as a motivating example, theoretically and empirically.  \nThe authors also empirically test their framework in a semi-synthetic setting involving two buyers, three open-source LLMs, and 10 sellers.", "questions": "- Mismatch between motivation and proposed framework. \n  - Although the proposed model is technically sound, it doesn't match the motivation of the work. The authors highlight the ethical concerns in LLM data acquisition, for example, the Sama company, Kenyan workers, and the OpenAI case ([11,12]). Accordingly to the theoretical framework, the sellers are actually companies like Sama, and not the Kenyan annotators. Additionally, the authors mention that the sellers would stop getting involved in the data market if not fairly treated, but as it was in this case, they simply move to new sellers because there is always someone willing to. So entry and exit of data buyers and sellers in the market should ideally be covered.\n  - There might be a decrease in data quality at time t, but a rise in quality at time t+x due to unfair data market or other factors outside of this. The model doesn't account for the changes in data quality at various time steps or for the entry and exit of different sellers and buyers in the data market, due to exploitative pricing and other factors. \n\n- Sections 3.1 and 3.2 and Appendix A, B and C\n  - I found Figure 1 confusing and unintuitive. For example, the data valuation methods require a model performance to compute the value of data, which is not clear from the figure. The relationship between the seller and buyer is oversimplified. For example, from the follow-up text, it seems to me there might be a back and forth between the two, sellers need potentially broadcasted buyers decision-making, and buyers need the data to compute value and set the price - to make decisions.\n  - The model of the seller's decision-making (Eqs. 3 and 4) has strong assumptions. For example, a lay seller, e.g., an annotator, might not have access to the seller utility or other data sources. As such, computing a net profit function is not trivial, and sellers might over- or underprice their data. Also how do we know the fixed price (line 158) is fair? Can buyers only get a fraction of seller data (Section 3.1 and Appendix B)?\n  - In lines 141 to 144, the authors show that utility (not the usual language of utility in dval literature) is a function of data values. It's unclear to me how well or accurately this formulation of utility captures gains such as commercial returns that buyers care about (line 196).   - The utility formulation has other limitations. In line 674, how high is high?  What's the threshold for determining that? All the utility formulations (C.1 to C.3) rely on marginal contribution-based data valuation methods defined in lines 141. I imagine it could be possible to combine various forms of valuating data to determine the utility.\n  - The setup for the results in Figure 3 is oversimplified. For example, why is utility u constant?\n\n- Experimental setups and analyses \n  - Given that authors consider varied tasks, why do they only use the affine mappings and not test the other formulations of utility? (c.2, c.3, c.4)? \n  - There is little justification or reasoning given for the choice of various parameters, e.g., how did the authors choose \\delta and T?\n\n- Limitations or discussions section \n  - Given that the authors make so many assumptions and the framework could be understood from various viewpoints, I think the paper should have a limitations section or discussions to include some of these perspectives and assumptions.", "limitations": "Given that the authors make so many assumptions and the framework could be understood from various viewpoints, I think the paper should have a limitations section or discussions to include some of these perspectives and assumptions.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n- The authors provide a nice background and motivation for their proposed framework of data pricing. \n- The theoretical model is well-motivated, technically sound, clearly written, and easy to parse. I especially like the authors' idea of incorporating both sellers' and buyers' utility and the long-term effects of (un)fair data pricing.\n- The motivating empirical results in Figures 2 and 3 are intuitive and complementary to their setup. \n- In the empirical experiments, the authors follow up with simulations specific to the LLM running example. \n\n**Weaknesses**\n\n- Mismatch between motivation and proposed framework. \n  - Although the proposed model is technically sound, it doesn't match the motivation of the work. The authors highlight the ethical concerns in LLM data acquisition, for example, the Sama company, Kenyan workers, and the OpenAI case ([11,12]). Accordingly to the theoretical framework, the sellers are actually companies like Sama, and not the Kenyan annotators. Additionally, the authors mention that the sellers would stop getting involved in the data market if not fairly treated, but as it was in this case, they simply move to new sellers because there is always someone willing to. So entry and exit of data buyers and sellers in the market should ideally be covered.\n  - There might be a decrease in data quality at time t, but a rise in quality at time t+x due to unfair data market or other factors outside of this. The model doesn't account for the changes in data quality at various time steps or for the entry and exit of different sellers and buyers in the data market, due to exploitative pricing and other factors. \n\n- Sections 3.1 and 3.2 and Appendix A, B and C\n  - I found Figure 1 confusing and unintuitive. For example, the data valuation methods require a model performance to compute the value of data, which is not clear from the figure. The relationship between the seller and buyer is oversimplified. For example, from the follow-up text, it seems to me there might be a back and forth between the two, sellers need potentially broadcasted buyers decision-making, and buyers need the data to compute value and set the price - to make decisions.\n  - The model of the seller's decision-making (Eqs. 3 and 4) has strong assumptions. For example, a lay seller, e.g., an annotator, might not have access to the seller utility or other data sources. As such, computing a net profit function is not trivial, and sellers might over- or underprice their data. Also how do we know the fixed price (line 158) is fair? Can buyers only get a fraction of seller data (Section 3.1 and Appendix B)?\n  - In lines 141 to 144, the authors show that utility (not the usual language of utility in dval literature) is a function of data values. It's unclear to me how well or accurately this formulation of utility captures gains such as commercial returns that buyers care about (line 196).   - The utility formulation has other limitations. In line 674, how high is high?  What's the threshold for determining that? All the utility formulations (C.1 to C.3) rely on marginal contribution-based data valuation methods defined in lines 141. I imagine it could be possible to combine various forms of valuating data to determine the utility.\n  - The setup for the results in Figure 3 is oversimplified. For example, why is utility u constant?\n\n- Experimental setups and analyses \n  - Given that authors consider varied tasks, why do they only use the affine mappings and not test the other formulations of utility? (c.2, c.3, c.4)? \n  - There is little justification or reasoning given for the choice of various parameters, e.g., how did the authors choose \\delta and T?\n\n- Limitations or discussions section \n  - Given that the authors make so many assumptions and the framework could be understood from various viewpoints, I think the paper should have a limitations section or discussions to include some of these perspectives and assumptions.", "was_revised": true, "final_justification": "The authors have addressed my concerns and those of other reviewers. Despite some weaknesses, the paper presents a novel and scalable idea, supported by theoretical analysis and empirical results. I have therefore decided to slightly increase my score, with the expectation that the authors will incorporate the revisions they have committed to in the rebuttals, such as improving the introduction, refining Section 3, and adding a more comprehensive limitations section.", "rebuttal": "We thank Reviewer AWaS for their detailed and thoughtful review. We address these points below.\n\n**1. Mismatch Between Motivation and Framework:**\n \nWe thank the reviewer for this insightful observation. It is absolutely correct that the real-world data supply chain often involves a three-party structure: Buyer, Intermediary, and Supplier. Our choice to model a two-party (Buyer-Seller) dynamic is a deliberate one, intended as a crucial first step. Importantly, this model directly represents many real-world markets, such as major crowdsourcing platforms (e.g., Amazon MTurk and Prolific), where buyers interact directly with sellers (annotators). \n\nOur “seller” is a tractable abstraction of the data supply side. The exploitative dynamic we model is often amplified in a three-party reality with an intermediary, a phenomenon explained by the economic literature on intermediation [1,3]. Specifically, the concept of \"double marginalization\" [2,3] shows how an intermediary's need to secure its own profit margin creates intense downward pressure on annotator wages to satisfy a price-sensitive buyer.\n\nThe fundamental conclusion of our work (i.e., under-compensation harms long-term data quality) remains the central tension in both two-party and three-party settings. Our current framework is valuable because it is a tractable model that also directly represents a large fraction of the data market, such as major crowdsourcing platforms. Modeling the additional strategic layer of an intermediary is a crucial and complex issue that demands future research, and we hope our work provides a clear foundation for these investigations.   \n\nOur two-party model serves as a tractable and powerful abstraction that isolates this core economic conflict. Formally modeling a three-party game, while a fascinating direction for future work, would introduce significant additional complexities (e.g., contract design, monitoring costs, three-way information asymmetry) that would obscure the foundational insight we aim to establish. We believe it is standard and valuable to first understand the core dyad before modeling the full supply chain.\n\nWe will revise the introduction and Section 3 to explicitly acknowledge the three-party structure of the real-world market. We will state that our two-party model is a deliberate abstraction and will briefly introduce the concept of \"double marginalization\" to argue that our model's core conclusions are likely amplified, not invalidated, in this more complex setting. We will also add a formal three-party game as a key direction for future work in our conclusion.\n\n**2. Exit of Buyers/Sellers:**\n\nThese are excellent points. Even when new sellers can enter the market, the core principle of our work remains the same: underpayment is a self-defeating strategy. This is because, in a transparent market, an exploitative buyer would develop a poor reputation, deterring new, high-quality sellers and leading to a \"lemons market\" of suppliers. Furthermore, constantly replacing exiting sellers is not a sustainable strategy but a high-cost, inefficient cycle of churn [4]. While formally modeling these entry/exit and reputation dynamics is a key direction for future work, we will add this discussion to our limitations section to scope our contribution and highlight this as an important avenue for research. \n\n**3. Change of Data Quality:**\n\nWe thank the reviewer for these insightful questions. Our framework is intentionally designed to be agnostic to the specific form of the utility function. Thus, it can readily accommodate these complexities, which represent key directions for future work. Fluctuating data quality can be directly modeled by treating the buyer's utility as a stochastic variable; our core conclusions still hold by simply using the expected utility in the decision-making process. We chose a deterministic setup in the paper for clarity and to crisply isolate the core strategic trade-offs, but the framework's ready extension to stochastic cases demonstrates its robustness.\n\n**4. On Unclear Figures and Simplified Setups:**\n\nWe thank the reviewer for this helpful feedback.  **Figure 1:** We will revise it and its caption to better illustrate the role of data valuation and the sequential buyer-seller interaction. **Figure 3:** This is a simplified illustration to clearly visualize our theoretical result in Lemma 3. We used a fixed utility to isolate the long-term strategic trade-off from stochastic noise.  In Section 4, our main experiments adopt stochastic utilities. We will revise Figure 3 to make its illustrative purpose explicit.\n\n**5. Complete Information Assumption:**\n\nWe thank the reviewer for raising this important point. Our choice to model a transparent market, where a platform provides public data valuation scores. This operationalizes information transparency and makes the problem tractable. Our choice to model this setting first was deliberate for three reasons:\n\n1. It serves as an essential first step to isolate the core economic interaction between buyers and sellers before introducing the complexities of asymmetric information.\n\n2. It follows the established precedent of seminal works in economics such as those by Akerlof [4] and Spence [5], which first established key insights in simpler settings. And follow-ups build the pipeline throughout more complex settings.\n\n3. Modeling incomplete information is a vast future research direction, and our work provides the critical first step.\n\nThis assumption also aligns with behavioral fairness models that highlight transparency’s role in sustaining cooperation [4,5,7]. We will further clarify in Section 3 that the seller's problem is made tractable by public valuation signals and emphasize how this benchmark lays the groundwork for future study of incomplete information settings. \n\n**6. Utility Function Mapping:**\n\nWe thank the reviewer for excellent questions about utility. Our framework was designed with generality and modularity in mind, allowing it to be adapted to diverse real-world scenarios.\n\n**Utility and Commercial Returns:** We use “utility” in its classic economic sense as a flexible measure of value [6,7]. Our framework is modular, separating the valuation score $v_k$ (technical impact) from a buyer-defined utility mapping $u_k(v_k)$. This allows practitioners to translate technical impact (i.e., model performance) into firm-specific value, such as commercial returns. This allows our theoretical framework to remain general, while letting practitioners plug in their own specific goals. \n\n**Thresholds:** The specific mappings in Appendix C are illustrative of this flexibility. Defining empirically-grounded thresholds is a firm-specific implementation detail that our framework is designed to accommodate, leaving this as a direction for future applied work with empirical data.\n\n**Combining Valuation Methods:** Absolutely agree. The framework is agnostic to how the valuation score $v_k$ is derived. A buyer can use a composite score (e.g., an average or minimum of various methods) for more robust signals, and our conclusions still hold.\n\nWe will revise Section 3 to clarify that the utility mapping is a modular, buyer-defined component, highlighting the generality of our framework.\n\n**8. On Experimental Setups and Analyses:**\n\nWe thank the reviewer for this excellent question. \n\n**Other utility mappings:** In our main experiments, we used the affine mapping (Appendix C.1) primarily for its clarity and direct interpretability. Our primary reason we did not include experiments on all mappings from Appendix C is that our framework is designed to be method-agnostic. The conclusions are expected to hold for other formulations. \n\nTo empirically validate this, we run new experiments using the Discrete Outcome mapping (Appendix C.2). The results are highly consistent with those in the main paper: Fairshare pricing continues to maximize long-term utility for buyers and profits for sellers. This confirms that our central conclusions are robust to the specific formulation of utility. \n\n**Cumulative Buyer's Utility (Discrete Mapping)**\n| Time Steps | Fairshare | Discount | Random | Exploitative |\n|---|---|---|---|---|\n| 20 | 51.11 | 115.66 | 90.98 | 72.87 |\n| 60 | 146.70 | 155.62 | 124.75 | 87.08 |\n| 100 | 236.49 | 162.67 | 134.55 | 99.53 |\n\n**Choice of $\\delta$ and $T$:** Our selections are grounded in standard practice and validated within our paper. \n\n$\\delta$: Following standard approaches in dynamic programming and economics [8,9], we set $\\delta=0.98$ to ensure future outcomes are properly considered and not overly discounted. This aligns with our Assumption 2, reflecting forward-looking buyers.\n\n$T$: 100 steps adequately capture the long-term market dynamics central to our analysis, allowing stable patterns and equilibrium outcomes to emerge.\n\nWe also thoroughly analyze the impact of these parameters. In Lemma 7 (line 791), we theoretically prove the trade-off, showing that a less patient buyer (a smaller $\\delta$) requires a longer time to converge to the optimal fairshare strategy. To complement this, we empirically demonstrate this exact relationship in our robustness check in Appendix E.2.\n\n**10. Limitations and discussions:**\n\nWe thank the reviewer for this important suggestion. While our current limitations section in Conclusion and Appendix G already encourages future work on incomplete information settings, we will add a more thorough discussion in the next version.\n***\n[1]Coase.The Nature of the Firm. \n\n[2]Spengler.Vertical Integration and Antitrust Policy. \n\n[3]Tirole.The Theory of Industrial Organization. \n\n[4]Akerlof.The market for “lemons”: Quality uncertainty and the market mechanism. \n\n[5]Spence.Job market signaling.\n\n[6]Mas-Colell et al.Microeconomic theory.\n\n[7]Neumann et al.Theory of games and economic behavior.\n\n[8]Sutton et al.Reinforcement Learning: An Introduction.\n\n[9]Stokey et al.Recursive Methods in Economic Dynamics."}, {"review_id": "C0nspSCMaD", "reviewer": "Reviewer_nL1m", "summary": "This paper proposes a theoretical framework to model the LLM data market, where buyers and sellers make decisions with transparent information. Theoretical and simulation results show that exploitative pricing leads to a lose-lose situation where data sellers exit the market, leaving buyers with data shortage. Furthermore, the authors propose a fairshare pricing mechanism that theoretically and empirically lead to a win-win situation, where data sellers obtain optimal profits and the buyers obtain long-run utility. Finally, across three LLMs, four tasks, and two budgets, the proposed fairshare pricing achieves the win-win situation as the theory suggests. These empirical results are consistent across several data valuation methods that are scalable to LLMs.", "questions": "- Intuitively, Assumptions 1 and 1.1 suggest that the sellers are quite likely to exit the market once they are exploited. Can the authors comment on how realistic this is in the real world? What if the lower bound in Assumption 1.1 is replaced with an upper bound, and $P_T$ in Assumption 1 is not multiplicative, suggesting that sellers are more tolerant to exploitation?\n- In Figure 2(c), at Buyer 1's MWP, why is the profit (~2.0) larger than the price (<2.0)?\n- This is a minor comment. Should $R_{\\ge 0}$ in line 139 be replaced by $R$ for the range of the data valuation function?", "limitations": "The limitations seem adequately addressed.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- This work formalizes the LLM data market with a theoretical framework.\n- This work proposes a fairshare pricing mechanism based on the theoretical framework. This can be impactful for policymaking and the fairness domain.\n- The lose-lose outcome from exploitative pricing and the win-win outcome of the proposed fairshare pricing are well supported with theoretical and simulation results, as well as empirical results with actual LLM fine-tuning.\n\n**Weaknesses**\n- The seller-side pricing under fairshare pricing (Section 3.3) does not seem to take the cost $c_j$ into consideration, which is inconsistent with the overall market framework. In particular, $p^*_j$ may be outside $\\cup_k MWP_k$ if $c_j > MWP_k$ for all $k = 1, ..., M$. I think a minor assumption on $c_j$ is needed for Lemma 2 to hold. Intuitively, $c_j$ should not be larger than a buyer's budget and marginal utility.\n- Overall, Assumptions 1, 1.1, and 2 are reasonable. However, the authors should comment on how their results would change if these assumptions are broken.", "was_revised": true, "final_justification": "I keep the initially recommended score of 5. My primary concerns were about (1) a lack of sufficient assumptions for Lemma 2 and (2) a lack of discussion about what happens when the assumptions made in the paper break. The authors have addressed my concerns with their rebuttal. While I agree with Reviewer 9cVB that the assumed seller behavior can be unrealistic, I view this paper as a reasonable first step toward formalizing data pricing in the LLM data market and initiating formal discussion.", "rebuttal": "We thank Reviewer nL1m for their highly constructive feedback and insightful technical questions. The suggestions and feedback by Reviewer nL1m are extremely valuable for this paper. We appreciate the opportunity to clarify these points and address them below. \n\n**1. On the Role of Cost cj​ in Fairshare Pricing (Lemma 2):** \n\nWe thank the reviewer for this sharp and accurate observation. The reviewer is correct that our characterization of the optimal price in Lemma 2 implicitly assumes that a profitable transaction is possible for the seller (i.e., $p_{j}^{*} \\geq c_{j}$). We will make this assumption explicit in Section 3.3 to improve the rigor of our claims. We appreciate the reviewer helping us strengthen our presentation.\n\nWe would also like to briefly clarify the justification for this assumption. It is grounded in the current dynamics of the LLM data market. Companies are making significant investments to acquire high-quality training data, as its value is a critical driver of model performance. On the other hand, the labor cost to produce this data is often exploitatively low [5-8]. This makes it a reasonable assumption for our model that the optimal price a buyer is willing to pay for valuable data ($p_{j}^{*}$) generally exceeds its annotation and production cost ($c_{j}$).\n\n**2. On the Justification of Assumptions 1, 1.1:**\n\nWe thank the reviewer for this excellent and nuanced question, which gets to the heart of our behavioral model. Our assumption that sellers are sensitive to exploitation is not just an intuition but is strongly grounded in both classical economic theory and direct evidence from the data annotation market.\n\nFirst, our model aligns with foundational studies in organizational justice and fairness theory, which we cite in our Related Work section. This body of literature shows that worker retention is driven by perceived fairness [1-4], not just absolute payment. When compensation is seen as disconnected from the value created – a core feature of exploitative pricing – it erodes trust and motivation, reducing the willingness of skilled participants to remain in a market. This principle has been documented specifically in the \"ghost work\" economy, where precarious conditions and low pay lead to high worker churn and disengagement. This has been highlighted in our Related Work (line 104-105).\n\nSecond, this long-standing theoretical principle is being validated in real-time in the current LLM data market. Recent investigations by outlets like WIRED [5] and The Washington Post [6] describe the industry as experiencing high worker churn due to persistently low and inconsistent pay. More directly, widespread reports from early 2024 revealed that when platforms like Scale AI's Remotasks abruptly reduced pay rates, it led to an immediate exodus of skilled annotators [7]. Further, a significant 2025 investment in Scale AI by Meta has raised concerns about gig workers being left behind, underscoring ongoing exploitation concerns [8]. These reports provide direct, contemporary evidence for our model's core assumption: exploitative pay strategies lead to high dropout rates among the very workers needed for high-quality data. We will incorporate these contemporary reports into our revised manuscript to further strengthen the real-world grounding for our model's behavioral assumptions.\n\n**3. On the Violation of Assumptions 1, 1.1, and 2:**\n\nWe thank the reviewer for this excellent suggestion. It is important to consider the boundaries of our model. If they were violated, the dynamics would change as follows:\n\n**If Assumptions 1 and 1.1 are broken:** We thank the reviewer for pointing this out and fully agree on the reviewer’s intuition. This implies that sellers are not sensitive to exploitative pricing and are highly tolerant of being underpaid. In this case, the primary long-term penalty for the buyer (i.e., losing access to high-quality data suppliers) is removed.\n\n**If Assumption 2 is broken:** This means the buyer is myopic and short-sighted, caring only about maximizing their immediate surplus ($u_t - p_t$) rather than their cumulative long-term utility.\n\nIn a scenario where both sets of assumptions are violated, the incentive structure that upholds our fairshare equilibrium might collapses. With tolerant sellers and a short-sighted buyer, the market would be dominated by exploitative pricing, as the buyer would have no rational incentive to pay more for long-term market sustainability. We will add a discussion to Section 3 analyzing how our results would change if these core assumptions were relaxed, making the scope of our framework clearer.\n\n**5. Clarification of Figure 2**\n\nWe thank the reviewer for this careful observation of Figure 2(c). The price point labeled \"Buyer 1's MWP\" is the highest price at which both buyers are willing to purchase the dataset. Therefore, the seller's total profit at this point is approximately (2 * price) - cost, which is why the profit value (~2.0) can be larger than the price itself (<2.0). If the seller were to increase the price beyond this point, Buyer 1 would drop out, leading to a lower total profit. We will revise the caption of Figure 2 to explicitly state that the simulation involves two buyers and that the plot shows the seller's total profit to ensure this is clear. \n\n**4. The Range of Domain:**\n\nThank you for your careful reading. The reviewer is correct. We will update the notation in line 139 from $R\\geq0$​ to $R$ for full generality. We thank the reviewer again for their supportive assessment and the detailed feedback, which will improve the rigor and clarity of our paper.\n\n***\n\n[1] Robert G Folger, Robert Folger, and Russell Cropanzano. *Organizational justice and human resource management, volume 7.* 1998. \n\n[2] Robert Folger and Russell Cropanzano. *Fairness theory: Justice as accountability.* Advances in organizational justice. 2001. \n\n[3] Jason A Colquitt, Jerald Greenberg, and Cindy P Zapata-Phelan. *What is organizational justice? a historical overview. In Handbook of organizational justice.* Psychology Press, 2013. \n\n[4] Mladen Adamovic. *Organizational justice research: A review, synthesis, and research agenda.* European Management Review. 2023.\n\n[5] Niamh Rowe. *Millions of Workers Are Training AI Models for Pennies.* WIRED. 2023\n\n[6] Rebecca Tan and Regine Cabato. *Behind the AI boom, an army of overseas workers in ‘digital sweatshops’.* The Washington Post.  2023.\n\n[7] Russell Brandom. *Scale AI’s Remotasks platform is dropping whole countries without explanation.* Rest of World. 2024\n\n[8] Billy Perrigo. *Meta’s $15 Billion Scale AI Deal Could Leave Gig Workers Behind.* TIME. 2025."}]

Author rebuttal:

Meta-review: The review team is largely supportive of the paper, with three strong supporters with scores of 5. Reviewers praised both the formalization/applied modelling aspects of the paper when it comes to markets for data involving LLMs, as well as the novel proposed fairshare pricing mechanism. Overall, the contribution is clearly novel, timely, and impactful. 

The one negative reviewer notes that they would like to see the problem studied through the lens of cooperative game theory. I agree with the reviewer that the paper, should, for the camera-ready, discuss the connection to cooperative game theory and explain the relationship to this line of work. However, I do fully agree with the authors' response that this is only one of possibly several possible framings, note that the authors provide compelling justification in their response for their non-cooperative framing, and that this critique is, in my opinion, insufficient grounds for rejection. I will note that the reviewer did not engage with the rest of the review team nor tried to adjust their score based on the authors' response, the other reviews, and the discussion where they were explicitly prompted to further support their disagreement with the rest of the review team. 

As such, I recommend the paper for acceptance as a poster.
Decision: Accept (poster)