Pricing and Competition for Generative AI
Authors: Rafid Mahmood
Venue: NeurIPS 2024 poster (2024)
Keywords: pricing
revenue management
model evaluation
foundation models
Primary area: human-AI_interaction
TL;DR: We propose a framework to characterize how the developers of generative AI models can price their technology in the face of competing alternative products

Abstract: Compared to classical machine learning (ML) models, generative models offer a new usage paradigm where (i) a single model can be used for many different tasks out-of-the-box; (ii) users interact with this model over a series of natural language prompts; and (iii) the model is ideally evaluated on binary user satisfaction with respect to model outputs. Given these characteristics, we explore the problem of how developers of new generative AI software can release and price their technology. We first develop a comparison of two different models for a specific task with respect to user cost-effectiveness. We then model the pricing problem of generative AI software as a game between two different companies who sequentially release their models before users choose their preferred model for each task. Here, the price optimization problem becomes piecewise continuous where the companies must choose a subset of the tasks on which to be cost-effective and forgo revenue for the remaining tasks. In particular, we reveal the value of market information by showing that a company who deploys later after knowing their competitor’s price can always secure cost-effectiveness on at least one task, whereas the company who is the first-to-market must price their model in a way that incentivizes higher prices from the latecomer in order to gain revenue. Most importantly, we find that if the different tasks are sufficiently similar, the first-to-market model may become cost-ineffective on all tasks regardless of how this technology is priced.

Pricing and Competition for Generative AI
Rafid Mahmood
NVIDIA University of Outawa
rmahmoodeénvddaa.com
Abstract
Compared to classical machine learning (ML) models, generative models offer a usage paradigm where (i) single model can be used for many different tasks out-of-the-box: (ii) users interact with this model over a series natural language prompts; and (iii) the model is ideally evaluated on binary user satisfaction with respect to model outputs. Given these characteristics, we explore the problem of how developers of new generative AI software can release and price their technol- ogy. We first develop comparison of two different models for a specific task with respect to user cost-effictivveess. We then model the pricing problem of generative AI software as game between two different companies who sequentially release their models before users choose their preferred model for each task. Here, the price optimization problem becomes piecewise continuous where the companies must choose subset of the tasks on which be cost-effective and forgo revenue for the remaining tasks. In particular, we reveal the value of market information by showing that a company who deploys later after knowing their competitor's price can always secure cost-effectiveness who the first-to-market must price their model a way that incentivizes higher prices from the latecomer in order to gain revenue. Most importantly, we find that if the different tasks are sufficiently similar, the trst-to-marke model may become cost-ineffective on all tasks regardless of how this technology is priced.
at least one task, whereas the company
Introduction
The recent explosion of generative artificial intelligence (AI) has introduced new machine learning (ML) frameworks for applications from chatbots robotics [Wu et al., 2023 Nasiriany et al., 2024 Whereas in classical ML, a user interacted with a single model designed for a specific predictive task (e.g., classification) via input data and output predictions, a single generative AI model can solve variety of tasks for a user out-of-the-boo [Brown et al. 2020). Moreover, users interact with the generative model over universal interface of natural language prompting [Arora et al., 2022].
The prompt-based paradigm has fostered two recent human-Al interaction trends. First, prompting facilitates such wide distribution of tasks (i.e., user inputs and model outputs) that conventional metrics for evaluating models have become insufficient, leaving the most effective evaluation metric be a binary score of whether the user is satisfied with the model output (Li et al. 2024 Chiang et 2024). For example. Ziegler et al. (2024) empirically analyzed the GitHub Copilot software to reveal that the frequency generated code approved by user 'is a better predictor of perceived [user] productivity than alternative measures." Second, if user does not receive satisfactory output, they can try again another prompting round by inputting to the model additional information [Castro et al. 2023]. For instance, the Anthropic HH and the Chatbot Arena datasets report on average 2.3 and 1.3 prompting rounds per conversation, respectively [Bai et al.) 2022 Chiang et al. 2024).
In this work, we study the impact of these interaction characteristics on the pricing of generative AI technology. While classical ML products can be priced by analyzing the user demand for model that can achieve given performance metric on a specific task (urran and de Véricourt) 2022 Mahmood
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

companies build generative models for different tasks
First prices deploys their mode: before the latecomer's actions
Second company coploys their model given the first-to-market's actions
Users chaose which models
Figure 1: Overview of the competitive pricing problem for generative Al models.
al., 2022]. generative Al model is priced per user prompt | This set price determines the user cost for multiple different tasks and variable number of prompting rounds, e.g., the cost of using GPT-4 for math reasoning or code generation depends only on the per-token price, and the length and number of prompts. Thus, developers of generative Al product must factor the demand for all potential use-case tasks of the technology when setting a price. This pricing problem becomes further challenging when considering the rapidly growing marketplace of competing generative Al models, since companies must also ensure that their products do not become unattractive to users as soon as competitor develops a newer and better model.
We first characterize when, for given task, a user will prefer one generative Al model versus another. We argue that users minimize their total cost, measured by the cost-per-prompl times the number of prompting rounds needed for the model to produce a satisfactory output; this leads to a comparison of price- perffrmner competitiveness between AI models. We then study game with two firms developing competing models used for set of tasks. Both firms know each other's model's performance the tasks. The first firm deploys their product and sets a price, followed by the second firm with their product and price. Finally, a user decides which models to use for each task. Both firms seek to maximize revenue, but the first firm acts without knowledge of their competitor's 's price. Figure summarizes the problem setting and insights. Our key observations include:
1.
The pricing problem reduces to piecewise optimization problem, where firms price their model to be competitive on a subset of the tasks while forgoing revenue from the others. This subset can be determined by ranking the tasks the competitive ratio between the two models for each task and selecting the most competitive tasks.
2. A firm who deploys late always obtains revenue from at least one task by leveraging the available market information. In contrast, the their prices to encourage the latecomer to set higher prices and focus on fewer tasks.
must strategically set
3. Under certain conditions on model performance and user demand, the frst-to-marrde may acquire zero revenue regardless of their price. In these settings, the latecomer naturally maximizes their revenue by being competitive for all tasks. Thus, developers that are first should have minimum model performance before deploying their product.
Related literature
Evaluating ML models. ML models are typically evaluated on generalization error for a task via out-of-sample test dataset benchmarks and competitions (Deng et al. 2009]. Generative Al and large language models (LLMMs) are evaluated on a suite benchmark tasks such for coding [Chen] et al. 202T], math [Cobbe et al. 2021], ], and problem solving [Hendrycks et al.
2021 However,
standardized benchmarks become uninformative over time as new models are trained to overfit to these metrics [Roelofs et al. 2019 Koch et al. [221]) Recently, Chiang et al. [0244 introduced Chatbot Arena for comparing LLMS head-to-head on human preference. Here, a user poses real-world prompt which is input to two models. The user reviews both model outputs and can even continue multiple conversation rounds, before ranking which model generated a satisfactory answer first. Although difficult to measure, informative metric as seen from generated code approvals on GitHub Copilot (Ziegler et al. 2024), perceived aesthetic quality of text-to-image generation such as MidJourney and Playground [Li et al. 2024). Our work combines user satisfaction with a user cost to construct a price-performance ratio for comparing different models.
satisfaction rate is increasingly viewed the most
In practice, generative Al models are typically siccedper-token. In Appendix B we show that all our results extend to the price-per- token setting with minor change of variables. For list of prices for current generative Al prices, see: fttaps://docsot... ai/tools/ wtt.tool//gggoooooaaaaeeeeeee-ee api- -pp-piiiini----iinnn cra.-cccccnnncaccclattrrrrr calculator

---

Human-genen -enerativee Al interaction. Prompt-based interaction has increased the diversity of tasks where these models can be applied (Eloundou et al. 2023]. Castro al. [023] analyze how and
when human users will use a generative Al model for a task versus performing it manually, as well as the characteristics of interacting over multiple prompt rounds. The quality of prompts crucial
to generating higher-qualit answers [Liu et al. 2023 Binz and Schulz, 2023). This has motivated studies on prompt techniques, such as chain-of tfhthughhtt (Wii et al. [022] and self-consistency [Wang] et al., 2022] In our work, we treat prompt quality as random variable and to simplify the structural analysis, assume that users will interact with generative Al model for as many prompting rounds as
needed to get satisfactory answer.
Pricing and competition. Duopolies of competitive products use game theoretic models such as the Bertrand (i.e., simultaneous pricing) and Stackelberg (i.e., sequential pricing) models of interaction [Gibbons] 1992]. Both deterministic and probabilistic demand can be used to study oligopolistic pricing of single or multiple products Chintagunta and Rao (199). Gallego and Hu [2014]. Specific structured demand frameworks allow for identifying market equilibria and failure settings where revenue is unobtainable [Federgruen and Hu 2015 2019). Our work is most closely related to the Stackelberg literature by modeling a sequential game and exploring the conditions under exponential demand that make certain generative Al models unattractive [Hamilton and Slutsky 1990].
Pricing Al products. Al technology can be priced various levels, ranging from training data to
model queries [Liu et al.] [202 Chen et al.] [019] Cong et al., [022]. A core aspect of the pricing problem involves valuating the ML model based on performance [Xu et al. 2024]. ML products are further susceptible to an Al flywheel effect where the release and price of an Al product will affect the subsequent collection of new training data from users, leading to a dynamic pricing problem [Gurkan and de Véricourt 2022, Chen and Xue 2023 More generally, novel technology products such as a new generative Al model with emergent use-cases may feature social-learning and dynamically growing market sizes [ellmaa et al.] 2019 Zhang et al., 2022] The closest to our work is Gurkan and de Véricourt [2022] who explore precing and contracting the development of classical ML model under the AT flywheel. In contrast, our work explores competition between ML model developers when faced with a diverse set of potential downstream tasks for which the model can be used.
3
Main model
We first define the characteristics of the pricing problem. We then propose model of user choice between two competing models from a price-performance perspective.
3.1 Problem setup
Tasks. We define a task as set of independent problem instances where for each instance, a user queries machine learning model with an input prompt and receives output generated the model. For example, a programming task may have instances where a user inputs a commented function definition and the model must complete the code to perform the function Chen et al. [2021] Ziegler ct al. 2224). Task instances are evaluated by a user via a binary correctness score. For tasks where correctness is unambiguous (c.g., whether the program runs), the score is equivalent to accuracy, whereas for open-ended tasks (e.g., whether the output meets the stylistic preferences of the user), we treat correctness simply as whether the user is satisfied with the output
Generative AI model. Given a set of T different tasks, a generative model an ML model that can be used to solve instances of any of the different tasks via prompts. We define this model tuple (p, Vi, V/,..., Vr) where p denotes the price for using the model, as measured dollars-per poolars-per-prompl (see Appendix B for the extension to pricing per-token), and for each / E [T'. V, E (0, 1) denotes the average score of the model over instances of each task. We assume that V - 1 implies that the model can always generate correct output for any task instance and V2- + 0 implics that the model will always generate an incorrect output. Therefore, for any random instance of task t, V; can also be interpreted as Bernoulli probability of the generative model producing a satisfactory output in single attempt.
For example in code completion. users prefer generated code that provides a good starting point for the users improve rather than code that is technically correct but confusing [Zigllr et al. 2024
3

---

Users will not use the generative model if it's price is too high with respect to the user's valuation of the specific task. For any given task t. let (p) E R, be the demand function, i.e., the number of users who will use a generative Al model for task l. function of the price p. Following standard assumptions, we assume that (,(p) differentiable and non-increasing in the model price, as well as being known to the developer of the Al model [Gallego and Van Ryzin 1994].
Multi-round use. Most ML benchmarks typically evaluate models on whether the models can generate the correct output under a single prompt round [Chen el al. 2021 Cobbe et al.) 2021 Hendrycks al. 2021] However, mn-the-wild users of generative models typically have interactive multi-round conversations where if the model generates an unsatisfactory solution after the first prompt, the user can provide feedback via their preferences or corrections in a second prompt round [Castro et al. [023 Liao et al.) 2024). For example in code completion, if the model fails to account for an edge-case input to the function, the user can identify the edge-case and ask the model to account for it. To characterize this multi-round use, we extend the single prompt to sequence of Bernoulli trials that continue until the user is satisfied with the model output. For simplicity of modeling, we make the following assumptions on user behavior.
Assumption 1. The total number of prompting rounds nu (v)))) that a user will engage with the model: (i) has a finite mean; and (ii) is independent of the model price p conditioned on the user knowing Vi.
Assumption | implies that the price of a model only determines demand via whether the model used at all, rather than how many times (i.e., prompting rounds) the model is used. Under this assumption, there are many choices for modeling the distribution of (V.). We give three examples:
Geometric: Because non-expen users tend to design uninformative prompts [amfireseu Pereira al. 2023], we may suppose the probability of success on any round does not depend on user input, and each round is an i.i.d. Bernoulli trial with probability Ve. Then, the total number of rounds is n4 Geom (V)), following Geometric distribution.
Truncated Geometric: Users may quit the model after maximum Tj rounds if it fails to generate a satisfactory response [Castro et al.] 2023). For instance, conversations in the Chatbot Arena dataset terminate after an average 1.3 rounds (Chiang et al., 2024]. Here, Prin-- = n} := (1 11---) V/3== VV for S n < T; and Pr(n, = T=}
Prompt-dependent: Suppose the success probability prompt-dependent V1(2a)) where ~ Pr(x) is the prompt on the i-th round. If users prompt until the model generates satisfactory answer, we have Pr(n,- = == E,,,,., 2n (V. II-- (1 V,(2;))).
Ultimately, the choice of characterizing na depends on the information available to the generative AI model provider. With limited information, the Geometric assumption may be most practical, but given knowledge of user prompts, we may consider more sophisticated models. Our results all hold independent of the distribution long as Assumption i satisfied. For ease of notation and interpretability, we assume n, ~ Geom (V.) in the remainder of this work.
3.2 Modeling user preference between AI models
Given the above task and user behavior framework, we analyze the problem of user who must choose between two competing generative models to solve instances of their tasks. Since user may prefer different models for different tasks, we consider single task and omit subscript
Consider two generative models: model A (q, W) and model B (p. V). Under Assumption I given a sufficient number of prompt rounds, both models eventually solve every task instance. Thus, rational user will seek to minimize the expected cost of completing a task instance, measured by the average price-per-prompt times the expected number of rounds required to complete the task instance. In this comparison, model B will incur a lower cost for the user if
pE[n(V)] < (W(n(W)] =
P
<
9
V
W
Otherwise, model A incurs lower user costs. We assume that model B is preferred in ties. Note that the implication follows from our Geometric assumption of n(VV).
Condition states that for any task, a specific generative Al model is preferred if the price- performance ratio for this task (i.c., the cost of using this model over the model's performance) is

---

lower than any other competing generative model. For example, if model B is twice as likely to generate user-satisfactory output as model A for given input prompt, i.e., V 2W. then the user will prefer model B as long the cost of prompting this model is not twice as high, i.e., P S 2g. Otherwise, the user will incur lower costs and still obtain their desired outputs by simply prompting the weaker model for twice as many rounds.
We note that (I) can also compare the use of a generative Al model versus manually performing the task (Castro et al. 2023 For instance, if we treat model B as a human and a prompt round as a timed attempt at completing task (e.g., coding the function within one hour), then V the probability of human being able to perform the task in the single attempt and is the value of this labor.
Finally, this framework can also compare free-to-use generative AI models such LLaMA Touvron et al. 22023. Although there may not be a given price-per-token for using these models, there is fixed cost to set up the infrastructure and environment. Given an expected total number of task instances, this fixed cost can be approximated to an equivalent
4 Pricing generative AI models
We now develop general framework under which a provider of a generative AI model can price their product. We first define the pricing problem as a game between two firms developing competing generative AI models. We then create tractable reformulations for these problems for both firms, representing pricing with and without considering competition.
Pricing problem. Consider a set of T tasks. Let (g, WI, W2...., WT) and (p, V1, V2,..., VT) be the generative models released by two competing firms, A and B, respectively. Firm A deploys their generative model (i.e., model A) first and sets the price q for this product. Then, firm deploys their competing model (i.e., model B) and sets their price p. After both firms deploy their models, user with demand functions (-).) for each will decide which models to use as determined by (I). We evaluate the total revenue obtained by each firm, given by RA (a(g))) and (B(pl4) respectively:
T RA(glp) = Σ D((1)
T Rn(plg): = P D((D) D.(DI
q W.
P V.
P V.
<
(2)
W.
Note that in practice, the user demand may depend on and q simultaneously; for simplicity, we assume the demand for a specific model to be determined only after the user preference condition ① is resolved. The results in this section easily generalize more complex demand functions. The revenue functions are composed of the demand for each task times the price set by the firm (Van Ryzin and Talluri 2005), summed over all tasks for which the firm's 's model is competitive according to the price-performance ratios. Each firm's 's objective is to maximize their revenue. However, because firm A is the first-te tor-t----rkeet and they do not know the action that firm B will take; instead, firm A must optimize for the worst-case scenario as determined by firm B. On the other hand, firm B first observes the price set by firm A. Thus, the two firms prices as follows:
q- := arg max (RA(gl))
arg max R (plg))
p" arg max {RB(plg"))
(3)
920
We assume both firms know V/ and We for all t E [T]. This is motivated by the availability of research papers and reported benchmark scores, and the predictability of model performance via power laws [Kaplan et al., 2020). In practice, firm may not know their competitor's performance, but they can forecast the state-of-the-art score in the short term.
4.1 Pricing in isolation
We first consider firm B's problem, who set their price after firm has already released competing model with price q² Firm B's problem depends on the competitive ratio Ne == E[n (W2) ///////E/(/// [V)) for task E (T), i.e., the relative ratio number of prompting rounds between the two firms, which under Geometric distribution assumption, simplifies to Ke = V./W, To maximize their revenue, firm can rank the tasks in order of 6s, select a subset of tasks in this order, and solve the optimization problem for each subset. This results in overall piecewise optimization problem.
5

---

Demand tasks
Revenue for tasks
400
Task
300
K34
kgg
K3q
Task
Task
Task
350
Task
250
Task
Demand (10k users)
300
Revenue ($10k/month)
Total Revenue
250
150
150
100
100
50
50
%
2
8
10
10
Price (50.01/prompt)
Price (50.01/promptt
Figure 2: (Leff) Three tasks with three different exponential demand functions D1 (p) 100e 0.5p (2(p) = 0055, D3(p) = 400e- (Right) The corresponding revenue from each task along with the total revenue function for firm (p). The vertical lines correspond to Kig, k2g, and 63g- where K1 > R2 > K3. For < K3g. revenue is obtained from all three tasks, for p (#aq, k2g.. revenue is obtained from only the first two tasks, and for E (Nog, Kig]. revenue is only obtained from the first task. No revenue can obtained if > kiq.
Theorem 1. Consider the following ordering a [T+ (7 + for which
Ba(1) > (a(2) > ... (T > (o(T+1) :=
(4)
Then, firm B's pricing problem is equivalent to the piecewise optimization problem:
(E(T) P (DDoo()0
P Kol()) 2 (o(1+))
max max
(5)
Theorem shows that a generative Al model should be priced by prioritizing a subset of the tasks and making the model price-performanee ignores the remaining tasks for which the model has low competitive ratios K2, since they would need extremely low prices in order satisfy (T) for these tasks. This strategy is due to the observation that for any task (t), if model satisfies (I). then the model will also satisfy the condition for all (t') for t' < Furthermore, Theorem holds without loss of generality with respect the strict inequalities on 4 since tasks with the same competitive ratios can be grouped together by summing the constituent demand functions. Finally, Theorem [ holds for arbitrary demand functions. In practice, the demand functions are known and typically have parametric structure. Figure gives givees an example using three tasks with demand that decays exponentially with price.
competitive for those tasks only. Consequently, the firm
Theorem D reveals two key implications on how firm can price generative mode when given competitor information. First, pricing reduces to solving T optimization problems, where each of these problems are of a single variable with a differentiable objective and boundary constraints. Thus, the inner problems can be solved via gradient descent. Second, long firm B sets price < (e(1), they will obtain some non-zero revenue, i.c., problem (5) always has a feasible solution. This advantage is due the fact that firm sets their price given fixed q.
4.2 Pricing when accounting for competition
We next consider firm A's problem of setting problem while assuming that firm will act optimally next. This pricing problem is bi-level optimization problem, but it can be solved by ranking the tasks according the competitive ratios for firm A and prioritizing subset of these tasks.

---

Theorem 2. Firm A's pricing problem is equivalent to the piecewise bi-level optimization problem:
T
Σ (a(s)(9)
max 1
max 920
a=t+1
t.
Σ
(o(s)())
p Ba(()
Na(t+1))
(6)
P
9
1¹ 2 11)
max P
(o's)(p') Ro((!)
AA
q
Theorem relies on the observation that the reverse order of (.)) in en rank the most to least competitive tasks for firm A. For any t, q < pk then model A is price-performance competitive for all tasks (l)..... ... ( and firm A will acquire revenue from all these tasks.
Firm A may not always be able to obtain revenue, Problem (6 is infeasible if for every t [T' 1]. there is no 2 0 that satisfies the bi-level constraint. This infcasibility implies for any q 2 0, firm B always maximizes their revenue by setting a low price p S Ko(ryy. Thus, the key motivation of firm A is that the firm benefits only when their competitor is incentivized to set high prices.
5 Structural analysis under exponential demand
Demand is typically modeled via structured parametric functions (Van Ryzin and Talluri [2005]. In this section, we specialize the pricing problem to the standard choice of exponentially decaying parametric demand to extend the previous general results. See (igure(](Left) for an example.
Assumption 2. For each task t, the demand function decays exponentially in price De(p) a, cxp(-bp), where ar > 0 the zero-price base demand and b > is the price-sensitivity of users. Furthermore, all tasks have the same price-sensitivty.
Under the exponential demand model, the demand for each task is equal to d, when p = 0, and decays at rate bb. We assume that the decay is the same for each task; this is motivated by the practical consideration that the different tasks should have relatively similar "user value' to have the same price. If one task uniquely seicessensitive to users, the firm may instead develop a finetuned model for that task or propose incentives such as task-specific discounts to better optimize revenue.
Below, revisit both firm B's and firm A's problems under this demand model to derive globally optimal solutions and structural insights on the market dynamics.
5.1 Pricing in isolation under exponential demand
Under Assumption 2 firm B's problem (5) now simplifies to the maximum of up to T possible values that are the globally optimal solution to each of the individual inner optimization problems.
Theorem 3. Suppose Assumption holds. For any I, let oo(1) = E'= Go(e)- Without loss of
generality, let to E [T] be the task index that satisfies Hatt-) q 2 1/b ko(1+ Then, firm B pricing problem is equivalent the following maximum value:
max
max 15
beace:
} jartese-1)
(7)
Theorem B] states that problem can be solved by by solving each of the inner problems starting from the lowest price range up until we arrive at the zero- -rooogrrdeett solution of the revenue function, i.e., 1/b. Furthermore, for each of the price ranges before this point, the optimal solution is the upper price boundary. Note that if 1/b >> (1) i.e., there does not exist any L" satisfying the condition, then we take the maximum of all T problems. Figure 2 (Right) visualizes the revenue function RB (pl4) for T = 3 tasks with exponential demand. Finally, this structure also reveals that the optimal price is bounded from both above and below via these optima.
Corollary 1. The optimal price for firm B is bounded 1/b 2 p" Ko(1)g.
7

---

Relationship between K2/Kı and a1/(a1 a2)
Ratio of relative performances (K2/kg)
1.0
kg/k3
Kg/K;
-W(3,
0.8
No revenue
0.6
Low price for q
0.4
0.2
High price q
0.0.0
0.2
0.4 0.6
0.8 1.0
Ratio of base demand ay/(a; + az)
Figure 3: The relationship between K2/ K1 and / (aj + a2) for firm A. the blue region, firm B will always set price that is competitive orange region, the maximum price that firm can set is upper bounded (see problem (TT). In the green region, the maximum price that firm A can set has higher upper bound (see problem [10
both tasks and firm A will acquire revenue. In the
5.2 Pricing when accounting for competition under exponential demand
We now revisit firm A's pricing problem, which is bi-level problem with multiple inner optimization constraints. To obtain structural insights on the challenges of pricing under competition, we explore special case with T tasks. Without loss of generality,
assume
K2, i.e., (t) t.
When there are only two tasks, firm will either set their price to be competitive for the first task only (i.e., the task with the higher competitive ratio) for both tasks. In the latter case, model A would unattractive
both tasks and firm A would obtain zero revenue. Therefore, firm A should set their price in such way that they maximize revenue the second task, while ensuring that firm motivated to be competitive only for the first task. Thus, problem (6) simplifies to
max qD2(q)
(8)
s.t. max (p) kig >p> k29 > max (p(Di()) +2(())
K2g pp
If firm A sets price g that is infeasible for this problem, they will get zero revenue. Furthermore, this problem reduces to two single-level optimization problems with only bounding constraints.
Theorem 4. Suppose that 2, that Assumption 2 holds, and withou loss of generality, assume (t) = f. For E R, let W(2) be the Lambert W function defined only over 2 > - If
B2 < -W
(- aj--
(9)
K1
+ +
then, the firm A's pricing problem is equivalent to
ape-!
max
qaze
by
W
2 29>0
(10)
br22
a1
+ a2
Otherwise, A's problem is equivalent to max
g02e
bg
1
K1 log log
01
2 q 0
(11)
b(k1
122)
02
Recall from Theorem the second-level problems in ( be solved analytically by checking the boundary points and zero- pero-graddent solution. Theorem 4 uses this property to map problem (8) to two sub-problems based whether condition (9) holds.
Determining which problem to solve to obtain the optimal price depends on a relationship between two constants N2/ //j and a //11 a2). Here, K2/K1 is the relative competitive ratio with respect to

---

the two tasks for firm B, where K1 > K2 > O. Thus, Kg/R1 + 1 suggests that the relative performance differences between model A and model are similar for both tasks, whereas Kg/Ki - 0 suggests that relative to model A, model B's s performance is much worse on the second task than for the first task. The second parameter a1 (a) + 02) is the fraction of the total demand that is occupied by the first task. If this is close to 1, then the first task has significantly higher demand than the second, but it is close to O, then the first task has significantly lower demand than the second.
We now discuss the structural insights obtained from Theorem (see Figure 3 for a visualization). First, note that when condition (9) is satisfied, firm is able to set higher prices than it could the condition were not satisfied. In the latter case, the optimal price that firm A can set must be upper bounded by the constraint in problem (T], which in this scenario, is less than the upper bound in problem (TO). Intuitively, this condition partitions firm A's pricing problem into two regimes: a high-price and low-price regime. Furthermore, the high-price regime is only attainable if the relative performance difference between the two tasks is greater than the Lambert W function of the fraction of total demand occupied by the first task.
Second, if firm B's relative performance difference between the two tasks is larger than the fraction of demand occupied by the first task, then firm A's pricing problem is infeasible. That is, no matter what price that firm A sets, firm B is always incentivized to set a price that ensures users will prefer model and consequently, leave no revenue for firm A.
Proposition 1. If 62/R1 > ai a1/(a) + 02), then for any price that firm A sets, firm B will always set price that allows them to be price-performance competitive for both tasks.
Intuitively, if the relative performance difference is high, then model B's performance relative to model A is approximately the same for both tasks. Consequently, the range (x2g, Kig) is small, meaning that a small perturbation from this price would not significantly decrease firm B's revenue from the first task, while permitting the firm to obtain revenue from the second task as well. Note that model does not necessarily have to be 'better' than model A, only that the performance on the tasks must be similar. Finally, this ratio K2/ K1 only needs to be larger than the fraction of total demand that is occupied by the first task. Consequently, even if the ratio is small, firm B can still be incentivized to set competitive price for both tasks if the potential revenue that can be obtained from the second task is high.
Theorem 4 and Proposition [ sets guideline for when a firm should deploy their generative Al model. Consider company that has developed a model for new application area with no competitors. This company knows that upon releasing their model to the public, competitors will release their own models. If the first-to-market company believes that the use-cases for this model are sufficiently similar to each other with respect to model performance, but are differentiated with respect to user demand, then they must ensure that the model performs exceedingly well on at least one specific use-case that their competitors cannot match. That is, the company must differentiate their product Schmalensee 1982]. or competitors can outprice the initial company with similar products.
Although firm A must price accounting for firm B's actions, if model A is sufficiently differentiated from model B. then firm A can acquire their maximum possible revenue. Recall that the zero- pero-gradient optimal price for problem (8) is 4" 1/b. Furthermore recall that if condition (9) is met, then firm can set higher prices for their model. We show below that if condition (9) satisfied and if x2 sufficiently low with respect to the demand ratio, then the optimal price is feasible.
Proposition 2. K2 < WWaaje (//(ie-/(((11 + +z)) min(K1, 1). then firm A maximizes their revenue by setting 9² = 1/b.
6
Conclusion
We study how a company developing generative AI model can set the price for this technology. Generative AI models are a fundamentally different technology product compared to classical AI models due to two factors. First, single generative AI model is performant on multiple distinct use-cases that each invite individual respective user demands. Second, generative AI models offer interactivity via prompting, thereby encouraging 'geometric' user interaction where users can repeat- edly prompt the model until it generates a satisfactory answer, compared to classical non-interactive Al models that invite one-shot 'Bernoulli' interaction. These features combined with a singular unit price per-prompt for model use warrant new revenue maximization frameworks for this technology.

---

We find that for generative Al models, the pricing problem reduces to ranking the different tasks in order of the relative performance of the model versus a competing alternative. In isolation, company can then always obtain non-zero revenue by setting price to be competitive for at least one downstream task. However, when considering the competition from alternative models that may be released after the price is set, the company faces strict upper bounds on the prices they can set based on the performance of the company's 's model relative to the latecomers. In particular, if the relative difference in performances on the tasks is sufficiently small, then a competitor will always be able to outprice the company on all tasks. This result reveals that an outsized performance improvement on at least one of the downstream applications of generative Al model is essential to maximizing revenue.
Limitations. We explore theoretical market problem where firms know user demand and the performance of competing Al models. In practice, these would be estimated with some noise. Furthermore, we study a static marketplace where each firm sets their price once. In contrast, AI technologies feature flywheel where low price and an early release of a product can allow for a firm to collect more training data and improve their model downstream [Gurkan and de Véricourt 2022]. In particular, the opportunity to acquire high-quality data can significantly improve model performance, thereby motivating the first mover position. Finally, we do not include the cost of developing the generative Al models themselves, but assumes that the fims have already decided to build these models. The market dynamics can change when considering how large of a model to build, how much resources (c.g., compute hours) to spend, or even whether to build a generative AI model. We see these scenarios important future extensions.
Societal impact. Pricing of generative Al can significantly affect the democratization of generative Al technology. This paper explores the conditions that incentivize firms to develop or deploy this technology, e.g., when firms can obtain revenue. Setting appropriate prices for these models can allow for this technology to be more easily accessible to wider set of users.
Acknowledgments and Disclosure of Funding
The author thanks the NeurIPS 2024 editorial chairs and anonymous referees for providing valuable feedback that significantly improved this paper. The author also thanks David Acuna, Sanja Fidler, Marc Law, and James Lucas for providing insightful discussion and valuable suggestions on early versions of this paper.
References
Simran Arora, Avanika Narayan, Mayee Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, and Christopher Re. Ask me anything: A simple strategy for prompting language models. In The Eleventh International Conference on Learning Representations, 2022.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0586) 2022.
Marcel Binz and Erie Schulz. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 12006))c2118521122 2023.
Tom Brown, Benjamin Mann, Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances neural information processing systems, 33:1877-1901. 1901, 2020.
Francisco Castro, Jian Gao, and Sébastien Martin. Human-ai interactions and societal pitfalls. arXiv preprint arXiv:2309. 10448, 2023.
Lingjaa Chen, Paraschos Koutris, and Arun Kumar. Towards model-based pricing for machine learning in data marketplace. Proceedings of the 2019 international conference on management of data, pages 1535-1552. 2019.
Mark Chen, Jerry Tworek, Heewoo Jun Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray. Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such,
10

---

Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Car,, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katic Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
Zhi Chen and Wenjie Xue. Learning by failing: The (unintended) consequence of test reporting autonomous vehicle training. Available at SSRN, 2023.
Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li. Hao Zhang, Banghua Zhu, Michael Jordan, Joseph Gonzalez, et Chatbot arena: An open platform for evaluating Ilms by human preference. arXiv preprint WxXiv2240300413. 2024.
Pradcep Chintagunta and Vithala R Rao. Pricing strategics a dynamic duopoly; A differential game model. Management Science, 42(11): 115111150111,,,, 1996.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jenry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprins arXiv:2110. 14168, 2021.
Zicun Cong, Xuan Luo, Jian Pei, Feida Zhu, and Yong Zhang. Data pricing in machine learning pipelines. Knowledge and Information Systems, 141-:144, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Puttern Recognition, pages 248-255.
Ieee, 2009.
Tyna Eloundou, Sam Manning, Pamela Mishkin, and Danicl Rock. Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint 23Xiv:2303.1013 2023.
Awi Federgruen and Ming Hu. Multi-product price and assortment compecttion. Operations Research, 63(3):
572 224,584.22 2015.
Awi Federgruen and Ming Hu. Stability in general oligopoly model. Naval Research Logistics (NRL), 66(1): 102,102,2220 2019.
Pnina Feldman, Yiangos Papanastasiou,, and Ella Segev. Social learning and the design of new experience goods. Management Science, 6504)1150-- 2019.
Guillermo Gallego and Ming Hu. Dynamic pricing of perishable assets under competition. Management Science, 60(5):1241 1259, 2014.
Guillermo Gallego and Garrett Van Ryzin. Optimal dynamic pricing of inventories with stochastic demand over finite horizons. Management science, 40(8):999-1020, 1994.
Robert Gibbons. Game theory for applied economists. Princeton University Press, 1992.
Huseyin Gurkan and Francis Véricourt. Contracting, pricing, and data collection under the flywheel effect. Management Science, 68(12))8791-88 188888 2022.
Jonathan H Hamilton and Steven M Slutsky, Endogenous timing in duopoly games: Stackelberg or cournot equilibria. Games and Economic Behavior, 2(1):29-45 2-. 1990.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray. Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.088 008361, 2020.
Bernard Koch, Emily Denton, Alex Hanna, and Jacob G Foster. Reduced, reused and recycled: The life of dataset in machine learning research. arXiv preprint arXiv:21 2021.
Daiqing Li. Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-iman -ooiiaaae generation. arXiv preprint 1724555 2024.
11

---

Yuan-Hong Liao, Rafid Mahmood. Sanja Fidler, and David Acuna. Can feedback enhance semantic grounding in large vision-language models? arXiv preprini arXiv:2404.06510, 2024.
Jinfei Liu, Jian Lou, Junxu Liu, Li Xiong, Jian Pei, and Jimeng Sun. Dealer: an end-to-end model marketplace with differential privacy. Proceedings of the VLDB Endowment, 14(6), 2021.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and prediet: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1 35, 2023.
Rafid Mahmood, James Lucas, Jose M Alvarez. Sanja Fidler, and Mare Law. Optimizing data collection for machine learning, Advances Neural Information Processing Systems, 35:29915-299922 2022.
Soroush Nasiriany, Fei Xia, Wenhao Yu, Ted Xiao, Jacky Liang, Ishita Dasgupta, Annie Xie, Danny Driess, Ayzaan Wahid, Zhuo Xu, et al. Pivot: lterative visual prompting elicits actionable knowledge vlms. arXiv preprint 4rXiv:2402.0887,, 2024.
Rebecca Roelofs, Vaishaal Shankar, Benjamin Recht, Sara Fridovich-Kan Keil, Moritz Hardt, John Miller, and Ludwig Schmidt. A meta-analysis of overfitting in machine learning. Advances in Neural Information Processing Systems, 32, 2019.
Richard Schmalensee. Product differentiation advantages of pioneering brands. The American Economic Review, 72(3):349-36 346-66.,..6611 1982.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière. Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023.
Garrett J Van Ryzin and Kalyan T Talluri. An introduction to revenue management. In Emerging theory, methods, and applications, pages 442-194, Informs, 2005.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe Le. Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-cons improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11177 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, al. Chain-of of thought prompting elicits reasoning large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5):1122 1136, 2023.
Xinyi Xu, Thanh Lam, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Model shapley: equitable model valuation with black-box access. Advances in Neural Information Processing Systems, 36, 2024.
JD Zamfirescu-Peree Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. Why johnny can't prompt how non-ai experts try (and fail) to design I1m prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-21, 2023.
Mengzhenyu Zhang, Hyun-Soo Ahn, and Joline Uichanco. Data-driven pricing for a new product. Operations Research, 70(2):847-8)) 866, 2022.
Albert Ziegler. Eirini Kalliamvakou, X Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittam- palam, and Edward Aftandilian. Measuring github copilot's impact productivity. Communications of the ACM, 67(3):44-63, 2024.
12

---

A Proofs
Proof of Theorem Consider the ordering a defined by 4 and note that for any if we constrain p to satisfy
(o(t) 2 P ko(t+1))
q
then all of the tasks ( (2)....., (t) become competitive for firm B's generative model compared
to firm A's model, i.e., by satisfying (T). whereas tasks oft + 1)..... (T") are competitive for firm
A's model instead. As a result, for prices constrained this range, the resulting problem becomes
maxp P E.= (e) (v). Therefore, to solve the overall pricing problem, we only need to solve this resulting constrained problem for each value of
Proof of Theorem 2 From Theorem for any value of g, there exists a I [T] such that firm will prioritize revenue from tasks o(1), o (2), ()(()))))))))))) ,()). This value of l will satisfy
}
i' max D Docss(o
max
P
(p)
No(0)22) No(L-11
Ko(eaa 2 2 no(e+2)a
for all t' * f. Furthermore, given this 1, firm A will only obtain revenue from tasks a(t + 1), o(t - 2). .. (T), thereby revealing the objective function. Finally, note that if T, then firm will not acquire any revenue since firm B will be price-performance competitive on all tasks. Therefore, firm A's optimization problem is to simultancously solve for and for t E [T' - -1.
Proof of Theorem 3 First, note that under an exponential demand function the revenue function for each piece of problem (5) is (1) exp(-bp). and therefore has a zero- prro-gradient point at p = 1/b. Thus, when solving problem (5). we solve the inner problem by breaking into 3 cases based on which to consider.
For t" satisfying the condition. Here, the zero- paro-gradient price p" is feasible solution to the inner pricing problem, meaning that this must be the optimal price. Substituting this into the revenue function yields ao(()) exp(-1))6b.
For t > t", Note that for any > t", the price is constrained to be less than +1( < 1/b. In this regime, the revenue function pa)(t) exp(-bp) is monotonically increasing, meaning that the optimal price will be the maximum possible value, i.e., p gka(t). Substituting this into the revenue function yields (o())) exp (-bbo
For f < t². Here, the set of feasible prices is strictly greater than (a(())) > 1/b. In this regime, the revenue function is monotonically decreasing, meaning that the optimal price will be the minimum value, i.e., - ((+((++))- approaching from above. However, for any such setting, decreasing p to equal the infinum would make firm B frice-performance thereby yield higher return. Therefore, the optimal solution for any / < l* can always be upper bounded by the optimal solution for I 1, leading up to 1..
competitive for the t + 1-th task well and
Proof of Corollary ! The upper bound follows from the proof of Theorem 3 as we show that the optimal revenue cannot be achieved without satisfying task (tt). The lower bound follows from observing the optimal solution for t T.
Proof of Theorem 4 Let P1 = arg (pg Ki9 2 > k2g). Furthermore, 1et P2 := arg max (plxxp((D)))) (p) D2(p)) k29 p 0}. Although Pi and p2 depend on q, omit this dependency to simplify the notation. We prove our theorem by exploring three regimes for g: (0, (/bx,)))),,, )]. (1/(bx1), 1/(b22)), and (1/(bkz), 20). We then show that whether condition (9) the problems for each regime simplify further into problems 10) and ITI.
For q E (1/(br2), oc). For any 1/(bR2), we have Ben 2 1/b > 0. From Theorem 3 Pg = 1/b the optimal price. Furthermore, since Kig 2 Pı > K2g, we have p1 > 1/b. From CorolTary 1 pi is priced too high and cannot achieve higher revenue than p2. Therefore in this regime, firm B will set price low enough that their model is competitive for both tasks, and consequently firm will achieve zero revenue.
13

---

For g E (1/(br1). 1/brg)) From Theorem 1/b and p2 = k2q are the optimal prices that firm B can set their two sub- sub-problems. For firm A achieve any revenue, the constraint in problem (8) becomes
a1 exp(-1)
b ai exp(-1) > k2q(ai +
exp( exp(e + bx2q emaqexp((bing
a1 + 02
cxp(-11
brid W
01 +02
01 exp(
g <
W
bk2
a1 + 42
bK2
Above, the first line follows from rearranging the terms and the second line follows from applying the definition of the Lambert W function, i.e., W(z) y + y exp(y) again from rearranging the terms. Finally, note that the Lambert W function is bounded in [-1, 0]. meaning that this constraint dominates the original upper bound. Therefore, for q in this regime, can solve the problem
The third line follows
1 w ( a1 exp(-1) bnz
max ga2 exp(-bg)
(12)
02
DKi
For q E (0,b/(bm From Theorem 3 P1 Ki4 and P2 k2d are the optimal prices that firm B
can set when < (bmb). Here, the constraint that ensures firm B will only prioritize the first task
reduces
a1
gbiai exp(-bx1p qM2(a1 + a2) exp(-pbippp =
exp(bb((1 K2)g)
K2
ai + a2
Ki
b(K1
k2)g < log
log
01 02
log K2 b
+ log a1+-2
Above, the first line follows from rearranging the terms and the second line follows from taking the log on both sides. The third line follows from rearranging the terms. Therefore, for q in this regime, we can solve the problem
max gaz exp(-
min
log
N1 R22
+ log
a1
+ a2 ))
2 2 20
(13)
bri
b((1
K22)
We now show that when condition ( is satisfied, problems 12 and (13 join to have one continuous is infeasible and the minimum
feasible set, whereas when the condition is not satisfied, problem 12
disappears.
First, to see that when the condition is satisfied, the two problems have one feasible set, we must prove
K1
a1
log
log
(14)
b(Ki K2)
02
bsi
Specifically,
K2 W
an exp(-1))
K2
K2
01 exp(epp -1)
exp
Ki
a1 + +02
K1
K1
01 + 02
K2
01
cxp
1
K1
K2
01 + 02
log
log
Ki
K22
01 + 02
R2
61
01
< log
+ log
K2
01 + 02
1
K1
01
<
log
+ log
bis
b(k1
K22)
K2
a1
02
14

---

Above, the first line follows from applying the Lambert W function. The second line follows from rearranging the terms and the third line follows from taking the log on both sides. The fourth and fifth lines follow from rearranging the terms and multiplying both sides by 1/b. Note that when this condition is satisfied, the lower bound for problem (12) and the upper bound for problem (13) are equivalent, meaning that we can merge the two disjunctive regions. This results in obtaining problem (10).
Next, we first note that when the condition is not satisfied, the inequality in (4 is reversed and the minimum on the seft-hand-side of the constraint (13) can be removed. We then show that problem becomes infeasible. Specifically,
12
K2g W (-1 _uepp(-11)
1 W
a1 exp(
a1 +
N1
K2
a1 + 02 exp(
1
W
bk1
b
an 02
The second line follows from multiplying both sides by 1/b. This condition means that problem (12) does not have a feasible region, meaning that the optimal price can only be obtained by solving problem IT.
Proof of Proposition The proof of the follows from observing that when this condition is satisfied,
the intermediate problems from the proof of Theorem [4] i... problems (12) and (13) both become
infeasible. If those problems become infeasible, the nominal problems TO and also must be infeasible. First,
K1
01 + 02
81 (1 + 02
<
log
log
R2
Q1
K2
d1
Kl
01
log
log
0
K2
01
02
The first line follows from taking the log and the second line follows from rearranging the terms and simplifying. When this condition is satisfied, problem (13) becomes infeasible.
Next, observe that for any = E [0, II. we have = 2 -W(zexp(-1)), Because ay u1/(a1 + a2) E [0, 1]. for any value aj and a2, we have
Q1 2 w a1 + 02
(1 exp((11)
Ri
(1 exp(
> WW
a1 + a2
11
a1 + a2
(al exp(-11)
W
bK1
bká
a1 +
Thus, problem 12 is also infeasible.
Proof of Proposition We show that if this condition holds, then problem (10) is the active problem- to-solve and that the zero-gradient solution 1/b is feasible solution. First, note that the condition implies that condition (9) holds, meaning that problem (IOp is the problem-to-sol sollee.. We then note that condition implies
1 K2
(- (-9 exp(-1) (-1eex((-1) min
Vi W, 1
w (w-- (xpcp111
< ---W w
+
R2
01 02
Multiplying both sides by 1/b completes the proof.
B Extensions from per-prompt to per-token pricing
In this paper, we assume that the generative Al models can be used at a fixed price per-proe per-prompt. In practice, generative models are priced either per-token or priced via a subscription mechanism where users pay fixed cost for (potentially unlimited) use. Furthermore, different tasks may feature statistically different numbers of tokens either via longer (or shorter) prompts, or longer (or shorter) model outputs. Consequently under per-token pricing, different tasks will have different costs
15

---

average. The price-pes per-prompt framework of this paper naturally extends to pricing per-token with small change of variables that preserves all fundamental results.
Suppose there are two firms, A and B, developing generative Al models with fixed price per-tokens Yo and Po, respectively. We assume that the price is the same for both input and output tokens. For each task t and model, there is now distribution of the number of tokens Φ and 0r, that the user sends and receives through the respective model in given prompt. Note that the corresponding price-per-prompt Pt and It are now random variables p: PoO, and gr goor, that depend on the specific task. For any given task a user will prefer model B if
PoB[ S goL]()n((W))
where the expectation is taken over the randomness in the number of prompting rounds times the number tokens per prompting round. The corresponding revenue functions for the firms are
T
Ra(9olpo) = qo
Σ
De(go)1 (go & poE])
t=1
RB(Pol90) = Po Σ
De(po) & go
1=1
To solve this problem, note that the inherent structure is equivalent to equation 3
Let Rt = B(d=n(W.))/(((nn(v)) be the corresponding competitive ratio of the models and note that the set of tasks can be ranked according to this re-defined competitive ratio. Then, Theorem follows using the same steps and the re-defined Kg. Moreover, all subsequent results carry over from this result.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract summarizes the key insights obtained from our model as well as the problem setting and model assumptions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have a Limitations subsection in the Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
All proofs are in the Appendix. All assumptions are in the main paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [NA]
Justification: This primarily theory-driven paper. The only numerical analysis is a computational example which detail in the main paper.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer:
[NA]
Justification:
We do not perform any experiments involving datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ccpppi.ccppulli//uuuii wwwti//ggui////eeeeceoocc CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [NA]
Justification:
We do not train any neural networks.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
We do not have any experiments with statistical noise.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
19

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [NA]
Justification: There is compute required.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that NeurIPS Code of Ethics https: http::://eeris..eeeee cc httppp/ccccc./ctcccccccie.....ea
didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification:
We have reviewed the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader societal impacts discussed in the conclusion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
20

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No data models are released in this paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: No assets are introduced in this paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppswpptseeee whworee.wthhhe....c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's is creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
21

---

Answer:
[NA]
Justification: No assets are introduced in this paper.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/codeemoodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No human studies performed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human studies performed.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
22


Reviews:
[{"review_id": "ixi0Jh2OGl", "reviewer": "Reviewer_USe1", "summary": "This paper develops a theoretical model of users deciding which generative AI system to use based on the price and probability of performing a task satisfactorily. Based on this theoretical model, the paper then explains how a firm should price their generative AI model in response to other firms, and then how a firm should price their model knowing that other firms will respond. They show that, depending on the ratio of ability to perform a task between the two models, a firm will either get no revenue since they are not price-competitive for anything, charge low enough prices to be price competitive on some tasks, or be so much better at a task than the other firm that they can charge prices which are bound based only on decreasing demand as price increases.", "questions": "1) Is there an alternative way of modeling task performance that does not assume that the AI system will always succeed?\n2) Different tasks might use different numbers of tokens while performing the task, which would result in higher costs for that specific task. Would it change the results a lot if all tasks shared a per-token price but had variable costs for the different tasks based on how many tokens the task requires?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact for their work.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Significance: This paper builds a model for analyzing pricing for generative AI services which can be used for multiple different tasks, which is a question which is very broadly applicable to analyzing AI pricing.\nOriginality: The pricing model is original, with several new definitions and problem formulations.\nQuality: The paper has proofs supporting its major claims.\nClarity: The paper is very clearly written, and I found its figures to be exceptionally helpful in understanding the claims of the paper.", "weaknesses": "1) The generative AI task performance model seems somewhat unrealistic -- it assumes that given enough user queries the model will always eventually succeed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your clear summary of our work, your positive comments on the originality and significance/broad applicability of our work, as well as on the clarity of our writing. \n\n---\n\n> Q1) Is there an alternative way of modeling task performance that does not assume that the AI system will always succeed?\n\n`Related: aKR2-Q4.`\n\nThis is a great point that we will update in our paper. We agree that for some complex tasks $t$, a user may give up after some maximum number of rounds $T_t$ before the model succeeds. **Our results all extend to this scenario (or any finite-mean distribution) with a small change of variables.**\n\n\nIn a setup where users terminate after some fixed $T_t$ rounds, the total number of rounds $n$ that a user prompts is a Truncated Geometric distribution \n\n$$Pr(n) = \n\\begin{cases}\n    (1-V_t)^{n-1}~V_t & \\text{ for } 1 < n < T_t \\\\\\\\\n     (1-V_t)^{T_t-1}  & \\text{ for } n = T_t\n\\end{cases}\n$$\n\nUsing the expected value of this distribution, the corresponding Eqn 1, i.e., when user prefers model B $(p, V_t)$ over model A $(q, W_t)$, is $p( \\frac{1-(1-V_t)^{T_t-1}}{V_t} + (1-V_t)^{T_t-1} ) \\leq q( \\frac{1-(1-W_t)^{T_t-1}}{W_t} + (1-W_t)^{T_t-1} )$. This principle plugs into a revised pricing problem (Eqn 2), and requires a re-defined competitive ratio between two models on a task:\n\n$$\n\\kappa_t := \\frac{ \\frac{1-(1-V_t)^{T_t-1}}{V_t} + (1-V_t)^{T_t-1} }{ \\frac{1-(1-W_t)^{T_t-1}}{W_t} + (1-W_t)^{T_t-1} }.\n$$\n\nWith this re-defined $\\kappa_t$, we can solve the pricing problem using Theorem 1 and recover all theoretical discussion and insights in our paper. Our main conclusions stay the same.\n\n\nThe maximum number of rounds $T_t$ is a parameter that can be estimated from historical data. For example in the Chatbot Arena dataset, users spend on average 1.3 rounds before determining the generative model output as satisfactory (i.e., $T = 2$ for many tasks). \n\n---\n\n> Q2) Different tasks might use different numbers of tokens while performing the task, which would result in higher costs for that specific task. Would it change the results a lot if all tasks shared a per-token price but had variable costs for the different tasks based on how many tokens the task requires?\n\n`Related: aKR2-W2.`\n\nWe agree that different tasks use different number of tokens, and thus, have different costs. **This extension will recover all our original results.** We will include this in revision.\n\n\nTo account for tasks with different prompt types, we define the price-per-prompt for each task $t$ as $p_t = \\theta_t p_0$ where $p_0$ is the price-per-token and $\\theta_t$ is the average number of tokens-per-prompt for task $t$ (equivalently $q_t = \\phi_t q_0$ for model A). Note that $p_0$ and $q_0$ are the price variables that the company can set whereas $\\theta_t$ and $\\phi_t$ are fixed parameters. Following the same steps as before, we can revise Eqn 1 to $\\theta_t \\frac{p_0}{V_t} \\leq \\phi_t \\frac{q_0}{W_t}$ and also update the pricing problem Eqn 2. All our results automatically extend after we re-define the competitive ratio to $\\kappa_t := \\frac{\\phi_t V_t}{\\theta_t W_t}$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h8Z6UVBaXH", "reviewer": "Reviewer_U4W8", "summary": "The paper identifies some unique characteristics of modern generative AI software which affect their pricing. It uses a notion of user cost-effectiveness to compare two models, capturing the cost per prompt and the number of prompting rounds needed to reach a satisfactory answer. The authors propose to model the pricing problem as a game between two companies sequentially releasing their models before users choose their preferred model for each task. A set of tasks is assumed and each task is priced separately. They show that the price optimization problem is piecewise continuous i.e. the companies must choose a subset of tasks on which to be cost-effective and forgo revenue on other tasks. Their analysis indicates that being the first to market may become cost-ineffective, as companies entering the market later can benefit from knowing their competitor's pricing.", "questions": "See above.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Pricing digital goods and services is an interesting problem in general due to their unique characteristics. The paper is a nice early attempt to characterize and price generative AI. \n\n2. The modeling is simple and accounts for user satisfaction, demand, and accuracy of the model for different tasks. \n\n3. Analysis with the above model reveals to the player (company) that companies have to choose a subset of tasks on which they can be cost-effective and forgo revenue on other tasks. It also shows that when the tasks are sufficiently similar, then the first-to-market may become cost-ineffective on all tasks regardless of the pricing.", "weaknesses": "1. The setup seems far from reality and loosely related to generative AI. First, in real life even if we assume two companies, there is nothing prohibiting them from updating their prices any number of times and the first mover is probably always in a better position due to customer acquisition, branding, and acquiring real data, system testing, etc. So the analysis seems counterintuitive. \n\n\n2. The connection with generative AI is not very clear. What unique characteristics of generative AI are actually used in the pricing model? It is generally applicable to any model with different performance on a set of tasks. \n\n\n3. There are no real data experiments or simulations provided in support of the modeling assumptions and analytical results. It is understandable the reality is complex and there are a lot of variables involved in pricing. It would be unreasonable to expect to model all of that but I do expect this study to be a bit more closer to reality than it is. It could be done by showing the real data and formulating the modeling assumptions from it and the pricing game with multiple parties going over multiple rounds and also improving their models. It would be great if with the proposed model the future price can be predicted accurately.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on the uniqueness of our problem as well as the simplicity and comprehensiveness of our model.\n\n---\n\n> W1) The setup seems far from reality and loosely related to generative AI. First, in real life even if we assume two companies, there is nothing prohibiting them from updating their prices any number of times and the first mover is probably always in a better position due to customer acquisition, branding, and acquiring real data, system testing, etc. So the analysis seems counterintuitive.\n\n\nThank you for several important points. Below, we justify our formulation to show that it reasonably captures real-world trends and we will revise our paper with this discussion.\n\n\n\n1. ( `Related: 6DrW-Q5, aKR2-Q3.`) Our framework can be extended to multiple companies by taking the highest price-performance ratio for each task. If a company $(p, V_t)$ has two competitors with $(q, W_t)$ and $(r, X_t)$, then a user will prefer the first model if  $\\frac{p}{V_t} < \\max( \\frac{q}{W_t} , \\frac{r}{X_t})$ (i.e., Eqn 1). With this, we can revise the pricing optimization problems (Eqn 2) and automatically extend Theorem 1 & 3 with a re-defined competitive ratio $\\kappa_t$. Although the general first-mover's problem (Theorem 2) becomes more difficult to solve, note that facing multiple competitors should preserve a first-mover disadvantage effect similar to what we currently observe. \n\n\n\n2. (`Related: CY6H-W3.`) We agree that real-world pricing may be dynamic with firms updating their prices over time. Our model is a natural baseline for this setting, since each turn of a dynamic problem can be naively tackled using static decisions. Furthermore, our results reveal key insights about the dynamic setting. Proposition 1 shows that if model performance is not competitive (i.e., $\\kappa_2 / \\kappa_1$ is large), then for any price that the company sets, their competitor can always beat them. **Thus, companies cannot just update prices, but must also continually improve their models or else be eventually beaten.** \n\n\n   We believe a full dynamic pricing study of this problem must include both pricing and model performance as variable, making it a difficult open problem. Our work is the necessary first step to tackling the extension.\n\n\n\n3. We agree that factors such as customer acquisition, branding, and the AI data/testing flywheel also significantly impact revenue. However, modeling every such component introduces complexity without meaningfully changing our key takeaway.\n\n   Specifically, we show that the first-mover advantage is most meaningful when the model has a particular specialization to at least one task (Proposition 1). This is a quantifiable example of the product differentiation principle and is relevant especially now as the top models perform statistically similarly on most application areas [1]. We observe the importance of pricing and product differentiation in, for example, releases of differentiated models (e.g., Claude 3 Opus, Sonnnet, Haiku), and price drops (e.g., GPT3.5 in Nov 2023 and Jan 2024).\n   \n \n\n[1] https://scale.com/leaderboard\n\n\n---\n\n> W2) The connection with generative AI is not very clear. What unique characteristics of generative AI are actually used in the pricing model? It is generally applicable to any model with different performance on a set of tasks.\n\nOur revision will clarify this below point. Our framework is suitable for modern LLMs (and VLMs/Multimodal LLMs) via two key traits: (1) there are multiple types of tasks; (2) users interact via prompting over multiple rounds. The combination of these two traits is not found in older ML technology. They imply that there is a single price variable affecting all tasks (i.e., price-per-token/price-per-prompt), and that users seek to minimize the number of prompting rounds. The prompting trait especially differentiates the use-case from general multi-task models, where a user can only input a task instance once. \n\n\n---\n\n> W3) There are no real data experiments or simulations provided in support of the modeling assumptions and analytical results. It is understandable the reality is complex and there are a lot of variables involved in pricing. It would be unreasonable to expect to model all of that but I do expect this study to be a bit more closer to reality than it is. It could be done by showing the real data and formulating the modeling assumptions from it and the pricing game with multiple parties going over multiple rounds and also improving their models. It would be great if with the proposed model the future price can be predicted accurately.\n\n\nTo the best of our ability, our model assumptions are based on real data and justification from the literature (as positively remarked by `aKR2`). For example, we studied user interactions from Chatbot Arena to validate that users will often interact for multiple rounds until they are satisfied. \n\n\nUnfortunately, numerically predicting future price accurately requires access to the true demand functions for LLMs, which to our knowledge, is not publicly available. Thus, our numerical analysis is limited to synthetic scenarios (e.g., Fig. 2, Fig. 3) where we impose a hypothetical demand function and demonstrate the outcome. \n\n\nWe also agree that observing multiple parties over multiple rounds to be a very interesting problem, but this changes the problem to a dynamic pricing setting, which requires more theoretical analysis. We envision exploring the dynamic problem in future work both theoretically and with synthetic numerical experiments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MLFClfdztJ", "reviewer": "Reviewer_6DrW", "summary": "This paper studies the pricing and competition of companies providing services using generative AI models. This paper proposes a stylized economic model that abstracts away from the technical details, and in particular, considers two companies entering the market sequentially. This paper assumes that the customer chooses the model that is cheaper per prompt, and each company solves a revenue maximization problem. Based on the model and analysis, this paper argues that companies should adopt different pricing strategies based on their order of entering the market.", "questions": "1. Is the sequential order of companies true? Would simultaneous competition be more reasonable? Furthermore, if so, do they get any first-mover advantage? The current model assumes that the first company is disadvantaged due to a lack of price information. \n1. The current modeling seems to assume that customers are very rational and could make a sensible decision based on knowledge of price and number of prompts $n$. What about the cases when consumers do not have a good idea of the number of prompts needed or the success rate? What if customers have heterogeneous types or they have heterogeneous tasks?\n1. The i.i.d. assumption of $V_t$ is concerning. Due to the conversational nature, the prompts are naturally highly correlated, and thus the independence of the success rate is not justified. \n1. Can authors start the motivation with some more practical background? \n1. To what extent does the current analysis extend to a more general scenario with multiple companies? \n1. Punctuations are missing from several math equations.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The pricing of generative models is an interesting and timely matter", "weaknesses": "1. Several fundamental model setups and assumptions are not very justified. \n1. The insights derived are not very informative nor it provide very meaningful guidance.  Overall, I understand that the parsimonious model aims to highlight certain main trade-offs, but this paper somewhat errs on the side of oversimplification. \n1. Another big concern is that: the nature of analysis in this paper is very stylized and economic, and it is unclear if this is the right venue for such kind of papers..", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on the interest and timeliness of our work.\n\n---\n\n> Q1) Is the sequential order of companies true? Would simultaneous competition be more reasonable? Furthermore, if so, do they get any first-mover advantage? The current model assumes that the first company is disadvantaged due to a lack of price information.\n\n`Related: aKR2-Q5.`\n\nOur sequential order of companies is motivated by real-world events of how ChatGPT had a large first-mover advantage before competing products were released. Furthermore, competitors were able to set their prices after observing the market performance of ChatGPT. Even now, firms release different models with prices asynchronously.\n\nMathematically, we could explore a simultaneous competition framework where neither firm knows the price that their opponent will set. Here, Theorem 1 is not possible since it requires competitor prices. A potential strategy is to use the robust problem of Theorem 2. However, this may not achieve a Nash equilibrium if both companies simultaneously use it.\n\nWe believe that the simultaneous pricing problem is important, but it requires non-trivial analysis to determine equilibrium-achieving conditions. We focus on the sequential problem in this work due to how it captures the real-world events, but we plan to study the simultaneous problem in future work.\n\n\n---\n\n> Q2) The current modeling seems to assume that customers are very rational and could make a sensible decision based on knowledge of price and number of prompts $n$. What about the cases when consumers do not have a good idea of the number of prompts needed or the success rate? What if customers have heterogeneous types or they have heterogeneous tasks?\n\n`Related: aKR2-W1.`\n\nOur ssumption that users are rational and know the expected number of prompts is reasonable for large-scale, commercial users of LLM applications. For example, the user could be a company building software ontop of LLM API calls (e.g., PDF AI Chat tools). They would be informed on task statistics and success rate.\n\nFurther, while our stylized model is defined for a 'single user with different tasks', the techniques naturally extend to multiple heterogeous users by aggregating the pricing objective (Eqn 2) into an expectation over the distribution of different types of users. Moreover, we expect that as a first-mover disadvantage exists for one user, an analogous disadvantage should exist over general multiple heterogeneous users.\n\nWe agree that consumers who do not know their task statistics or success rates, may not be rational, but may be subject to additional factors, such as marketing or word-of-mouth adoption. This is an interesting extension that we will discuss in our paper and study in future work.\n\n---\n\n> Q3) The i.i.d. assumption of $V_t$ is concerning. Due to the conversational nature, the prompts are naturally highly correlated, and thus the independence of the success rate is not justified.\n\n\nThis is a great point. We agree that for some tasks, the success probability and number of rounds may not be independent. **Our framework and all results extend to arbitrary distributions of the number of prompting rounds required to complete a task $n(V)$.** Our initial Geometric assumption was a stylistic choice to make the downstream insights more interpretable, but we will revise our paper with the below generalization.\n\n\nFor any distribution for $n(V)$ with finite expected value $E[n(V)]$, the principle for determining whether company B $(p, V)$ is preferred over company A $(q, W)$, i.e., Eqn 1, is $p E[n(V)] \\leq q E[n(W)]$ and the pricing problem is \n$R(p|q) := p \\sum_t D_t(p)\\mathbf{1} [ p E[n(V_t)] \\leq q E[n(W_t)] ]$. This optimization problem can still be solved using Theorem 1 if we re-define the competitive ratio between tasks as $\\kappa_t := \\frac{E[n(W_t)]}{E[n(V_t)]}$. Using this $\\kappa_t$, we can recover all of our theoretical results. \n\n\n---\n\n> Q4) Can authors start the motivation with some more practical background?\n\n\nWe will revise our paper introduction to better highlight our motivation. \n\nOne motivation of our work was the first-mover position of ChatGPT, which led to several competing products being released shortly after. This technology invited a new paradigm of interacting with a single model to solve various different tasks via multiple prompts. While this product has seemed to be dominant, there are several competing products that all perform relatively similarly on these tasks [1]. For this emerging market, when should users prefer one competing model over another, and what is the market advantage in the current rapid pace of development where companies are quickly trying to release better models as fast as possible? We study these problems by analyzing the relationships between model performance, competition, and price.\n\n\n[1] https://scale.com/leaderboard\n\n\n---\n\n> Q5) To what extent does the current analysis extend to a more general scenario with multiple companies?\n\n`Related: U4W8-W1, aKR2-Q3.`\n\nOur duopoly framework and corresponding pricing problems (Eqn 2) naturally extend multiple companies by simply taking the highest price-performance ratio for each task. If company A $(p, V)$ has two competitors,  $(q, W)$ and $(r, X)$, a user will prefer company A over the competitors if $\\frac{p}{V_t} \\leq \\max( \\frac{q}{W_t}, \\frac{r}{X_t}  )$ (i.e. Eqn 1). We can then rewrite the pricing problem (Eqn 2), re-define the competitive ratio $\\kappa_t$ to be a worst-case ratio over all competitors, and directly recover many of the theoretical results (e.g., Theorem 1, Theorem 3) in our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zHqmqbePrI", "reviewer": "Reviewer_aKR2", "summary": "* The paper explores the optimal pricing problem for generative AI services in a two-firm Stackelberg competition setting.\n* In the setting, two generative AI firms compete over users. Each generative model is characterized by a fixed price $p$ per query, and success probabilities $(V_1, \\\\ldots, V_T)\\\\in[0,1]^T$ for each task. The demand for task $t$ at price $p$ is characterized by a non-increasing demand function $D_t(p)$. Users are assumed to query the generative model repeatedly until a satisfactory result is obtained, and success probability of each query ($V_t$) is assumed to be independent and identical. Two generative AI firms compete over users in a Stackelberg setting (firm A sets prices first, firm B follows), and their goal is to maximize revenue.\n* For firm B, Theorem 1 characterizes the price optimization problem as a piecewise optimization problem. For firm A, Theorem 2 characterizes the price optimization problem as a bi-level optimization problem. Section 5 assumes an exponential parametric form for the demand function, and derives a globally-optimal solution based on them.", "questions": "* What is the computational complexity of the optimization problem in Theorem 1?\n* In the presented setting, can the firms benefit from colluding?\n* What are the expected consequences of having more than two firms participating in the market?\n* How would results change if the generative model was assumed to be able to \"fail\" in generating a satisfactory response? (either due to lack of ability, or users that churn before reaching satisfactory results) \n* How would results change if the two firms are allowed play simultaneously?\n* Minor typos:\n  * L223 - “Theorem holds..” - Missing theorem number?\n  * L239 - “cn” - Can?\n  * L356 - “assumes” - assume?\n  * L357 - “fims” - firms?", "limitations": "Most limitations are discussed explicitly, and in sufficient detail. I feel that the paper can benefit from discussing the additional limitations that appear above.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Topic of the paper is well-motivated.\n* Paper is well-written. Presentation is clear and very easy to follow.\n* Assumptions are introduced gradually and explicitly, and seem to be justified by existing literature.\n* The paper proposes guidelines for pricing in practical scenarios.", "weaknesses": "* The analysis seems to rely on the assumption that a full taxonomy of tasks is available, and that demand curves for each task are independent. However, due to the general-purpose nature of generative models, tasks may change over time, and therefore identifying and estimating all demand curves may be infeasible.\n* Assumed cost model (pay-per-query) doesn’t seem to be common - Current common pricing schemes for generative models are per-token (e.g, OpenAI’s GPT-4 API), or per-month (e.g, ChatGPT), possibly inducing different incentive structures.\n* Code is not provided, making it harder to reproduce results and build upon them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on our motivation, clarity of writing, and overall presentation of our work. \n\n---\n> W1) The analysis seems to rely on the assumption that a full taxonomy of tasks is available, and that demand curves for each task are independent. However, due to the general-purpose nature of generative models, tasks may change over time, and therefore identifying and estimating all demand curves may be infeasible.\n\n`Related: 6DrW-Q2.`\n\nWe agree that for general LLM consumers, the number and type of tasks will vary over time. In practice, a company can update prices to reflect changing user demands. Our most effective use-case is for consistent/stationary users, such as companies that are building tools ontop of LLM APIs (e.g., PDF AI Chat software). These users have relatively consistent tasks and can quantify task statistics, user demand, and success rates.\n\n---\n> W2) Assumed cost model (pay-per-query) doesn’t seem to be common - Current common pricing schemes for generative models are per-token (e.g, OpenAI’s GPT-4 API), or per-month (e.g, ChatGPT), possibly inducing different incentive structures.\n\n`Related: USe1-Q2.`\n\nWe agree that most payment structures are per-token or subscription-based. Our per-prompt pricing is **a stylistic choice to reduce notation and our results automatically extend to pricing per-token** with a small change of variables. We will update our paper with the below extension.\n\nWith per-token pricing, each task $t$ has a different average price $p_t = \\theta_t p_0$ where $p_0$ is the price-per-token and $\\theta_t$ is the average prompt length for the task (equivalently $q_t = \\phi_t q_0$). Here, $p_0$ and $q_0$ are the price variables to optimize, whereas $\\theta_t$ and $\\phi_t$ are fixed parameters.\n\nFor task $t$, a user will prefer model B if $\\theta_t \\frac{p_0}{V_t} \\leq \\phi_t \\frac{q_0}{W_t}$ (i.e., Eqn 1). Using this principle, the corresponding pricing optimization problems (Eqn 2) can be solved by  redefining the competitive ratio between models on each task to $\\kappa_t := \\frac{\\phi_t V_t }{\\theta_t W_t}$. This redefined $\\kappa_t$ can be substituted into the subsequent analyses to recover all our theoretical results.\n\nWe agree that subscription pricing is an important setting, but it is typically for more general consumers. We will explore this problem in future work.\n\n---\n> Q1) What is the computational complexity of the optimization problem in Theorem 1?\n\nThe problem is a max of $T$ single-variable inner optimization problems with only bounding constraints. If the demand functions $D_t(p)$ are differentiable, each inner problem can be solved by gradient descent. For standard choices of $D_t(p)$ (e.g., exponential, linear), we can directly check the zero-derivative and boundary points. For example under Assumption 2, the problem automatically reduces to checking $2T$ possible values of $p$.\n\n---\n> Q2) In the presented setting, can the firms benefit from colluding?\n\nThanks for this great question! Our analysis reveals that for a company to maximize revenue, they should be particularly specialized in at least one task (Proposition 1). This is the generative AI equivalent of product differentiation in  goods. Intuitively, two companies could collude by specializing their models to different tasks (e.g., one company focuses on programming and quantitative tasks, while another focuses on language and writing tasks), thereby ensuring both companies receive revenue. However, quantifying this effect, as well as the social outcomes, is non-trivial and requires extensive effort. We would be happy to explore this more thoroughly in future work.\n\n---\n> Q3) What are the expected consequences of having more than two firms participating in the market?\n\n`Related: U4W8-W1, 6DrW-Q5.`\n\nWe will include this point in our updated paper. Our pricing problems can be extended to multiple competitors by taking the highest price-performance ratio for each task. If a company $(p, V_t)$ has two competitors, $(q, W_t)$ and $(r, X_t)$, a user will prefer the first model over all others if  $p/V_t \\leq \\max( q/W_t , r/X_t)$. This revised version of Eqn 1 can be used to re-define new pricing problems and recover Theorem 1 & 3 by re-defining $\\kappa_t$ as a worst-case competitive ratio. \n\n---\n> Q4) How would results change if the generative model was assumed to be able to \"fail\" in generating a satisfactory response? (either due to lack of ability, or users that churn before reaching satisfactory results)\n\n`Related: USe1-Q1.`\n\nThis is a great question that we will include in our updated paper! **Our framework generalizes to the case where for each task $t$, the user churns after a maximum $T_t$ rounds.** Here, the total number of rounds for a task is a Truncated Geometric distribution (`omitted details for space, see USe1-Q1`). Using the expected value of this distribution, we can re-derive Eqn 1, revise the pricing problem (Eqn 2), and redefine the competitive ratio $\\kappa_t$ between models to recover all of our theoretical results. Our main insights stay the same.\n\n---\n> Q5) How would results change if the two firms are allowed play simultaneously?\n\n`Related: 6DrW-Q1.`\n\nMathematically, we could explore a simultaneous competition framework where neither firm knows the price that their opponent will set a priori. Here, the pricing solution in Theorem 1 is not possible. A potential strategy is to use the robust problem in Theorem 2, which if solvable, would guarantee a non-zero revenue. However, this strategy is not optimal and may not achieve a Nash equilibrium if two companies are simultaneously playing.\n\nWe agree that the simultaneous pricing problem is important, but it requires non-trivial analysis to determine equilibrium-achieving conditions. Since the sequential setting characterizes recent events where ChatGPT held a first-mover position over competitors, we focus on the sequential problem. We plan to explore the simultaneous problem in future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L10JkQj0BK", "reviewer": "Reviewer_CY6H", "summary": "The paper presents a theoretical analysis of pricing strategies for generative AI models in a competitive market. It investigates how companies can optimally set prices for AI models that are used across various tasks, considering the impact of competition from other firms. The study uses a game-theoretical approach to model the interactions between firms, exploring the implications of different pricing strategies on market dynamics and revenue optimization.", "questions": "See previous comments.", "limitations": "See previous comments.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper tackles an emerging and relevant problem in the field of AI economics, particularly the pricing of generative AI in a competitive landscape. The study is timely.\n2.\tThe formulation is clear and the presentation is good. The paper in general is very easy to follow.\n3.\tThe paper offers insightful conclusions about the strategic behaviors of firms in a duopoly, especially the advantages of being a second mover in the market.", "weaknesses": "1.\tFrom the perspective of a stylized model, the work is solid and interesting. But I am not sure about the practice relevant. I am not sure whether the real world LLM company is thinking their pricing strategy in this way. I won’t push along this line, and would like leave the decision on this point to more senior reviewers and ACs.\n2.\tThe main technical contribution to me is the modeling the GenAI’s prompt-based service. The analysis seems very standard.\n3.\tI am not fully convinced that in the current GenAI market, the companies are following the dynamics of model A and model B in the paper. All the companies should be keeping updated their accuracy and be almost homogenous in terms of pricing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on the relevance and timeliness of our work, clarity of our formulation and writing, as well as the key insights drawn from our work.\n\n---\n\n> W1) From the perspective of a stylized model, the work is solid and interesting. But I am not sure about the practice relevant. I am not sure whether the real world LLM company is thinking their pricing strategy in this way. I won’t push along this line, and would like leave the decision on this point to more senior reviewers and ACs.\n\n\nThank you for the positive comments. Although we study the effects of competition, model performance, and user demand, we agree that the real-world pricing will likely incorporate additional factors (e.g., marketing, model development).\n\nRather than algorithms for prescribing explicit prices, our goal is to reveal insights on market dynamics drawn from the unique traits of generative AI technology. Specifically, we show the importance of knowing competitor price and performance information on revenue. For example, Proposition 1 shows that if the model built by firm A is not differentiated, then no matter their price, firm B can always set a price that limits firm A's revenue. The implication is that the only way for firm A to profit is to either improve their model to specialize on at least one task, or use external factors (e.g., marketing).\n\n\n---\n\n> W2) The main technical contribution to me is the modeling the GenAI’s prompt-based service. The analysis seems very standard.\n\n\nWe agree that the key methodological contribution is the modeling and problem formulation of the pricing problem in generative AI. The goal of our follow-through analysis is to derive insights, rather than new algorithms. Further, we believe that our model is the first step towards more advanced analyses of generative AI pricing.\n\n---\n\n> W3) I am not fully convinced that in the current GenAI market, the companies are following the dynamics of model A and model B in the paper. All the companies should be keeping updated their accuracy and be almost homogenous in terms of pricing.\n\n`Related: U4W8-Q1`\n\nThis is a great point that our analysis also agrees with and we will include the following discussion in our revision.\n\nOur analysis in Proposition 1 shows that firms should frequently introduce new, better models, in addition to competing on price. Specifically, if the model performance is weak (i.e., $\\kappa_2 / \\kappa_1$ is large), then it doesn't matter what price they set because their competitor can set a more competitive price. Thus, we must continue to improve our models -- but by how much? Proposition 1 states that to guarantee revenue, we should ensure the model sufficiently outperforms competitors on at least one task (relative to other tasks), which allows the product to be sufficiently differentiated. This may be more efficient than competing on every task.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their positive comments: 
- Originality, novelty, and timeliness of our study (`USe1, U4W8, 6DrW, CY6H`)
- Broad applicability of our model & insightful analysis/guidelines/conclusions drawn (`USe1, U4W8, aKR2, CY6H`)
- Clarity of our written presentation (`USe1, aKR2, CY6H`)

Our work is motivated by the recent trends in generative AI (i.e., LLMs/Multimodal LLMs) where ChatGPT enjoyed a first-mover position. The prompt-based interaction combined with diverse set of applications make pricing these ML models different from others. We are not trying to prescribe exact prices, but instead we are studying the interaction between the performance vs price of these models when given competing alternative models. 

Our general formulation allows us to derive high-level insights on the generative AI market. Our key insight is the observation that if model performance over different tasks fails to satisfy a relative ratio requirement (i.e., $\kappa_2 / \kappa_1$ is large), then the company's revenue can always be limited by a competing alternative.

We received questions mainly on several extensions of our model. With minor change of variables, **most extensions can be easily accommodated while preserving all our conclusions**. We summarize these extensions below and will include them in revision:

- **If users stop after a number of rounds or if prompts are non-i.i.d. (`USe1-Q1, 6DrW-Q1, aKR2-Q4`):** We can use any arbitrary distribution on the number of prompting rounds $n(V)$. In general, Eqn 1 becomes $p E[n(V_t)] \leq q E[n(W_t)]$ and the pricing problem (Eqn 2) changes to reflect this. If we re-define the competitive ratio to $\kappa_t := \frac{E[n(W_t)]}{E[n(V_t)]}$, then we can recover all our results.

- **Different prompt lengths and pricing per-token (`USe1-Q2, aKR2-W2`):** For task $t$, the price per-prompt becomes $p_t := p_0 \theta_t$ where $p_0$ is the price per-token and $\theta_t$ is the average prompt length (respectively $q_t := q_0 \phi_t$ for the competitor). Here, Eqn 1 becomes $\theta_t \frac{p_0}{V_t} \leq \phi_t \frac{q_0}{W_t}$ and the competitive ratio becomes $\kappa_t := \frac{\phi_t V_t}{\theta_t W_t}$. We can recover all our results.

- **If there are more than 2 companies (`U4W8-Q1, 6DrW-Q5, aKR2-Q3`):** If there are three models $(p, V_t), (q, W_t), (r, X_t)$, then users will prefer the first model if $p/V_t \leq \max(q/W_t , r/X_t)$.  We can update the pricing problem (Eqn 2) and still solve the latecomer's pricing problem (Theorem 1 & Theorem 3) by re-defining $\kappa_t := V_t / \max(W_t, X_t)$.

Meta-review: The paper studies the problem of pricing generative AI technologies. Their setting assumes different tasks and two firms that sequentially price their tasks based on their performance to maximize their profit.

While the reviewers generally liked the motivation behind the paper and the analysis of the simplified model, they were concerned about several assumptions of the paper. These include the one-shot nature of the game, the assumption of iid results for repeated queries and the lack of any accounting for the cost it takes to generate the technology. The authors had provided to some of these concerns in the rebuttal. While I do not expect a published work to address all these issues, sI encourage the authors to incorporate these to the limitations/discussion of the paper. I would also encourage the authors to include a more in-depth discussion of why the game applies to generative AI companies more clearly in the final version.
Decision: Accept (poster)