Incentivizing Quality Text Generation via Statistical Contracts
Authors: Eden Saig
Ohad Einav
Inbal Talgam-Cohen
Venue: NeurIPS 2024 poster (2024)
Keywords: Contract Theory
Contract Design
Moral Hazard
Natural Language Generation
LLM evaluation
Hypothesis Testing
Primary area: algorithmic_game_theory
TL;DR: We design pay-for-performance contracts that incentivize the use of high-quality LLMs for text generation.

Abstract: While the success of large language models (LLMs) increases demand for machine-generated text, current pay-per-token pricing schemes create a misalignment of incentives known in economics as moral hazard: Text-generating agents have strong incentive to cut costs by preferring a cheaper model over the cutting-edge one, and this can be done “behind the scenes” since the agent performs inference internally. In this work, we approach this issue from an economic perspective, by proposing a pay-for-performance, contract-based framework for incentivizing quality. We study a principal-agent game where the agent generates text using costly inference, and the contract determines the principal’s payment for the text according to an automated quality evaluation. Since standard contract theory is inapplicable when internal inference costs are unknown, we introduce cost-robust contracts. As our main theoretical contribution, we characterize optimal cost-robust contracts through a direct correspondence to optimal composite hypothesis tests from statistics, generalizing a result of Saig et al. (NeurIPS’23). We evaluate our framework empirically by deriving contracts for a range of objectives and LLM evaluation benchmarks, and find that cost-robust contracts sacrifice only a marginal increase in objective value compared to their cost-aware counterparts.

Incentivizing Quality Text Generation via Statistical Contracts
Eden Saig', Ohad Einav', Inbal Talgam-Cohen"
Technion Israel Institute of Technology
Tel Aviv University
(edens, ohadeinav, italgamm@csssssss wet.aa//e.eecceioooo....ooo.o techniin.cn....c ac.
Abstract
While the success of large language models (LLMs) increases demand for machine- generated text, current pay-pe may-per-token pricing schemes create a misalignment incentives known in economics strong incentive to cut costs by preferring a cheaper model over the cutting-edge one, and this be done "behind the scenes" since the agent performs inference internally. In this work, we approach this issue from an economic perspective, by proposing may-for-performann
moral hazard: Text-generating agents have
contract-basee framework for incentivizing quality. We study principal-agent game where the agent generates text using costly inference, and the contract determines the principal's 's payment for the text according to an automated quality evaluation. Since standard contract theory is inapplicable when internal inference costs are unknown, we introduce cost-robust contracts. As our main theoretical contribution, we characterize optimal cost-robust contracts through direct correspondence to optimal composite hypothesis tests from statistics, generalizing result of Saig et al. (NeurIPS 23). We evaluate our framework empirically by deriving contracts
a range of objectives and LLM evaluation benchmarks, and find that cost-robust contracts sacrifice only marginal increase in objective value compared to their cost-aware counterparts.
Introduction
Modern-day LLMs are showing increasingly impressive capabilities, and simultaneously becoming increasingly costly. With rising success handling complex tasks, conversational AI systems are seeing ubiquitous usage across critical domains such as healthcare [19, 34], financial risk assessment [27], and law [24, 35]. To achieve such levels of performance, contemporary LLM architectures contain billions and even trillions of parameters, leading computational pipeline that requires dedicated facilities and substantial energy to operate [33].
Due the high computational requirements of modern LLMs, language generation tasks are typically outsourced to commercial firms which generate text for a fee. These firms either maintain dedicated infrastructure optimized for inference workloads, or act as intermediaries that facilitate access to such resources. To address the tension between performance and computational costs, such firms typically have multiple service options, each offering different trade-off between model quality and cost [1. 7, 36, 37]. Currently, the most common pricing scheme for such services is pay-per-token token, in which users agree advance to pay a fixed rate for each token of text generated by the system [10].
While simple and intuitive, the pay-per-token pricing scheme creates a misalignment of economic incentives between the firms and their consumers, known in the economic literature as moral hazurd: As inference is performed internally and fixed price is agreed upon in advance, firms strategically increase their profit margin by generating text using cheaper, lower-quality model. Due to the stochastic nature of language generation, consumers may be able reliably determine the quality of the model being used, exposing them to this kind of hazard.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Moral hazard is especially prevalent in cases where the text generation task is complex, and so evaluation is hard: Consider a scenario where a healtheare provider hires a firm to use conversational Al for summarizing medical notes. As medical diagnosis is an intricate and critical task, the healthcare provider wishes the medical summaries to be generated using the most advanced language model. Under the pay-per-token pricing scheme, the healthcare provider agrees in advance to pay the firm a fixed amount for each token generated. However, it is not hard to imagine that the firm may attempt to increase profit margins by routing some of the summarization requests to cheaper language models, instead of the most advanced one, without taking into account their purpose, and knowing that any lower-quality results would be attributed to the stochastic nature of LLM inference.
In the economic literature, the canonical solution to moral hazard problems is for-performance, or P4P [17]. Instead of paying fixed price for any outcome, the parties agree in advance on contract that specifies differential payment scheme for example, agreeing in advance that the firm will receive higher pay when the generated text is considered to be of higher quality. When designed correctly, contracts incentivize rational agents to invest more effort, thus providing a way to align incentives. Interaction around contracts is modeled as principal-agent game, where the principal commits to a payment scheme, and the agent responds by rationally selecting utility- maximizing action. Within this framework, the principal seeks to design a contract which satisfies some notion of optimality, such as requiring the least amount of expected pay ("min-pay contract"), or the lowest budget ("min-budget contract").
From pay-per-
pay-for-performace
In this work, we extend the theory of contract design, and use it to design optimal pay-for- for-performance pricing schemes for delegated text generation. Applying contract design this setting requires us to overcome the challenges of automated evaluation and cost uncertainty. The former stems from the need for a scalable measure of performance to support pay-for-perte -orf---rfformacce pricing, while the latter arises from the principal's 's uncertainty about the agent's 's true internal cost structure, as commercial firms often regard operational costs and implementation details as proprietary information.
Our results. To tackle automated evaluation, we draw upon recent advances in the L.MM evaluation literature [9], and propose a modular contract design framework which uses LLM evaluators as subroutines. More specifically, upon receiving generated text, our pricing scheme is implemented by evaluating the prompt-repponse pair using an automated evaluator and paying accordingly. The choice of evaluator can be tailored to the task: optimal pricing schemes in code generation tasks, for example, would rely on a pass/fail code evaluator [11, 4], whereas evaluation of linguistic tasks can be achieved using an "LLM-as LLLM-as-a-jddee judge" approach [41, 28, 25]. In our theoretical analysis, we show that our framework is applicable even to intricate tasks where current evaluation methods are noisy and undecisive, as the principal can compensate for the noise by paying more (Proposition 1).
To address the challenge of cost uncertainty, we propose new notion of cost-robust contracts, which are pay-for-performacce schemes guaranteed to incentivize effort even when the internal cost structure is uncertain. Our main theoretical contribution is a statistical characterization of optimal cost-robust contracts (Theorem 1): We prove direct correspondence between optimal cost-robust contracts and statistical hypothesis tests by showing that the min-budget and min-pay contract objectives correspond to minimax risk functions of composite hypothesis tests (Type-1+Type-2 errors and FP/TP, respectively). This significantly generalizes a recent result by Saig et al. [31] to arbitrary action spaces and multiple optimality objectives. The statistical connection provides intuition and interpretation for numerical results, and the applicability to multiple objectives allows system designers to accommodate different business requirements. Intriguingly, the relation between the optimal contract and the optimal statistical risk have the same functional form in both objectives (min-buce buineb and min-pay). Moreover, multiplying optimal hypothesis tests by a constant whose value depends only on the statistical risk yields spproximately-optinaa contracts (Theorem 2).
Finally, we evaluate the empirical performance of cost-robust contracts by analyzing LLM evaluation benchmarks for two families of tasks. In the first experiment, we compare the performance of two-outcome contracts across code generation tasks with varying difficulty; results show that what determines the pricing scheme is the relative success rates of the models, not the task difficulty. the second experiment, we compute multi-outcome contracts for an intricate conversational task evaluated via LLM--s---judee judge. Numerical results show that the optimal monotone cost-robust pricing scheme has an intuitive 3-vell structure: pay nothing if the quality is poor, pay extra if it is exceptional, and pay a fixed baseline otherwise. We show our framework's 's flexibility by providing comprehensive comparison across various contract objectives and simplicity constraints.
2

---

Principal
Agent
Interaction
Text Generators
Commit pay [m] Rao
Prompt wle
generator 8 - a(t)
Generate: resporse aty ((ng) at cost [ploig]
Automated Evaluator
Response dig
Evaluate quality q(sip mig) E [mm]
Payment r(a)
Receive (()) monetary units
Figure 1: Interaction protocol. Principal commits to pay t(g) monetary units according to response quality, and sends prompt wo: Agent selects text generator g (L), and generates response wn = g(wo) at cost ag]wa]l: Principal evaluates response quality g WR). and pays accordingly.
1.1 Related work
Our main technical tool is algorithmic contract design (sce [5, 20, 14] and subsequent works). Many works in this area address distributional robustness, e.g. [8]. [14] which also studies approximation guarantees of simple contracts, and the recent [3] which presents distributionallly-rbuu contract knowledge, none address cost-robustness.
design approach for delegation of learning. However, Connections between contract design and statistics have long been known to exist at a high level (see, e.g.. [32]), and were recently explored by [6] in the context of adverse selection, and [31] for two-action min-budget contract. From a technical standpoint, our work is closest to [31], which only proves the statistical connection for the special case of two-action min-budget contracts. Finally, we note that our cost-robustness framework is general, and our characterization results may be of independent interest. Additional related work appears in Appendix A.
2
Problem Setting: Contract Design for Text Generation
We study the delegation of a text generation task from a strategic principal to agent, with a payment scheme designed to incentivize quality. Here we formulate the problem as contract design instance.
2.1 Quality text generation (agent's perspective)
The core of our setting is standard language generation task. Let V be vocabulary of tokens, and denote the set of all token sequences by V* A text generator g V* - V* is a mapping from textual prompt Wo to a response wR. We assume that prompts sampled from distribution w/ ~ D E A [V*), and denote by D, the distribution of prompt-response pairs, where the prompt sampled from D and the response is generated by generator g. Given a prompt and generated response, a quality evaluator is a function q V x V > [m] which scores the response on scale of 1,..., m. We use Fg denote the distribution over scores [m]] induced by applying the quality evaluator to a random pair (Wo, WR) ~ D,, and Fai to denote the probability of score j E [m]]
The agent has access to collection of possible text generators refer to for convenience by their indices [n]. Each model g. E 9 is associated with a model-dependent cost a; 2 0, which is the average cost (borne by the agent) of generating single token from gi- For convenience we write D; Dg. and F; Fg.- Denote by & = ag (n) [| the expected cost of using the ith generator. We assume w.l.o.g. that the costs non-decreasing, i.e.,
(91....,, .........0... .h.) which also
e1 <... < Ca. and that they reflect the inherent quality of the models. In contract design terminology. the generators are the agent's possible actions. The agent can choose a single (pure) action, distribution over text generators E A (G) known in game theory as a mixed action.' The cost ei of the lowest-cost action is the agent's 's "opportunity cost", and unless stated otherwise e₁ 0.2
As abstract contract design problem. The above setting is precisely a contract design setting with n actions and m outcomes [21]. Such a setting is defined by the pair (F,c), where F is an n x m matrix with distribution F; as its ith row for every i (known the distribution matrix), and where is a vector of costs. For every action i, F, and c, are the outcome distribution and cost, respectively.
For example, the agent can generate responses using larger model for 95% of requests, and apply the smaller model for the remaining 5%, corresponding to the mixed action a (0.05, 0.95). *Choosing the first action can be thought of as opting out of the task at cost e1 If c: = 0 then the agent participates in the contract only if the expected utility is non-negative a property known as individual rationality.
3

---

may-for-performande and agent's utility. To incentivize high quality text generation, the principal commits in advance to may-for-performant
contract, which specifies the amount of payment to the agent for generating response with a certain quality. More formally, given a quality evaluator with an output scale 1,..., m, a contract : [m]] + R2 is a mapping from the estimated quality to the size of monetary transfer. Note that transfers are non-negative; this standard restriction is known as limited liability, and it mirrors the fact that when a principal hires an agent to perform a task, money flows in one way only (from principal to agent, and not vice versa). If transfers are increasing with score, we say & is a monotone contract. Monotonicity is not without loss of generality, but is a desirable property as monotone contracts are generally simpler and easier to explain [14].
Given a contract E I2%0 and an action o A (g), the agent's 's expected utility u u o) (a.k.a the agent's 's profit) is the difference between the expected reward and the expected cost of text generation:
( σ) =
[f(q(wo, wx)) - as [ull = Eg:
[t(j) - c.),
where (wa, wx) ~ D, are the prompt and generated response, t(q(w), Wx)) is the payment transferred to the agent based on the quality of response, and ag [ is the agent's 's cost of generating the response. We assume the agent is rational and therefore selects, when facing contract t, an action (t) which
maximizes their expected profit (also known as the agent's 's best response):
(t) arg max (t) rEA(S)
As is standard in contract theory, we assume the agent breaks ties consistently and in way that agrees with the principal's 's preferences." 3 The interaction model is summarized in Figure 1.
2.2 Designing the contract (principal's perspective)
We assume that the principal seeks to obtain text generated by the model 9n E G, the most advanced model with the (strictly) highest associated cost C,. > Ca-1- We refer to In as the target action, i.e. the action which the principal wishes to incentivize. Taking the role of the principal, our goal is to design the "best" contract /* that incentivizes the agent to generate responses using the target model In- This is formalized by the following optimization problem:
t" arg min
||+]]
s.t. o(t) ōgn.
(1)
where ||4|| is norm of L representing the principal's economic objective (see below), and 6g. is point-mass distribution over text generators, supported by the target generator Ja- We denote the set of contracts incentivizing action 9n by T(gn) {t E 2 I (t) = .g. }. and further note that the assumption of a single target action serves as foundational step towards more complex contract design scenarios (see Appendix B.
Information structure (who knows what). The agent's available actions G and the possible scores [m] are known to both players. As the quality distributions F, can be learned from past data, we assume they are known to both principal and agent. As the costs of inference }a;} depend on internal implementation details, we assume the costs are known to the agent but uncertain to the principal. We thus aim for contract optimization framework which maximizes different types of objectives, and allows for optimization of even when the costs incurred by the agent are uncertain to the principal.
Objectives: min-budget, min-pay pay and min-variance contracts. In eq. (1), different norms |||]] correspond to different possible optimization goals of the principal: For example, a contract is min-pay if it incentivizes the target action using minimum total expected payment Ej-p,, [(())] among all contracts in T(gx) [14]; In eq. (1), this corresponds to the ε norm weighted by the quality distribution of the target action. Similarly, contract is min-budget if it incentivizes the target action using minimum budget B, = max, t()) [31]; In eq. (1), this corresponds to the foo norm. Additionally, studied to our knowledge. A min-vat variancr contract minimizes the objective Var(t), corresponding to weighted & norm. Optimal contracts for these objectives can be computed in polynomial time by solving a corresponding linear or convex-quadratic program (see Appendix D. We also consider approximately-optimal contracts:
also consider natural nni-vvaiancce objective, which was not previously
Definition 1 (7--optimal contract). Let 1) 2 1. For contract setting (F,e), let t' E T(gg) be the optimal contract with respect to objective |[4]]. A contruct i E T(gn) is oppptiaal if ||4|| S 1 [|]|]]
'n our context, this means that if action g., is a best response for the agent, then the agent will choose (t)) that plays gn with probability I (see Section 2.2).
4

---

3 Hypothesis Testing and Contracts
This section sets the stage for connecting cost-robust contracts to statistical tests in Section 4.
3.1 Preliminaries
Simple hypothesis tests Consider two distributions Fo: Fi A ([ml). Given [m] which is sampled from either Fo or Fi, hypothesis test is a function db : [m]] + [0, 1] which outputs if is likely to have been sampled from Fi, and 0 otherwise* In the hypothesis testing literature, Fo is simple null hypothesis, and P1 is a simple alternative hypothesis. Performance measures of hypothesis tests are derived from the probabilities of making different types of errors: For a test y, the probability of false positives FP = j=1 Fo,j measures the rate of type- errors; This is when the test rejects the null hypothesis despite the sample being drawn from Fg. Similarly, the probability of false negatives FN = شك= Fl v/y)) measures the rate of type-2 errors, i.e. when the test does not reject the null hypothesis despite the sample being drawn from Fi. We also denote the true positives by TP = Σラ= Fly; TP is also known as the test's power, and equal to -FN.
m
Composite hypothesis tests. Consider now two sets of distributions {Fr)zz). (Fr), where F3 E A ([ml) for all E [n]. In hypothesis testing terms, {Fa [F&)R=] is composite rull hypothesis. {Fr) a simple alternative hypothesis as before, and composite hypothesis test té outputs given [m]] is likely to have been sampled from F,. To define performance in the composite case, we denote by FP, = سØ= Feoyb the standard type-1 error between distributions F& and Fn. As the alternative hypothesis is still simple, definitions of FN and TP remain as before, using F, as the reference distribution. To measure the performance of hypothesis tests, it is common to take a worst-case approach, and define the composite FP as the standard type-1 error FPx against the worst-case distribution Fx in the null hypothesis set.
3.2 Risk and minimax tests
To formalize the notion of worst-case error, let sb be a composite hypothesis test for [Fk)R=] k= ( (Fa) For any [n 1]. define risk function Tk : [0. 1] R20 be mapping from & to a risk score, treating ab as a simple hypothesis test between distributions Fi and Fn. A natural way of measuring risk is by combining the test's two error types. One measure is the sum of errors, denoted by (1)) : FP + FN. A classic result by Neyman and Pearson shows that (xb))) is minimized by the likelihood ratio test for any fixed & [29]. Another measure the ratio of false positives to
true positives, denoted by (1bb == FPx / TP. To generalize a risk measure тж to a composite hypothesis test, we adopt the worst-case approach and define r(v) == max (v))). Thus, R(v) max [ } = FP + FN, and p(t) = maxxe(n- [ FP TP.
Definition (Minimax hypothesis test). Let 3 be a composite hypothesis test for [Fi]ez], {Fr). and fix risk function rk- The test V is minimax optimal w.r.t if minimizes the worst-case risk:
bp arg min max (rx(v)) arg min(r()))+ ve kE(n-1)
1e[0.
The minimax sum-optimal test and minimax ratio-optimal test are the minimax optimal tests with respect to the sum R and the ratio p. respectively.
Observe that the optimal risk of both types of tests is bounded by r(t) < 1, the constant test vi = 0.5 satisfies R(e)) = p(t) = 1. We assume at least a small difference between the hypotheses, such that r(v) < 1. This allows us define contracts based on these tests.
3.3 From tests to contracts and back
We derive "statistical" contracts from hypothesis tests by multiplying them by a function of the risk, and derive "contractual" tests from contracts by normalizing them: Consider a contract setting (F, c). with either known costs and b == C1, or cost upper bound b 2 Cn e₁ Fix a risk function and corresponding budget function B, b) E R.
Test-to-contract: Let be test for sets (F (F,) with budget (, b) E oo). The corresponding statistical contract {(r,)) is (, b) <b.
Contract-to-test Let t be a contract. The corresponding contractual test vft is t/ ||0|
'When (()) is fractional, consider the output of the test to be 1 with probability +0). and 0 otherwise.
5

---

Economic objective
Objective function
Statistical objective
Risk function
Min-budget
FP + FN
Pre. (v = 1) + Pre, (vb 0)
Min-pay
Ey [ty]
FP/ /TP
Prry (0)1) Pre, (W=x)
Table
Correspondence between cost-robust contracts and hypothesis tests, arising from Theorem 1.
We are interested in the following statistical contracts corresponding to the tests from Definition 2:
Definition Consider a contract setting (F, c), with either known costs and == Cn - C1 or cost upper bound b 2 -1. The sum-optimal statistical contract (bx(b) V is obtained from the minimax sum-optimal test R multiplied by (b) := . The ratio-optimal statistical contract
(b) V is obtained from the minimax ratio-optimal test 3b multiplied by (b) TP(W:)-PPWP))
4 Cost-Robust Contracts
In this section state and prove our main result- direct connection between composite hypothesis testing and cost-robust contracts. Consider a contract design setting (F, c) with increasing costs a1 S... S Cn-1 < Cu. where n. is the target action. In real-world settings, the principal may not have full knowledge of the agent's internal cost structure. We model this by assuming the principal is oblivious to the precise costs, but knows an upper bound b 2 Cn C1. We are interested in robust contracts that incentivize the target action for any cost vector compatible with the upper bound:
Definition 4 (Cost-robust contracts). Consider a distribution matrix F and bound b on the costs. Let Co be an ambiguity set of all increasing cost vectors e such that C₂ - C1 < b. A contract is b-cost-robust if implements action n for any cost vector c Cs.
Informally, our main theoretical result shows that optimal cost-robust contracts are optimal hypothesis tests up to scaling, where the scaler depends on the risk measure which the test optimizes. Our approach can be applied to several notions of optimality, and each optimality criterion for contracts corresponds to a different optimality criterion for hypothesis tests. Specifically, we derive the correspondence for min-budget and min-pay optimality of contracts. Formally (recall Definition 3):
Theorem 1 (Optimal cost-robust contracts). For every contract setting with distribution matrix F and an upper bound b on the (unknown) costs, let Vi (resp., 15)) be the minimax sum-optimal (ratio-optimal) test with risk R (p*) among all composite hypothesis tests for {Fw)R=1- {Fa}. Then:
The sum-optimal statistical contract (b) VRR is b-cost-robust with budget b/(1 R*), and has the lowest budget among all b-cost-robust contracts.
The ratio-optimal statistical contract (b) < is b-cost-robuse with expected total payment b/(1 )²), and has the lowest expected total payment among all b-cost-robust contracts.
Table summarizes the contract vs. test equivalences arising from Theorem 1. In the special case of contract design settings combining (i) binary action space (n = 2), (ii) zero-cost action (c. = 0). and (iii a tight upper bound (b = c2). the first half of Theorem 1 recovers recently- discovered correspondence between hypothesis testing and (non-cost-robust) two-action min-budget contracts [31, Theorem 2]. Theorem is more general since it applies to any number of actions as well as the standard min-pay objective. Thus, Theorem can also be seen as extending the interpretable format of optimal contracts for binary-action settings beyond two actions.
In Appendix D.2, we derive our main lemmas, which are used in Appendix D.4 to prove Theorem 1. Our proofs rely on two main assumptions. The first assumption is that the cost uncertainty bounds are known; as these bounds become looser, the required budget and expected payout increase linearly. The second assumption is that the contract design problem is implementable - i.e., that there exists some contract incentivizing the target action. In particular, implementabilityy holds when text generated by the target model has the highest quality in expectation (see Appendix C.I). Generally, contract design problem is implementable when the observed quality distribution of the target model can't an't be emulated by combination of alternative models at a lower cost (see, e.g., [14]).

---

4.1 Additional properties of optimal cost-robust contracts
In this section, we focus for concreteness on min-budger cost-robust contracts, and establish their approximation guarantees as well as their functional form under structural assumptions. First, in analogy to [14], it is natural to examine the approximation guarantees of cost-robust contracts. We show (recall Definition I):
Theorem (Approximation guarantees). For every contract setting (F,c), ler 0 < a be lower and upper bound on the difference between the larget cost and any other cost, i.e., (Ca - c;) E |a, for all i E [n -11. 1]. Then the min-budget b-cost-robust contract for (F, c) is "ooptmmll with respect to the budget objective [[o] and the approximation ratio & is tight.
Proof in Appendix D.5. As corollary, combining this result with Theorem 1 shows that statistical contracts are approximately optimal in the global sense: For any contract setting (F, c) with corre- sponding minimax sum-optimal hypothesis test vh the contract T- V is 17-optimal with respect to the budget metric and Cn-Cs cost-robust contracts.
We also note that similar results hold for min-pay
We next turn to consider the functional form of optimal cost-robust contracts (i.e., why their payments are as they are). One of the criticisms of optimal (non-robust) contracts is that the payments seem arbitrary and opaque. Compared to this, cost-robust contracts are more transparent and explainable. We show two additional results regarding their format, leveraging the connection to minimax hypothesis testing.
The first result explains the budget: By the minimax principle, there is a "least favorable distribution" " (to use terminology from statistics) or, equivalently, a mixed strategy over the rows of F (to use terminology from game theory) such that no test can achieve for it better risk than the minimax risk R We show that the budget of the optimal cost-robust contract can be interpreted using this distribution. Formally, let [| lV be the total variation distance between distributions F1, F2. then the budget is as follows (see Appendix D.5 for a proof):
Proposition (Distribution distance determines budget). For every contract setting with distribution matrix F and spread of costs Cn C1 b, the minimum budget of a b-cost-robust contract is maX) 6/ F Eien F
The distribution Eien A; that maximizes the above expression is the least favorable one. Intuitively, the closer it is to the target distribution F,, the larger the budget needed for the agent to distinguish among them and prefer the target action. We also note that the required budget approaches infinity when the worst-case distribution distance approaches zero, coinciding with the implementability characterization known from the literature (see Appendix C. The second set of results adds standard structure to the distribution matrix to obtain even simpler contract formats:
Definition 5 (Monotone Likelihood Ratio (MLR)). A distribution matrix satisfies MLR F..//FF. is monotonically increasing in j E for all >>>> i'
Intuitively, if F satisfies MLR, then the higher the outcome j. the more likely is to origin from more costly distribution F; than from F, (recall that costs q are increasing in i). Consider minimax composite hypothesis tests for {Fi k==- {Fx }; if F does not satisfy MLR, optimal such tests may require randomization (i.e., bb E (0, 1) for some outcome j) and/or non- mon-ooottonicity (i.e. 3 >> Bhyy for some pair of outcomes < j'). However, if MLR holds for F, then nice properties (determinism. monotonicity) hold for minimax tests (e.g.. by the Karlin-Rubin theorem [29]), and consequently also for cost-robust contracts (see Appendix D.6 for proof):
Propasition 2 (MLR induces threshold simplicity). For every contract setting with distribution matrix F that satisfies MLR, and with spread of costs 6n C1 S b, the min-budget b-cost-robust contract for F is monotone threshold contract, which pays full budget to the agent for every outcome j above some threshold j"
For min-pay rather than bin-budget, under MLR we get monotone contract with a single positive payment, which is optimal (see [14, Lemma 7J). Finally, we note that finding cost-robust min-budget contracts with two levels of pay ("all-or-nothing" [31]) computationally hard in the general case, as the reduction by [31, Theorem 3] also applies in the cost-robust case (see Appendix D.8).
7

---

MBPP outsomes
contracts
MT-Bench outcomes
MT-Bench contracts
0.4
Model
HEval
12.5
min-hulget
700
1:3
MBPP
Probabiity
705
13B
10,0
min
7.5
EvalnEval outcomes
Faymunt 12
0.3
5.0
Model
130
2.5
708
0.0
0%
20%
43%
60%
coot-robust Cost-anare
4
Quality
Quality
Figure 2: Empirical evaluation results. (Left) Outcome distributions and optimal contracts for Code Generation data, Section 5.1. (Right) Outcome distributions and optimal contracts for MTT-Bench data, Section 5.2. For the contracts plot, solid lines represent cost-robust contracts, dashed lines represent cost-aware contracts, and dotted lines represent threshold contracts.
5 Empirical Evaluation
We evaluate the empirical performance of our cost-robust contracts using LLM evaluation benchmarks.
We compute binary and multi-outcome contracts for two distinct families of tasks based on evaluation scores from known benchmark datasets, optimizing the contract objectives set forth in Section 2.2. Our action space consists of the 7B, 13B, and 70B parameter model versions of the open-source Llama2 and Code LLMs [37, 30). which share the same architecture and hence similar inference costs. The benchmark data is used to create an empirical outcome distribution for each LLM in the action space. In both cases, contract optimization targets of the largest model variant (70B), which is the most performant and costly. Implementation details are provided in Appendix E.3, and code is available at: https: //g thub. com/ eeweeeign mampsaaagggmmmmmmmma contracts.
Cost estimation. To estimate the inference costs of the language models, we leverage their open-source availability. We use energy consumption data from the popular Hugging Face LLM- Performance Leaderboard [22, 23]. which we then convert to dollar values using conservative cost estimates (see Appendix E. As a first-order assumption of cost uncertainty, we assume that infer- ence costs of alternative generators are bounded from below by the cost of the most energy energy-efficient alternative model (c1), and bounded from above by the cost of the alternative model with the highest energy consumption (cx-1).
5.1 Binary-outcome contracts across tasks (code generation)
We begin by analyzing a simple contract design setting across benchmarks of varying difficulties. We use the LLM task of code-generation which has m 2 outcomes: pass or fail. The analysis of binary outcome space is motivated by the following theoretical property:
Proposition 3. For any contract design problem with = 2 where the most-costly (target) action has the highest pass rate, the optimal contract is identical for all optimality objectives (min-pay, min-budget, and min-variance). Moreover, the optimal contract satisfies fpass > tiail O.
Proof in Appendix D.7. Proposition 3 allows us to compare performance across different evaluation tasks without being sensitive to the choice of contract objective and constraints such monotonicity.
Datasets. We use evaluation data from two distinct benchmarks, which represent differing degrees of task difficulty. The Mostly Basic Programming Problems (MBPP) benchmark [4] contains 974 entry-level programming problems with 3 unit tests per problem. The HumanEval benchmark [11] consists of 164 hand-written functional programming problems. Included in each programming problem is a function signature, a doc-string prompt, and unit tests. There is an average of 7.7 unit tests per problem, and the overall score based on pass/fail evaluation of the responses. For each of these benchmarks, we create binary-outcom (m 2) contract from the pass rates of the CodeLlama model family (CodeLlamd ama- (7B, 13B, 70B}). We use the pass@ values from the CodeLlama paper [30] (success rates for single response), as they capture setting where the agent gets paid for each response.
8

---

Cost-aware
Cost-robust
E
Budget
stdev(t)
[[/]
Budget
stdev(t)
Min-Pay 4.19
10.6
5.2
4.82
(+15%)
12.2
5,98
Min-Budget
4.73
6.16
1.71
5.48
6.63 (+1.7%)
1.83
4.73
6.16
1.71
5.48
6.63
1.83 (+6.7%)
Table 2:
Monotone contracts curated the MT-hench dataset. Costs in units of S/1M tokens, and are order-of-magnitude estimates based on typical prices of electricity (see Appendix E.1). The numbers in bold denote the relative increase from the optimal monotone contract that the cost- robustness sacrifices in each setting. The percentages denote the price of cost-robustness: how much the objective values increase relative the cost-aware setting.
Task difficulty and optimal pay. Figure 2 (Left-Center) presents optimal cost-aware and cost- robust contracts for code-generation. We observe that cost uncertainty entails a consistent increase in payment across tasks: For MBPP, we observe 14.9% increase, and for HumanEval we observe similar 14.7% increase. Additionally, while the MBPP task is easier than HumanEval (i.e. character- ized by higher pass rates), the resulting contracts for MBPP are more expensive. This demonstrates the fundamental connection between contracts and statisties: In MBPP, there is smaller gap between the performance of the target model (70B) and the performance of the alternatives. This makes the highest-performing model harder to detect, increasing the cost of the contract. The required payments this case thus depend on the absolute differences between pass rates, rather than absolute values.
5.2 Multi-outcome contracts
To understand the relation between different optimality objectives and constraints, we analyze optimal contracts in an expressive multi-outcome (m = 10) environment based on MT-Bench
Dataset. The MT-Bench benchmark [41] is designed to evaluate the conversational and instruction- following abilities of LLMs in multi-turn (MT) conversational settings. The benchmark consists of 80 prompts in the format of multi-turn questions, and the evaluation dataset includes LLM-as-a-jude- evaluations on pprompt,responsee
pairs from various models, using GPT-4 the judge. In the dataset, each (prompt.response) pair is given discrete response quality scores in the range 1..... 10. These scores define contract outcome space. In consistence with the analysis in section 5.1, the outcome distributions of (Llama-2- 2- 13B, 70B} -chat, and target the 70B model. Outcome distributions for the MT--Bench dataset are presented in Figure
Simple optimal contracts. In practical applications, contracts with simple functional form are often preferred since they are easier to comprehend. We compute optimal contracts with two types of simplicity constraints: monotone contracts (weakly increasing payout), and threshold contracts (full budget for all scores above a threshold, and zero otherwise). Results are presented Figure (Right), and Table 2. For the min-budget and min-variance criteria, monotone cost-robust contracts have an intuitive three-level structure: Zero pay for outputs with the lowest quality score, base pay for intermediate scores, and extra pay for outputs with the highest quality score. While threshold contracts may resemble current pricing schemes more closely (see Appendix E. monotone contracts enable lower overall budget while still maintaining simple functional form. For min-pay, monotone cost-robust contracts are in themselves threshold contracts, however they may deter risk-averse agents they only pay for highest-quality outputs. In Appendix E.2, additionally analyze con-onotoone contracts, and show that further economic efficiency can be achieved by sacrificing simplicity.
Price of cost-robustness. Table 2 compares cost-robust and cost aost-aware monotone contracts across different performance metrics. We observe that cost-robust contracts setting sacrifice a marginal increase in objective values: 15% increase in the min-pay objective, an 1.7% increase if optimizing for budget, and 6.7% increase when optimizing for minimum variance. We refer to Appendix H.2 for further analysis of cost-robustness in the mo--ooooton setting.

---

6 Discussion
In this paper, we introduce cost-robust contracts as a means to address the emerging problem of moral hazards LLM inference. Our aim to offer flexible payment schemes that ensure integrity in current LLM markets, even when facing challenges of incomplete information. One of the key insights from our study is that cost-robust contracts can be relevant and effective in practical settings. Moreover, we generalize the work paved by Saig et al. [31] by uncovering stronger connections between the fields of contract design and statistical hypothesis testing. These connections underscore the statistical intuition that is prevalent in contract design.
Despite the promising results, our work still has several limitations that would do well to be addressed in future research. For one, the data we capture through the evaluation benchmarks does not accurately reflect real-world distributions, where the prompt space is much richer. A natural direction for future work is to explore approximation guarantees when learning contracts from data. Additionally, our analysis relies on set assumptions regarding the cost uncertainty and estimations, which should be carefully considered when designing contracts for Generative AI. Lastly, it would also be interesting to see our contract design framework applied to markets with a more elaborate action space.
The authors would like to thank Nir Rosenfeld, Ariel Procaccia, Stephen Bates, and Michael Toker for their insightful remarks and valuable suggestions. Eden Saig is supported by the Israel Council for Higher Education PBC scholarship for Ph.D. students in data science. This work received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant No.: 101077862, project: ALGOCONTRACT. Pl: Inbal Talgam-Cohes by the Israel Science Foundation (grant No.: 3331/24), by the NSF-BSF (grant No.: 2021680), and by a Google Research Scholar Award.
Acknowledgements.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint #rXiv:2303.08774, 2023.
[2] Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
[3] Nivasini Ananthakrishnan, Stephen Bates, Michael Jordan, and Nika Haghtalab. Delegating data collection in decentralized machine learning. In International Conference on Artificial Intelligence and Statistics, pages 478 486. PMLR, 2024.
[4] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.0773: 077322 2021.
[5] Moshe Babaioff, Michal Feldman, Noam Nisan, and Eyal Winter. Combinatorial agency. Journal of Economic Theory, 147(3):999-10) 1034, 2012.
[6] Stephen Bates, Michael I Jordan, Michael Sklar, and Jake A Soloff. Principal-agent hypothesis testing, arXiv preprint arXiv:2205.0688 2022.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[8] Gabriel Carroll. Robustness and linear contracts. American Economic Review, 105(2):536-563, 2015.
[9] Yupeng Chang, Xu Wang. Jindong Wang, Yuan Wu, Linyi Yang. Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang. Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3): 1-5,, 2024.
[10] Lingjaa Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.051 05176, 2023.
10

---

[11] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et Evaluating large language models trained on code. arXiv preprint arXiv:2107.03332 2021.
[12] Steven Diamond and Stephen Boyd. CVXPY: A Python-embeddded modeling language for convex optimization. Journal of Machine Learning Research, 17(83):1-5, 2016.
[13] Paul Duetting, Vahab Mirrokni, Renato Paes Leme, Haifeng Xu, and Song Zuo. Mechanism design for large language models. In Proceedings of the ACM on Web Conference 2024, pages 144-155, 2024.
[14] Paul Dütting, Tim Roughgarden, and Inbal Talgam-Cohen. Simple versus optimal contracts. In Proceedings of the 2019 ACM Conference Economics and Computation, pages 369-387, 2019.
[15] Sara Fish, Paul Gölz, David C Parkes, Ariel Procaccia, Gili Rusak, Itai Shapira, and Manuel Wüthrich. Generative social choice. arXiv preprint arXiv:2309.0/299, 2023.
[16] Paul J. Goulart and Yuwen Chen. Clarabel: An interior- point solver for conic programs with quadratic objectives, 2024.
[17] Stuart E Greene and David B Nash. Pay for performance: an overview of the literature. American Journal of Medical Quality, 1402),140-163, 2009.
[18] Keegan Harris, Nicole Immorlica, Brendan Lucier, and Aleksandrs Slivkins. Algorithmic persuasion through simulation: Information design the age of generative ai. arXiv preprint arXiv:231111813., 2023.
[19] Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. arXiv preprini arXiv:2310.056622 2023.
[20] Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan. Adaptive contract design for crowdsoureing markets: Bandit algorithms for repeated principal-agent problems. Journal of Artificial Intelligence Research, 55:317-359, 2016.
[21] Bengt Holmström. Moral hazard and observability. The Bell journal of economics, pages 74-91, 1979.
[22] Régis Pierrard Ilyas Moutawwakil. Lim-perf leaderboard. https://huugiingffcc. cttpaacce..c./pppeeeeeeeeeeeeeee co/spa ves/oppimum// /11m 1llm-perf--f Leaderbardd 2023.
[23] Régis Pierrard Ilyas Moutawwakil. Optimum-benchmark: A framework for benchmarking the performance of transformers models with different hardwares, backends and optimizations., 2023.
[24] Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip Yu. Large language models in law: A survey. arXiv preprint arXiv:23 2023.
[25] Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Ze Fan, Hai Zhao, and Pengfei Liu. Genera- tive judge for evaluating alignment. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[26] Linyang Li, Pengyu Wang, Ke Ren, Tianxiang Sun, and Xipeng Qiu. Origin tracing and detecting of lms. arXiv preprint ar0iv:2304. 12022444444 2023.
[27] Yinheng Li. Shaofei Wang, Han Ding, and Hang Chen. Large language models in finance: A survey. In Proceedings of the Fourth ACM International Conference on Al in Finance, pages 374-382, 2023.
[28] Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 2511 2521-2522. Association for Computational I Linguistics, 2023. URL. httph:tt/////..tt haap:/:///cnaathoooyyyo org/ org/2023. 2023. emnlp emnlp-main. 153.

---

[29] Phillippe Rigollet and Chn-Christian Hütter. High dimensional statistics. Lecture notes for course 18$997, 8 81:(81)):46, 2015.
[30] Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code Ilama: Open foundation models for code. arXiv preprint arXiv:2308. 12950, 2023.
[31] Eden Saig, Inbal Talgam- and Nir Rosenfeld. Delegated classification. Advances in Neural Information Processing Systems, 36, 2024.
[32] Bernard Salanié. The Economics of Contracts: Primer. MIT press, 2017.
[33] Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. From words to watts:
Benchmarking the energy costs of large language model inference. 2023 IEEE High Performance Extreme Computing Conference (HPEC), pages 1-9. IEEF, 2023.
[34] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620(7972):
2023.
[35] Zhongxiang Sun. short survey of viewing large language models in legal aspect. arXiv preprini arXiv:2303.09) 00036,660000066 2023.
[36] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint (rXXiv:3302.1971 2023.
[37] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint
2023.
[38] Adaku Uchendu, Thai Le. Kai Shu, and Dongwon Lee. Authorship attribution for neural text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8384-8388 8395, 2020.
[39] Xianjun Yang. Liangming Pan, Xuandong Zhao, Haifeng Chen, Linda Petzold, William Yang Wang, and Wei Cheng. A survey on detection of lIms-generater content. arXiv preprint arXiv:2310. 15654, 2023.
[40] Xianjun Yang. Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. DNA-GPT: divergent n-gram analysis for training-free free detection of gpt-generated text. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria,
May 7-11, 2024. OpenReview.net, 2024.
[41] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024.
12

---

A Additional Related Work
Detection, As a possible alternative to a contract-design approach, the LLM content detection literature develops tools which attempt detect machine-generated text, and distinguish between different text generators [38, 26, 40, 39]. Using such tools, a principal could deploy an LLM content detector and penalize firms who are not labeled as using target text generator. From this perspective, contract design is a complementary approach which provides guidelines for positive incentives in case generated text gets accepted, an approach considered more effective at encouraging participation. Additionally, our pay-for-performance framework supports richer outcomes spaces beyond binary pass/fail, enabling more granular, and thus more efficient, control of incentives.
AGT and LLMs. On a broader perspective, our work further promotes the role of Algorithmic Game Theory in the economics of Generative AL. Previous works include: Duetting et al. [13] who design auctions that merge outputs from multiple LLMs; Harris et al. [18] who offer Bayesian Persuasion setting where the sender can use Generative Al to simulate the receiver's 's behavior; and Fish et al. [15] who leverage the creative nature of LLMSs to enhance social choice settings.
B
Extensions
B.1 Targeting a set of high-quality models
For high-stake tasks such summarizing medical information, it makes sense target the most advanced model. other scenarios, specifying a single target action can act as an intermediate step toward the final contract design.
For instance, a principal might aim to incentivize the use of any model that meets a certain quality threshold, c.g., requiring text generation from any LLM with more than 70B parameters. Formally, given a set of models G = {9....... In. with associated costs 6₁ < C2 <... < Cn. we assume that higher-cost models offer higher quality. Let & E [ni] represent the index of the minimum quality model that the principal seeks to target, such that the goal to incentivize any model 9: E {9kk....
To compute the optimal contract in this setting, the principal enumerates over the different target models gi E {gk: ............. ... designs an optimal single-target contract for each, and selects the "best" contract among the resulting designs formally, the contract minimizing ||4]] for the appropriate norm (see Section 2.2). Since all enumerated contracts are designed to satisfy incentive compatibility and potentially cost-robustness, the optimal contract among them also satisfies these properties.
Contract Implementability
C.1 Conditions for implementablity
The implementatbility of min-pay contracts is discussed in [14, Appendix A.2], and the imple- mentatbility of min-budget contracts is discussed [31, Appendix B.3. both cases, the implementability of the contract design problems is characterized by the following condition:
Proposition 4 (Implementablity; [e.g.. 14, Proposition 2D). In a contract design problem (F, c) with possible actions, an action i E [m] is implementable if and only if there is no convex combination of alternative actions that results the same outcome distribution Ex yi A F. F; but lower cost
Evzi Cp < Cz.
We note that Proposition 4 holds for all objectives ||t]] described in eq. (I), as feasibility only depends on the incentive compatibility constraints. Adding to the result above, show that implementability holds in the the intuitive case where the target model has the highest expected quality:
Proposition 5 (Highest expected quality implies implementability). For contract design problem (F, e), denote the expected quality of action by q Eyer, l.). If:; < qn for all i < n, then the contract is implementable.
Proof. By contradiction. Assume that 4 < gn for all i < n but the contract design problem is not implementable. By Proposition 4, there exists convex combination of actions such that Eien X,F, = F,, and E, G- < Gn- Denote the convex combination of distributions by Fx. By definition, it holds that:
E E ErBByabl
Σ Apa
ien
13

---

But from the infeasibility assumption Eicn A.Fi Fn, and therefore it also holds that:
Ejwex b6 = By~FF [s] =
and therefore Eien X On. which contradicts the initial assumption that < for all
C.2 Designing cost-robust contracts for stricil-interrmedatee target actions
Our main analysis technique for cost-robust contracts requires a separation between the interval covering the costs of target actions, and the interval covering the costs of alternative actions (see proofs Lemma and Lemma 3). This assumption does not hold when the principal targets strictly intermediate model, formally Di G such that i < т. and < This captures scenarios where principal seeks to incentivize text generation from medium-sized model, but not from larger, costlier models that are available. In such instances, guaranteeing the implementability of cost-robust contracts requires additional assumptions.
To illustrate this, consider the following design setting, with actions and m outcomes:
Fi (1,0)
C 0
F2 10
02
P3 (0.5, 0.5)
C3 10
Assume that the principal targets action 8" = 2. In this setting, the cost difference is upper-bounded by oz- G S C2 -i = 1, and the cost-robust min-budget contract corresponding to the bound 6 =
given by Theorem 1:
(- P3 FallTV
0, 0.5
(0, 2)
This contract is incentive compatible with respect to the target action, yielding expected utilitics (t, 0 for action 1, (t,2)) 1 for action 2, and u. (f,, -9 for action 3.
However, consider the following setting, with identical target and different outcome distributions:
Fi (1,0)
Ci
F2 (0.5. 0.5) F3 (0, 1) C3
C2 =
In this setting, an incentive-compatibe contract exists (e.g., / = (0, 4)), and the same cost upper bound b holds, however the cost-robust contract design problems are unfeasible by Proposition as in the truncated contract design setting (F = (F1, F2, F3). c.onst (0, 1, 0)) there exists a convex
combination of alternative actions A1 = 0.5, X3 = 0.5 which satisfies Eie(1,3) A,E, = F2 and
(ie(1,3 dzc < 1.
D Deferred Proofs
D.1 Convex programs and equivalent forms
In this section, include linear programs (LPs) for optimizing contracts and hypothesis tests. Non-negativity constraints on the variables are ommitted where clear from context.
By definition (see Section 2.2), contract is min-budget with respect target action i if and only if it is an optimal solution to the following MIN-BUDGET LP, where IC stands for incentive compatibility (i.e., the constraints that ensure the agent's best response to / choosing action i):
min
B
s.t.
Eesty - cv < Σ Figt
Wi' ti (IC)
(2)
)
t < B
Vj E [m]
(BUDGET)
14

---

Proposition 6 (Equivalent form to the MIN-BUDGET LP; [31, B.2]). When eq. (2) is feasible, the variable transformation (t, (t (#/8,1/3) yields an equivalent LP which we refer as the statistical L.P:
maX *E|0,11 DE
B
(3)
s.L. Σ Posis + B3)-
#i
Similarly to the MIN-BUDGET LP, have the MIN-PAY LP:
min
Fut
th
(4)
Pesls cv S PP #i (IC)
-
There are also natural LP formulations for hypothesis testing. hypothesis test & is the minimax sum-optimal test w.r.t risk (see Definition 2) if and only if is optimal solution to the following
LP:
min "E(0,I]"".ER
(5)
8.b.
Deerh + +
1255) Sr
vinn
Proposition 7 (Dual of statistical LP).
The dual of eq. (3) is given by:
Σ Mj
min
AER ..ERR
20
20
jE[m]
8.
Σ (Fn -Fi5)
A S4j
fA E [m]
(6)
Σ (cn a) 21
M4 20 0
Vj E [m]
A 2 0
Vi E |n - 1]
Proof. Denote: == -1h1 (tb1....,,, vm..8) E m=++
Fi,1 Fn.
F1 Fi.m mm
Fn.m
On C1
Fig Fo.i
Cn Ca
A
F(n-1),1 Fn.1
F(m-11,m 1),m
Fu,m
Cn C2-1
Imxm
= (O....,. 0 1.....1 E RR" 1+m
times
times
0, Rm+1
15

---

Using these notations, eq. (3) can be written as:
cº
max
#20
(7)
s.t.
Ar S
The symmetric dual LP of eq. (7) is given by:
min b" y
#20
(8)
s.t.
A y 2
Denote:
y == (A1..... An- M1, M1........ pm)
Unpacking the matrix notations in eq. (8) yields the following:
Σ M2
min AERR-
PER
jE/m]
s.t.
Σ (Fa.) Fn. d H33 20
Vj E [m]
K<n
Σ (Gr - -) 21
i<n
which is equivalent to eq. (6).
Proposition 8. A min-variance contract is optimal solution for the following quadratic program:
min
m
s.t.
Fests -r S Fitt Vi' +
(IC)
Where is positive semi-definite matrix depending on the target action distribution F,.
Proof. Denote the contract by E IR and the probability distribution of the target action by p E A ([m]). We use the following matrix notations:
P1
P1
p
:
diag (p)
Pm
Pm
Pm
The variance of t is given by:
Var (t) = Egnp (tt - -E(t))]
2
Σr, P3 11 Σ Pklk
= diag (p) [t - 1p 1
= |iaag (vp) (I 1p" t|l
Denote R = diag (vp) (I - 1)7), and V = RT R. Then:
Var (t) = |Rt||
t RT Rt
Note that V R. R is a Gram matrix. therefore positive semi-definite, and the quadratic program is convex.
D
16

---

D.2 Main lemmas
The next lemmas are the workhorses of our theoretical results. We use T(F,c) to denote the set of contracts incentivizing the target action are also known as the feasible solutions of the setting. For simplicity we focus on settings for which the set feasible solutions nonempty (i.e., the target action is implementable). Given either non-decreasing cost vector c= (CI, Cn-1, Cn)
contract design setting c); the contracts in T(F.c)
R2 and an index k [n 1]. cost On and
constant c'
Cu. define
(k)
(Ck, (Chy Ck, Cn) 20i count = (c,..,,, d,,,,,,,)
These vectors with uniform costs (up Cn)) Note that the costs in c(1) are (weakly) lower than those in c, and vice versa for c(n-1);
<cs (csc(m1)) c(n-1)
Where the relation S is defined element-win wise. Intuitively, since the agent gravitates towards lower costs, it is harder to incentivize the target action against lower costs. We formalize this as follows:
Lemma (Incentivizing against lower costs is harder). Consider a distribution matrix F, and two cost vectors с S Z Ro0 satisfying = En (i.e., c is dominated by i and the cost of the target action coincides). Then the sets of feasible solutions for contract design settings (F,c) and (F,c) satisfy T(F.s) S ST(P)))
Corollary 1. For every contract design setting (F,c), the set T(r.e) feasible solutions satisfies T(F.D))) E T(r.e) E ((F
Consider now a contract setting (F, where the action costs are uniformly equal to d except for the target action (which is more costly). We show that for such setting, the optimal contract for incentivizing the target action has an interpretable format closely related to hypothesis testing. Recall the notions of sum-optimal and ratio-optimal statistical contracts from Definition 3; then:
Lemma 2 Minn-bddee optimality in uniform-cost settings). For every contract design setting (F, const)) the min-budge contract coincides with the sum-optimal statistical contract Bx c') Vi and the optimal budget is cd)((1 R'')
Lemma 3 (Min-pay optimality in uniform-cost settings). For every contruct design setting (F, crosst, the min-pay contract coincides with the ratio-oplimal statistical contract (2, - cd) - .. and the optimal expected total payment is (c₂- cc)/ p²))
Proofs appear in Appendix D.3, establishing also the other direction:
Observation 1. Let (F, const) be contract design setting. Then the minimax sum-optimal test among the composite hypothesis tests for {F FR)) {Fa) is obtained by normalizing the min-budget contract, and the minimax ratio-optimal test is obtained by normalizing the min-pay contract.
D.3 Proofs main lemmas
Proof of Lemma I. For target action and any alternative action í n, the (IC) constraint of the MIN-BUDGET LP (eq. (2)) is given by:
Σ Fots - - -4 - Fagly - Cn.
Rearranging the terms yields:
Σ (Fly -Fy- 5
(10)
Let 7 be cost vector satisfying e < and Cn = Zn. The costs are assumed to be increasing in k. and therefore 7 Tn C2 Cn for all Moreover, as e; S 7, for all < n, the RHS of eq. (10)
satisfies:
<.
RHS of (F.
RHS (F,
Hence, the (IC) constraints of the e) contract design problem are more restrictive than the (IC) constraints the (F, c) design problem. Since the design problems (F,c). (F, e), only differ the RHS of the (IC) constraints, the sets of feasible solutions satisfy the desired inclusion relation. L
17

---

Proof of Lemma 2. Under the "statistical" variable transformation (t, B) - (158.1/8), the MIN- BUDGET LP for (F, cronst) is given by eq. (3):
max # ReR20
B
E Pups + Σ Fns(1 11:1-1)))))) 1b))) S - (cn - c3)3 Vi< n
s.t.
Applying the variable transformation r = - (cn - c)} yields the following equivalent LP:
min rE(O.I]".rER>>>
s.t. Porus + Pn - 15)) Sr vinn
This LP is equivalent to the minimax sum-optimal test VR in eq. (5), and therefore the optimal solution zi" is precisely this test. By the same equivalence, the optimal value of the optimization parameter satisfies r- R,, where Re is the minimax risk of the testing problem (i.e., the risk of (1.. By construction, the optimal B satisfies 3* is B" + f- which in the notation of Definition 3 is (c, c'). Reversing the variable transformation we get 30 /3* B" ui", which equal the sum- cum-optimal statistical contract (G, d)- WR as required.
1-r"
1-R" Ca c.-c and therefore the minimal budget
ca-c
Proof of Lemma 3. For the min-pay contract design problem, introduce an auxiliary variable 3 > and define a "statistical" variable transformation + Eume'' t., where x E [0, 1)m. The MIN-PAY LP (eq. (4)) transforms into:
min " 1800
On -
Σ
Fngtis
B
(11)
s.l.
3 s Σ
Pniyii - Σ Pagals
4k E [n 1]
For any given vh the optimal value of is:
8 min ke [E(n=1] 1]
Σ Faj
Σ
Fkydy
Σ Pasiis
Σ Fas
max kE sE(n-1)
Therefore, eq. (11) is equivalent to:
E, Fns cd) (c--c) E, Fag2is max E, Fu
min vE(0._1
(cn
(12)
Divide the numerator and the denominator by E, Frgily to obtain the transformed objective:
Σ, Engyy
E, Frgdi - maxxe(n-1) E, FlyDi
Σ, Fiss V/
maX Σ, Faj
maxxe[n-11] (1())
And hence eq. (12) can be written compactly as:
Cn d
min
(13)
maxxe(n- Pk (1))
18

---

The optimal solution for eq. (13) is the minimizer of maxke[n-11] Pa(()), which is equivalent to the minimax ratio-optimal test dp by Definition The optimal expected pay is 9- and the optimal contract is given by:
Cn
On (P( - FP(())) V.p. = Be - 6) vp.
C'
gr
where (-) is as in Definition 3. We conclude that t" is the ratio-optimal statistical contract, as required.
D.4 Proof of main theorem
We are now ready prove our main theorem:
Proof of Theorem I. We prove the first half of the theorem, i.c., that the sum-optimal statistical contract is b-cost-robust and has the lowest budget 6/(1 - R²) among all b-cost-robust contracts. The second half of the theorem follows by swapping Lemma with Lemma
We first show that the sum-optimal statistical contract is b-cost-robust, i.e., incentivizes the target action for every cost vector in the ambiguity set C: Define c := (0,.... O, b). By Lemma 2, the min- budget contract for the setting (F, c²) is the sum-optimal statistical contract (Tz i.e., BTx(2 e)).vhr where 7 the minimax sum-optimal composite test for distribution sets (Fije(nnuj- { {Fn). Its
(F.)
budget is (b -)* = b/(1 -.))
-
In particular, 1 incentivizes the target action and so belongs to T(P.e9). Observe that any increasing cost vector with C.. dominates the cost vector co, and therefore by Lemma 1, contract R also belongs to T(F.a) for any such cost vector e that dominates c.. Furthermore, any cost vector c in the ambiguity set C. has corresponding cost vector in which all costs identical except for On I c b. Lowering the target action's 's cost from T, to Cn can only help incentivize it, thus we conclude that 'R T(P.e). as required.
We now show optimality of the budget: Since tğ is the min-budget contract for the setting (F,c²) c.), and since e" is within the ambiguity region C, it holds that any b-cost-robust contract must satisfy B, 2 b/(1- R.). ). As satisfies this bound exactly, it has the lowest budget among all b-cost-robust contracts.
D.5 Proof of properties of optimal cost-robust contracts
Proof of Theorem 2. Let ca = (cn, - a.... On a, Cn) and c = (cn, bb... ......... On b, cr)) be two uniform-cost profiles; for brevity we refer to these as e , Since in contract setting (F, c)
it holds that (c,, c;) E [a, b] for all i E [n 1]. we have that c; S e S o for all Thus by Lemma I it holds that
T(r..) S T(FEe"))
(14)
that is, any contract that incentivizes the target action in setting (F, c) will incentivize it also in setting ())
Consider the min-budget contracts for settings (F,c-"), (F, c), (F,c*))) Denote their budgets by Bi BiF.o-a): Bir B Big.e- respectively. We deduce from Equation (14) that
B(F BiF. "
(15)
since the min-budget contract for (F, c) is feasible for (F,c-"). Now recall that Lemma 2 gives us an expression for the optimal budgets of the two uniform-cost settings. This expression depends the risk R' of the minimax sum-optimal hypothesis test for { (F,), which static across the two settings. It also depends on the difference between the highest and lowest cost in each setting. Thus:
B BiE a
(16)
B BiP,a-")
Combining Equation (15) and Equation (16) we get : BiFr) 2 Big.e-y- We conclude that the min-budget contract for setting (F,c-b) is m-min-budget contract for (F,c), By Lemma 2 the
19

---

min-budget contract is the sum cum-optimal statistical contract I-R VR which by Theorem is the b-cost-robust contract with the lowest budget, as required.
We now turn to the claim of tightness. Consider the following contract design setting:
F = (1, 0)
F2 1 (c)
F3 = (0,1)
Where costs are increasing 61 < 02 < C3, and is parameter satisfying:
C3 C2
(17)
C3 C1
The target distribution F3 is only supported on j = 3, and therefore the minimax sum-optimal test for this setting is:
/ (0, 1)-
As F3 do not overlap, the minimax risk given with respect F2 by the Neyman-Pearson Lemma [29]:
R F2 Fallrv
c,
and therefore the approximate contract given by Theorem 2 is:
Cn 3/
C3
t
0.
1 R
As for the optimal contract, it satisfies t* (0, B*) because the target distribution only supported
on j = 2, and it has threshold form due to [31, Lemma 4]. The optimal budget is:
C3 C2
B" max C3 Ct
When satisfies Equation (17), the optimal budget B*
02 and therefore:
C3
C1
|t|loc
03 C1
C3 02
C3 C2 E
B
On
C
||
- Cn-1
required.
Remark (Extending the proof of Theorem cost-robust min-pay contracts). By Lemma 3, the min-pay contracts for the design settings (F,c (F, c- defined in the proof depend linearly on a, respectively, and thus their expected pay is also linear the bounds. The inclusion argument in eq. (14) holds for min-pay contracts as well, and thus argument analogous to eq. (16) can be constructed for the ratio of expected payments. The rest of the proof follows similarly.
Proof of Proposition 1. By Theorem and Lemma 2, the b-cost-robust contract with minimum budget is the min-budget contract for setting (F, cconst where counst (O,..., b). For this setting,
plugging variables A, into the dual in eq. (6), we get:
Σ 74
min
ÀERB n- 20
PERm
20
jE/m]
s.t.
Σ (Fn.
F..)) d $ j5
Vj [m]
Σ
65, 2
20

---

Define the following variable transformation:
bā;
1 bj.
Under this transformation, the dual LP is equivalent to:
min A 20 MERT
20
jc|m]
s.t.
Σ
(Fr.)
Vj [m]
A21
When the contract is implementable, the optimal solution the primal statistical LP (eq. (3)) satisfies 3 >> O, corresponding to the last constraint in the dual LP. Therefore, the last constraint of the dual
tight due to complementary slackness:
Σ4=1, icn
and the LP is equivalent to:
Σ Pj /E(m]
min XEA((=-1) PER 20
s.t.
Fn.
Σ ixn
X.Fi, S
Vj E [m]
As 2 O we can write:
Σ
Fig.j Σ
min XEA((n-I)
5E[m]
ien
and by the definition of total variation distance (e.g by [31, Claim 4J), the optimization objective satisfies:
Σ
Fn. ΣxFis)
Fw Σ X
/E/m]
t<n
Applying the inverse transformation yields:
Σ
jE(m)
jE(m)
minxes((a-1)
Pr Eien Fie
Then, by strong LP duality 3 E3 Ai. and the final result is obtain by applying the nonlinear This gives the desired expression for the minimum budget B* of
1. g.. b-cost-robust contract: maxxes b/ //I|F. Fr Eien A:FlllTv
variable transformation B" =
D.6 MLR
In this section, we prove that cost-robust min-budget contracts for distributions satisfying the Mono- tone Likelihood Ratio (MLR) property have threshold functional form.
Proof of Proposition 2. Let (F, be contract design setting with e, ca < 6, such that F' satisfies monotone likelihood ratio. By the -arlin-Rubin theorem, the hypothesis test for the composite hypotheses {F& 1==1 (Fa) minimizing FP FN is threshold function, and therefore there exists jo E [m] such that (f) [ 2 jo]- By Theorem 1, the optimal contract in this case of the form t" Bb* for some scalar B > 0, and therefore t" is a threshold contract.
21

---

D.7 Two-outcome settings
Proposition 9. Let (F,c) c) be two-outcome contract design setting (m = 2). A contract (to, t1) with t to implements the target action if and only the contract t' (0, t1 to) implements the target action.
Proof. For any action i (n 1], the corresponding (IC) constraint is:
Σ
figh
Σ
Ingli
(18)
jE(0,1)
JE(0.1)
As fi, fn are probability distributions, the following holds for any to:
Σ
fi,t0= = Σ fasto
(19)
JE(O.1)
jE(O.1)
Subtracting eq. (19) from eq. (18) does not change the (IC) constraint, as both sides of eq. (19) are equal. Performing the subtraction and rearranging the terms gives:
Σ jE(0.1)
fallts to) - S
Σ jE(0.1)
Inilts to) -cn Cn
which is equivalent to:
(ti to) S fn, (ti - to) Cn
and this is the (IC) constraint for the contract t' = (0, t to). Therefore, the contract is feasible if and only the contract e' is feasible.
Proposition 10. Let (F, c) be two-outcome contract design setting (m 2), and let (to, t1).
Then the contract t' = (0, t - (o) has weakly-better expected pay, weakly-better budget requirements, and the same variance as t.
Proof. For the min-pay objective, we obtain from linearity of expectation:
Ejes, It, - tol S Eyer, Ity] and therefore has weakly-better expected pay. For the min-budget objective, it holds that
max {0, t1 to} S max (tu, +1}
and therefore i' has weakly-better budget requirement. for the min-variance objective, adding constant to a random variable does not affect its variance:
Var(t) Var(t')
and therefore has the same variance as
Proof of Proposition 3. For all [n], let Bernoulli(ps.. As any contract & is two-dimensional vector. By proposition 9, proposition 10, holds that the optimal contract is of the form 1" (0, (0,t)) for any of the three objectives. To prove that the optimal payment t is the same for all objectives, observe that all objective functions are monotonically-ineree
in t
max t" t Ey-n, [t'] = fn,iti
(Required budget)
(Expected pay)
Var(t") fn.1 fn,1) (ti)²
(Variance)
Since the optimization problems are of single variable with identical (IC) constraints, their optimal solutions all identical.
D
22

---

Llama2 cost (S/IM tokens)
CodeLlama cost (S/IM tokens)
Model size
7B
$0.182
S0.183 $0.24
13B
$0.24
70B
$0.64
$0.64
Table 3: Estimated energy costs for the Llama2 and CodeLlama model families, according to the methodology described in Appendix E.1.
D.8 Hardness of all-or-nothing cost-robust contracts
While cost-robust contracts can be computed in polynomial time by solving the corresponding convex programs (see Appendix D.I), restricting the functional form of the solution to have only two levels of payment entails computational hardness:
Definition 6 (All-or-nothing contract; [31]) A contract has all-or-nothing finctionna form if there exists B such that t, E B3 for all j E [m].
In [31, Theorem 31, it is shown that computing a min- mhin-budget all-or- oilthing contract is NP-hard. We show that the same reduction is applicable for min-budget cost-robust contracts:
Proposition 11 (Hardness). Computing min- min-budget all all-or-nothing contract is NP-hard.
Proof. By reduction from 3SAT. Given 3-CNF formula, show that there exists cost-robust contract design problem such that the required budget of an all-or-nothing cost-robust min-budget contract below a certain threshold if and only the formula is satisfyiable.
First, given a 3-CNF formula, apply the polynomial-tin time reduction described in [31, Appendix B.5.3] construct min-budget contract design problem (F, c). Denote the target action of the design problem by n E [n]. By the construction described in [31, Equation (27)), it holds that c, = 1 and n. Thus, the contract design problem tightly satisfies the cost difference upper bound Cn S 1. Moreover, since all alternative actions have the same cost = 0, the cost vector it is also the worst-case cost vector in the cost uncertainty set induced by the bound b (by Lemma 1). Therefore, (F is also cost-robust contract design setting for the bound
G for all
By the proof of [31, Theorem 3], there exists threshold By such that the required budget of an all-or-nothing min-budget contract in the design setting (F, c) is below certain threshold if and only the 3-CNF formula is satisfiable.
E Experineets/Empirrcal Evaluation
E.1 Inference Costs
To calculate the costs for each model, we energy data from the Hugging Face LLM Performance Leaderboard The energy efficiency for each model is expressed in the leaderboard in units of output tokens per kWH. To convert actionable costs we assume a rate of .105 $/KWH, aligning with conservative energy costs in the United States and giving us order-of-magnitudee approximations of the actual inference costs. The inference costs are presented in units of S/1M tokens. The leaderboard data was taken from the experiments on the A100 GPUs. For each model, we took the PTTT-4bi++xxl.
v2 quantization benchmark. Table 3 shows the energy costs on the leaderboard for Llama2 and CodeLlama. We note that energy data was missing for CodeLlama-70B, so we extrapolated from Ll amma--7B-chht..
Model verbosity. Calculation of the per-token inference costs are not complete without an analysis of the response length produced by the various models. Table 4 shows the average verbosity (output length) of the Llama models on the single-turn prompts in the MT-bench evaluation set. Since the values are of the same order of magnitude, we simplify and assume that the choice of model does not influence the verbosity, and therefore we do not include this our cost calculations.
hrr leaderboard.
23

---

Model
Verbosity
Llama-2-7B-coe
1625
aaam-2-13B-chat 1573
Llama-2-70B-chat 1695
Table 4: Model verbosities (average output length) of the models in consideration. Since the values are of the same order of magnitude, we assume for simplification that the choice model does not significantly affect verbosity (see Appendix E.
Cost-aware
Cost-robust
[
max lj
sldev(()
[1]
max l
stdev(()
Min-Pay 0.86
73.4
7.63
0.92
(+6.9%)
73.4
8.16
Min-Budget 2.48
3.59
1.60
2.78
3.91 (+8.7%)
1.70
Marranrience
3.52
6.31
1.45
3.84
6.58
1.53 (+6.5%)
Table 5: Cost-aware vs Cost-robust contracts in the non-monotone setting. The numbers in bold denote the optimal values achieved for the 3 objectives. The percentages denote the relative increase from the optimal that the cost-robustness sacrifices in each setting.
Current market pricing schemes described Section 1, current market pricing schemes for LLM generation involve pay-per-token rates for which the user pays regardless of the response quality. For open-source models such as Ll ama2, there exist API services to run model inference, such as AWS and Microsoft Azure. In other scenarios, some pricing schemes behave threshold contracts: an unsatisfied user may request from the API to regenerate a response free charge, and hence will only pay if the response quality is above some threshold. For this reason threshold contracts can offer a "satisfaction guarantee" while retaining a simple form.
E.2 Multi-outcome contracts: Further Analysis
Non-Monotone Contracts Table 5 shows the statistics of the various contract objectives in contracts when optimized without a monotonicity constraint, and displays how they match up to each other in cost-aware and cost-rok cost-robust settings. We observe that the min-pay contract minimizes expected pay at the expense of high budget and variance. The min-budget contract, on the other hand, is not the worst in any of the objectives. Additionally, the cost-robust setting sacrifices only marginal increase in objective values: a 6.9% increase in the Min-pay objective, an 8.7% increase optimizing for budget, and 6.5% increase when optimizing for minimum variance.
Price of monotonicity It is of interest to analyze the relative difference in resulting contracts that occurs due to removing the monotonicity constraint. Table 6 shows the discrepancy in contract objectives for cost-robust contracts. We can observe that while the monotone contracts as whole are simpler, more intuitive, and closely resemble threshold contracts, it is not without cost as they suffer a sizeable increase in contract objectives, most notably an increase of 388% when trying to minimize expected pay.
Non-Monotone
Monotone
E[(]
max L
stdev(t)
max 1
stdev(t)
Min-Pay
0.92
73.4
8.16
4.82
12.23
5.98
Min-Budget
2.78
3.91
1.70
5,48
6.63
1.83
Min- Variance
3.84
6.58
1.53
5.48
6.63
1.83
Table 6: Cost-robust contracts, monotone non-monotone setting. The numbers in bold denote the optimal values achieved for the 3 objectives.

---

E.3 Implementation details
Code. We implement our code Python. Our code relies on cvxpy [12, 2] and Clarabel [16] for solving linear and quadratic programs.
Code is available at: https:/ / htt///githbbb www
Hardware. All experiments were run on single Macbook Pro laptop, with 16GB of RAM, and M2 processor, and with no GPU support. Overall computation time is approximately one minute.
implement cost-robustness of a contract setting with costs (c1, (21,C2 C2...... Cn), assume knowledge of only the range of costs, and calculate the contract using costs (0, .0., c1). This modeling assumption provides us with the flexibility of solving contracts in settings without call-information while maintaining the approximation guarantee set forth in Theorem 2.
Implementation of cost-robustness.
E.3.1 Contract design solvers
To compute optimal contracts, we implemented the following solvers:
Convex programming solvers: Given outcome distributions {fii and costs {c),, we solve the MIN-PAY LP (eq. (4)), the MIN-BUDGET LP (eq. (2)), and the MIN- VARIANCE QP (eq. (9) using cvxpy. All optimization programs enforce incentive compatibility (IC) constraints for the target action n against all other actions E [1, n 1]. We note that the Clarabel solver supports both linear and quadratic programs,
To find threshold contracts for problems with low-dimensional outcome and action spaces (such as with MT-bench), we implement a brute-force solver which performs full enumerations of all possible thresholds, as proposed by [31]. We refer to the Full Enumeration inn Solver in [31, Appendix C.2.1] for further implementation details.
Threshold contract solver:
25

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: See Section and Appendix C.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: See Appendix C, and Appendix
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Section 5, Appendix E, and attached code.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See Appendix E.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
9. Code Of Ethics
26

---

Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://ttturpsss.ccco///////// htt.p//s...cpploatctccccccoooaao
Question:
Answer: [Yes]
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section and Section
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Licenses existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
27


Reviews:
[{"review_id": "Y5BndYhabh", "reviewer": "Reviewer_RQsK", "summary": "* The paper concerns the problem of incentivizing LLMs to use the most costly model, which is assumed to be the model with the best performance. Without proper incentive, the LLM company has the incentive to charge customers for the highest payment, but deliver the service using a lower-cost model, because the performance of the model is usually not verified. Therefore, the paper proposes to use contract design to solve this problem. In particular, an automatic detector first gives an integer score for the performance of an LLM. Then, the task is to design a payment for the company for each of the integer scores. The goal is to minimize the total payment conditioned on incentivizing the best-performed model. \n* The main contribution is the discussion of the cost-robust contract, meaning how to design the optimal contract while the costs of LLMs are unknown. Empirical evaluations have shown how to use the theory in practical settings given LLM performance data.", "questions": "See weakness", "limitations": "Limitations are reasonably stated.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "* (I’m not an expert in contract design.) I appreciate the theoretical contributions of the paper. To me, section 4 has several interesting insights into connecting cost-robust contracts with hypothesis tests. As claimed, this is the first paper considering cost-robust contract design. However, the real contribution should be better evaluated by experts.\n* In general, incentive issues of LLM uses have been very critical and challenging. I also like the connection between contract design and the production of LLMs.", "weaknesses": "Although I believe contract design can speak with the production of LLMs, I’m not fully convinced that the proposed model is a good idea to solve the considered problem. \n\n* In practice, each company pricing its own AIs, so who should be the principal? In other words, the paper assumes there is a trust-worthy third party who can run the quality-detector and commits to a contract with the LLM companies. I’m not sure this is feasible in practice. I hope the authors can explain more carefully the application scenarios of their theory.\n* Furthermore, there is no evidence in the paper (and, I guess, on the Internet) that can prove LLM companies are cheating about their service quality. I also don’t think this is very likely because LLM companies have other incentives to provide high-quality services, e.g. their reputation. So, how do we know we are not solving a problem that does not exist?\n* Even though we go with the assumption that there is a contract that the company (agent) agrees on, I doubt cost-robustness is the first-order concern. The cost data is usually publicly obtainable from energy reports, as the authors did in their experiments. Even though this data is not public, the energy cost is usually easy to estimate. Therefore, I don’t think incentivizing LLM production is a suitable application for cost-robust contracts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the insightful review! We address the points below:\n\n> **In practice, each company pricing its own AIs, so who should be the principal? In other words, the paper assumes there is a trust-worthy third party who can run the quality-detector and commits to a contract with the LLM companies.**\n\nThank you for this question. One application our solution applies to is cases in which the principal (a consumer, or a group) has some bargaining power. While this doesn’t cover all possible scenarios, in many cases the principal is in fact in a strong position: Large organizations typically have bargaining power as they license software in bulk (through \"enterprise licensing\"). Additionally, our results also suggest that users with common interests are likely to benefit from negotiating together, or purchasing LLM services through agencies with negotiation power as in the online ad market. We will update the paper to emphasize these aspects.\n\nRegarding the need for third-party evaluation, it is first important to note that evaluation can be performed locally by the principal in a verifiable way, and thus there is no need for a third-party. This applies naturally in important use-cases such as LLM-assisted code generation. For evaluation settings which are more intricate, following your remark we have extended our framework to support contracts with principal-side sampling, where the principal only evaluates a fixed proportion of results that are chosen uniformly at random (e.g. 1% of responses at random). This results in a contract which requires more budget but demands less evaluation resources. Allowing this trade-off can eliminate the need for a trusted third-party in additional key use-cases, such as evaluation based on an LLM-as-a-judge that runs locally. We will add a full formal description to the paper, together with supporting proofs.\n\nLastly, we also note that evaluator integrity is a common assumption in the contract design literature, and any economic/cryptographic solution to the evaluator integrity problem can be applied in our case as well. \n\n> **There is no evidence in the paper (and, I guess, on the Internet) that can prove LLM companies are cheating about their service quality. I also don’t think this is very likely because LLM companies have other incentives to provide high-quality services, e.g. their reputation.**\n\nWhile strategic misreporting in LLMs has not yet been officially acknowledged, there is ample evidence for strategic misreporting in similar industries with more mature markets, and related evidence suggesting that strategic behavior is also plausible in the context of LLMs:\n\n* In related industries, there are certainly examples of trust violations from companies who provide black box services - an extreme example is Theranos, and another example is throttling by internet service providers, which purposefully and intermittently slow down service.\n* For LLMs, there has already been a documented period of alleged low-quality service from ChatGPT around January 2024, dubbed “ChatGPT gone lazy”, which OpenAI acknowledged but never fully explained (see The Guardian, “What is going on with ChatGPT?”, Jan 12, 2024). With the current pricing schemes, it is not clear what will prevent such periods from recurring in the future.\n\nWhile indirect systems such as reputation or the litigation system allow for some level of quality assurance, contracts are increasingly being applied in real-world applications - such as pay-for-performance healthcare, or revenue sharing for internet content creators. For reputation, one of the issues is that quality is non-trivial to assess, and quality measures are noisy. Considering contracts in conjunction with complementary systems like reputation appears very promising for future work, and we are very grateful for this insightful remark.\n\n> **The cost data is usually publicly obtainable from energy reports, as the authors did in their experiments. Even though this data is not public, the energy cost is usually easy to estimate.**\n\nTo our knowledge, fine-grained cost analysis is only publicly available for open-source models. Our experiments use data from the open-source Llama-2 models with publicly-disclosed computational setups. In contrast, the full architecture and computational setup of the commercial models common today are not publicly disclosed (for example, GPT-4, Gemini, and Claude are all closed-source). For closed models, many factors affect the cost (architecture, batching policy, GPU type, exact energy prices, etc.), and many of them remain undisclosed - Motivating cost-robustness.\n\nMoreover, there is no doubt that simplicity and explainability are important practical concerns in pricing, and our contracts based on hypothesis tests achieve exactly these properties, while also having a deep theoretical justification as optimal cost-robust contracts. This mirrors seminal results in contract theory that justify the use of simple linear contracts due to their robust optimality - whether to the full action set [1] or to the distributional details [2].\n\n–\n\nWe would like to thank you again for the insightful questions and remarks! We hope this discussion clarifies the plausibility of moral hazard problems in LLMs, the applicability of contract design to them, and the significance of the contribution to the foundations of contract design theory. \n\nPlease do not hesitate to follow up if you feel some questions remain unanswered.\n\nReferences:\n* [1] Carroll, Robustness and Linear Contracts, AER 2015.\n* [2] Dutting et al., Simple vs Optimal Contracts, EC 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m9WzIwZdKt", "reviewer": "Reviewer_4RXi", "summary": "This paper addresses the issue of moral hazard in pay-per-token pricing for large language model (LLM) services. Firms may use cheaper, lower-quality models to cut costs, compromising text quality. Moreover, the firms's costs may be unknown to the clients. To counter this, the authors propose a pay-for-performance framework using cost-robust contracts that incentivize high-quality text generation under the uncertainty about the firms's costs. These contracts are designed based on and have a one-to-one correspondence to optimal composite hypothesis tests. Approximation guarantees are provided with respect to the optimal contracts. Empirical evaluations show that these contracts provide robust incentives with small cost increases.", "questions": "1. what is the computational complexity of finding the optimal cost-robust contract that incentivizes $c_n$ (equivalently, the complexity of finding the optimal test)?\n2. Can the characterization results and computational efficiencies be directly applied to the cases where actions with lower costs may be the best action to incentivize?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The results of characterizing the forms of optimal cost-robust contracts using hypothesis testing, as well as the approximation guarantees, are interesting and have valuable contributions.", "weaknesses": "1. The model's complexity is unnecessary. The problem could actually be studied in the most basic contract setting. \n2. The authors do not discuss the computational complexity of finding the optimal cost-robust contract. \n3. The authors do not discuss the cases where the action with the highest cost may not be the best action to incentivize.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful review, and for the excellent questions!\n\n> **What is the computational complexity of finding the optimal cost-robust contract that incentivizes (equivalently, the complexity of finding the optimal test)?**\n\nThis is a very good question. Optimal cost-robust contracts (and equivalently, optimal hypothesis tests) can be found by solving linear programs, and can therefore be computed in $\\\\mathrm{poly}(n,m)$ time, where $n$ is the size of the action space (number of possible text generators), and $m$ is the size of the outcome space (number of possible evaluation outcomes). This is a corollary of Theorem 1 and Lemmas 2-3:  By Theorem 1, the optimal cost-robust contract is equivalent to an optimal statistical contract, and by the lemmas this in turn is equivalent to a min-budget or min-pay contract in an appropriate uniform-cost setting. Equations (1) and (3) encode min-budget and min-pay contract LPs, respectively. \n\nFurther expanding on this point, it is also worth noting that while all optimal contracts presented in this paper can be computed in polynomial time, a common theme in the contract design literature suggests that complexity may change when restricting optimization to contracts with a \"simple\" functional form (for example, when restricting optimal contracts to have only two levels of payment). Following your remark, we looked into the hardness results of Saig et al. [1] for the full-information min-budget setting, and found that their result is also applicable to cost-robust contracts when their functional form is restricted to have only two levels of payment (Theorem 3 in Saig et al. [1] shows that computing a min-budget contract with two levels of payment is NP-hard in the full-information setting). We are thankful for this remark, and will include this additional insight in the paper.\n\n> **Can the characterization results and computational efficiencies be directly applied to the cases where actions with lower costs may be the best action to incentivize?**\n\nThis is an excellent question. Our model indeed relies on the assumption that the output of costlier LLMs has higher quality in general. While it doesn’t capture scenarios such as LLM fine-tuning, we believe that this is a reasonable starting point.\n\nA generalization of this approach currently appears in footnote 4 in the paper. Expanding upon that point, we note that our contract design scheme can be naturally generalized to scenarios where the agent seeks to incentivize any LLM above a certain quality threshold, rather than the single most costly LLM. Taking OpenAI’s products as an example, this corresponds to cases where the agent would like to incentivize text generation using GPT-3 and above, rather than GPT-3 strictly. This generalization still maintains the correspondence between cost-robust contracts and hypothesis tests, and the contract is computed by iteratively solving cost-robust design problems with a single target action. We will update the paper to include a detailed discussion of these aspects.\n\nBeyond this extension, our main analysis technique is not directly applicable, as it requires a separation between the interval covering the costs of target actions and the interval covering the costs of alternative actions (see Lemmas 2-3). In this regard, extending cost-robustness to problems with cost interval overlap seems to be a very interesting direction for future research. We will add to the paper a discussion of our theoretical techniques and their potential applicability in more general scenarios, as well as counterexamples that illustrate limitations.\n\n> **The model's complexity is unnecessary. The problem could actually be studied in the most basic contract setting.**\n\nWe are not sure what “most basic'' means here and apologize in advance if we misunderstood, but if you mean a setting in which the costs are known, we note that many factors affecting cost (such as model architecture, batching policy, and exact energy costs) remain undisclosed in practical commercial applications, motivating the development of cost-robust contracts. If this wasn’t the intention, please let us know. There is always a trade-off between simplicity and expressiveness when defining models, and having a concrete reference for a basic model will help us make a more precise comparison and clarify the relations.\n\n–\n\nPlease let us know if our response has addressed your questions regarding computational complexity, multiple target actions, and the conciseness of the model. If you have any further questions or thoughts, we are more than happy to clarify and discuss!\n\nReference:\n* [1] Saig et al., Delegated Classification, NeurIPS 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VALHYDRcuw", "reviewer": "Reviewer_Bg3E", "summary": "The authors formulate a theoretical setup for a LLM text generation service to incentivize the service to output high quality text the consumer. The authors formulate this setup as having the service having a set of models that has quality (as rated by a evaluator on the end of the consumer) that increases with the cost of running the LLM. The goal is to derive a framework for paying the LLM service based on that quality of the text generated that incentivizes the LLM service to always use its best model. The authors go about this by formulating the definition of a contract in this setting, and defining various metric (max payment, avg. payment, etc.) that consumer aims optimize. They show that the set of contracts that will incentivize the LLM service to output text with the best model can be derived from the set of optimal hypothesis tests that distinguish which model is being used from the evaluator outputs. They derive how the optimal contract can be formulated from these hypothesis tests, when only bounds are known on the costs of running each model for the LLM service.", "questions": "Should the principal know the outcome distribution for each generator? This seems slightly unrealistic since consumers only ever have black box access to the API, and never know precisely which model they're responses are from. They do know that previous agent actions could only be mixed over the old (worse) models though.", "limitations": "I think the limitations are sufficiently addressed.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The theoretical setup and monotone assumption of model performance, cost, etc. is quite reasonable and tackles and interesting and relevant problem with LLM queries. The results are simple and intuitive, and connect nicely with previous work on contract theory and hypothesis testing.", "weaknesses": "The main issue is the theoretical setup does require an assumption that the bounds on cost are known, which seems somewhat impractical. It might be useful to explicitly comment on how the different metrics degrade with increased looseness of the cost bounds (linearly, it seems like), since one can always pick extremely conservative cost bounds.\n\nMinor issue:\n\n- In Definition 3, it would be helpful to illustrate why $B_R^*$ and $B^*_\\rho$ are defined as they are, correctly. Further, the definition with $b \\geq c_n$ is a bit strange, since the definition of minmax hypothesis test does not involve worst case over cost vectors, so it doesn't seem correct to use $b$ to derive the minimax contract (and instead it should remain a function of $c_n - c_1$) --- maybe dropping that $b \\geq c_n$ case would be more accurate, since it is used in the definition of cost-robust later.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful and encouraging review! We address your questions and remarks below:\n\n> **The main issue is the theoretical setup does require an assumption that the bounds on cost are known, which seems somewhat impractical.**\n\nThis is a very good point. We note that virtually all robust optimality results in the mechanism design literature assume the principal knows something about the setting (e.g., a subset of available actions in [1], or the first moments of the distributions in [2]), and we believe that supporting uncertainty regarding cost is certainly better than assuming full knowledge as in the standard contract setting. We hope that the characterization of cost-robust contracts can serve as a building block towards the development of contracts which are robust in additional ways.\n\n> **It might be useful to explicitly comment on how the different metrics degrade with increased looseness of the cost bounds (linearly, it seems like), since one can always pick extremely conservative cost bounds.**\n\nWe will gladly add the analysis you suggest of degradation with looseness of the cost bounds (it is indeed linear).\n\n> **Minor issue in Definition 3: it would be helpful to illustrate why $B_R$ and $B_\\\\rho$ are defined as they are...**\n\nWe will also revise Definition 3 - thank you for these suggestions!   \n\n> **Should the principal know the outcome distribution for each generator? This seems slightly unrealistic since consumers only ever have black box access to the API, and never know precisely which model they're responses are from. They do know that previous agent actions could only be mixed over the old (worse) models though.**\n\nThis is an excellent question, which suggests a very interesting direction for future research. In our model, the principal is indeed assumed to know the score distribution of the generators, and indeed can never know (without further assumptions) whether the agent “cheated” in the past to influence this knowledge. \n\nIt seems that to gain from such cheating, responses should come from a weaker generator, to appear as if easy tasks require more effort from the agent than they actually do, and thus justify more compensation from the principal. In mechanism design, this is somewhat similar to systematically underbidding in auctions, so that the auctioneer will charge less in a future auction, never finding out the true value distribution of the bidder. From the game-theoretic perspective, formulating and studying a model with such non-myopic agent behavior is a very interesting question for future research. From the statistical learning perspective, it will be very interesting to identify distributional assumptions which are reasonable in practice, and also provide meaningful guarantees for learning from samples.\n\n--\n\nAgain, we would like to thank you for the insightful feedback! If any additional questions or thoughts arise, please do not hesitate to let us know.\n\nReferences:\n* [1] Carroll, Robustness and Linear Contracts, AER 2015.\n* [2] Dutting et al., Simple vs Optimal Contracts, EC 2019.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper takes a contract design approach for LLMs, proposing a pay-for-performance, contract-based framework for incentivizing quality.

The review team (including the AC) agrees that the paper makes a solid theoretical contribution to the literature, by offering an interesting statistical characterization of optimal cost-robust contracts by connecting them to statistical hypothesis tests. This contribution is perhaps of independent interest (thus applicable beyond the LLMs setting). 

On the other hand, the review team pointed out that there is little to no evidence on the misalignment of incentives in quality text generation. More generally, the application of cost-robust contract design to LLMs might seem somewhat superficial / not particularly suitable. Overall, the practical motivation of the problem and its applications can be significantly improved.

Despite these concerns, the review team weighted more positively the theoretical and conceptual contribution of the paper.
Decision: Accept (poster)