SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Using Prop. 1, a solution to Eq. (3) can be obtained by iteratively solving Eq. (4): For all  $n_i \in \mathcal{A}$ , solve Eq. (4) with target action  $n^* = n_i$ , and return  $t^*(n_i)$  for the best implementable  $n_i$  whose budget does not exceed  $B$ . The budget-optimal problem thus reduces to solving multiple min-budget problems.
To compute each  $t^*(n_i)$  in Eq. (4), we formulate a novel MIN-BUDGET linear program (LP),<sup>3</sup> detailed in Appx. B.2. One way to solve this LP is with generic solvers—an approach which is valid, but can be costly. One of our contributions is in identifying natural cases where min-budget contracts take on simple forms, which are easier to optimize, and have practical merit. In particular, we show that binary-action contracts have all-or-nothing structure  $(t(j) \in \{0, B\})$ , and plausible structural assumptions on the learning curve give rise to threshold contracts  $(t(j) = B \mathbb{1}[j \geq j_0]$  for some  $j_0$ ). Our theoretical results are summarized in Table 1, and detailed in the rest of the section.
3.3 All-or-nothing contracts: Binary action space and the statistical connection
We begin with a simple delegated learning setting in which the agent can choose one of two actions,  $\mathcal{A} = \{n_1,n_2\}$ , e.g., a 'small' vs. 'large' training set, and the principal seeks to incentivize training with more data. This reduced case will be useful as a building block for the general case (which we return to in Sec. 3.4), and for making a precise connection between contract design and hypothesis tests.
Simple min-budget contracts for binary action space. Our first result shows that optimal binary-action contracts are all-or-nothing contracts whose budget is determined by the total variation distance between outcome distributions  $f_{2}$  and  $f_{1}$ , namely  $\| f_{2} - f_{1}\|_{\mathrm{TV}} = \frac{1}{2}\sum_{j = 0}^{m}|f_{2,j} - f_{1,j}|$ .
Theorem 1 (Optimal binary-action contract). In a binary-action contract setting with outcome distributions  $f_{1}, f_{2}$  and costs  $c_{1}, c_{2}$ , the min-budget contract is an all-or-nothing contract, given by:
$$
t ^ {*} (j) = B \mathbb {1} [ f _ {2} (j) \geq f _ {1} (j) ] \quad \forall j \in \{0, \dots , m \}, \quad \text {w h e r e} \quad B = \left(c _ {2} - c _ {1}\right) / \| f _ {2} - f _ {1} \| _ {\mathrm {T V}}. \tag {5}
$$
The proof (in Appendix B.4.2) is by LP duality. Intuitively, the optimal contract pays the agent for outcomes that are more likely to come from  $f_{2}$  than from  $f_{1}$ . Moreover, it requires a higher budget the smaller the distance is between the two distributions  $f_{1}, f_{2}$ . At the extremes, if their distance is 1 (i.e. no overlap among their supports), the required budget for incentivizing  $n_{2}$  is  $c_{2} - c_{1}$ , whereas if their distance is 0 (i.e.  $f_{1} = f_{2}$ ) it becomes impossible to incentivize  $n_{2}$ .
3The MIN-BUDGET LP is closely related to the well-known MIN-PAY LP from non-budgeted contract design [e.g., 19], but with a different objective, and hence very different optimal solutions (see Appx. B.3).
4When  $n_2$  is not the target or when it is not implementable, then the solution is immediate: always pay  $c_{1}$
Formal connection to optimal hypothesis testing. Theorem 1 also uncovers a direct correspondence between optimal contracts and optimal hypothesis tests. Intuitively, given the outcome distributions  $\{f_1, f_2\}$ , and in order to incentivize  $n_2$ , the principal wishes to pay the agent if the observed outcome  $j \in \{0, \dots, m\}$  is more likely to have originated from  $f_2$ . The principal can attempt to identify whether the outcome  $j$  is drawn from distribution  $f_1$  or  $f_2$  through hypothesis testing, where a hypothesis test  $\psi : \{0, \dots, m\} \to \{0, 1\}$  maps a sample  $j$  to either  $f_2$  (indicated by 1) or to the null hypothesis  $f_1$  (indicated by 0). For our purpose it is convenient to allow tests to be non-integral, in which case  $\psi : \{0, \dots, m\} \to [0, 1]$  maps  $j$  to a probability with which it originates from  $f_2$ . The quality of a hypothesis test is measured by summing its type-1 and type-2 errors:  $\sum_{j=0}^{m} f_{1,j} \psi_j + \sum_{j=0}^{m} f_{2,j} (1 - \psi_j)$ . The test that minimizes this sum is known as the most powerful hypothesis test, and has been characterized by Neyman and Pearson [45, 4.3].
We now turn to formally establishing the connection. For a fixed  $B$ , observe that every contract with budget  $B$  can be mapped to a hypothesis test via the bijection  $\psi(j) = t^{(j)} / B$ . Then:
Theorem 2 (Optimal contract vs. test). Consider binary-action contract design with distributions  $f_{1}, f_{2}$  and costs  $c_{1}, c_{2}$ . A contract  $t$  with budget  $B$  is optimal if and only if its corresponding hypothesis test  $\psi = t / B$  is maximum power with type-1 and type-2 errors summing to  $1 - \frac{c_{2} - c_{1}}{B}$ .
The proof (Appendix B.4.2) is by a non-linear variable transformation to the MIN-BUDGET LP. Theorem 2 implies that the optimal contract for the binary-action case (Theorem 1) is equivalent to the well-known Neyman-Pearson lemma characterizing the most powerful hypothesis test:
Lemma 1 (Neyman-Pearson [e.g., 45]). Let  $f_{1}, f_{2}$  be two discrete probability distributions. Then the most powerful hypothesis test for  $f_{1}, f_{2}$  is the likelihood ratio test  $\psi(j) = \mathbb{1}[f_{2}(j) \geq f_{1}(j)]$ , which attains the optimal bound  $1 - \| p - q\|_{\mathrm{TV}}$  on the sum of type-1 and type-2 errors.
Theorem 2 establishes a new formal connection between the two domains of contract design and hypothesis testing. In the context of contract design, it provides a statistical interpretation: A min-budget contract can be interpreted as an optimal hypothesis test, and the ability to distinguish between the two hypotheses determines the required budget. In the converse direction, it enables a new proof for the Neyman-Pearson using the min-budget contract given by Theorem 1 (see Appendix B.4.2).
3.4 All-or-nothing contracts: Beyond binary action
For general action spaces, optimal contracts are not guaranteed to be all-or-nothing. In fact, we show that determining whether there exists an all-or-nothing contract that is optimal is NP-hard:
Theorem 3 (Hardness). Finding a min-budget all-or-nothing contract is NP-hard.
The proof is by reduction from 3SAT, and appears in Appx. B.5.2. Nonetheless, there are special but important cases—notably the binary outcome case<sup>5</sup>—in which results from Sec. 3.3 hold, suggesting that ideas from Thm. 1 apply more broadly. The following algorithm makes use of these ideas, showing good empirical performance, and provable performance under an MLRP condition (Sec. 3.5).
Single binding action algorithm. Revisiting the closed-form all-or-nothing contract in Eq. (5), we observe that the result is based on the fact that binary action spaces have only one alternative action. Building upon this observation, we propose the single binding action (SBA) algorithm, which computes a solution to Eq. (4) in the general (many-actions) case: Given target action  $n^* \in \mathcal{A}$ , loop over all actions  $n \neq n^*$ , and apply the closed-form formula in Eq. (5) to obtain an (all-or-nothing) contract  $t^*(n, n^*)$ . If the agent's best response (Eq. (2)) satisfies  $n(t^*(n, n^*)) = n^*$ , return  $t^*$ . If the loop ends without returning a contract, return 'fail'. The following claim shows the algorithm is sound:
Proposition 2 (Soundness of SBA). When the single binding action algorithm terminates successfully, it returns an optimal contract which is an all-or-nothing contract.
Proof in Appendix B.6. Prop. 2 ensures that if SBA succeeds, then the returned all-or-nothing contract  $t^*$  is optimal. Moreover, failure does not preclude the existence of an optimal all-or-nothing contract. If SBA fails, we solve Eq. (4) with a generic LP solver. Empirically, SBA was successful in more than  $85\%$  of cases, and is  $\sim 10^3$  times faster than the LP solver (Appendix C.3). Thus, this optimistic 'try SBA first' approach typically succeeds, adds negligible overhead if not, and guarantees correctness.
In this case there are binary outcomes  $(m = 1)$ , such as when the agent's efforts can result in either success or failure [6, 28, 21]. For this case, we show that the optimal contract is also all-or-nothing (Appx. B.5.1).
3.5 Threshold contracts: MLRP assumption
In this section we provide sufficient conditions, in the form of natural structural properties of (stochastic) learning curves, that guarantee the optimality of even simpler contracts—namely threshold contracts—and the success of SBA. With inspiration from hypothesis testing [40] and contract theory [26, 19], it is natural to consider the monotone likelihood ratio property (MLRP) assumption:
Definition 1 (MLRP [e.g., 26]). A contract design setting satisfies MLRP if for every pair of actions  $a, a'$  such that  $c_{a} < c_{a'}$ , the likelihood ratio  $f_{a'}(j) / f_{a}(j)$  is monotonically increasing in  $j$ .
In our context, MLRP states that the better the validation-set performance of a classifier, the more likely it was trained on more data. This holds in particular for monotone learning curves with binomial outcome distribution [19, B.1]. For a binary action space, MLRP ensures that  $n_2$  is always implementable, $^6$  and that the optimal contract is a threshold contract:  $t^* (j) = B\mathbb{1}[j\geq j_0]$ . This is by Theorem 1, and by the fact that  $\exists j_0$  such that  $f_{2}(j) / f_{1}(j)\geq 1$  iff  $j\geq j_0$  (see also Appendix B.7.1). Interestingly, this is similar to the relation between the Neyman-Pearson lemma and the Karlin-Rubin theorem, which characterizes the most powerful hypothesis test under monotone likelihood ratio [40].
MLRP for general action space. MLRP does not guarantee threshold contracts in general: In Appendix B.7.3, we give a constructive counterexample satisfying MLRP, but for which the optimal contract is not threshold. However, refining MLRP to also capture 'diminishing returns' turns out to be sufficient for recovering guarantees generally. For target action  $a_{N}$ , denote the survival probability of an action  $a_{i}$  by  $s_i = \mathbb{P}_{j\sim f_i}[j\geq j^*]$ , where  $j^{*}$  is the minimal outcome at which action  $a_{N}$  is more likely than  $a_{N - 1}$ . These will serve as formal means for capturing the concavity of learning curves.
Definition 2 (C-MLRP). A contract design setting satisfies Concave-MLRP (C-MLRP) if it satisfies MLRP, and additionally the actions' survival probability is concave as a function of the actions' cost.
Our final result shows that C-MLRP guarantees optimality of threshold contracts, and success of SBA:
Theorem 4 (Sufficiency for threshold). Consider a contract design setting with C-MLRP. Then the optimal contract is a threshold contract, and is recovered by the SBA algorithm.
We prove this claim by showing that concavity implies that only one alternative action is binding in the linear program equivalent to Eq. (4), reducing the problem to the two-action case. By applying Theorem 1, we obtain optimality of threshold contracts in this case as well (proof in Appendix B.7.3). This also implies that SBC always terminates successfully on inputs that satisfy C-MLRP.
In practice, we believe that C-MLRP is a reasonable assumption for realistic learning curves: in Appendix B.8.1, we prove it is satisfied by a standard theoretical model of learning curves, and in Appendix C.3, we empirically demonstrate that it is (approximately) satisfied in settings where threshold contracts are optimal. Interestingly, in our empirical study, threshold contracts were often optimal even when C-MLRP did not hold—suggesting the condition is sufficient, but not necessary (see Sec. 4.1).
We now turn to our empirical investigation of delegated learning under full and partial information. We base our experiments on the recently curated Learning Curves Database (LCDB) [43], which includes a large collection of stochastic learning curves for multiple classification datasets and methods. For each dataset and method, the database includes held-out accuracy measurements obtained for increasing sample sizes  $n \in \{2^4, 2^{4.5}, \dots, 2^{15}\}$ , with multiple repetitions per  $n$ ; these provide us with stochastic learning curves. Here we focus primarily on the popular MNIST dataset [39] as our case study, and on MLP and GBDT as representative classifiers, but we refer the reader to Appendix C for further experiments on additional datasets and methods. Code is available at: https://github.com/edensaig/delegated-classification.
4.1 Full information
We begin with the full information setting to explore in a clean environment how different parameters of the learning setting and environment affect predictive performance and economic outcomes.
As a corollary of a similar result for min-pay contracts [19, Lemma 7].
This also holds for binary outcomes in an arbitrary action space, see Appendix B.7.2.
Figure 3: Delegating with full information. (Left) Typical learning curves for two learning algorithms on MNIST. (Center) Required budget for target accuracy of 0.85 per validation set size  $m$ . (Right) Different cost regimes, indicating per accuracy region which of the two methods is cheaper to delegate.
Validation set size. Fig. 3 (left) presents typical stochastic learning curves for two learning algorithms: Multi-Layered Perceptron (MLP) and Gradient-Boosted Decision Trees (GBDT). We take an arbitrary accuracy point on the curve at  $\mathrm{acc}(n) = 0.85$  (dotted line) to examine the effects of validation set size  $m$  on min-budget contracts. Notice that MLP requires larger  $n$  to obtain 0.85; Fig. 3 (center) shows how this translates to a larger required budget  $B^{*}$ , which holds for all  $m$ . As  $m$  increases, required budgets and the difference between them both decrease. Larger validation sets are therefore useful for reducing required budget. Nonetheless, even for reasonable  $m$ , obtained budgets still remain higher than their theoretical lower bounds (target action costs  $c_{n^{*}}$ ).
Budget regimes. Fig. 3 (left) also indicates two points in which the learning curves cross (dashed lines), at  $\sim 0.74$  and  $\sim 0.94$  accuracy. These correspond to sample sizes  $n$  for which both methods obtain matching accuracies (in expectation). For a self-sufficient learner, the implication is that at each of these points, both methods are equally costly, i.e., both cost  $c_{n}$ . Interestingly, and in contrast, delegation can entail different required budgets despite equal accuracies. Fig. 3 (right) shows for each target accuracy the gap in required budgets between both methods,  $\Delta B^{*} = B_{\mathrm{GBDT}}^{*} - B_{\mathrm{MLP}}^{*}$ . As can be seen, each method is comparatively more (or less) costly in different accuracy regimes (up to 0.6; between 0.6 and 0.92; and above 0.92). Crucially, the budget gap can be large even when accuracies match (dashed lines). For example, even though both MLP and GBDT require  $n = 362 \approx 2^{17/2}$  samples to obtain  $\sim 0.74$  accuracy, GBDT is cheaper ( $\Delta B^{*} = -10^{2}$ ); for  $\sim 0.94$  which requires  $n = 23170 \approx 2^{29/2}$  from both, GBDT is significantly more expensive ( $\Delta B^{*} = 10^{5}$ ). The reason for this is that optimal budgets are determined by the ability to distinguish between distributions (Sec. 3.3).
Prevalence of simple contracts. To understand the applicability of our theoretical findings, in Appendix C.3 we conduct an empirical prevalence evaluation on additional learning algorithms, and across target actions. We observed that min-budget contracts assume a threshold form and the SBC algorithm returns correct results in more than  $85\%$  of cases overall. Restricting optimization to simple contracts, budget requirements were generally less than  $1\%$  higher than that of a min-budget contract, suggesting that simple contracts may provide a good approximation even when min-budget contracts do not assume a simple form.
4.2 Partial information
We now turn to consider delegation under partial information, in which the principal must rely on an estimated learning curve. We instantiate this idea by assuming that the principal has access to a small 'pilot' dataset of size  $k$ , where  $k$  is considered small. Using this set, the principal creates an estimated learning curve  $\hat{F}$  by fitting a curve to accuracies obtained for up to some  $n_0 \leq k$ , and extrapolating to larger  $n > n_0$ . In particular, we experiment with fitting parametric power-law curves of the form  $\mathbb{E}[\alpha_n] = a - bn^{-c}$ , which have been shown to provide good fit in various scenarios both empirically and theoretically [49, 34, 11]. Since power-law curves are monotone, composition with binomial distributions increasing in  $p$  provably results in MLRP stochastic curves [19, B.1].
Bias-variance tradeoff. Given  $k$  pilot examples, there are different ways in which the principal can use them to construct an estimated curve. Here we consider a simple tradeoff: setting  $n_0$  to be small but with more samples per  $n < n_0$  (low variance), or setting  $n_0$  to be large but with few samples per  $n < n_0$  (low bias). We define  $r$  as the number of samples per  $n$  (so low  $r$  means larger  $n_0$ ). Then, for a
Figure 4: Delegating with partial information. (Left) Extrapolated learning curves for different  $r$ . (Center-left) Accuracy obtained via delegation per pilot set size  $k$ . (Center-right) Multiplicative gain in effective number of samples due to delegation. (Right) Implications of over vs. under-estimation.
given  $r$ , we set  $n_0$  such that  $\sum_{n \leq n_0} r \cdot n \leq k$  (i.e., such that the total number of used samples does not exceed  $k$ ). Fig. 4 (left) shows different curve fits for  $r \in \{1, 3, 5\}$ , and corresponding  $n_0$ . Then, Fig. 4 (center-left) shows for a certain fixed budget the accuracy level that can be attained for increasing  $k$ , and as a function of  $r$ . As can be seen, having sufficient points  $k$  for constructing  $\hat{F}$  is important, but performance grows quickly with  $k$  (note log-scale x-axis). It is also apparent in our example that low bias (via larger  $n_0$ ) is much more important than low variance for constructing useful  $\hat{F}$ .
Cost-efficiency tradeoff. Because the pilot set provides the principal a basic means for obtaining minimal accuracy, we can ask: given  $k$  examples, and for a fixed budget  $B$ , what is the added benefit of delegating learning? For this, we define  $\mu(k) = n(\hat{t}) / k$  to be the sample-size multiplier, i.e., the multiplicative gain in the effective number of samples due to delegation. Fig. 4 (center-right) shows  $\mu(k)$  for increasing  $k$  and across  $r$ . For  $r = 1$  (which is superior in terms of performance and outcomes),  $\mu$  begins at  $\sim 10$ , increases to  $\sim 30$  at around  $k = 190$ , and slowly decreases back to  $\sim 10$  towards  $k = 1,000$ . For  $r > 1$ , we observe that  $\mu \approx 1$ , i.e., there is effectively no gain from delegation, until around  $k = 100$ , only after which some gain is restored. This highlights the importance of obtaining an accurate estimate  $\hat{F}$  in terms of the economic consequences of delegation.
Over vs. under-estimation. Typically in curve-fitting, over and under-estimation are treated equally, since both types of error can negatively affect goodness of fit and extrapolation quality. However, for delegation, the implications of over vs. under-estimation on contract outcomes are highly asymmetric. Fig. 4 (right) shows for a target incentivized number of samples  $n(t^{*})$  the relation between the (theoretical) signed extrapolation error  $n(t^{*})$  (i.e., over- or under-estimate, measured in accuracy points) and the eventual loss in accuracy obtained through delegation, relative to perfect estimation. Each point in the plot corresponds to one curve-fitting instance, with points shown for varying  $k$ ,  $n_0$ , and  $r$ , and with multiple independent repetitions. Results show that in the under-estimation regime (i.e., negative extrapolation error), loss in accuracy degrades gracefully with the estimation error. In stark contrast, even minimal over-estimation (positive extrapolation error) causes accuracy to plummet dramatically, as the agent's rational response in those cases was to use the smallest dataset possible. We interpret this as a consequence of 'setting the bar too high'—a rational decision to minimize effort in response to unrealistic expectations. This has important implications for the choice of how to fit and extrapolate learning curves, suggesting that contracts can be tolerant to under-estimation, while over-estimation should be avoided at all costs.
Motivated by the increasingly-common practice of outsourcing learning tasks, this paper sets out to introduce and study the novel problem of delegated classification. Our findings suggest that conflict of interests should not be overlooked, and that contracts hold potential as a means for aligning them. Our analysis relies on a set of assumptions, which should be carefully considered by practitioners and empiricists alike; we also believe that there are likely further fruitful connections to explore between contracts and statistical hypothesis testing. As a problem of contract design, and when the learning task is reasonably well-behaved, delegated learning manifests in the form simple threshold contracts. A natural question for future work is whether simplicity also implies robustness to partial knowledge—as is often the case [19].
Acknowledgements. The authors would like to thank Ruth Heller, Shafi Goldwasser, Jonathan Shafer, Ohad Einav, and anonymous reviewers for their insightful remarks and valuable suggestions. Nir Rosenfeld is supported by the Israel Science Foundation grant no. 278/22. Eden Saig is supported by the Israel Council for Higher Education PBC scholarship for Ph.D. students in data science. Funded by the European Union (ERC, ALGOCONTRACT, 101077862, PI: Inbal Talgam-Cohen).