SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tonghan Wang
twang1@g.harvard.edu
Paul Dütting
Google Switzerland
duetting@google.com
Israel Institute of Technology
divanov@campus.technion.ac.il
Inbal Talgam-Cohen
italgam@cs.technion.ac.il
David C. Parkes
parkes@eecs.harvard.edu
Contract design involves a principal who establishes contractual agreements about payments for outcomes that arise from the actions of an agent. In this paper, we initiate the study of deep learning for the automated design of optimal contracts. We introduce a novel representation: the Discontinuous ReLU (DeLU) network, which models the principal's utility as a discontinuous piecewise affine function of the design of a contract where each piece corresponds to the agent taking a particular action. DeLU networks implicitly learn closed-form expressions for the incentive compatibility constraints of the agent and the utility maximization objective of the principal, and support parallel inference on each piece through linear programming or interior-point methods that solve for optimal contracts. We provide empirical results that demonstrate success in approximating the principal's utility function with a small number of training samples and scaling to find approximately optimal contracts on problems with a large number of actions and outcomes.
Contract theory studies the setting where a principal seeks to design a contract for rewarding an agent on the basis of the uncertain outcomes caused by the agent's private actions [8, 45]. Typical examples include a landlord who enters into a summer rental with a contract that includes penalties in the case of damage; a homeowner who engages a firm to complete a kitchen renovation with a contract that conditions payments on timely completion or functioning appliances; or an individual who employs a freelancer to do some design work with a contract that includes bonuses for completing the job.
A contract specifies payments to the agent, conditioned on outcomes. The principal is self-interested, with a value for each outcome and a cost for making payments. The agent is also self-interested, and responds to a contract by choosing an action that maximizes its expected utility (expected payment minus the cost of an action). The problem is to find a contract that maximizes the principal's utility (expected value minus expected payment), given that the agent will best respond to the contract. In economics, this is referred to as a problem of moral hazard, in that the agent is willing to privately act in its best interest given the contract (the "hazard" is that the behavior of the agent may be to the detriment of the principal).
The importance of contract design is evidenced by the 2016 Nobel Prize awarded to O. Hart and B. Holmström [51] and its broad application to real-world problems. Contract design is one of the three fundamental problems in the realm of economics involving asymmetric information and incentives, along with mechanism design [9] and signalling (Bayesian persuasion) [41]. However, while
*Also DeepMind, London UK
(a)  $\pmb{u}^{\pmb{p}}$  function
(b) ReLU approximation (baseline)
Figure 1: DeLU and ReLU approximation on a randomly generated contract design instance with 1,000 actions and 2 outcomes (see Sec. 4.3). The x- and y-coordinates are the payments for each of the two outcomes, and the z-coordinate is the utility of the principal. (a) The exact surface of the principal's utility function  $u^p$ . Different colors represent the action selected by the agent upon receiving the contract. (b) A learned ReLU network cannot model the discontinuity of the  $u^p$  function and yields an incorrect contract as shown in (a). Colors in (b) and (c) represent different activation patterns (linear regions) of the networks. (c) A learned DeLU network represents a discontinuous function and can well-approximate  $u^p$ , yielding the optimal contract as shown in (a).
(c) DeLU approximation
both mechanism design [10, 11, 12, 7, 36, 37] and signalling [26, 27, 18] have been studied extensively from a computational perspective, the contract design problem has only recently received attention and presented distinct computational challenges. For many combinatorial contract settings [6], the conventional approach based on linear programming becomes computationally infeasible and the problem of finding or even approximating the optimal contract becomes intractable [33, 28, 29]. In addition, learning-theoretic results give worst-case exponential sample complexity bounds for learning an approximately-optimal contract [40, 55]. As a result, there is a growing demand for a scalable, general purpose, and beyond-worst case approach for computing (near-)optimal contracts.
We are thus motivated to initiate the study of deep learning for optimal contract design (deep contract design). This falls within the broader framework of differentiable economics, which seeks to leverage parameterized representations of differentiable functions for the purpose of optimal economic design [30]. In regard to learning contracts, recent work [40, 21, 55, 31] considers the distinct setting of online learning for contract design with bandit feedback, characterizing regret bounds without appeal to deep learning.
A first innovation of this paper is to introduce a neural network architecture that can well-approximate the principal's utility function. A close examination of the geometry of the principal's utility function (Sec. 3) reveals its similarity to fully-connected feed-forward neural networks with ReLU activations [1]: both of them are piecewise affine functions [22]. However, whereas a ReLU network models a continuous function, the principal's utility function is discontinuous at the boundary of linear regions, where the best response of the agent changes. Accurate approximation in the vicinity of boundaries is critical because an optimal contract is always located on the boundary (Lemma 3 in Sec. 3), but this is exactly where ReLU approximation error can be large. To handle this, we introduce the Discontinuous ReLU (DeLU) network, which recognizes that the linear regions of a ReLU network are decided by activation patterns (the status of all activation units in the network), and conditions a piecewise bias on these activation patterns. In this way, each linear region has a different bias parameter and can be discontinuous at the boundaries. Fig. 1 illustrates the DeLU and its use to represent the principal's utility function, contrasting this with a continuous ReLU function.
A second innovation in this paper is to introduce a scalable inference technique that can find the contract that maximizes the network output (i.e., the principal's utility). We show that linear regions (pieces) of DeLU networks implicitly learn closed-form expressions for the incentive constraints of the agent and the utility maximization objective of the principal, so that we can use linear programming (LP) on each piece to find the global optimum. However, the time efficiency of this LP method is impeded by the overhead of solving individual LPs, and worsens with problem size in the absence of suitable parallel computing resources. To increase the computational efficiency, we develop a gradient-based inference algorithm based on the interior-point method [49] that only requires a few forward and backward passes of the DeLU network to find the contract that maximizes the network output. This method scales well to large-scale problems and can be readily run in parallel on GPUs.
The effectiveness of introducing piecewise discontinuity into neural networks is demonstrated by our experimental results. The DeLU network well-approximates the principal's utility function even with
a small number of training samples, and the inference method remains accurate and efficient as the problem size grows. By synergistically harnessing these two innovations, our method consistently finds near-optimal solutions on a wide range of contract design problems, significantly surpassing the capability of conventional continuous networks.
Related work. A longstanding challenge in the deep learning community has been to approximate discontinuous functions with neural networks. While the Universal Approximation Theorem guarantees the approximation of continuous functions, many problems involve discontinuity, including solar flare imaging [46] and problems coming from mathematics [24]. However, establishing a network to represent discontinuous functions is not an easy task. Discontinuities were considered as early as in the 1950s, when Rosenblatt [50] introduced the perceptron model and its single-layer of step activation functions. Following this work, others modeled discontinuity through the use of different discontinuous activation functions [34, 2, 44]. However, training these models is more challenging than the more typical models that make use of continuous activation functions [24], and this has hindered their application. To the best of our knowledge, this paper presents the first discontinuous network architecture with continuous activation functions and stable optimization performance.
There is a vast and still-growing economics literature on contracts [see, e.g., 52, 15], to which the computational lens has recently been applied. In classic settings, computing the optimal contract is tractable by solving one LP per agent's action, and taking the overall best solution [see, e.g., 32]. However, LP becomes computationally infeasible for combinatorial contracts, including settings with exponentially-many outcomes [33], actions [28], or agent combinations [29]. Another issue with the LP-based approach is that it requires complete information in regard to the problem facing the agent (its type). If the agent has a hidden type, this makes the optimal contract hard to compute [16, 38, 17], or otherwise complex [4]. One way to deal with these complexities is to focus on simple contracts, in particular linear contracts (commission-based), which are popular in practice [e.g., 13, 32, 14].
A distinct but related approach combines contract design with learning, where efforts have concentrated on online learning theory [40, 21, 55, 31]. These works show exponential lower bounds on the achievable regret for general contracts in worst-case settings, and sublinear regret bounds for linear contracts. Additionally, contracts have been studied from the perspective of multi-agent reinforcement learning [48, 19, 25]. The problem of strategic classification has also established a formal connection between strategy-aware classifiers and contracts [43, 42, 3]. There is also interest in the application of contract theory to the AI alignment problem [39]. However, the use of learning methods for solving optimal contracts, as opposed to auctions [30], remains largely unexplored. This gap in the literature can be partly attributed to the inherent complexity of finding optimal contracts, as the principal utility depends on the agent's action, which must conform to incentive compatibility (IC) constraints that are not observable by the principal.
Offline contract learning problem. The contract design problem is defined with elements  $\mathcal{C} = \langle \mathcal{A},\mathcal{O},\mathcal{F},c,p,v\rangle$ , and involves a single principal and a single agent. The agent selects an action  $a$  in the finite action space  $\mathcal{A}$ ,  $|\mathcal{A}| = n$ . Action  $a$  leads to a distribution  $p(\cdot |a)$  over the outcomes in  $\mathcal{O}$ ,  $|\mathcal{O}| = m$ , and incurs a cost  $c(a)\in \mathbb{R}_{\geq 0}$  to the agent. The valuation of each outcome  $o_{j}\in \mathcal{O}$  for the principal is decided by the value function  $v:\mathcal{O}\to \mathbb{R}_{\geq 0}$ . The principal sets up a contract,  $\pmb {f}\in \mathcal{F}\subset \mathbb{R}_{\geq 0}^{m}$ , which influences the action selected by the agent. A contract  $\pmb {f} = (f_1,f_2,\dots ,f_m)^\top$  specifies the payment  $f_{j}\geq 0$  made to the agent by the principal in the event of outcome  $o_j$ . On receiving a contract  $\pmb{f}$ , the agent selects the action  $a^* (\pmb {f})$  maximizing its utility  $u^{a}(\pmb {f};a) = \mathbb{E}_{o\sim p(\cdot |a)}[f_{o}] - c(a)$ . For a given action  $a$  of the agent, the principal gets utility  $u^{p}(\pmb {f};a) = \mathbb{E}_{o\sim p(\cdot |a)}[v_{o} - f_{o}]$ , which is the principal's expected value minus payment. In the case that the induced action is the best response of the agent given the contract, we write  $u^{p}(\pmb {f}) = \mathbb{E}_{o\sim p(\cdot |a^{*}(\pmb {f}))}[v_{o} - f_{o}]$ . The goal in optimal contract design is to find the contract that maximizes the principal's utility without access to  $p(\cdot |a)$  and the action taken by the agent:
$$
\boldsymbol {f} ^ {*} = \underset {\boldsymbol {f}} {\arg \max } u ^ {p} (\boldsymbol {f}) = \underset {\boldsymbol {f}} {\arg \max } \mathbb {E} _ {o \sim p \left(\cdot | a ^ {*} (\boldsymbol {f})\right)} \left[ v _ {o} - f _ {o} \right]. \tag {1}
$$
ReLU piecewise-affine networks and activation patterns. A fully-connected neural network with a piecewise linear activation function (e.g., ReLU and leaky ReLU) and a linear output layer represents a continuous piecewise affine function [5, 23]. A piecewise affine function is defined as follows.
Definition 1 [Piecewise affine function]. A function  $g: \mathbb{R}^d \to \mathbb{R}$  is piecewise affine if there exists a finite set of polytopes  $\{\mathcal{D}_i\}_{i=1}^P$  such that  $\cup_{i=1}^P \mathcal{D}_i = \mathbb{R}^d$ ,  $\mathcal{D}_i \cap \mathcal{D}_{j \neq i} = \emptyset$ , and  $g$  is a linear function  $\rho_i: \mathcal{D}_i \to \mathbb{R}$  when restricted to  $\mathcal{D}_i$ . We call  $\rho_i$  a linear piece of  $g$ .
We now follow [22] and introduce some local properties of the piecewise affine function that is represented by a ReLU network. Suppose there are  $L$  hidden layers in a network  $g$ , with sizes  $[n_1, n_2, \dots, n_L]$ .  $\pmb{W}^{(l)} \in \mathbb{R}^{n_l \times n_{l-1}}$  and  $\pmb{b}^{(l)} \in \mathbb{R}^{n_l}$  are the weights and biases of layer  $l$ . Let  $n_0 = d$  denote the input space dimension. We consider a ReLU network with one-dimensional outputs, and the output layer has weights  $\pmb{W}^{(L+1)} \in \mathbb{R}^{1 \times n_L}$  and a bias  $b^{(L+1)} \in \mathbb{R}$ . With input  $\pmb{x} \in \mathbb{R}^d$ , we have the pre- and post-activation output of layer  $l$ :  $\pmb{h}^{(l)}(\pmb{x}) = \pmb{W}^{(l)}\pmb{o}^{(l-1)}(\pmb{x}) + \pmb{b}^{(l)}$  and  $\pmb{o}^{(l)}(\pmb{x}) = \sigma(\pmb{h}^{(l)}(\pmb{x}))$ , where  $\sigma: \mathbb{R} \to \mathbb{R}$  is an activation function. In this paper, we consider ReLU activation  $\sigma(x) = \max\{x, 0\}$  [35, 54], but the proposed method can be extended to other piecewise linear activation functions (e.g., LeakyReLU and PReLU [54]). For each hidden unit, the ReLU activation status has two values, defined as 1 when pre-activation  $h$  is positive and 0 when  $h$  is strictly negative. The activation pattern of the entire network is defined as follows.
Definition 2 [Activation Pattern]. An activation pattern of a ReLU network  $g$  with  $L$  hidden layers is a binary vector  $\mathbf{r} = [\mathbf{r}^{(1)},\dots ,\mathbf{r}^{(L)}]\in \{0,1\}^{\sum_{l = 1}^{L}n_{l}}$ , where  $\mathbf{r}^{(l)}$  is a layer activation pattern indicating activation status of each unit in layer  $l$ .
The activation pattern depends on the input  $\pmb{x}$ , and we define function  $r: \mathbb{R}^d \to \{0,1\}^{\sum_{l=1}^{L} n_l}$  that maps the input to the corresponding activation pattern. For a ReLU network, inputs that have the same activation pattern lie in a polytope, and the activation pattern determines the boundaries of this polytope. To see this, we can write the output of layer  $l$ ,  $h^{(l)}(\pmb{x})$ , as
$$
\boldsymbol {h} ^ {(l)} (\boldsymbol {x}) = \boldsymbol {W} ^ {(l)} \boldsymbol {R} ^ {(l - 1)} (\boldsymbol {x}) \left(\boldsymbol {W} ^ {(l - 1)} \boldsymbol {R} ^ {(l - 2)} (\boldsymbol {x}) \left(\dots \left(\boldsymbol {W} ^ {(1)} \boldsymbol {x} + \boldsymbol {b} ^ {(1)}\right) \dots\right) + \boldsymbol {b} ^ {(l - 1)}\right) + \boldsymbol {b} ^ {(l)}, \tag {2}
$$
where  $\pmb{R}^{(k)}$  is a diagonal matrix with diagonal elements equal to the layer activation pattern  $\pmb{r}^{(k)}$ . Eq. 2 indicates that, when  $\pmb{r}$  is fixed,  $\pmb{h}^{(l)}$  is a linear function  $\pmb{h}^{(l)}(\pmb{x}) = \pmb{M}^{(l)}\pmb{x} + \pmb{z}^{(l)}$ , where  $\pmb{M}^{(l)} = \pmb{W}^{(l)}\left(\prod_{k=1}^{l-1}\pmb{R}^{(l-k)}(\pmb{x})\pmb{W}^{(l-k)}\right)$  and  $\pmb{z}^{(l)} = \pmb{b}^{(l)} + \sum_{k=1}^{l-1}\left(\prod_{j=1}^{l-k}\pmb{W}^{(l+1-j)}\pmb{R}^{(l-j)}(\pmb{x})\right)\pmb{b}^{(k)}$ . We thus get  $\sum_{l=1}^{L}n_l$  half-spaces, with the half-space corresponding to unit  $i$  of layer  $l$  defined as:
$$
\Gamma_ {l, i} = \left\{\boldsymbol {y} \in \mathbb {R} ^ {d} \mid \Delta_ {i} ^ {(l)} \left(\boldsymbol {M} _ {i} ^ {(l)} \boldsymbol {y} + \boldsymbol {z} _ {i} ^ {(l)}\right) \geq 0 \right\}, \tag {3}
$$
where  $M_{i}^{(l)}\pmb{y} + \pmb{z}_{i}^{(l)}$  is the output of unit  $i$  at layer  $l$ , and  $\Delta_{i}^{(l)}$  is 1 if  $\pmb{h}_{i}^{(l)}(\pmb{x})$  is positive, and is -1 otherwise. The input  $\pmb{x}$  is in the polytope that is defined by the intersection of these half-spaces:  $\mathcal{D}(\pmb{x}) = \cap_{l=1,\dots,L} \cap_{i=1,\dots,n_l} \Gamma_{l,i}$ . When restricted to  $\mathcal{D}(\pmb{x})$ , the ReLU network is a linear function:  $g(\pmb{x}) = \pmb{W}^{(L+1)}\pmb{R}^{(L)}\pmb{h}^{(L)}(\pmb{x}) + b^{(L+1)}$ .
Interior-point method for optimization problems with inequality constraints. For a minimization problem with objective function  $q(\pmb{x})$  and inequality constraints  $p_i(\pmb{x}) > 0, i = 1, \dots, M$ , the interior-point method [53] introduces a logarithmic barrier function,  $\phi(\pmb{x}) = -\sum_{i=1}^{M} \log(p_i(\pmb{x}))$ , and finds the minimizer of  $q(\pmb{x}) + \frac{1}{t} \phi(\pmb{x})$ , for some  $t > 0$ . This new objective function is defined on the set of strictly feasible points  $\{\pmb{x} | p_i(\pmb{x}) > 0, i = 1, \dots, M\}$ , and approximates the original objective as  $t$  becomes large. Given this, we can solve for a series of optimization problems for increasing values of  $t$ . In the  $k$ -th round,  $t^{(k)}$  is set to  $\mu \cdot t^{(k-1)}$ , where  $\mu > 1$  is a constant,  $t^{(0)} > 0$  is an initial value, and the problem is solved (e.g., by Newton initialized at  $\pmb{x}^{(k-1)}$ ) to yield  $\pmb{x}^{(k)}$ . Assume that we solve the barrier problem exactly for each iterate, then to achieve a desired accuracy level of  $\epsilon > 0$ , we need  $n_{\text{barrier}} = \log(M / (t^{(0)}\epsilon)) / \log(\mu)$  rounds of optimization.
3 Geometry of Optimal Contracts
In this section, we introduce some properties of the principal's utility function,  $u^p: \mathcal{F} \to \mathbb{R}$ , which will motivate our method in Sec. 4. First, we show that  $u^p$  is a piecewise affine function.
Lemma 1. The principal's utility function  $u^p$  is a piecewise affine function.
Proof. The principal utility depends on the agent action. For contracts in the intersection of the following  $n - 1$  half spaces that represent incentive compatibility constraints, the agent's action is  $a_{i}$ :
$$
\Gamma_ {i, j} = \left\{\boldsymbol {f} \in \mathcal {F} \mid \mathbb {E} _ {o \sim p (\cdot | a _ {i})} [ f _ {o} ] - c (a _ {i}) \geq \mathbb {E} _ {o \sim p (\cdot | a _ {j})} [ f _ {o} ] - c (a _ {j}) \right\}, \forall j \neq i. \tag {4}
$$
Moreover, when agent action  $a_i$  remains unchanged,  $u^p$  changes linearly with  $\pmb{f}$ :  $\alpha u^{p}(\pmb {f};a_{i}) + \beta u^{p}(\pmb{f}^{\prime};a_{i}) = \alpha \mathbb{E}_{o\sim p(\cdot |a_{i})}[v_{o} - f_{o}] + \beta \mathbb{E}_{o\sim p(\cdot |a_{i})}[v_{o} - f_{o}^{\prime}] = \mathbb{E}_{o\sim p(\cdot |a_{i})}[v_{o} - (\alpha f_{o} + \beta f_{o}^{\prime})] = u^{p}(\alpha \pmb {f} + \beta \pmb{f}^{\prime};a_{i})$ . Therefore, when restricted to the polytope  $\mathcal{Q}_i = \cap_{j\neq i}\Gamma_{i,j},\forall i,u^p$  is linear.
Based on Lemma 1, we define a linear piece of function  $u^p$  as  $\mu_i^p: \mathcal{Q}_i \to \mathbb{R}$ , where  $\mathcal{Q}_i$  defines the set of contracts that motivates the agent to take action  $i$ . Lemma 1 can be easily extended to the agent's utility function  $u^a$ , and the linear pieces of function  $u^p$  and  $u^a$  share the same set of domains  $\{\mathcal{Q}_i\}$ . We define a linear piece of function  $u^a$  as  $\mu_i^a: \mathcal{Q}_i \to \mathbb{R}$ . We next observe:
Lemma 2. The principal's utility function  $u^p$  can be discontinuous on the boundary of linear pieces.
The proof in Appx. A follows the idea that the agent is indifferent between action  $a_i$  and  $a_j$  given a contract  $f$  on the boundary of two neighboring linear pieces  $\mu_i^p$  and  $\mu_{j\neq i}^p$ :  $\mu_i^a (\pmb {f}) = \mu_j^a (\pmb {f}) = u^a (\pmb {f})$ . However, the principal's utility equals the expected value minus the cost of an action minus  $u^{a}(\pmb {f})$ . As the expected value minus the cost can be different in  $\mu_i^p$  and  $\mu_j^p$ ,  $u^p$  can be discontinuous at  $f$ .
We then define the optimality of a contract and analyze the structure of optimal contracts.
Definition 3 [Piecewise and Global Optimal Contracts]. A contract  $\pmb{f}_i^* \in \mathcal{Q}_i$  is piecewise optimal if  $u^{p}(\pmb{f}_{i}^{*}) \geq u^{p}(\pmb{f}), \forall \pmb{f} \in \mathcal{Q}_{i}$ . A contract  $\pmb{f}^{*}$  is global optimal if  $u^{p}(\pmb{f}^{*}) \geq u^{p}(\pmb{f}), \forall \pmb{f} \in \mathcal{F}$ .
Lemma 3. The global optimal contract is on the boundary of a linear piece.
Lemma 3 can be proved by contradiction (as detailed in Appx. A): if a global optimal contract is not on the boundary, we can always find a solution with greater principal utility. As analyzed in Sec. 4, Lemma 3 serves as a compelling rationale for introducing our discontinuous networks. Besides discontinuity, there is another network design consideration motivated by the following property.
Lemma 4. The principal's utility function  $u^p$  can be written as a summation of a concave function and a piecewise constant function.
The proof in Appx. A commences by establishing the convexity of function  $u^a$  and subsequently formulating  $u^p$  as a function of  $u^a$ . This property motivates us to introduce concavity into our network design. In Appx. C, we discuss how to achieve this by imposing non-negativity constraints on network weights and analyze the impact of this design choice on the performance.
4 DeLU Neural Networks
Given the piecewise-affine geometry, the preceding analysis shows a close connection between the principal's utility function  $u^p$  (Lemma 1) and fully-connected ReLU networks (Sec. 2). However, ReLU functions are continuous and cannot represent the abrupt changes in  $u^p$  at the boundaries of the linear pieces. This lack of representational capacity is problematic because the optimal contracts are on the boundary (Lemma 3), which is precisely where the ReLU approximation errors can be large. Consequently, ReLU networks are not well suited to deep contract design. In this section, we introduce the new Discontinuous ReLU (DeLU) network, which provides a discontinuity at boundaries between pieces, making it a suitable function approximator for the  $u^p$  function.
4.1 Architecture
The DeLU network architecture supports different biases for different linear pieces. Since a linear piece can be identified by the corresponding activation pattern, we propose to condition these piecewise biases on activation patterns. Specifically, we learn a DeLU network  $\xi : \mathcal{F} \to \mathbb{R}$  (Fig. 2) to approximate the principal's utility function, mapping a contract to the corresponding utility of the principal. The first part of a DeLU network is a sub-network similar to a conventional ReLU network. This sub-network  $\eta$  has  $L$  fully-connected hidden layers with ReLU activation and a weight matrix at the output layer, i.e.,  $\eta$  is parameterized as  $\theta_{\eta} = \{\pmb{W}^{(1)}, \pmb{b}^{(1)}, \dots, \pmb{W}^{(L)}, \pmb{b}^{(L)}, \pmb{W}^{(L + 1)}\}$ , which includes weights and biases for  $L$  hidden layers and weights for the output  $((L + 1)$ -th) layer. The