SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Figure 2: The DeLU architecture.
Algorithm 1 Parallel Gradient-Based Inference
Input:  $\mathbf{X}^{(0)} \in \mathbb{R}^{K \times m}$ ; DeLU network  $\xi$  with trained parameters;  $\epsilon, \mu > 1$ ;  $t^{(0)}$ ;  $\alpha$ .
for  $k\in \{1,\dots ,\lceil \frac{\log(N / t^{(0)}\epsilon)}{\log(\mu)}\rceil \}$  do
$t^{(k)} = \mu^{k - 1}t^{(0)};\bar{X}^{(k,0)}\gets X^{(k - 1)};/*$  Each round starts with the optimal solution in the last round.\*/
for  $j = 1,2,\dots$  do
$\pmb{H}^{(l)} = \pmb{M}_i^{(l)}\pmb{X}^{(k,j - 1)} + \pmb{z}_i^{(l)} / * \text{Get hidden layer}$  outputs in a single forward pass.*/
$\Phi^{(k,j - 1)} = -\sum_{l,i}\log \Delta_i^{(l)}H_i^{(l)} / *Barrier$  term.\*/
$$
d \boldsymbol {X} = \frac {\partial}{\partial \boldsymbol {X} ^ {(k , j - 1)}} \left[ \boldsymbol {W} ^ {(L + 1)} \boldsymbol {R} _ {i} ^ {(L)} \boldsymbol {H} ^ {(L)} - \frac {\boldsymbol {\Phi} ^ {(k , j - 1)}}{t ^ {(k)}} \right]
$$
if  $\| dX\|_{\infty} <   \epsilon$  then  $t^{(k)}* /$
$\pmb{X}^{(k)} \gets \pmb{X}^{(k,j-1)}$ ; break; /*To the next round*/
else  $X^{(k,j)} = X^{(k,j - 1)} + \alpha dX / *Gradient$  ascent\*/and for
DeLU network is different from a conventional ReLU network at the bias of the output (last) layer  $(b^{(L + 1)})$ , and we introduce a new method to generate the last-layer biases.
Since there are no activation units at the output layer, given an input contract  $\pmb{f}$ , we can obtain the activation pattern  $r(\pmb{f})$  by a forward pass of the network up until the last layer. To condition  $b^{(L + 1)}$  on the activation pattern, we train another sub-network  $\zeta : [0,1]^{\sum_{l = 1}^{L}n_{l}} \to \mathbb{R}$ , that maps  $r(\pmb{f})$  to the bias of the output layer. We use a two-layer fully-connected network with Tanh activation to represent  $\zeta$  and denote its parameters as  $\theta_{\zeta}$ . In this way, contracts in the same linear piece of the DeLU network share the same bias value, enabling the network to express discontinuity at the boundaries while keeping other properties of network  $\eta$  unchanged. In Appx. B, we discuss why we adopt a neural network, instead of a simpler function, to model the bias term.
4.2 Training and inference
The DeLU network  $\xi$ , including sub-networks  $\eta$  and  $\zeta$ , is end-to-end differentiable. For training, we randomly sample  $K$  contracts and the corresponding principal's utilities:  $\mathcal{T}_K = \{(\pmb {f}_i,u^p (\pmb {f}_i))\}_{i = 1}^K$ . Feeding a training sample  $\pmb{f}_i$  as input, we get an approximated principal's utility:  $\xi (\pmb {f}_i;\theta_\eta ,\theta_\zeta) = \eta (\pmb {f}_i;\theta_\eta) + \zeta (r(\pmb {f}_i);\theta_\zeta)$ , and the network  $\xi$  is trained to minimize the following loss function in  $T$  epochs (Appx. D gives more details on network architecture, infrastructure, and training):
$$
\mathcal {L} _ {\mathcal {T} _ {K}} \left(\theta_ {\eta}, \theta_ {\zeta}\right) = \frac {1}{K} \sum_ {i = 1} ^ {K} \left[ \xi \left(\boldsymbol {f} _ {i}; \theta_ {\eta}, \theta_ {\zeta}\right) - u ^ {p} \left(\boldsymbol {f} _ {i}\right) \right] ^ {2}. \tag {5}
$$
Given a trained DeLU network,  $\xi : \mathcal{F} \to \mathbb{R}$ , we have an approximation of the principal's utility function. The next step is to find a contract that maximizes the learned utility function, that is  $f^{*} = \arg \max_{\boldsymbol{f}} \xi(\boldsymbol{f})$ . We call this the inference process. Since the function represented by a DeLU network is discontinuous, conventional first- or second-order optimization methods are not applicable, and, formally, we need to develop a suitable optimization approach to solve
$$
\xi (\boldsymbol {f} ^ {*}) = \max  _ {\rho_ {i}} \max  _ {\boldsymbol {f} \in \mathcal {D} _ {i}} \rho_ {i} (\boldsymbol {f}), \tag {6}
$$
where  $\rho_{i}:\mathcal{D}_{i}\to \mathbb{R}$  is a linear piece of network  $\xi$ . This motivates us to first find the piecewise optimal contracts and then get the global optimum by comparing the piecewise optima.
4.2.1 Linear programming based inference
A first approach finds the optimal contract for each piece using linear programming (LP). Contracts in the same linear piece  $\rho$  result in the same activation pattern  $r$ , and the DeLU network is a linear function on the piece. In particular, the optimization problem of finding the piecewise optimum is:
$$
\arg \max  _ {\boldsymbol {f}} \boldsymbol {W} ^ {(L + 1)} \boldsymbol {R} ^ {(L)} \left[ \boldsymbol {M} ^ {(L)} \boldsymbol {f} + \boldsymbol {z} ^ {(L)} \right] \text {s . t .} \Delta_ {i} ^ {(l)} \left(\boldsymbol {M} _ {i} ^ {(l)} \boldsymbol {f} + \boldsymbol {z} _ {i} ^ {(l)}\right) \geq 0, \forall l \in [ L ], i \in [ n _ {l} ],
$$
Figure 3: The gradient-based inference method for finding piecewise optima, illustrated on the same instance as Fig. 1.
where  $[L] = \{1, \dots, L\}$ ,  $[n_l] = \{1, \dots, n_l\}$ , the objective is the linear function represented by the DeLU network on piece  $\rho$ , and the constraints require that the activation pattern remains  $r$ . The definitions of  $\Delta_i^{(l)}$ ,  $\pmb{R}^{(L)}$ ,  $\pmb{M}^{(l)}$ , and  $\pmb{z}^{(l)}$  are the same as in Sec. 2, but with the activation pattern fixed to  $r$ . We omit the last-layer bias in the objective because the activation pattern is fixed for this piece, and thus the bias is constant and does not influence the piecewise optimal solution.
For each LP, there are  $m$  decision variables, representing payments for  $m$  outcomes, and  $N = \sum_{l=1}^{L} n_l$  (the number of neurons) constraints. LPs for each piece can be solved in polynomial time of  $m$  and  $N$ , and quickly in practice via the simplex method. However, a challenge is that there are  $2^N$  activation patterns. For a DeLU network with a moderate size, we can enumerate all these pieces. For a larger DeLU network, we can approximate by collecting random contract samples and solving LPs with the corresponding activation patterns. Parallelizing these LPs may achieve better time efficiency. However, this improvement is limited by the overhead of solving a single LP and the required amount of suitable parallel computational resources. In the next section, we introduce a gradient-based method that provides better efficiency in solving single problems and better parallelism through making use of GPUs. We compare these two inference methods in Sec. 5.1.
4.2.2 Gradient-based inference
The gradient-based inference method is based on the interior-point method (Sec. 2) and finds piecewise optimal solutions via forward and backward passes of the DeLU network. Specifically, on each linear piece,  $\rho$ , we adopt the following barrier function:
$$
\phi (\boldsymbol {f}) = - \sum_ {l = 1} ^ {L} \sum_ {i = 1} ^ {n _ {l}} \log \left[ \Delta_ {i} ^ {(l)} \left(\boldsymbol {M} _ {i} ^ {(l)} \boldsymbol {f} + \boldsymbol {z} _ {i} ^ {(l)}\right) \right]. \tag {7}
$$
At the  $k$ -th round, the objective function is
$$
\phi^ {(k)} (\boldsymbol {f}) = - \boldsymbol {W} ^ {(L + 1)} \boldsymbol {R} ^ {(L)} \left[ \boldsymbol {M} ^ {(L)} \boldsymbol {f} + \boldsymbol {z} ^ {(L)} \right] + \frac {1}{t ^ {(k)}} \phi (\boldsymbol {f}). \tag {8}
$$
We use gradient descent initialized at  $\pmb{f}^{(k-1)}$ , and update  $\pmb{f}$  with  $\partial \phi^{(k)}(\pmb{f}) / \partial \pmb{f}$  to find the minimizer. A forward pass of the DeLU network gives  $\phi^{(k)}(\pmb{f})$  and a backward pass is sufficient to calculate  $\partial \phi^{(k)}(\pmb{f}) / \partial \pmb{f}$ . Since forward and backward passes are naturally parallelized in modern deep learning frameworks, this method can be parallelized by processing multiple  $\pmb{f}$  simultaneously. Alg. 1 gives the matrix-form expression of this parallel computation. The input  $X^{(0)} \in \mathbb{R}^{K \times m}$  can be the training set or a random sample set, and we discuss the difference of these two settings in Appx. E.
Inference performance and boundary alignment degree. The performance of the proposed inference algorithms depends on the DeLU approximation quality, especially on the degree of alignment between the DeLU boundaries and the ground-truth linear piece boundaries. An interesting question is whether our training setup can achieve a high boundary alignment degree. A reason to think this is possible comes from observing that the MSE training loss (Eq. 5) is sensitive to misalignment between the DeLU and true boundaries. In particular, given that the jump of the utility function at boundary points can be arbitrarily large, a slight misalignment between DeLU and true boundaries can lead to a large increase in the MSE loss. Related to this, we explore an extension of the gradient-based inference method to make it more robust to possible boundary misalignment. When
1Previous research [20] found some patterns are invalid, and there are actually  $n_{N,m} = \sum_{i = 0}^{m}\binom{N}{m - i}$  pieces.
Figure 4: Optimality (normalized principal utility) of DeLU, ReLU and a direct LP solver (Oracle LP), for problems with increasing sizes.
annealing the coefficient of the barrier function, we can check whether the principal's utility increases for each  $t^{(k)}$  value. A non-increasing utility indicates that we encounter inaccurate boundaries, and we can stop the inference to seek more robustness. We call inference with this "early stop" mechanism the sub-argmax gradient-based inference method.
In Sec. 5.3, we empirically evaluate the boundary alignment degree achieved by DeLU, and demonstrate that near-optimal contract-design performance is closely associated with high boundary alignment. We also show that the sub-argmax variation can improve performance of gradient-based inference, especially on tasks where the boundary alignment degree is low.
4.3 Illustrating DeLU-based contract design
We first illustrate the DeLU approach on a randomly generated contract design instance, in this case with 2 outcomes and 1,000 actions. In this instance, we sample the values for outcomes and costs for actions uniformly from [0, 10]. The outcome distribution for an action is further generated by applying SoftMax to a Gaussian random vector in  $\mathbb{R}^m$ . In Fig. 1 (a), we show the exact surface of the principal's utility function  $u^p$  on such a randomly generated instance. We observe that  $u^p$  is a discontinuous, piecewise affine function, where each piece corresponds to an action of the agent. In Fig. 1 (b) and (c), we show the  $u^p$  surface approximated by each of a ReLU and a DeLU network trained with  $40K$  samples. These two networks have a similar architecture, with a single hidden layer of 8 ReLU units. The difference is the additional, last-layer biases of the DeLU network, which are dependent on activation patterns. Whereas the ReLU network cannot represent the discontinuity of  $u^p$ , and thus gives an incorrect optimal contract, the DeLU network replicates the  $u^p$  surface, and gives an accurate optimal contract (Fig. 1 (a)). Another interesting observation is that the DeLU network may use multiple pieces to represent an original linear region (Fig. 1 (c)). In Fig. 3, we further illustrate the inference process of the gradient-based method on this instance.
In this section, we design experiments to study the following aspects of DeLU contract design: (1) Optimality (Sec. 5.1): Can DeLU networks give solutions close to the optimal contracts? How do the solutions compare against those generated by continuous neural networks? (2) Sample efficiency (Sec. 5.1): How many training samples are required for accurate DeLU approximation? Is the proposed method applicable to large-scale problems? (3) Time efficiency (Sec. 5.2): How does the computation overhead required for DeLU learning and inference compare to those of other solvers? (4) Inference (Sec. 5.2): Does the gradient-based inference method provide a good tradeoff between accuracy and time efficiency? (5) Boundary alignment degree (Sec. 5.3): How does the boundary alignment degree affect optimality?
Problem generation. Experiments are carried out on random synthetic examples. The outcome distributions  $p(\cdot |a)$  are generated by applying SoftMax on a Gaussian random vector in  $\mathbb{R}^m$ . The outcome value  $v_{o}$  is uniform on [0, 10]. The action cost is a mixture,  $c(a) = (1 - \beta_p)c_r(a) + \beta_p c_i(a)$ , where  $c_{r}(a) = \alpha_{p}\mathbb{E}_{o\sim p(\cdot |a)}[v_{o}]$  for scaling factor  $\alpha_{p} > 0$  is a correlated cost that is proportional to the expected value of the action,  $c_{i}(a)$  is an independent cost and uniform on [0, 1], and  $\beta_{p}$  controls the weight of the independent cost. We test different problem sizes by changing the number of outcomes  $m$  and actions  $n$ . For each problem size, we test various combinations of  $(\alpha_{p},\beta_{p})$  to consider the influence of correlation costs. Different methods are compared on the same set of problems.
Figure 5: (a) DeLU training and inference time compared against Oracle LP. (b) DeLU performance on large-scale problems. (c) DeLU performance with a small number of training samples.
Figure 6: Comparing the optimality of the two inference methods for solving the global optimum given a learned DeLU network, considering increasing problem sizes.
5.1 Optimality and efficiency.
In Fig. 4, we compare DeLU against ReLU networks as well as a baseline linear programming (LP) solver (Oracle LP). Oracle LP refers to the use of LP for directly solving the contract design problem, not for inference on a trained DeLU network. It solves  $n$  LP problems, one for each action  $a$ , where the objective is to maximize  $u^p$  with the incentive compatibility (IC) constraints associated with the action  $a$ . The best of these  $n$  solutions becomes the optimal contract. Oracle LP has access to the outcome distributions  $p(\cdot | a)$  (to construct the IC constraints) that are unobservable to the DeLU and ReLU learners, but gives a benchmark for the optimality of the proposed method.
We test different problem sizes in Fig. 4. Specifically, we set the number of outcomes  $m$  to 25 (1 $^{\text{st}}$  column), 50 (2 $^{\text{nd}}$  column), and 100 (3 $^{\text{rd}}$  column) and increase the number of actions  $n$  from  $2^2$  to  $2^8$ . For each problem size, 12 combinations of  $\alpha_p$  and  $\beta_p$  are tested, with  $(\alpha_p, \beta_p) \in \{0.5, 0.7, 0.9\} \times \{0, 0.3, 0.6, 0.9\}$ . The median performance as well as the first and third quartile (shaped area) of these 12 combinations are shown. When reporting optimality, we normalize the principal's utility achieved by DeLU/ReLU LP-based inference via dividing them by the value returned by Oracle LP. To ensure a fair comparison, DeLU and ReLU networks have the same architecture, with one hidden layer of 32 hidden units. They are trained for 100 epochs with  $50K$  random samples.
DeLU consistently achieves better optimality than ReLU networks  $(+28.29\%)$  and the best contract in the training set  $(+23.84\%)$  across all problem sizes. The performance gap is particularly large for large-scale problems. For example, when the number of outcomes is larger than 1,000 (Fig. 5 (b)), the ReLU networks return solutions much worse  $(< 20\%)$  than training samples, while DeLU can obtain a solution at least 1.7 times better than the best training data. As for sample efficiency, as shown in Fig. 5 (c), DeLU achieves near-optimality even with a very small training set.
5.2 Comparing the two DeLU inference methods.
In Fig. 6, we fix  $m$  to 25, 50, 100 and increase  $n$  from  $2^{2}$  to  $2^{8}$  to compare the optimality of the proposed inference methods. We again test the same 12 combinations of  $(\alpha_{p}, \beta_{p})$ . The median performance and the first and third quartile are shown for each problem size. Gradient-based inference is parallelized for  $50K$  training samples on GPUs, while LP inference is parallelized for 5 linear pieces on CPUs. We can see that LP inference consistently provides a better solution, as the optimality of gradient-based inference  $(-7.56\%)$  is limited by the number of gradient descent steps.
The advantage of gradient-based inference is its time efficiency. In Fig. 5 (a), we compare the inference time for different problem sizes (with  $m$  fixed to 25). Gradient-based inference saves around  $50 - 90\%$  overhead compared to LP inference. We also note that the DeLU training and
Figure 7: Correlation between boundary alignment degree (x-axis) and DeLU optimality (y-axis; left: LP-based inference; middle: gradient-based inference); also, the improvement achieved by sub-argmax inference compared to gradient-based inference (right). Trend lines (linear regression) are shown. Point sizes  $(\log(mn))$  indicate the corresponding problem size.
inference costs do not increase with the problem size. By comparison, the overhead of the direct LP solver grows quickly with the problem scale.
5.3 Boundary alignment degree and its influence on DeLU optimality
In Fig. 7, we study the relationship between DeLU performance and the degree of boundary alignment.
For each contract design problem, we first calculate the boundary alignment degree achieved by the DeLU network. For this, we test a large number (50K) of contracts, and check whether they are simultaneously on the DeLU model and true utility boundary. Specifically, for each contract we randomly sample 10K directions and assess linearity of the DeLU utility model and the true utility function in each direction. The principal's utility function is piecewise linear in the proximity of an interior point. Conversely, when a point lies on a boundary, there is a jump in utility within its proximity, rendering the utility unable to pass a linearity test in some direction. In particular, if a function is non-linear in  $>20\%$  of random directions, we mark the contract sample as being on a boundary (we use the exact same approach to check for boundaries of the DeLU utility function and the principal's true utility function). The boundary alignment degree is calculated as the percentage of overlapped boundary points (# contract samples on both DeLU and true boundaries / # contract samples on true boundaries).
Each point in Fig. 7 represents a contract design problem, and we use the same set of problems as in the previous experiments. The x-axis is the DeLU boundary alignment degree, and the y-axis is the optimality of DeLU with LP-based inference (Fig. 7 left) and gradient-based inference (Fig. 7 middle). The right plot gives the optimality improvement achieved by the sub-argmax inference method compared to standard gradient-based inference. From Fig. 7 left, we observe that DeLU achieves good boundary alignment degrees ( $>80\%$ ) for most contract design problems, and that a strong positive correlation exists between the boundary alignment degree and the optimality of LP-based inference. This result indicates that the alignment degree between DeLU and true boundaries is important in supporting good performance of LP-based inference. A similar observation can be made for gradient-based inference. Fig. 7-right shows that as the boundary alignment degree increases, the optimality improvement from the sub-argmax inference compared to standard gradient-based inference becomes less significant. This confirms that the sub-argmax inference is especially helpful for contract design problems where the DeLU boundaries are less accurate.
6 Closing Remarks
This paper initiates the investigation of contract design from a deep learning perspective, introducing a family of piecewise discontinuous networks and inference techniques that are tailored for deep contract learning. In future work, it will be interesting to take this framework to real-world settings, provide theory in regard to the expressiveness of the function class comprising these piecewise discontinuous functions, and extend to the setting of online learning. We expect that our exploration of discontinuous networks can also draw attention to other economics problems involving discontinuity and thereby contribute to advancing AI progress in computational economics.
This work received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement: 101077862, project name ALGO-CONTRACT). We extend our heartfelt appreciation to the anonymous NeurIPS reviewers for their insightful questions and constructive interactions during the review process, which has inspired us to delve deeper into critical inquiries, such as the boundary alignment issue. Their feedback has been important in shaping the quality and depth of our work.
[1] Abien Fred Agarap. Deep learning using rectified linear units (ReLU). arXiv preprint arXiv:1803.08375, 2018.
[2] Marat Akhmet and Enes Yilmaz. Neural networks with discontinuous/impact activations. Springer, 2014.
[3] Tal Alon, Magdalen Dobson, Ariel D. Procaccia, Inbal Talgam-Cohen, and Jamie Tucker-Foltz. Multiagent evaluation mechanisms. In AAAI 2020, pages 1774-1781, 2020.
[4] Tal Alon, Paul Dütting, Yingkai Li, and Inbal Talgam-Cohen. Bayesian analysis of linear contracts. In EC '23, page 66. ACM, 2023.
[5] Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units. In ICLR 2018, 2018.
[6] Moshe Babaioff, Michal Feldman, Noam Nisan, and Eyal Winter. Combinatorial agency. Journal of Economic Theory, 147(3):999-1034, 2012.
[7] Moshe Babaioff, Nicole Immorlica, Brendan Lucier, and S Matthew Weinberg. A simple and approximately optimal mechanism for an additive buyer. Journal of the ACM, 67(4):1-40, 2020.
[8] Patrick Bolton and Mathias Dewatripont. Contract theory. MIT press, 2004.
[9] Tilman Börgers. An introduction to the theory of mechanism design. Oxford University Press, USA, 2015.
[10] Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. An algorithmic characterization of multi-dimensional mechanisms. In STOC 2012, pages 459-478, 2012.
[11] Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. Optimal multi-dimensional mechanism design: Reducing revenue to welfare maximization. In FOCS 2012, pages 130-139, 2012.
[12] Yang Cai, Nikhil R Devanur, and S Matthew Weinberg. A duality based unified approach to Bayesian mechanism design. In STOC 2016, pages 926-939, 2016.
[13] Gabriel Carroll. Robustness and linear contracts. American Economic Review, 105(2):536-63, 2015.
[14] Gabriel Carroll. Robustness in mechanism design and contracting. Annual Review of Economics, 11:139-166, 2019.
[15] Gabriel Carroll. Contract theory. In Federico Echenique, Nicole Immorlica, and Vijay V. Vazirani, editors, Online and Matching-Based Market Design. Cambridge University Press, 2022. To appear.
[16] Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Bayesian agency: Linear versus tractable contracts. Artificial Intelligence, 307:103684, 2022.
[17] Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Designing menus of contracts efficiently: The power of randomization. In EC 2022, pages 705-735, 2022.