Title: Deep Contract Design via Discontinuous Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: Automated contract design

discontinuous neural networks
Abstract: Contract design involves a principal who establishes contractual agreements about payments for outcomes that arise from the actions of an agent. In this paper, we initiate the study of deep learning for the automated design of optimal contracts. We introduce a novel representation: the Discontinuous ReLU (DeLU) network, which models the principal's utility as a discontinuous piecewise affine function of the design of a contract where each piece corresponds to the agent taking a particular action. DeLU networks implicitly learn closed-form expressions for the incentive compatibility constraints of the agent and the utility maximization objective of the principal, and support parallel inference on each piece through linear programming or interior-point methods that solve for optimal contracts. We provide empirical results that demonstrate success in approximating the principal's utility function with a small number of training samples and scaling to find approximately optimal contracts on problems with a large number of actions and outcomes.

Deep Contract Design via Discontinuous Networks

Abstract

Contract design involves a principal who establishes contractual agreements about payments for outcomes that arise from the actions of an agent . In this paper, we initiate the study of deep learning for the automated design of optimal contracts. We introduce a novel representation: the Discontinuous ReLU (DeLU) network , which models the principal's utility as a discontinuous piecewise affine function of the design of a contract where each piece corresponds to the agent taking a particular action. DeLU networks implicitly learn closed-form expressions for the incentive compatibility constraints of the agent and the utility maximization objective of the principal, and support parallel inference on each piece through linear programming or interior-point methods that solve for optimal contracts. We provide empirical results that demonstrate success in approximating the principal's utility function with a small number of training samples and scaling to find approximately optimal contracts on problems with a large number of actions and outcomes.

1 Introduction

Contract theory studies the setting where a principal seeks to design a contract for rewarding an agent on the basis of the uncertain outcomes caused by the agent's private actions [8, 45]. Typical examples include a landlord who enters into a summer rental with a contract that includes penalties in the case of damage; a homeowner who engages a firm to complete a kitchen renovation with a contract that conditions payments on timely completion or functioning appliances; or an individual who employs a freelancer to do some design work with a contract that includes bonuses for completing the job.

A contract specifies payments to the agent, conditioned on outcomes. The principal is self-interested, with a value for each outcome and a cost for making payments. The agent is also self-interested, and responds to a contract by choosing an action that maximizes its expected utility (expected payment minus the cost of an action). The problem is to find a contract that maximizes the principal's utility (expected value minus expected payment), given that the agent will best respond to the contract. In economics, this is referred to as a problem of moral hazard , in that the agent is willing to privately act in its best interest given the contract (the 'hazard" is that the behavior of the agent may be to the detriment of the principal).

The importance of contract design is evidenced by the 2016 Nobel Prize awarded to O. Hart and B. Holmström [51] and its broad application to real-world problems. Contract design is one of the three fundamental problems in the realm of economics involving asymmetric information and incentives, along with mechanism design [9] and signalling (Bayesian persuasion) [41]. However, while

Dmitry Ivanov

(a)

𝒖

𝒑

function

(b)

ReLU

approximation (

baseline

)

(c)

DeLU

approximation

DeLU

&

Real

Opt.

ReLU

Opt.

3

0

both mechanism design [10, 11, 12, 7, 36, 37] and signalling [26, 27, 18] have been studied extensively from a computational perspective, the contract design problem has only recently received attention and presented distinct computational challenges. For many combinatorial contract settings [6], the conventional approach based on linear programming becomes computationally infeasible and the problem of finding or even approximating the optimal contract becomes intractable [33, 28, 29]. In addition, learning-theoretic results give worst-case exponential sample complexity bounds for learning an approximately-optimal contract [40, 55]. As a result, there is a growing demand for a scalable, general purpose, and beyond-worst case approach for computing (near-)optimal contracts.

We are thus motivated to initiate the study of deep learning for optimal contract design ( deep contract design ). This falls within the broader framework of differentiable economics , which seeks to leverage parameterized representations of differentiable functions for the purpose of optimal economic design [30]. In regard to learning contracts, recent work [40, 21, 55, 31] considers the distinct setting of online learning for contract design with bandit feedback, characterizing regret bounds without appeal to deep learning.

Afirst innovation of this paper is to introduce a neural network architecture that can well-approximate the principal's utility function. A close examination of the geometry of the principal's utility function (Sec. 3) reveals its similarity to fully-connected feed-forward neural networks with ReLU activations [1]: both of them are piecewise affine functions [22]. However, whereas a ReLU network models a continuous function, the principal's utility function is discontinuous at the boundary of linear regions, where the best response of the agent changes. Accurate approximation in the vicinity of boundaries is critical because an optimal contract is always located on the boundary (Lemma 3 in Sec. 3), but this is exactly where ReLU approximation error can be large. To handle this, we introduce the Discontinuous ReLU (DeLU) network, which recognizes that the linear regions of a ReLU network are decided by activation patterns (the status of all activation units in the network), and conditions a piecewise bias on these activation patterns. In this way, each linear region has a different bias parameter and can be discontinuous at the boundaries. Fig. 1 illustrates the DeLU and its use to represent the principal's utility function, contrasting this with a continuous ReLU function.

A second innovation in this paper is to introduce a scalable inference technique that can find the contract that maximizes the network output (i.e., the principal's utility). We show that linear regions (pieces) of DeLU networks implicitly learn closed-form expressions for the incentive constraints of the agent and the utility maximization objective of the principal, so that we can use linear programming (LP) on each piece to find the global optimum. However, the time efficiency of this LP method is impeded by the overhead of solving individual LPs, and worsens with problem size in the absence of suitable parallel computing resources. To increase the computational efficiency, we develop a gradient-based inference algorithm based on the interior-point method [49] that only requires a few forward and backward passes of the DeLU network to find the contract that maximizes the network output. This method scales well to large-scale problems and can be readily run in parallel on GPUs.

The effectiveness of introducing piecewise discontinuity into neural networks is demonstrated by our experimental results. The DeLU network well-approximates the principal's utility function even with

a small number of training samples, and the inference method remains accurate and efficient as the problem size grows. By synergistically harnessing these two innovations, our method consistently finds near-optimal solutions on a wide range of contract design problems, significantly surpassing the capability of conventional continuous networks.

Related work. A longstanding challenge in the deep learning community has been to approximate discontinuous functions with neural networks. While the Universal Approximation Theorem guarantees the approximation of continuous functions, many problems involve discontinuity, including solar flare imaging [46] and problems coming from mathematics [24]. However, establishing a network to represent discontinuous functions is not an easy task. Discontinuities were considered as early as in the 1950s, when Rosenblatt [50] introduced the perceptron model and its single-layer of step activation functions. Following this work, others modeled discontinuity through the use of different discontinuous activation functions [34, 2, 44]. However, training these models is more challenging than the more typical models that make use of continuous activation functions [24], and this has hindered their application. To the best of our knowledge, this paper presents the first discontinuous network architecture with continuous activation functions and stable optimization performance.

There is a vast and still-growing economics literature on contracts [see, e.g., 52, 15], to which the computational lens has recently been applied. In classic settings, computing the optimal contract is tractable by solving one LP per agent's action, and taking the overall best solution [see, e.g., 32]. However, LP becomes computationally infeasible for combinatorial contracts, including settings with exponentially-many outcomes [33], actions [28], or agent combinations [29]. Another issue with the LP-based approach is that it requires complete information in regard to the problem facing the agent (its type ). If the agent has a hidden type, this makes the optimal contract hard to compute [16, 38, 17], or otherwise complex [4]. One way to deal with these complexities is to focus on simple contracts, in particular linear contracts (commission-based), which are popular in practice [e.g., 13, 32, 14].

A distinct but related approach combines contract design with learning, where efforts have concentrated on online learning theory [40, 21, 55, 31]. These works show exponential lower bounds on the achievable regret for general contracts in worst-case settings, and sublinear regret bounds for linear contracts. Additionally, contracts have been studied from the perspective of multi-agent reinforcement learning [48, 19, 25]. The problem of strategic classification has also established a formal connection between strategy-aware classifiers and contracts [43, 42, 3]. There is also interest in the application of contract theory to the AI alignment problem [39]. However, the use of learning methods for solving optimal contracts, as opposed to auctions [30], remains largely unexplored. This gap in the literature can be partly attributed to the inherent complexity of finding optimal contracts, as the principal utility depends on the agent's action, which must conform to incentive compatibility (IC) constraints that are not observable by the principal.

2 Preliminaries

Offline contract learning problem. The contract design problem is defined with elements C = ⟨A , O , F , c, p, v ⟩ , and involves a single principal and a single agent. The agent selects an action a in the finite action space A , |A| = n . Action a leads to a distribution p ( ·| a ) over the outcomes in O , |O| = m , and incurs a cost c ( a ) ∈ R ≥ 0 to the agent. The valuation of each outcome o j ∈ O for the principal is decided by the value function v : O → R ≥ 0 . The principal sets up a contract , f ∈ F ⊂ R m ≥ 0 , which influences the action selected by the agent. A contract f = ( f 1 , f 2 , · · · , f m ) ⊤ specifies the payment f j ≥ 0 made to the agent by the principal in the event of outcome o j . On receiving a contract f , the agent selects the action a ∗ ( f ) maximizing its utility u a ( f ; a ) = E o ∼ p ( ·| a ) [ f o ] -c ( a ) . For a given action a of the agent, the principal gets utility u p ( f ; a ) = E o ∼ p ( ·| a ) [ v o -f o ] , which is the principal's expected value minus payment. In the case that the induced action is the best response of the agent given the contract, we write u p ( f ) = E o ∼ p ( ·| a ∗ ( f )) [ v o -f o ] . The goal in optimal contract design is to find the contract that maximizes the principal's utility without access to p ( ·| a ) and the action taken by the agent:

ReLU piecewise-affine networks and activation patterns. A fully-connected neural network with a piecewise linear activation function (e.g., ReLU and leaky ReLU) and a linear output layer represents a continuous piecewise affine function [5, 23]. A piecewise affine function is defined as follows.

̸

Definition 1 [Piecewise affine function] . A function g : R d → R is piecewise affine if there exists a finite set of polytopes {D i } P i =1 such that ∪ P i =1 D i = R d , D i ∩ D j = i = ∅ , and g is a linear function ρ i : D i → R when restricted to D i . We call ρ i a linear piece of g .

We now follow [22] and introduce some local properties of the piecewise affine function that is represented by a ReLU network. Suppose there are L hidden layers in a network g , with sizes [ n 1 , n 2 , · · · , n L ] . W ( l ) ∈ R n l × n l -1 and b ( l ) ∈ R n l are the weights and biases of layer l . Let n 0 = d denote the input space dimension. We consider a ReLU network with one-dimensional outputs, and the output layer has weights W ( L +1) ∈ R 1 × n L and a bias b ( L +1) ∈ R . With input x ∈ R d , we have the pre- and post-activation output of layer l : h ( l ) ( x ) = W ( l ) o ( l -1) ( x ) + b ( l ) and o ( l ) ( x ) = σ ( h ( l ) ( x ) ) , where σ : R → R is an activation function . In this paper, we consider ReLU activation σ ( x ) = max { x, 0 } [35, 54], but the proposed method can be extended to other piecewise linear activation functions (e.g., LeakyReLU and PReLU [54]). For each hidden unit, the ReLU activation status has two values, defined as 1 when pre-activation h is positive and 0 when h is strictly negative. The activation pattern of the entire network is defined as follows.

Definition 2 [Activation Pattern] . An activation pattern of a ReLU network g with L hidden layers is a binary vector r = [ r (1) , · · · , r ( L ) ] ∈ { 0 , 1 } ∑ L l =1 n l , where r ( l ) is a layer activation pattern indicating activation status of each unit in layer l .

The activation pattern depends on the input x , and we define function r : R d →{ 0 , 1 } ∑ L l =1 n l that maps the input to the corresponding activation pattern. For a ReLU network, inputs that have the same activation pattern lie in a polytope, and the activation pattern determines the boundaries of this polytope. To see this, we can write the output of layer l , h ( l ) ( x ) , as

where R ( k ) is a diagonal matrix with diagonal elements equal to the layer activation pattern r ( k ) . Eq. 2 indicates that, when r is fixed, h ( l ) is a linear function h ( l ) ( x ) = M ( l ) x + z ( l ) , where M ( l ) = W ( l ) ( ∏ l -1 k =1 R ( l -k ) ( x ) W ( l -k ) ) and z ( l ) = b ( l ) + ∑ l -1 k =1 ( ∏ l -k j =1 W ( l +1 -j ) R ( l -j ) ( x ) ) b ( k ) . We thus get ∑ L n l half-spaces, with the half-space corresponding to unit i of layer l defined as:

where M ( l ) i y + z ( l ) i is the output of unit i at layer l , and ∆ ( l ) i is 1 if h ( l ) i ( x ) is positive, and is -1 otherwise. The input x is in the polytope that is defined by the intersection of these half-spaces: D ( x ) = ∩ l =1 , ··· ,L ∩ i =1 , ··· ,n l Γ l,i . When restricted to D ( x ) , the ReLU network is a linear function: g ( x ) = W ( L +1) R ( L ) h ( L ) ( x ) + b ( L +1) .

Interior-point method for optimization problems with inequality constraints. For a minimization problem with objective function q ( x ) and inequality constraints p i ( x ) > 0 , i = 1 , · · · , M , the interior-point method [53] introduces a logarithmic barrier function , ϕ ( x ) = -∑ M i =1 log( p i ( x )) , and finds the minimizer of q ( x ) + 1 t ϕ ( x ) , for some t > 0 . This new objective function is defined on the set of strictly feasible points { x | p i ( x ) > 0 , i = 1 , · · · , M } , and approximates the original objective as t becomes large. Given this, we can solve for a series of optimization problems for increasing values of t . In the k -th round, t ( k ) is set to µ · t ( k -1) , where µ > 1 is a constant, t (0) > 0 is an initial value, and the problem is solved (e.g., by Newton initialized at x ( k -1) ) to yield x ( k ) . Assume that we solve the barrier problem exactly for each iterate, then to achieve a desired accuracy level of ϵ > 0 , we need n barrier = log( M/ ( t (0) ϵ )) / log( µ ) rounds of optimization.

3 Geometry of Optimal Contracts

In this section, we introduce some properties of the principal's utility function, u p : F → R , which will motivate our method in Sec. 4. First, we show that u p is a piecewise affine function.

Lemma 1. The principal's utility function u p is a piecewise affine function.

l =1

Proof. The principal utility depends on the agent action. For contracts in the intersection of the following n -1 half spaces that represent incentive compatibility constraints , the agent's action is a i :

̸

̸

Moreover, when agent action a i remains unchanged, u p changes linearly with f : αu p ( f ; a i ) + βu p ( f ′ ; a i ) = α E o ∼ p ( ·| a i ) [ v o -f o ] + β E o ∼ p ( ·| a i ) [ v o -f ′ o ] = E o ∼ p ( ·| a i ) [ v o -( αf o + βf ′ o )] = u p ( α f + β f ′ ; a i ) . Therefore, when restricted to the polytope Q i = ∩ j = i Γ i,j , ∀ i , u p is linear.

Based on Lemma 1, we define a linear piece of function u p as µ p i : Q i → R , where Q i defines the set of contracts that motivates the agent to take action i . Lemma 1 can be easily extended to the agent's utility function u a , and the linear pieces of function u p and u a share the same set of domains {Q i } . We define a linear piece of function u a as µ a i : Q i → R . We next observe:

Lemma 2. The principal's utility function u p can be discontinuous on the boundary of linear pieces.

̸

The proof in Appx. A follows the idea that the agent is indifferent between action a i and a j given a contract f on the boundary of two neighboring linear pieces µ p i and µ p j = i : µ a i ( f ) = µ a j ( f ) = u a ( f ) . However, the principal's utility equals the expected value minus the cost of an action minus u a ( f ) . As the expected value minus the cost can be different in µ p i and µ p j , u p can be discontinuous at f .

We then define the optimality of a contract and analyze the structure of optimal contracts.

Definition 3 [Piecewise and Global Optimal Contracts] . A contract f ∗ i ∈ Q i is piecewise optimal if u p ( f ∗ i ) ≥ u p ( f ) , ∀ f ∈ Q i . A contract f ∗ is global optimal if u p ( f ∗ ) ≥ u p ( f ) , ∀ f ∈ F .

Lemma 3. The global optimal contract is on the boundary of a linear piece.

Lemma 3 can be proved by contradiction (as detailed in Appx. A): if a global optimal contract is not on the boundary, we can always find a solution with greater principal utility. As analyzed in Sec. 4, Lemma 3 serves as a compelling rationale for introducing our discontinuous networks. Besides discontinuity, there is another network design consideration motivated by the following property.

Lemma 4. The principal's utility function u p can be written as a summation of a concave function and a piecewise constant function.

The proof in Appx. A commences by establishing the convexity of function u a and subsequently formulating u p as a function of u a . This property motivates us to introduce concavity into our network design. In Appx. C, we discuss how to achieve this by imposing non-negativity constrains on network weights and analyze the impact of this design choice on the performance.

4 DeLU Neural Networks

Given the piecewise-affine geometry, the preceding analysis shows a close connection between the principal's utility function u p (Lemma 1) and fully-connected ReLU networks (Sec. 2). However, ReLU functions are continuous and cannot represent the abrupt changes in u p at the boundaries of the linear pieces. This lack of representational capacity is problematic because the optimal contracts are on the boundary (Lemma 3), which is precisely where the ReLU approximation errors can be large. Consequently, ReLU networks are not well suited to deep contract design. In this section, we introduce the new Discontinuous ReLU (DeLU) network , which provides a discontinuity at boundaries between pieces , making it a suitable function approximator for the u p function.

4.1 Architecture

The DeLU network architecture supports different biases for different linear pieces. Since a linear piece can be identified by the corresponding activation pattern, we propose to condition these piecewise biases on activation patterns. Specifically, we learn a DeLU network ξ : F → R (Fig. 2) to approximate the principal's utility function, mapping a contract to the corresponding utility of the principal. The first part of a DeLU network is a sub-network similar to a conventional ReLU network. This sub-network η has L fully-connected hidden layers with ReLU activation and a weight matrix at the output layer, i.e., η is parameterized as θ η = { W (1) , b (1) , · · · , W ( L ) , b ( L ) , W ( L +1) } , which includes weights and biases for L hidden layers and weights for the output ( ( L +1) -th) layer. The

Data flow

in

𝜂

Data flow in the

bias network

𝜁

𝑾

("#$)

𝑾

($)

𝒇 + 𝒃

($)

𝑏

("#$)

Input contract

𝒇

Output

𝑾

"

𝒐

"&$

(𝒇) + 𝒃

(")

1

0

1

0

…

ReLU activation

…

0

ReLU activation

1

0

1

1

…

Layer 1

Layer

𝐿

Gradient

flow

MSE loss (

𝑢

!

(𝒇)

)

Sub-network

𝜂

𝜁

Algorithm 1 Parallel Gradient-Based Inference

Input: X (0) ∈ R K × m ; DeLU network ξ with trained parameters; ϵ ; µ > 1 ; t (0) ; α .

DeLU network is different from a conventional ReLU network at the bias of the output (last) layer ( b ( L +1) ), and we introduce a new method to generate the last-layer biases.

Since there are no activation units at the output layer, given an input contract f , we can obtain the activation pattern r ( f ) by a forward pass of the network up until the last layer. To condition b ( L +1) on the activation pattern, we train another sub-network ζ : [0 , 1] ∑ L l =1 n l → R , that maps r ( f ) to the bias of the output layer. We use a two-layer fully-connected network with Tanh activation to represent ζ and denote its parameters as θ ζ . In this way, contracts in the same linear piece of the DeLU network share the same bias value, enabling the network to express discontinuity at the boundaries while keeping other properties of network η unchanged. In Appx. B, we discuss why we adopt a neural network, instead of a simpler function, to model the bias term.

4.2 Training and inference

The DeLU network ξ , including sub-networks η and ζ , is end-to-end differentiable. For training, we randomly sample K contracts and the corresponding principal's utilities: T K = { ( f i , u p ( f i )) } K i =1 . Feeding a training sample f i as input, we get an approximated principal's utility: ξ ( f i ; θ η , θ ζ ) = η ( f i ; θ η ) + ζ ( r ( f i ); θ ζ ) , and the network ξ is trained to minimize the following loss function in T epochs (Appx. D gives more details on network architecture, infrastructure, and training):

Given a trained DeLU network, ξ : F → R , we have an approximation of the principal's utility function. The next step is to find a contract that maximizes the learned utility function, that is f ∗ = arg max f ξ ( f ) . We call this the inference process . Since the function represented by a DeLU network is discontinuous, conventional first- or second-order optimization methods are not applicable, and, formally, we need to develop a suitable optimization approach to solve

where ρ i : D i → R is a linear piece of network ξ . This motivates us to first find the piecewise optimal contracts and then get the global optimum by comparing the piecewise optima.

4.2.1 Linear programming based inference

A first approach finds the optimal contract for each piece using linear programming (LP). Contracts in the same linear piece ρ result in the same activation pattern r , and the DeLU network is a linear function on the piece. In particular, the optimization problem of finding the piecewise optimum is:

(a) DeLU surface and

contour lines

of the barrier function

on one piece.

(b) Gradient-based

inference

(c)

Parallel

inference

Maximization

vector

where [ L ] = { 1 , · · · , L } , [ n l ] = { 1 , · · · , n l } , the objective is the linear function represented by the DeLU network on piece ρ , and the constraints require that the activation pattern remains r . The definitions of ∆ ( l ) i , R ( L ) , M ( l ) , and z ( l ) are the same as in Sec. 2, but with the activation pattern fixed to r . We omit the last-layer bias in the objective because the activation pattern is fixed for this piece, and thus the bias is constant and does not influence the piecewise optimal solution.

For each LP, there are m decision variables, representing payments for m outcomes, and N = ∑ L l =1 n l (the number of neurons) constraints. LPs for each piece can be solved in polynomial time of m and N , and quickly in practice via the simplex method. However, a challenge is that there are 2 N activation patterns. 1 For a DeLU network with a moderate size, we can enumerate all these pieces. For a larger DeLU network, we can approximate by collecting random contract samples and solving LPs with the corresponding activation patterns. Parallelizing these LPs may achieve better time efficiency. However, this improvement is limited by the overhead of solving a single LP and the required amount of suitable parallel computational resources. In the next section, we introduce a gradient-based method that provides better efficiency in solving single problems and better parallelism through making use of GPUs. We compare these two inference methods in Sec. 5.1.

4.2.2 Gradient-based inference

The gradient-based inference method is based on the interior-point method (Sec. 2) and finds piecewise optimal solutions via forward and backward passes of the DeLU network. Specifically, on each linear piece, ρ , we adopt the following barrier function :

At the k -th round, the objective function is

We use gradient descent initialized at f ( k -1) , and update f with ∂ϕ ( k ) ( f ) /∂ f to find the minimizer. A forward pass of the DeLU network gives ϕ ( k ) ( f ) and a backward pass is sufficient to calculate ∂ϕ ( k ) ( f ) /∂ f . Since forward and backward passes are naturally parallelized in modern deep learning frameworks, this method can be parallelized by processing multiple f simultaneously. Alg. 1 gives the matrix-form expression of this parallel computation. The input X (0) ∈ R K × m can be the training set or a random sample set, and we discuss the difference of these two settings in Appx. E.

Inference performance and boundary alignment degree . The performance of the proposed inference algorithms depends on the DeLU approximation quality, especially on the degree of alignment between the DeLU boundaries and the ground-truth linear piece boundaries. An interesting question is whether our training setup can achieve a high boundary alignment degree . A reason to think this is possible comes from observing that the MSE training loss (Eq. 5) is sensitive to misalignment between the DeLU and true boundaries. In particular, given that the jump of the utility function at boundary points can be arbitrarily large, a slight misalignment between DeLU and true boundaries can lead to a large increase in the MSE loss. Related to this, we explore an extension of the gradient-based inference method to make it more robust to possible boundary misalignment. When

2

4

6

8

0

20

40

60

80

100

2

4

6

8

0

20

40

60

80

100

2

4

6

8

0

20

40

60

80

100

DeLU

Best Training Data

ReLU

Linear Programming (Oracle, has access to unobservable information)

Optimality %

Optimality %

Optimality %

# Outcomes = 100

annealing the coefficient of the barrier function, we can check whether the principal's utility increases for each t ( k ) value. A non-increasing utility indicates that we encounter inaccurate boundaries, and we can stop the inference to seek more robustness. We call inference with this 'early stop" mechanism the sub-argmax gradient-based inference method .

In Sec. 5.3, we empirically evaluate the boundary alignment degree achieved by DeLU, and demonstrate that near-optimal contract-design performance is closely associated with high boundary alignment. We also show that the sub-argmax variation can improve performance of gradient-based inference, especially on tasks where the boundary alignment degree is low.

4.3 Illustrating DeLU-based contract design

We first illustrate the DeLU approach on a randomly generated contract design instance, in this case with 2 outcomes and 1,000 actions. In this instance, we sample the values for outcomes and costs for actions uniformly from [0 , 10] . The outcome distribution for an action is further generated by applying SoftMax to a Gaussian random vector in R m . In Fig. 1 (a), we show the exact surface of the principal's utility function u p on such a randomly generated instance. We observe that u p is a discontinuous, piecewise affine function, where each piece corresponds to an action of the agent. In Fig. 1 (b) and (c), we show the u p surface approximated by each of a ReLU and a DeLU network trained with 40 K samples. These two networks have a similar architecture, with a single hidden layer of 8 ReLU units. The difference is the additional, last-layer biases of the DeLU network, which are dependent on activation patterns. Whereas the ReLU network cannot represent the discontinuity of u p , and thus gives an incorrect optimal contract, the DeLU network replicates the u p surface, and gives an accurate optimal contract (Fig. 1 (a)). Another interesting observation is that the DeLU network may use multiple pieces to represent an original linear region (Fig. 1 (c)). In Fig. 3, we further illustrate the inference process of the gradient-based method on this instance.

5 Empirical Evaluation

In this section, we design experiments to study the following aspects of DeLU contract design: (1) Optimality (Sec. 5.1): Can DeLU networks give solutions close to the optimal contracts? How do the solutions compare against those generated by continuous neural networks? (2) Sample efficiency (Sec. 5.1): How many training samples are required for accurate DeLU approximation? Is the proposed method applicable to large-scale problems? (3) Time efficiency (Sec. 5.2): How does the computation overhead required for DeLU learning and inference compare to those of other solvers? (4) Inference (Sec. 5.2): Does the gradient-based inference method provide a good tradeoff between accuracy and time efficiency? (5) Boundary alignment degree (Sec. 5.3): How does the boundary alignment degree affect optimality?

Problem generation. Experiments are carried out on random synthetic examples. The outcome distributions p ( ·| a ) are generated by applying SoftMax on a Gaussian random vector in R m . The outcome value v o is uniform on [0 , 10] . The action cost is a mixture, c ( a ) = (1 -β p ) c r ( a ) + β p c i ( a ) , where c r ( a ) = α p E o ∼ p ( ·| a ) [ v o ] for scaling factor α p > 0 is a correlated cost that is proportional to the expected value of the action, c i ( a ) is an independent cost and uniform on [0, 1], and β p controls the weight of the independent cost. We test different problem sizes by changing the number of outcomes m and actions n . For each problem size, we test various combinations of ( α p , β p ) to consider the influence of correlation costs. Different methods are compared on the same set of problems.

2

4

6

8

0

200

400

600

800

1000

LP Infer

DeLU Train

Grad. Infer

Oracle LP

Time (s)

(a) Time Efficiency

(1K,1K)

(10K,10K)

(50K,2K)

(100K,1K)

(1M,100)

0

50

100

150

200

DeLU

ReLU

Problem size (# outcomes, # actions)

Opt. vs. Best Train Data %

(b) Large-Scale Problems

5

100

2

5

1000

2

5

10k

2

5

0

20

40

60

80

100

(n, m)

(5, 16)

(50, 16)

(50, 32)

# Training samples

Optimality %

(c) Limited Training Data

2

4

6

8

60

80

100

2

4

6

8

60

80

100

2

4

6

8

60

80

100

DeLU (LP-Based Inference)

DeLU (Gradient-Based Inference)

DeLU (Sub-argmax Gradient-Based Inference)

Optimality %

Optimality %

Optimality %

# Outcomes = 100

5.1 Optimality and efficiency.

In Fig. 4, we compare DeLU against ReLU networks as well as a baseline linear programming (LP) solver ( Oracle LP ). Oracle LP refers to the use of LP for directly solving the contract design problem, not for inference on a trained DeLU network. It solves n LP problems, one for each action a , where the objective is to maximize u p with the incentive compatibility (IC) constraints associated with the action a . The best of these n solutions becomes the optimal contract. Oracle LP has access to the outcome distributions p ( ·| a ) (to construct the IC constraints) that are unobservable to the DeLU and ReLU learners, but gives a benchmark for the optimality of the proposed method.

We test different problem sizes in Fig. 4. Specifically, we set the number of outcomes m to 25 ( 1 st column), 50 ( 2 nd column), and 100 ( 3 rd column) and increase the number of actions n from 2 2 to 2 8 . For each problem size, 12 combinations of α p and β p are tested, with ( α p , β p ) ∈ { 0 . 5 , 0 . 7 , 0 . 9 } × { 0 , 0 . 3 , 0 . 6 , 0 . 9 } . The median performance as well as the first and third quartile (shaped area) of these 12 combinations are shown. When reporting optimality, we normalize the principal's utility achieved by DeLU/ReLU LP-based inference via dividing them by the value returned by Oracle LP . To ensure a fair comparison, DeLU and ReLU networks have the same architecture, with one hidden layer of 32 hidden units. They are trained for 100 epochs with 50 K random samples.

DeLU consistently achieves better optimality than ReLU networks ( +28 . 29% ) and the best contract in the training set (+ 23 . 84% ) across all problem sizes. The performance gap is particularly large for large-scale problems. For example, when the number of outcomes is larger than 1 , 000 (Fig. 5 (b)), the ReLU networks return solutions much worse ( < 20% ) than training samples, while DeLU can obtain a solution at least 1 . 7 times better than the best training data. As for sample efficiency , as shown in Fig. 5 (c), DeLU achieves near-optimality even with a very small training set.

5.2 Comparing the two DeLU inference methods.

In Fig. 6, we fix m to 25, 50, 100 and increase n from 2 2 to 2 8 to compare the optimality of the proposed inference methods. We again test the same 12 combinations of ( α p , β p ) . The median performance and the first and third quartile are shown for each problem size. Gradient-based inference is parallelized for 50 K training samples on GPUs, while LP inference is parallelized for 5 linear pieces on CPUs. We can see that LP inference consistently provides a better solution, as the optimality of gradient-based inference ( -7 . 56% ) is limited by the number of gradient descent steps.

The advantage of gradient-based inference is its time efficiency. In Fig. 5 (a), we compare the inference time for different problem sizes (with m fixed to 25). Gradient-based inference saves around 50 -90% overhead compared to LP inference. We also note that the DeLU training and

0

0.5

1

40

60

80

100

Boundary alignment degree %

Opt. (LP inference) %

0

0.5

1

40

60

80

100

Boundary alignment degree %

Opt. (grad. inference) %

0

0.5

1

-4

-2

0

2

4

6

Boundary alignment degree %

Improvement by sub-argmax %

inference costs do not increase with the problem size. By comparison, the overhead of the direct LP solver grows quickly with the problem scale.

5.3 Boundary alignment degree and its influence on DeLU optimality

In Fig. 7, we study the relationship between DeLU performance and the degree of boundary alignment.

For each contract design problem, we first calculate the boundary alignment degree achieved by the DeLU network. For this, we test a large number (50K) of contracts, and check whether they are simultaneously on the DeLU model and true utility boundary. Specifically, for each contract we randomly sample 10K directions and assess linearity of the DeLU utility model and the true utility function in each direction. The principal's utility function is piecewise linear in the proximity of an interior point. Conversely, when a point lies on a boundary, there is a jump in utility within its proximity, rendering the utility unable to pass a linearity test in some direction. In particular, if a function is non-linear in >20% of random directions, we mark the contract sample as being on a boundary (we use the exact same approach to check for boundaries of the DeLU utility function and the principal's true utility function). The boundary alignment degree is calculated as the percentage of overlapped boundary points (# contract samples on both DeLU and true boundaries / # contract samples on true boundaries).

Each point in Fig. 7 represents a contract design problem, and we use the same set of problems as in the previous experiments. The x-axis is the DeLU boundary alignment degree, and the y-axis is the optimality of DeLU with LP-based inference (Fig. 7 left) and gradient-based inference (Fig. 7 middle). The right plot gives the optimality improvement achieved by the sub-argmax inference method compared to standard gradient-based inference. From Fig. 7 left, we observe that DeLU achieves good boundary alignment degrees (>80%) for most contract design problems, and that a strong positive correlation exists between the boundary alignment degree and the optimality of LPbased inference. This result indicates that the alignment degree between DeLU and true boundaries is important in supporting good performance of LP-based inference. A similar observation can be made for gradient-based inference. Fig. 7-right shows that as the boundary alignment degree increases, the optimality improvement from the sub-argmax inference compared to standard gradient-based inference becomes less significant. This confirms that the sub-argmax inference is especially helpful for contract design problems where the DeLU boundaries are less accurate.

6 Closing Remarks

This paper initiates the investigation of contract design from a deep learning perspective, introducing a family of piecewise discontinuous networks and inference techniques that are tailored for deep contract learning. In future work, it will be interesting to take this framework to real-world settings, provide theory in regard to the expressiveness of the function class comprising these piecewise discontinuous functions, and extend to the setting of online learning. We expect that our exploration of discontinuous networks can also draw attention to other economics problems involving discontinuity and thereby contribute to advancing AI progress in computational economics.

References

Abien Fred Agarap. Deep learning using rectified linear units (ReLU). arXiv preprint arXiv:1803.08375 , 2018.

Marat Akhmet and Enes Yılmaz. Neural networks with discontinuous/impact activations . Springer, 2014.

Tal Alon, Magdalen Dobson, Ariel D. Procaccia, Inbal Talgam-Cohen, and Jamie Tucker-Foltz. Multiagent evaluation mechanisms. In AAAI 2020 , pages 1774-1781, 2020.

Tal Alon, Paul Dütting, Yingkai Li, and Inbal Talgam-Cohen. Bayesian analysis of linear contracts. In EC '23 , page 66. ACM, 2023.

Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. Understanding deep neural networks with rectified linear units. In ICLR 2018 , 2018.

Moshe Babaioff, Michal Feldman, Noam Nisan, and Eyal Winter. Combinatorial agency. Journal of Economic Theory , 147(3):999-1034, 2012.

Moshe Babaioff, Nicole Immorlica, Brendan Lucier, and S Matthew Weinberg. A simple and approximately optimal mechanism for an additive buyer. Journal of the ACM , 67(4):1-40, 2020.

Patrick Bolton and Mathias Dewatripont. Contract theory . MIT press, 2004.

Tilman Börgers. An introduction to the theory of mechanism design . Oxford University Press, USA, 2015.

Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. An algorithmic characterization of multi-dimensional mechanisms. In STOC 2012 , pages 459-478, 2012.

Yang Cai, Constantinos Daskalakis, and S Matthew Weinberg. Optimal multi-dimensional mechanism design: Reducing revenue to welfare maximization. In FOCS 2012 , pages 130-139, 2012.

Yang Cai, Nikhil R Devanur, and S Matthew Weinberg. A duality based unified approach to Bayesian mechanism design. In STOC 2016 , pages 926-939, 2016.

Gabriel Carroll. Robustness and linear contracts. American Economic Review , 105(2):536-63, 2015.

Gabriel Carroll. Robustness in mechanism design and contracting. Annual Review of Economics , 11:139-166, 2019.

Gabriel Carroll. Contract theory. In Federico Echenique, Nicole Immorlica, and Vijay V. Vazirani, editors, Online and Matching-Based Market Design . Cambridge University Press, 2022. To appear.

Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Bayesian agency: Linear versus tractable contracts. Artificial Intelligence , 307:103684, 2022.

Matteo Castiglioni, Alberto Marchesi, and Nicola Gatti. Designing menus of contracts efficiently: The power of randomization. In EC 2022 , pages 705-735, 2022.

Yu Cheng, Ho Yee Cheung, Shaddin Dughmi, Ehsan Emamjomeh-Zadeh, Li Han, and ShangHua Teng. Mixture selection, mechanism design, and signaling. In FOCS 2015 , pages 1426-1445, 2015.

Phillip J.K. Christoffersen, Andreas A. Haupt, and Dylan Hadfield-Menell. Get it in writing: Formal contracts mitigate social dilemmas in multi-agent RL. In AAMAS 2023 , page 448-456, 2023.

Lingyang Chu, Xia Hu, Juhua Hu, Lanjun Wang, and Jian Pei. Exact and consistent interpretation for piecewise linear neural networks: A closed form solution. In KDD 2018 , pages 1244-1253, 2018.

Alon Cohen, Argyrios Deligkas, and Moran Koren. Learning approximately optimal contracts. In SAGT 2022 , pages 331-346, 2022.

Francesco Croce, Maksym Andriushchenko, and Matthias Hein. Provable robustness of ReLU networks via maximization of linear regions. In AISTATS 2019 , pages 2057-2066, 2019.

Francesco Croce and Matthias Hein. A randomized gradient-free attack on ReLU networks. In GCPR 2019 , pages 215-227, 2019.

Francesco Della Santa and Sandra Pieraccini. Discontinuous neural networks and discontinuity learning. Journal of Computational and Applied Mathematics , 419:114678, 2023.

Heng Dong, Tonghan Wang, Jiayuan Liu, Chi Han, and Chongjie Zhang. Birds of a feather flock together: A close look at cooperation emergence via multi-agent rl. arXiv preprint arXiv:2104.11455 , 2021.

Shaddin Dughmi. On the hardness of signaling. In FOCS 2014 , pages 354-363, 2014.

Shaddin Dughmi and Haifeng Xu. Algorithmic Bayesian persuasion. In STOC 2016 , pages 412-425, 2016.

Paul Dütting, Tomer Ezra, Michal Feldman, and Thomas Kesselheim. Combinatorial contracts. In FOCS 2021 , pages 815-826, 2022.

Paul Dütting, Tomer Ezra, Michal Feldman, and Thomas Kesselheim. Multi-agent contracts. In STOC 2023 , pages 1311-1324, 2023.

Paul Dütting, Zhe Feng, Harikrishna Narasimhan, David C. Parkes, and Sai Srivatsa Ravindranath. Optimal auctions through deep learning: Advances in differentiable economics. Journal of the ACM , Forthcoming 2023. First version, ICML 2019, pages 1706-1715. PMLR, 2019.

Paul Dütting, Guru Guruganesh, Jon Schneider, and Joshua Ruizhi Wang. Optimal no-regret learning for one-sided Lipschitz functions. In ICML 2023 , volume 202, pages 8836-8850. PMLR, 2023.

Paul Dütting, Tim Roughgarden, and Inbal Talgam-Cohen. Simple versus optimal contracts. In EC 2019 , pages 369-387, 2019.

Paul Dütting, Tim Roughgarden, and Inbal Talgam-Cohen. The complexity of contracts. SIAM Journal on Computing , 50(1):211-254, 2021.

Mauro Forti and Paolo Nistri. Global convergence of neural networks with discontinuous neuron activations. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications , 50(11):1421-1435, 2003.

Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In AISTATS 2011 , pages 315-323, 2011.

Yannai A Gonczarowski. Bounding the menu-size of approximately optimal auctions via optimal-transport duality. In STOC 2018 , pages 123-131, 2018.

Yannai A Gonczarowski and S Matthew Weinberg. The sample complexity of up-toε multidimensional revenue maximization. Journal of the ACM , 68(3):1-28, 2021.

Guru Guruganesh, Jon Schneider, and Joshua R Wang. Contracts under moral hazard and adverse selection. In EC 2021 , pages 563-582, 2021.

Dylan Hadfield-Menell and Gillian K. Hadfield. Incomplete contracting and AI alignment. In AIES 2023 , pages 417-422, 2019.

Chien-Ju Ho, Aleksandrs Slivkins, and Jennifer Wortman Vaughan. Adaptive contract design for crowdsourcing markets: Bandit algorithms for repeated principal-agent problems. Journal of Artificial Intelligence Research , 55:317-359, 2016.

Emir Kamenica and Matthew Gentzkow. Bayesian persuasion. American Economic Review , 101(6):2590-2615, 2011.

Jon Kleinberg and Manish Raghavan. Algorithmic classification and strategic effort. ACM SIGecom Exchanges , 18(2):45-52, 2020.

Jon M. Kleinberg and Manish Raghavan. How do classifiers induce agents to invest effort strategically? ACM Transactions on Economics and Computation , 8(4):19:1-19:23, 2020.

Xiaoyang Liu and Jinde Cao. Robust state estimation for neural networks with discontinuous activations. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , 40(6):1425-1437, 2010.

David Martimort and Jean-Jacques Laffont. The theory of incentives: The principal-agent model . Princeton University Press, 2009.

Paolo Massa, Sara Garbarino, and Federico Benvenuto. Approximation of discontinuous inverse operators with neural networks. Inverse Problems , 38(10):105001, 2022.

Stuart Mitchell, Michael OSullivan, and Iain Dunning. Pulp: a linear programming toolkit for python. The University of Auckland, Auckland, New Zealand , 65, 2011.

Tong Mu, Stephan Zheng, and Alexander Trott. Solving dynamic principal-agent problems with a rationally inattentive principal. arXiv preprint arXiv:2202.01691 , 2022.

Florian A Potra and Stephen J Wright. Interior-point methods. Journal of Computational and Applied Mathematics , 124(1-2):281-302, 2000.

Frank Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review , 65(6):386, 1958.

Royal Swedish Academy of Sciences. Scientific background on the 2016 Nobel Prize in Economic Sciences, 2016.

Bernard Salanie. The Economics of Contracts: A Primer . MIT press, 2017.

Stephen J Wright. On the convergence of the Newton/log-barrier method. Mathematical Programming , 90:71-100, 2001.

Bing Xu, Naiyan Wang, Tianqi Chen, and Mu Li. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853 , 2015.

Banghua Zhu, Stephen Bates, Zhuoran Yang, Yixin Wang, Jiantao Jiao, and Michael I. Jordan. The sample complexity of online contract design. In EC '23 , page 1188. ACM, 2023.

A Proofs of Lemma 2-4

In Sec. 3, we study the geometric structure of optimal contracts by establishing four lemmas. While some of them have been stated and proved for linear contracts [32], and the first two lemmas, at least, are not surprising, we give formal proofs of these properties of the principal's utility and optimal contracts in the general case. An important property of the principal's utility function u p that we state in Lemma 2 is that u p can be a discontinuous function.

Lemma 2. The principal's utility function u p can be discontinuous on the boundary of linear pieces.

̸

Proof. For a contract f on the boundary of two neighboring linear pieces µ p i and µ p j = i , the agent is indifferent between action a i and a j given f : µ a i ( f ) = µ a j ( f ) . The principal's utility

̸

It is possible that

Function u p is discontinuous in this case.

Another property of the principal's utility function u p that motivates our discontinuous neural networks is Lemma 3.

Lemma 3. The global optimal contract is on the boundary of a linear piece.

Proof. Suppose that the global optimal f ∗ is not on a boundary but is an interior point of a linear piece Q i (contracts in Q i encourage the agent to take action a i .):

̸

̸

It follows that

where the inequality is strict. Let

̸

It holds that ϵ i,j > 0 , ∀ j = i .

Now we consider the contract f ′ = f ∗ -δp ( ·| a i ) for some small δ > 0 . For any a j = a i , we have

̸

When

̸

(where note the denominator is > 0 ), we have

where

̸

̸

which means f ′ incentivizes the agent to take action a i . Therefore, for δ in this range, the principal's utility given f ′ is:

We thus find a contract f ′ that induces greater utility for the principal, contradicting with the fact that f ∗ is the global optimal contract. This finishes the proof.

Note that here we consider the boundary resulting from changes in the agent's best responses. We can extend the proof to cover another type of boundary, which pertains to the requirement that contracts are non-negative, by defining Q i to be Q i = { f | f ≥ 0 } ∩ Γ i, 1 ∩ · · · ∩ Γ i,i -1 ∩ Γ i,i +1 ∩ · · · .

Lemma 4 claims another property that may influence the design of DeLU networks.

Lemma 4. The principal's utility function u p can be written as a summation of a concave function and a piecewise constant function.

Proof. We first prove the agent's utility function u a ( f ) is a convex function.

We need to prove that, for any two contracts f (1) ∈ F and f (2) ∈ F , it holds that u p ( λ f (1) +(1 -λ ) f (2) ) ≤ λu p ( f (1) ) + (1 -λ ) u p ( f (2) ) , ∀ λ ∈ [0 , 1] . Denote d = f (2) -f (1) . It suffices to prove that the derivative of u p ( f (1) + δ d ) with respect to δ is a non-decreasing function for δ ∈ [0 , 1] .

We have

where a δ is the agent's action given the contract f (1) + δ d .

Case 1: When a δ does not change,

is a constant, which is a non-decreasing function.

Case 2: When a δ changes. Suppose that there exists δ 1 ∈ [0 , 1] such that f (1) + δ 1 d is on the boundary of linear piece Q i and Q j . Let

Because the agent is self-interested, it follows that a j is the best response when the contract is f (1) + δ + 1 d :

It follows that where ϵ is a small number and

We now look at the two terms in the numerator of Eq. 30:

and

Therefore,

This finishes the proof that u a is a convex function. Furthermore, we have that

where -u a ( f ) is a concave function and -c ( a ∗ ( f )) + E o ∼ p ( ·| a ∗ ( f )) [ v o ] is a piecewise constant function with the value -c ( a i ) + E o ∼ p ( ·| a i ) [ v o ] when f ∈ Q i .

B Why we use another network to generate the last-layer bias?

To model model the dependency of the last-layer bias on the activation pattern, we use a neural network, rather than a simpler, linear, and learnable function. The reason is that the bias does not always depend linearly on the activation pattern. Here is an example to illustrate this. There are two outcomes with values v = [20 , 1] , four actions with costs c = [1 . 0 , 2 . 1 , 2 . 3 , 4 . 7] , and the action-outcome transition kernel is

Suppose we consider linear contracts, where f = α v , α > 0 . Then the principal's utility function for different contracts is:

Suppose that we have a 2-dimensional activation pattern, and the linear function converting activation patterns to the bias has parameters [ b 1 , b 2 ] . Then the bias for each of the four pieces would be 0 , b 1 , b 2 , and b 1 + b 2 , respectively. The difference between each piece's bias needs to model the discontinuity at contract parameter α = 0 . 3 , 0 . 4 , 0 . 5 , but this is impossible with this linear model. To see this, we first assume that the piece 0 . 2 < α < 0 . 3 has bias 0. Then the differences of biases of the other 3 pieces would need to be 2.5, 2.86, and 5.28, which cannot be achieved with b 1 , b 2 , and b 1 + b 2 . It can be easily verified that the cases where other pieces have a bias of 0 are similar, demonstrating that a linear bias function cannot express the discontinuity. By contrast, appealing to a second network allows for non-linear dependency on activation, and can handle this problem.

2

4

6

8

log

2[# Outcomes]

0

25

50

75

100

Optimality %

# Actions = 5

2

4

6

8

log

2[# Outcomes]

0

50

100

150

200

Time (in seconds)

# Actions = 5

2

4

6

8

log

2[# Outcomes]

0

25

50

75

100

Optimality %

# Actions = 50

2

4

6

8

log

2[# Outcomes]

0

500

1000

1500

Time (in seconds)

# Actions = 50

2

4

6

8

log

2[# Actions]

0

25

50

75

100

Optimality %

2

4

6

8

log

2[# Actions]

0

1000

2000

Time (in seconds)

2

4

6

8

log

2[# Actions]

0

25

50

75

100

Optimality %

2

4

6

8

log

2[# Actions]

0

500

1000

1500

2000

Time (in seconds)

DeLU

DeLU Inference Time

Concave DeLU

Concave DeLU Inference Time

Linear Programming (Oracle, has access to unobservable information)

C Introducing Concavity into the DeLU Network

From Lemma 4, the principal's utility function u p is a summation of a concave function and a piecewise constant function. Further, our DeLU network, which is used to approximate the function u p , can be written as

In particular, ζ ( r ( f i ); θ ζ ) is a piecewise constant function, because given an activation pattern r ( f i ) , ζ is a constant. However, the first term in Eq. 41, in a general DeLU network, is an arbitrary function. This raises the question as to whether it is useful to further restrict the network architecture, constraining the sub-network η to be a concave function. In this section, we discuss how to introduce concavity into η , and how this restriction affects the performance of DeLU-based contract design.

C.1 Concave DeLU architecture

We can make η a concave function by (1) enforcing its weights (for all but the first layer) to be non-negative, and (2) taking the negation of its output. The first modification will make η a convex function because η uses ReLU activation, which is a convex and non-decreasing function. When the weights after a ReLU activation are non-negative, η becomes a composition of several convex, non-decreasing functions, which is still a convex function. Since the negation of a convex function is a concave function, the second modification will make η concave.

Formally, in Concave DeLU , the sub-network η calculates

where R ( k ) is a diagonal matrix with diagonal elements equal to the layer activation pattern r ( k ) . The other components, as well as the training process, of Concave DeLU are the same as a DeLU network. In the next sub-section, we evaluate the performance of Concave DeLU .

C.2 Experiments on Concave DeLU networks

In Fig. 8, we compare Concave DeLU against DeLU and the direct LP solver ( Oracle LP ). For this, we fix DeLU and Concave DeLU to have the same size, and both use the LP inference algorithm. We test different problem sizes. In the first and second column, we set the number of actions n to 5 and 50, respectively, and increase the number of outcomes m from 2 1 to 2 8 . In the third and fourth column, we set the number of outcomes m to 5 and 50, respectively, and increase the number of actions n from 2 1 to 2 8 . For each problem size, the same 12 combinations of α p and β p are tested. The median

performance as well as the first and third quartile (shaped area) of these 12 combinations are shown. The first row compares optimality, while the second row compares inference time efficiency. We again report normalized principal utilities when it comes to optimality.

A surprising result is that for most problem sizes, DeLU achieves better optimality than Concave DeLU . Although the function class represented by Concave DeLU is a better fit with u p , it seems that the larger function class of DeLU aids optimization and leads to better performance. At the same time, it is somewhat surprising that Concave DeLU can reduce inference time substantially. Unlike DeLU, the inference time of Concave DeLU remains relatively stable when the problem size increases. We conjecture that this behavior can be attributed to the non-negativity constraint on network weights, which reduces the number of valid activation patterns and speeds up LP inference.

D Experimental Setups

D.1 Network architecture and training

We use a simple architecture for the DeLU network. In all experiments, the sub-network η has one hidden layer with 32 hidden units and ReLU activations. We deliberately restrict the size of this subnetwork to limit the number of valid activation patterns and speedup LP-based inference. However, this restriction may reduce the representational capacity of DeLU networks: it is in contrast to the common practice of overparameterization, which has contributed to the success of deep learning. To alleviate this concern, we employ a relatively larger network for the bias network ζ . In our experiments, ζ has a hidden layer with 512 (Tanh-activated) neurons.

The two sub-networks η and ζ are trained in an end-to-end manner by the MSE loss (Eq. 5). The optimization is conducted using RMSprop with a learning rate of 1 × 10 -3 , α of 0.99, and with no momentum or weight decay. For the DeLU and the baseline ReLU networks, training samples are randomly shuffled in each of the training epochs.

D.2 Infrastructure

Across all experiments, DeLU, and the baseline ReLU networks are trained on a NVIDIA A100 GPU. Gradient-based inference is also parallelized on the A100 GPU. The direct LP solver ( Oracle LP ) and the LP-based inference algorithm are based on the linear programming toolkit PuLP [47], and we parallelize five LP solvers on CPUs.

E More Experiments

In this section, we carry out experiments to study (1) using random samples to initialize gradientbased inference (Appx. E.1); and (2) the influence of cost correlation on the optimality of DeLU learners (Appx. E.2).

E.1 Gradient-based inference initialized with random samples

In Sec. 4.2.2, we introduce a gradient-based inference algorithm, and Alg. 1 gives the matrix-form expression of its parallel implementation. There is a choice in using Alg. 1, as to whether the input ('probe points") X (0) ∈ R K × m is taken from the training set or a different, random sample set. In this section, we report the results of experiments to empirically compare these two setups.

We start from the same trained DeLU networks on small ( n = 5 , m = 16 ), middle ( n = 32 , m = 50 ), and large ( n = 50 , m = 128 ) problem sizes, where n is the number of actions and m is the number of outcomes. For each problem size, we consider 12 different combinations of ( α p , β p ) as in other experiments and report the mean and variance of the performance. We run Alg. 1 with two different inputs X (0) : Training Set uses 50 K training samples while Random Set uses 50 K randomly generated contracts. In Table. 1, we present the normalized principal utility (divided by the result obtained by Oracle LP ) of these two setups.

We can observe that the optimality of these two setups are very close, especially when the problem size is large. We thus recommend running Alg. 1 initialized with the training set to reduce the possible time and memory overhead of generating a new random sample set.

E.2 Influence of correlated costs

Across our experiments, we test 12 different combinations of α p and β p , where ( α p , β p ) ∈ { 0 . 5 , 0 . 7 , 0 . 9 } × { 0 , 0 . 3 , 0 . 6 , 0 . 9 } . Recall that a greater β p value means we put more weights on the independent cost, and a greater α p value indicates that the correlated cost is more close to the expected value of the action. It is interesting to investigate the influence of these two parameters on the performance of DeLU contract designers.

In Table 2, we show the optimality of DeLU learners (using the LP inference algorithm) under different α p and β p values. For each ( α p , β p ) combination, we test 24 different problem sizes: ( m,n ) ∈ { 25 , 50 , 100 } × { 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 } . We give the median and standard deviation of these 24 instances. We observe that β p exerts influence on the performance of DeLU networks: optimality of the learned contracts generally increases for greater values of β p , whatever the value of α p , and we see that DeLU seems better suited to handle problems in which the costs of actions are relatively independent of the expected values of actions. This claim is also supported by the results regarding α p , where the optimality under α p = 0 . 5 is typically better than those under other α p values for most β p values. It will be interesting in future work to further study this phenomenon, and to see whether suitable modifications can be made to the DeLU architecture to further improve optimality in regimes with higher cost correlation.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of contract design, focusing on a scenario where a single principal aims to design the reward structure for different outcomes, followed by a self-interest agent. The principal's utility function, which relies on the agent's best response strategy, is a piecewise affine funciton with points of discontinuity at the boundaries. Based on that, the authors introduce a novel approach by using a discontinuous ReLU (DeLU) network to model the principal's utility function. \n\nTo infer the optimal contract using a trained DeLU network, the authors propose two inference methods: a Linear Programming (LP)-based approach and a gradient-based approach. \n\nThe authors provide empirical evidence through various experiments to showcase the optimality, sample complexity, and time efficiency of their DeLU network in comparison to the conventional ReLU network, as well as the effectiveness of their proposed inference techniques.\n\nSTRENGTHS:\n1. The idea of representing the principal's utility function using a neural network is novel and insightful. It paves the way for a new research direction in contract design that harnesses the power of neural networks.\n2. The motivation behind the DeLU network is well-founded. The authors thoroughly analyze the piecewise affine linear nature of the principal's utility function, including its discontinuity at boundary points. Subsequently, they construct the DeLU network, which aligns with and preserves these crucial properties.\n3. The paper exhibits excellent organization throughout, seamlessly integrating theoretical analysis, methodology descriptions, and the experimental section. The logical flow makes it easy to follow and comprehend the content.\n\nWEAKNESSES:\nFrom my view, one significant weakness of the paper is the lack of motivation for utilizing deep learning in the context of optimal contract design.  Why is deep learning a relevant and beneficial approach in this domain? To enhance the paper, it would be essential to address this gap and explain the motivation behind incorporating deep learning techniques for contract design.\n\nFor instance, researchers have explored deep learning for optimal auction design because the theoretical research on optimal auctions has encountered bottlenecks. Similarly, it is crucial to identify the motivation for leveraging deep learning in contract design.\n\nQUESTIONS:\n1. As I mentioned in 'Weakness', what is the significance of using deep learning in optimal contract design? why do we care about this? \n2. Why model the utility function and infer the optimal contract instead of directly modeling the contract and optimizing it based on the utility function?\n\nLIMITATIONS:\nThe authors have discussed the limitations in Section 6.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors focus on the problem of contract design. In this problem, there is an agent which can take some costly action, each action resulting in a distribution over possible outcomes. A principle gets utility based on outcomes, and incentivizes the agent to take actions which will benefit them by transferring them a payment that depends on the outcome. The agent chooses the action that maximizes their expected received payment minus cost; the principal gets a utility equal to the utility of the outcome minus the required payment.\n\nIn mechanism design for selling goods (e.g. auctions) there is a recent thread of work on “differentiable economics” for learning good mechanisms. The authors to some extent work within this broad area, although the techniques they represent depart from prior work in interesting ways. In particular, rather than using a differentiable neural network to represent the mechanism or the agent’s utility as a function of agent types, they instead represent the principal’s utility as a function of the mechanism (contract) itself. The authors present a new network architecture (DeLU) well-suited to learning to represent principal utilities (which are always piecewise affine, though unlike ReLU network not necessarily continuous). By training these networks in a supervised manner, the authors approximate the utility of a given contract. They then present various techniques to optimize over network inputs to find a high-utility contract, and show that these approaches do in fact give good-performing contracts.\n\nSTRENGTHS:\nThe problem of contract design is relevant and interesting, and the authors tackle it in a novel way. The new network architecture and new techniques for optimizing over its input are interesting technical contributions. The experiments are carefully done and support the empirical claims made.\n\nWEAKNESSES:\nA major concern I have is that essentially, the authors take many examples of contracts, use them to supervised-learn an approximate utility function, and then optimize on that utility function. Does this reliably work much better than just taking the best contract observed from the training data, skipping this intermediate step? This general problem also shows up in other mechanism design problems where utility functions are learned via supervised learning -- does learning the utility function and using it significantly improve downstream performance compared to just using the fixed training dataset as a \"pointwise\" utility function?\n\nI don't think this is a trivial issue and if unaddressed it is hard to tell if the paper's contribution serves a purpose or not. This is the main reason for my low score and if it can be addressed and no other problems emerge during the discussion phase, I would significantly raise my score.\n\nAfter author response: this issue is satisfactorily addressed.\n\nQUESTIONS:\nFigure 1, although a valuable and informative plot, is unpleasant looking. Now that you are not up against NeurIPS deadline it might make sense to take the time to do it and make it better looking (e.g. change the strange camera perspective, make axes not cut off, etc.)\n\nHow would this approach differ from directly doing gradient-based optimization in the space of contracts, similar to RochetNet/MenuNet for auctions?\n\nIt is possible to embed ReLU networks in a mixed-integer program and globally optimize over their inputs. This approach, along with some preprocessing and pruning, has been shown to work quite well even at relatively large scales (e.g. Tjeng et al., https://arxiv.org/abs/1711.07356), especially since good MIP solvers are so fast these days. Could this technique be used effectively to globally optimize over DeLU networks too?\n\nLIMITATIONS:\nThe authors adequately discuss limitations of their work, except for the issue mentioned in \"Weaknesses\". I think better discussion of the size of problem instances in which their approach can be used would be warranted.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers an offline learning problem of optimal contract through neural network. The authors propose a novel neural network architecture, called Discontinuous ReLU (DeLU) network, which models a piecewise affine function with discontinuous boundaries --- a representation that captures the principal (contract designer)'s utility function with respect to the different contracts. With the neural network to model the principal's utility, the paper showcases two methods to determine the optimal contract, linear programming or the gradient-based interior-point method. The gradient based method is shown to be more efficient in the experiments.\n\nSTRENGTHS:\n1. The paper is well written and easy to follow. The authors provide a good introduction to the contract design problem and the related literature. The paper is also well organized, with a clear description of the proposed method and the experiments. The figures in this paper are very well designed for readers to intuitively understand the idea.\n2. The paper proposes a novel neural network architecture for training a piecewise affine function with discontinuous boundaries, as well as the gradient based optimization for inference. This architecture is very interesting and useful for many other applications in multi-agent learning. The authors also provide a good explanation of the architecture and the intuition behind it.\n3. The paper provides extensive empirical experiments on simulated data with comparison of different network architecture, inference methods.\n\nWEAKNESSES:\n1. One major concern on the proposed method of this paper is the fact that it attempts to approximate the optimal contract simply as the argmax to the approximated principal’s utility. This does not seem to be a reasonable choice, because there may be a constant gap between the expected agent response and the actual agent response. In particular, the argmax contract at some boundary of the piecewise affine function in the approximated principal’s utility (as pointed out in Lemma 3) is not robustified from the small inaccuracies at the boundary --- at least I do not see how the proposed method can ensure an accurate boundary can be learnt. And this necessarily leads to a constant drop in the testing against the actual agent in a large class of problem instances. If this understanding is indeed correct, it is actually surprising that the proposed method can still achieve a good performance in the experiments. The authors should provide some explanation on this issue; I suspect this is due to the special structure in the synthesized data.\n\n\n\n2. The paper in general lacks theoretical analysis.  It would be interesting to see some theoretical analysis on the generalization error/sample complexity of the proposed network architecture. The authors could also provide some theoretical analysis on the convergence of the gradient based optimization method, as well as the convergence of the linear programming method.\n\n3. I also expect the paper to include some discussion on the novelty of neural network architecture. I wonder if similar attempts have been made to design neural architecture for special function structures in the hypothesis class.\n\n> Both the first and second concern are resolved given the additional experiments and details provided by the authors in the rebuttal.\n\nQUESTIONS:\nPlease answer to my concern in the first point of weakness section.\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a deep-learning approach for contract design. The rationale is:\n- the principal's utility function should be learned from the data \n- the problem of approximating accurately the utility function is non-trivial, and the utility function may be discontinuous --- suggesting the development of better models such as those proposed in the paper\n\nIn addition to the model approximation problem, the authors improve the available algorithms for training and inference. Finally, the authors experimentally evaluate their paradigm according to several dimensions.\n\nSTRENGTHS:\nThe authors clearly posed the problem and the presentation is clear.\n\nWEAKNESSES:\nAt the current stage, I have some doubts about the significance of the contributions and I need that the authors help me to understand better that issue. \n\nMy feeling is that the machine-learning contribution is not very strong and that the work is primarily a slight variation of deep-learning tools applied to contract design. That is, from a machine-learning perspective, this paper does not provide substantial advancements. This is not necessarily a critical issue for the acceptance of the paper. Many papers just apply machine learning tools to design important applications.  More importantly for me, the advancement in the contract design perspective is not really strong. Yes, the authors are providing better approximations, but it is not clear to me the applicability of the results, e.g., due to the need for thousands of samples for the training and it is unlikely such data are available.\n\nI agree with the authors that the main works done so far focus on online/bandit approaches and that the authors are providing a completely different perspective. While online/bandit approaches are directed at a small number of samples, deep learning makes the reverse and, in my opinion, could require a too large number of samples for a good approximation. Personally, I believe that something in the middle would be useful.\n\nQUESTIONS:\n*Technicalities*\n- are the techniques for training and inference provided by the authors important advancements (i.e., non-trivial, non-direct) of the deep learning techniques? if yes, why and how\n\n*Applicability*\n- is correct my understanding that several thousands of samples are necessary for training? if yes, do you believe that making thousands of queries to the principal is reasonable (using $10^6$ in the plot does not help and I would suggest the authors use semi-log plots to show what happens with a small number of samples).\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an automated optimal contract design method from offline data using deep learning. In this setting, a principle deigns a contract establishing an agreement of payments the principle will undertake for the outcomes arising from the actions of an agent. Given a contract, the agent privately selects an action to maximize its expected utility under an action-outcome transition kernel. The principle, unaware of the agent's actions and the transition kernel, aims to maximize its own expected utility. The authors propose the use of neural network function approximation to learn the principle's utility function from offline samples. Due to piecewise discontinuous nature of the principle's utility function and the continuous nature of existing neural network (eg. ReLU), the authors propose the Discontinuous ReLU (DeLU) network to model the the principle's utility function as a discontinuous piecewise affine function, where each piece represents a particular action taken by the agent. Moreover, the authors present a computationally efficient inference method for contract designing based on interior-point method. Finally, the paper empirically evaluates the validity and the performance of the methods presented here on synthetic data.\n\nSTRENGTHS:\n1) Originality:\nNoting that I am not quite familiar with this field, this is the first ever use of neural network learning of utility functions for contract design based on the author's claims. The authors not only combine two existing ideas, but also innovate a new neural network architecture as the existing models are not capable of capturing the critical discontinuities in the utility function. Moreover, they propose an efficient implementation of the inference computations by utilizing highly parallelized existing deep learning frameworks. They also propose an alternative concave neural network architecture. \n\n2) Quality\nThe claims made here were supported by sound theoretical and empirical evidence. \n\n3) Clarity \nEven though I am out of the field, it was very easy to follow the paper. The problem and its motivation are explained very clearly. The theory and the experimental results are stated neatly and discussed properly. It was a pleasure to read this paper.\n\n4) Significance\nAgain, since this is outside my expertise, it is less obvious to me why this problem should be studied. The examples provided by the authors as well as a quick Google search were helpful to clarify this. Aside from the real-world significance, the proposed DeLU network and its concave counterpart are novel and worthy of studying further.\n\nWEAKNESSES:\nI don't have much to say here.\n\nQUESTIONS:\nI have some question to the authors out of curiosity.\n1) How easy is it to extend your method to contract with mixed payment-penalty structure (ie. negative values for f)?\n2) Is there a sequential version of contract design problem with cumulative utility maximization? I am thinking of a case where the principle also is deemed responsible of its actions in the contract design and there is a dynamics interaction between the agent and the principle.\n3) Is there a specific reason to pick a mother neural network to models the bias term? It'd be more straightforward to write the bias term  as a linear function of activation pattern $ b^{L+1}(\\mathbf{f}) = \\sum_{l=1}^{L} \\mathbf{r_l(\\mathbf {f})}^T \\mathbf{b}_l $.\n4) What are some real world applications where the the offline learning with previously collected data could be feasible and reasonable? Are these type of data readily available in practice?\n5) Does it require a significantly new treatment to extend this to multi-agent (single principle) case?\n\nLIMITATIONS:\nLimitations, societal considerations and future works are all discussed.", "TITLE": "Official Review", "RECOMMENDATION": 9, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-19"}