SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Neural Network Architecture. For the single buyer setting, any IC mechanism can be represented as a menu of experiments. For this, we extend the RochetNet architecture to represent a menu of priced statistical experiments. Specifically, the parameters correspond to a menu of  $P$  choices, where each choice  $p \in [P]$  is associated with an experiment, with parameters  $\gamma^p \in \mathbb{R}^{|\Omega| \times |A|}$ , and a price,  $\beta^p \in \mathbb{R}$ . Given this, we define a menu entry as,
$$
\pi_ {\omega , j} ^ {p} = \frac {\exp \left(\gamma_ {\omega , j} ^ {p}\right)}{\sum_ {k \in [ m ]} \exp \left(\gamma_ {\omega , k} ^ {p}\right)}, \quad t ^ {p} = \beta^ {p} \tag {6}
$$
For the single bidder settings, ex post IC and IR are the same as BIC and IIR.
The input layer takes  $\rho = (v,\theta)$  as input and the network computes the  $P$  utility values corresponding to each menu entry. We do not impose obedience constraint explicitly. However, while computing the utility of a menu, we take into consideration the best possible deviating action an agent can take. From the perspective of buyer, if action  $k$  is recommended by an experiment choice  $p$ , then  $Pr[\omega = j|a_k,\theta ] = \frac{\theta_j\pi_{j,k}^p}{Pr[s_k]}$ . The best action deviation is thus  $\delta (k) = \arg \max_{j\in [m]}\frac{\theta_j\pi_{j,k}^p}{Pr[s_k]}$ , yielding a utility of  $v\cdot \max_{j\in [m]}\frac{\theta_j\pi_{j,k}^p}{Pr[s_k]} = \frac{v\cdot\max_{j\in[m]}\theta_j\pi_{j,k}^p}{Pr[s_k]}$ . Taking an expectation over all signals, buyer  $i$  receives the following utility for the choice  $p\in [P]$ :
$$
h ^ {p} (\rho) = v \cdot \left(\sum_ {k \in [ m ]} \max  _ {j \in [ m ]} \left\{\theta_ {j} \pi_ {j, k} ^ {p} \right\}\right) - \beta^ {p} \tag {7}
$$
We also include an additional experiment,  $\pi^0$ , which corresponds to a null experiment that generates the same signal regardless of state, and thus provides no useful information. This menu entry has a price of  $t^0 = 0$ , and thus  $h^0 (\rho) = v\cdot (\max_{j\in [m]}\theta_j)$ . In particular, this menu entry has the same utility as that of opting out.
Lemma 3.1. Let  $p^*(\rho) = \arg \max_{p \in \{0, \dots, P\}} h^p(\rho)$  denote the optimal menu choice. Then the mechanism  $\mathcal{M} = (\sigma^w, t^w)$  where  $\sigma^w(\omega, \rho) = \pi_\omega^{p^*(\rho)}$  and  $t^w(\rho) = t^{p^*(\rho)}$  satisfies IC and IR.
This mechanism is IC as it is agent optimizing, and is IR as it guarantees a buyer at least the utility of opting out.
Training Problem. The expected revenue of such an IC and IR mechanism that is parameterized by  $\gamma, \beta$  is given by  $\mathbb{E}_{\rho \sim \mathcal{P}}[\beta^{p^*(\rho)}]$ . Rather than minimizing a loss function that measures errors against ground truth labels (as in a supervised learning setting), our goal is to minimize expected negated revenue. To ensure that the objective is differentiable, we replace the argmax operation with a softmax during training. The loss function, for parameters  $\gamma$  and  $\beta$ , is thus given by  $\mathcal{L}(\gamma, \beta) = \mathbb{E}_{\rho \sim \mathcal{P}}\left[-\sum_{p=1}^{P} \beta^p \tilde{\nabla}^p(\rho)\right]$  where  $\tilde{\nabla}^p(\rho) = \text{softmax}\left(\frac{h^1(\rho)}{\tau}, \ldots, \frac{h^P(\rho)}{\tau}\right)$ , and  $\tau > 0$  controls the quality of approximation. Note that, at test time, we revert to using the hard max, so as to guarantee exact IC and IR from a trained network.
During training, this null experiment  $(\pi^0,t^0)$  remains fixed, while the parameters  $\gamma \in \mathbb{R}^{p\times |\Omega |\times m}$  and  $\beta \in \mathbb{R}^p$  are optimized through SGD on the empirical version of the loss calculated over  $\ell$  i.i.d samples  $S = \{\rho^{(1)},\dots \rho^{(\ell)}\}$  drawn from  $\mathcal{P}$ . We report our results on a separate test set sampled from  $\mathcal{P}$ . We refer to the Appendix A.2 for more details regarding the hyperparameters.
4 Optimal Data Market Design in the Multi-Buyer Setting
In the multi-buyer setting, the goal is to learn parameters  $w \in \mathbb{R}^d$  that maximize  $\mathbb{E}_{\rho \sim \mathcal{S}}[\sum_{i=1}^{n} t_w^i(\rho)]$ , for a parametric class of mechanisms,  $(\sigma^w, t^w) \in \mathcal{M}$ , such that  $(\sigma^w, t^w)$  satisfy IC (or BIC) and IR (or IIR). By restricting our loss computations to differentiable functions, we can again use tools from automatic differentiation and SGD. For the multi-buyer setting, we use differentiable approximations to represent the rules of the mechanism and compute the degree to which IC constraints are violated during training adopting an augmented Lagrangian method [32].
Neural Network Architecture. The neural network architecture has two components: one that encodes the experiments and another that encodes payments. We model these components in a straightforward way as feed-forward, fully connected neural networks with Leaky ReLU activation functions. The input layer consists of the reported type profile,  $\rho$ , encoded as a vector. For both IC and BIC settings, the component that encodes experiments outputs a matrix  $\pi$  of dimensions  $n \times |\Omega| \times |\Omega|$ , which represents an experiment that corresponds to each of the  $n$  agents. In order to ensure feasibility, i.e., the probability values of sending signals for each agent under each state realization is non-negative and sums up to 1, the neural network first computes an output matrix,  $\tilde{\pi}$ , of the same dimension. For all  $i,j,\omega$ , we then obtain  $\pi_{i,\omega,j}$  by computing  $\exp(\tilde{\pi}_{i,\omega,j}) / \sum_k \exp(\tilde{\pi}_{i,\omega,k})$ .
For the setting of IC (vs. BIC), we define payments by first computing a normalized payment,  $\tilde{t}_i^w (\rho) \in [0,1]$ , for each buyer  $i$ , using a sigmoid activation unit. The payment  $t_i^w (\rho)$  is computed as
follows:
$$
t _ {i} ^ {w} (\rho) = \tilde {t} _ {i} ^ {w} (\rho) \cdot v _ {i} \cdot \left(\sum_ {k \in [ m ]} \pi_ {i, k, k} ^ {w} (\rho) \theta_ {i, k} - \bar {\alpha} \sum_ {k \in [ m ]} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} (\rho) \theta_ {j, k} - (\max  \theta_ {i} - \alpha)\right) \tag {8}
$$
Lemma 4.1. Any mechanism  $\mathcal{M} = (\sigma^w, t^w)$  where  $\sigma_i^w(\omega, \rho) = (\pi_{i,\omega}^w(\rho))$  and  $t^w$  satisfies Eqn 8 is ex post IR constraint for any  $w \in \mathbb{R}^d$ .
For the BIC setting we only need to compute the interim payment, as we can replace  $\mathbb{E}_{\rho_{-i}\sim \mathcal{P}_{-i}}[t_i(\rho_i,\rho_{-i})]$  by  $t_i(\rho_i)$ . For this, we compute an interim normalized payment,  $\hat{t}_i\in [0,1]$ , for each buyer  $i$  by using a sigmoid unit. We compute the interim payment as:
$$
t _ {i} ^ {w} (\rho_ {i}) = \tilde {t} _ {i} ^ {w} (\rho_ {i}) \cdot v _ {i} \cdot \hat {\mathbb {E}} _ {- i} \left[ \sum_ {k \in [ m ]} \pi_ {i, k, k} ^ {w} (\rho) \theta_ {i, k} - \bar {\alpha} \sum_ {k \in [ m ]} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} (\rho) \theta_ {j, k} - (\max  \theta_ {i} - \alpha) \right] \tag {9}
$$
Lemma 4.2. Any mechanism  $\mathcal{M} = (\sigma^w, t^w)$  where  $\sigma_i^w(\omega, \rho) = (\pi_{i,\omega}^w(\rho))$  and  $t^w$  satisfies Eqn 9 is interim IR for any  $w \in \mathbb{R}^d$ .
Training Problem. In order to train the neural network, we need to minimize the negated revenue subject to incentive constraints. Following Dütting et al. [32], we measure the extent to which a mechanism violates the IC (or BIC) constraints through the notion of ex post regret (or interim regret) and then appeal to Lagrangian optimization. The regret for an agent is given by the maximum increase in utility, considering all possible misreports and all possible deviations for a given misreport in consideration while fixing the truthful reports of others (or in expectation over truthful reports of others for the BIC setting) when the others are truthful and obedient.
We define the ex post regret for a buyer  $i$  as  $RGT_{i}^{w} = \mathbb{E}_{\rho \in \mathcal{P}}\left[\max_{\rho_{i}^{\prime}\in \mathcal{P}_{i}}rgt_{i}^{w}(\rho_{i}^{\prime},\rho)\right]$  where  $rgt_{i}^{w}(\rho_{i}^{\prime},\rho)$  is defined as:
$$
\begin{array}{l} r g t _ {i} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho\right) = v _ {i} \cdot \left(\sum_ {k \in [ m ]} \max  _ {k ^ {\prime} \in [ m ]} \left\{\pi_ {i, k ^ {\prime}, k} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) \theta_ {i, k ^ {\prime}} \right\} - \bar {\alpha} \sum_ {k \in [ m ]} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) \theta_ {j, k}\right) \\ - v _ {i} \cdot \left(\sum_ {k \in [ m ]} \pi_ {i, k, k} ^ {w} (\rho) \theta_ {i, k} - \bar {\alpha} \sum_ {k \in [ m ]} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} (\rho) \theta_ {j, k}\right) - \left(t _ {i} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) - t _ {i} ^ {w} (\rho)\right) \tag {10} \\ \end{array}
$$
Lemma 4.3. Any mechanism  $\mathcal{M} = (\sigma^w, t^w)$  where  $\sigma_i^w(\omega, \rho) = (\pi_{i,\omega}^w(\rho))$  is ex post IC if and only if  $RGT_i^w = 0 \forall i \in [n]$ , except for measure zero events.
We define the interim regret for a buyer  $i$  as  $\widehat{RGT}_i^w = \mathbb{E}_{\rho \in \mathcal{P}}\left[\max_{\rho_i'\in \mathcal{P}_i}\widehat{rgt}_i^w (\rho_i',\rho_i)\right]$  where  $\widehat{rgt}_i^w (\rho_i',\rho_i)$  is defined as:
$$
\begin{array}{l} \widehat {r g t} _ {i} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {i}\right) = v _ {i} \cdot \left(\sum_ {k \in [ m ]} \max  _ {k ^ {\prime} \in [ m ]} \hat {\mathbb {E}} _ {- i} \left[ \pi_ {i, k ^ {\prime}, k} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) \theta_ {i, k ^ {\prime}} - \bar {\alpha} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) \theta_ {j, k} \right]\right) \\ - v _ {i} \cdot \hat {\mathbb {E}} _ {- i} \left[ \sum_ {k \in [ m ]} \pi_ {i, k, k} ^ {w} \left(\rho_ {i}, \rho_ {- i}\right) \theta_ {i, k} - \bar {\alpha} \sum_ {k \in [ m ]} \sum_ {j \in [ n ] \backslash i} \pi_ {j, k, k} ^ {w} \left(\rho_ {i}, \rho_ {- i}\right) \theta_ {j, k} \right] - \left(t _ {i} \left(\rho_ {i} ^ {\prime}\right) - t _ {i} \left(\rho_ {i}\right)\right) \tag {11} \\ \end{array}
$$
Lemma 4.4. Any mechanism  $\mathcal{M} = (\sigma^w, t^w)$  where  $\sigma_i^w(\omega, \rho) = (\pi_{i,\omega}^w(\rho))$  is interim IC if and only if  $\widehat{RGT}_i^w = 0 \forall i \in [n]$ , except for measure zero events.
Given regret, we compute the Lagrangian objective on the empirical version of loss and regret over samples  $S = \{\rho^{(1)},\ldots ,\rho^{(L)}\}$ . We solve this objective using augmented Lagrangian optimization.
The BIC setting also involves an inner expectation to compute the interim representations. There, for each agent  $i$ , we sample a separate subset  $S_{-i} = \{\rho_{-i}^{(1)},\ldots ,\rho_{-i}^{(K)}\}$ , and replace the inner expectations with an empirical expectation over these samples. We sample several misreports, compute the best performing misreport among these samples, and use this as a warm-start initialization for the inner maximization problem. For the BIC setting, rather than sampling fresh misreports and computing the interim experiments and payments, we re-use the other samples from the minibatch and their already computed interim values to find the best initialization. This leads to a dramatic speed-up as the number of forward passes required to compute the regret reduces by an order magnitude of the number of initialized misreports. This, however, works only for the BIC setting, as the constraints are only dependent on the interim values.
We report all our results on a separate test set sampled from  $\mathcal{P}$ . Please refer to Appendix B.6 for more details regarding the hyperparameters.
5 Experimental Results for the Single Buyer Setting
In this section, we demonstrate the use of RochetNet to recover known existing results from the economic theory literature. Additionally, we also show how we can use this approach to conjecture the optimal menu structure for settings outside the reach of current theory. Further, we give in the Appendix C.2 representative examples of how we can characterize the differential informativeness of the menu options when we change different properties of the prior distribution. This builds economic intuition in regard to the shape of optimal market designs.
Buyers with world prior heterogeneity. We first show that RochetNet can recover the optimal menu for all the continuous distribution settings considered in [7] where the input distributions are a continuum of types. We specifically consider the following settings with binary states and binary actions with payoffs  $v = 1$  and interim beliefs drawn from:
A. an unit interval, i.e,  $\theta \sim U[0,1]$
B. an equal weight mixture of  $Beta(8,30)$  and  $Beta(60,30)$ .
Figure 1: The differential informativeness  $q$  of the menu(s) learned by RochetNet for Setting D, varying the parameter  $c$  that instantiates the economic environment.
The optimal menus for each of these settings are given by [7]. For the first setting, the optimal menu consists of a fully informative experiment with a price of 0.25. For the second setting, the seller offers two menu options: a
partially informative experiment and a fully informative experiment. In each case, RochetNet recovers the exact optimal menu. We describe the optimal menus, associated prices, revenue, and RochetNet revenue in the Appendix C.1.
We also give the results from additional experiments in Appendix C.2, where we demonstrate how the informativeness of the menu  $q$  changes, as we vary different properties of the economic environment.
Buyers with payoff and world prior heterogeneity. We are aware of no theoretical characterization of optimal data market designs when both  $v$  and  $\theta$  vary. In such cases, we can use RochetNet to conjecture the structure of an optimal solution. For this, we consider the following settings with enlarged buyer types with binary states and binary actions with:
C.  $v\sim U[0,1]$  and the interim beliefs are drawn from  $U[0,1]$
D.  $v \sim U[c, 1]$  and the interim beliefs are drawn from an equal weight mixture of  $Beta(8, 30)$  and  $Beta(60, 30)$  for  $c \in [0, 1]$ .
For Setting C, RochetNet learns a menu consisting of a single fully informative experiment with a price of 0.14. For Setting D, RochetNet learns a menu consisting of a single fully informative experiment when  $c < 0.55$ . For higher values of  $c$ , RochetNet learns an additional partially informative menu option. In Figure 1, we show how the differential informativeness  $q$  of the partially informative menu changes with  $c$ .
Figure 2: Experiments learned for Settings E and F, for BIC constraints (top) and ex post IC constraints (bottom). The plot shows the probability of recommending the correct action to buyer 1 (left subplot) and buyer 2 (right subplot) for varying values of  $v_{1}$  (x-axis) and  $v_{2}$  (y-axis). For each setting, the theoretically optimal experiments are represented by solid lines separating the regions.
6 Experimental Results for the Multi-Buyer Setting
In this section, we show how we can use RegretNet to recover known existing results for the multi-buyer, BIC setting. We also use RegretNet to conjecture the optimal solution for a multi-buyer problem in the ex post IC setting, and then prove the optimality of the design. Lastly, we give results for a setting where it is analytically hard to compute the optimal solution but RegretNet is used to understand how revenue varies with changes to the intensity of negative externality, again building economic intuition for the market design problem.
BIC settings. [12] study the multi-buyer market design problem with two buyers, binary actions, and fixed interim beliefs, the same for each agent. They show that a deterministic signaling scheme is optimal, and thus characterize their solution in terms of recommending a correct action (selling a fully informative menu). Since the obedience constraints are defined on the interim representation, the optimal mechanism is also able to sometimes recommend incorrect actions to a buyer (in effect sending recommendations that are opposed with the realized state).
We consider the following settings from [12] with interim beliefs are given by  $\theta_{1} = \theta_{2} = (0.5, 0.5)$  and  $\alpha = 0.5$  and:
E. The payoffs  $v_{1}, v_{2}$  are sampled from the unit interval  $U[0, 1]$ .
F. The payoffs  $v_{1}, v_{2}$  are sampled from the exponential distribution with  $\lambda = 1$ .
The results for these settings with a BIC constraint are shown in Figure 2 (top). The theoretically optimum solution contains two kinds of recommendations, those recommending the correct action (the optimal action to take given the realized state), and those recommending incorrect actions, separated by the black solid lines. The heatmap showed in the plots denotes our computational results, which show the probability of recommending the correct action. In particular, we can confirm visually (and from its expected revenue) that RegretNet is able to recover the optimal revenue as well the optimal experiment design. The test revenue and regret along with additional results are in Appendix D.1.
Ex post IC settings. We apply RegretNet to the same three settings, but now adopting ex post IC constraints. This is of interest because there is no known analytical solution for the optimal mechanism for two buyers in the ex post IC setting. See Figure 2 (bottom). Based on this, and additional experiments (in Appendix D.2), we were able to conjecture that the structure of the optimal design in this ex post IC setting, and we prove its optimality by following Myerson's framework for single item auctions. We defer the proof to the appendix D.4.
Theorem 6.1. Consider the setting with Binary State and Binary Actions where the buyers have a common interim belief  $\theta_{i} = \theta$ . The payoff  $v_{i}$  for a buyer  $i$  is drawn from a regular distribution  $\mathcal{V}_{i}$  with a continuous density function. Define virtual value  $\phi_{i}(v_{i}) = v_{i} - \frac{1 - F(v_{i})}{f(v_{i})}$ , for pdf  $f$  and cdf  $F$  of distribution  $\mathcal{V}_{i}$ . The revenue-optimal mechanism satisfying ex post IC and IR is a mechanism that sells the fully informative experiment to buyer  $i$  if  $\phi_{i}(v_{i}) \geq \bar{\alpha} \sum_{i \neq j} \phi_{j}(v_{j})$ . Otherwise buyer  $i$  receives an uninformative menu, where the signal corresponds to the most likely state based on the prior.
Studying the effect of varying the intensity of externalities. Following [12], we study the effect of the negative externality parameter  $\alpha$  on the revenue for the BIC, multi-buyer setting with a common and known prior for each buyer and payoffs sampled from  $U[0,1]$ . We also consider the case where the payoffs are constant but the interim beliefs are independently sampled from  $U[0,1]$ . While there is no known analytical characterization for the latter, we show how easy it is to study the models learnt by RegretNet to analyze economic properties of interest.
Figure 3 shows the effect of  $\alpha$  on the revenue. In the context of uncertain priors, we note that the effect on revenue is similar to the setting where payoffs are uncertain. As we enhance competition intensity via  $\alpha$ , the revenue grows. Even though the buyers are not recommended the correct action, the seller manages to generate revenue by threatening to share exclusive information with a competitor. This effect becomes fiercer in the settings with uncertain priors.
We have introduced a new deep neural network architecture and learning framework to study the design of optimal data markets. We have demonstrated through experimental work the flexibility of the framework, showing that it can
Figure 3: Effect of  $\alpha$  on revenue
adapt to relatively complex scenarios and facilitate the discovery of new optimal designs. Note that while we have only considered matching utility payoffs in this paper, our approach can easily be extended to non-matching utility payoffs as well.
We also point out some limitations. First, for our approach to continue to provide insights into the theoretically optimal design for larger problems (e.g., with more buyers, more signals, more actions), it will be important to provide interpretability to the mechanisms learned by RegretNet (designs learned by RochetNet, on the other hand, are immediately interpretable). Second, while our approach scales well with the number of buyers or states in the ex post IC setting, it does not scale as easily with the number of buyers in the BIC setting. The challenge in the BIC setting comes from the interim computations involving conditional expectations over reports of others and scaling beyond what is studied in this paper will require new techniques, for example, exploiting symmetry. Third, we are making use of gradient-based approaches, which may suffer from local optima in non-convex problems. At the same time, deep learning has shown success in various problem domains despite non-convexity. The experiments reported here align with these observations, with our neural network architectures consistently recovering optimal solutions, when these are known, and thus optimality can be verified, and despite non-convex formulations. Fourth, we attain in the multi-buyer setting only approximate, and not exact, incentive alignment, and this leaves the question of how much alignment is enough for agents to follow the intended advice of a market design (there is little practical or theoretical guidance in this regard). Moreover, we do not know for a given solution whether there is an exact IC solution nearby. While there is some recent guiding theory [22, 26, 15] that provides transformations between  $\epsilon$ -BIC and BIC without revenue loss in the context of auction design, extending these transformations to approximate IC settings and to problems with both types and actions presents an interesting avenue for future research.
Lastly, we return to where we started, and underline that markets for trading data about individuals raise substantive ethical concerns [1, 2, 5, 8, 11, 19, 33]. Our hope is that machine learning frameworks such as those introduced here can be used to strike new kinds of trade-offs, for example allowing individuals to also benefit directly from trades on data about themselves and to embrace privacy and fairness constraints.
We would like to thank Zhe Feng and Alessadro Bonatti for the initial discussions. The source code for all experiments is available from Github at https://github.com/saisrivatsan/deep-data-markets/