SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Sai Srivatsa Ravindranath* Yanchen Jiang* David C. Parkes  
Harvard John A. Paulson School of Engineering and Applied Sciences {saisr, yanchen_jiang, parkes} @g.harvard.edu
The data market design problem is a problem in economic theory to find a set of signaling schemes (statistical experiments) to maximize expected revenue to the information seller, where each experiment reveals some of the information known to a seller and has a corresponding price [7]. Each buyer has their own decision to make in a world environment, and their subjective expected value for the information associated with a particular experiment comes from the improvement in this decision and depends on their prior and value for different outcomes. In a setting with multiple buyers, a buyer's expected value for an experiment may also depend on the information sold to others [12]. We introduce the application of deep learning for the design of revenue-optimal data markets, looking to expand the frontiers of what can be understood and achieved. Relative to earlier work on deep learning for auction design [32], we must learn signaling schemes rather than allocation rules and handle obedience constraints—these arising from modeling the downstream actions of buyers—in addition to incentive constraints on bids. Our experiments demonstrate that this new deep learning framework can almost precisely replicate all known solutions from theory, expand to more complex settings, and be used to establish the optimality of new designs for data markets and make conjectures in regard to the structure of optimal designs.
Many characterize the current era as the Information Age. Companies such as Google and Meta (search and social media), Experian and TransUnion (credit agencies), and Amazon and American Express (commerce) hold vast quantities of data about individuals. In turn, this has led to data markets, where information about an individual can be purchased in real-time to guide decision-making (e.g., LiveRamp, Segment, Bloomreach). In this paper, we advance the design of market rules that govern the structuring and sale of this kind of information. Using machine learning, we leverage existing theoretical frameworks [7, 12] to replicate all known solutions from theory, expand to more complex settings, establish the optimality of new designs, and make conjectures in regard to the structure of optimal designs. Although this is not our focus, there are also important ethical questions in regard to data markets, in regard to privacy, ownership, informed consent, and the use of data [2, 11, 8, 5, 19, 33, 1].
In settings with a single buyer, Bergemann et al. [7] introduce a framework in which there is a data buyer who faces a decision problem under uncertainty, and has a payoff depending on their choice of action and the underlying state of the world. There is a data seller who knows the world state, and can disclose information through a menu of statistical experiments, each experiment offering a stochastic signal that reveals some (or all) of the seller's information and each of which is associated with a price. The buyer's willingness to pay for information is determined by their type, which defines their prior belief and their value for different outcomes in the world (these arising from the action that the buyer will choose to take).
The optimal data market design problem is to find a set of experiments and associated prices with which to maximize expected revenue, given a distribution over the buyer's type. Bergemann et al. [7] characterize the optimal design when there is a binary world state and a buyer with a binary action. In some settings there may be multiple buyers and buyers may compete downstream, and the more informed one buyer is, the lower the payoff may become for others. Bonatti et al. [12] take up this problem, and characterize the optimal design for multiple buyers in the binary state/binary action setting, and further limited to buyers who each have a common prior on the world state. It remains an open problem to obtain theoretical results for richer, multi-buyer settings, and this motivates the need for gaining new understanding and making progress through computational approaches.
Contributions. Inspired by the recent advances in differential economics ([32, 34, 35, 38, 46, 47, 44, 45, 23, 25, 28, 36, 24, 29] etc.), we initiate the use of deep learning for the automated design of data markets. This market sells information rather than allocates resources, and the value of information depends on the way in which a buyer will use the information—and this downstream action by a buyer needs to be modeled. Following the revelation principle for dynamic games ([43], Section 6.3), it is without loss of generality to model an experiment as generating a recommended action and insisting on designs in which the buyer will prefer to follow this action. This brings about new challenges, most notably to extend this framework to handle obedience. While the other aspect of incentive compatibility that we need to handle is more typical, i.e., that of achieving incentive alignment so that a buyer will choose to truthfully report their prior beliefs and values for different outcomes, we also need to ensure there are no useful double deviations, where a buyer simultaneously misreports their type and acts contrary to the seller's recommended action.
In settings with a single buyer, we learn an explicit, parameterized representation of a menu of priced experiments to offer the buyer and with which to model the action choices available to the buyer. In this way, we extend the RochetNet architecture [32] that has been used successfully for optimal auction design with a single buyer. This enables us to obtain exact incentive compatibility for the single buyer setting: a buyer has no useful deviation from a recommended action, no useful deviation in reporting their type, and no useful double deviation. In settings with multiple buyers, we seek to learn revenue-maximizing designs while also minimizing deviations in disobeying action recommendations and misreporting types and including double deviations. This extends the RegretNet framework [32] that has been used successfully for optimal auction design with multiple buyers and gives approximate incentive alignment.
Our first experimental result is to show through extensive experiments that these new neural network architectures and learning frameworks are able to almost exactly recover all known optimal solutions from Bergemann et al. [7] and Bonatti et al. [12]. Following the economic theory literature on data market design, we consider the notion of Bayesian incentive compatibility (BIC). To handle this, we use samples of a buyer's type to compute an interim experiment and interim payment, averaging over samples drawn from the type distribution (this builds on the BIC methods for deep learning for auctions that were used in the context of differentiable economics for budget-constrained auctions [34]). We give a training method that enables the efficient reuse of computed interim allocations and interim payments from other samples to swiftly calculate the interim utility of misreports, dramatically speeding up training.
Whereas analytical results are only available for the BIC setting, which is, in effect, lower-dimensional, and easier to analyze, we are able to study through computational techniques the design of data markets in the ex post IC setting, which is a setting without existing theory. A second result, in the setting of multiple buyers, is to use our framework to conjecture the structure of an optimal design and prove its optimality (for this, we make use of virtual values analogous to Myerson's framework [42]). We see this as an important contribution, as ex post IC is a stronger notion of IC than BIC.
A third result is to demonstrate how our framework extends its utility beyond empirical results and serves as a toolbox to guide economic theory. To illustrate this, we study how revenue varies with competition in the multi-buyer setting where the prior information is uncertain. Despite the absence of existing theoretical results for this particular setting, our framework enables us to derive trends in revenue effortlessly. We also conjecture the structure of solutions for problems in the single buyer setting with an enlarged type, where both the buyer payoffs and priors are uncertain. For this case, we again derive empirical results using our proposed framework and use it to conjecture the properties of the underlying theoretical solution.
Related work. Conitzer and Sandholm [20, 21] introduced the use of automated mechanism design (AMD) for economic design and framed the problem as an integer linear program (or just a linear program for Bayesian design). Responding to challenges with the enumerative representations used in these early approaches, Dütting et al. [32] introduced the use of deep neural networks for auction design, attaining more representational flexibility. Since then, there has been a line of work on this so-called approach of differentiable economics, including to problems with budget-constrained bidders [34], for minimizing agents' payments [47], applying to multi-facility location problems [35], balancing fairness and revenue ([38]), and applying to two-sided matching [46]. To the best of our knowledge, none of this work has considered the setting of the design of optimal data markets, which introduce new challenges in regard to handling agent actions (obedience) as well as incorporating negative externalities.
In regard to the data market problem, Bergemann et al. [7] build upon the decision-theoretic model pioneered by Blackwell [10] and study a setting with a single buyer. Cai and Velegkas [13] also give computational results in this model, making use of linear programs to compute the optimal menu for discrete type distributions. They also investigate a generalization of this model that allows multiple agents to compete for useful information. In this setting, at most, one agent receives the information. While this approach can be used for continuous type distribution by applying the LP to a discretized valuation space, solving it for even a coarser discretization can be prohibitively expensive. Further advancing economic theory, Bergemann et al. [9] consider more general type distributions and investigate both the cardinality of the optimal menu and the revenue achievable when selling complete information. Bonatti et al. [12] also study the multi-buyer setting modeling competition through negative externality.
Babaioff et al. [6] give a related framework, with the key distinction that the seller is not required to commit to a mechanism before the realization of the world state. As a result, the experiments and prices of the data market can be tailored to the realized state of the world. Chen et al. [18] extend their setting, considering budget-constrained buyers, formulating a linear program to find the solution of the problem for a discrete type space.
There exist several other models that study revenue-optimal mechanisms for selling data. For example, Liu et al. [40] characterize the revenue-optimal mechanism for various numbers of states and actions, and considering general payoff functions. However, in their setup, the state only impacts the payoff of the active action taken by a buyer, which provides considerable simplification and may not be realistic. Li [39] investigates a setting where the buyer can conduct their own costly experiment at a cost after receiving the signal. Different from the model considered in this paper, after receiving the signal from the data broker, the agent can subsequently acquire additional information with costs. The model also assumed that the valuation function of the agent is separable and that the private type of the agent represents her value of acquiring more information, which is different from the single buyer model studied in this paper, where the prior belief is also drawn from a distribution and constitutes a part of the buyer's type. Agarwal et al. [3] explore data marketplaces where each of multiple sellers sells data to buyers who aim to utilize the data for machine learning tasks, and Chen et al. [17] consider scenarios where neither the seller nor the buyer knows the true quality of the data. Mehta et al. [41] and Agarwal et al. [4] also incorporate buyer externalities into the study of data marketplaces.
Another line of research studies problems of information design, for example, the problem of Bayesian Persuasion [37]. There, the model is different in that the sender of information has preferences on the action taken by the receiver, setting up a game-theoretic problem of strategic misrepresentation by the sender. Dughmi and Xu [31] studied this from an algorithmic perspective, and Castiglioni et al. [16] also brought in considerations from mechanism design by introducing hidden types; see also [30], [27], and [14] for more work on information design and Bayesian persuasion.
Model. We consider a setting with  $n$  data buyers,  $N = \{1, \dots, n\}$ , each facing a decision problem under uncertainty. The state of the world,  $\omega$ , is unknown and is drawn from a finite state space  $\Omega = \{\omega_1, \dots, \omega_m\}$ . Each buyer  $i$  can choose an action, from a finite set  $\mathcal{A}_i$ . Let  $\mathcal{A} = \Pi_{i=1}^n \mathcal{A}_i$ . The ex post payoff to buyer  $i$  for choosing action  $a_i \in \mathcal{A}_i$  under state  $\omega$  is given by  $\mathcal{U}_i(\omega, a_i)$ . Unless otherwise specified, we consider the case of matching utility payoffs where the buyer seeks to match
the state and the action  $^2$ . In such settings, we have  $|\Omega| = |\mathcal{A}_i|$  for each  $i \in [n]$ , and the payoff is given by  $\mathcal{U}_i(\omega, a_i) = v_i \cdot \mathbf{1}\{\omega = a_i\}$  where each  $v_i$  is drawn independently from a distribution  $\mathcal{V}_i$ . Let  $\mathcal{V} = \Pi_{i=1}^n \mathcal{V}_i$ .
Each buyer  $i$  also has an interim belief,  $\theta_{i} \in \Delta(\Omega)$ , about the world state. Each buyer's belief,  $\theta_{i}$  is drawn independently from a distribution  $\Theta_{i}$ . Let  $\Theta = \Pi_{i=1}^{n} \Theta_{i}$ . The type of a buyer is given by the tuple  $(v_{i}, \theta_{i})$  and is denoted by  $\rho_{i}$ . If the buyers don't vary in  $v_{i}$  or their interim beliefs  $\theta_{i}$ , then  $\rho_{i} = \theta_{i}$  or  $\rho_{i} = v_{i}$ , respectively. Let  $\mathcal{P}_{i} \triangleq \mathcal{V}_{i} \times \Theta_{i}$ . The utility of a buyer  $i \in [n]$  is given by the utility function  $u_{i}: \mathcal{A} \times \Omega \times \mathcal{P}_{i} \to \mathbb{R}$ . We assume that the utility of buyer  $i$  depends only on its own type  $\rho_{i}$  but can depend on other players' actions as a negative externality. Additionally, we follow [12] and assume that this externality is separable, thus simplifying our utility  $u_{i}(a, \omega, \rho_{i}) = v_{i} \cdot \mathbf{1}\{a_{i} = \omega\} - E_{-i}(a_{-i}, \omega, \rho_{i})$ . In this paper, we consider settings where the negative externality is given by  $E_{-i}(a_{-i}, \omega, \rho_{i}) = \frac{\alpha}{n-1} \sum_{j \in [n] \setminus i} v_{i} \cdot \mathbf{1}\{a_{j} = \omega\}$  where  $\alpha \in \mathbb{R}_{\geq 0}$  where  $\alpha$  captures the degree of competitiveness among buyers. Let  $\bar{\alpha} = \frac{\alpha}{n-1}$
Statistical Experiments. There is a data seller who observes the world state and wishes to sell information to one or more buyers to maximize expected revenue. The seller sells information through signaling schemes, where each scheme is called an experiment. The seller chooses, for each buyer  $i \in [n]$ , a set of signals  $S_{i}$  and a signaling scheme  $\sigma_{i}: \Omega \to \triangle S_{i}$ . If state  $\omega \in \Omega$  is realized, then the seller sends a signal drawn from the distribution  $\sigma_{i}(\omega)$ . Upon receiving a signal, the buyers update their prior beliefs and choose an optimal action accordingly. The signaling scheme  $\sigma_{i}$  can also be represented as a matrix  $\pi_{i} -$  a collection of  $m$  row vectors each of dimensions  $|S_{i}|$ . The  $j$ -th row vector (for  $j \in [m]$ ) specifies the likelihood of each signal when state  $\omega_{j}$  is realized. Thus we have  $\sigma_{i}(\omega_{j}) = \pi_{i,j}$ .
The Mechanism Design Problem. The mechanism design goal is to design a set of experiments and corresponding prices to maximize the expected revenue of the seller. Let  $\mathcal{B}$  denote a message (bid) space. Define the signaling schemes for a buyer  $i\in [n]$  as  $\sigma_{i}:\Omega \times \mathcal{B}\to \triangle S_{i}$  and a payment function  $t_i:\mathcal{B}\rightarrow \mathbb{R}_{\geq 0}$ . Given bids  $b = (b_{1},\ldots b_{n})\in \mathcal{B}$ , if state  $\omega \in \Omega$  is realized, then the seller sends a signal drawn from the distribution  $\sigma_{i}(\omega ,b)$  to buyer  $i$  and collects payment  $t_i(b)$ . The sequence of interactions between one or more buyers and the seller takes place as follows:
1. The seller commits to a mechanism,  $\mathcal{M} = (\sigma, t)$ , where  $\sigma = (\sigma_1, \dots, \sigma_n)$  is a choice and  $t = (t_1, \dots, t_n)$ .
2. Each buyer  $i$  observes their type,  $\rho_{i}$ . The seller observes the state,  $\omega$ .
3. Each buyer reports a message  $b_{i}$ .
4. The seller sends buyer  $i$  a signal,  $S_{i} \in S_{i}$ , generated according to the signaling scheme  $\sigma_{i}(\omega, b)$ , and collects payment  $t_{i}(b)$ .
5. Each buyer  $i$  chooses an action  $a_i \in \mathcal{A}_i$ , and obtains utility  $u_i(a, \omega, \rho_i) - t_i(b)$ , where  $a = (a_1, \ldots, a_n)$ .
By the revelation principle for dynamic games [43, Section 6.3], as long as we consider incentive-compatible mechanisms, it is without loss of generality for the message space to be the type space of buyers, and for the size of the signal space to be the size of the action space, i.e., for each  $i \in [n]$ ,  $|\mathcal{S}_i| = |\mathcal{A}_i|$ . In such mechanisms, the seller designs experiments where every signal leads to a different optimal choice of action. Following [7], we can then replace every signal as an action recommended by the seller.
Incentive Compatibility. A mechanism  $(\sigma, t)$  is Bayesian incentive compatible (BIC) if the buyer maximizes their expected utility (over other agents' reports) by both reporting their true type as well as by following the recommended actions. For the sake of notational convenience, let  $\hat{\mathbb{E}}_{-i} \coloneqq \mathbb{E}_{\rho_{-i} \sim \mathcal{P}_{-i}}$  denote the operator used for computing the interim representations.
For each  $(\rho_i,\rho_i^{\prime})\in \mathcal{P}_i^2$  and for each deviation function  $\delta :\mathcal{A}_i\to \mathcal{A}_i$  , a BIC mechanism satisfies:
$$
\hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a \sim \sigma (\omega , \rho) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (a, \omega , \rho) - t _ {i} (\rho) ] \right] \geq \hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a \sim \sigma (\omega ; \rho_ {i} ^ {\prime}, \rho_ {- i}) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (\delta (a _ {i}), a _ {- i}, \omega , \rho) - t _ {i} \left(\rho_ {i} ^ {\prime}, \rho_ {- i}\right) ] \right] \tag {1}
$$
It is easier to extend our approach to non-matching utilities as well
In particular, this insists that double deviations (misreporting the type and disobeying the recommendation) are not profitable.
We also consider the stronger notion of ex post incentive compatible (IC), which requires, for every agent  $i$ , and for each  $\rho \in \mathcal{P}$ , and assuming that every other agent reports its type truthfully and follows the recommended action, then for each misreport  $\rho_i' \in \mathcal{P}_i$ , and each deviation function,  $\delta: \mathcal{A}_i \to \mathcal{A}_i$ , the following condition:
$$
\underset { \begin{array}{c} a \sim \sigma (\omega , \rho) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (a, \omega , \rho) - t _ {i} (\rho) ] \geq \underset { \begin{array}{c} a \sim \sigma (\omega ; \rho_ {i} ^ {\prime}, \rho_ {- i}) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (\delta (a _ {i}), a _ {- i}, \omega , \rho) - t _ {i} (\rho_ {i} ^ {\prime}, \rho_ {- i}) ] \tag {2}
$$
Individual Rationality. A mechanism is interim individually rational (IIR) if reporting an agent's true type guarantees at least as much expected utility (in expectation over other agents' reports) as opting out of the mechanism. Let  $\sigma_{-i}:\Omega \times \mathcal{P}_{-i}\to \triangle \mathcal{A}_{-i}$  denote the recommendations to other participating buyers when buyer  $i$  opts out. For each agent  $i$ , and each  $\rho_{i}\in \mathcal{P}_{i}$ , IIR requires
$$
\hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a \sim \sigma (\omega ; \rho) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (a; \omega , \rho) - t _ {i} (\rho) ] \right] \geq \max  _ {\tilde {a} _ {i} \in \mathcal {A} _ {i}} \left(\hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a _ {- i} \sim \sigma_ {- i} (\omega ; \rho_ {- i}), \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (\tilde {a} _ {i}, a _ {- i}; \omega , \rho) ] \right]\right) \tag {3}
$$
It is in the seller's best interest to instantiate the recommendation function to the other buyers when buyer  $i$  opts out,  $\sigma_{-i}$ , to minimize the value of the RHS of this IIR inequality. For this reason, it is without loss of generality to rewrite the above equation as:
$$
\hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a \sim \sigma (\omega ; \rho) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (a; \omega , \rho) - t _ {i} (\rho) ] \right] \geq \min  _ {\sigma_ {- i}} \max  _ {\tilde {a} _ {i} \in \mathcal {A} _ {i}} \left(\hat {\mathbb {E}} _ {- i} \left[ \underset { \begin{array}{c} a \sim \sigma_ {- i} (\omega ; \rho_ {- i}) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (\tilde {a} _ {i}, a _ {- i}; \omega , \rho) ] \right]\right) \tag {4}
$$
In particular, and recognizing that a buyer's utility decreases the more informed other buyers are, the seller can achieve this by sending optimal recommendations to participating buyers in order to minimize the utility of a non-participating buyer.
We also consider a stronger version of IR, namely ex post individual rationality (or simply IR). In this case, for each agent  $i$ , and each  $\rho \in \mathcal{P}$ , we require:
$$
\underset { \begin{array}{c} a \sim \sigma (\omega ; \rho) \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (a; \omega , \rho) - t _ {i} (\rho) ] \geq \min  _ {\sigma_ {- i}} \max  _ {\tilde {a} _ {i} \in \mathcal {A} _ {i}} \left(\underset { \begin{array}{c} a \sim \sigma_ {- i} (\omega ; \rho_ {- i}), \\ \omega \sim \theta_ {i} \end{array} } {\mathbb {E}} [ u _ {i} (\tilde {a} _ {i}, a _ {- i}; \omega , \rho) ]\right) \tag {5}
$$
3 Optimal Data Market Design in the Single-Buyer Setting
In this section, we formulate the problem of optimal data market design for a single buyer as an unsupervised learning problem. We study a parametric class of mechanisms,  $(\sigma^w,t^w)\in \mathcal{M}$ , for parameters,  $w\in \mathbb{R}^d$ , where  $d > 0$ . For a single buyer, the goal is to learn parameters  $w$  that maximize  $\mathbb{E}_{\rho \sim \mathcal{P}}[t^w (\rho)]$ , such that  $(\sigma^w,t^w)$  satisfy ex post IC and IR. $^3$  By adopting differentiable loss functions, we can make use of tools from automatic differentiation and stochastic gradient descent (SGD). For notational convenience, we drop the subscript in this case, as there is only one buyer.