SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Guanghui Yu
guanghuiyu@wustl.edu
Wei Tang
wt2359@columbia.edu
Saumik Narayanan
saumik@wustl.edu
Chien-Ju Ho
chienju.ho@wustl.edu
We initiate the study of behavioral information design through deep learning. In information design, a sender aims to persuade a receiver to take certain actions by strategically revealing information. We address scenarios in which the receiver might exhibit different behavior patterns other than the standard Bayesian rational assumption. We propose HAIDNet, a neural-network-based optimization framework for information design that can adapt to multiple representations of human behavior. Through extensive simulation, we show that HAIDNet can not only recover information policies that are near-optimal compared with known analytical solutions, but also can extend to designing information policies for settings that are computationally challenging (e.g., when there are multiple receivers) or for settings where there are no known solutions in general (e.g., when the receiver behavior does not follow the Bayesian rational assumption). We also conduct real-world human-subject experiments and demonstrate that our framework can capture human behavior from data and lead to more effective information policy for real-world human receivers.
The problem of information design, where a player with an information advantage (the sender) can strategically reveal information to influence another player (the receiver) to take certain actions, is ubiquitous in everyday applications. For example, online retailers can highlight a subset of product features to influence buyers to make purchases [32, 46]. Recommendation systems might selectively display other users' ratings to persuade users to follow recommendations [75]. Politicians can influence voters' decisions by designing different policy experiments [1]. There have been various research efforts from economics [51, 62, 29, 30, 44], machine learning and artificial intelligence [79, 12, 2, 33, 6, 25], and general computer science [23, 18] devoted to the study of information design. Among the growing literature on information design, the model of Bayesian persuasion proposed by Kamenica and Gentzkow [39] is one of the most prominent, and has inspired a rich body of studies.
While Bayesian persuasion offers an elegant framework for formulating the information design problem, it has two limitations. First, the receiver is assumed to be Bayesian rational. This means that the receiver can form a posterior in a Bayesian manner and chooses the action that maximizes his expected utility. However, as consistently observed in empirical studies [53, 37], humans often
<sup>1</sup>We use she/he to denote the sender/receiver, respectively.
deviate from being Bayesian or rational. Directly applying the techniques from the information design literature that assume Bayesian rational receivers could lead to suboptimal outcomes. In this work, we address this limitation by proposing a general optimization framework that can integrate a wide range of human behavior, expressed either as traditional analytical closed-form behavioral models or as data-driven models, and design optimal information policies with respect to the provided human behavior.
Second, despite a decade of effort, characterizing the optimal information policy remains notoriously difficult. Dughmi and Xu [18] have shown that it is #P-hard to compute the optimal expected sender utility, and in multi-receiver settings where each receiver only has binary actions, it is #P-hard to even approximate the optimal sender utility within any constant multiplicative factor [78]. Moreover, most previous works have assumed that the receiver follows the Bayesian rational assumption. When this assumption is relaxed [14, 70], there are generally no known analytical solutions for finding the optimal information policy yet.
In this work, we initiate the study of automated information design that encodes human behavior into the design process. Inspired by the recent effort in utilizing deep learning for auction design [20, 58], we propose HAIDNet, an optimization framework that leverages neural-network architectures for information design. Unlike existing works that assume rational human behavior, our optimization framework can adjust to multiple representations of human behavior patterns, including standard behavioral models represented in analytic forms, and data-driven models trained using machine learning approaches. More specifically, We encode receiver behavior as a function and represent the loss in our optimization framework as a function of the receiver's responses to the disclosed information. This approach enables our optimization framework to accommodate different representations of human behavior and can lead to corresponding optimal information policies. Our contributions can be summarized as follows:
- We initiate the study of automated information design that encodes human behavior in the design process. The proposed end-to-end Human behavior encoded neural network mechanism for Automated Information Design, namely HAIDNet, enables us to optimize the sender's information policy based on a given model of human behavior. We incorporate a neural network architecture for information design problems. Moreover, we extend the literature on deep learning for mechanism design to encode realistic human behavior in the design process.
- We evaluate our approach via extensive simulations. In simpler settings with known analytical solutions, we show that HAIDNet can recover the optimal information policies. We also show that HAIDNet can extend to design information policies for settings that are computationally challenging (e.g., multiple receivers involved), or for settings with no known solutions in general (e.g., when the receiver's behavior does not follow the standard Bayesian rationality assumption).
- Through real-world human-subject experiments, we demonstrate that our framework can adapt to scenarios where we do not have access to human models a priori. We demonstrate that our approach can accurately learn a human descriptor from behavioral data, incorporate it in our optimization framework, and result in more effective information design policy in the real world.
Related Work. Our work joins a growing line of research that leverages computational tools for automated mechanism design [11, 65, 10, 66]. More recently, deep neural networks have been utilized for optimizing auction design [20, 27, 31, 13, 42, 61, 58, 45, 12]. Our work differs from this line of work in two ways. First, we extend the approach beyond auction design to address the automated information design problem. Second and more importantly, we have incorporated human behavior in our design, while prior works mostly require standard rationality assumptions.
Our information design formulation builds on the seminal work of Bayesian persuasion [39], which inspired a rich line of research in information design [e.g., see the recent surveys by 38, 4]. In particular, given the practical relevance of information design, there is an increasing number of information design studies in the research community in machine learning and artificial intelligence [79, 12, 2, 33, 25], economics [51, 62, 29, 30, 34], and operations research [44, 76]. Our work differs from most of the existing works in that we integrate human behavior into the design of information policy. There have been a few works addressing non-Bayesian belief updating [14] and non-rational receiver behavior [70, 26] with stylized models in information design. Our work extends previous research by designing an framework that can accommodate both the analytical form and the data-driven form of human behavior. More discussions on related work can be found in Appendix A.
2 Preliminary - Bayesian Persuasion Basics
In Bayesian persuasion, there are two players: a sender and a receiver. The sender's goal is to design an information disclosure policy that persuades the receiver to take certain actions maximizing the sender's objective. The state of nature  $\theta$  is drawn from a finite set  $\Theta \triangleq \{1,\dots,m\}$  according to a prior distribution  $\lambda \triangleq (\lambda (\theta))_{\theta \in \Theta}\in \Delta (\Theta)$ . The prior is common knowledge to both the sender and the receiver. The receiver's utility  $u^{R}(a,\theta)$  depends on the receiver action  $a\in \mathcal{A}$  from an action set  $\mathcal{A}$  and the state  $\theta$ . The sender's utility  $u^{S}(a,\theta)$  also depends on the receiver's action and the state.
The sender can observe the realized state while the receiver cannot, and the sender can utilize this information advantage to persuade the receiver to take the desired action. In particular, before observing the realized state, the sender can commit to an information policy  $\pi$ , specifying what signal to present to the receiver conditional on the realized state. More formally, an information policy  $\pi$  consists of a signal space  $\Sigma$  and a set of conditional probabilities  $\{\pi(\cdot|\theta)\}_{\theta \in \Theta}$  where  $\pi(\cdot|\theta) = (\pi(\sigma|\theta))_{\sigma \in \Sigma} \in \Delta(\Sigma)$  and  $\pi(\sigma|\theta) \in [0,1]$  denotes the probability to send signal  $\sigma \in \Sigma$  given the realized state  $\theta$ . This information disclosure policy is known to the receiver and specifies how the sender discloses information to the receiver. When a state  $\theta \in \Theta$  is realized, the sender sends a signal  $\sigma \sim \pi(\cdot|\theta)$  according to the policy.
In Bayesian persuasion, the receiver is assumed to be Bayesian rational in the sense that upon seeing the signal  $\sigma$ , the receiver forms his posterior belief about the state in a Bayesian manner and takes an action that maximizes his expected utility. Formally, upon seeing the signal realization  $\sigma$ , the receiver updates his posterior belief over the state of nature, denoted by  $\mu(\sigma) \triangleq (\mu(\theta | \sigma))_{\theta \in \Theta} \in \Delta(\Theta)$ , by applying Bayes' rule:  $\mu(\theta | \sigma) \triangleq \frac{\pi(\sigma | \theta) \lambda(\theta)}{\sum_{\theta' \in \Theta} \pi(\sigma | \theta') \lambda(\theta')}$ .
Given the posterior induced from the observed signal  $\sigma \in \Sigma$ , the receiver takes an action  $a^{\mathsf{BR}}(\sigma) \in \mathcal{A}$  that maximizes his expected utility<sup>2</sup>, namely,  $a^{\mathsf{BR}}(\sigma) \triangleq \arg \max_{a \in \mathcal{A}} \sum_{\theta \in \Theta} \mu(\theta | \sigma) u^{R}(a, \theta)$ . The sender's information design problem is to find the optimal information policy that maximizes her expected payoff induced by the receiver's action, as follows:
$$
\max  _ {\pi} \sum_ {\theta \in \Theta} \lambda (\theta) \sum_ {\sigma \in \Sigma} \pi (\sigma | \theta) u ^ {S} (a ^ {\mathsf {B R}} (\sigma), \theta). \tag {PBR}
$$
In this work, our goal is to design an automated framework to solve the above bi-level optimization problem while encoding realistic human behavior in the design process (i.e., replacing the Bayesian rational human model  $a^{\mathrm{BR}}(\sigma)$  with general human behavior).
Example. Consider the scenario in which an online retailer (the sender) aims to persuade a buyer (the receiver) to make a purchase. The retailer's products are directly coming from the factory, and the product quality (represented by the binary state  $\theta$ ) is drawn from a prior distribution  $\lambda$ . The buyer's utility  $u^{R}(a,\theta)$  depends on both his binary purchase decision  $a$  and the binary product quality  $\theta$ , while the retailer's utility  $u^{S}(a,\theta) \equiv u^{S}(a), \forall \theta$  is state-independent and only depends on the buyer's purchase decision. For example, the goal of the retailer is to persuade the buyer to make a purchase, i.e.,  $u^{S}(a) = 1$  for  $a = 1$  and  $u^{S}(a) = 0$  for  $a = 0$ . The buyer only wants to purchase when the product is good, i.e.,  $u^{R}(a,\theta) = 1$  if  $\theta = a$ , and  $u^{R}(a,\theta) = 0$  otherwise.
In order to persuade the buyer to purchase, the retailer can commit to performing (noisy) product inspections  $\pi (\sigma |\theta)$  to reveal information about the product quality. For example, the inspection might signal the product quality is satisfactory with  $80\%$  chance if the quality of the product is indeed satisfactory (i.e.,  $\pi (\sigma = 1|\theta = 1) = 0.8$ ) and signal the product quality is unsatisfactory with  $90\%$  chance if the quality is indeed unsatisfactory (i.e.,  $\pi (\sigma = 0|\theta = 0) = 0.9$ ). The information design problem for the retailer is to identify an inspection policy that maximizes the probability on selling the product to the buyer.
3 HAIDNet: Encoding Human Behavior in Automated Information Design
In this section, we introduce HAIDNet, an optimization framework based on a neural network architecture that can adjust to various forms of human behavior. In the following discussion, we first describe how we modularize human behavior in information design. We then explain the neural
$^{2}$ BR here stands for Bayesian rational.
Figure 1: The overall HAIDNet framework. The human descriptor module is given to the optimization module before training. The optimization is performed through back propagation which evaluates the gradient of the loss to update the weights in the neural network structure.
network architecture of our proposed HAIDNet that can adapt to different forms of human behavior. Finally, we outline the procedures for optimizing the information policy in HAIDNet.
3.1 Encoding Human Behavior
Bayesian persuasion assumes that the receiver is Bayesian rational. However, in practice, this assumption often does not hold. The receiver may exhibit systematic biases both in belief updating and in decision making. In the following discussion, we formulate the sender's problem on finding the optimal information policy when taking more general human behavior into account.
Definition 3.1 (Human Behavior Descriptor). For any receiver utility  $u^R$ , prior  $\lambda$ , sender information policy  $\pi$  (and signal space  $\Sigma$ ), a human behavior descriptor is denoted by  $H_{u^R, \lambda, \pi}(\sigma, a)$ , representing the probability for a human receiver to choose action  $a \in \mathcal{A}$  when seeing a realized signal  $\sigma \in \Sigma$ .
When the context is clear, we omit the subscripts and write  $H_{u^R,\lambda,\pi}(\sigma,a)$  as  $H(\sigma,a)$  for notational simplicity. With the above definition, we can rewrite the sender's information design problem as:
$$
\max  _ {\pi} \sum_ {\theta \in \Theta} \lambda (\theta) \sum_ {\sigma \in \Sigma} \pi (\sigma | \theta) \sum_ {a \in \mathcal {A}} H (\sigma , a) u ^ {S} (a, \theta).
$$
Below we give a few examples of human behavior descriptors.
Bayesian rational (BR). In standard Bayesian persuasion, the receiver updates his posterior in a Bayesian manner and takes action that maximizes the expected utility. Following the definition in Section 2, the human descriptor can be written as  $H(\sigma, a) = \mathbf{1}\{a = a^{\mathrm{BR}}(\sigma)\}$ .
Probability weighting and discrete choice (TH-Model). We present another human behavior descriptor based on the work by Tang and Ho [70] (denoted as the TH-model in the description of this work). In particular, they combine probability weighting, assuming the receiver's posterior is distorted based on a function  $\omega(\cdot): \Delta(\Theta) \to \Delta(\Theta)$ , and discrete choice model, assuming the receiver's action is stochastic, with a higher probability in taking an action with higher expected utility (based on the distorted posterior belief).
Formally, let  $\omega (\theta |\sigma)$  be the receiver's distorted posterior belief after seeing signal  $\sigma$  and  $\beta_{H}$  be a parameter in the discrete choice model that tunes how stochastic the receiver's action is (when  $\beta_{H}\rightarrow \infty$ , the discrete choice model reduces to standard expected utility maximization), the human behavior descriptor for this model can be written as:
$$
H (\sigma , a) = \frac {\exp \left(\beta_ {H} \sum_ {\theta \in \Theta} \omega (\theta | \sigma) u ^ {R} (a , \theta)\right)}{\sum_ {a ^ {\prime}} \exp \left(\beta_ {H} \sum_ {\theta \in \Theta} \omega (\theta | \sigma) u ^ {R} \left(a ^ {\prime} , \theta\right)\right)}. \tag {1}
$$
Data-driven human behavior descriptor. Note that in our formulation, we use the function  $H(\sigma, a)$  to represent human behavior. Suppose we have access to sufficient human behavioral data, instead of expressing  $H(\sigma, a)$  using a closed-form analytical expression, we can train a machine learning model to approximate this function and utilize the learned model as the human behavior descriptor.
3.2 HAIDNet Framework and Optimization
We now introduce the framework of HAIDNet and explain how we utilize it to optimize the sender's information policy for a given human descriptor.
HAIDNet framework. As presented in Figure 1, HAIDNet consists of two modules: the sender's optimization module and the human descriptor. The sender's optimization module is a neural network responsible for optimizing the sender's optimal information policy. It takes the information design problem instances as input, including the prior distribution  $\lambda$  over the states and the payoff functions  $u^S$ ,  $u^R$  for all players. The module outputs an information policy which consists of a set of conditional probabilities  $\{\pi(\cdot|\theta)\}_{\theta \in \Theta}$  over the signal space for each state  $\theta \in \Theta$ .
The human descriptor can either be model-based (e.g., Bayesian rational model or TH model in Equation (1)), or data-driven (e.g., a neural network modeling the receiver's behavior). The human descriptor is treated as a black box from the perspective of the sender's optimization module, and is fixed before HAIDNet begins training. The input of the descriptor consists of the receiver utility  $u^{R}$ , the prior distribution  $\lambda$ , and the information policy  $\pi$  (i.e. the output of the sender's optimization module), while the output is the receiver's response strategy  $H(\sigma, a) = H_{u^{R},\lambda,\pi}(\sigma, a)$ .
**Optimization procedure.** For the sender's optimization, we follow the recent line of research on using deep learning for auction design [20]: we randomly draw problem instances from a pre-specified distribution and perform stochastic gradient descent to minimize the loss function in the training process. The loss function is defined to be the negative of the sender's expected utility, since the goal of the sender is to find the optimal information policy that maximizes her expected utility.
$$
\mathcal {L} _ {u ^ {S}, \lambda} (\pi , H) = - \sum_ {\theta \in \Theta} \lambda (\theta) \sum_ {\sigma \in \Sigma} \pi (\sigma | \theta) \sum_ {a \in \mathcal {A}} H (\sigma , a) u ^ {S} (a, \theta). \tag {2}
$$
Our work differs from previous works in that we incorporate the human behavior descriptor in the definition of the loss function. The requirement is that the human descriptor  $H(\sigma, a)$  needs to be differentiable. This requirement is naturally satisfied in many cases, e.g., when the human descriptor follows the model defined in (1) or is a neural-network-based model, the gradient always exists. However, in the Bayesian rational model, since the receiver chooses the action that maximizes his expected utility, this argmax operation makes the human model not differentiable. To overcome this issue, we approximate the Bayesian rational model using softmax instead of argmax with a sufficiently large softmax scale parameter  $\beta$ .<sup>3</sup> More concretely, let  $u(a)$  be the expected utility for action  $a$ . The softmax operator approximates the receiver's behavior by using  $\exp(\beta u(a)) / \sum_{a'} \exp(\beta u(a'))$  to denote the probability of choosing action  $a$ . As a sanity check, when  $\beta \to \infty$ , this expression reduces to argmax, choosing the action maximizing the expected utility.
Optimization implementation. To optimize HAIDNet, we train a neural network with 3 fully connected layers employing ReLU activation functions and the Adam optimizer. The model is trained on 100 batches of size 1024, for a total of 102, 400 uniformly drawn problem instances (i.e., data points for training). Evaluation of the model is conducted on a test set consisting of 1000 problem instances. The specification of hyperparameters and implementation details are included in the appendix. We have also included the source code in the supplementary materials.
We have conducted extensive simulations to evaluate HAIDNet. Our results demonstrate that HAIDNet can find the near-optimal information policy in various settings. Specifically, we show its effectiveness in settings where efficient methods exist to obtain the optimal information policy (Section 4.1.1) and in computationally challenging settings where finding the optimal information policy is difficult (Section 4.1.2). Moreover, even in settings where no known solutions exist in general, HAIDNet can generate information policy with good performance (Section 4.1.3). We have conducted additional simulations, including examining the convergence of the training, investigating
3The notation  $\beta$  here is different from  $\beta_{H}$  used to model human behavior in (1).
the scalability of the approach, accounting for varying number of receivers, comparing with random policy, and examining empirical run-time. Due to space constraints, these additional simulation results are included in Appendix B.1.
4.1.1 Settings with efficient solutions
We start our evaluations with a simple setting where there exist efficient solutions to find the optimal policy. In this setting, we leverage the efficient solutions as ground truth to examine whether our approach can also identify the optimal information policy.
In particular, we consider the setting with a single Bayesian rational receiver. In this setting, when there are only two actions available for the receiver and there are only two states, there exists a closed-form characterization of the optimal information policy. When the numbers of actions and states are finite constants, the optimal information policy can still be computed efficiently [39]. Therefore, we can evaluate the performance of our approach by comparing the information policy generated by HAIDNet with the optimal policy.
Binary actions and binary states. We first examine the simplest setting with binary actions and binary states (a classical setting in Bayesian persuasion [39]), namely, the action space  $\mathcal{A} = \{0,1\}$  and the state space  $\Theta = \{0,1\}$ , and observe whether HAIDNet produces near-optimal information policies. For the sender utility, we adopt a stylized setting where the sender obtains utility 1 when the receiver takes action 1 and utility 0 when the receiver takes action 0. The receiver aims to take the action that aligns with the true state, i.e.,  $u^{R}(0,1) = u^{R}(1,0) = 0$ , and we randomly draw each value for  $u^{R}(0,0)$  and  $u^{R}(1,1)$  from  $[0,1]$ . In plain words, the receiver prefers action 1 when the state is 1 and action 0 when the state is 0, and the goal of the sender is to persuade the receiver to take action 1. The prior distribution  $\lambda$  is drawn from a Dirichlet distribution. We then simulate data using the setting above and optimize HAIDNet.
We first examine whether the policy generated by HAIDNet matches the known optimal policy. Note that in this simple setting, via revelation principle [39], an information policy can be characterized by two signals, i.e.,  $\sigma \in \{0,1\}$ , where each signal corresponds to a recommended action. Moreover, in the optimal policy, we have  $\pi^{*}(\sigma = 1|\theta = 1) = 1$ , and therefore the optimal policy can be characterized by a single parameter  $\pi^{*}(\sigma = 1|\theta = 0)$ . To examine whether HAIDNet generates the same policy as the optimal policy, we compare the value of this parameter on different scenarios.