SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let us remark that representing promise-form signaling schemes requires a number of bits polynomial in the size of the problem instance and in  $|\mathcal{I}|$ , which is the cardinality of the set of promises. While  $|\mathcal{I}|$  could be arbitrarily large in general, the algorithm that we will present in the following Section 5 guarantees that  $|\mathcal{I}|$  has "small" size, by means of a clever choice of the functions  $I_h$ .
4.2 From promise-form to history-dependent signaling schemes
In the following, we show how the sender can implement promise-form signaling schemes, proving that they represent a subclass of history-dependent ones.
Notice that, in order to completely specify a promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$ , it is sufficient to specify the functions  $\varphi_{h}$  and  $g_{h}$ , since the functions  $I_{h}$  can always be inferred by looking at the images of the functions  $g_{h}$ . However, we included  $I_{h}$  in the definition of promise-form signaling schemes since this will considerably ease notation when dealing with them in Sections 5 and 6.
The sender can implement a promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  as follows. After committing to  $\sigma$  (Line 1 of Algorithm 1), at each step  $h\in \mathcal{H}$ , in Line 6 of Algorithm 1 they select which action-recommendation strategy to use by reconstructing the current promise on the basis of the history  $\tau_{h}$ . Such a reconstruction is done by recursively "composing" functions  $g_{h'}$  for the preceding steps  $h' < h$ , by means of the procedure in Algorithm 2 with  $\sigma$  and  $\tau_{h}$  as inputs. By letting  $\iota \in I_h(s_h)$  be the continuation value obtained by running Algorithm 2, the action recommendation  $a_{h}$  in Line 6 is then sampled from  $\varphi_h(s_h,\iota ,\theta_h)$ . Algorithm 2 clearly runs in time polynomial in the instance size, and, thus, the sender can implement a promise-form signaling scheme efficiently.
In the rest of this section, given a promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  we denote by  $\phi^{\sigma}\coloneqq \{\phi_{\tau}^{\sigma}\}_{\tau \in \mathcal{T}}$  the history-dependent signaling scheme induced by  $\sigma$  thorough the implementation procedure described above. Formally, for every history  $\tau = (s_1,a_1,\dots ,s_{h - 1},a_{h - 1},s_h)\in \mathcal{T}_h$  up to step  $h\in \mathcal{H}$  , the function  $\phi_{\tau}^{\sigma}:\Theta \to \Delta (\mathcal{A})$  is defined so that  $\phi_{\tau}^{\sigma}(\theta) = \varphi (s_h,\iota_\tau^\sigma ,\theta)$  for every  $\theta \in \Theta$  , where  $\iota_{\tau}^{\sigma}\in I_{h}(s_{h})$  denotes the promise value correspond
Algorithm 2 From histories to promises
Require:  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}},$
$$
\tau = \left(s _ {1}, a _ {1}, \dots , s _ {h - 1}, a _ {h - 1}, s _ {h}\right) \in \mathcal {T} _ {h}
$$
1: Initialize  $\iota \gets 0\in I_1(s_1)$
2: for each step  $h' = 1, \ldots, h - 1$  do
3:  $\iota \gets g_{h'}(s_{h'}, a_{h'}, \iota, s_{h' + 1})$
4: return  $\iota$
ing to history  $\tau$ , as computed by Algorithm 2 with  $\sigma$  and  $\tau$  as inputs. As it is easy to see, implementing the promise-form signaling scheme  $\sigma$  as described above is equivalent to using  $\phi^{\sigma}$  in Algorithm 1.
Next, we show that the value functions of the sender and the receiver associated with the induced history-dependent signaling scheme  $\phi^{\sigma}$  can be efficiently computed by only accessing the components of the promise-form signaling scheme  $\sigma$ . In the following, given any  $\sigma := \{(I_h, \varphi_h, g_h)\}_{h \in \mathcal{H}}$ , for every step  $h \in \mathcal{H}$  we introduce the functions  $\mathcal{V}_h^{\mathsf{R},\sigma}: \mathcal{A} \times \mathcal{S} \times \mathcal{I} \to \mathbb{R}$  and  $\mathcal{V}_h^{\mathsf{R},\sigma}: \mathcal{S} \times \mathcal{I} \to \mathbb{R}$ , which are jointly recursively defined so that, for every  $a \in \mathcal{A}$ ,  $s \in \mathcal{S}$ , and  $\iota \in I_h(s)$ , it holds:
$$
\mathcal {V} _ {h} ^ {\mathsf {R}, \sigma} (a, s, \iota) = \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \varphi_ {h} (a | s, \iota , \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} (s ^ {\prime} | s, a, \theta) \mathcal {V} _ {h + 1} ^ {\mathsf {R}, \sigma} \left(s ^ {\prime}, g _ {h} (s, a, \iota , s ^ {\prime})\right)\right)
$$
and  $\mathcal{V}_h^{\mathsf{R},\sigma}(s,\iota) = \sum_{a\in \mathcal{A}}\mathcal{V}_h^{\mathsf{R},\sigma}(a,s,\iota)$ . Similarly,  $\mathcal{V}_h^{\mathsf{S},\sigma}:\mathcal{S}\times \mathcal{I}\to \mathbb{R}$  is such that, for  $s\in S,\iota \in I_h(s)$ :
$$
\mathcal {V} _ {h} ^ {\mathbb {S}, \sigma} (s, \iota) = \sum_ {a \in \mathcal {A}} \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \varphi_ {h} (a | s, \iota , \theta) \left(r _ {h} ^ {\mathbb {S}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} (s ^ {\prime} | s, a, \theta) \mathcal {V} _ {h + 1} ^ {\mathbb {S}, \sigma} (s ^ {\prime}, g _ {h} (s, a, \iota , s ^ {\prime}))\right).
$$
Then, we can prove the following lemma:
Lemma 1. Given a promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  for every  $h\in \mathcal{H}$  and history  $\tau = (s_1,a_1,\dots ,s_{h - 1},a_{h - 1},s_h)\in \mathcal{T}_h$  up to step  $h$ , the following holds:
$$
V _ {h} ^ {\mathsf {R}, \phi^ {\sigma}} (a, \tau) = \mathcal {V} _ {h} ^ {\mathsf {R}, \sigma} (a, s _ {h}, \iota_ {\tau} ^ {\sigma}), V _ {h} ^ {\mathsf {R}, \phi^ {\sigma}} (\tau) = \mathcal {V} _ {h} ^ {\mathsf {R}, \sigma} (s _ {h}, \iota_ {\tau} ^ {\sigma}), \text {a n d} V _ {h} ^ {\mathsf {S}, \phi^ {\sigma}} (\tau) = \mathcal {V} _ {h} ^ {\mathsf {S}, \sigma} (s _ {h}, \iota_ {\tau} ^ {\sigma}).
$$
Intuitively, Lemma 1 establishes that the functions  $\nu_{h}^{\mathsf{R},\sigma}$ ,  $\nu_{h}^{\mathsf{R},\sigma}$ , and  $\nu_{h}^{\mathsf{S},\sigma}$  "correctly" encode the value functions of the sender and the receiver for a promise-form signaling scheme  $\sigma$ , when it is implemented according to the procedure described at the beginning of the section.
Finally, given how a promise-form signaling scheme  $\sigma$  is implemented, in the following we say that  $\sigma$  is  $\epsilon$ -persuasive (for some  $\epsilon \geq 0$ ) if the induced history-dependent signaling scheme  $\phi^{\sigma}$  is  $\epsilon$ -persuasive according to Definition 1. However, using such a definition to check whether  $\sigma$  is  $\epsilon$ -persuasive is clearly computationally inefficient, since it would require working with exponentially-many histories. In Section 4.3, we introduce an easy way to ensure that a promise-form signaling scheme "keeps its promises", and we show that this allows to encode  $\epsilon$ -persuasiveness constraints in an efficient way.
4.3 The power of honesty
Next, we introduce a particular class of promise-form signaling schemes which always guarantee that the sender honestly (approximately) assures promised rewards to the receiver. We call  $\eta$ -honest the promise-form signaling schemes with such a property, which are formally defined as follows:
Definition 2 ( $\eta$ -honesty). Let  $\eta \geq 0$ . A promise-form signaling scheme  $\sigma := \{(I_h, \varphi_h, g_h)\}_{h \in \mathcal{H}}$  is  $\eta$ -honest if, for every step  $h \in \mathcal{H}$ , state  $s \in S$ , and promise  $\iota \in I_h(s)$ , the following holds:
$$
\sum_ {a \in \mathcal {A}} \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \varphi_ {h} (a | s, \iota , \theta) \left(r _ {h} ^ {R} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} \mid s, a, \theta\right) g _ {h} (s, a, \iota , s ^ {\prime})\right) \geq \iota - \eta . \tag {1}
$$
Then, we can prove the following result on  $\eta$ -honest promise-form signaling schemes:
Lemma 2. Let  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  be a promise-form signaling scheme. If  $\sigma$  is  $\eta$ -honest, then, for every step  $h\in \mathcal{H}$  and state  $s\in S$ , it holds that  $\mathcal{V}_h^{\mathsf{R},\sigma}(s,\iota)\geq \iota -\eta (H - h + 1)$  for all  $\iota \in I_h(s)$ .
Since by Lemma 1 the function  $\nu_h^{\mathsf{R},\sigma}$  encodes the receiver's value function when  $\sigma$  is implemented by the sender, Lemma 2 intuitively establishes that, at each step  $h\in \mathcal{H}$  and state  $s\in S$ , the signaling scheme actually "keeps the promise" of giving at least  $\iota \in I_h(s)$  future rewards to the receiver, up to an error depending on  $\eta$ . Checking whether a promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  is  $\eta$ -honest or not can be done in time polynomial in the instance size and  $|\mathcal{I}|$ .
Now, we are ready to prove the following crucial lemma:
Lemma 3. Let  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  be an  $\eta$ -honest promise-form signaling scheme such that, for every  $h\in \mathcal{H}$ ,  $s\in S$ ,  $\iota \in I_h(s)$ , and  $a,a'\in \mathcal{A}$ , the following constraint is satisfied:
$$
\begin{array}{l} \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \varphi_ {h} (a | s, \iota , \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} | s, a, \theta\right) g _ {h} (s, a, \iota , s ^ {\prime})\right) \geq \\ \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \varphi_ {h} (a | s, \iota , \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a ^ {\prime}, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} \mid s, a ^ {\prime}, \theta\right) \widehat {V} _ {h + 1} ^ {\mathsf {R}} \left(s ^ {\prime}\right)\right). \tag {2} \\ \end{array}
$$
Then, we can conclude that  $\sigma$  is  $(\eta H)$ -persuasive.
Intuitively, Lemma 3 states that for an  $\eta$ -honest promise-form signaling scheme, the constraint in Equation (2) is equivalent to the one in Definition 1. The crucial advantage of Equation (2) is that it allows to express persuasiveness conditions as "local" constraints which do not require recursion.
4.4 Promise-form signaling schemes are sufficient
Finally, by exploiting Lemma 3, we can prove the main result of this section: promise-form signaling schemes represent a sufficient subclass of history-dependent ones. Formally:
Theorem 3. There is always a persuasive promise-form signaling scheme  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  with sender's expected reward equal to OPT. More formally, it holds that  $V^{\mathcal{S},\phi^{\sigma}} = \mathrm{OPT}$  for the history-dependent signaling scheme  $\phi^\sigma$  induced by  $\sigma$ .
5 Approximation scheme
Theorem 3 shows that, in order to find an optimal signaling scheme, one can focus on promise-form signaling schemes that have the nice property of being polynomially representable in the instance size and the cardinality  $|\mathcal{I}|$  of the set of promises. In this section, we show how to compute an  $\epsilon$ -persuasive promise-form signaling scheme with sender's expected rewards at least OPT in polynomial time.
We design an algorithm working with sets  $I_h(s)$  defined on a suitable grid, whose size can be properly controlled by a discretization step  $\delta$ . The algorithm solves a recursively-defined optimization problem for each  $h \in \mathcal{H}$ ,  $s \in S$ , and  $\iota \in I_h(s)$ , by starting from step  $H$  and proceeding in bottom up fashion.
For every step  $h \in \mathcal{H}$  and state  $s \in S$ , the set  $I_h(s)$  of promises is defined as a suitable subset of a grid  $\mathcal{D}_{\delta} := \{k \delta \mid k \in \mathbb{N} \land k \leq \lfloor H / \delta \rfloor\}$ , where  $\delta > 0$  is a discretization step to be set depending on the desired relaxation  $\epsilon$  of the persuasiveness constraints. This also allows us to control the representation size of promise-form signaling schemes, as well as the running time of the algorithm. In particular, it holds  $|\mathcal{D}_{\delta}| = O(1 / \delta)$  and, thus,  $|\mathcal{I}| = O(1 / \delta)$  since  $\mathcal{I} \subseteq \mathcal{D}_{\delta}$  by definition. In the following, for ease of notation, we let  $\lfloor x \rfloor_{\delta} := \min_{k \in \mathbb{N}: k \delta \geq x} k \delta$  be the smallest multiple of  $\delta$  greater than  $x$ , while  $\lfloor x \rfloor_{\delta} := \max_{k \in \mathbb{N}: k \delta \leq x} k \delta$  is the greatest multiple of  $\delta$  smaller than  $x$ .
The algorithm keeps track of recursively-computed values in a set of tables, one for each step. The table at step  $h \in \mathcal{H}$  is encoded by means of a function  $M_h^\delta : S \times \mathcal{D}_\delta \to \mathbb{R} \cup \{-\infty\}$ . Intuitively, for every  $s \in S$  and  $\iota \in \mathcal{D}_\delta$ , the entry  $M_h^\delta(s, \iota)$  is related to the expected rewards achieved by the sender when "promising" the receiver expected rewards "approximately equal" to  $\iota$  in state  $s$  at step  $h$ . We also admit the functions  $M_h^\delta$  to take value  $-\infty$ , which semantically corresponds to the case in which it is impossible to guarantee the promise  $\iota$  to the receiver in state  $s$  at step  $h$ . The entry  $M_h^\delta(s, \iota)$  of
the table  $M_h^\delta$  at step  $h$  is computed recursively by solving a problem  $\mathcal{P}_{h,s,\iota}(M_{h+1}^\delta)$  that we define in the following, where  $M_{h+1}^\delta$  is the (previously-computed) table at step  $h + 1$ .
By letting  $M: S \times \mathcal{D}_{\delta} \to \mathbb{R} \cup \{-\infty\}$  be a function encoding a generic table over  $S \times \mathcal{D}_{\delta}$ , for every step  $h \in \mathcal{H}$ , state  $s \in S$ , and promise  $\iota \in I_h(s)$ , we define the value  $\Pi_{h,s,\iota}(M)$  of the optimization problem  $\mathcal{P}_{h,s,\iota}(M)$  as follows:
$$
\Pi_{h,s,\iota}(M):= \max_{\substack{\kappa :\Theta \to \Delta (\mathcal{A})\\ q:\mathcal{A}\times \mathcal{S}\to \mathcal{D}_{\delta}}}F_{h,s,M}(\kappa ,q)\quad \text{s.t.}\quad (\kappa ,q)\in \Psi_{\iota}^{h,s},
$$
where problem variables are encoded by the functions  $\kappa : \Theta \to \Delta(\mathcal{A})$  and  $q: \mathcal{A} \times \mathcal{S} \to \mathcal{D}_{\delta}$ , which represent an action-recommendation strategy and a promise function, respectively. The objective function  $F_{h,s,M}(\kappa, q)$  of the optimization problem is defined as:
$$
F _ {h, s, M} (\kappa , q) := \sum_ {\theta \in \Theta} \sum_ {a \in \mathcal {A}} \mu_ {h} (\theta | s) \kappa (a | \theta) \left(r _ {h} ^ {\mathsf {S}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} (s ^ {\prime} | s, a, \theta) M (s ^ {\prime}, q (a, s ^ {\prime}))\right),
$$
which encodes the sender's expected reward when their values for the next step  $h + 1$  are those specified by the table  $M$ . We assume that  $0 \cdot (-\infty) = -\infty$ . Moreover, the set  $\Psi_{\iota}^{h,s}$  is comprised of the functions  $\kappa : \Theta \to \Delta(\mathcal{A})$  and  $q: \mathcal{A} \times \mathcal{S} \to \mathcal{D}_{\delta}$  that satisfy the following constraints:
$$
\sum_ {a \in \mathcal {A}} \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \kappa (a | \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} \mid s, a, \theta\right) q \left(a, s ^ {\prime}\right)\right) \geq \iota \tag {4a}
$$
$$
\begin{array}{l} \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \kappa (a | \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} \mid s, a, \theta\right) q \left(a, s ^ {\prime}\right)\right) \geq \\ \sum_ {\theta \in \Theta} \mu_ {h} (\theta | s) \kappa (a | \theta) \left(r _ {h} ^ {\mathsf {R}} (s, a ^ {\prime}, \theta) + \sum_ {s ^ {\prime} \in \mathcal {S}} p _ {h} \left(s ^ {\prime} \mid s, a ^ {\prime}, \theta\right) \widehat {V} _ {h + 1} ^ {\mathsf {R}} \left(s ^ {\prime}\right)\right) \quad \forall a, a ^ {\prime} \in \mathcal {A}, \tag {4b} \\ \end{array}
$$
where Equation (4a) and Equation (4b) play the role of the honesty and the persuasiveness constraints, respectively. Notice that relaxing the honesty constraint yields larger feasible sets. Formally, for any  $\iota \geq \iota' \geq 0$  we have that the following holds:  $\Psi_{\iota}^{h,s} \subseteq \Psi_{\iota'}^{h,s}$ .
If  $F_{h,s,M}(\kappa,q) = -\infty$  for all  $(\kappa,q) \in \Psi_\iota^{h,s}$ , we have that  $\Pi_{h,s,\iota}(M) = -\infty$ . This intuitively comes from the fact that, if  $\Pi_{h,s,\iota}(M) = -\infty$ , then the value  $\iota$  promised to the receiver is not realizable.
The optimization problem  $\mathcal{P}_{h,s,\iota}(M)$  could be easily cast as a mixed-integer quadratic program, which are too general to be solved efficiently. Thus, we need specifically-tailored procedures to find an approximate solution to it. This discussion is deferred to Section 6. In the following, we assume to have access to an oracle  $\mathcal{O}_{h,s,\iota}(M)$  that provides a suitable approximate solution to  $\mathcal{P}_{h,s,\iota}(M)$ . In particular,  $\mathcal{O}_{h,s,\iota}(M)$  must satisfy the requirements introduced by the following definition:
Definition 3 (Approximate Oracle). An algorithm  $\mathcal{O}_{h,s,\iota}(M)$  is an approximate oracle for  $\mathcal{P}_{h,s,\iota}(M)$  if it returns a tuple  $(\kappa, q, v)$  such that  $\Pi_{h,s,\iota}(M) \leq v \leq F_{h,s,M}(\kappa, q)$  and  $(\kappa, q) \in \Psi_{\iota - \delta}^{h,s}$ .
Intuitively, we ask that an approximate oracle finds a solution  $(\kappa ,q)$  in a slightly larger set  $\Psi_{\iota -\delta}^{h,s}$ . The oracle also returns a value  $v$  to be inserted into  $M_h^\delta (s,\iota)$ , where  $v$  is possibly different from the value  $F_{h,s,M}(\kappa ,q)$  of the objective function. This is needed for technical reasons in order to recover some concavity properties of the functions defining the tables used by the algorithm, which may be lost due to approximations. A complete discussion on this last aspect can be found in Section 6.
Equipped with an approximate oracle as in Definition 3, we are ready to design our approximation scheme that computes an  $\epsilon$ -persuasive promise-form signaling scheme attaining sender's expected reward at least OPT (Algorithm 3). The algorithm iteratively builds each table  $M_h^\delta$  by filling it with the values  $v$  returned by the approximate oracle. Moreover, it sets action-recommendations strategies  $\varphi_h$  and promises functions  $g_h$  of the signaling scheme  $\sigma := \{(I_h, \varphi_h, g_h)\}_{h \in \mathcal{H}}$  to be equal to the functions  $\kappa, q$  returned by the approximate oracle.
The optimization problem over functions  $\kappa$  and  $q$  can be rewritten as an equivalent program with tabular variables, since the functions  $\kappa$  and  $q$  map discrete sets to discrete sets (or a randomization over them).
In order to clarify the semantic of the tables  $M_h^\delta$  built by Algorithm 3, we prove the following preliminary result, which states that the sender's expected rewards for the signaling scheme returned by Algorithm 3 constitute an upper bound on the entries of the tables  $M_h^\delta$ . Formally:
Lemma 4. Let  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$  be returned by Algorithm 3 instantiated with any oracle  $\mathcal{O}_{h,s,\iota}$  as in Definition 3. For every  $h\in \mathcal{H}$ ,  $s\in S$ ,  $\iota \in I_h(s)$ , it holds that  $\mathcal{V}_h^{\mathsf{S},\sigma}(s,\iota)\geq M_h^\delta (s,\iota)$ .
Moreover, the entries of the tables  $M_h^\delta$  built by Algorithm 3 are upper bounds on the sender's expected rewards provided by an optimal promise-form signaling scheme. Formally:
Lemma 5. Let  $M_h^\delta$ ,  $I_h$  (for  $h \in \mathcal{H}$ ) be computed by Algorithm 3 instantiated with any oracle  $\mathcal{O}_{h,s,\iota}$
as in Definition 3, and let  $\sigma^{\star} = \{(I_h^{\star},\varphi_h^{\star},g_h^{\star})\}_{h\in \mathcal{H}}$  be an optimal promise-form signaling scheme. For every  $h\in \mathcal{H}$ $s\in S$  , and  $\iota \in I_h^\star (s)$  , it holds that  $\lceil \iota \rceil_{\delta}\in I_h(s)$  and  $M_h^\delta (s,\lceil \iota \rceil_\delta)\geq \mathcal{V}_h^{\mathbb{S},\sigma^{\star}}(s,\iota)$
Algorithm 3 Approximation scheme
Require:  $\delta \in (0,1)$    
1:  $M_{H + 1}^{\delta}(s,0)\gets 0$  for all  $s\in S$    
2:  $M_{H + 1}^{\delta}(s,\iota)\gets -\infty$  for  $s\in S,\iota \in \mathcal{D}_{\delta}\backslash \{0\}$    
3: for  $h = H,\dots ,1$  do   
4: for  $s\in S$  do   
5:  $I_h(s) = \{\emptyset \}$    
6: for  $\iota \in \mathcal{D}_{\delta}$  do   
7:  $(\kappa ,q,v)\gets \mathcal{O}_{h,s,\iota -\delta}(M_{h + 1}^{\delta})$    
8:  $M_h^\delta (s,\iota)\gets v$    
9:  $\varphi_h(a|s,\iota ,\theta)\gets \kappa (a|\theta)$    
10:  $g_{h}(s,a,\iota ,s^{\prime})\gets q(a,s^{\prime})$    
11: if  $v > - \infty$  then   
12:  $I_{h}(s)\gets I_{h}(s)\cup \{\iota \}$    
13: return  $\sigma \coloneqq \{(I_h,\varphi_h,g_h)\}_{h\in \mathcal{H}}$
By combining Lemma 5 and Lemma 4 we can promptly state and prove the main result of this section.
Theorem 4. For any  $\epsilon >0$ , given an approximate oracle as in Definition 3, Algorithm 3 instantiated with  $\delta = \epsilon /2H$  runs in time polynomial in  $1 / \epsilon$  and the instance size, while it finds an  $\epsilon$ -persuasive promise-form signaling scheme that guarantees expected reward at least OPT to the sender.
6 Building a polynomial-time approximate oracle
In the previous section, we described an approximation scheme (Algorithm 3) that works provided it has access to an oracle that approximately solves the problem  $\mathcal{P}_{h,s,\ell}(M)$  while satisfying the requirements of Definition 3. In this section, we exploit the specific structure of the problem to design an algorithm implementing such an oracle.
First, we relax the problem  $\mathcal{P}_{h,s,\iota}(M)$  into  $\mathcal{R}_{h,s,\iota}(M)$ , which optimizes the expected value of the sender's rewards over randomizations of promises  $\iota \in \mathcal{D}_{\delta}$ . Moreover, we define for all  $s' \in S$  the set  $\mathcal{D}_{\delta}(s', M) = \{\iota \in \mathcal{D}_{\delta} : M(s', \iota) > -\infty\}$  as the set of realizable promises, which semantically means
that those promises can be realized, conditioned on what is stored in the table  $M$ .<sup>10</sup> Specifically, the variables of  $\mathcal{R}_{h,s,\iota}(M)$  are encoded by a function  $\kappa: \Theta \to \Delta(\mathcal{A})$  and a randomized function  $\tilde{q}: \mathcal{S} \times \mathcal{A} \to \Delta(\mathcal{D}_{\delta})$ . The objective function is
$$
\widetilde{F}_{h,s,M}(\kappa ,\tilde{q}):= \sum_{\theta \in \Theta}\sum_{a\in \mathcal{A}}\mu_{h}(\theta |s)\kappa (a|\theta)\Bigg(r_{h}^{\mathsf{S}}(s,a,\theta) + \sum_{s^{\prime}\in \mathcal{S}}p_{h}(s^{\prime}|s,a,\theta)\sum_{\iota^{\prime}\in \mathcal{D}_{\delta}(s^{\prime},M)}\tilde{q} (\iota^{\prime}|a,s^{\prime})M(s^{\prime},\iota^{\prime})\Bigg),
$$
which modifies the objective function  $F_{h,s,M}(\kappa ,q)$  of  $\mathcal{P}_{h,s,\iota}(M)$  by introducing the expectation over the possible promises  $\iota^{\prime}\in \mathcal{D}_{\delta}$ , which are randomized through  $\tilde{q} (\iota^{\prime}|a,s^{\prime})$ .
To simplify notation, for any set  $\mathcal{X}$ , table  $M$ , and randomized function  $\lambda: \mathcal{X} \times S \to \Delta(\mathcal{D}_{\delta})$ , we denote by  $\lambda_{\mathbb{E}}: \mathcal{X} \times S \to \operatorname{co}(\mathcal{D}_{\delta})$  a quantity related to its average, where co denotes the convex hull of a set. Formally,  $\lambda_{\mathbb{E}}(x, s') = \sum_{\iota' \in \mathcal{D}_{\delta}(s', M)} \iota' \cdot \lambda(\iota'|x)$  for all  $x \in \mathcal{X}$ . Moreover, we denote by  $\lfloor \lambda_{\mathbb{E}} \rfloor_{\delta}: \mathcal{X} \to \mathcal{D}_{\delta}$  its discrete average such that  $\lfloor \lambda_{\mathbb{E}} \rfloor_{\delta}(x, s') = \lfloor \lambda_{\mathbb{E}}(x, s') \rfloor_{\delta}$  for all  $x \in \mathcal{X}$  and