SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 4 Approximate oracle
Require:  $\delta \in (0,1), h \in \mathcal{H}, s \in \mathcal{S}, \iota \in \mathcal{D}_{\delta}, M : \mathcal{S} \times \mathcal{D}_{\delta} \to \mathbb{R}$   
1: if  $\mathcal{R}_{h,s,\iota}(M)$  is feasible then  
2:  $(\kappa, \tilde{q}) \gets \text{Solution to } \mathcal{R}_{h,s,\iota}(M)$   
3:  $v \gets \widetilde{F}_{h,s,M}(\kappa, \tilde{q})$   
4:  $q \gets \lfloor \tilde{q}_{\mathbb{E}} \rfloor_{\delta}$   
5: else  
6:  $(\kappa, q)$  arbitrary  
7:  $v \gets -\infty$   
8: return  $(\kappa, q, v)$
<sup>10</sup>Note that set is always not empty when instantiating  $M = M_h^\delta$  for some  $h \in \mathcal{H}$ , as it always contains 0 for all  $s' \in S$ . This can be easily seen from Lemma 5 with  $\iota = 0$ .
$s' \in S$ . Notice that, for ease of notation, we drop the dependence of the operator  $\mathbb{E}$  from  $M$ , as such a dependence is always clear from the context.
By exploiting the notation introduced above, the relaxed optimization problem  $\mathcal{R}_{h,s,\iota}(M)$  reads as:
$$
\Omega_{h,s,\iota}(M):= \max_{\substack{\kappa :\Theta \to \Delta (\mathcal{A})\\ \tilde{q}:\mathcal{A}\times \mathcal{S}\to \Delta (D_{\delta})}}\widetilde{F}_{h,s,M}(\kappa ,\tilde{q}) \text{s.t.} (\kappa ,\tilde{q}_{\mathbb{E}})\in \Psi_{\iota}^{h,s},
$$
where we relax the functions  $\tilde{q}$  to be distributions over  $\mathcal{D}_{\delta}$ , rather than deterministic. Then, we can prove the following:
Lemma 6. The problem  $\mathcal{R}_{h,s,\iota}(M)$  can be solved in time polynomial in  $1 / \delta$  and the instance size.
By relying on a solution  $(\kappa, \tilde{q})$  to  $\mathcal{R}_{h,s,t}(M)$ , Algorithm 4 is an approximate oracle for  $\mathcal{P}_{h,s,t}(M)$ . In particular, when the relaxed problem is feasible, the algorithm returns  $k$  and the function  $q$  obtained by de-randomizing the randomized function  $\tilde{q}$  with its discrete average. Formally:
Theorem 5. For every  $h \in \mathcal{H}$ ,  $s \in S$ , and  $\iota \in \mathcal{D}_{\delta}$ , if Algorithm 4 is used for all  $h' > h$  as approximate oracle  $\mathcal{O}_{h',s,\iota}$  in Algorithm 3, then it implements an approximate oracle as in Definition 3.
Notice that the value  $v$  returned by Algorithm 4 is the optimal value of the relaxed problem  $\mathcal{R}_{h,s,\iota}(M)$ . It is easy to prove (see the proof of Theorem 5) that the tables built by Algorithm 3 through Algorithm 4 are encoded by concave functions. This is the key feature that allows us to go from a randomized solution to its discrete average without loss in sender's expected rewards, and it is the reason why we need to assume that Algorithm 4 is employed at every step  $h$  in Theorem 5.
In this paper, we provided a crucial advancement in Bayesian persuasion for sequential decision making, mastering the setting in which a farsighted receiver interacts with an MDP, a problem that was previously thought to be intractable [Wu et al., 2022, Gan et al., 2022a,b].
First, we have shown that the class of Markovian signaling schemes, the standard choice in previous works and most MDP settings, do not constitute the "right" class of policies, since finding an optimal Markovian signaling scheme is NP-hard. Instead, we demonstrated that history-dependent signaling schemes allow to both circumvent the negative complexity result affecting Markovian signaling schemes and guarantee higher expected rewards to the sender in general. Specifically, we designed an algorithm to find an optimal  $\epsilon$ -persuasive history-dependent signaling scheme in time polynomial in  $1 / \epsilon$  and in the instance size. The crucial component of the algorithm is to restrict the optimization to a convenient subclass of history-dependent signaling schemes, which we call *promise-form* as they encode the history into a promise of future rewards to the receiver. We showed that promise-form signaling schemes are both efficient to represent and as powerful as the general class of history-dependent signaling schemes. An interesting insight of our analysis reveals that being *honest*, i.e., keeping promises of future rewards, is the key to persuade farsighted receivers in MDPs.
As a future work, we will investigate the online learning variant of the problem studied in this paper, as done in [Zu et al., 2021, Wu et al., 2022, Bernasconi et al., 2022], in which the transition function is not known. Moreover, it would be interesting to study the problem faced by a sender that needs to persuade a stream of receivers having adversarially-selected types, as done in [Balcan et al., 2015, Castiglioni et al., 2020b, 2021b, Bernasconi et al., 2023].
This paper is supported by FAIR (Future Artificial Intelligence Research) project, funded by the NextGenerationEU program within the PNRR-PE-AI scheme (M4C2, Investment 1.3, Line on Artificial Intelligence), and by the EU Horizon project ELIAS (European Lighthouse of AI for Sustainability, No. 101120237).
Paola Alimonti and Vigo Kann. Some APX-completeness results for cubic graphs. Theoretical Computer Science, 237:123-134, 04 2000.
Ricardo Alonso and Odilon Câmara. Persuading voters. American Economic Review, 2016.
Yakov Babichenko and Siddharth Barman. Algorithmic aspects of private Bayesian persuasion. In ITCS, 2017.
Maria-Florina Balcan, Avrim Blum, Nika Haghtalab, and Ariel D Procaccia. Commitment without regrets: Online learning in stackelberg security games. In EC, 2015.
Martino Bernasconi, Matteo Castiglioni, Alberto Marchesi, Nicola Gatti, and Francesco Trovò. Sequential information design: Learning to persuade in the dark. In NeurIPS, 2022.
Martino Bernasconi, Matteo Castiglioni, Andrea Celli, Alberto Marchesi, Francesco Trovò, and Nicola Gatti. Optimal rates and efficient algorithms for online bayesian persuasion. In ICML, 2023.
Dimitri Bertsekas. Network optimization: continuous and discrete models, volume 8. Athena Scientific, 1998.
Umang Bhaskar, Yu Cheng, Young Kun Ko, and Chaitanya Swamy. Hardness results for signaling in bayesian zero-sum and network routing games. In EC, 2016.
Peter Bro Miltersen and Or Sheffet. Send mixed signals: earn more, work less. In EC, 2012.
Ozan Candogan. Persuasion in networks: Public signals and k-cores. In  $EC$ , 2019.
Matteo Castiglioni and Nicola Gatti. Persuading voters in district-based elections. In AAAI, 2021.
Matteo Castiglioni, Andrea Celli, and Nicola Gatti. Persuading voters: It's easy to whisper, it's hard to speak loud. In AAAI, 2020a.
Matteo Castiglioni, Andrea Celli, Alberto Marchesi, and Nicola Gatti. Online Bayesian persuasion. In NeurIPS, 2020b.
Matteo Castiglioni, Andrea Celli, Alberto Marchesi, and Nicola Gatti. Signaling in Bayesian network congestion games: the subtle power of symmetry. In AAAI, 2021a.
Matteo Castiglioni, Alberto Marchesi, Andrea Celli, and Nicola Gatti. Multi-receiver online Bayesian persuasion. In ICML, pages 1314-1323, 2021b.
Jiarui Gan, Rupak Majumdar, Goran Radanovic, and Adish Singla. Bayesian persuasion in sequential decision-making. In AAAI, 2022a.
Jiarui Gan, Rupak Majumdar, Goran Radanovic, and Adish Singla. Sequential decision making with information asymmetry. In CONCUR, 2022b.
Itay Goldstein and Yaron Leitner. Stress tests and information disclosure. Journal of Economic Theory, 177:34-69, 2018.
Emir Kamenica and Matthew Gentzkow. Bayesian persuasion. American Economic Review, 101(6): 2590-2615, 2011.
Anton Kolotilin. Experimental design to persuade. Games and Economic Behavior, 90:215-226, 2015.
Yishay Mansour, Aleksandrs Slivkins, Vasilis Syrgkanis, and Zhiwei Steven Wu. Bayesian exploration: Incentivizing exploration in Bayesian games. In EC, 2016.
Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
Zinovi Rabinovich, Albert Xin Jiang, Manish Jain, and Haifeng Xu. Information disclosure as a means to security. In AAMAS, 2015.
Jibang Wu, Zixuan Zhang, Zhe Feng, Zhaoran Wang, Zhuoran Yang, Michael I Jordan, and Haifeng Xu. Markov persuasion processes and reinforcement learning. In EC, 2022.
Haifeng Xu, Rupert Freeman, Vincent Conitzer, Shaddin Dughmi, and Milind Tambe. Signaling in Bayesian Stackelberg games. In AAMAS, 2016.
You Zu, Krishnamurthy Iyer, and Haifeng Xu. Learning to persuade on the fly: Robustness against ignorance. In EC, 2021.
A History-dependent signaling schemes are necessary
In this section, we provide an illustrative MDP instance in which an history-dependent signaling scheme is required in order to optimally solve a Bayesian persuasion problem with farsighted receiver. This instance also proves Theorem 1. The instance is depicted in Figure 1. It is easy to check that the receiver can achieve expected rewards
$$
\widehat {V} _ {1} ^ {\mathsf {R}} (s _ {0}) = 5, \quad \widehat {V} _ {2} ^ {\mathsf {R}} (s _ {1}) = 1 0, \quad \widehat {V} _ {2} ^ {\mathsf {R}} (s _ {2}) = 0, \quad \widehat {V} _ {3} ^ {\mathsf {R}} (s _ {3}) = 0, \quad \widehat {V} _ {3} ^ {\mathsf {R}} (s _ {4}) = 0,
$$
without sender's recommendations. If the sender commits to a non-stationary Markovian signaling scheme  $\phi^{\star} = \{\phi_h\}_{h\in \mathcal{H}}$ , where  $\phi_h:\mathcal{S}\times \Theta \to \Delta (\mathcal{A})$ , they can achieve an expected reward of  $V^{\mathsf{S},\phi^{\star}} = V_{1}^{\mathsf{S},\phi^{\star}}(s_{0}) = 25$  while being persuasive. The latter is obtained by recommending actions deterministically as follows:
$$
\phi_ {2} ^ {\star} (s _ {2}) = a _ {1}, \quad \phi_ {3} ^ {\star} (s _ {3}, \theta_ {0}) = a _ {0}, \quad \phi_ {3} ^ {\star} (s _ {3}, \theta_ {1}) = a _ {0}.
$$
Instead, an history-dependent signaling scheme  $\phi^{\dagger} := \{\phi_{\tau}\}_{\tau \in \mathcal{T}}$  can provide the sender with an expected reward of  $V^{\mathsf{S},\phi^{\dagger}} = V_{1}^{\mathsf{S},\phi^{\dagger}}(s_{0}) = 30$  while being persuasive. This can only be obtained by adapting the action recommendation strategy in state  $s_3$  according to whether the receiver passed through state  $s_1$  or state  $s_2$ , as follows:
$$
\phi_ {(s _ {0}, s _ {1}, s _ {3})} ^ {\dagger} (\theta_ {0}) = a _ {0}, \quad \phi_ {(s _ {0}, s _ {1}, s _ {3})} ^ {\dagger} (\theta_ {1}) = a _ {0}, \quad \phi_ {(s _ {0}, s _ {2}, s _ {3})} ^ {\dagger} (\theta_ {0}) = a _ {0}, \quad \phi_ {(s _ {0}, s _ {2}, s _ {3})} ^ {\dagger} (\theta_ {1}) = a _ {1},
$$
which makes the recommendation  $\phi_{(s_0,s_2)}^\dagger = a_0$  persuasive as well.
Figure 1: Visual representation of a Bayesian persuasion problem defined over an MDP with  $\mathcal{S} = \{s_0, s_1, s_2, s_3\}$ ,  $\mathcal{A} = \{a_0, a_1\}$ ,  $\Theta = \{\theta_0, \theta_1\}$ ,  $\mathcal{H} = [1, 2, 3]$ , and  $\beta(s_0) = 1$ . The probabilities of the state transitions  $p_h$  and the private observations  $\mu_h$  are reported on the dashed edges. The reward functions  $r_h^{\mathrm{S}}$ ,  $r_h^{\mathrm{R}}$  for the sender and the receiver, respectively, are reported on the solid edges, when different from 0. For the sake of clarity, the visualization omits some irrelevant information, such as private observations in  $s_0, s_1, s_2, s_4$  and deterministic transitions.
States where action selection is irrelevant are omitted for brevity.
The latter example proves the existence of an MDP instance where history-dependent signaling schemes are necessary. The reported instance does not require bizarre constructions, and we believe that exploiting history to optimize action recommendations is crucial in practical scenarios as well.
Intuition of Figure 1. In such a scenario, the MDP in Figure 1 can model a ride-sharing platform, where the drivers act as receivers and the platform is the sender. The platform can recommend routes and matching orders with the drivers, which then share with the platform a portion of the profit originating from the trips. In this example,  $s_3$  can be thought as a location with high demand that is costly to reach, such as an airport terminal far from the city center. We can further think of the path  $s_1 \rightarrow s_3$  as a driver's trip to the airport while serving an order, and  $s_2 \rightarrow s_3$  as an empty trip instead. The platform can then adapt its order matching strategy according to whether the driver suffered empty costs or got a profit to come to the airport, such as guaranteeing a quick yet cheap match to the former, while waiting a more lucrative trip for the latter. This adaptive strategy can only be executed through an history-dependent signaling scheme. Instead, a Markovian signaling scheme cannot lure a driver into coming to the airport without serving an order, with diminishing profits for the platform.
B Proofs omitted from Section 3
Proof. We reduce from a promise version of vertex cover in cubic graphs. In particular, given a cubic graph  $(V,E)$ , there exists a  $\gamma \in (0,2]$  such that it is NP-hard to decide whether there exists a vertex cover of size  $k$  or all the vertex covers have size at least  $(1 + \gamma)k$  [Alimonti and Kann, 2000].
We design an instance such that the set of states  $S$  includes  $s_0, s_1, s_2$  and  $s_3$ . Moreover, it includes a state  $s_e$  for each  $e \in E$ , and a state  $s_v$  for each  $v \in V$ . The time horizon is set as  $H = 3$ . All the transition function and reward are independent from the time step  $h \in [H]$ . Hence, we will remove the subscript  $h$  from the notation.
The rewards and transition in the different states are as follows:
- In state  $s_0$  there is a single state of nature  $\theta_0$  and two actions  $a_{0,1}$  and  $a_{0,2}$ . The transition function is such that  $p(s_3|s_0,a_{0,1},\theta_0) = 1$  while  $p(s_e|s_0,a_{0,2},\theta_0) = \frac{1}{|E|}$  for each  $e \in E$ . The rewards of the receiver in state  $s_0$  are  $r^{\mathsf{R}}(a_{0,1},s_0,\theta_0) = 1$  and  $r^{\mathsf{R}}(a_{0,2},s_0,\theta_0) = 0$ . The rewards of the sender in state  $s_0$  are  $r^{\mathsf{S}}(a_{0,1},s_0,\theta_0) = 0$  and  $r^{\mathsf{S}}(a_{0,2},s_0,\theta_0) = 1$ .
- In state  $s_1$ , there is a single state of nature  $\theta_1$  and a single action  $a_1$ . The transition function is  $p(s_2 | s_1, a_1, \theta_1) = 1$ , while the rewards of the sender and the receiver in  $s_1$  are always 0.
- In state  $s_2$ , there is a single state of nature  $\theta_2$  and a single action  $a_2$ . The transition function is  $p(s_v | s_2, a_2, \theta_2) = \frac{1}{|V|}$  for each  $v \in V$ . The sender's and receiver's rewards in  $s_2$  are 0.
- In state  $s_3$ , there is a single state of nature  $\theta_3$  and a single action  $a_3$ . The transition function is such that  $p(s_3 | s_3, a_3, \theta_3) = 1$ . The rewards of the sender and the receiver in  $s_3$  are always 0.
- For each  $e = (v, u) \in E$ , in the state  $s_e$ , there is a single state of nature  $\theta_e$ , and two actions  $a_{e,v}$  and  $a_{e,u}$ . The transition function is such that  $p(s_v|s_e, a_{e,v}, \theta_e) = 1$  and  $p(s_u|s_e, a_{e,u}, \theta_e) = 1$ . The rewards of the sender and the receiver in state  $s_e$  are 0.
- For each  $v \in V$  in the state  $s_v$ , there are two states of nature  $\theta_{v,1}$  and  $\theta_{v,2}$ . There are three actions  $a_{v,1}, a_{v,2}$ , and  $a_{v,3}$ . The transition function is such that  $p(s_3|s_v,a,\theta) = 1$  for each  $a$  and  $\theta$ . The rewards of the receiver in  $s_v$  are  $r^{\mathsf{R}}(s_v,a_{v,1},\theta_{v,1}) = 1$ ,  $r^{\mathsf{R}}(s_v,a_{v,2},\theta_{v,2}) = 1$ ,  $r^{\mathsf{R}}(s_v,a_{v,3},\theta_{v,1}) = r^{\mathsf{R}}(s_v,a_{v,3},\theta_{v,2}) = \frac{1}{2}$  and 0 otherwise. The rewards of the sender in  $s_v$  is  $\frac{1}{2}$  if the receiver plays  $a_{v,3}$ , i.e.,  $r^{\mathsf{S}}(s_v,a_{v,3},\theta_{v,1}) = r^{\mathsf{S}}(s_v,a_{v,3},\theta_{v,2}) = \frac{1}{2}$ , and 0 otherwise.
- All nature's states are drawn uniformly on the nature's outcome at each node.
- Finally, the initial distribution over the states is  $\beta$  such that  $\beta(s_0) = \beta(s_1) = \frac{1}{2}$ .
Figure 2: Illustrative instance of a cubic graph  $(V,E)$ .
In Figure 2 we have a simple instance of a cubic graph (undirected graph with degree bounded by 3), and in Figure 3 we have the instance the persuasion MDP built from the cubic graph in Figure 2.
We recall that a non-stationary Markovian signaling scheme is defined by a set of functions  $\{\phi_h\}_{h\in \mathcal{H}}$  where  $\phi_h:S\times \Theta \to \Delta (\mathcal{A})$ . However, we use both states  $s_1$  and  $s_2$  so that it takes two steps to arrive in states  $s_v$  both when starting from  $s_0$  and when starting from  $s_1$ . Hence, each state  $s$  (excluding the sink state  $s_3$ ) can be reached only with a specific time step  $h$ . This allows us to work only with stationary signaling scheme, i.e., such that the signals do not depend on the time step. Hence, we can remove the subscript  $h$  from the signaling scheme notation.
(a) Instance of Bayesian Persuasion in MDP
(b) Gadget  $\mathsf{G}_v$
Figure 3: Instance of Bayesian persuasion in MDP constructed from the cubic graph  $(V,E)$  of Figure 2. Sender's and receiver's rewards are reported only when different from 0. Solid lines represent actions (that are reported on the corresponding edges) while dashed lines represent stochastic outcomes (either of the transition model or of the prior). For each  $v\in V$  the gadget  $\mathsf{G}_v$  is reported in Figure 3b.
We will show that if there exists a vertex cover of size  $k$  then there exists a persuasive signaling scheme such that the sender's expected rewards is at least  $\frac{3}{4} - \frac{k}{4|V|}$ , while if all the vertex covers have size at least  $(1 + \gamma)k$  then all the  $\epsilon$ -persuasive signaling schemes have sender's expected rewards strictly less than  $\alpha \left( \frac{3}{4} - \frac{k}{4|V|} \right)$ , where  $\epsilon = \frac{1}{3} \cdot 10^{-4}\gamma^2$  and  $\alpha = 1 - \frac{\gamma}{100}$ . This will conclude the
proof as we could use an algorithm for our problem with approximation factor less or equal to  $\alpha$  to distinguish between the two cases of the original problem.
Completeness. Suppose there exists a vertex cover  $V^{*}$  of size  $k$ .
Consider the signaling scheme  $\phi^{*}$  such that  $\phi^{*}(a_{0,2}|s_{0},\theta_{0}) = 1$ ,  $\phi^{*}(a_{1}|s_{1},\theta_{1}) = 1$ ,  $\phi^{*}(a_{2}|s_{2},\theta_{2}) = 1$ , and  $\phi^{*}(a_{3}|s_{3},\theta_{3}) = 1$ . Moreover, by the definition of vertex cover, for each  $e \in E$  there exists a vertex  $v_{e} \in V^{*}$  such that  $v_{e} \in e = (v_{e},u)$ . We set  $\phi^{*}(a_{e,v_{e}}|s_{e},\theta_{e}) = 1$ . Finally, for each  $v \in V^{*}$ , we set  $\phi^{*}(a_{v,1}|s_{v},\theta_{v,1}) = 1$ ,  $\phi^{*}(a_{v,2}|s_{v},\theta_{v,2}) = 1$ , and  $\phi^{*}(a_{v,3}|s_{v},\theta_{v,1}) = \phi^{*}(a_{v,3}|s_{v},\theta_{v,2}) = 1$  for all  $v \notin V^{*}$ .
First, we show that the signaling scheme is persuasive. Consider the state  $s_0$ . The receiver following the recommendations will always reach a state  $v \in V^*$  and in this state will get utility 1. It is easy to see that playing action  $a_{0,1}$  the utility is 1. Hence, in state  $s_0$  the receiver will not deviate. In states  $s_1, s_2$ , and  $s_3$ , there is only one action available, hence the signaling scheme is trivially persuasive in these states. In a state  $s_e$ ,  $e \in E$ , the receiver has no incentive to deviate. Indeed, following the recommendation they will transition to a state  $s_v$ ,  $v \in V^*$ , and will get 1, while not following the recommendation they can get at most 1. In all the states  $s_v$ , for  $v \in V^*$ , the receiver gets 1, while deviating can get at most 1. Finally, in a state  $s_v$ ,  $v \notin V^*$ , the receiver gets  $\frac{1}{2}$ , while deviating can get at most  $\frac{1}{2}$ . Hence, the signaling scheme is persuasive.
Moreover, the sender's expected rewards for playing according to  $\phi^{*}$  is 1 if the initial state is  $s_0$ , as in this state the signaling scheme recommends  $a_{0,2}$ . On the other hand, if the initial state is  $s_1$ , then a state  $s_v$  with  $v \in V^*$  is reached with probability  $k / |V|$  and a state  $s_v$  with  $v \notin V^*$  is reached with probability  $1 - k / |V|$ . Then, it is easy to see that the sender's expected rewards in reaching a state  $s_v$  with  $v \in V^*$  is 0 and  $\frac{1}{2}$  otherwise. Thus the total sender's expected rewards of the signaling scheme  $\phi^{*}$  is:
$$
\frac {1}{2} + \frac {1}{4} \left(1 - \frac {k}{| V |}\right) = \frac {3}{4} - \frac {k}{4 | V |}.
$$
Soundness. Consider a signaling scheme  $\phi$ , and suppose that all the vertex covers have size at least  $(1 + \gamma)k$ . We show that the sender's expected rewards is strictly less than  $\alpha\left(\frac{3}{4} - \frac{k}{4|V|}\right)$ . First, notice that if the sender's expected rewards is at least  $\alpha\left(\frac{3}{4} - \frac{k}{4|V|}\right)$  it has to recommend  $a_{0,2}$  with probability at least  $\ell := 3/10$ , i.e.,  $\phi(a_{0,2}|s_0, \theta_0) \geq \frac{3}{10}$ . Indeed, if this is not the case the sender's expected rewards is at most
$$
\frac {1}{2} \frac {3}{1 0} \left(1 + \frac {1}{2}\right) + \frac {1}{2} \frac {1}{2} = \frac {1 9}{5 0} <   \frac {\alpha}{2} \leq \alpha \left(\frac {3}{4} - \frac {k}{4 | V |}\right),
$$
as it gives at most  $1 + \frac{1}{2}$  from  $s_0$  when  $a_{0,2}$  is played (which happens with probability less than  $3/10$ ) and  $\frac{1}{2}$  from  $s_1$ . Thus we can only consider the cases in which  $\phi(a_{0,2}|s_0,\theta_0) \geq \frac{3}{10}$ .
Then, suppose that for  $n$  states  $s_e$ ,  $e \in E$ , the receiver's expected rewards following the recommendations from there on is at most  $1 - \delta$ , where  $\delta = \gamma / 100$ . Denote with  $\widehat{E} \subseteq E$  be this set, where  $|\widehat{E}| = n$ , and with  $\widetilde{E} \coloneqq E \setminus \widehat{E}$ . The  $\epsilon$ -persuasiveness constraint in state  $s_0$  when the recommended action is  $a_{0,2}$  implies:
$$
\phi (a _ {0, 2} | s _ {0}, \theta_ {0}) \bigg [ \frac {n}{| E |} (1 - \delta) + \bigg (1 - \frac {n}{| E |} \bigg) \bigg ] \geq \phi (a _ {0, 2} | s _ {0}, \theta_ {0}) r ^ {\mathsf {S}} (a _ {0, 1}, s _ {0}, \theta_ {0}) - \epsilon ,
$$
as the left hand side is an upper bound on the receiver's expected rewards starting from  $s_0$ . The equation above implies that,
$$
\frac {n}{| E |} \left(1 - \delta\right) + \left(1 - \frac {n}{| E |}\right) \geq 1 - \frac {\epsilon}{\ell},
$$
and by rearranging it:
$$
n \leq \frac {| E | \epsilon}{\ell \delta} = \frac {\frac {1}{3} | E | \gamma^ {2} 1 0 ^ {- 4}}{3 \gamma / 1 0 0 0} = \frac {\gamma | E |}{9 0}.
$$
Let  $\widetilde{V}$  be the set of states  $s_v$ ,  $v \in V$ , such that the receiver's expected rewards from there on is at least  $1 - \delta$ . Notice that for each state  $s_e$ , with  $e \in \widetilde{E}$ , there exists a state  $s_v$ , with  $v \in \widetilde{V}$ , such that  $v \in e$ , i.e.,  $v$  covers  $e$ . This implies that  $\widetilde{V}$  covers  $\widetilde{E}$ .
By contradiction, we show that  $|\widetilde{V}| \geq (1 + \gamma)k - n$ . Suppose otherwise. Then, the set  $\widetilde{V}$  is such that  $|\widetilde{V}| < (1 + \gamma)k - n$  and covers at least  $|\widetilde{E}|$  edges. Hence, there exists a super-set of  $\widetilde{V}$  of size strictly smaller than  $(1 + \gamma)k$  that covers all the edges, reaching a contradiction. This holds as adding a vertex  $v$  for each  $e = (v,u) \in \widehat{E}$  to  $\widetilde{V}$ , we obtain a vertex cover of  $E$ .
Then, we show that in all the states  $s_v$ ,  $v \in \widetilde{V}$  the sender's expected rewards from there on is at most  $\delta$ . It is easy to see that action  $a_3$  is recommended with probability at most  $2 / \delta$  in each state  $s_v$ ,  $v \in \widetilde{V}$  and the sender collects  $\frac{1}{2}$ . If this is not the case, the receiver's expected rewards is strictly smaller than  $2\delta \frac{1}{2} + (1 - 2\delta)1 = 1 - \delta$ . This implies that the sender's expected rewards from a state  $s_v$ ,  $v \in \widetilde{V}$  is at most  $\delta$ . A similar argument shows that the sender's expected rewards from a state  $s_e$ ,  $e \in \widetilde{E}$ , is at most  $\delta$ .