SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Since  $\frac{M_a}{n + M_a} \leq \frac{M_a}{n}$ , by Lemma E.3, this can be achieved with  $\eta \geq O(M_a / \sqrt{N_0})$ , which proves part (a).
For part (b), set  $\eta$  to be large enough such that  $|\widehat{\mu}_{a,n}^{\mathrm{tape}} - \rho_{a,n}| \leq \frac{1}{2} \cdot \sqrt{\frac{\eta}{n}}$ . Given, Lemmas E.3 and E.4, this can be achieved with  $\eta \geq O(M_a / \sqrt{N_0})$ . Since  $M - 1 \geq n$ , we can further guarantee  $\frac{\ln(2 / \zeta)}{M - 1} \leq \frac{\eta}{4n}$  by setting  $\eta \geq O(\ln(1 / \zeta))$ , which finishes the proof together with Corollary E.2.
F The corollary from Section 6
As a corollary of Theorem 6.1, we derive a 0-sampling failure, leading to  $\Omega (T)$  Bayesian regret. Specifically, the agents never try arm 2 when it is in fact the best arm. This happens whenever the prior is independent across arms and has a positive density on the entire [0, 1] interval.
Corollary F.1. In the setting of Theorem 6.1, suppose the prior  $\mathcal{P}$  is independent across arms and has a positive density for each arm (i.e., has probability density function that is strictly positive on  $[0,1]$ ). Then  $\mathbb{E}[\mathrm{Regret}(T)]\geq c_{\mathcal{P}}\cdot T$ , where the constant  $c_{\mathcal{P}} > 0$  depends only on the prior  $\mathcal{P}$ .
This follows from a more explicit, but more cumbersome corollary.
Corollary F.2. Denote  $\mu_1^0 = \mathbb{E}[\mu_1]$  and  $\mu_2^0 = \mathbb{E}[\mu_2]$ . Consider independent priors such that  $\operatorname*{Pr}\left[\mu_1 = 1\right] < \left(\mu_1^0 - \mu_2^0\right)/2$ . Pick any  $\alpha > 0$  such that  $\operatorname*{Pr}\left[\mu_1 \geq 1 - 2\alpha\right] \leq \left(\mu_1^0 - \mu_2^0\right)/2$ . Then Bayesian regret is at least  $T \cdot \left(\frac{\alpha}{2}\left(\mu_1^0 - \mu_2^0\right)\operatorname*{Pr}\left[\mu_2 > 1 - \alpha\right]\right)$ .
Proof. Let  $\mathcal{E}_1$  be the event that  $\mu_1 < 1 - 2\alpha$  and arm 2 is never chosen. By Theorem 6.1 and the definition of  $\alpha$ , we have  $\operatorname*{Pr}\left[\mathcal{E}_1\right] \geq \left(\mu_1^0 - \mu_2^0\right) / 2$ .
Let  $\mathcal{E}_2$  be the event that  $\mu_2 > 1 - \alpha$ . Under event  $\mathcal{E}_1 \cap \mathcal{E}_2$ , each round contributes  $\mu_2 - \mu_1 \geq \alpha$  to regret, so  $\mathbb{E}[R(T) \mid \mathcal{E}_1 \cap \mathcal{E}_2] \geq \alpha T$ .
Since event  $\mathcal{E}_1$  is determined by the prior on arm 1 and the rewards of arm 2, it is independent from  $\mathcal{E}_2$ . It follows that
$$
\begin{array}{l} \mathbb {E} [ R (T) ] \geq \mathbb {E} [ R (T) \mid \mathcal {E} _ {1} \cap \mathcal {E} _ {2} ] \cdot \Pr [ \mathcal {E} _ {1} \cap \mathcal {E} _ {2} ] \\ \geq \alpha T \cdot \left(\mu_ {1} ^ {0} - \mu_ {2} ^ {0}\right) / 2 \cdot \Pr \left[ \mathcal {E} _ {2} \right]. \\ \end{array}
$$