SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. The event  $I_{i}^{b_{k}}$  requires  $|\hat{p}_{i,b_{k}} - \hat{p}_{i,b_{k-1}}| \geq \frac{1}{\sqrt{\log N}}$ . Hence by a standard Chernoff estimate, regardless of the true reward probability  $p_{i}$ ,
$$
\mathbb {E} \left[ I _ {i} ^ {b _ {k}} \right] \leq e ^ {- \Omega_ {\alpha , \varrho , \varrho_ {1}} \left(b _ {k} / \log N\right)}.
$$
Since by construction  $b_0 \geq \varrho_1 \log^2 N$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \left(e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} - 1\right) \cdot I _ {i} ^ {b _ {k}} \right] \leq e ^ {b _ {k} \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N} - \Omega_ {\alpha , \varrho , \varrho_ {1}} (b _ {k} / \log N)} \\ <   e ^ {- \Omega_ {\alpha , \varrho , \varrho_ {1}} (\log N)} \\ = N ^ {- \Omega_ {\alpha , \varrho , \varrho_ {1}} (1)}. \\ \end{array}
$$
Since  $k_0 \leq O(\log N)$ , summing gives the desired conclusion.
Propositions C.2 and C.3 imply that the total contribution from rejections in the small and medium sample phases is at most  $c(\alpha, \varrho) / 2$ . It remains to analyze the large sample phase in the following subsection.
C.4 Analysis of Algorithm 3 in the Large Sample Phase
Similarly to the previous section, the main part of the analysis concerns the large sample phases  $b_{k_0 + j}$  for  $j \geq 1$ . Our goal is to precisely estimate the rejection probability at each time  $b_{k_0 + j}$ . Note that these estimates should not depend on the true average rewards  $p_i$ .
Our approach is based on exchangeability and avoids any consideration of  $p_i$ . For a given value  $j$  and a large constant  $L = L(\varrho)$ , consider the sequence of times
$$
b _ {k _ {0} + j - L}, b _ {k _ {0} + j - L + 1}, \dots , b _ {k _ {0} + j}
$$
and the associated sequence of empirical average rewards
$$
\hat {p} _ {i, b _ {k _ {0} + j - L}}, \hat {p} _ {i, b _ {k _ {0} + j - L + 1}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j}}. \tag {C.6}
$$
It follows from the algorithm description that for  $I_{i}^{b_{k_{0} + j}}$  to occur, we must have
$$
\hat {p} _ {i, b _ {k _ {0} + j}} - \hat {p} _ {i, b _ {k _ {0} + j - \ell}} \geq \ell \cdot \frac {d _ {F} (\alpha , \beta) \varrho (1 - \varrho_ {2})}{\log N}, \quad \forall 1 \leq \ell \leq L. \tag {C.7}
$$
This is clear for  $j > L$ , but it holds also for  $0 \leq j \leq L$  as for  $N$  sufficiently large,
$$
\alpha - \varrho - \frac {k _ {0}}{\sqrt {\log N}} - L \cdot \frac {d _ {F} (\alpha , \beta) \varrho (1 - \varrho_ {2})}{\log N} \geq \alpha - 2 \varrho .
$$
By exchangeability, conditioned on the future values  $\hat{p}_{i,b_{k_0} + j},\dots,\hat{p}_{i,b_{k_0} + j - \ell}$  the law of  $\hat{p}_i,b_{k_0 + j - \ell -1}$  depends only on  $\hat{p}_i,b_{k_0 + j - \ell}$  and is given explicitly by a hypergeometric variable. Recalling that
$R_{i,t} = n_{i,t}\hat{p}_{i,t}$  is the total reward from the first  $n_{i,t}$  samples of arm  $i$ ,  $R_{i,b_{k_0 + j - \ell -1}}$  has hypergeometric conditional law given by:
$$
\begin{array}{l} \mathbb {P} \Big [ R _ {i, b _ {k _ {0} + j - \ell - 1}} = k \mid (\hat {p} _ {i, b _ {k _ {0} + j}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j - \ell}}) \Big ] = \mathbb {P} \Big [ R _ {i, b _ {k _ {0} + j - \ell - 1}} = k \mid \hat {p} _ {i, b _ {k _ {0} + j - \ell}} \Big ] \\ = \frac {\binom {b _ {k _ {0} + j - \ell - 1}} {k} \binom {b _ {k _ {0} + j - \ell} - b _ {k _ {0} + j - \ell - 1}} {R _ {k _ {0} + j - \ell} - k}}{\binom {b _ {k _ {0} + j - \ell}} {R _ {k _ {0} + j - \ell}}}. \tag{C.8} \\ \end{array}
$$
We will refer to this as the HyperGeom  $\left(b_{k_0 + j - \ell}, b_{k_0 + j - \ell - 1}, R_{k_0 + j - \ell}\right)$  distribution. Importantly, this distribution is independent of  $\mu$ . We exploit this below to control the probability of a given sequence  $\left(\hat{p}_i, b_{k_0 + j - L}, \hat{p}_i, b_{k_0 + j - L + 1}, \dots, \hat{p}_i, b_{k_0 + j}\right)$  of empirical average rewards. The following useful result states that hypergeometric variables automatically inherit tail bounds from the corresponding binomial random variables.
Lemma 1 ([LP14, Hoe94]). Fix non-negative integers  $A \geq B, C$  and let  $X \sim \text{HyperGeom}(A, B, C)$  and  $Y \sim \text{Bin}(B, C / A)$ . Then for any convex function  $f: \mathbb{R} \to \mathbb{R}$ ,
$$
\mathbb {E} [ f (X) ] \leq \mathbb {E} [ f (Y) ].
$$
Lemma 2. For any  $0 < \underline{q} < \overline{q} < 1$  and constants  $\varrho > 0$  there exists  $\Delta_0(\underline{q},\overline{q},\varrho)$  and  $N_{0}(\underline{q},\overline{q},\varrho)$  such that the following holds for all  $p \in [\underline{q},\overline{q}]$ . For  $n \geq n_0$  sufficiently large and  $\frac{1}{\Delta_0\sqrt{n}} \leq \Delta \leq \Delta_0$ ,
$$
\mathbb {P} \left[ \frac {\mathrm {H y p e r G e o m} (n (1 + \varrho) , n , n p (1 + \varrho))}{n} \leq p - \Delta \right] \leq e ^ {\left(- \frac {\Delta^ {2}}{2 p (1 - p)} + \varrho\right) n}.
$$
Proof. The corresponding binomial result Lemma 4 is proved in Theorem 2.2 in [DA92] by upper bounding an exponential moment. The same proof applies here by Lemma 1.  $\square$
It will be convenient to define a restricted set of good sequences  $(q_{L}, q_{L-1}, \ldots, q_{0})$ . These satisfy the key properties of empirical average reward sequences (C.6) for which  $I_{i}^{b_{k_{0}+j}}$  holds. We say such a length  $L+1$  sequence is good if the following conditions are satisfied:
1.  $q_0 \in [q, \overline{q}] \subseteq (0, 1)$  for constants  $0 < q < \overline{q} < 1$  depending only on  $\varrho, L$ .
$$
\max  _ {\ell_ {1}, \ell_ {2}} \left| q _ {\ell_ {1}} - q _ {\ell_ {2}} \right| \leq O \left(1 / \sqrt {\log N}\right). \tag {C.9}
$$
3. For each  $1 \leq \ell \leq L$ :
$$
\begin{array}{l} \theta (q _ {0}) \leq \theta (\alpha - 2 \varrho) - j \cdot \frac {d _ {F} (\alpha , \beta) \varrho (1 - \varrho_ {2})}{\log N} \\ \leq \theta (\alpha - 2 \varrho) - (j - \ell) \cdot \frac {d _ {F} (\alpha , \beta) \varrho (1 - \varrho_ {2})}{\log N} \\ \leq \theta (q _ {\ell}). \\ \end{array}
$$
The third condition above is necessary for  $I_{i}^{b_{k_{0} + j},i} = 1$ , and these together imply the first condition. Indeed for fixed  $\underline{q},\overline{q}$  and small  $\varrho \in (0,1 / 10)$  one always has
$$
\frac {\hat {p} _ {i , b _ {k _ {0} + j - 1}}}{\hat {p} _ {i , b _ {k _ {0} + j}}}, \frac {1 - \hat {p} _ {i , b _ {k _ {0} + j - 1}}}{1 - \hat {p} _ {i , b _ {k _ {0} + j}}} \in [ 1 - 2 \varrho , (1 - 2 \varrho) ^ {- 1} ]
$$
for large enough  $N$  and any  $j$ . Hence it suffices to take  $\underline{q} = \beta (1 - 2\varrho)^L$  and  $\overline{q} = 1 - (1 - \alpha)(1 - 2\varrho)^L$ . With this choice, if
$$
\hat {p} _ {i, b _ {k _ {0} + j - L}}, \hat {p} _ {i, b _ {k _ {0} + j - L + 1}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j}}.
$$
is not good and  $I_{i}^{b_{k_{0} + j}} = 1$ , then the second condition must be the only violated one. The following easy lemma controls the failure probability of the second condition. Recall from (C.8) that conditioning on  $\hat{p}_{i,b_{k_{0} + j}}$  determines the joint conditional law of the previous conditional rewards, regardless of  $\mu$ .
Lemma 3. All sequences violating only the second condition (C.9) above have probability at most
$$
e ^ {- \Omega_ {L, \varrho} (b _ {k _ {0} + j} / \log N)},
$$
even after conditioning on an arbitrary value for  $\hat{p}_{i,b_{k_0 + j}}$ .
Proof. The claim follows by an elementary Chernoff estimate for hypergeometric variables, which hold just as for binomial variables by Lemma 1. Indeed the assumption implies that some adjacent difference  $|\hat{p}_{i,b_{k_0 + j - \ell}} - \hat{p}_{i,b_{k_0 + j - \ell +1}}|$  has size  $\Omega (1 / \sqrt{\log N})$ . (Note for applying the Chernoff bound that  $L$  is a constant independent of  $N$ , and so  $b_{k_0 + j - L} \geq \Omega_{L,\varrho}(b_{k_0 + j})$ .)
We now focus on upper-bounding the probability of any good sequence  $(q_{L},\ldots ,q_{0})$  appearing, conditionally on  $q_{0}$
Lemma 4. For any good sequence  $(q_{L}, q_{L-1}, \ldots, q_{0})$  and  $j \geq 0$ ,
$$
\begin{array}{l} \mathbb {P} \Big [ \left(\hat {p} _ {i, b _ {k _ {0} + j - L}}, \hat {p} _ {i, b _ {k _ {0} + j - L + 1}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j}}\right) = \left(q _ {L}, q _ {L - 1}, \dots , q _ {0}\right) \mid p _ {i, b _ {k _ {0} + j}} = q _ {0} \Big ] \\ \leq \exp \left(- \frac {(1 - O (\varrho))}{2 q _ {0} (1 - q _ {0}) \varrho} \sum_ {\ell = 0} ^ {L - 1} b _ {k _ {0} + j - \ell} (q _ {\ell} - q _ {\ell + 1}) ^ {2}\right). \\ \end{array}
$$
Proof. It suffices to show that
$$
\mathbb {P} \left[ \hat {p} _ {i, b _ {k _ {0} + j - \ell - 1}} = q _ {\ell + 1} \mid q _ {\ell} \right] \leq \exp \left(- \frac {(1 - O (\varrho))}{2 q _ {0} (1 - q _ {0}) \varrho} b _ {k _ {0} + j - \ell} (q _ {\ell} - q _ {\ell + 1}) ^ {2}\right)
$$
This follows by applying Lemma 2 to the hypergeometric random variable
$$
\hat {p} _ {i, b _ {k _ {0} + j - \ell}} \cdot b _ {k _ {0} + j - \ell} - \hat {p} _ {i, b _ {k _ {0} + j - \ell - 1}} \cdot b _ {k _ {0} + j - \ell - 1} = R _ {b _ {k _ {0} + j - \ell}} - R _ {b _ {k _ {0} + j - \ell - 1}}.
$$
$$
b _ {k _ {0} + j - \ell + 1} - b _ {k _ {0} + j - \ell} = \varrho \cdot b _ {k _ {0} + j - \ell} \pm O (1)
$$
leads to the factor of  $\underline{\varrho}$  in the denominator of the desired result.
Lemma 5. For fixed problem parameters and  $N$  large, any good sequence  $(q_{L},\ldots ,q_{0})$  satisfies
$$
q _ {\ell} \geq q _ {0} + \frac {\ell \cdot d _ {F} (\alpha , \beta) \varrho (1 - 2 \varrho_ {2}) \cdot \sqrt {q _ {0} (1 - q _ {0})}}{(\log N)}
$$
Proof. Recall that  $\theta'(q) = \frac{1}{\sqrt{q(1 - q)}}$  and that  $\theta$  is smooth on  $[\underline{q}, \overline{q}] \subseteq (0, 1)$ . By Item 2 above, all  $q_{\ell}$  are within  $o_N(1)$  of each other, so the result follows from the inverse function theorem. (Notice that the factor  $(1 - \varrho_2)$  changed to  $(1 - 2\varrho_2)$  above.)
Lemma 6. For  $1 \leq m \leq L$  and any good sequence  $(q_{L},\ldots ,q_{0})$ , we have
$$
\sum_ {\ell = 0} ^ {m - 1} (q _ {\ell} - q _ {\ell + 1}) ^ {2} \geq \frac {m \cdot d _ {F} (\alpha , \beta) ^ {2} \varrho^ {2} (1 - 4 \varrho_ {2}) \cdot q _ {0} (1 - q _ {0})}{\log^ {2} N}.
$$
Proof. The result follows from Lemma 5 and Cauchy-Schwarz in the form
$$
\sum_ {\ell = 0} ^ {m - 1} \left(q _ {\ell} - q _ {\ell + 1}\right) ^ {2} \geq m ^ {- 1} \left(\sum_ {\ell = 0} ^ {m - 1} \left| q _ {\ell} - q _ {\ell + 1} \right|\right) ^ {2}.
$$
Lemma 7. For any good sequence  $(q_{L},\ldots ,q_{0})$  and  $j\geq 0$  we have
$$
\sum_ {\ell = 0} ^ {L - 1} b _ {k _ {0} + j - \ell} (q _ {\ell} - q _ {\ell + 1}) ^ {2} \geq (1 - O (\varrho_ {2})) \cdot \frac {b _ {k _ {0} + j} \varrho d _ {F} (\alpha , \beta) ^ {2} \cdot q _ {0} (1 - q _ {0})}{\log^ {2} N}.
$$
Proof. We break the sum into parts and apply Lemma 6 to each one. We have:
$$
\begin{array}{l} \sum_ {\ell = 0} ^ {L - 1} b _ {k _ {0} + j - \ell} (q _ {\ell} - q _ {\ell + 1}) ^ {2} = b _ {k _ {0} + j - L + 1} \sum_ {\ell = 0} ^ {L - 1} (q _ {\ell} - q _ {\ell + 1}) ^ {2} + \sum_ {m = 1} ^ {L - 1} (b _ {k _ {0} + j - m + 1} - b _ {k _ {0} + j - m}) \sum_ {\ell = 0} ^ {m - 1} (q _ {\ell} - q _ {\ell + 1}) ^ {2} \\ \geq \sum_ {m = 1} ^ {L - 1} b _ {k _ {0} + j} \cdot \frac {\varrho}{(1 + \varrho) ^ {m + 1 0}} \cdot (1 - 4 \varrho_ {2}) \frac {m \varrho^ {2} d _ {F} (\alpha , \beta) ^ {2} \cdot q _ {0} (1 - q _ {0})}{\log^ {2} N} \\ \geq \left(1 - O (\varrho + \varrho_ {2})\right) \cdot b _ {k _ {0} + j} \cdot \frac {\varrho^ {3} d _ {F} (\alpha , \beta) ^ {2} \cdot q _ {0} (1 - q _ {0})}{\log^ {2} N} \cdot \sum_ {m = 1} ^ {L - 1} \frac {m}{(1 + \varrho) ^ {m}}. \\ \end{array}
$$
For  $L = L(\varrho) = O\bigl (\varrho^{-1}\log (\varrho^{-1})\bigr)$  sufficiently large,
$$
\begin{array}{l} \sum_ {m = 1} ^ {L - 1} \frac {m \varrho}{(1 + \varrho) ^ {m}} \geq (1 - \varrho) \sum_ {m = 1} ^ {\infty} \frac {m}{(1 + \varrho) ^ {m}}. \\ = (1 - \varrho) \left(\sum_ {m = 1} ^ {\infty} \frac {1}{(1 + \varrho) ^ {m}}\right) ^ {2} \\ = \frac {1 - \varrho}{\varrho^ {2}}. \\ \end{array}
$$
Substituting and recalling that  $\varrho \ll \varrho_{2}$  completes the proof.
Combining with Lemma 4 yields the second inequality below (the first is trivial).
Corollary C.4. For any  $\mu$  and  $q_0$ , we have
$$
\begin{array}{l} \mathbb {P} ^ {p _ {i} \sim \mu} \left[ \left(\hat {p} _ {i, b _ {k _ {0} + j - L}}, \hat {p} _ {i, b _ {k _ {0} + j - L + 1}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j}}\right) = \left(q _ {L}, q _ {L - 1}, \dots , q _ {0}\right) \right] \\ \leq \mathbb {P} \left[ \left(\hat {p} _ {i, b _ {k _ {0} + j - L}}, \hat {p} _ {i, b _ {k _ {0} + j - L + 1}}, \dots , \hat {p} _ {i, b _ {k _ {0} + j}}\right) = \left(q _ {L}, q _ {L - 1}, \dots , q _ {0}\right) \mid p _ {i, b _ {k _ {0} + j}} = q _ {0} \right] \\ \leq \exp \Big (- \big (1 - O (\varrho_ {2}) \big) \frac {b _ {k _ {0} + j} d _ {F} (\alpha , \beta) ^ {2}}{2 \log^ {2} N} \Big). \\ \end{array}
$$
Lemma 8. Let  $j_0$  be the largest  $j$  such that  $b_{k_0 + j} \leq N$ . Then for  $N$  sufficiently large,
$$
\sum_ {j = 1} ^ {j _ {0}} \mathbb {E} \left[ e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} \cdot I _ {i} ^ {b _ {k _ {0} + j}} \right] \leq c (\alpha , \varrho) / 4.
$$
Proof. Recall that  $c_{\alpha, \beta} = \frac{d_F(\alpha, \beta)^2}{2}$ , and observe that the number of total sequences  $(q_L, \ldots, q_0) \in [0,1]^{L+1}$  with  $b_{k_0 + j + \ell} q_\ell \in \mathbb{Z}$  is at most  $N^{L+1}$  for each  $j \leq j_0$ . Combining Lemma 3 and Corollary C.4 and noting that the latter always gives the main contribution, we find for each  $j \leq j_0$ ,
$$
\begin{array}{l} \mathbb {E} \left[ e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} \cdot I _ {i} ^ {b _ {k _ {0} + j}} \right] \leq N ^ {L + 1} \exp \left(\frac {b _ {k _ {0} + j}}{\log^ {2} N} \cdot \big ((c _ {\alpha , \beta} - \varrho_ {3}) - (1 - O (\varrho_ {2})) c _ {\alpha , \beta} \big)\right) \\ \leq \exp \left(- \Omega \left(\frac {\varrho_ {3} b _ {k _ {0} + j}}{\log^ {2} N}\right)\right) \\ \end{array}
$$
so long as  $\varrho_{3}$  is chosen so that  $\varrho_{3} \gg \max (\varrho, \varrho_{2})$ . In the last line we used the fact that  $b_{k_0 + j} \geq b_{k_0} \geq \log^4 N$  to absorb the factor  $N^{L + 1} \leq e^{\varrho \log^{3/2} N}$  for large  $N$ . Summing over  $j$  gives the
desired result, since for  $\varrho_4 = \Omega (\varrho_3)$  and  $N$  sufficiently large,
$$
\begin{array}{l} \sum_ {j = 1} ^ {\infty} e ^ {- \Omega \left(\frac {\varrho_ {3} b _ {k _ {0} + j}}{\log^ {2} N}\right)} \leq \sum_ {m = 1} ^ {\infty} e ^ {- \frac {\varrho_ {4} (m + b _ {k _ {0}})}{\log^ {2} N}} \\ = e ^ {- \varrho_ {4} \log^ {2} N} \sum_ {m = 1} ^ {\infty} e ^ {- \frac {\varrho_ {4} m}{\log^ {2} N}} \\ \leq e ^ {- \varrho_ {4} \log^ {2} N} \cdot O \left(\frac {\log^ {2} N}{\varrho_ {4}}\right) \\ \leq e ^ {- \frac {\varrho_ {4} \log^ {2} N}{2}} \\ \leq c (\alpha , \varrho) / 4. \\ \end{array}
$$
We now use Lemma 3 to conclude.
Proof that Algorithm 3 achieves the guarantee of Theorem 3.1. By combining Lemma 8 with the previous Propositions C.2 and C.3, it follows that
$$
\mathbb {E} \left[ e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho 3}{\log^ {2} N}} \cdot I _ {i} \right] \leq 1.
$$
Lemma 3 now implies that the total amount of time spent on eventually rejected arms is at most  $N(1 - \varrho)$  with probability
$$
e ^ {- \frac {\left(c _ {\alpha , \beta} - \varrho_ {3}\right) (1 - \varrho) N}{\log^ {2} N}}.
$$
On this event, the output arm  $i^{*}$  satisfies  $n_{i^{*},N} \geq \varrho N$  by definition. Since  $i^{*}$  was not rejected, for  $j_{1}$  be the largest value such  $b_{k_0 + j_1} \leq \varrho N$  we have
$$
\hat {p} _ {i ^ {*}, b _ {k _ {0} + j _ {1}}} \geq \beta + \varrho .
$$
The probability for this to hold if  $p_i \leq \beta$  is at most  $e^{-\Omega_{\varrho}(N)}$ . Altogether we find that
$$
\mathbb {P} \left[ p _ {i ^ {*}} \geq \beta \right] \geq 1 - \exp \left(- \frac {\left(c _ {\alpha , \beta} - \varrho_ {5}\right) N}{\log^ {2} N}\right) - e ^ {- \Omega_ {\varrho} (N)} \tag {C.10}
$$
for  $\varrho_{5}$  arbitrarily small. This concludes the analysis of Algorithm 3 (since the last error term is negligible).
C.5 Finding Many Good Arms with a Fixed Budget
In this final subsection we observe that Algorithm 3 can be modified to output as many as  $\log N$  distinct arms each of which satisfies the same  $(\eta, \varepsilon, \delta)$ -PAC guarantee<sup>2</sup>, with no degradation in the asymptotic failure probability. With other parameters fixed, we denote the  $N$ -sample version of Algorithm 3 by  $\mathcal{A}_N$  to emphasize the dependence on  $N$ . In particular,  $N$  both equals the number of steps in  $\mathcal{A}_N$  and appears (via its logarithm) in the description of  $\mathcal{A}_N$ 's individual steps.
Let  $\tilde{N} = N + \lceil \frac{2N}{\log^{1 / 2}(N)} \rceil$ . We consider a modified algorithm  $\tilde{\mathcal{A}}_{\tilde{N}}$  which mimicks the behavior of  $\mathcal{A}_N$  with two changes:
1.  $\tilde{\mathcal{A}}_N$  is a  $\tilde{N}$ -sample algorithm.
2. If an arm  $a_i$  has not yet been rejected after  $M = \lceil N / \log^{3/2}(N) \rceil$  samples, then  $\tilde{\mathcal{A}}_{\tilde{N}}$  accepts  $a_i$  and continues to  $a_{i+1}$ . In particular,  $\tilde{\mathcal{A}}_{\tilde{N}}$  may accept several arms instead of just one.
Theorem C.9. With probability  $1 - \exp \left(-\frac{(c_{\alpha,\beta} - \varrho_5 - o_N(1))N}{\log^2N}\right)$ ,  $\tilde{\mathcal{A}}_{\tilde{N}}$  accepts at least  $\log (N)$  distinct arms  $a_i$ , all of which satisfy  $p_i \geq \beta$ .
2 In fact  $\log N$  can be replaced by anything  $o_N(\log^2 N)$  by more precisely defining  $M$  and  $\tilde{N}$ .
The change from  $N$  to  $\tilde{N}$  is almost irrelevant in the actual statement of Theorem C.9 since  $\log (N)\geq \log (\tilde{N}) - o_N(1)$ . In particular,  $\tilde{\mathcal{A}}_{\tilde{N}}$  is a  $\tilde{N}$ -sample algorithm which outputs at least  $\log (\tilde{N}) - 1$  arms with probability  $1 - \exp \left(-\frac{(c_{\alpha,\beta} - \varrho_5 - o_{\tilde{N}}(1))\tilde{N}}{\log^2\tilde{N}}\right)$ . It is certainly not really necessary to use the value  $\log (N)$  rather than  $\log (\tilde{N})$  to describe the individual steps taken by  $\tilde{A}_{\tilde{N}}$ . However introducing  $\tilde{N}$  streamlines the proof below by letting us treat  $\mathcal{A}_N$  as a blackbox.
Proof. To show that all accepted arms  $a_i$  satisfy  $p_i \geq \beta$  with sufficiently high probability, it suffices to consider (C.10) with the final term replaced by  $e^{-\Omega_e(N / \log^{3/2}(N))}$ . In particular, observe that the main term does not change, even after multiplying the failure probability by  $O\big(\log^{3/2}(N)\big)$  (the maximum possible number of arms accepted by  $\tilde{\mathcal{A}}_{\tilde{N}}$ ). Thus we focus on showing that  $\tilde{\mathcal{A}}_{\tilde{N}}$  outputs at least  $\log(N)$  arms with high probability.
Consider yet another  $N$ -sample algorithm  $\widehat{\mathcal{A}}_N$  which deletes each arm independently with probability  $1/N$  and follows  $\mathcal{A}_N$  on the set of non-deleted arms in order of increasing index. (Like  $\mathcal{A}_N$ ,  $\widehat{\mathcal{A}}_N$  never accepts arms before time  $N$ .) We simulate  $\tilde{\mathcal{A}}_{\tilde{N}}$  and  $\widehat{\mathcal{A}}_N$  on the same reward sequences, i.e. we couple them so that the  $t$ -th sample of arm  $a_i$  always gives the same result for each  $(t,i)$ . We claim that in this coupling, conditioned on  $\tilde{\mathcal{A}}_{\tilde{N}}$  failing to accept  $\log(N)$  arms within the first  $\tilde{N}$  samples,  $\widehat{\mathcal{A}}_N$  has probability  $\Omega(N^{-\log(N)})$  to fail (i.e. output  $a_i$  with  $p_i < \beta$ ) when run for  $N$  samples.
First let us assume the claim and deduce Theorem C.9. Denote by  $p(N)$  the probability for  $\mathcal{A}_N$  to fail. Note that  $\widehat{\mathcal{A}}_N$  has the same failure probability  $p(N)$ , having in fact the same behavior as  $\mathcal{A}_N$  in distribution (as the set of deleted arms is independent of everything else). Moreover let  $\tilde{p}(\tilde{N}, k)$  denote the probability that  $\tilde{\mathcal{A}}_{\tilde{N}}$  fails to accept at least  $k$  arms. The claim above implies that
$$
\begin{array}{l} \tilde {p} (\tilde {N}, \log N) \leq O \left(N ^ {\log N}\right) \cdot p (N, 1) \\ \leq e ^ {o _ {N} \left(N / \log^ {2} N\right)} \cdot p (N, 1) \\ \leq \exp \left(- \frac {\left(c _ {\alpha , \beta} - \varrho_ {5} - o _ {N} (1)\right) N}{\log^ {2} N}\right). \\ \end{array}
$$
It remains to prove the above claim. Let us say the infinite i.i.d. reward sequence  $(r_{i,n})_{n\geq 1}$  of arm  $a_i$  is acceptable if  $\mathcal{A}_N$  would not reject  $a_i$  within  $M$  samples, i.e.  $\tilde{\mathcal{A}}_{\tilde{N}}$  will either accept  $a_i$  or run out of samples before doing so. We take the point of view that each  $a_i$  is either acceptable or not (by randomly fixing the reward sequences at the start). Then with probability  $\Omega (N^{-\log (N)})$ , the first  $\log (N)$  acceptable arms are skipped by  $\hat{\mathcal{A}}$ , and the first  $\hat{N}$  unacceptable arms are not skipped. On this event, the first  $\hat{N} - M\geq N$  samples obtained by  $\hat{\mathcal{A}}_N$ , i.e. all  $N$  of its samples, are drawn from unacceptable arms. On this event,  $\hat{\mathcal{A}}_N$  fails with constant probability, which establishes the claim and completes the proof.