SlowGuess/NeurIPS_2023_OCR huggingface dataset:
It is easy to see that  $B$  thus defined is  $\varrho$ -slowly increasing for any  $\varrho > 0$  and  $N$  sufficiently large. We denote  $b_{k} = \lfloor N^{6\varrho}(1 + \varrho)^{k}\rfloor$  so that  $|b_{k + 1} - (1 + \varrho)b_k|\leq 2$ . (This choice of indexing differs from that of Definition B.1, which will not be used in the sequel.)
We next construct our randomness distorting adversary  $\mathbb{A} = \mathbb{A}(N,\varrho)$ . For each arm  $i$ , the adversary  $\mathbb{A}$  acts as follows depending on the current number of samples  $n_{i,t}$ .
1. If  $n_{i,t} \leq N^{2\varrho}$ , then  $\mathbb{A}$  does nothing.
2. When  $N^{2\varrho} \leq n_{i,t} < N^{6\varrho}$  increases by  $N^{\varrho}$ ,  $\mathbb{A}$  declares that the average reward of this batch of  $N^{\varrho}$  samples is at most  $\overline{\gamma} - N^{-\varrho}$ .
3. When  $n_{i,t}$  increases from  $b_{k} \geq N^{6\varrho}$  to  $b_{k+1}$ :
(a) If  $\hat{p}_i(b_k) > \beta$  holds, then  $\mathbb{A}$  declares that
$$
\theta \left(\hat {p} _ {i} \left(b _ {k + 1}\right)\right) \leq \theta \left(\hat {p} _ {i} \left(b _ {k}\right)\right) - \frac {\varrho (1 + 1 0 \varrho) d _ {F} (\alpha , \beta)}{\log N}. \tag {B.15}
$$
(b) If  $\hat{p}_i(b_k)\leq \beta$  holds, then A declares that
$$
\hat {p} _ {i} \left(b _ {k + 1}\right) \leq \beta .
$$
4. When the  $\mathcal{A}$  chooses the arm  $a_{i^*}$  to output,  $\mathbb{A}$  declares that  $p_{i^*} < \beta$ .
Due to step 4, the declarations made by  $\mathbb{A}$  ensure that  $p_{i^*} < \beta$ . Recalling Lemma 4 and Proposition B.2, it remains to show the upper bound
$$
\operatorname {s t r e n g t h} (\mathbb {A}) \leq \frac {\left(c _ {\alpha , \beta} + C * \varrho\right) N}{\log^ {2} (N)}
$$
for a constant  $C_* = C_*(\underline{\gamma}, \overline{\gamma}, \underline{f}, \overline{f}, \beta, \alpha)$  independent of  $\varrho$  (and  $N$ ). We show this bound in several parts. Recalling (3.4), we refer to the cost of a step above as the contribution to Cost from the corresponding declarations by  $\mathbb{A}$ . The most important parts are Lemmas 10 and 11, which bound the cost of the main step 3a and form the dominant contribution to Cost. Note that throughout the analysis below, all cost upper bounds hold almost surely and we assume that all of  $\mathbb{A}$ 's declarations hold true.
Lemma 8. The total cost from step 2 is at most  $C_* N^{1 - \varrho}$ , for  $N \geq C(\underline{\gamma}, \overline{\gamma}, \underline{f}, \overline{f}, \beta, \alpha, \varrho)$  sufficiently large.
Proof. The probability for each such declaration by  $\mathbb{A}$  is at least
$$
\mathbb {P} \left[ \operatorname {B i n} \left(N ^ {2 \varrho}, \bar {\gamma}\right) \leq \bar {\gamma} N ^ {2 \varrho} - N ^ {\varrho} \right] \tag {B.16}
$$
since  $p_i \leq \overline{\gamma}$  almost surely. Recall that a  $\operatorname{Bin}(N^{2\varrho}, \overline{\gamma})$  random variable obeys a central limit theorem centered at  $\overline{\gamma} N^{2\varrho}$  with standard deviation at least  $C(\overline{\gamma}) N^{\varrho}$ . Therefore the probability in (B.16) is at least  $\frac{1}{3}$  for  $N$  is sufficiently large depending on  $\varrho$ . Hence each such declaration costs at most  $C_*$  for  $N$  sufficiently large. Moreover such declarations can occur only  $N^{1 - \varrho}$  times because each one involves  $N^{\varrho}$  samples, and the base algorithm  $\mathcal{A}$  is an  $N$ -sample algorithm. This completes the proof.
Lemma 9. The total cost from step 3b is at most  $C_* N^{1 - 6\rho}$  as long as  $N \geq C(\underline{\gamma}, \overline{\gamma}, \underline{f}, \overline{f}, \varrho)$ .
Proof. It suffices to show that the cost per step 3b declaration is at most  $C_*$ . This follows from (B.13) and stochastic monotonicity.
Lemma 10. The total cost from step 3a is at most
$$
\frac {N}{\log^ {2} (N)} \cdot \left(c _ {\alpha , \beta} + C _ {*} \varrho + o _ {N} (1)\right).
$$
Proof. We claim that the cost from a single instance of step 3a when increasing from  $b_{k}$  to  $b_{k + 1}$  samples is at most
$$
\left(\frac {\left(b _ {k + 1} - b _ {k}\right)}{\log^ {2} (N)}\right) \left(c _ {\alpha , \beta} + C _ {*} \varrho + o _ {N} (1)\right).
$$
This implies the desired result since  $\mathcal{A}_N$  is an  $N$ -sample algorithm. Taking  $\Delta = (1 + 10\varrho)d_F(\alpha ,\beta) / \log (N)$  in Lemma 7, we find that the declared event has probability at least
$$
\exp \left(- \frac {(b _ {k + 1} - b _ {k}) (1 + 1 0 \varrho) ^ {2} d _ {F} (\alpha , \beta) ^ {2} (1 + o _ {N} (1))}{2 \log^ {2} (N)}\right) \geq \exp \left(- \frac {(b _ {k + 1} - b _ {k})}{\log^ {2} (N)} \big (c _ {\alpha , \beta} + C _ {*} \varrho + o _ {N} (1) \big)\right).
$$
This implies the desired claim and completes the proof.
Lemma 11. For any  $a_i$  sampled  $b_0 = \lfloor N^{6\varrho}\rfloor$  times,  $\hat{p}_i(b_0)\leq \overline{\gamma}$
Proof. By definition of  $\mathbb{A}$
$$
\begin{array}{l} \hat {p} _ {i} (b _ {0}) \leq \frac {N ^ {2 \varrho} + (N ^ {6 \varrho} - N ^ {2 \varrho}) (\overline {{\gamma}} - N ^ {- \varrho})}{N ^ {6 \varrho}} \\ = \overline {{\gamma}} - \frac {1}{N ^ {\varrho}} + \frac {(1 - \overline {{\gamma}})}{N ^ {4 \varrho}} + \frac {1}{N ^ {5 \varrho}} \\ \leq \overline {{\gamma}}. \\ \end{array}
$$
In the last step we used the fact that
$$
\frac {1}{N ^ {\varrho}} \geq \frac {(1 - \bar {\gamma})}{N ^ {4 \varrho}} + \frac {1}{N ^ {5 \varrho}}
$$
for any  $\varrho > 0$  if  $N$  is sufficiently large.
Lemma 12. For  $\varrho \in (0,1 / 100)$ , if  $n_{i,t} \geq N^{1 - \varrho}$  and the declarations of  $\mathbb{A}$  hold, then  $\hat{p}_{i,t} \leq \beta$ .
Proof. We analyze the rate at which the adversary forces  $\theta (\hat{p}_i(b_k))$  to decrease. From (B.15) and (11) it follows that for  $k$  with  $b_{k}\geq N^{1 - \varrho}$ , we have
$$
\begin{array}{l} \theta (\hat {p} _ {i} (b _ {k})) \leq \theta (\bar {\gamma}) - \frac {\varrho (1 + 1 0 \varrho) d _ {F} (\alpha , \beta) \log_ {1 + \varrho} (N ^ {1 - 8 \varrho})}{\log N} \\ = \theta (\overline {{\gamma}}) - \frac {\varrho (1 + 1 0 \varrho) (1 - 8 \varrho) d _ {F} (\alpha , \beta)}{\log (1 + \varrho)} \\ \leq \theta (\bar {\gamma}) - (1 + \varrho) d _ {F} (\alpha , \beta) \\ \stackrel {(B. 8)} {<  } \theta (\beta). \\ \end{array}
$$
Here we used the fact that  $\log (1 + \varrho) \leq \varrho$  and  $(1 + 10\varrho)(1 - 8\varrho) \geq 1$  for  $\varrho \in (0,1 / 100)$ . Since  $\theta$  is increasing, this shows that  $\hat{p}_{i,t} = \hat{p}_i(b_k) < \beta$  for  $b_{k} \geq N^{1 - \varrho}$ , completing the proof.
Lemma 13. The cost from step 4 is at most  $C_* \left( N^{1 - \varrho} + 1 \right)$ .
Proof. First, if  $\hat{p}_{i^*,N} \leq \beta$  then the cost from step 4 is at most  $C_*$ . On the other hand if  $\hat{p}_{i^*,N} > \beta$ , then Lemma 11 implies  $n_{i^*,N} \leq N^{1 - \varrho}$ . Since the prior  $\mu$  is supported in  $[\underline{\gamma},\overline{\gamma}]$ , the likelihood ratio of updates from  $N^{1 - \varrho}$  samples is almost surely bounded by  $e^{C_*N^{1 - \varrho}}$ . Therefore
$$
\begin{array}{l} \mathbb {P} ^ {x \sim \mu_ {i, N}} [ x <   \beta ] \geq e ^ {- C _ {*} N ^ {1 - \varrho}} \mathbb {P} ^ {x \sim \mu} [ x <   \beta ] \\ \geq e ^ {- C _ {*} N ^ {1 - \varrho}} \frac {(\beta - \underline {{\gamma}}) \underline {{f}}}{\bar {f}}. \\ \end{array}
$$
We now combine the lemmas above to conclude Theorem 3.1 via (B.7).
Proof of Theorem 3.1. Let  $C_*'$  be a larger constant depending on the same parameters. Then by Lemmas 8, 9, and 13, the total cost from Steps 2, 3b, 4 combines to  $C_*'(N^{1 - \varrho}) \leq o_N(N / \log^2 N)$ . The main cost contribution of
$$
\frac {N}{\log^ {2} N} \left(c _ {\alpha , \beta} + C _ {*} \varrho + o _ {N} (1)\right).
$$
comes from Lemma 10, and all other terms are of strictly smaller order. We have thus constructed a reservoir sequence  $(\mu_N(\varrho))_{N\geq 1}$  satisfying (B.7) for arbitrary  $\varrho >0$  , completing the proof.
C An Optimal Algorithm with Fixed Budget
Here we provide an asymptotically optimal algorithm which establishes Theorems B.1, B.2, and B.3. In the next subsection in which we show how to reduce the other results mentioned to Theorem 3.1 (in which  $\alpha$  is given) using Proposition A.1. Our main focus will then be to prove Theorem 3.1.
We will fix  $\varrho > 0$  small and construct a sequence of  $N$ -sample algorithms  $(\mathcal{A}(N, \varrho))$  satisfying the slightly relaxed guarantee
$$
\lim  _ {N \rightarrow \infty} \frac {\left(- \log \left(\mathbb {P} ^ {\mu_ {N} (\varrho)} \left[ p _ {i ^ {*}} <   \beta \right]\right)\right) \cdot \log^ {2} N}{N} \geq c _ {\alpha , \beta} - \lambda (\varrho) \tag {C.1}
$$
for a (possibly different) function  $\lambda$  satisfying  $\lim_{\varrho \to 0} \lambda(\varrho) = 0$  (for fixed  $\alpha, \beta, \eta$ ). Here  $(\mu_N)_{N \geq 1}$  is any sequence of reservoir distributions satisfying  $G_{\mu_N}^{-1}(1 - \eta) = \alpha$ . An elementary diagonalization argument then implies Theorem 3.1. Thus it suffices to construct algorithms satisfying (C.1) for any desired  $\varrho > 0$ .
C.1 Reduction to Known  $\alpha$
We explain why Theorems B.1, B.2, and B.3 all follow from Theorem 3.1 (more precisely, the uniform statement given in Remark B.1). We begin with Theorem B.1, where
$$
\alpha_ {N} = \frac {1}{\eta_ {1} - \eta_ {2}} \cdot \int_ {1 - \eta_ {1}} ^ {1 - \eta_ {2}} G _ {\mu_ {N}} ^ {- 1} (x) d x.
$$
Let  $J = \left\lceil \frac{6}{\varepsilon(\eta_1 - \eta_2)}\right\rceil$  and define
$$
\eta^ {(j)} = \frac {(J - j) \eta_ {1} + j \eta_ {2}}{J}, \quad j \in [ J ].
$$
It is easy to see that  $\eta^{(j + 1)} - \eta^{(j)}\leq \eta^{(j)}$  for all  $j$ . We next apply Alg. 4 on  $(\eta^{(j)},\eta^{(j + 1)} - \eta^{(j)},\varepsilon^{\prime},\delta^{\prime})$  for  $0\leq j\leq J - 1$ , with:
$$
\varepsilon^ {\prime} = \log^ {- 1 / 3} (N),
$$
$$
\delta^ {\prime} = e ^ {- \frac {1 0 N}{\log^ {2} (N)}} / J.
$$
This requires sample complexity
$$
N _ {A} \leq \frac {C \left(\eta_ {1} , \eta_ {2}\right) N \log \log (N)}{\log (N)} \leq o _ {N} (N). \tag {C.2}
$$
Let  $\hat{\alpha}_j$  be the resulting output. With probability  $1 - J\delta$ , we have for each  $0 \leq j \leq J - 1$ ,
$$
\hat {\alpha} _ {j} \in \left[ G ^ {- 1} \left(1 - \eta^ {(j)}\right) - \frac {\varepsilon}{3}, G ^ {- 1} \left(1 - \eta^ {(j + 1)}\right) + \frac {\varepsilon}{3} \right]. \tag {C.3}
$$
Note that the function  $G_{\mu}^{-1}$  is increasing and [0, 1]-valued. Therefore if (C.3) holds for each  $j$ , then
$$
\left| \frac {1}{J} \cdot \sum_ {j = 0} ^ {J - 1} \hat {\alpha} _ {j} - \frac {1}{\eta_ {1} - \eta_ {2}} \cdot \int_ {1 - \eta_ {1}} ^ {1 - \eta_ {2}} G _ {\mu_ {N}} ^ {- 1} (x) d x \right| \leq \frac {\varepsilon}{3} + \frac {1}{J} \leq \frac {\varepsilon}{2}.
$$
Therefore the estimator
$$
\hat {\alpha} _ {A} = \frac {1}{J} \cdot \sum_ {j = 0} ^ {J - 1} \hat {\alpha} _ {j}
$$
$$
\mathbb {P} \left[ \left| \hat {\alpha} _ {A} - \frac {1}{\eta_ {1} - \eta_ {2}} \cdot \int_ {1 - \eta_ {1}} ^ {1 - \eta_ {2}} G _ {\mu_ {N}} ^ {- 1} (x) d x \right| \leq \varepsilon / 2 \right] \geq 1 - J \delta^ {\prime} = 1 - e ^ {- \frac {1 0 N}{\log^ {2} (N)}}.
$$
Finally,  $c_{\alpha,\alpha-\varepsilon} \leq \pi < 10$  for any  $\alpha,\varepsilon \in [0,1]$  (see (B.10)). Therefore the  $\delta' = e^{-\frac{10N}{\log^2(N)}}$  failure probability above has a negligible contribution in Theorem B.1. It follows that applying Theorem 3.1 with  $\alpha = \hat{\alpha}_A$  as above and  $N' = N - N_A$  implies Theorem B.1.
We now turn to Theorem B.2, where  $\mu_N$  is required to satisfy  $G_{\mu_N}^{-1}(1 - \eta) \geq \frac{1 + \varepsilon}{2}$ . We run Alg. 4 with parameters
$$
\eta_ {1} = \eta ,
$$
$$
\eta_ {2} = \log^ {- 1 / 3} (N),
$$
$$
\delta^ {\prime} = e ^ {- \frac {1 0 N}{\log^ {2} (N)}}.
$$
The sample complexity  $N_B$  again satisfies  $N_B \leq o(N)$  exactly as in (C.2). Let  $\hat{\alpha}_B + \varepsilon'$  be the resulting output. Then with probability at least  $1 - e^{-\frac{10N}{\log^2(N)}}$ ,
$$
\hat {\alpha} _ {B} \geq G _ {\mu_ {N}} ^ {- 1} (1 - \eta) - 2 \varepsilon^ {\prime}
$$
and so with  $\varepsilon'' = \varepsilon - 2\varepsilon'$ , we have
$$
\hat {\alpha} _ {B} - \varepsilon^ {\prime \prime} \geq G _ {\mu_ {N}} ^ {- 1} (1 - \eta) - \varepsilon .
$$
Moreover, also with probability at least  $1 - e^{-\frac{10N}{\log^2(N)}}$ ,
$$
\hat {\alpha} _ {B} \leq G _ {\mu_ {N}} ^ {- 1} (1 - \eta + \eta_ {2}).
$$
It follows that applying the algorithm of Theorem 3.1 with
$$
(N, \alpha , \eta , \varepsilon) = (N - N _ {B}, \hat {\alpha} _ {B}, \eta - \eta_ {2}, \varepsilon - 2 \varepsilon^ {\prime})
$$
suffices to recover Theorem B.2, since  $\eta_{2}$  and  $\varepsilon^{\prime}$  tend to 0 as  $N\to \infty$ . As in our discussion of Theorem B.1 above, the failure probability  $e^{-\frac{10N}{\log^2(N)}}$  is negligible compared to the relevant rate in Theorem B.2.
Finally, Theorem B.3 relies on the simple fact
$$
\lim  _ {\eta \rightarrow 0} G _ {\mu} ^ {- 1} (1 - \eta) = \mu^ {.} \tag {C.4}
$$
Recall that  $\mu^{*} \in [0,1]$  denotes the maximum value in the support of  $\mu$ . We run Alg. 4 on  $(\eta_{1},\eta_{2},\varepsilon^{\prime},\delta^{\prime})$  where:
$$
\eta_ {1} = \log^ {- 1 / 3} (N),
$$
$$
\eta_ {2} = \eta_ {1} / 2,
$$
$$
\varepsilon^ {\prime} = \varepsilon_ {1} - \varepsilon ,
$$
It follows from Proposition A.1 that the resulting output  $\hat{\alpha}_C + \frac{\varepsilon_1 - \varepsilon}{2}$  is computed using  $O\left(\frac{N\log\log(N)}{\log(N)}\right)\leq o(N)$  samples as in the previous cases. Moreover for  $N$  sufficiently large:
$$
\begin{array}{l} \mathbb {P} \left[ \hat {\alpha} _ {C} + \frac {\varepsilon_ {1} - \varepsilon}{2} \geq \mu^ {*} - \frac {\varepsilon^ {\prime}}{3} - o _ {N} (1) \right] \stackrel {(\mathrm {C . 4})} {\geq} \mathbb {P} \left[ \hat {\alpha} _ {C} + \frac {\varepsilon_ {1} - \varepsilon}{2} \geq G _ {\mu} ^ {- 1} (1 - \eta_ {1}) - \frac {\varepsilon^ {\prime}}{3} \right] \\ \geq 1 - \delta^ {\prime} \\ = 1 - e ^ {- \frac {1 0 N}{\log^ {2} (N)}}. \\ \end{array}
$$
Since  $\varepsilon_1 > \varepsilon$ , this means for  $N \geq N_0(\mu, c', \ldots)$  large enough,
$$
\mathbb {P} \left[ \hat {\alpha} _ {C} \geq \mu^ {*} - (\varepsilon_ {1} - \varepsilon) \right] \geq 1 - e ^ {- \frac {1 0 N}{\log^ {2} (N)}}.
$$
Note that Alg. 4 also ensures that with probability  $1 - e^{-\frac{10N}{\log^2(N)}}$
$$
\begin{array}{l} \hat {\alpha} _ {C} \leq \mu^ {*} + \frac {\varepsilon^ {\prime}}{3} - \frac {\varepsilon_ {1} - \varepsilon}{2} = \mu^ {*} - \frac {\varepsilon_ {1} - \varepsilon}{6} \\ \leq G _ {\mu} ^ {- 1} (1 - \eta^ {\prime}) \\ \end{array}
$$
for some  $\eta'(\mu, \varepsilon_1, \varepsilon) > 0$ . It follows that applying Theorem 3.1 with
$$
(N, \alpha , \eta , \varepsilon) = (N - N ^ {\prime}, \hat {\alpha} _ {C}, \eta^ {\prime}, \varepsilon)
$$
implies Theorem B.3.
C.2 The Fixed Budget Algorithm
We now present Algorithm 3 for the fixed budget problem (recall the informal discussion in Section 3). Algorithm 3 studies one arm  $a_{i}$  at a time, moving to  $a_{i + 1}$  if  $a_{i}$  is rejected. Similarly to the previous section, some details are needed while  $n_{t,i}$  is small, since large deviation asymptotics may not have kicked in yet. As explained at the start of the section, we choose a small constant  $\varrho > 0$ . In fact, we will eventually choose small constants
$$
0 <   \varrho \ll \varrho_ {1} \ll \varrho_ {2} \ll \varrho_ {3} \ll \varrho_ {4} \ll \varrho_ {5} \ll 1
$$
which all tend to 0 as  $\varrho \to 0$ . These constants will be defined throughout the proof. More formally, these values can be obtained by choosing  $\varrho_5 > 0$  arbitrarily small, then  $\varrho_4 > 0$  sufficiently small depending on  $\varrho_5$ , and so on.
Algorithm 3 operates in a batch-compressed way, for a sequence  $(b_{1}, b_{2}, \ldots)$  defined as follows:
$$
b _ {k} = b _ {0} (1 + \underline {{\varrho}}) ^ {k}, \quad k \leq k _ {0}
$$
$$
b _ {k _ {0} + j} = \lceil (1 + \varrho) ^ {j} b _ {k _ {0}} \rceil , \quad j \geq 1
$$
$$
\tau_ {k _ {0} + j} = \theta (\alpha - 2 \varrho) - j \cdot \frac {d _ {F} (\alpha , \beta) \varrho (1 - \varrho_ {2})}{\log N}, \quad j \geq 1.
$$
Note in particular that  $b_{k_0} \geq \log^4 (N)$ . We denote by  $\hat{p}_{i,t}$  the empirical average reward collected by  $a_i$  from its first  $t$  samples.
Algorithm 5: Output arm with  $p_i \geq \beta$  using  $N$  samples with high probability
1 input: an infinite sequence of arms  $i = 1,2,\ldots$    
2 initialize:  $i = 0$    
3 while fewer than  $N$  samples have been collected do   
4  $\begin{array}{l}i\leftarrow i + 1\\ \mathrm{Collect~}b_{0}\mathrm{~samples~of~arm~}i.\\ \mathrm{if~}\hat{p}_{i,b_0}\leq \alpha -\varrho \mathrm{then}\\ \mid \mathrm{Reject~arm~}i\\ \mathrm{end}\\ \mathrm{for~}k = 1,2,\dots ,k_0\mathrm{~do}\\ \mid \mathrm{Collect~}bk - bk - 1\mathrm{~samples~of~arm~}i\mathrm{~for~a~total~of~}bk\mathrm{~samples}.\\ \mathrm{if~}\hat{p}_{i,b_k}\leq \alpha -\varrho -\frac{k}{\sqrt{\log N}}\mathrm{~then}\\ \mid \mathrm{Reject~arm~}i;\\ \mathrm{end}\\ \mathrm{end}\\ \mathrm{for~}j = 1,2,\dots \mathrm{~do}\\ \mid \mathrm{Collect~}bk_{0 + j} - bk_{0 + j - 1}\mathrm{~samples~of~arm~}i\mathrm{~for~a~total~of~}bk_{0 + j}.\\ \mathrm{if~}\theta (\hat{p}_{i,b_{k_0 + j}})\leq \theta (\alpha -2\varrho) - j\cdot \frac{d_F(\alpha,\beta)\varrho(1 - \varrho_2)}{\log N}\mathrm{~then}\\ \mid \mathrm{Reject~arm~}i\\ \mathrm{end}\\ \mathrm{end}\\ \mathrm{return~arm~}i. \end{array}$    
5   
6 if  $\hat{p}_{i,b_0}\leq \alpha -\varrho$  then   
7 Reject arm i   
8 end   
9 for  $k = 1,2,\dots ,k_0$  do   
10 Collect  $bk - bk - 1$  samples of arm i for a total of  $bk$  samples.   
11 if  $\hat{p}_{i,b_k}\leq \alpha -\varrho -\frac{k}{\sqrt{\log N}}$  then   
12 |Reject arm i;   
13 end   
14 end   
15 for  $j = 1,2,\dots$  do   
16 Collect  $bk_{0 + j} - bk_{0 + j - 1}$  samples of arm i for a total of  $bk_{0 + j}$  .   
17 if  $\theta (\hat{p}_{i,b_{k_0 + j}})\leq \theta (\alpha -2\varrho) - j\cdot \frac{d_F(\alpha,\beta)\varrho(1 - \varrho_2)}{\log N}$  then   
18 |Reject arm i   
19 end   
20 end   
21 end   
22 Return arm i.
The role of the values  $b_{j}$  is as follows. When an arm  $a_{i}$  reaches  $b_{k}$  samples for some  $k \geq 0$ , it is checked for possible rejection by comparing its empirical average reward to the threshold  $\tau_{k}$ . Algorithm 3 rejects arm  $i$  and moves to arm  $a_{i+1}$  if the empirical average  $\hat{p}_{i,b_{k}}$  of arm  $a_{i}$  drops below a moving threshold  $\tau_{k}$ . The threshold  $\tau_{k}$  begins close to  $\alpha$  and gradually decreases until reaching  $\beta + \varrho$  by the time  $\tau_{k} \geq \Omega(N)$ .
So for, our informal description of Alg. 3 also applies to the algorithm proposed in [GM20]. We now highlight two important differences. The first is that our algorithm is defined more carefully during the "early" phases when an arm has been sampled at most  $N^{O(\varrho)}$  times. This is crucial for carrying out a rigorous analysis. The second difference is that in the main phase, we increase the sample size for a given arm in powers of  $1 + \varrho$  rather than powers of 2, and also move the rejection thresholds  $\tau_{k}$  based on the Fisher information distance via the function  $\theta$ . The latter ingredients allow us to obtain the optimal constant factor.
We begin the analysis of Alg 3 by proving Lemma 3.
Proof of Lemma 3. Let  $M_{j} = \prod_{1\leq i\leq j}Y_{i}$  and observe that  $M_j^c$  is a positive supermartingale with  $M_0 = 0$ . The result follows by Doob's maximal inequality.
We will apply Lemma 3 in the following way. Let  $X_{i}$  be the number of samples used by arm  $a_{i}$  before rejection, and  $I_{i} \in \{0,1\}$  be the indicator of the event that  $a_{i}$  is ever rejected, even if Algorithm 3 were to continue past time  $N$  and sample arm  $i$  an infinite number of times. We set
$$
Y _ {i} = e ^ {X _ {i}} \cdot I _ {i},
$$
With  $M$  defined from  $(Y_{i})_{i\geq 1}$  as in Lemma 3, it follows that  $\log (M)$  is at most the amount of time spent on eventual rejections before the first eventually accepted arm. Therefore if  $\log (M)\le N(1 - \varrho)$ , we conclude that the last arm to be studied was sampled at least  $N\varrho$  times. Since it was not rejected during that time, we can conclude this arm has  $p_i\geq \beta$  with probability  $1 - e^{-\Omega_{\varrho}(N)}$ . The main contribution to the failure probability of Algorithm 3 comes from the event  $\{M\geq A\}$  above, for suitable  $A$ . Correspondingly, the main work will be to verify  $\mathbb{E}[Y_i^c ]\leq 1$  for suitable  $c$ .
Note that  $Y_{i} \in \{0\} \cup [1, \infty)$  almost surely for each  $i$ . Therefore a necessary first step in showing  $\mathbb{E}[Y_i^c] \leq 1$  is to lower bound  $\mathbb{P}[Y_i = 0]$ , the probability that Algorithm 3 never rejects  $a_{i}$ . We now give a sufficient lower bound from the event  $p_{i} \geq \alpha$ .
Proposition C.1. Let  $x_{1}, x_{2}, \ldots$  be an i.i.d. Bernoulli(p) sequence for  $p \geq \alpha$ , and let  $S_{k} = \sum_{i=1}^{k} x_{i}$  and set
$$
\underline {{S}} = \inf  _ {k \geq 1} S _ {k} / k.
$$
Then  $\underline{S} \geq \alpha - \varrho$  holds with probability at least  $c(\alpha, \varrho) > 0$ . Thus  $\mathbb{E}[I_i] \leq 1 - c(\alpha, \varrho)$ .
Proof. Since the probability that  $\underline{S} \geq \alpha - \varrho$  is increasing in  $p$  it suffices to take  $p = \alpha$  and show the probability is positive for any  $\varrho > 0$ . Assume not. Then by restarting the indexing every time  $S_{k} \leq k(\alpha - \varrho)$  holds, we find that
$$
\liminf_{n\to \infty}S_{n} / n\leq \alpha -\varrho .
$$
This contradicts the strong law of large numbers, thus completing the proof of the first assertion. The second assertion follows since if  $S_{k} / k \geq \alpha - \varrho$  for all  $k$  where  $x_{1}, \ldots$  are the rewards of arm  $i$ , then arm  $i$  will never be rejected by Algorithm 3.
Based on Proposition C.1 above, to show
$$
\mathbb {E} \left[ e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} \cdot I _ {i} \right] \leq 1
$$
(which is essentially what we want in light of Lemma 3), it suffices to show that
$$
\mathbb {E} \left[ \left(e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} - 1\right) \cdot I _ {i} \right] \leq c (\alpha , \varrho). \tag {C.5}
$$
We let  $I_{i}^{t} = I_{i} \cdot 1_{X_{i} = t}$  be the event that arm  $i$  was rejected after exactly  $t$  steps. Since Alg 3 can only reject after  $b_{j}$  samples, we have
$$
I _ {i} = \sum_ {j = 0} ^ {\infty} I _ {i} ^ {b _ {j}}
$$
We use this to break the left-hand side of (C.5) into three separate parts and estimate the parts separately. The parts correspond to  $b_{0}, b_{1}$  through  $b_{k_{0}}$ , and  $b_{k_{0} + 1}$  onward. The first two parts are easier and handled in Subsection C.3 below. The final term is the main contribution and is handled in Subsection C.4.
C.3 Analysis of Algorithm 3 in the Small and Medium Sample Phases
Proposition C.2 bounds the contribution to (C.5) from the small sample phase, i.e. the first rejection condition in line 7 of Alg 3.
Proposition C.2. For any  $\alpha, \varrho$  there is  $\varrho_{1} > 0$  sufficiently small that with  $b_{0}$  as defined above, and with  $N$  sufficiently large,
$$
\mathbb {E} \left[ \left(e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} - 1\right) \cdot I _ {i} ^ {b _ {0}} \right] \leq c (\alpha , \varrho) / 4
$$
Proof. It suffices to observe that for fixed  $\alpha, \varrho$  and  $\varrho_{1}$  small and  $N$  sufficiently large, we have
$$
e ^ {b _ {0} \cdot \frac {c _ {\alpha , \beta} - \varrho_ {3}}{\log^ {2} N}} - 1 \leq e ^ {\varrho_ {1}} - 1 \leq 2 \varrho_ {1}.
$$
Proposition C.3 bounds the contribution to (C.5) from the medium sample phase, i.e. the second rejection condition in line 12 of Alg 3.
Proposition C.3. For any  $\alpha, \varrho, \varrho_1$  and for  $N$  sufficiently large,
$$
\sum_ {k = 1} ^ {k _ {0}} \mathbb {E} \left[ \left(e ^ {X _ {i} \cdot \frac {c _ {\alpha , \beta} - e _ {3}}{\log^ {2} N}} - 1\right) \cdot I _ {i} ^ {b _ {k}} \right] \leq c (\alpha , \varrho) / 4
$$