SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {P} \left[ N _ {\overline {{\alpha}}} \leq K \left(\eta_ {1} - \frac {3 \eta_ {2}}{4}\right) \right] \geq e ^ {- \Omega (K \eta_ {2} ^ {2} / \eta_ {1})} \geq 1 - \frac {\delta}{8}.
$$
When both  $N(\varepsilon) \leq \frac{K\eta_2}{6}$  and  $N_{\overline{\alpha}} \leq K\left(\eta_1 - \frac{3\eta_2}{4}\right)$  hold, it follows by definition that  $\hat{\alpha} \leq \overline{\alpha} + \frac{\varepsilon}{3}$ . Hence recalling (A.2) above, we conclude that
$$
\mathbb {P} \left[ \hat {\alpha} \leq \overline {{\alpha}} + \frac {\varepsilon}{3} \right] \geq 1 - \frac {\delta}{4},
$$
establishing (A.3). The other direction is similar. With  $\alpha = G^{-1}(1 - \eta_1)$  as usual, we set
$$
N _ {\alpha} \equiv | \{i \in [ K ]: p _ {i} \geq \alpha \} |. \tag {A.4}
$$
This time,  $N_{\alpha}$  stochastically dominates a  $\mathrm{Bin}(K,\eta_1)$  random variable. Yet another Chernoff estimate yields
$$
\mathbb {P} \left[ N _ {\alpha} \geq K \left(\eta_ {1} - \frac {\eta_ {2}}{4}\right) \right] \geq 1 - \frac {\delta}{8}.
$$
Using (A.2) in the same way as above, we find
$$
\mathbb {P} \left[ \hat {\alpha} \geq \alpha - \frac {\varepsilon}{3} \right] \geq 1 - \frac {\delta}{4}.
$$
Proof of Theorem 2.1. First we analyze the expected sample complexity. On the event that
$$
\hat {\alpha} \in \left[ G ^ {- 1} (1 - \eta) - \frac {\varepsilon}{3}, G ^ {- 1} \left(1 - \frac {\eta}{2}\right) + \frac {\varepsilon}{3} \right] \tag {A.5}
$$
we claim that Alg. 2 terminates with probability  $\eta /4$  for each  $a_{i}$ . Indeed, if
$$
\hat {p} _ {i} \geq G ^ {- 1} \left(1 - \frac {\eta}{2}\right)
$$
then termination always happens by definition. This has probability at least  $1/4$  if  $p_i \geq G^{-1}\left(1 - \frac{\eta}{2}\right)$  by Theorem 1 in [GM14], and the latter condition holds with probability at least  $\eta/2$  by definition. It follows that when (A.5) holds, the expected sample complexity of Alg. 2 is  $O\left(\frac{\log(1/\eta\delta)}{\eta\varepsilon^2}\right)$ . On the other hand, (A.5) fails to hold with probability less than  $\delta$ . Because of the explicit termination condition in Alg. 2, this yields a additional sample complexity contribution of smaller order  $O\left(\delta \log(1/\delta) \frac{\log(1/\eta\delta)}{\eta\varepsilon^2}\right)$ . Finally Alg. 4 has sample complexity
$$
O \left(\frac {\log (1 / \eta) \log (1 / \delta)}{\eta \varepsilon^ {2}}\right)
$$
which clearly forms the dominant contribution. This completes the proof of the sample complexity bound and we now turn to proving correctness with probability  $1 - \delta$ . First, it is easy to see that Alg. 4 outputs some arm  $a_{i}$  with probability at least  $1 - \frac{\delta}{2}$ . It therefore suffices to show that for any fixed  $\hat{\alpha}$  satisfying (A.5), conditioned on the event  $\hat{p}_i \geq \hat{\alpha} - \frac{\varepsilon}{3}$ , the conditional probability that  $p_i \geq \alpha - \varepsilon$  is at least  $1 - \frac{\delta}{2}$ .
We do this using Bayes' rule. If  $p_i \geq G^{-1}(1 - \frac{\eta}{2})$ , then as above Theorem 1 in [GM14] implies
$$
\mathbb {P} \left[ \hat {p} _ {i} \geq \hat {\alpha} - \frac {\varepsilon}{3} \right] \geq \mathbb {P} [ \hat {p} _ {i} \geq p _ {i} ] \geq 1 / 4.
$$
This event hence contributes probability at least  $\eta /4$  to the event  $p_i\geq G^{-1}(1 - \eta)$ . On the other hand, if  $p_i\leq G^{-1}(1 - \eta) - \varepsilon \leq \hat{\alpha} -\frac{2\varepsilon}{3}$ , then
$$
\mathbb {P} \left[ \hat {p} _ {i} \geq \hat {\alpha} - \frac {\varepsilon}{3} \right] \leq \mathbb {P} \left[ \hat {p} _ {i} \geq p _ {i} + \frac {\varepsilon}{3} \right] \leq \eta \delta / 8
$$
for an absolute constant  $C$ . Combining these via Bayes' rule implies the desired result.
B Lower Bound for Fixed Budget
Fixed Budget with Unknown  $\alpha$
Before giving the proof, we give some qualitative discussion of the role of unknown  $\alpha$ . We consider Theorem 3.2 to be a definitive lower bound, since e.g. being given the value of  $\alpha$  only makes the result stronger. When  $\alpha$  is unknown, it is possible to give an essentially matching algorithm, but more care is required when stating the result. This is inherent and stems from the fact that the value  $\alpha = G_{\mu}^{-1}(1 - \eta)$  can be difficult or even impossible to estimate, yet determines the constant  $c_{\alpha,\beta}$  in the desired rate.
Let us illustrate the issue by a counterexample. Consider  $\mu_N$  defined by:
$$
\mathbb {P} ^ {p \sim \mu_ {N}} [ p = 0. 4 ] = \frac {1}{2} + e ^ {- 1 0 N}, \tag {B.1}
$$
$$
\mathbb {P} ^ {p \sim \mu_ {N}} [ p = 0. 2 ] = \frac {1}{2} - e ^ {- 1 0 N}.
$$
Similarly define  $\tilde{\mu}_N$  by:
$$
\mathbb {P} ^ {p \sim \tilde {\mu} _ {N}} [ p = 0. 4 ] = \frac {1}{2} - e ^ {- 1 0 N},
$$
$$
\mathbb {P} ^ {p \sim \tilde {\mu} _ {N}} [ p = 0. 3 ] = 2 e ^ {- 1 0 N}, \tag {B.2}
$$
$$
\mathbb {P} ^ {p \sim \tilde {\mu} _ {N}} [ p = 0. 2 ] = \frac {1}{2} - e ^ {- 1 0 N}.
$$
Then  $\mu_N$  and  $\tilde{\mu}_N$  are not distinguishable using  $N$  samples, yet  $G_{\mu}^{-1}(1/2) = 0.4$  while  $G_{\tilde{\mu}}^{-1}(1/2) = 0.3$ . Using non-distinguishability it follows that the lower bound of Theorem 3.2 applies to  $\tilde{\mu}_N$  with threshold  $\alpha = G_{\mu_N}^{-1}(1/2) = 0.4$ , as opposed to the direct application using  $G_{\tilde{\mu}_N}^{-1}(1/2) = 0.3$ . It is not hard to show using monotonicity of  $\frac{1}{\sqrt{x(1 - x)}}$  that
$$
c _ {0. 4, 0. 4 - \varepsilon} <   c _ {0. 3, 0. 3 - \varepsilon}
$$
for all  $\varepsilon \leq 0.3$ . As a result, it is information-theoretically impossible to achieve the rate (3.1) for  $\tilde{\mu}_N$  if the target quantile value  $\alpha$  is not given. The core reason is that the value  $G_{\tilde{\mu}}^{-1}(1 / 2) = 0.3$  is too sensitive to the choice  $\eta = 1 / 2$  of quantile.
Fortunately, this issue is more of an annoyance than a real difficulty. It can be fixed in several ways. In Theorems B.1, B.2, and B.3 below we give three concrete formulations under which the guarantee (3.1) can be achieved, as mentioned in the main body.
Theorem B.1. For fixed  $\eta_1, \eta_2, \varepsilon$ , there is a sequence  $(\mathcal{A}_N)_{N \geq 1}$  of  $N$ -sample algorithms outputting  $a_i^*$  such that the following holds for any sequence  $(\mu_N)_{N \geq 1}$  of reservoir distributions. Letting
$$
\alpha_ {N} = \frac {1}{\eta_ {1} - \eta_ {2}} \cdot \int_ {1 - \eta_ {1}} ^ {1 - \eta_ {2}} G _ {\mu_ {N}} ^ {- 1} (x) d x
$$
be a quantile average of  $\mu_N$ , we have
$$
\lim  _ {N \rightarrow \infty} \frac {\left(- \log \mathbb {P} \left[ p _ {i ^ {*}} <   \alpha_ {N} - \varepsilon \right]\right) \cdot \log^ {2} N}{c _ {\alpha_ {N} , \alpha_ {N} - \varepsilon} N} \geq 1. \tag {B.3}
$$
Theorem B.2. For fixed  $\eta, \varepsilon$ , there is a sequence  $(\mathcal{A}_N)_{N \geq 1}$  of  $N$ -sample algorithms outputting  $a_i^*$  such that for any sequence of reservoir distributions  $\mu_N$  satisfying
$$
\alpha_ {N} \equiv G _ {\mu_ {N}} ^ {- 1} (1 - \eta) \geq \frac {1 + \varepsilon}{2},
$$
$$
\lim  _ {N \rightarrow \infty} \frac {\left(- \log \mathbb {P} \left[ p _ {i ^ {*}} <   G _ {\mu_ {N}} ^ {- 1} (1 - \eta) - \varepsilon \right]\right) \cdot \log^ {2} N}{c _ {\alpha_ {N} , \alpha_ {N} - \varepsilon} N} \geq 1. \tag {B.4}
$$
Theorem B.3. For any fixed  $\varepsilon_1 > \varepsilon$ , there is a sequence  $(\mathcal{A}_N)_{N\geq 1}$  of  $N$ -sample algorithms outputting  $a_{i^*}$  such that for any fixed reservoir distribution  $\mu$  with  $\mu^{*} > \varepsilon$ ,
$$
\lim  _ {N \rightarrow \infty} \frac {\left( \right.- \log \mathbb {P} \left[ \right. p _ {i ^ {*}} <   \mu^ {*} - \varepsilon_ {1}\left. \right) \cdot \log^ {2} N \left. \right.}{N} \geq c _ {\mu^ {*}, \mu^ {*} - \varepsilon}. \tag {B.5}
$$
We emphasize that the rate (3.1) is optimal in all cases since the lower bound of Theorem 3.2 is for an easier problem. The first formulation above may be the most principled choice. The idea is that an averaged quantile depends continuously on  $\mu$ , and can in fact be estimated by applying Proposition A.1 for several pairs  $(\eta_1, \eta_2)$  and computing a Riemann sum. The second formulation requires only the mild condition that  $\alpha \geq \frac{1 + \varepsilon}{2}$  and uses monotonicity of  $c_{\alpha, \alpha - \varepsilon}$  on this set. (In other words, if the average reward values  $p$  appearing in (B.1), (B.2) were larger than 0.5, there would be no counterexample.) The third formulation allows us to almost send  $\eta$  all the way down to 0. It uses the fact that
$$
\mu^ {*} - \left(\varepsilon_ {1} - \varepsilon\right) \leq G _ {\mu} ^ {- 1} \left(1 - \eta^ {\prime}\right)
$$
for some  $\eta' = \eta'(\mu, \varepsilon_1, \varepsilon) > 0$ . These results show that (3.1) is achievable even without knowledge of  $\alpha$ , up to a choice of technical modification to sidestep the counterexample discussed above.
Remark B.1. In fact uniformity in  $(\alpha, \beta)$  holds in the following sense. For any sequence  $(\alpha_N, \beta_N)_{N \geq 1}$  of pairs with  $\min \left(\beta_N, \alpha_N - \beta_N, 1 - \alpha_N\right)$  uniformly bounded below, there is a sequence  $(\mathcal{A}_N)_{N \geq 1}$  of  $N$ -sample algorithms such that for any  $\eta \in (0, 1)$  and any sequence of reservoir distributions  $\mu_N$  with  $G_{\mu_N}^{-1}(1 - \eta) \geq \alpha_N$ ,
$$
\lim  _ {N \rightarrow \infty} \frac {\left(- \log \mathbb {P} \left[ p _ {i ^ {*}} <   \beta_ {N} \right]\right) \cdot \log^ {2} N}{c _ {\alpha_ {N} , \beta_ {N}} N} \leq 1. \tag {B.6}
$$
This can be shown identically to Theorem 3.1, though we don't give the proof in this generality. It is useful for the reduction arguments in Theorems B.1, B.2, and B.3.
B.1 Preparation for the Proof
Here we prove Theorem 3.2. For any  $\alpha, \beta, \eta, \varrho > 0$  we construct a reservoir  $\mu = \mu_{\alpha, \beta, \eta, \varrho}$  such that
$$
\lim  _ {N \rightarrow \infty} \inf  _ {N \rightarrow \infty} \frac {\left(- \log \mathbb {P} ^ {\mu} \left[ p _ {i ^ {*}} <   \beta \right]\right) \cdot \log^ {2} N}{N} \leq c _ {\alpha , \beta} + \lambda (\varrho) \tag {B.7}
$$
holds for any sequence of  $N$ -sample algorithms  $\mathcal{A}_N$ , and where  $\lim_{\varrho \to 0} \lambda(\varrho) = 0$  for fixed  $\alpha, \beta, \eta$ .
B.2 Admissible Reservoirs and Bayesian Perspective
In proving Theorem 3.2, we will use reservoir distributions  $\mu$  of a specific form. Namely, we require each  $\mu$  to be supported on an interval  $[\underline{\gamma},\overline{\gamma} ]$ , where
$$
0 <   \beta - \underline {{\varrho}} <   \underline {{\gamma}} <   \beta <   \alpha <   \overline {{\gamma}} <   \alpha + \underline {{\varrho}} <   1.
$$
In fact we define  $\underline{\gamma},\overline{\gamma}$  explicitly (recall that  $\varrho >0$  is a small constant which we eventually send to 0) by
$$
\theta (\underline {{\gamma}}) = \theta (\beta) - \varrho^ {2};
$$
$$
\begin{array}{l} \theta (\underline {{\gamma}}) = \theta (\beta) - \underline {{\varrho}} ^ {2}, \\ \theta (\overline {{\gamma}}) = \theta (\alpha) + \underline {{\varrho}} ^ {2}. \end{array} \tag {B.8}
$$
We say  $\mu$  is  $(\underline{\gamma},\overline{\gamma},\underline{f},\overline{f})$  admissible if  $\mu$  has density  $\mu(dx) = f(x)dx$  for a Borel measurable function  $f$  and satisfies for constants  $0 < \underline{f} < \overline{f} < \infty$ ,
$$
f (x) \in [ \underline {{f}}, \overline {{f}} ], \quad \forall x \in [ \underline {{\gamma}}, \overline {{\gamma}} ].
$$
Towards proving Theorem 3.2, we fix throughout this section some  $(\underline{\gamma},\overline{\gamma},\underline{f},\overline{f})$  admissible  $\mu$  such that  $G_{\mu}^{-1}(\alpha) = \eta$  holds, for appropriate constants  $(\underline{f},\overline{f})$  depending only on  $(\eta ,\varepsilon ,\alpha ,\beta ,\underline{\gamma},\overline{\gamma})$ . It is easy to see that this is always possible.
An admissible  $\mu$  is roughly comparable to the uniform distribution on an interval. Using admissible reservoirs gives each  $a_{i}$  the potential to slowly degrade in observed quality over time. We remark that while it is more convenient to work with reservoirs supported away from the boundaries, i.e. in  $[\underline{\gamma},\overline{\gamma} ]\subseteq (0,1)$ , we do not expect this to be essential.
It will be helpful throughout this section to take a Bayesian point of view. We treat  $\mu_N$  as known to  $\mathcal{A}_N$ , since  $\mathcal{A}_N$  is in fact allowed to depend on  $\mu_N$ . Thus at each time  $t$ , each  $p_i$  has a posterior probability distribution which we denote by  $\mu_{i,t}$ . Note that each  $\mu_{i,t}$  depends only on  $(n_{i,t},\hat{p}_{i,t})$  and is initialized at  $\mu_{i,0} = \mu$ . We denote by
$$
\boldsymbol {\mu} ^ {t} = \left(\mu_ {1, t}, \mu_ {2, t}, \dots\right) \tag {B.9}
$$
the sequence of posterior distributions  $\mu_{i,t}$ . Since arms are independent,  $\pmb{\mu}^t$  is the full time- $t$  posterior of the algorithm.
B.3 Batched Algorithms and Adversaries
In pure exploration problems, it is possible to significantly simplify the structure of any algorithm at the cost of a small multiplicative increase in the sample complexity. We carry this out using the notion of a batch-compressed algorithm.
Definition B.1. Given an increasing sequence  $B = (b_{1}, b_{2}, \ldots)$  of positive integers, an algorithm  $\mathcal{A}$  is  $B$ -batch-compressed if  $\mathcal{A}$  can only act by increasing the number of times  $n_{i}$  that  $a_{i}$  has been sampled from  $b_{k}$  to  $b_{k+1}$ , so that  $n_{i} \in B$  holds at all times.  $B$  is  $\varrho$ -slowly increasing if
$$
\frac {b _ {k + 1}}{b _ {k} + 1} \leq 1 + \varrho , \quad \forall k \geq 1.
$$
Finally if  $\mathcal{A}$  is  $B$ -batch-compressed and  $B$  is  $\varrho$ -slowly increasing, we say that  $\mathcal{A}$  is  $\varrho$ -batch-compressed.
Unlike the batched algorithms studied in [PRCS16, GHRZ19], batch-compression is only important for us as an analysis technique. Indeed the following proposition shows that it does not fundamentally affect pure exploration algorithms.
Proposition B.2. If  $B$  is  $\varrho$ -slowly increasing, then for any  $N$ -sample algorithm  $\mathcal{A}$ , there exists an  $B$ -batch-compressed  $\lfloor N(1 + \varrho)\rfloor$  algorithm  $\mathcal{A}'$  with the same output.
Proof. We show how to simulate  $\mathcal{A}$  using the  $B$ -batch-compressed  $\mathcal{A}'$ , assuming that the sequence of rewards for each  $a_i$  is fixed. Each time  $\mathcal{A}$  samples arm  $i$  for the  $n_i = (a_k + 1)$ -st time for  $a_k \in A$ ,  $\mathcal{A}'$  samples arm  $i$  until  $n_i = a_{k+1}$ . Then  $\mathcal{A}'$  has all the information of  $\mathcal{A}$  at all times, hence can simulate the behavior and output of  $\mathcal{A}$ . Moreover by the definition of  $\varrho$ -slowly increasing, the sample complexity of  $\mathcal{A}'$  is larger than that of  $\mathcal{A}$  by at most a factor  $(1 + \varrho)$ .
We will use the above with  $\varrho \to 0$  slowly as  $N\to \infty$ . Then the sample complexity increase  $1 + \varrho$  is absorbed into the  $1 + o(1)$  factor in Theorem 3.2. As a result it suffices to establish (B.7) under the additional assumption that  $\mathcal{A}_N$  is  $\varrho$ -batch-compressed.
B.4 Fisher Information Distance
Determining the tight constant  $c_{\alpha, \beta}$  requires significant care. In particular the adversary must decrease the empirical average rewards  $\hat{p}_{i,t}$  at a precise rate depending on  $n_{i,t}$ . This rate turns out to involve the Fisher information distance. For  $a, b \in [0,1]$  we define the Fisher information distance  $d_F(a,b)$  between  $a$  and  $b$  to be
$$
d _ {F} (a, b) = \left| \int_ {a} ^ {b} \frac {d x}{\sqrt {x (1 - x)}} \right|.
$$
This agrees with the more general Fisher information metric when each  $a \in [0,1]$  is identified with the corresponding Bernoulli distribution. We refer the reader to [Nie20] for a survey on information geometry. In short, the Fisher information yields a natural Riemannian metric on families of probability distributions which are parametrized by smooth manifolds. However we will use only elementary properties of  $d_F$ .
We parametrize  $[0,1]$  using the function  $\theta :[0,1]\to [0,\pi ]$  defined by
$$
\theta (a) = d _ {F} (0, a) = \int_ {0} ^ {a} \frac {d x}{\sqrt {x (1 - x)}} = \operatorname {a r c c o s} (1 - 2 a). \tag {B.10}
$$
$$
d _ {F} (a, b) = | \operatorname {a r c c o s} (1 - 2 a) - \operatorname {a r c c o s} (1 - 2 b) | \geq 2 | a - b |
$$
and so  $d_F(0,1) = \pi$ . The main property of  $\theta$  that we will use is the resulting differential equation
$$
\theta^ {\prime} (a) = \frac {1}{\sqrt {\theta (a) (1 - \theta (a))}}. \tag {B.11}
$$
In our case,  $\theta^{-1}$  parametrizes a "constant speed" path through the space of Bernoulli variables, viewing the Fisher information. Correspondingly, our adversary will ensure that  $\theta (\hat{p}_i(n_{i,t}))$  decreases linearly in  $\log (n_{i,t})$ .
B.5 Preliminary Lemmas from Moderate Deviations
Recall that for positive integers  $a$  and  $b$ , the  $\operatorname{Beta}(a, b)$  distribution has probability density function
$$
\frac {(a + b - 1) !}{(a - 1) ! (b - 1) !} x ^ {a - 1} (1 - x) ^ {b - 1}
$$
for  $x \in [0,1]$ . We now recall a moderate deviations principle for the binomial distribution and a central limit theorem for the beta distribution.
Lemma 4 (Theorem 2.2 in [DA92]). For any  $0 < \underline{q} < \overline{q} < 1$  and constant  $\varrho > 0$  there exists  $\Delta_0(\underline{q},\overline{q},\varrho)$  and  $M_0(\underline{q},\overline{q},\varrho)$  such that the following holds for all  $p \in [\underline{q},\overline{q}]$ . For  $n \geq n_0(\underline{q},\overline{q},\varrho)$  sufficiently large and any  $\frac{1}{\Delta_0\sqrt{n}} \leq \Delta \leq \Delta_0$  we have
$$
e ^ {\left(- \frac {\Delta^ {2}}{2 p (1 - p)} - \varrho\right) n} \leq \mathbb {P} \left[ \frac {B i n (n , p)}{n} \leq p - \delta \right] \leq e ^ {\left(- \frac {\Delta^ {2}}{2 p (1 - p)} + \varrho\right) n}.
$$
Lemma 5 (Lemma A.1 in [MNS16]). Let  $\{a_{n}\}_{n\geq n_{0}}$  be a sequence satisfying
$$
\underline {{\gamma}} \leq \frac {a _ {n}}{n} \leq \overline {{\gamma}}.
$$
Then the  $\mathrm{Beta}(n - a_n + 1, a_n + 1)$  distribution on  $[0,1]$  obeys a central limit theorem with mean  $\frac{a_n}{n}$  and standard deviation  $\sqrt{\frac{(a_n / n)(1 - (a_n / n))}{n}}$  in the sense that for any bounded sequence  $(w_{n})_{n\geq n_{0}}$  of real numbers and with  $\Phi$  the normal CDF,
$$
\lim  _ {n \rightarrow \infty} \left| \Phi (w _ {n}) - \mathbb {P} ^ {x \sim \operatorname {B e t a} (n - a _ {n} + 1, a _ {n} + 1)} \left[\left(x - (a _ {n} / n)\right) \cdot \sqrt {\frac {n}{(a _ {n} / n) (1 - (a _ {n} / n))}} \leq w _ {n} \right]\right| = 0.
$$
In the next two lemmas, we lower bound the probability that  $\hat{p}_{i,t}$  changes significantly when the number  $n_{i,t}$  of samples for  $a_i$  increases by a factor  $(1 + \varrho)$ .
Lemma 6. Assume  $\mu$  is  $(\underline{\gamma},\overline{\gamma},\underline{f},\overline{f})$ -admissible. Suppose that arm  $i$ 's average reward  $\hat{p}_{i,t}$  after  $n = n_{i,t}$  samples satisfies
$$
\hat {p} _ {i, t} \in [ \beta , \bar {\gamma} ]. \tag {B.12}
$$
Then for  $n \geq C(\underline{\gamma},\overline{\gamma},\underline{f},\overline{f},\beta)$  sufficiently large,
$$
\mathbb {P} ^ {x \sim \mu_ {i, n}} [ x \leq \hat {p} _ {i, t} ] \geq \frac {f}{3 \bar {f}}. \tag {B.13}
$$
Proof. Let  $R_{i,t} = n\hat{p}_{i,t}$  be the total reward from arm  $i$  so far. The posterior distribution  $\mu_{i,t}$  for  $p_i$  takes the form
$$
\mu_ {i, t} (d x) = \frac {x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}} f (x) d x}{\int_ {\underline {{\gamma}}} ^ {\overline {{\gamma}}} x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}} f (x) d x}.
$$
For  $x\in [\gamma ,\overline{\gamma} ]$  we estimate
$$
\frac {x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}} f (x)}{\int_ {\underline {{\gamma}}} ^ {\overline {{\gamma}}} x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}} f (x) d x} \geq (\underline {{f}} / \overline {{f}}) \cdot \frac {x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}}}{\int_ {0} ^ {1} x ^ {R _ {i , t}} (1 - x) ^ {n - R _ {i , t}} d x}.
$$
The right-hand side is the density of a beta variable with parameters  $(R_{i,t} + 1, n - R_{i,t} + 1)$ . We conclude that
$$
\mathbb {P} ^ {x \sim \mu_ {i, t}} \left[ x \in [ \underline {{\gamma}}, \hat {p} _ {i, t} ] \right] \geq (\underline {{f}} / \overline {{f}}) \cdot \mathbb {P} ^ {z \sim \mathrm {B e t a} (n - R _ {i, t} + 1, R _ {i, t} + 1)} \left[ z \in [ \underline {{\gamma}}, \hat {p} _ {i, t} ] \right]
$$
For  $n$  sufficiently large, it follows from Lemma 5 and (B.12) that
$$
\mathbb {P} ^ {z \sim \mathrm {B e t a} (n - R _ {i, t} + 1, R _ {i, t} + 1)} \left[ z \in [ \underline {{\gamma}}, \hat {p} _ {i, t} ] \right] \geq \frac {1}{3}.
$$
Therefore  $\mathbb{P}^{\mu_i,t}[p_i\leq \hat{p}_{i,t}]\geq \frac{1}{3}$ , proving (B.13).
Lemma 7. Assume  $\mu$  is  $(\underline{\gamma},\overline{\gamma},\underline{f},\overline{f})$ -admissible and that (B.12) holds. For  $n = n_{i,t}$ , let  $\tilde{n} \geq 1$  satisfy  $|\tilde{n} - \varrho n| \leq 2$ . Let
$$
\tilde {p} _ {i} = \frac {R _ {i , n + \tilde {n}} - R _ {i , n}}{\tilde {n}}
$$
be the average reward from the  $(n + 1)$ -th through  $(n + \tilde{n})$ -th samples of arm  $i$ . Then as  $n \to \infty$ , for any sequence  $\Delta_n = \Theta(1 / \log n)$ ,
$$
\mathbb {P} ^ {t} \left[ \tilde {p} _ {i} \leq \theta^ {- 1} \left(\theta \left(\hat {p} _ {i, t}\right) - \Delta_ {n}\right) \right] \geq \exp \left(- \frac {n \varrho \Delta_ {n} ^ {2} \left(1 + o _ {n} (1)\right)}{2}\right). \tag {B.14}
$$
Proof. Stochastic monotonicity implies that
$$
\mathbb {P} \left[ \frac {\mathrm {B i n} (\tilde {n} , p)}{\tilde {n}} \leq \theta^ {- 1} \big (\theta (\hat {p} _ {i, t}) - \Delta_ {n} \big) \right]
$$
is a decreasing function of  $p \in [0,1]$ . Combining with Lemma 6, it follows that
$$
\begin{array}{l} \mathbb {P} ^ {t} [ \tilde {p} _ {i} \leq \theta^ {- 1} (\theta (\hat {p} _ {i, t}) - \Delta_ {n}) ] = \int \mathbb {P} \left[ \frac {\operatorname {B i n} (\tilde {n} , x)}{\tilde {n}} \leq \theta^ {- 1} \left(\theta (\hat {p} _ {i, t}) - \Delta_ {n}\right) \right] d \mu_ {i, t} (x) \\ \geq \mathbb {P} ^ {\mu_ {i, t}} [ p _ {i} \leq \hat {p} _ {i, t} ] \cdot \mathbb {P} \left[ \frac {\operatorname {B i n} (\tilde {n} , \hat {p} _ {i , t})}{\tilde {n}} \leq \theta^ {- 1} \left(\theta (\hat {p} _ {i, t}) - \Delta_ {n}\right) \right] \\ \geq \frac {f}{3 \bar {f}} \cdot \mathbb {P} \left[ \frac {\operatorname {B i n} (\tilde {n} , \hat {p} _ {i , t})}{\tilde {n}} \leq \theta^ {- 1} \left(\theta (\hat {p} _ {i, t}) - \Delta_ {n}\right) \right]. \\ \end{array}
$$
Since  $\theta$  is smooth with smooth inverse on  $[\underline{\gamma},\overline{\gamma} ]$  and  $\Delta_{n}\leq o_{n}(1)$  , we have
$$
\begin{array}{l} \hat {p} _ {i, t} - \theta^ {- 1} \left(\theta \left(\hat {p} _ {i, t}\right) - \Delta_ {n}\right) = \left(1 \pm o _ {n} (1)\right) \Delta_ {n} \cdot \left(\theta^ {- 1}\right) ^ {\prime} \left(\theta \left(\hat {p} _ {i, t}\right)\right) \\ = \frac {\left(1 \pm o _ {n} (1)\right) \cdot \Delta_ {n}}{\theta^ {\prime} \left(\theta^ {- 1} \left(\hat {p} _ {i , t}\right)\right)} \\ = (1 \pm o _ {n} (1)) \cdot \Delta_ {n} \sqrt {\hat {p} _ {i , t} (1 - \hat {p} _ {i , t})}. \\ \end{array}
$$
The result now follows from Lemma 4, where we absorb the factor  $\frac{f}{(3\overline{f})}$  into the  $o_n(1)$ .
B.6 Proof of Theorem 3.2
Recall the definition (B.8) of  $\underline{\gamma}$  and  $\overline{\gamma}$ . We require  $\mathcal{A}$  to be  $B$ -batch-compressed for  $B = B(N, \underline{\varrho})$  containing:
1. All positive integers at most  $N^{2\varrho}$
2. All positive multiples of  $\lfloor N^{\varrho}\rfloor$  at most  $N^{6\varrho}$
3. Integers of the form  $\lfloor N^{6\varrho}(1 + \varrho)^j\rfloor$  for  $j\geq 0$