SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let's define  $T(\delta, D_0) \coloneqq \sup \{n > K \mid n - 1 < D_0 \sum_{i \in \mathcal{I}_{\varepsilon}(\mu)} T_{\varepsilon_0,\beta}(\mu,i)(\sqrt{c(n - 1,\delta)} + \sqrt{(2 + s)\log n})^2\}$ . Therefore, we have  $\mathcal{E}_n \cap \{n < \tau_{\varepsilon_0,\delta}\} = \emptyset$  (i.e.  $\mathcal{E}_n \subset \{\tau_{\varepsilon_0,\delta} \leq n\}$ ) for all  $n \geq \max \{T(\delta, D_0), T_\mu + 1\}$ . This concludes the construction.
To conclude the proof, we will show that Lemma 34 for many choices of  $D_0$  and  $T_{\mu}$
Vanilla Proof of Lemma 34. Let  $g_{\varepsilon}(n) = \frac{4H_{\mu,\varepsilon_0}(\varepsilon)}{\min\{\beta,1 - \beta\}} (2 + s)\log n + 3K(K - 1) / 2$ . Using Lemma 5, under the event  $\mathcal{E}_n$ ,
$$
\sum_ {i \in \mathcal {I} _ {\varepsilon} (\mu)} \sum_ {j} T _ {n} (i, j) \geq n - 1 - g _ {\varepsilon} (n).
$$
Therefore, the pigeonhole principle yields that there exists  $i_0 \in \mathcal{I}_{\varepsilon}(\mu)$  such that
$$
\sum_ {j} T _ {n} \left(i _ {0}, j\right) \geq v _ {i _ {0}} \left(n - 1 - g _ {\varepsilon} (n)\right).
$$
The crux of the problem is to relate  $N_{t,C_t}^{i_0} / (t - 1)$  and  $w_{\beta}^{\star}(i_0)_{C_t^{\mathrm{TC}}\varepsilon_0}$ . To do so, we use the approach of [20] that builds on the idea behind the proof for APT from [30]: consider an arm being over-sampled and study the last time this arm was pulled.
By the pigeonhole principle, at time  $n$ , there is an index  $k_{1} \neq i_{0}$  such that
$$
N _ {n, k _ {1}} ^ {i _ {0}} \geq \frac {w _ {\beta} ^ {\star} \left(i _ {0}\right) _ {k _ {1}}}{1 - \beta} \sum_ {j \neq i _ {0}} N _ {n, j} ^ {i _ {0}}, \tag {16}
$$
and we take such  $k_{1}$ . Let  $t_1 \coloneqq \sup \left\{t < n \mid (B_t, C_t) = (i_0, k_1)\right\}$  be the last time at which  $i_0$  was the leader and  $k_{1}$  was the challenger. If  $I_{t_1} = B_{t_1}$  then  $N_{t_1, k_1}^{i_0} = N_{n, k_1}^{i_0}$ , otherwise  $N_{t_1, k_1}^{i_0} = N_{n, k_1}^{i_0} - 1$ .
In both cases, we have  $N_{t_1,k_1}^{i_0} \geq N_{n,k_1}^{i_0} - 1$ . Combined with the above and using that  $w_{\beta}^{\star}(i_0)_{k_1} \leq 1 - \beta$  and  $v_{i_0} \leq 1$ , we obtain
$$
\begin{array}{l} N _ {t _ {1}, k _ {1}} ^ {i _ {0}} \geq N _ {n, k _ {1}} ^ {i _ {0}} - 1 \geq \frac {w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}}}{1 - \beta} \sum_ {j \neq i _ {0}} N _ {n, j} ^ {i _ {0}} - 1 \\ \geq w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} \left(\sum_ {j \neq i _ {0}} T _ {n} (i _ {0}, j) - \frac {K - 1}{2 (1 - \beta)}\right) - 1 \\ \geq w _ {\beta} ^ {\star} \left(i _ {0}\right) _ {k _ {1}} v _ {i _ {0}} (n - 1) - (1 - \beta) v _ {i _ {0}} g _ {\varepsilon} (n) - \frac {K + 1}{2} \\ \geq w _ {\beta} ^ {\star} \left(i _ {0}\right) _ {k _ {1}} v _ {i _ {0}} (n - 1) - h _ {\varepsilon} (n), \\ \end{array}
$$
with  $h_{\varepsilon}(n) = \frac{4(1 - \beta)H_{\mu,\varepsilon_0}(\varepsilon)}{\min\{\beta,1 - \beta\}} f_2(n) + (1 - \beta)3K(K - 1)/2 + \frac{K + 1}{2}$ .
Let  $w_{i_0, - } > 0$  be a lower bound on  $w_{\beta}^{\star}(i_0)_{k_1}$ , e.g.  $w_{i_0, - } = \min_{i\neq i_0}w_{\beta}^{\star}(i_0)_i$ . For instances  $\mu$  such that  $w_{\beta}^{\star}(i_0)_{k_1}$  is small, the equation (16) can be satisfied at the very beginning, hence  $t_1$  might be sublinear in  $n$ . Due to the missing link between  $t_1$  and  $n$ , we use the following inequality
$$
1 / N _ {t _ {1}, i _ {0}} ^ {i _ {0}} + 1 / N _ {t _ {1}, k _ {1}} ^ {i _ {0}} \leq \frac {1}{v _ {i _ {0}} (n - 1)} \left(1 / \beta + \frac {v _ {i _ {0}} (n - 1)}{N _ {t _ {1} , k _ {1}} ^ {i _ {0}}}\right) \left(\frac {N _ {t _ {1} , k _ {1}} ^ {i _ {0}}}{N _ {t _ {1} , i _ {0}} ^ {i _ {0}}} + 1\right),
$$
which is a suboptimal step which artificially introduces  $1 / \beta$ .
Let  $\gamma \in (0,1 / 2]$ . It remains to control both terms. First, we obtain
$$
1 / \beta + \frac {v _ {i _ {0}} (n - 1)}{N _ {t _ {1} , k _ {1}} ^ {i _ {0}}} \leq 1 / \beta + \frac {1}{w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} - \frac {h _ {\varepsilon} (n)}{v _ {i _ {0}} (n - 1)}} \leq (1 + \gamma) \left(1 / \beta + 1 / w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}}\right),
$$
for all  $n > C_1(w_{i_0, - },v_{\mu ,\varepsilon_0}(\varepsilon))$  . The last inequality is obtained by definition of
$$
C _ {1} (w, v) := \sup  \left\{n \geq 1 \mid n - 1 <   \frac {h _ {\varepsilon} (n)}{w v} \left(1 + \gamma^ {- 1}\right) \right\}, \tag {17}
$$
which ensures that, for all  $n > C_1(w_{i_0, - },v_{\mu ,\varepsilon_0}(\varepsilon))$  , the last condition of the equivalence
$$
w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} - \frac {h _ {\varepsilon} (n)}{v _ {i _ {0}} (n - 1)} \geq (1 + \gamma) ^ {- 1} w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} \quad \Longleftrightarrow \quad n - 1 \geq \frac {h _ {\varepsilon} (n)}{v _ {i _ {0}} w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}}} \left(1 + \gamma^ {- 1}\right)
$$
is satisfied since  $w_{\beta}^{\star}(i_0)_{k_1}\geq w_{i_0, - },v_{i_0}\geq v_{\mu ,\varepsilon_0}(\varepsilon)$  and  $n > C_1(w_{i_0, - },v_{\mu ,\varepsilon_0}(\varepsilon))$
Second, using Lemma 46, we can show that
$$
N _ {t _ {1}, i _ {0}} ^ {i _ {0}} \geq T _ {t _ {1}} (i _ {0}, k _ {1}) - N _ {t _ {1}, k _ {1}} ^ {i _ {0}} \geq \beta T _ {t _ {1}} (i _ {0}, k _ {1}) - 1 \geq \frac {\beta}{1 - \beta} N _ {t _ {1}, k _ {1}} ^ {i _ {0}} - \frac {1}{1 - \beta}.
$$
Then, re-ordering the above and using that  $N_{t_1,k_1}^{i_0}\geq w_\beta^\star (i_0)_{k_1}v_{i_0}(n - 1) - h_\varepsilon (n)$ , we obtain
$$
\frac {N _ {t _ {1} , k _ {1}} ^ {i _ {0}}}{N _ {t _ {1} , i _ {0}} ^ {i _ {0}}} + 1 \leq \left(\frac {\beta}{1 - \beta} - \frac {1}{(1 - \beta) \left(w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} v _ {i _ {0}} (n - 1) - h _ {\varepsilon} (n)\right)}\right) ^ {- 1} + 1 \leq (1 + \gamma) / \beta ,
$$
for all  $n > C_2(w_{i_0, - },v_\mu ,\varepsilon_0(\varepsilon))$  . The last inequality is obtained by definition of
$$
C _ {2} (w, v) := \sup  \left\{n \in \mathbb {N} ^ {\star} \mid n - 1 <   \frac {1}{w v} \left(h _ {\varepsilon} (n) + \frac {\gamma + 1 - \beta}{\beta \gamma}\right) \right\}, \tag {18}
$$
which ensures that, for all  $n > C_2(w_{i_0, - },v_{\mu ,\varepsilon_0}(\varepsilon))$  , the last condition of the equivalence
$$
\begin{array}{l} \left(\frac {\beta}{1 - \beta} - \frac {1}{(1 - \beta) \left(w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} v _ {i _ {0}} (n - 1) - h _ {\varepsilon} (n)\right)}\right) ^ {- 1} + 1 \leq (1 + \gamma) / \beta \\ \Longleftrightarrow \quad \frac {1}{w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} v _ {i _ {0}} (n - 1) - h _ {\varepsilon} (n)} \leq \frac {\beta \gamma}{\gamma + 1 - \beta} \\ \Longleftrightarrow \quad n - 1 \geq \frac {1}{w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}} v _ {i _ {0}}} \left(h _ {\varepsilon} (n) + \frac {\gamma + 1 - \beta}{\beta \gamma}\right) \\ \end{array}
$$
is satisfied since  $w_{\beta}^{\star}(i_0)_{k_1}\geq w_{i_0, - },v_{i_0}\geq v_{\mu ,\varepsilon_0}(\varepsilon)$  and  $n > C_2(w_{i_0, - },v_{\mu ,\varepsilon_0}(\varepsilon))$
Let  $T_{\mu} \geq \max_{k \in [2]} C_k(\min_{i \in \mathcal{I}_{\varepsilon}(\mu)} w_{i, -}, v_{\mu, \varepsilon_0}(\varepsilon))$  and  $D_0 = (1 + \gamma)^2 / \beta$ . In summary, we have shown that for all  $n > T_{\mu}$ , there exists  $i_0 \in \mathcal{I}_{\varepsilon}(\mu)$  and  $t_1 \leq n$  with  $B_{t_1} = i_0$  and such that (15) holds.
For  $\beta = 1/2$ , using that  $K \geq 2$ ,  $1 < 1 + \gamma^{-1}$  and  $5/2 + \gamma^{-1} \leq 3(1 + \gamma^{-1})/2$  (since  $\gamma \in (0,1/2]$ ), we obtain that
$$
\begin{array}{l} \max  _ {i \in [ 2 ]} C _ {i} (w, v) \leq \sup  \left\{n \geq 1 \mid n - 1 <   \frac {1 + \gamma^ {- 1}}{w v} \left(4 (2 + s) H _ {\mu , \varepsilon_ {0}} (\varepsilon) \log n + 3 K ^ {2} / 4 + 1\right) \right\} \\ <   h _ {1} \left(\frac {4 (2 + s) (1 + \gamma^ {- 1})}{w v} H _ {\mu , \varepsilon_ {0}} (\varepsilon), \frac {(1 + \gamma^ {- 1}) (3 K ^ {2} / 4 + 1)}{w v} + 1\right), \\ \end{array}
$$
where the last inequality uses Lemma 51 and  $h_1(x,y)$  defined therein.
Using  $w_{i_0, -} = \min_{i \neq i_0} w_\beta^*(i_0)_i$  and  $\beta = 1/2$ , the first part of Theorem 5 (i.e.  $x = 0$ ) is obtained by taking the infimum over  $(\varepsilon, x) \in [0, \varepsilon_0] \times \{0\}$ .
Refined Proof of Lemma 34. When considering large  $K$  or instances with unbalanced  $\beta$ -optimal allocation,  $\min_{i_0 \in \mathcal{I}_{\varepsilon}(\mu)} \min_{i \neq i_0} w_{\beta}^{\star}(i_0)_i$  can become arbitrarily small, hence a dependence in its inverse is undesired. Thanks to the refined analysis from [20], we can clip it with a value of our choosing which is away from zero.
Let  $x \in (0,1/(K-1)]$  be an allocation threshold and  $d_{\mu}(x,i) := |\{j \in [K] \setminus \{i\}| w_{\beta}^{\star}(i)_j < (1-\beta)x\}|$  for all  $i \in \mathcal{I}_{\varepsilon}(\mu)$ . The equation (16) is only informative when (1)  $w_{\beta}^{\star}(i_0)_{k_1} \geq (1-\beta)x$  or (2)  $w_{\beta}^{\star}(i_0)_{k_1} < (1-\beta)x$  and  $N_{n,k_1}^{i_0} \geq x \sum_{j \neq i_0} N_{n,j}^{i_0}$ . The problematic case occurs when (3)  $w_{\beta}^{\star}(i_0)_{k_1} < (1-\beta)x$  and  $N_{n,k_1}^{i_0} < x \sum_{j \neq i_0} N_{n,j}^{i_0}$ .
When  $w_{\beta}^{\star}(i_0)_{k_1} \geq (1 - \beta)x$ , we can use the above computations by considering  $w_{i_0, -} = \max \{(1 - \beta)x, \min_{i \neq i_0} w_{\beta}^{\star}(i_0)_i\}$ .
When  $w_{\beta}^{\star}(i_0)_{k_1} < (1 - \beta)x$  and  $N_{n,k_1}^{i_0} \geq x \sum_{j \neq i_0} N_{n,j}^{i_0}$ , the above proof can be conducted by using  $(1 - \beta)x$  instead of  $w_{\beta}^{\star}(i_0)_{k_1}$  since we will obtain that
$$
N _ {t _ {1}, k _ {1}} ^ {i _ {0}} \geq (1 - \beta) x v _ {i _ {0}} (n - 1) - h _ {\varepsilon} (n) \quad \text {a n d} \quad 1 / \beta + \frac {1}{(1 - \beta) x} \leq 1 / \beta + 1 / w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}}.
$$
Then, we can likewise consider  $w_{i_0, - } = \max \{(1 - \beta)x,\min_{i\neq i_0}w_\beta^\star (i_0)_i\}$  to conclude.
When  $w_{\beta}^{\star}(i_0)_{k_1} < (1 - \beta)x$  and  $N_{n,k_1}^{i_0} < x\sum_{j\neq i_0}N_{n,j}^{i_0}$ , as in (16), the pigeonhole principle yields that there exists  $k_{2}\in [K]\setminus \{i_{0},k_{1}\}$  such that
$$
N _ {n, k _ {2}} ^ {i _ {0}} \geq \frac {w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {2}}}{1 - \beta - w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {1}}} \sum_ {j \notin \{i _ {0}, k _ {1} \}} N _ {n, j} ^ {i _ {0}} \geq \frac {(1 - x) w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {2}}}{1 - \beta} \sum_ {j \neq i _ {0}} N _ {n, j} ^ {i _ {0}}.
$$
Based on  $k_{2}$  the same dichotomy as for  $k_{1}$  happens: either we can conclude the proof when we are in case 1 or 2 or we cannot since we are in case 3. If case 3 occurs also for  $k_{2}$ , i.e.  $w_{\beta}^{\star}(i_0)_{k_2} < (1 - \beta)x$  and  $N_{n,k_2}^{i_0} < x\sum_{j\notin \{i_0,k_1\}}N_{n,j}^{i_0}$ , we should also ignore it since it is non informative and construct a third arm  $k_{3}$ .
The main idea is then to peel off arms that are not informative, till we find an informative one. By induction, we construct a sequence  $(k_{a})_{a\in [d]}$  of such arms, where  $k_{d}$  is the first arm for which either (1)  $w_{\beta}^{\star}(i_0)_{k_d}\geq (1 - \beta)x$  or (2)  $w_{\beta}^{\star}(i_0)_{k_d} < (1 - \beta)x$  and  $N_{n,k_d}^{i_0}\geq x\sum_{j\notin \{i_0\} \cup \{k_a\}_{a\in [d - 1]}}N_{n,j}^{i_0}$  holds. This means that for all  $a\in [d - 1]$ , we have  $w_{\beta}^{\star}(i_0)_{k_a} < (1 - \beta)x$  and
$$
N^{i_{0}}_{n,k_{a}} <   x\sum_{\substack{j\notin \{i_{0}\} \cup \{k_{b}\}_{b\in [a - 1]}}}N^{i_{0}}_{n,j}.
$$
Using the pigeonhole principle for  $i \in [K] \setminus \left( \{i_0\} \cup \{k_a\}_{a \in [d-1]} \right)$  yields that  $k_d$  satisfies
$$
N _ {n, k _ {d}} ^ {i _ {0}} \geq \frac {w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {d}}}{1 - \beta - \sum_ {a \in [ d - 1 ]} w _ {\beta} ^ {\star} (i _ {0}) _ {k _ {a}}} \sum_ {j \notin \{i _ {0} \} \cup \{k _ {b} \} _ {b \in [ a - 1 ]}} N _ {n, j} ^ {i _ {0}},
$$
Using a simple recurrence on the arms  $\{k_{a}\}_{a\in [d - 1]}$ , we obtain that
$$
\sum_ {j \notin \{i _ {0} \} \cup \{k _ {b} \} _ {b \in [ a - 1 ]}} N _ {n, j} ^ {i _ {0}} \geq (1 - x) ^ {d - 1} \sum_ {j \neq i _ {0}} N _ {n, j} ^ {i _ {0}}.
$$
Let  $t_d \coloneqq \sup \left\{t < n \mid (B_t, C_t) = (i_0, k_d)\right\}$ . Since the arm  $k_d$  satisfies case 1 or case 2, we can conclude similarly as above with an extra multiplicative factor  $(1 - x)^{d-1}$ .
To remove the dependency in the random variable  $d$ , we consider the worst case scenario where  $\{k_{a}\}_{a\in [d - 1]} = \{i\in [K]\setminus \{i_0\} \mid w_{\beta}^{\star}(i_0)_i < (1 - \beta)x\}$ , i.e.  $d - 1\leq d_{\mu}(x,i_0)$ . In words, it means that we had to enumerate over all arms with small allocation, such that case 2 didn't hold, before finding an arm with large allocation, i.e. satisfying case 1. Let  $d_{\mu ,\varepsilon}(x) = \max_{i\in \mathcal{I}_{\varepsilon}(\mu)}d_{\mu}(x,i)$ . Using that
$$
(1 - x) ^ {d - 1} \geq (1 - x) ^ {d _ {\mu} (x, i _ {0})} \geq (1 - x) ^ {d _ {\mu , \varepsilon} (x)},
$$
$$
w _ {i _ {0}, -} = (1 - x) ^ {d _ {\mu} (x, i _ {0})} \max  \{(1 - \beta) x, \min  _ {i \neq i _ {0}} w _ {\beta} ^ {\star} (i _ {0}) _ {i} \},
$$
we can conclude the proof by taking  $T_{\mu} \geq \max_{k \in [2]} C_k(\min_{i \in \mathcal{I}_{\varepsilon}(\mu)} w_{i, -}, v_{\mu, \varepsilon_0}(\varepsilon))$  and  $D_0 = \frac{(1 + \gamma)^2}{\beta(1 - x)^{d_{\mu, \varepsilon}(x)}}$ . In other words, we have shown that  $n > T_{\mu}$ , there exists  $i_0 \in \mathcal{I}_{\varepsilon}(\mu)$  and  $t_d \leq n$  with  $B_{t_d} = i_0$  and such that (15) holds. Using  $\beta = 1/2$ , the second part of Theorem 5 is obtained by taking the infimum over  $(\varepsilon, x) \in [0, \varepsilon_0] \times (0, 1/(K - 1)]$ .
E.2.1 Consequence of tighter concentration
It is direct to see that the proof of Theorem 5 will also work when using the concentration event  $\tilde{\mathcal{E}}_n = \tilde{\mathcal{E}}_{n,1}^1\cap \tilde{\mathcal{E}}_{n,1}^2$  with  $(\tilde{\mathcal{E}}_{n,\delta}^{1})_{n > K}$  and  $(\tilde{\mathcal{E}}_{n,\delta}^{2})_{n > K}$  as in (27) and (29). Let us define
$$
\tilde {f} _ {1} (n) = \frac {1}{2} \bar {W} _ {- 1} (2 s \log n + 2 \log (2 + \log n) + 2),
$$
$$
\tilde {f} _ {2} (n) = \bar {W} _ {- 1} (s \log n + 2 \log (2 + \log n) + 2).
$$
Using Lemma 50, we know that  $\tilde{f}_1(n) \leq \tilde{f}_2(n)$ . Then, we will be able to show
$$
\mathbb {E} _ {\nu} [ \tau_ {\delta} ] \leq \inf  _ {(\varepsilon , x) \in [ 0, \varepsilon_ {0} ] \times [ 0, 1 / (K - 1) ]} \max  \left\{\tilde {T} _ {\mu , \varepsilon_ {0}} (\delta , \varepsilon , x) + 1, \tilde {S} _ {\mu , \varepsilon_ {0}} (\varepsilon , x) \right\} + \zeta (s) K (K + 1) / 2,
$$
$$
\begin{array}{l} \tilde {T} _ {\mu , \varepsilon_ {0}} (\delta , \varepsilon , x) = \sup  \left\{n \mid n - 1 \leq \frac {2 (1 + \gamma) ^ {2} \sum_ {i \in \mathcal {I} _ {\varepsilon} (\mu)} T _ {\varepsilon_ {0} , 1 / 2} (\mu , i)}{(1 - x) ^ {d _ {\mu , \varepsilon_ {0}} (\varepsilon , x)}} \right. \\ \left. \left(\sqrt {c (n - 1 , \delta)} + \sqrt {\tilde {f} _ {2} (n)}\right) ^ {2} \right\}, \\ \end{array}
$$
$$
\tilde {S} _ {\mu , \varepsilon_ {0}} (\varepsilon , x) = \tilde {h} _ {1} \left(\frac {4 (1 + \gamma^ {- 1})}{a _ {\mu , \varepsilon_ {0}} (\varepsilon , x) v _ {\mu , \varepsilon_ {0}} (\varepsilon)} H _ {\mu , \varepsilon_ {0}} (\varepsilon), \frac {(1 + \gamma^ {- 1}) (3 K ^ {2} / 4 + 1)}{a _ {\mu , \varepsilon_ {0}} (\varepsilon , x) v _ {\mu , \varepsilon_ {0}} (\varepsilon)} + 1\right),
$$
with  $\tilde{h}_1(z,y) > \sup \left\{n\geq 1\mid n <   zf_2(n) + y\right\}$ . Using Lemma 49, we obtain that
$$
\begin{array}{l} n \geq z \tilde {f} _ {2} (n) + y \quad \Longleftrightarrow \quad \frac {n - y}{z} - \log (n - y) \geq s \log n + 2 \log (2 + \log n) + 2 - \log z \\ \Longleftarrow \frac {n}{z (1 + s)} - \log \frac {n}{z (1 + s)} \geq \log (1 + s) + \frac {s}{s + 1} \log z + \frac {1}{1 + s} (2 \log (2 + \log n) + 2 + \frac {y}{z}). \\ \end{array}
$$
Therefore, we can consider  $\tilde{h}_1(z,y)\geq z(1 + s)h_2(z,y)$  with
$$
h _ {2} (z, y) = \sup \left\{x \mid x - \log x - \frac {2}{1 + s} \log (\log x + 2 + \log (z (1 + s))) <   C (z, y) \right\}
$$
where  $C(z,y) = \frac{y}{(1 + s)z} + \frac{s}{s + 1}\log z + \log (1 + s) + \frac{2}{1 + s}$ . Therefore, we will obtain a multiplicative factor  $(1 + s)$  instead of  $(2 + s)$  in front of  $H_{\mu,\varepsilon_0}(\varepsilon)$ . This improvement is rather mild, hence we don't elaborate on it further.
F Probability of error and simple regret
Let  $\varepsilon_0 > 0$  and  $\beta \in (0,1)$ . In Appendix F, we provide an analysis on the probability of error and the simple regret of  $\mathrm{EB - TC}_{\varepsilon_0}$  with fixed proportions  $\beta$  and slack  $\varepsilon_0 > 0$  (Theorem 6 proved in Appendix F.1). In Appendix F.2, we provide an upper bound on the unverifiable expected sample complexity of  $\mathrm{EB - TC}_{\varepsilon_0}$  (Theorem 7). Then, by using Theorem 6, we prove that the policy pulling arm  $\hat{i}_n$  at time  $n$  has a constant induced regret (Corollary 2 proved in Appendix F.3). In practice, we will mostly be interested in the case  $\beta = 1/2$ .
Theorem 3 and Corollary 1 is a corollary of Theorem 6 up to some additional computations on the complexities  $(H_{i}(\mu ,\varepsilon_{0}))_{i\in [C_{\mu} - 1]}$ , which we will detail below. Theorem 6 provides upper bound on  $\mathbb{P}_{\nu}\left(i_n\notin \mathcal{I}_\varepsilon (\mu)\right)$  holding for any time  $n$  and any error  $\varepsilon \geq 0$ , and upper bound on  $\mathbb{E}_{\nu}[\mu_{\star} - \mu_{i_n}]$  holding for any time  $n$ .
Theorem 6. Let  $\varepsilon_0 > 0$ . For all  $\varepsilon \geq 0$ , let  $i_{\mu}(\varepsilon) = i$  if  $\varepsilon \in [\Delta_i, \Delta_{i+1})$  and  $i \in [C_\mu - 1]$ , otherwise  $i_{\mu}(\varepsilon) = C_\mu$ . For all  $i \in [C_\mu - 1]$ , let  $C_i(\varepsilon_0) = 2\Delta_i^{-1} - \varepsilon_0^{-1}$  and  $C_{i,j}(\varepsilon_0) = 2\frac{\Delta_j / \varepsilon_0 + 1}{\Delta_i - \Delta_j} + 3\varepsilon_0^{-1}$ . For all  $i \in [C_\mu - 1]$ , let  $H_i(\mu, \varepsilon_0) \coloneqq \min_{j \in [i]} \max \{\tilde{H}_{i,j}(\mu, \varepsilon_0), \tilde{H}_{i,j}(\mu, \varepsilon_0)\}$  where
$$
\begin{array}{l} \bar {H} _ {i, j} (\mu , \varepsilon_ {0}) := | i ^ {\star} (\mu) | \max  \left\{\sqrt {2} \Delta_ {j + 1} ^ {- 1}, C _ {i + 1, j} (\varepsilon_ {0}) \right\} ^ {2} \\ + \max  \left\{C _ {j + 1} \left(\varepsilon_ {0}\right), C _ {i + 1, j} \left(\varepsilon_ {0}\right) \right\} ^ {2} \left(\sum_ {k = 2} ^ {j} \left| \mathcal {C} _ {\mu} (k) \right| + \sum_ {k = i + 1} ^ {C _ {\mu}} \left| \mathcal {C} _ {\mu} (k) \right|\right) \\ + \sum_ {k = j + 1} ^ {i} | \mathcal {C} _ {\mu} (k) | \max \left\{C _ {j + 1} (\varepsilon_ {0}), C _ {i + 1, j} (\varepsilon_ {0}), \sqrt {2} \Delta_ {k} ^ {- 1} \right\} ^ {2}, \\ \end{array}
$$
$$
\begin{array}{l} \tilde {H} _ {i, j} (\mu , \varepsilon_ {0}) := \frac {2 | i ^ {\star} (\mu) |}{\Delta_ {j + 1} ^ {2}} + \max  \left\{C _ {j + 1} (\varepsilon_ {0}), \varepsilon_ {0} ^ {- 1} \right\} ^ {2} \sum_ {k = 2} ^ {j} | \mathcal {C} _ {\mu} (k) | + \frac {2 \sum_ {k = 1} ^ {j} | \mathcal {C} _ {\mu} (k) |}{(\Delta_ {i + 1} - \Delta_ {j}) ^ {2}} \\ + \sum_ {k = j + 1} ^ {C _ {\mu}} | \mathcal {C} _ {\mu} (k) | \max  \left\{C _ {j + 1} (\varepsilon_ {0}), \varepsilon_ {0} ^ {- 1}, \sqrt {2} \Delta_ {k} ^ {- 1} \right\} ^ {2}. \\ \end{array}
$$
The  $EB-TC_{\varepsilon_0}$  algorithm with fixed proportions  $\beta = 1/2$  satisfies that, for all  $\nu \in \mathcal{D}^K$  and all  $n \geq D_\mu$ , if the algorithm has not stopped yet, then
$$
\begin{array}{l} \forall \varepsilon \geq 0, \quad \mathbb {P} _ {\nu} \left(\hat {i} _ {n} \notin \mathcal {I} _ {\varepsilon} (\mu)\right) \leq \mathbb {1} \left(\varepsilon <   \Delta_ {\max }\right) \frac {K (K + 1)}{2} e ^ {2} (2 + \log n) ^ {2} p \left(\frac {n - 5 K ^ {2} / 2}{8 H _ {i _ {\mu} (\varepsilon)} (\mu , \varepsilon_ {0})}\right), \\ \mathbb {E} _ {\nu} \left[ \mu_ {\star} - \mu_ {\hat {i} _ {n}} \right] \leq \frac {K (K + 1)}{2} e ^ {2} (2 + \log n) ^ {2} \sum_ {i \in \left[ C _ {\mu} - 1 \right]} \left(\Delta_ {i + 1} - \Delta_ {i}\right) p \left(\frac {n - 5 K ^ {2} / 2}{8 H _ {i} \left(\mu , \varepsilon_ {0}\right)}\right). \\ \end{array}
$$
where  $p(x) = x \exp(-x)$  and  $D_{\mu} = 8H_1(\mu, \varepsilon_0)h_2 \left(8H_1(\mu, \varepsilon_0), 5K^2/2, 2 + \log(K(K+1)/2)\right) + 5K^2/2$  with  $h_2$  defined in Lemma 51.
**Hardness of BAI** We have  $C_2(\varepsilon_0) = 2\Delta_{\mathrm{min}}^{-1} - \varepsilon_0^{-1}$  and  $C_{2,1}(\varepsilon_0) = 2\Delta_{\mathrm{min}}^{-1} + 3\varepsilon_0^{-1}$ . Then, we obtain  $H_{1}(\mu ,\varepsilon_{0}) = \bar{H}_{1,1}(\mu ,\varepsilon_{0}) = K(2\Delta_{\mathrm{min}}^{-1} + 3\varepsilon_{0}^{-1})^{2}$  since
$$
\tilde {H} _ {1, 1} (\mu , \varepsilon_ {0}) = 4 | i ^ {\star} (\mu) | \Delta_ {\mathrm {m i n}} ^ {- 2} + \sum_ {k = 2} ^ {C _ {\mu}} | \mathcal {C} _ {\mu} (k) | \max \left\{C _ {2} (\varepsilon_ {0}), \varepsilon_ {0} ^ {- 1}, \sqrt {2} \Delta_ {k} ^ {- 1} \right\} ^ {2}.
$$
This corresponds to the complexity of the two-groups instances which are defined as  $\mathcal{D}_{\Delta} := \{\nu \in \mathcal{D}^K \mid C_\mu = 2, \Delta_2 = \Delta\}$  where  $\Delta > 0$ . More importantly, it corresponds to the complexity of identifying one of the best arms, i.e. taking  $\varepsilon = 0$  yields
$$
\mathbb {P} _ {\nu} \left(\hat {i} _ {n} \notin i ^ {\star} (\mu)\right) \leq \frac {K (K + 1)}{2} e ^ {2} (2 + \log n) ^ {2} p \left(\frac {n - 5 K ^ {2} / 2}{8 K (2 \Delta_ {\operatorname* {m i n}} ^ {- 1} + 3 \varepsilon_ {0} ^ {- 1}) ^ {2}}\right).
$$
Hardness of  $\varepsilon$ -BAI Let  $i > 1$  and  $j \in [i]$ . We have  $C_{i + 1,j}(\varepsilon_0) = 2\frac{\Delta_j / \varepsilon_0 + 1}{\Delta_{i + 1} - \Delta_j} + 3\varepsilon_0^{-1}$ ,  $C_{j + 1}(\varepsilon_0) = 2\Delta_{j + 1}^{-1} - \varepsilon_0^{-1} \leq 2\Delta_{j + 1}^{-1}$  and  $\sqrt{2}\Delta_k^{-1} \leq 2\Delta_{j + 1}^{-1}$  for all  $k \geq j + 1$ . Direct manipulations show that, for all  $i > 1$  and all  $j \in [i]$ , we have
$$
\max \{\bar {H} _ {i, j} (\mu , \varepsilon_ {0}), \tilde {H} _ {i, j} (\mu , \varepsilon_ {0}) \} \leq K \max \left\{2 \Delta_ {j + 1} ^ {- 1}, 2 \frac {\Delta_ {j} / \varepsilon_ {0} + 1}{\Delta_ {i + 1} - \Delta_ {j}} + 3 \varepsilon_ {0} ^ {- 1} \right\} ^ {2}.
$$
$$
H _ {i} (\mu , \varepsilon_ {0}) \leq K \min _ {j \in [ i ]} \max \left\{2 \Delta_ {j + 1} ^ {- 1}, 2 \frac {\Delta_ {j} / \varepsilon_ {0} + 1}{\Delta_ {i + 1} - \Delta_ {j}} + 3 \varepsilon_ {0} ^ {- 1} \right\} ^ {2}.
$$
Taking  $j = 1$  and then  $j = 2$ , we see that, for all  $i > 1$
$$
H _ {i} (\mu , \varepsilon_ {0}) \leq K \max  \left\{2 \Delta_ {\min } ^ {- 1}, 3 \varepsilon_ {0} ^ {- 1} + 2 \Delta_ {i + 1} ^ {- 1} \right\} ^ {2} <   H _ {1} (\mu , \varepsilon_ {0}),
$$
$$
H _ {i} (\mu , \varepsilon_ {0}) \leq K \max \left\{2 \Delta_ {3} ^ {- 1}, 2 \frac {\Delta_ {\min} / \varepsilon_ {0} + 1}{\Delta_ {i + 1} - \Delta_ {\min}} + 3 \varepsilon_ {0} ^ {- 1} \right\} ^ {2} \leq K \max \left\{2 \frac {\Delta_ {\min} / \varepsilon_ {0} + 1}{\Delta_ {3} - \Delta_ {\min}} + 3 \varepsilon_ {0} ^ {- 1} \right\} ^ {2},
$$
where the last inequality uses that  $\Delta_3\leq \Delta_{i + 1}$  for all  $i > 1$
Similarly, we can derive a lower bound on  $H_{i}(\mu ,\varepsilon_{0})$  by noting that  $H_{i}(\mu ,\varepsilon_{0})\geq \min_{j\in [i]}\bar{H}_{i,j}(\mu ,\varepsilon_{0})$  For all  $j\in [i]$  , we have that  $\sqrt{2}\Delta_{j + 1}^{-1}\geq \sqrt{2}\Delta_{i + 1}^{-1}$ $C_{j + 1}(\varepsilon_0)\geq 2\Delta_{i + 1}^{-1} - \varepsilon_0^{-1}$  and  $C_{i + 1,j}(\varepsilon_0)\geq$ $2\Delta_{i + 1}^{-1} + 3\varepsilon_0^{-1}$  , hence we obtain
$$
H _ {i} (\mu , \varepsilon_ {0}) \geq \min  _ {j \in [ i ]} \bar {H} _ {i, j} (\mu , \varepsilon_ {0}) \geq 2 K \Delta_ {i + 1} ^ {- 2}.
$$
Three-groups instances The three-groups instances are defined as  $\mathcal{D}_{\Delta, \Lambda} := \{\nu \in \mathcal{D}^K \mid C_\mu = 3, \Delta_2 = \Delta, \Delta_3 = \Lambda\}$  where  $\Delta > 0$  and  $\Lambda > \Delta$ . Above results yield