SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
T _ {\varepsilon_ {0}} (\mu) \left(\Delta_ {\max } + \varepsilon_ {0}\right) ^ {2} \geq T _ {\varepsilon_ {1}} (\mu) \left(\Delta_ {\max } + \varepsilon_ {1}\right) ^ {2} \quad a n d \quad T _ {\varepsilon_ {0}, \beta} (\mu) \left(\Delta_ {\max } + \varepsilon_ {0}\right) ^ {2} \geq T _ {\varepsilon_ {1}, \beta} (\mu) \left(\Delta_ {\max } + \varepsilon_ {1}\right) ^ {2}.
$$
Proof. Let  $i^{\star}(\mu) = \{i^{\star}\}$ . Let  $\varepsilon_0 > \varepsilon_1$ . Using Lemma 9, we have
$$
2 T _ {\varepsilon} (\mu) ^ {- 1} (\Delta_ {\min } + \varepsilon) ^ {- 2} = \max  _ {w \in \triangle_ {K}} \min  _ {j \neq i ^ {*}} \frac {\tilde {\Delta} _ {j} (\varepsilon) ^ {2}}{1 / w _ {i ^ {*}} + 1 / w _ {j}} \quad \text {w i t h} \quad \tilde {\Delta} _ {j} (\varepsilon) = \frac {\mu_ {\star} - \mu_ {j} + \varepsilon}{\Delta_ {\min } + \varepsilon}.
$$
To conclude the first part of the first result, a sufficient condition is to show that  $\tilde{\Delta}_j(\varepsilon_1) \geq \tilde{\Delta}_j(\varepsilon_0)$  for all  $j \neq i^{\star}$ . Direct manipulations show that, for all  $j \neq i^{\star}$ ,
$$
\tilde {\Delta} _ {j} \left(\varepsilon_ {1}\right) \geq \tilde {\Delta} _ {j} \left(\varepsilon_ {0}\right) \iff 1 - \frac {\varepsilon_ {0} - \varepsilon_ {1}}{\mu_ {\star} - \mu_ {j} + \varepsilon_ {0}} \geq 1 - \frac {\varepsilon_ {0} - \varepsilon_ {1}}{\Delta_ {\min } + \varepsilon_ {0}} \iff \mu_ {\star} - \mu_ {j} \geq \Delta_ {\min },
$$
hence the result holds. The same proof can be used to obtain the second part of the first result.
Let  $\varepsilon_0 < \varepsilon_1$ . Using Lemma 9, we have
$$
2 T _ {\varepsilon} (\mu) ^ {- 1} \left(\Delta_ {\max } + \varepsilon\right) ^ {- 2} = \max  _ {w \in \triangle_ {K}} \min  _ {j \neq i ^ {\star}} \frac {\bar {\Delta} _ {j} (\varepsilon) ^ {2}}{1 / w _ {i ^ {\star}} + 1 / w _ {j}} \quad \text {w i t h} \quad \bar {\Delta} _ {j} (\varepsilon) = \frac {\mu_ {\star} - \mu_ {j} + \varepsilon}{\Delta_ {\max } + \varepsilon}.
$$
To conclude the first part of the second result, a sufficient condition is to show that  $\bar{\Delta}_j(\varepsilon_1) \geq \bar{\Delta}_j(\varepsilon_0)$  for all  $j \neq i^{\star}$ . Direct manipulations show that, for all  $j \neq i^{\star}$ ,
$$
\bar {\Delta} _ {j} \left(\varepsilon_ {1}\right) \geq \bar {\Delta} _ {j} \left(\varepsilon_ {0}\right) \iff 1 + \frac {\varepsilon_ {1} - \varepsilon_ {0}}{\mu_ {\star} - \mu_ {j} + \varepsilon_ {0}} \geq 1 + \frac {\varepsilon_ {1} - \varepsilon_ {0}}{\Delta_ {\max } + \varepsilon_ {0}} \iff \mu_ {\star} - \mu_ {j} \leq \Delta_ {\max },
$$
hence the result holds. The same proof can be used to obtain the second part of the second result.
D Asymptotic analysis
Let  $\varepsilon_0 > 0$ ,  $\beta \in (0,1)$  and  $\delta \in (0,1)$ . In this section, we provide an asymptotic analysis of  $\mathrm{EB - TC}_{\varepsilon_0}$  slack  $\varepsilon_0 > 0$  combined with the stopping rule (3) with parameters  $(\varepsilon_0,\delta)$ . First, we detail IDS proportions in Appendix D.1. Then, we sketch the proof for fixed proportions  $\beta$  in Appendix D.2.
In the following, we consider a sub-Gaussian bandit with distribution  $\nu \in \mathcal{D}^K$  having mean parameter  $\mu \in \mathbb{R}^K$  with a unique best arm, i.e.  $i^{\star}(\mu) = \{i^{\star}\}$ . We restate Theorem 1 below.
Theorem 4 (Theorem 1). Let  $\varepsilon_0 > 0$ ,  $\varepsilon_1 \geq 0$  and  $\delta \in (0,1)$ . Using the threshold (4) in the stopping rule (3) with slack  $\varepsilon_1$ , the  $EB-TC_{\varepsilon_0}$  algorithm is  $(\varepsilon_1, \delta)$ -PAC. For IDS proportions, it satisfies that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^\star(\mu)| = 1$ ,
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\nu}[\tau_{\varepsilon_{1},\delta}]}{\log(1 / \delta)}\leq T_{\varepsilon_{0}}(\mu)\left(1 + \max_{i\neq i^{\star}}\frac{\varepsilon_{0} - \varepsilon_{1}}{\mu_{i^{\star}} - \mu_{i} + \varepsilon_{1}}\right)^{2}.
$$
Let  $\beta \in (0,1)$ . For fixed proportions  $\beta$ , it satisfies that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^{\star}(\mu)| = 1$ ,
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\nu}[\tau_{\varepsilon_{1},\delta}]}{\log(1 / \delta)}\leq T_{\varepsilon_{0},\beta}(\mu)\left(1 + \max_{i\neq i^{\star}}\frac{\varepsilon_{0} - \varepsilon_{1}}{\mu_{i^{\star}} - \mu_{i} + \varepsilon_{1}}\right)^{2}.
$$
Theorem 1 provides asymptotic upper bound on the expected sample complexity of the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm when combined with the stopping rule (3). When  $\varepsilon_{1} = \varepsilon_{0}$  and for Gaussian distributions, combining Lemma 1 and Theorem 1 shows that the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm is asymptotically optimal for  $\varepsilon_0$ -BAI when using IDS proportions and asymptotically  $\beta$ -optimal for  $\varepsilon_0$ -BAI when using fixed proportions  $\beta$ . Until recently [40], proving asymptotic optimality of Top Two algorithms with adaptive choice  $\beta$  was an open problem in the BAI literature. Our work extends their analysis to tackle the  $\varepsilon$ -BAI setting. Compared to previous work, we removed the assumption that sub-optimal arms should have distinct means.
Considering  $\varepsilon_1 \neq \varepsilon_0$  can be interesting when the practitioner decides to tackle  $\varepsilon$ -BAI with an error  $\varepsilon$  which is different from the slack  $\varepsilon_0$  used by the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm. Those situations occur when the practitioner is observing the data collected by the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm without having control over it. Theorem 1 show that the resulting sequential hypothesis testing has still asymptotic guarantees on the expected sample complexity. Quite naturally, those guarantees are not asymptotically optimal even for Gaussian distributions. The sub-optimality multiplicative gap of IDS proportions (that is, the ratio of the upper bound in Theorem 1 and the characteristic time for  $\varepsilon_1$ -BAI,  $T_{\varepsilon_1}(\mu)$ ) is
$$
\frac {T _ {\varepsilon_ {0}} (\mu)}{T _ {\varepsilon_ {1}} (\mu)} \frac {\left(\Delta_ {\min } + \varepsilon_ {0}\right) ^ {2}}{\left(\Delta_ {\min } + \varepsilon_ {1}\right) ^ {2}} \geq 1 \quad \text {w h e n} \varepsilon_ {0} > \varepsilon_ {1}, \text {o t h e r w i s e} \quad \frac {T _ {\varepsilon_ {0}} (\mu)}{T _ {\varepsilon_ {1}} (\mu)} \frac {\left(\Delta_ {\max } + \varepsilon_ {0}\right) ^ {2}}{\left(\Delta_ {\max } + \varepsilon_ {1}\right) ^ {2}} \geq 1,
$$
where the inequalities comes from Lemma 11 and justify that there is no contradiction with the asymptotic lower bound on the expected sample complexity (Lemma 1). The same reasoning applies to fixed proportions  $\beta$  by replacing  $T_{\varepsilon}(\mu)$  by  $T_{\varepsilon, \beta}(\mu)$ .
Concentration result For both proofs, we use the following concentration result of the empirical mean for sub-Gaussian observations (Lemma 12). This is a standard tool for the asymptotic analysis of Top Two algorithms (e.g. Lemma 3 in [32], Lemma 5 in [36] or Lemma 14 in [21]), hence we omit the proof.
Lemma 12. There exists a sub-Gaussian random variable  $W_{\mu}$  such that almost surely, for all  $i \in [K]$  and all  $n$  such that  $N_{n,i} \geq 1$ ,
$$
| \mu_ {n, i} - \mu_ {i} | \leq W _ {\mu} \sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}}.
$$
In particular, any random variable which is polynomial in  $W_{\mu}$  has a finite expectation.
D.1 IDS proportions
Using Lemma 9, the optimal allocation for  $\varepsilon_0$ -BAI are defined as
$$
w_{\varepsilon_{0}}(\mu):= \operatorname *{arg  max}_{w\in \triangle_{K}}\min_{i\neq i^{\star}}\frac{(\mu_{i^{\star}} - \mu_{i} + \varepsilon_{0})^{2}}{1 / w_{i^{\star}} + 1 / w_{i}}  .
$$
Since  $|i^{\star}(\mu)| = 1$ , let  $\mu_{\varepsilon_0} = \mu_{\varepsilon_0}(i^\star)$  where  $\mu_{\varepsilon}(i^{\star})$  as in Lemma 9. Since we have  $T_{\varepsilon_0}(\mu) = T^{\star}(\mu_{\varepsilon_0})$  and  $w_{\varepsilon_0}(\mu) = w^{\star}(\mu_{\varepsilon_0})$ , Lemma 6 and 8 yield that  $w_{\varepsilon_0}(\mu) = \{w^{\star}\}$  is a singleton with unique element denotes by  $w^{\star}$  which satisfies that  $\min_{i\in [K]}w_i^{\star} > 0$ ,
$$
\sum_ {i \neq i ^ {\star}} \left(\frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}\right) ^ {2} = 1, \tag {8}
$$
$$
\forall i \in [ K ] \backslash \{i ^ {\star} \}, \quad \frac {\left(\mu_ {i ^ {\star}} - \mu_ {i} + \varepsilon_ {0}\right) ^ {2}}{1 / w _ {i ^ {\star}} ^ {\star} + 1 / w _ {i} ^ {\star}} = 2 T _ {\varepsilon_ {0}} (\mu) ^ {- 1}. \tag {9}
$$
[40] refers to (8) as the overall balance equation. The key novelty of their analysis is to show that asymptotically the empirical proportions satisfy the same equation, referred to as the empirical overall balance, when using IDS proportions with sampling to choose among the leader and the challenger. The key novelty in our work lies in proving that the empirical overall balance equation is also satisfied when using IDS proportions with  $K(K - 1)$  tracking procedures to select between the leader and the challenger (see Appendix D.1.2).
Convergence time We follow an asymptotic analysis similar to the ones used in [32, 36, 21] with the improvement from [40]. Let  $\gamma > 0$ . Let us define the convergence time  $T_{\varepsilon_0,\gamma}$ , which is a random variable quantifies the number of samples required for the empirical allocations  $(N_{n,i} / N_{n,i^*})_{i \neq i^*}$  to be  $\gamma$ -close to  $(w_i^\star / w_{i^*})_{i \neq i^*}$ :
$$
T _ {\varepsilon_ {0}, \gamma} := \inf  \left\{T \geq 1 \mid \forall n \geq T, \max  _ {i \neq i ^ {\star}} \left| \frac {N _ {n , i}}{N _ {n , i ^ {\star}}} - \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} \right| \leq \gamma \right\}. \tag {10}
$$
Lemma 13 gives a sufficient condition to prove Theorem 1 for IDS proportions. The case  $\varepsilon_0 = \varepsilon_1$  is a direct consequence of combining Theorem 13 and Proposition 14 in [40]. The proof is inspired by existing methods, e.g. Theorem 2 in [21] or Theorem 3 in [32].
Lemma 13. Let  $\varepsilon_0 > 0$ ,  $\varepsilon_1 \geq 0$  and  $\delta \in (0,1)$ . Assume that there exists  $\gamma_1(\mu) > 0$  such that for all  $\gamma \in (0,\gamma_1(\mu)], \mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$ . Using the threshold (4) in the stopping rule (3) with slack  $\varepsilon_1$  yields an algorithm such that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^{\star}(\mu)| = 1$ ,
Proof. Let  $\gamma > 0$ . Let us define by
$$
\tilde {T} _ {\gamma , \varepsilon_ {0}} := \inf  \left\{T \geq 1 \mid \forall n \geq T, \left\| \frac {N _ {n}}{n - 1} - w ^ {\star} \right\| _ {\infty} \leq \gamma \right\}.
$$
Using that
$$
\sum_ {i \neq i ^ {\star}} \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}}} = \frac {1}{w _ {i ^ {\star}}} - 1 \quad \text {a n d} \quad \sum_ {i \neq i ^ {\star}} \frac {N _ {n , i}}{N _ {n , i ^ {\star}}} = \frac {n - 1}{N _ {n , i ^ {\star}}} - 1,
$$
it is direct to see that for all  $n\geq T_{\gamma}$
$$
\begin{array}{l} \left| \frac {N _ {n , i ^ {\star}}}{n - 1} - w _ {i ^ {\star}} ^ {\star} \right| = \left| \frac {n - 1}{N _ {n , i ^ {\star}}} - \frac {1}{w _ {i ^ {\star}} ^ {\star}} \right| \frac {N _ {n , i ^ {\star}}}{n - 1} w _ {i ^ {\star}} ^ {\star} \leq \left| \frac {n - 1}{N _ {n , i ^ {\star}}} - \frac {1}{w _ {i ^ {\star}} ^ {\star}} \right| \leq \gamma (K - 1), \\ \left| \frac {N _ {n , i}}{n - 1} - w _ {i} ^ {\star} \right| = w _ {i ^ {\star}} ^ {\star} \left| \frac {N _ {n , i}}{n - 1} \left(\frac {1}{w _ {i ^ {\star}} ^ {\star}} - \frac {n - 1}{N _ {n , i ^ {\star}}}\right) + \frac {N _ {n , i}}{N _ {n , i ^ {\star}}} - \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} \right| \\ \leq \left| \frac {1}{w _ {i ^ {\star}} ^ {\star}} - \frac {n - 1}{N _ {n , i ^ {\star}}} \right| + \left| \frac {N _ {n , i}}{N _ {n , i ^ {\star}}} - \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} \right| \leq \gamma K. \\ \end{array}
$$
Therefore, for all  $\gamma \in (0,\gamma_1(\mu) / K],\mathbb{E}_\nu [\tilde{T}_{\gamma ,\varepsilon_0}] <   + \infty$
Let  $\mu_{\varepsilon_0}$  as in Lemma 9. Since we have  $T_{\varepsilon_0}(\mu) = T^{\star}(\mu_{\varepsilon_0})$  and  $w_{\varepsilon_0}(\mu) = w^{\star}(\mu_{\varepsilon_0})$ , we can leverage existing results from the BAI literature, e.g. Theorem 2 in [21] or Theorem 3 in [32]. The sole criterion on the stopping threshold is to be asymptotically tight (Definition 1).
Definition 1. A threshold  $c:\mathbb{N}\times (0,1]\to \mathbb{R}_+$  is said to be asymptotically tight if there exists  $\alpha \in [0,1)$ ,  $\delta_0\in (0,1]$ , functions  $f,\bar{T}:(0,1]\to \mathbb{R}_{+}$  and  $C$  independent of  $\delta$  satisfying: (1) for all  $\delta \in (0,\delta_0]$  and  $n\geq T(\delta)$ , then  $c(n,\delta)\leq f(\delta) + Cn^{\alpha}$ , (2)  $\lim \sup_{\delta \to 0}f(\delta) / \log (1 / \delta)\leq 1$  and (3)  $\lim \sup_{\delta \to 0}\bar{T} (\delta) / \log (1 / \delta) = 0$ .
Since  $\mathcal{C}_G$  defined in (23) satisfies  $\mathcal{C}_G\approx x + \log (x)$ , it is direct to see that
$$
c (n, \delta) = 2 \mathcal {C} _ {G} \left(\frac {1}{2} \log \left(\frac {K - 1}{\delta}\right)\right) + 4 \log \left(4 + \log \frac {n}{2}\right)
$$
is asymptotically tight, e.g. by taking  $(\alpha, \delta_0, C) = (1/2, 1, 4)$ ,  $f(\delta) = 2\mathcal{C}_G\left(\frac{1}{2}\log \left(\frac{K - 1}{\delta}\right)\right)$  and  $\bar{T}(\delta) = 1$ .
Based on continuity arguments and using that  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$ , the proof of Theorem 2 in [21] yields that
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\nu}[\tau_{\delta}]}{\log(1 / \delta)}\leq \left(\min_{i\neq i^{\star}}\frac{(\mu_{i^{\star}} - \mu_{i} + \varepsilon_{1})^{2}}{2(1 / w_{i^{\star}}^{\star} + 1 / w_{i}^{\star})}\right)^{-1} = T_{\varepsilon_{0}}(\mu)\max_{i\neq i^{\star}}\left(\frac{\mu_{i^{\star}} - \mu_{i} + \varepsilon_{0}}{\mu_{i^{\star}} - \mu_{i} + \varepsilon_{1}}\right)^{2},
$$
where the equality uses the condition at the equilibrium (9). This concludes the proof.
Using Lemma 13, the proof of Theorem 1 for IDS proportions boils down to showing that  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$ . As in [32, 36, 21, 40], the proof is divided in several steps. First, we show that all the arms are sufficient explored for  $n$  large enough (Appendix D.1.1). Second, we prove that the empirical overall balance equation approximately holds for  $n$  large enough (Appendix D.1.2). Finally, we show convergence of the empirical ratio of proportions towards the ratio of optimal allocation, i.e.  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$  (Appendix D.1.3).
D.1.1 Sufficient exploration
To upper bound the expected convergence time, as prior work we first establish sufficient exploration. Given an arbitrary threshold  $L \in \mathbb{R}_+^*$ , we define the sampled enough set and its arms with highest mean (when not empty) as
$$
S _ {n} ^ {L} := \left\{i \in [ K ] \mid N _ {n, i} \geq L \right\} \quad \text {a n d} \quad \mathcal {I} _ {n} ^ {\star} := \underset {i \in S _ {n} ^ {L}} {\arg \max } \mu_ {i}. \tag {11}
$$
In all generality  $\mathcal{I}_n^\star$  is a set, yet we obtain  $\mathcal{I}_n^\star = \{i^\star\}$  as soon as  $i^\star \in S_n^L$ . We define the highly and the mildly under-sampled sets
$$
U _ {n} ^ {L} := \left\{i \in [ K ] \mid N _ {n, i} <   \sqrt {L} \right\} \quad \text {a n d} \quad V _ {n} ^ {L} := \left\{i \in [ K ] \mid N _ {n, i} <   L ^ {3 / 4} \right\}. \tag {12}
$$
[21] identifies the properties that the leader and the challenger should satisfy to ensure sufficient exploration. We improve on their analysis by removing the need for the distinct means assumption, and supporting IDS instead of a fixed allocation  $\beta$ .
Lemma 14 shows that the transportation cost is strictly positive and increases linearly. Compared to previous results, the key improvement lies in the fact that the lower bound holds for  $(i,j) \in \mathcal{I}_n^\star \times S_n^L$  instead of  $(i,j) \in \mathcal{I}_n^\star \times (S_n^L \setminus \mathcal{I}_n^\star)$ . This is due to the  $\varepsilon_0$  regularization, which removes the need to assume that the means are distinct.
Lemma 14. Let  $S_{n}^{L}$  and  $\mathcal{I}_{n}^{\star}$  as in (11). There exists  $L_{0}$  with  $\mathbb{E}_{\mu}[(L_0)^\alpha ] < +\infty$  for all  $\alpha >0$  such that if  $L\geq L_0$ , for all  $n$  such that  $S_{n}^{L}\neq \emptyset$ , for all  $(i,j)\in \mathcal{I}_n^\star \times S_n^L$  such that  $i\neq j$ , we have
$$
\frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \geq \sqrt {L} \frac {\varepsilon_ {0}}{2 \sqrt {2}}.
$$
Proof. Using Lemma 12 and  $\min \{N_{n,i}, N_{n,j}\} \geq L$ , we obtain
$$
\begin{array}{l} \mu_ {n, i} - \mu_ {n, j} \geq \mu_ {i} - \mu_ {j} - W _ {\mu} \left(\sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}} + \sqrt {\frac {\log (e + N _ {n , j})}{N _ {n , j}}}\right) \\ \geq - 2 W _ {\mu} \sqrt {\frac {\log (e + L)}{L}} \geq - \frac {\varepsilon_ {0}}{2}, \\ \end{array}
$$
where the last inequality is obtained for  $L \geq L_0 = L_1 - e$  which is defined as
$$
L _ {1} = \sup  \left\{x \mid x <   1 6 W _ {\mu} ^ {2} \log (x) / \varepsilon_ {0} ^ {2} + e \right\} \leq h _ {1} \left(1 6 W _ {\mu} / \varepsilon_ {0} ^ {2}, e\right).
$$
The last inequality is obtained by using Lemma 51, and we recall that  $h_1$  is defined in Lemma 51. Since  $h_1(x,e)\sim_{x\to +\infty}x\log x$ , we have  $\mathbb{E}_{\mu}[(L_0)^{\alpha}] < + \infty$  for all  $\alpha >0$  by using Lemma 12 (polynomial in  $W_{\mu}$ ). Then, we have
$$
\frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \geq \frac {\varepsilon_ {0} / 2}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \geq \sqrt {L} \frac {\varepsilon_ {0}}{2 \sqrt {2}}.
$$
Lemma 15 gives an upper bound on the transportation costs between a sampled enough arm and an under-sampled one.
Lemma 15. Let  $S_{n}^{L}$  as in (11). There exists  $L_{1}$  with  $\mathbb{E}_{\mu}[(L_1)^\alpha ] < +\infty$  for all  $\alpha >0$  such that for all  $L\geq L_{1}$  and all  $n\in \mathbb{N}$ ,
$$
\forall (i, j) \in S _ {n} ^ {L} \times \overline {{S _ {n} ^ {L}}}, \quad \frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \leq \sqrt {L} (D _ {1} + 4 W _ {\mu}),
$$
where  $D_{1} > 0$  is a problem dependent constant and  $W_{\mu}$  is the random variables defined in Lemma 12.
Proof. Using Lemma 12 and  $N_{n,i} \geq L \geq N_{n,j} \geq 1$ , we obtain
$$
\begin{array}{l} \frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \leq \sqrt {N _ {n , j}} | \mu_ {n, i} - \mu_ {n, j} + \varepsilon_ {0} | \\ \leq \sqrt {L} \left(| \mu_ {i} - \mu_ {j} + \varepsilon_ {0} | + W _ {\mu} \left(\sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}} + \sqrt {\frac {\log (e + N _ {n , j})}{N _ {n , j}}}\right)\right) \\ \leq \sqrt {L} \left(\left| \mu_ {i} - \mu_ {j} + \varepsilon_ {0} \right| + 2 W _ {\mu} \sqrt {\log (e + 1)}\right) \\ \leq \sqrt {L} \left(D _ {1} + 4 W _ {\mu}\right), \\ \end{array}
$$
for  $D_{1} = \max_{i\neq j}|\mu_{i} - \mu_{j} + \varepsilon_{0}|$
Lemma 16 shows the desired property for the EB leader. Since it corresponds to Lemma 17 in [21], we omit the proof.
Lemma 16 (Lemma 17 in [21]). There exists  $L_{2}$  with  $\mathbb{E}_{\nu}[(L_2)^{\alpha}] < +\infty$  for all  $\alpha >0$  such that if  $L\geq L_{2}$ , for all  $n$  (at most polynomial in  $L$ ) such that  $S_n^L\neq \emptyset$ ,  $B_n^{EB}\in S_n^L$  implies  $B_n^{EB}\in \mathcal{I}_n^\star$ .
Lemma 17 show that the desired property for the TC challenger. Compared to existing result, we improve by removing the assumption that all arms have distinct means thanks to the regularization  $\varepsilon_0 > 0$ .
Lemma 17. There exists  $L_{3}$  with  $\mathbb{E}_{\nu}[L_3] < +\infty$  such that if  $L \geq L_{3}$ , for all  $n$  (at most polynomial in  $L$ ) such that  $U_{n}^{L} \neq \emptyset$ ,  $B_{n}^{EB} \notin V_{n}^{L}$  implies  $C_{n}^{TC\varepsilon_{0}} \in V_{n}^{L}$ .
Proof. In the following, we consider  $U_{n}^{L} \neq \emptyset$  (hence  $V_{n}^{L} \neq \emptyset$ ) and  $B_{n}^{\mathrm{EB}} \notin V_{n}^{L}$ . Let  $\mathcal{J}_n^\star = \arg \max_{i\in \overline{V_n^L}}\mu_i$  and  $L_{2}$  as in Lemma 16. Then, for  $L \geq L_2^{4 / 3}$ , we have  $B_{n}^{\mathrm{EB}} \in \mathcal{J}_{n}^{\star}$ .
Let  $L_{0}$  and  $(L_1,D_1)$  as in Lemmas 14 and 15. Then, for all  $L\geq \max \{L_0^{4 / 3},L_2^{4 / 3},L_1^2\}$
$$
B _ {n} ^ {\mathrm {E B}} \in \mathcal {I} _ {n} ^ {\star},
$$
$$
\forall (i, j) \in \mathcal {J} _ {n} ^ {\star} \times \overline {{V _ {n} ^ {L}}}, \text {s . t .} i \neq j, \frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \geq L ^ {3 / 8} \frac {\varepsilon_ {0}}{2 \sqrt {2}},
$$
$$
\forall (i, j) \in \overline {{U _ {n} ^ {L}}} \times U _ {n} ^ {L}, \frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} \leq L ^ {1 / 4} (D _ {1} + 4 W _ {\mu}).
$$
Since  $\mathcal{J}_n^\star \subseteq \overline{V_n^L} \subseteq \overline{U_n^L}$ , for all  $L \geq L_3 \coloneqq \max\{L_0^{4/3}, L_2^{4/3}, L_1^2, \left(\frac{2\sqrt{2}}{\varepsilon_0}(D_1 + 4W_\mu)\right)^8 + 1\}$  we have
$$
\forall (i, k, j) \in \mathcal {J} _ {n} ^ {\star} \times U _ {n} ^ {L} \times \overline {{V _ {n} ^ {L}}}, \text {s . t .} i \neq j, \quad \frac {\mu_ {n , i} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , j}}} > \frac {\mu_ {n , i} - \mu_ {n , k} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i} + 1 / N _ {n , k}}}.
$$
As  $B_n^{\mathrm{EB}} \in \mathcal{I}_n^\star$ , the definition  $C_n^{\mathrm{TC}\varepsilon_0}$  yields that  $C_n^{\mathrm{TC}\varepsilon_0} \in V_n^L$ . Otherwise the above strict inequality would yield a contradiction. Using Lemma 12, we show
$$
\mathbb {E} _ {\boldsymbol {F}} \left[ L _ {3} \right] \leq \mathbb {E} _ {\boldsymbol {F}} \left[ \left(\frac {2 \sqrt {2}}{\varepsilon_ {0}} \left(D _ {1} + 4 W _ {\mu}\right)\right) ^ {8} + 1 \right] + \mathbb {E} _ {\boldsymbol {F}} \left[ \left(L _ {0}\right) ^ {4 / 3} \right] + \mathbb {E} _ {\boldsymbol {F}} \left[ \left(L _ {2}\right) ^ {4 / 3} \right] + \mathbb {E} _ {\boldsymbol {F}} \left[ \left(L _ {1}\right) ^ {2} \right] <   + \infty ,
$$
hence this concludes the proof.
Lemma 18 shows that all arms are sufficiently explored and that the leader is the unique best arm for  $n$  large enough.
Lemma 18. There exist  $N_{1}$  with  $\mathbb{E}_{\nu}[N_1] < +\infty$  such that for all  $n \geq N_{1}$  and all  $i \in [K]$ ,  $N_{n,i} \geq \sqrt{n / K}$  and  $B_{n}^{EB} = i^{\star}$ .
Proof. Let  $L_{2}$  and  $L_{3}$  as in Lemmas 16 and 17. Therefore, for  $L \geq L_{4} := \max \{L_{3}, L_{2}^{4/3}\}$ , for all  $n$  such that  $U_{n}^{L} \neq \emptyset$ ,  $B_{n}^{\mathrm{EB}} \in V_{n}^{L}$  or  $C_{n}^{\mathrm{TC}\varepsilon_{0}} \in V_{n}^{L}$ . We have  $\mathbb{E}_{\nu}[L_{4}] < +\infty$ . There exists a deterministic  $L_{5}$  such that for all  $L \geq L_{5}$ ,  $\lfloor L \rfloor \geq KL^{3/4}$ . Let  $L \geq \max \{L_{5}, L_{4}\}$ .
Suppose towards contradiction that  $U_{[KL\downarrow]}^L$  is not empty. Then, for any  $1 \leq t \leq \lfloor KL \rfloor$ ,  $U_t^L$  and  $V_t^L$  are non-empty as well. Using the pigeonhole principle, there exists some  $i \in [K]$  such that  $N_{[L],i} \geq L^{3/4}$ . Thus, we have  $\left|V_{[L]}^L\right| \leq K - 1$ . Our goal is to show that  $\left|V_{[2L]}^L\right| \leq K - 2$ . A sufficient condition is that one arm in  $V_{[L]}^L$  is pulled at least  $L^{3/4}$  times between  $[L]$  and  $[2L] - 1$ .
If we have
$$
\sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(\left\{B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \right\} \subseteq V _ {t} ^ {L}\right) \geq K L ^ {3 / 4},
$$
then, using that  $V_{t}^{L}\subseteq V_{\lfloor L\rfloor}^{L}$  , we obtain