SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
[ \mathbf {I D S} ] \lim  _ {\delta \rightarrow 0} \sup  _ {\log (1 / \delta)} \frac {\mathbb {E} _ {\nu} [ \tau_ {\varepsilon , \delta} ]}{\log (1 / \delta)} \leq T _ {\varepsilon_ {0}} (\mu) D _ {\varepsilon , \varepsilon_ {0}} (\mu) \quad a n d \quad [ \text {f i x e d} \beta ] \lim  _ {\delta \rightarrow 0} \sup  _ {\log (1 / \delta)} \frac {\mathbb {E} _ {\nu} [ \tau_ {\varepsilon , \delta} ]}{\log (1 / \delta)} \leq T _ {\varepsilon_ {0}, \beta} (\mu) D _ {\varepsilon , \varepsilon_ {0}} (\mu),
$$
where  $D_{\varepsilon, \varepsilon_0}(\mu) = (1 + \max_{i \neq i^{\star}} (\varepsilon_0 - \varepsilon) / (\mu_\star - \mu_i + \varepsilon))^2$ .
While Theorem 1 holds for all sub-Gaussian distributions, it is particularly interesting for Gaussian ones, in light of Lemma 1. When choosing  $\varepsilon = \varepsilon_0$  (i.e.  $D_{\varepsilon_0,\varepsilon_0}(\mu) = 1$ ), Theorem 1 shows that  $\mathrm{EB - TC}_{\varepsilon_0}$  is asymptotically optimal for Gaussian bandits when using IDS proportions and asymptotically  $\beta$ -optimal when using fixed proportions  $\beta$ . We also note that Theorem 1 is not conflicting with the lower bound of Lemma 1, as shown in Lemma 11 in Appendix C. Empirically we observe that the empirical stopping time can be drastically worse when taking  $\varepsilon_0 < \varepsilon$ , and close to the optimal one when  $\varepsilon_0 > \varepsilon$  (Figures 5, 6 and 7 in Appendix J.2.1).
Until recently [40], proving asymptotic optimality of Top Two algorithms with adaptive choice  $\beta$  was an open problem in BAI. In this work, we prove that their choice of IDS proportions also yield asymptotically optimal algorithms for  $\varepsilon$ -BAI. While the proof of Theorem 1 assumes the existence of a unique best arm, it holds for instances having sub-optimal arms with the same mean. This is an improvement compared to existing asymptotic guarantees on Top Two algorithms which rely on the assumption that the means of all arms are different [32, 36, 21]. The improvement is possible thanks to the regularization induced by the slack  $\varepsilon_0 > 0$ .
While asymptotic optimality in the  $\varepsilon$ -BAI setting was already achieved for various algorithms (e.g.  $\varepsilon$ -Track-and-Stop (TaS) [16], Sticky TaS [10] or L $\varepsilon$ BAI [19]), none of them obtained non-asymptotic guarantees. Despite their theoretical interest, asymptotic results provide no guarantee on the performance for moderate  $\delta$ . Furthermore, asymptotic results on Top Two algorithms require a unique best arm regardless of the considered error  $\varepsilon$ : reaching asymptotic ( $\beta$ -)optimality implicitly
means that the algorithm eventually allocates samples in an optimal way that depends on the identity of the unique best arm, and that requires the unique best arm to be identified. As our focus is  $\varepsilon$ -BAI, our guarantees should only require that one of the  $\varepsilon$ -good arms is identified and should hold for instances having multiple best arms. The upper bound should scale with  $\varepsilon_0^{-2}$  instead of  $\Delta_{\mathrm{min}}^{-2}$  when  $\Delta_{\mathrm{min}}$  is small. Theorem 2 satisfies these requirements.
Theorem 2. Let  $\delta \in (0,1)$  and  $\varepsilon_0 > 0$ . Combined with  $GLR_{\varepsilon_0}$  stopping (3), the  $EB-TC_{\varepsilon_0}$  algorithm with fixed  $\beta = 1/2$  is  $(\varepsilon_0, \delta)$ -PAC and satisfies that, for all  $\nu \in \mathcal{D}^K$  with mean  $\mu$ ,
$$
\mathbb {E} _ {\nu} [ \tau_ {\varepsilon_ {0}, \delta} ] \leq \inf _ {\tilde {\varepsilon} \in [ 0, \varepsilon_ {0} ]} \max  \left\{T _ {\mu , \varepsilon_ {0}} (\delta , \tilde {\varepsilon}) + 1, S _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}) \right\} + 2 K ^ {2},
$$
where  $T_{\mu ,\varepsilon_0}(\delta ,\tilde{\varepsilon})$  and  $S_{\mu ,\varepsilon_0}(\tilde{\varepsilon})$  are defined by
$$
T _ {\mu , \varepsilon_ {0}} (\delta , \tilde {\varepsilon}) = \sup  \left\{n \mid n - 1 \leq 2 (1 + \gamma) ^ {2} \sum_ {i \in \mathcal {I} _ {\tilde {\varepsilon}} (\mu)} T _ {\varepsilon_ {0}, 1 / 2} (\mu , i) \left(\sqrt {c (n - 1 , \delta)} + \sqrt {4 \log n}\right) ^ {2} \right\},
$$
$$
S _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}) = h _ {1} \left(\frac {1 6 (1 + \gamma^ {- 1})}{a _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon})} H _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}), \frac {(1 + \gamma^ {- 1}) K ^ {2}}{a _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon})} + 1\right),
$$
$$
a _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}) = \frac {\min _ {i \in \mathcal {I} _ {\tilde {\varepsilon}} (\mu)} T _ {\varepsilon_ {0} , 1 / 2} (\mu , i)}{\sum_ {i \in \mathcal {I} _ {\tilde {\varepsilon}} (\mu)} T _ {\varepsilon_ {0} , 1 / 2} (\mu , i)} \min _ {i \in \mathcal {I} _ {\tilde {\varepsilon}} (\mu), j \neq i} w _ {\varepsilon_ {0}, 1 / 2} (\mu , i) _ {j},
$$
where  $\gamma \in (0,1 / 2]$  is an analysis parameter and  $h_1(y,z)\approx z + y\log (z + y\log (y))$  as in Lemma 51.  $T_{\varepsilon_0,1 / 2}(\mu ,i)$  and  $w_{\varepsilon_0,1 / 2}(\mu ,i)$  are defined in (1) and
$$
H _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}) := \frac {2 | i ^ {\star} (\mu) |}{\Delta_ {\mu} (\tilde {\varepsilon}) ^ {2}} + (| \mathcal {I} _ {\tilde {\varepsilon}} (\mu) \backslash i ^ {\star} (\mu) |) C _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}) ^ {2} + \sum_ {i \notin \mathcal {I} _ {\tilde {\varepsilon}} (\mu)} \max  \left\{C _ {\mu , \varepsilon_ {0}} (\tilde {\varepsilon}), \sqrt {2} \Delta_ {i} ^ {- 1} \right\} ^ {2}, \tag {5}
$$
with  $\Delta_{\mu}(\tilde{\varepsilon}) = \min_{k\notin \mathcal{I}_{\varepsilon}(\mu)}\Delta_{k}$  and  $C_{\mu ,\varepsilon_0}(\tilde{\varepsilon}) = \max \{2\Delta_{\mu}(\tilde{\varepsilon})^{-1} - \varepsilon_0^{-1},\varepsilon_0^{-1}\}$ .
The upper bound on  $\mathbb{E}_{\nu}[\tau_{\varepsilon_0,\delta}]$  involves a  $\delta$ -dependent term  $T_{\mu, \varepsilon_0}(\delta, \tilde{\varepsilon})$  and a  $\delta$ -independent term  $S_{\mu, \varepsilon_0}(\tilde{\varepsilon})$ . The choice of  $\tilde{\varepsilon}$  influences the compromise between those, and the infimum over  $\tilde{\varepsilon}$  means that our algorithm benefits from the best possible trade-off. In the asymptotic regime, we take  $\tilde{\varepsilon} = 0$  and  $\gamma \to 0$  and we obtain  $\lim_{\delta \to 0} \mathbb{E}_{\nu}[\tau_{\varepsilon_0,\delta}] / \log(1/\delta) \leq 2|i^{\star}(\mu)|T_{\varepsilon_0,1/2}(\mu)$ . When  $|i^{\star}(\mu)| = 1$ , we recover the asymptotic result of Theorem 1 up to a multiplicative factor 2. For multiple best arms, the asymptotic sample complexity is at most a factor  $2|i^{\star}(\mu)|$  from the  $\beta$ -optimal one.
Given a finite confidence, the dominant term will be  $S_{\mu,\varepsilon_0}(\tilde{\varepsilon})$ . For  $\tilde{\varepsilon} = 0$ , we show that  $H_{\mu,\varepsilon_0}(0) = \mathcal{O}(K\min\{\Delta_{\min}, \varepsilon_0\}^{-2})$ , hence we should consider  $\tilde{\varepsilon} > 0$  to avoid the dependency in  $\Delta_{\min}$ . For  $\tilde{\varepsilon} = \varepsilon_0$ , there exists instances such that  $\max_{i \in \mathcal{I}_{\varepsilon_0}(\mu)} T_{\varepsilon_0,1/2}(\mu, i)$  is arbitrarily large, hence  $S_{\mu,\varepsilon_0}(\varepsilon_0)$  will be very large as well. The best trade-off is attained in the interior of the interval  $(0, \varepsilon_0)$ . For  $\tilde{\varepsilon} = \varepsilon_0/2$ , Lemma 10 shows that  $T_{\varepsilon_0,1/2}(\mu, i) = \mathcal{O}(K/\varepsilon_0^2)$  for all  $i \in \mathcal{I}_{\varepsilon_0/2}(\mu)$  and  $H_{\mu,\varepsilon_0}(\varepsilon_0/2) = \mathcal{O}(K/\varepsilon_0^2)$ . Therefore, we obtain an upper bound  $\mathcal{O}(|\mathcal{I}_{\varepsilon_0/2}(\mu)| K\varepsilon_0^{-2}\log\varepsilon_0^{-1})$ .
Likewise, Lemma 10 shows that  $\min_{j\neq i}w_{\varepsilon_0,1 / 2}(\mu ,i)_j\geq (16(K - 2) + 2)^{-1}$  for all  $i\in \mathcal{I}_{\varepsilon_0 / 2}(\mu)$ . While the dependency in  $a_{\mu ,\varepsilon_0}(\varepsilon_0 / 2)$  is milder in  $\varepsilon$ -BAI than in BAI (as it is bounded away from 0), we can improve it by using a refined analysis (see Appendix E). Introduced in [20], this method allows to clip  $\min_{j\neq i}w_{\varepsilon_0,1 / 2}(\mu ,i)_j$  by a fixed value  $x\in [0,(K - 1)^{-1}]$  for all  $i\in \mathcal{I}_{\hat{\varepsilon}}(\mu)$ .
Comparison with existing upper bounds The LUCB algorithm [24] has a structure similar to a Top Two algorithm, with the differences that LUCB samples both the leader and the challenger and that it stops when the gap between the UCB and LCB indices is smaller than  $\varepsilon_0$ . As LUCB satisfies  $\mathbb{E}_{\mu}[\tau_{\varepsilon_0,\delta}] \leq 292H_{\varepsilon_0}(\mu)\log (H_{\varepsilon_0}(\mu) / \delta) + 16$  where  $H_{\varepsilon_0}(\mu) = \sum_i(\max \{\Delta_i,\varepsilon_0 / 2\})^{-2}$ , it enjoys better scaling than EB-TC for finite confidence. However, since the empirical allocation of LUCB is not converging towards  $w_{\varepsilon_0,1 / 2}(\mu)$ , it is not asymptotically 1/2-optimal. While LUCB has better moderate confidence guarantees, there is no hope to prove anytime performance bounds since LUCB indices depend on  $\delta$ . In contrast, EB-TC enjoys such guarantees (see Section 4).
Key technical tool for the non-asymptotic analysis We want to ensure that  $\mathrm{EB - TC}_{\varepsilon_0}$  eventually selects only  $\varepsilon$ -good arms as leader, for any error  $\varepsilon \geq 0$ . Our proof strategy is to show that if the leader is not an  $\varepsilon$ -good arm and empirical means do not deviate too much from the true means, then either the current leader or the current challenger was selected as leader or challenger less than a given quantity. We obtain a bound on the total number of times that can happen.
Lemma 4. Let  $\delta \in (0,1]$  and  $n > K$ . Let  $T_{n}(i)\coloneqq \sum_{j\neq i}(T_{n}(i,j) + T_{n}(j,i))$  be the number of times arm  $i$  was selected in the leader/challenger pair. Assume there exists a sequence of events  $(A_{t}(n,\delta))_{n\geq t > K}$  and positive reals  $(D_{i}(n,\delta))_{i\in [K]}$  such that, for all  $t\in \{K + 1,\dots ,n\}$ , under the event  $A_{t}(n,\delta)$ ,
$$
\exists i _ {t} \in [ K ], \quad T _ {t} (i _ {t}) \leq D _ {i _ {t}} (n, \delta) \quad a n d \quad T _ {t + 1} (i _ {t}) = T _ {t} (i _ {t}) + 1. \tag {6}
$$
Then, we have  $\sum_{t = K + 1}^{n}\mathbb{1}(A_t(n,\delta))\leq \sum_{i\in [K]}D_i(n,\delta)$
To control the deviation of the empirical means and empirical gaps to their true value, we use a sequence of concentration events  $(\mathcal{E}_{n,\delta})_{n > T}$  defined in Lemma 45 (Appendix G.2) such that  $\mathbb{P}_{\nu}(\mathcal{E}_{n,\delta}^{\mathbb{C}})\leq K^{2}\delta n^{-s}$  where  $s\geq 0$  and  $\delta \in (0,1]$ . For the EB-TC  $\varepsilon_0$  algorithm with fixed  $\beta = 1 / 2$  we prove that, under  $\mathcal{E}_{n,\delta}$ ,  $\{B_t^{\mathrm{EB}}\notin \mathcal{I}_\varepsilon (\mu)\}$  is a "bad" event satisfying the assumption of Lemma 4. This yields Lemma 5, which essentially says that the leader is an  $\varepsilon$ -good arm except for a logarithmic number of rounds.
Lemma 5. Let  $\delta \in (0,1]$ ,  $n > K$  and  $\varepsilon \geq 0$ . Under the event  $\mathcal{E}_{n,\delta}$ , we have
$$
\sum_ {i \in \mathcal {I} _ {\varepsilon} (\mu)} \sum_ {j} T _ {n} (i, j) \geq n - 1 - 8 H _ {\mu , \varepsilon_ {0}} (\varepsilon) f _ {2} (n, \delta) - 3 K ^ {2},
$$
where  $f_{2}(n,\delta) = \log (1 / \delta) + (2 + s)\log n$  and  $H_{\mu ,\varepsilon_0}(\varepsilon)$  is defined in (5).
Noticeably, Lemma 5 holds for any  $\varepsilon \geq 0$  even when there are multiple best arms. As expected the number of times the leader is not among the  $\varepsilon_0$ -good arms depends on  $H_{\mu,\varepsilon_0}(\varepsilon_0) = \mathcal{O}(K/\varepsilon_0^2)$ . The number of times the leader is not among the best arms depends on  $H_{\mu,\varepsilon_0}(0) = \mathcal{O}(K(\min\{\Delta_{\min}, \varepsilon_0\})^{-2})$ .
Time-varying slack Theorem 1 shows the asymptotic optimality of the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm with IDS for  $\varepsilon_0$  -BAI (where  $\varepsilon_0 > 0$ ). To obtain optimality for BAI, we consider time-varying slacks  $(\varepsilon_{n})_{n}$  where  $(\varepsilon_{n})_{n}$  is decreasing,  $\varepsilon_{n} > 0$  and  $\varepsilon_{n}\rightarrow +\infty$  0. A direct adaptation of our asymptotic analysis on  $\mathbb{E}_{\nu}[\tau_{0,\delta}]$  (see Appendix D), regardless of the choice of  $(\varepsilon_{n})_{n}$ , one can show that using  $\mathrm{GLR}_0$  stopping, the  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  algorithm with IDS is  $(0,\delta)$ -PAC and is asymptotically optimal in BAI. Its empirical performance is however very sensitive to the choice of  $(\varepsilon_{n})_{n}$  (Appendix J.2.3).
4 Beyond fixed-confidence guarantees
Could an algorithm analyzed in the fixed-confidence setting be used for the fixed-budget or even anytime setting? This question is especially natural for  $\mathrm{EB - TC}_{\varepsilon_0}$ , which does not depend on the confidence parameter  $\delta$ . Yet its answer is not obvious, as it is known that algorithms that have optimal asymptotic guarantees in the fixed-confidence setting can be sub-optimal in terms of error probability. Indeed [28] prove in their Appendix C that for any asymptotically optimal (exact) BAI algorithm, there exists instances in which the error probability cannot decay exponentially with the horizon, which makes them worse than the (minimax optimal) uniform sampling strategy [5].
Their argument also applies to  $\beta$ -optimal algorithms, hence to  $\mathrm{EB - TC_0}$  with  $\beta = 1 / 2$ . However, whenever  $\varepsilon_0$  is positive, Theorem 3 reveals that the error probability of  $\mathrm{EB - TC\varepsilon_0}$  always decays exponentially, which redeems the use of optimal fixed-confidence algorithms for a relaxed BAI problem in the anytime setting. Going further, this result provides an anytime bound on the probability to recommend an arm that is not  $\varepsilon$ -optimal, for any error  $\varepsilon \geq 0$ . This bound involves instance-dependent complexities depending solely on the gaps in  $\mu$ . To state it, we define  $C_\mu \coloneqq |\{\mu_i \mid i \in [K]\}|$  as the number of distinct arm means in  $\mu$  and let  $\mathcal{C}_{\mu}(i) \coloneqq \{k \in [K] \mid \mu_{\star} - \mu_k = \Delta_i\}$  be the set of arms having mean gap  $\Delta_i$  where the gaps are sorted by increasing order  $0 = \Delta_1 < \Delta_2 < \dots < \Delta_{C_\mu}$ . For all  $\varepsilon \geq 0$ , let  $i_\mu(\varepsilon) = i$  if  $\varepsilon \in [\Delta_i, \Delta_{i+1})$  (with the convention  $\Delta_{C_\mu + 1} = +\infty$ ). Theorem 3 shows that the exponential decrease of  $\mathbb{P}_\nu(\hat{i}_n \notin \mathcal{I}_\varepsilon(\mu))$  is linear.
Theorem 3. (see Theorem 6 in Appendix F) Let  $\varepsilon_0 > 0$ . The  $EB-TC_{\varepsilon_0}$  algorithm with fixed proportions  $\beta = 1/2$  satisfies that, for all  $\nu \in \mathcal{D}^K$  with mean  $\mu$ , for all  $\varepsilon \geq 0$ , for all  $n > 5K^2/2$ ,
$$
\mathbb {P} _ {\nu} \left(\hat {i} _ {n} \notin \mathcal {I} _ {\varepsilon} (\mu)\right) \leq K ^ {2} e ^ {2} (2 + \log n) ^ {2} \exp \left(- p \left(\frac {n - 5 K ^ {2} / 2}{8 H _ {i _ {\mu} (\varepsilon)} (\mu , \varepsilon_ {0})}\right)\right).
$$
where  $p(x) = x - \log x$  and  $(H_{i}(\mu ,\varepsilon_{0}))_{i\in [C_{\mu} - 1]}$  are such that  $H_{1}(\mu ,\varepsilon_{0}) = K(2\Delta_{\mathrm{min}}^{-1} + 3\varepsilon_{0}^{-1})^{2}$  and  $K / \Delta_{i + 1}^{-2}\leq H_i(\mu ,\varepsilon_0)\leq K\min_{j\in [i]}\max \{2\Delta_{j + 1}^{-1},2\frac{\Delta_j / \varepsilon_0 + 1}{\Delta_{i + 1} - \Delta_j} +3\varepsilon_0^{-1}\} ^2$  for all  $i > 1$
This bound can be compared with the following uniform  $\varepsilon$ -error bound of the strategy using uniform sampling and recommending the empirical best arm:
$$
\mathbb {P} \left(\hat {\imath} _ {n} ^ {\mathrm {U}} \notin \mathcal {I} _ {\varepsilon} (\mu)\right) \leq \sum_ {i \notin \mathcal {I} _ {\varepsilon} (\mu)} \exp \left(- \frac {\Delta_ {i} ^ {2} \lfloor n / K \rfloor}{4}\right) \leq K \exp \left(- \frac {n - K}{4 K \Delta_ {i _ {\mu} (\varepsilon) + 1} ^ {- 2}}\right)
$$
Recalling that the quantity  $H_{i}(\mu, \varepsilon_{0})$  in Theorem 3 is always bounded from below by  $2K\Delta_{i+1}^{-1}$ , we get that our upper bound is larger than the probability of error of the uniform strategy, but the two should be quite close. For example for  $\varepsilon = 0$ , we have
$$
\mathbb {P} _ {\nu} \left(\hat {i} _ {n} \notin i ^ {\star} (\mu)\right) \leq \exp \left(- \Theta \left(\frac {n}{K (\Delta_ {\min} ^ {- 1} + \varepsilon_ {0} ^ {- 1}) ^ {2}}\right)\right), \mathbb {P} _ {\nu} \left(\hat {i} _ {n} ^ {\mathrm {U}} \notin i ^ {\star} (\mu)\right) \leq \exp \left(- \Theta \left(\frac {n}{K \Delta_ {\min} ^ {- 2}}\right)\right).
$$
Even if they provide a nice sanity-check for the use of a sampling rule with optimal fixed-confidence guarantees for  $\varepsilon_0$ -BAI in the anytime regime, we acknowledge that these guarantees are far from optimal. Indeed, the work of [41] provides tighter anytime uniform  $\varepsilon$ -error probability bounds for two algorithms: an anytime version of Sequential Halving [25] using a doubling trick (called DSH), and an algorithm called Bracketting Sequential Halving, that is designed to tackle a very large number of arms. Their upper bounds are of the form  $\mathbb{P}_{\nu}\left(\hat{i}_n \notin \mathcal{I}_{\varepsilon}(\mu)\right) \leq \exp\left(-\Theta\left(n / H(\varepsilon)\right)\right)$  with  $H(\varepsilon) = \frac{1}{g(\varepsilon / 2)} \max_{i \geq g(\varepsilon) + 1} \frac{i}{\Delta_i^2}$  where  $g(\varepsilon) = |\{i \in [K] \mid \mu_i \geq \mu^\star - \varepsilon\}|$ . Therefore, they can be much smaller than  $K \Delta_{i_\mu(\varepsilon) + 1}^{-2}$ .
The BUCB algorithm of [26] is also analyzed for any level of error  $\varepsilon$ , but in a different fashion. The authors provide bounds on its  $(\varepsilon, \delta)$ -unverifiable sample complexity, defined as the expectation of the smallest stopping time  $\tilde{\tau}$  satisfying  $\mathbb{P}(\forall t \geq \tilde{\tau}, \hat{i}_n \in \mathcal{I}_{\varepsilon}(\mu)) \geq 1 - \delta$ . This notion is different from the sample complexity we use in this paper, which is sometimes called verifiable since it is the time at which the algorithm can guarantee that its error probability is less than  $\delta$ . Interestingly, to prove Theorem 3 we first prove a bound on the unverifiable sample complexity of  $\mathrm{EB-TC}_{\varepsilon_0}$  which is valid for all  $(\varepsilon, \delta)$ , neither of which are parameters of the algorithm. More precisely, we prove that  $\mathbb{P}_{\nu}(\forall n > U_{i_{\mu}(\varepsilon)}, \delta(\mu, \varepsilon_0), \hat{i}_n \in \mathcal{I}_{\varepsilon}(\mu)) \geq 1 - \delta$  for  $U_{i, \delta}(\mu, \varepsilon_0) =_{\delta \to 0} 8H_i(\mu, \varepsilon_0) \log(1/\delta) + O(\log \log(1/\delta))$ . As this statement is valid for all  $\delta \in (0, 1)$ , applying it for each  $n$  to  $\delta_n$  such that  $U_{i_{\mu}(\varepsilon), \delta_n}(\mu, \varepsilon_0) = n$ , we obtain Theorem 3. We remark that such a trick cannot be applied to BUCB to get uniform  $\varepsilon$ -error bounds for any time, as the algorithm does depend on  $\delta$ .
Simple regret As already noted by [41], uniform  $\varepsilon$ -error bounds easily yield simple regret bounds. We state in Corollary 1 the one obtained for  $\mathrm{EB - TC}_{\varepsilon_0}$ . As a motivation to derive simple regret bounds, we observe that they readily translate to bounds on the cumulative regret for an agent observing the stream of recommendations  $(\hat{i}_n)$  and playing arm  $\hat{i}_n$ . An exponentially decaying simple regret leads to a constant cumulative regret in this decoupled exploration/exploitation setting [2, 33].
Corollary 1. Let  $\varepsilon_0 > 0$ . Let  $p(x)$  and  $(H_i(\mu, \varepsilon_0))_{i \in [C_\mu - 1]}$  be defined as in Theorem 3. The  $EB-TC_{\varepsilon_0}$  algorithm with fixed  $\beta = 1/2$  satisfies that, for all  $\nu \in \mathcal{D}^K$  with mean  $\mu$ , for all  $n > 5K^2/2$ ,
$$
\mathbb {E} _ {\nu} \left[ \mu_ {\star} - \mu_ {i _ {n}} \right] \leq K ^ {2} e ^ {2} (2 + \log n) ^ {2} \sum_ {i \in [ C _ {\mu} - 1 ]} \left(\Delta_ {i + 1} - \Delta_ {i}\right) \exp \left(- p \left(\frac {n - 5 K ^ {2} / 2}{8 H _ {i} (\mu , \varepsilon_ {0})}\right)\right).
$$
Following the discussion above, this bound is not expected to be state-of-the-art, it rather justifies that  $\mathrm{EB - TC}_{\varepsilon_0}$  with  $\varepsilon_0 > 0$  is not too much worse than the uniform sampling strategy. Yet, as we shall see in our experiments, the practical story is different. In Section 5, we compare the simple regret of  $\mathrm{EB - TC}_{\varepsilon_0}$  to that of DSH in synthetic experiments with a moderate number of arms, revealing the superior performance of  $\mathrm{EB - TC}_{\varepsilon_0}$ .
We assess the performance of the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm on Gaussian instances both in terms of its empirical stopping time and its empirical simple regret, and we show that it performs favorably compared to existing algorithms in both settings. For the sake of space, we only show the results for large sets of arms and for a specific instance with  $|i^{\star}(\mu)| = 2$
Figure 2: (a) Empirical stopping time on “ $\alpha = 0.3$ ” instances for varying  $K$  and stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$ . The BAI algorithms T3C, EB-TCI and TTUCB are modified to be  $\varepsilon$ -BAI ones. (b) Empirical simple regret on instance  $\mu = (0.6, 0.6, 0.55, 0.45, 0.3, 0.2)$ , in which  $\mathrm{EB - TC}_{\varepsilon_0}$  with slack  $\varepsilon_0 = 0.1$  and fixed  $\beta = 1/2$  is used.
Empirical stopping time We study the impact of large sets of arms (up to  $K = 1000$ ) in  $\varepsilon$ -BAI for  $(\varepsilon, \delta) = (0.1, 0.01)$  on the "  $\alpha = 0.3$ " scenario of [18] which sets  $\mu_i = 1 - ((i - 1) / (K - 1))^{\alpha}$  for all  $i \in [K]$ . EB-TC  $\varepsilon_0$  with IDS and slack  $\varepsilon_0 = \varepsilon$  is compared to existing  $\varepsilon$ -BAI algorithms having low computational cost. This precludes algorithms such as  $\varepsilon$ -Track-and-Stop (TaS) [16], Sticky TaS [10] or  $\varepsilon$ -BAI adaptation of FWS [39] and DKM [11]. In Appendix J.2.2, we compare EB-TC  $\varepsilon$  to those algorithms on benchmarks with smaller number of arms. We show that EB-TC  $\varepsilon$  performs on par with  $\varepsilon$ -TaS and  $\varepsilon$ -FWS, but outperforms  $\varepsilon$ -DKM. As Top Two benchmarks with fixed  $\beta = 1/2$ , we consider T3C [36], EB-TCI [21] and TTUCB [20]. To provide a fair comparison, we adapt them to tackle  $\varepsilon$ -BAI by using the stopping rule (3) and by adapting their sampling rule to use the TC  $\varepsilon$  challenger from (2) (with a penalization  $\log N_{n,i}$  for EB-TCI). We use the heuristic threshold  $c(n, \delta) = \log((1 + \log n)/\delta)$ . While it is too small to ensure the  $(\varepsilon, \delta)$ -PAC property, it still yields an empirical error which is several orders of magnitude lower than  $\delta$ . Finally, we compare with LUCB [24] and uniform sampling. For a fair comparison, LUCB uses  $\sqrt{2c(n - 1, \delta)/N_{n,i}}$  as a bonus, which is also too small to yield valid confidence intervals. Our results are averaged over 100 runs, and the standard deviations are displayed. In Figure 2(a), we see that EB-TC  $\varepsilon$  performs on par with the  $\varepsilon$ -T3C heuristic, and significantly outperforms the other algorithms. While the scaling in  $K$  of  $\varepsilon$ -EB-TCI and LUCB appears to be close to the one of EB-TC  $\varepsilon$ ,  $\varepsilon$ -TTUCB and uniform sampling obtain a worse one. Figure 2(a) also reveals that the regularization ensured by the TC  $\varepsilon$  challenger is sufficient to ensure enough exploration, hence other exploration mechanisms are superfluous (TS/UCB leader or TCI challenger).