SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The right column of Figure 6 reveals that the choice of  $\varepsilon_0$  has few impact on the empirical simple regret, at least for reasonably chosen  $\varepsilon_0$ . Likewise, there is few differences between IDS proportions and fixed  $\beta = 1/2$ . The good empirical performance of IDS proportions as regards the simple regret hints that  $\mathrm{EB-TC}_{\varepsilon_0}$  with IDS has (most likely) similar theoretical guarantees as the ones obtained in Section 4 for  $\mathrm{EB-TC}_{\varepsilon_0}$  with fixed  $\beta = 1/2$  (Theorem 3 and Corollary 1). This is an interesting open problem that we leave for future work.
Two-groups instances We conclude our sensitivity analysis on  $\mathrm{EB - TC}_{\varepsilon_0}$  by considering the "two-groups" instances  $\mu \in \{0.6, 0.4\}^{10}$  for varying number of best arms, i.e.  $|i^{\star}(\mu)| \in [8]$ . Our aim is to assess the impact of having multiple best arms on the empirical stopping time. We average on 1000 runs, and the standard deviations are displayed.
Figure 7: Empirical stopping time with stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$  on instances  $\mu \in \{0.6, 0.4\}^{10}$  for varying  $|i^{\star}(\mu)|$ . "c-" denotes fixed  $\beta = 1/2$ , without refers to IDS.
On this specific instances where the number of arms is fixed, it would be intuitive to think that the problem is easier when there are more best arms. However, this is actually the opposite since there less bad arms (which were easy to detect) and more good arms which need to be estimated well enough. While Figure 7 shows how poor the performances are when taking  $\varepsilon_0 < \varepsilon$ , it also shows that the impact of having multiple best arms is rather mild for slacks  $\varepsilon_0 \geq \varepsilon$ . We will see in Figure 12 that  $\mathrm{EB - TC}_{\varepsilon_0}$  is the most resilient algorithm with respect to multiple arms, meaning that its slope is the smallest.
"Two-groups" instances are interesting since they are the instances for which the ratio  $T_{\varepsilon,1/2}(\mu) / T_{\varepsilon}(\mu)$  is the largest. Using Lemma 9 and the theoretical results of [20] (see their Lemma C.6), we conjecture that  $T_{\varepsilon,1/2}(\mu) \leq r_K T_{\varepsilon}(\mu)$  with  $r_K = 2K / (1 + \sqrt{K - 1})^2$  and that the equality occurs for "two-groups" instances. Since  $r_{10} = 5/4$ , using fixed  $\beta = 1/2$  should be roughly  $25\%$  slower than using IDS. While Figure 7 confirms that using IDS yields better empirical performance than using fixed  $\beta = 1/2$ , the empirical ratio between their performance is lower than the one suggested by the theory.
Table 4: Comparison between the original sampling rule and the modified sampling rule for BAI algorithms when combined with the  $\mathrm{GLR}_{\varepsilon}$  stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$ . We display the empirical stopping time (and standard deviation) on random instances with  $K \in \{5, 10, 20\}$ .
K	T3C	ε-T3C	EB-TCI	ε-EB-TCI
5	9138 (±7988)	2259 (±1243)	22505 (±66624)	2290 (±1202)
10	17418 (±9726)	3793 (±1524)	58086 (±135655)	3975 (±1509)
20	30771 (±13038)	6631 (±1992)	115861 (±195797)	6905 (±1879)
J.2.2 Comparison with modified BAI algorithms
Appendix J.2.2 is meant to provide further empirical evidence that  $\mathrm{EB - TC}_{\varepsilon_0}$  outperforms its competitors on different tasks and for different types of instances. Overall, we will use the same instances as used in Appendix J.2.1 since they provide a wide range of interesting problems. Throughout this section, we will be using  $\mathrm{EB - TC}_{\varepsilon_0}$  with slack  $\varepsilon_0 = 0.1$ . While IDS proportions are used for the experiments on the empirical stopping time, we use fixed  $\beta = 1/2$  for the empirical simple regret.
Figure 8: Empirical stopping time on (a) “ $\alpha = 0.6$ ” instances and (b) “sparse” instances for varying  $K$  and stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$ . The BAI algorithms T3C, EB-TCI and TTUCB are modified to be  $\varepsilon$ -BAI ones.
Figure 9: Empirical stopping time on random instances with stopping rule (3) using (left)  $(\varepsilon, \delta) = (0.05, 0.01)$  and (right)  $(\varepsilon, \delta) = (0.1, 0.01)$  with (top)  $K = 5$ , (middle)  $K = 10$  and (bottom)  $K = 20$ . The BAI algorithms T3C, EB-TCI, TTUCB, TaS, FWS, DKM are modified for  $\varepsilon$ -BAI.
As benchmarks we consider modified BAI which are tailored to tackle  $\varepsilon$ -BAI, and we combine them with the  $\mathrm{GLR}_{\varepsilon}$  stopping rule. One can still wonder whether the unmodified sampling rule would perform well for  $\varepsilon$ -BAI when combined with the  $\mathrm{GLR}_{\varepsilon}$  stopping rule. While we could hope that modifying the stopping rule is enough, our experiments reveal that it is not the case. If we don't adapt the sampling rule as well, Table 4 shows that the empirical stopping time can be more than ten times larger than their modified version.
Large sets of arms To complement Figure 2(a), we consider the “ $\alpha = 0.6$ ” instances and “sparse” instances for varying  $K$ . We average on 100 runs, and the standard deviations are displayed.
Figure 8 confirms the observations made in Figure 2(a). We see that  $\mathrm{EB - TC}_{\varepsilon}$  performs on par with the  $\varepsilon$ -T3C, and outperforms the other algorithms. It also highlights that the regularization ensured by the  $\mathrm{TC}_{\varepsilon}$  challenger is sufficient to ensure enough exploration. As a consequence, other exploration mechanism are superfluous when using Top Two algorithms for  $\varepsilon$ -BAI (e.g. using TS/UCB leader or TCI challenger).
Random instances For  $\varepsilon \in \{0.05, 0.1\}$ , we assess the performance on 1000 random Gaussian instances with  $K \in \{5, 10, 20\}$  as described above. We display the boxplots of the empirical stopping time on 1000 runs.
Figure 9 confirms that  $\mathrm{EB - TC}_{\varepsilon_0}$  performs on par with the state-of-the-art  $\varepsilon$ -BAI algorithms and greatly outperform  $\varepsilon$ -DKM, LUCB and uniform sampling. Interestingly, for larger sets of arms  $(K = 20)$ ,  $\mathrm{EB - TC}_{\varepsilon_0}$  appears to be more robust than its competitors, closely followed by  $\varepsilon$ -T3C and  $\varepsilon$ -EB-TCI. The performance  $\varepsilon$ -TTUCB,  $\varepsilon$ -TaS and  $\varepsilon$ -FWS is slightly worse, while the one of  $\varepsilon$ -DKM is greatly impacted. LUCB seems relatively robust to larger sets of arms, but it is still significantly worse than  $\mathrm{EB - TC}_{\varepsilon_0}$ . It is even the best when considering slack  $\varepsilon_0 = \varepsilon$  in the bottom row of Figure 9. The slightly worse (but still competitive) empirical performance of  $\mathrm{EB - TC}_{\varepsilon_0}$  in the top row of Figure 9 is explained by the slack  $\varepsilon_0 = 0.1$  which differs from the error  $\varepsilon = 0.05$  considered in the GLR $_{\varepsilon}$  stopping rule.
Specific instances To complement Figure 2(b), we consider the two other instances from Table 3. We display the boxplots of the empirical stopping time on 1000 runs, and the empirical simple regret on 10000 runs (with associated standard deviation).
The left column of Figure 10 validates the previous conclusions as regards the good empirical performance of  $\mathrm{EB - TC}_{\varepsilon_0}$  compare to existing  $\varepsilon$ -BAI algorithms. We also see that it seems to perform even better when there are multiple  $\varepsilon$ -good arms and multiple best arms, i.e.  $|i^{\star}(\mu)| > 1$ . The right column of Figure 10 corroborates the observations made in Figure 2(b):  $\mathrm{EB - TC}_{\varepsilon_0}$  outperforms uniform sampling, as well as DSR and DSH.
Two groups We consider the "two-groups" instances  $\mu \in \{0.6, 0.4\}^{10}$  for varying number of best arms, i.e.  $|i^{\star}(\mu)| \in [8]$ . We average on 1000 runs, and the standard deviations are displayed.
Figure 11 reveals that  $\mathrm{EB - TC}_{\varepsilon_0}$  is the most robust to increased number of best arms, i.e. its slope is the smallest. The good performance of  $\mathrm{EB - TC}_{\varepsilon_0}$  is closely followed by other Top Two algorithms. This confirms that additional regularization mechanisms (e.g. TS/UCB leader or TCI challenger) are superfluous since the  $\mathrm{TC}\varepsilon$  challenger is enough to ensure sufficient exploration.  $\varepsilon$ -TaS and  $\varepsilon$ -FWS have slightly worse slope, and the one of  $\varepsilon$ -DKM seems to become less steep after a large initial increase. The highest slope is achieved by LUCB. Quite surprisingly, uniform sampling seems to reach a plateau after which it is scaling better than  $\mathrm{EB - TC}_{\varepsilon_0}$  (in terms of slope), while still having worse empirical performance. It would be interesting to quantify theoretically the good scaling of  $\mathrm{EB - TC}_{\varepsilon_0}$  with respect to increased number of best arms.
Figure 10: (Left) Empirical stopping time for the stopping rule (3) using  $\delta = 0.01$  and (right) empirical simple regret on instances (top)  $\mu_{1}$  with  $\varepsilon = 0.1$ , (middle)  $\mu_{2}$  with  $\varepsilon = 0.15$  and (bottom)  $\mu_{3}$  with  $\varepsilon = 0.1$ . The BAI algorithms T3C, EB-TCI, TTUCB, TaS, FWS, DKM are modified to be  $\varepsilon$ -BAI ones.
Figure 11: Empirical stopping time with stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$  on instances  $\mu \in \{0.6, 0.4\}^{10}$  for varying  $|i^{\star}(\mu)|$ . The BAI algorithms T3C, EB-TCI, TTUCB, TaS, FWS, DKM are modified to be  $\varepsilon$ -BAI ones.
Figure 12: Empirical simple regret on instances  $\mu \in \{0.6, 0.4\}^{10}$  for  $|i^{\star}(\mu)| \in [6]$ , i.e. (top left)  $|i^{\star}(\mu)| = 1$  and (bottom right)  $|i^{\star}(\mu)| = 6$ .
Figure 12 provides additional empirical validation that  $\mathrm{EB - TC}_{\varepsilon_0}$  outperforms uniform sampling as well as DSR and DSH in terms of empirical simple regret. While the empirical stopping time increased with the number of best arms, Figure 12 confirms the intuition that the problem gets easier with respect to the simple regret. First, the range of empirical simple regret becomes smaller with increased number of best arms. This was expected as we have more chances to recommend one of the best arms at early stage even though this recommendation is close to random. Second, we observe a steeper decrease of the empirical simple regret with increased number of best arms. Likewise, this is a natural phenomenon since the collected data will reveal that there multiple good options very fast.
Quite surprisingly,  $\mathrm{EB - TC}_{\varepsilon_0}$  is still better than DSH when the number of best arms increases. To understand why it is surprising, we recall that the exponential decrease of DSH's probability of error is linear with a rate proportional to a hardness constant  $H_{\mathrm{DSH}}(\mu)$ . By definition,  $H_{\mathrm{DSH}}(\mu)$  gets small when there are multiple best arms, hence we would expect to observe lower empirical simple regret (i.e. a "speed-up"). In contrast, our hardness constant  $H_{1}(\mu ,\varepsilon_{0}) = K(2\Delta_{\min}^{-1} + 3\varepsilon_{0}^{-1})^{2}$  does not enjoy such property. Therefore, we would expect that DSH will outperform  $\mathrm{EB - TC}_{\varepsilon_0}$  when the number of best arms increases. Theoretically proving that  $\mathrm{EB - TC}_{\varepsilon_0}$  has a better scaling than the one given in Theorem 3 is still an open problem. Solving it would allow to better understand its good empirical performance on this task.
Fixed-budget performance In addition, we compare the performance of  $\mathrm{EB - TC}_{\varepsilon_0}$  with  $\varepsilon_0$  and  $\beta = 1 / 2$  to the one of SR and SH. Since SR and SH are fixed-budget algorithms, we ran a different instance for each budget  $T$ . Therefore, this comparison gives an unfair advantage to the fixed-budget
algorithms which fully leverage the knowledge of  $T$  but have no theoretical guarantees at time  $n \neq T$  (even at time  $T \pm 1$ ). As a result, the empirical performances of SR and SH are better than the ones of DSR and DSH. We consider the instances from Table 3, as well as "two-groups" instances.
Figure 13: (Left) Empirical simple regret and (right) empirical error on instances (top to bottom)  $\mu_{3} =$ $(0.6,0.6,0.55,0.45,0.3,0.2)$ ,  $\mu_{2} = (0.8,0.75,0.7,0.6,0.5,0.4)$ ,  $\mu_{1} = (0.7,0.55,0.5,0.4,0.2)$  and  $\mu \in \{0.6,0.4\}^{10}$  with  $|i^{\star}(\mu)| \in \{1,3\}$ . Average over 200 runs. We ran a different instance of SR and SH for each budget  $T$ .
Overall, according to Figure 13,  $\mathrm{EB - TC}_{\varepsilon_0}$  seems to perform on par with SR and SH in all instances considered, and to outperform SH for some instances. In our experiments, we consider SH in which the samples are dropped between each phase, as analyzed theoretically in [1]. This loss of information explains why SR has better empirical performance than SH. To the best of our knowledge, there is no theoretical analysis for the heuristic SH where all the samples are preserved, even though it has better empirical performance.
J.2.3 Varying slack parameter
In Appendix J.2.3, we study the BAI problem. Those experiments have two goals: (1) sensitivity analysis of  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  described in Section 3 and (2) performance assessment of  $\mathrm{EB - TC}_{\varepsilon_0}$  on BAI problems. We consider fixed proportions  $\beta = 1 / 2$ , and use  $\varepsilon_0 = 0.1$  for  $\mathrm{EB - TC}_{\varepsilon_0}$ . For the rate of decrease of  $(\varepsilon_{n})_{n}$ , we will be considering two archetypal choices: (a) polynomial by taking  $\varepsilon_{n} = n^{-\alpha /2}$  and (b) polylogarithmic by taking  $\varepsilon_{n} = \log (n)^{-\alpha /2}$ . We compare empirically those two rates of decrease for different choices of  $\alpha$ , namely  $\alpha \in \{0.05,0.1,0.5\}$ . To tackle BAI, we consider the GLR $_0$  stopping rule (3) with  $(\varepsilon ,\delta) = (0,0.01)$  and the heuristic threshold  $c(n,\delta) = \log ((1 + \log n) / \delta)$ . Even though this choice is not sufficient to prove  $(0,\delta)$ -PAC, it yields an empirical error which is several orders of magnitude lower than  $\delta$ . As benchmark, we use the unmodified BAI algorithms.
Random instances We assess the performance on 1000 random Gaussian instances with  $K \in \{5,10,20\}$  such that  $\mu_1 = 1$  and  $\mu_i \sim \mathcal{U}([0.5,08])$  for  $i \neq 1$ . We display the boxplots of the empirical stopping time on 1000 runs.
Figure 14: Empirical stopping time for the stopping rule (3) using  $(\varepsilon, \delta) = (0, 0.01)$  on random instances with (a)  $K = 5$ , (b)  $K = 10$  and (c)  $K = 20$ . "l" denotes  $\varepsilon_{n} = \log(n)^{-\alpha/2}$ , "p" denotes  $\varepsilon_{n} = n^{-\alpha/2}$ . "a..." refers to  $\alpha \in \{0.5, 0.1, 0.05\}$ .
The most stricking feature of Figure 14 is that it reveals how poor the performance of  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  can be for many choices of  $(\varepsilon_{n})_{n}$ . For the considered  $\alpha$ , we observe that polynomial decrease is always bad since it yields roughly the same empirical performance as uniform sampling. This can be explained by the fact that this decrease is too fast, hence  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  will resemble  $\mathrm{EB - TC}_0$  which is known to have poor empirical performance [21]. For the considered  $\alpha$ , we see that polylogarithmic decrease is often bad, except for  $\alpha = 0.5$ . In that case, it performs on par with TaS, slightly worse than standard Top Two algorithms for BAI and better than LUCB and uniform sampling. This can be explained by the fact that  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  is similar to uniform sampling when the decrease is too slow. The above sensitivity analysis truly shows how difficult it is to choose  $(\varepsilon_{n})_{n}$  beforehand. The
best "trade-off" between a slow decrease (yet not too slow) highly depends on the complexity of the unknown instance. While  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  reaches asymptotic optimality for BAI, we recommend the practitioner to use other Top Two algorithms which enjoy the same theoretical guarantees and better empirical performance. When one needs to have a deterministic algorithm, EB-TCI seems to be the best. Without the deterministic constraint, T3C has great performance.
In Figure 14, we also see that, when combined with the  $\mathrm{GLR}_0$  stopping rule,  $\mathrm{EB - TC}_{\varepsilon_0}$  has good empirical performance for BAI. It outperforms  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$ , performs on par with TaS and is only slightly worse than standard Top Two algorithms for BAI. Experiments conducted in Appendix J.2.1 already provide experimentally confirmation that  $\mathrm{EB - TC}_{\varepsilon_0}$  has still good empirical performance on  $\varepsilon$ -BAI problems when  $\varepsilon_0 \geq \varepsilon$ . Theoretically, Theorem 1 provides some intuition on why this is true. It shows that the gap between the asymptotic lower bound  $T_0(\mu)$  and the asymptotic upper bound of  $\mathrm{EB - TC}_{\varepsilon_0}$  is bounded by
$$
\frac{T_{\varepsilon_{0},1 / 2}(\mu)}{T_{0}(\mu)}\left(1 + \frac{\varepsilon_{0}}{\Delta_{\min}}\right)^{2}\leq 4\frac{\sum_{i\neq i^{\star}}(\Delta_{i} + \varepsilon_{0})^{-2}}{\sum_{i\neq i^{\star}}\Delta_{i}^{-2}}\left(1 + \frac{\varepsilon_{0}}{\Delta_{\min}}\right)^{2}\approx \left\{ \begin{array}{ll}4 & {\mathrm{if}\varepsilon_{0}\gg \Delta_{\min}}\\ 4 & {\mathrm{if}\varepsilon_{0}\ll \Delta_{\min}} \end{array} \right.,
$$
where we used Lemmas 6, 9 and 7.
Specific instances We consider the two instances  $\mu_{1}$  and  $\mu_{2}$  from Table 3 with  $\varepsilon = 0$ . Since most theoretical guarantees on BAI algorithms assume that there is a unique best arm, we don't study  $\mu_{3}$ . We display the boxplots of the empirical stopping time on 1000 runs.
Figure 15: Empirical stopping time for the stopping rule (3) using  $(\varepsilon, \delta) = (0, 0.01)$  on instances (a)  $\mu_1$  and (b)  $\mu_2$ . "l" denotes  $\varepsilon_n = \log(n)^{-\alpha/2}$ , "p" denotes  $\varepsilon_n = n^{-\alpha/2}$ .
Figure 15 confirms the empirical observations from Figure 14. Overall,  $\mathrm{EB - TC}_{(\varepsilon_n)_n}$  performs poorly and  $\mathrm{EB - TC}_{\varepsilon_0}$  has good empirical performance for BAI.