SlowGuess/NeurIPS_2023_OCR huggingface dataset:
As in [40], we show that a challenger will not be sampled next when the ratio of its empirical proportion compared to the one of  $i^{\star}$  overshoots the ratio of the optimal allocations (Lemma 62).
Lemma 62. Let  $\gamma >0$ . There exists  $N_{5}$  with  $\mathbb{E}_{\nu}[N_5] < +\infty$  such that for all  $n\geq N_5$  and all  $i\neq i^{\star}$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma \quad \Longrightarrow \quad C _ {n} ^ {T C _ {\varepsilon_ {0}} ^ {m}} \neq i.
$$
Let  $N_4$  as in Lemma 61. As in Lemma 22, we can show for all  $n \geq N_4$
$$
\exists j \notin \{i ^ {\star}, i \}, \quad \frac {N _ {n , j}}{N _ {n , i ^ {\star}}} \leq \frac {w _ {j} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}.
$$
Let  $N_{1}$  as in Lemma 18. Using that  $B_{n}^{\mathrm{EB}} = i^{\star}$  for all  $n\geq N_1$  and the definition of  $C_n^{\mathrm{TC}_{\varepsilon_0}^m}$ , we know that
$$
\frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , i}}{\sqrt {1 / N _ {n , i ^ {\star}} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , i}}} > \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , j}}{\sqrt {1 / N _ {n , i ^ {\star}} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \quad \Rightarrow \quad C _ {n} ^ {\mathrm {T C} _ {\varepsilon_ {0}} ^ {\mathrm {m}}} \neq i.
$$
$$
\begin{array}{l} \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , i}}{\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , j}} \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , j}}{1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , i}}} \\ \geq \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , i}}{\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , j}} \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} w _ {i ^ {\star}} ^ {\star} / w _ {j} ^ {\star}}{1 + (1 - \varepsilon_ {0}) ^ {2} \left(w _ {i} ^ {\star} / w _ {i ^ {\star}} ^ {\star} + \gamma\right) ^ {- 1}}}. \\ \end{array}
$$
Let  $\tilde{\gamma} > 0$ . Using Lemmas 12 and 18, we have, for all  $n \geq N_1$  and all  $k \neq i^\star$ ,
$$
\begin{array}{l} \left| \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , k}}{\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {k}} - 1 \right| \leq \frac {W _ {\mu} \left((1 - \varepsilon_ {0}) \sqrt {\frac {\log (e + N _ {n , k})}{N _ {n , k}}} + \sqrt {\frac {\log (e + N _ {n , i ^ {\star}})}{N _ {n , i ^ {\star}}}}\right)}{\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {k}} \\ \leq \frac {(2 - \varepsilon_ {0}) W _ {\mu}}{\min _ {k \neq i ^ {\star}} (\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {k})} \sqrt {\frac {\log (e + \sqrt {n / K})}{\sqrt {n / K}}} \leq \tilde {\gamma} \\ \end{array}
$$
$$
\begin{array}{l} X _ {0} = \sup  \left\{x \geq 1 \mid x <   \frac {(2 - \varepsilon_ {0}) ^ {2} W _ {\mu} ^ {2}}{\tilde {\gamma} ^ {2} \min  _ {k \neq i ^ {\star}} (\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {k}) ^ {2}} \log x + e \right\} \\ <   h _ {1} \left(\frac {(2 - \varepsilon_ {0}) ^ {2} W _ {\mu} ^ {2}}{\tilde {\gamma} ^ {2} \min _ {k \neq i ^ {\star}} (\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {k}) ^ {2}}, e\right). \\ \end{array}
$$
$$
\begin{array}{l} \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , i}}{\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , j}} \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , j}}{1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , i}}} \\ \geq \frac {\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {i}}{\mu_ {i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {j}} \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} w _ {i ^ {\star}} ^ {\star} / w _ {j} ^ {\star}}{1 + (1 - \varepsilon_ {0}) ^ {2} (w _ {i} ^ {\star} / w _ {i ^ {\star}} ^ {\star} + \gamma) ^ {- 1}}} \frac {1 - \tilde {\gamma}}{1 + \tilde {\gamma}} \\ = \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} w _ {i ^ {\star}} ^ {\star} / w _ {i} ^ {\star}}{1 + (1 - \varepsilon_ {0}) ^ {2} \left(w _ {i} ^ {\star} / w _ {i ^ {\star}} ^ {\star} + \gamma\right) ^ {- 1}} \frac {1 - \tilde {\gamma}}{1 + \tilde {\gamma}} . \\ \end{array}
$$
where the equality uses that the transportation costs are equal at the equilibrium, i.e. (31). Taking  $\tilde{\gamma}$  small enough, we have that shown that, for all  $n\geq \max \{N_1,N_4,N_6\}$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma \quad \Longrightarrow \quad \frac {\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , i}}{\mu_ {n , i ^ {\star}} - (1 - \varepsilon_ {0}) \mu_ {n , j}} \sqrt {\frac {1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , j}}{1 + (1 - \varepsilon_ {0}) ^ {2} N _ {n , i ^ {\star}} / N _ {n , i}}} > 1,
$$
hence  $C_n^{\mathrm{TC}_{\varepsilon_0}^m}\neq i$  . This concludes the proof.
Lemma 63 is obtained with the same proof as Lemma 23.
Lemma 63. Let  $\gamma >0$ . There exists  $N_{6}$  with  $\mathbb{E}_{\nu}[N_6] < +\infty$  such that for all  $n\geq N_6$  and all  $i\neq i^{\star}$ ,
Lemma 64 is obtained with the same proof as Lemma 24.
Lemma 64. Let  $\varepsilon_0\in (0,1)$ ,  $\gamma >0$  and  $T_{\varepsilon_0,\gamma}$  introduced before (adaptation of(10)). Under the  $EB-TC_{\varepsilon_0}^m$  sampling rule with IDS proportions, we have  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$ .
Combining Lemmas 54 and 64 yields the first of Theorem 8.
I.3.4 Convergence towards  $\beta$ -optimal allocation
Combining Lemmas 58 and 27 with the proof of Lemma F.9 in [20] yields Lemma 65.
Lemma 65. Let  $\gamma >0$ . There exists  $N_{5}$  with  $\mathbb{E}_{\nu}[N_5] < +\infty$  such that for all  $n\geq N_5$  and all  $i\neq i^{\star}$ ,
$$
\frac {N _ {n , i}}{n - 1} \geq w _ {\beta , i} ^ {\star} + \gamma \quad \Longrightarrow \quad C _ {n} ^ {T C _ {\varepsilon_ {0}} ^ {m}} \neq i.
$$
Lemma 66 is obtained with the same proof as Lemmas 29 and 30.
Lemma 66. Let  $\varepsilon_0\in (0,1)$ ,  $\beta \in (0,1)$ ,  $\gamma >0$  and  $T_{\varepsilon_0,\beta ,\gamma}$  introduced before (adaptation of (14)). Under the  $EB-TC_{\varepsilon_0}^m$  sampling rule with fixed proportions  $\beta$ , we have  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\beta ,\gamma}] < +\infty$ .
Combining Lemmas 54 and 66 yields the second part of Theorem 8.
J Implementation details and additional experiments
After presenting the implementations details in Appendix J.1, we display supplementary experiments in Appendix J.2.
J.1 Implementation details
Top Two sampling rules Given a fixed  $\beta$ , most Top Two algorithms use randomization to select  $I_{n} \in \{B_{n}, C_{n}\}$ , namely they sample  $B_{n}$  with probability  $\beta$ , otherwise sample  $C_{n}$ . Instead of randomization, TTUCB [20] uses  $K$  tracking procedures to select  $I_{n} \in \{B_{n}, C_{n}\}$ . Namely, they choose  $I_{n} = B_{n}$  if  $N_{n,B_{n}}^{B_{n}} \leq \beta \sum_{i \neq B_{n}} T_{n+1}(B_{n}, i)$ , and  $I_{n} = C_{n}$  otherwise. In comparison, EB-TC $_{\varepsilon_0}$  relies on  $K(K - 1)$  tracking procedures both for fixed proportion  $\beta$  and IDS proportions [40]. In [40], they consider IDS proportions to define  $\beta_{n}$  adaptively. For Gaussian with known homoscedastic variance, their update mechanism is
$$
\beta_ {n} = \frac {N _ {n , B _ {n}} d _ {\mathrm {K L}} (\mu_ {n , B _ {n}} , u _ {n} (B _ {n} , C _ {n}))}{N _ {n , B _ {n}} d _ {\mathrm {K L}} (\mu_ {n , B _ {n}} , u _ {n} (B _ {n} , C _ {n})) + N _ {n , C _ {n}} d _ {\mathrm {K L}} (\mu_ {n , C _ {n}} , u _ {n} (B _ {n} , C _ {n}))} = \frac {N _ {n , C _ {n}}}{N _ {n , B _ {n}} + N _ {n , C _ {n}}} ,
$$
where the second equality is obtained by direct computations which uses that
$$
u _ {n} (i, j) = \inf _ {x \in \mathbb {R}} \left[ N _ {n, i} d _ {\mathrm {K L}} (\mu_ {n, i}, x) + N _ {n, j} d _ {\mathrm {K L}} (\mu_ {n, j}, x) \right] = \frac {N _ {n , i} \mu_ {n , i} + N _ {n , j} \mu_ {n , j}}{N _ {n , i} + N _ {n , j}}.
$$
[8] proposed a new methodology called Balancing Optimal Large Deviations (BOLD) to select  $I_{n} \in \{B_{n}, C_{n}\}$  adaptively. While their approach aims at minimizing the probability of incorrect selection (i.e. fixed-budget setting), it is possible to adapt their idea to minimize the expected sample complexity (i.e. fixed-confidence setting) by "swapping" the arguments of the KL divergence. Namely, the BOLD procedure can be rewritten in our setting as selecting  $I_{n} = B_{n}$  if
$$
\sum_ {i \neq B _ {n}} \frac {d _ {\mathrm {K L}} (\mu_ {n , B _ {n}} , u _ {n} (B _ {n} , i))}{d _ {\mathrm {K L}} (\mu_ {n , i} , u _ {n} (B _ {n} , i))} > 1,
$$
and  $I_{n} = C_{n}$  otherwise. For Gaussian with known homoscedastic variance, their approach recovers the heuristic "adaptive Welch divergence" algorithm of proposed in [37], i.e.  $I_{n} = B_{n}$  if  $N_{n,B_n}^2 < \sum_{i\neq B_n}N_{n,i}^2$ , and  $I_{n} = C_{n}$  otherwise. The analysis of [8] is focused on asymptotic guarantees in probability. It is an interesting direction for future research to show that the BOLD procedure also yields asymptotically optimal algorithms.
TTTS [34] uses a TS (Thompson Sampling) leader and a RS (Re-Sampling) challenger based on a sampler  $\Pi_{n}$ . For Gaussian bandits, the sampler  $\Pi_{n}$  is the posterior distribution  $\times_{i\in [K]}\mathcal{N}(\mu_{n,i},1 / N_{n,i})$  given the improper prior  $\Pi_1 = (\mathcal{N}(0, + \infty))^K$ . The TS leader is  $B_{n}^{\mathrm{TS}}\in \arg \max_{i\in [K]}\theta_{i}$  where  $\theta \sim \Pi_{n}$ . The RS challenger samples vector of realizations  $\theta \in \Pi_{n}$  until  $B_{n}\notin \arg \max_{i\in [K]}\theta_{i}$ , then it is defined as  $C_n^{\mathrm{RS}}$  arg  $\max_{i\in [K]}\theta_i$  for this specific vector of realization. When the posterior  $\Pi_{n}$  and the leader  $B_{n}$  have almost converged towards the Dirac distribution on  $\mu$  and the best arm  $i^{\star}(\mu)$  respectively, the event  $B_{n}\notin \arg \max_{i\in [K]}\theta_{i}$  becomes very rare. The experiments in [21] reveals that computing the RS challenger can require more than millions of re-sampling steps. Therefore, the RS challenger can become computationally intractable even for Gaussian distribution where sampling from  $\Pi_{n}$  can be done more efficiently.
T3C [36] combines the TS leader and the TC challenger. EB-TCI [21] combines the EB leader with the TCI challenger defined as
$$
C_{n}^{\mathrm{TCI}} = \operatorname *{arg  min}_{i\neq B_{n}}\mathbb{1}\left(\mu_{n,B_{n}} > \mu_{n,i}\right)\frac{\left(\mu_{n,B_{n}} - \mu_{n,i}\right)^{2}}{2(1 / N_{n,B_{n}} + 1 / N_{n,i})} +\log (N_{n,i}).
$$
TTUCB [20] combined the TC challenger with a UCB leader which is defined as
$$
B _ {n} ^ {\mathrm {U C B}} = \underset {i \in [ K ]} {\arg \max} \{\mu_ {n, i} + \sqrt {g (n) / N _ {n , i}} \},
$$
where  $\sqrt{g(n) / N_{n,i}}$  is a bonus coping for uncertainty. TS-KKT [40] uses the TS leader and a challenger based on transportation costs with a different penalization than the one used in [21], namely
$$
C _ {n}) \in \underset {i \neq B _ {n}} {\arg \min} \left\{\frac {(\mu_ {n , B _ {n}} - \mu_ {n , i}) ^ {2}}{2 (1 / N _ {n , B _ {n}} + 1 / N _ {n , i})} - \frac {\rho}{n} \log (1 / N _ {n, B _ {n}} + 1 / N _ {n, i}) \right\},
$$
where  $\rho$  is a parameter that needs to be selected beforehand. The authors highlight that the choice of  $\rho$  has an important impact on the empirical performance.
While the above Top Two algorithms are BAI algorithms, we can adapt them straightforwardly to tackle the  $\varepsilon$ -BAI setting by (1) using the stopping rule as in (3) and (2) using transportation costs for  $\varepsilon$ -BAI, i.e. the ones in (2). The modification (1) is inspired by [19] and the modification (2) by [22]. As an example, the modified TCI challenger can be written as
$$
C _ {n} ^ {\mathrm {T C I}} = \underset {i \neq B _ {n}} {\arg \min } \mathbb {1} \left(\mu_ {n, B _ {n}} > \mu_ {n, i}\right) \frac {(\mu_ {n , B _ {n}} - \mu_ {n , i} + \varepsilon) ^ {2}}{2 (1 / N _ {n , B _ {n}} + 1 / N _ {n , i})} + \log (N _ {n, i}).
$$
Fixed-confidence BAI algorithms At each time  $n$ , Track-and-Stop (TaS) [15] computes the optimal allocation for the current empirical mean,  $w_{n} = w^{\star}(\mu_{n})$ . Given  $w_{n} \in \triangle_{K}$ , it uses a tracking procedure to obtain an arm  $I_{n}$  to sample. On top of this tracking a forced exploration is used to enforce convergence towards the optimal allocation for the true unknown parameters. The optimization problem defining  $w^{\star}(\mu)$  can be rewritten as solving an equation  $\psi_{\mu}(r) = 0$  where
$$
\forall r \in (1 / \min _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}, + \infty), \psi_ {\mu} (r) = \sum_ {i \neq i ^ {\star}} \frac {1}{(r (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2} - 1) ^ {2}} - 1
$$
The function  $\psi_{\mu}$  is decreasing, and satisfies  $\lim_{r\to +\infty}\psi_{\mu}(r) = -1$  and  $\lim_{y\to 1 / \min_{i\neq i^*}(\mu_i\star -\mu_i)^2}F_\mu (y) = +\infty$ . For the practical implementation of the optimal allocation, we use the approach of [15] and perform binary searches to compute the unique solution of  $\psi_{\mu}(r) = 0$ . A faster implementation based on Newton's iterates was proposed by [3] after proving that  $\psi_{\mu}$  is convex. While this improvement holds only for Gaussian distributions, the binary searches can be used for more general distributions. Similarly, one can adapt TaS to tackle  $\varepsilon$ -BAI as done in [16], i.e. by tracking  $w_{n} = w_{\varepsilon}(\mu_{n})$ . Using Lemma 9, it is direct to see that one can use the same implementation to compute it efficiently.
DKM [11] view  $T^{\star}(\mu)^{-1}$  as a min-max game between the learner and the nature, and design saddle-point algorithms to solve it sequentially. At each time  $n$ , a learner outputs an allocation  $w_{n}$ , which is used by the nature to compute the worst alternative mean parameter  $\lambda_{n}$ . Then, the learner is updated based on optimistic gains based on  $\lambda_{n}$ . Similarly, one can adapt DKM to tackle  $\varepsilon$ -BAI by using a modified alternative mean parameter. This coincides with the L-BAI algorithm [19] when used on unstructured multi-armed bandits.
FWS [39] alternates between forced exploration and Frank-Wolfe (FW) updates. Similarly, FWS can be adapted to tackle  $\varepsilon$ -BAI
LUCB [24] samples and stops based on upper/ lower confidence indices for a bonus function  $g$ . For Gaussian distributions, it rewrites for all  $i \in [K]$  as
$$
U _ {n, i} = \mu_ {n, i} + \sqrt {\frac {2 c (n - 1 , \delta)}{N _ {n , i}}} \quad \text {a n d} \quad L _ {n, i} = \mu_ {n, i} - \sqrt {\frac {2 c (n - 1 , \delta)}{N _ {n , i}}}.
$$
At each time  $n$ , it samples  $\hat{i}_n$  and  $\arg \max_{i \neq \hat{i}_n} U_{n,i}$ . For  $\varepsilon$ -BAI, LUCB stops when  $L_{n,\hat{i}_n} + \varepsilon \geq \max_{i \neq \hat{i}_n} U_{n,i}$ .
Fixed-budget BAI algorithms We consider Successive Reject (SR) [1] and Sequential Halving (SH) [25]. SR eliminates one arm with the worst empirical mean at the end of each phase, and SH eliminates half of them but drops past observations between each phase. Within each phase, both algorithms use a round-robin uniform sampling rule on the remaining active arms. It is possible to convert the fixed-budget SH algorithm into an anytime algorithm by using the doubling trick. It considers a sequence of algorithms that are run with increasing budgets  $(T_{k})_{k\geq 1}$ , with  $T_{k + 1} = 2T_{k}$  and  $T_{1} = 2K[\log_{2}K]$ , and recommend the answer outputted by the last instance that has finished to run. It is well known that the "cost" of doubling is to have a multiplicative factor 4 in front of the hardness constant. The first two-factor is due to the fact that we forget half the observations. The second two-factor is due to the fact that we use the recommendation from the last instance of SH that has finished. The doubling version of SR and SH are named Doubling SR (DSR) and Doubling SH (DSH).
Reproducibility Our code is implemented in Julia 1.9.0, and the plots are generated with the StatsPlots.jl package. Other dependencies are listed in the Readme.md. The Readme.md file also provides detailed julia instructions to reproduce our experiments, as well as a script.sh to run them all at once. The general structure of the code (and some functions) is taken from the
tidnabbil library. This library was created by [11], see https://bitbucket.org/wmkoolen/tidnabbil. No license were available on the repository, but we obtained the authorization from the authors. Our experiments are conducted on an institutional cluster with 4 Intel Xeon Gold 5218R CPU with 20 cores per CPU and an x86_64 architecture.
J.2 Supplementary experiments
For the sake of space, we only presented a small subset of our experiments in Section 5, hence we provide additional empirical evidence below. In Appendix J.2.1, we assess the impact of the choice of the slack  $\varepsilon_0$  on the empirical performance of  $\mathrm{EB - TC}_{\varepsilon_0}$  with IDS or with fixed proportion  $\beta = 1 / 2$ . Appendix J.2.2 complements Section 5 by providing more comparison between  $\mathrm{EB - TC}_{\varepsilon_0}$  and existing algorithms. In Appendix J.2.3, we study the BAI problem and show the empirical performance of the  $\mathrm{EB - TC}_{(\varepsilon_n)_{n > K}}$  with fixed proportion  $\beta = 1 / 2$  and using varying slack parameters  $(\varepsilon_{n})_{n > K}$  (as described in Sections 3 and 4). As in Section 5, we consider  $\delta = 0.01$  and use the heuristic threshold  $c(n,\delta) = \log ((1 + \log n) / \delta)$ , which yields an empirical error which is several orders of magnitude lower than  $\delta$ .
J.2.1 Numerical simulations on  $\mathbf{EB - TC}_{\varepsilon_0}$
Appendix J.2.1 is meant to be a sensitivity analysis of the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm. It will allow us to numerically substantiate our recommendation to the practitioner on how to set the slack  $\varepsilon_0$ , and why using IDS yields better empirical performance. For slacks  $\varepsilon_0 \in \{0.15, 0.1, 0.05\}$ , we will consider the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithms with IDS proportions and fixed proportions  $\beta = 1/2$ .
Figure 4: Empirical stopping time with stopping rule (3) using  $(\varepsilon, \delta) = (0.1, 0.01)$  on (a) "sparse" instances", (b)  $\alpha = 0.3$  instances and (c) "  $\alpha = 0.6$  instances. "c- denotes fixed  $\beta = 1/2$ , without refers to IDS.
Large sets of arms First, we evaluate the impact of larger number of arms (up to  $K = 1000$ ). The "  $\alpha = 0.6$  " scenario of [18] sets  $\mu_{i} = 1 - ((i - 1) / (K - 1))^{\alpha}$  for all  $i \in [K]$ . The "sparse" scenario of [18] sets  $\mu_{1} = 1 / 4$  and  $\mu_{i} = 0$  otherwise. We average on 100 runs, and the standard deviations are displayed.
In Figure 4, we see that all instances of  $\mathrm{EB - TC}_{\varepsilon_0}$  with IDs or fixed  $\beta = 1 / 2$  have the same scaling with the dimension  $K$ . Figure 4 also reveals that using IDS instead of fixed  $\beta = 1 / 2$  consistently yields smaller empirical stopping time. Therefore, IDS should be preferred to choosing fixed  $\beta = 1 / 2$ . Finally, Figure 4 shows that better empirical performance are obtained when using a slack  $\varepsilon_0$  which matches the error  $\varepsilon$ , i.e. choosing  $\varepsilon_0 = \varepsilon$  to tackle  $\varepsilon$ -BAI. While choosing  $\varepsilon_0 > \varepsilon$  only slightly damage the empirical performance, taking  $\varepsilon_0 < \varepsilon$  is clearly detrimental to the empirical performance.
Figure 5: Empirical stopping time on random instances with stopping rule (3) using (left)  $(\varepsilon, \delta) = (0.05, 0.01)$  and (right)  $(\varepsilon, \delta) = (0.1, 0.01)$  with (top)  $K = 5$ , (middle)  $K = 10$  and (bottom)  $K = 20$ . "c-" denotes fixed  $\beta = 1/2$ , without refers to IDS.
Random instances We further investigate the phenomenon described above on random instances. For  $\varepsilon \in \{0.05, 0.1\}$ , we assess the performance on 1000 random Gaussian instances with  $K = 5$  (resp.  $K = 10$  and  $K = 10$ ) such that  $\mu_1 = 1$ ,  $\mu_i \sim \mathcal{U}([1 - \varepsilon, 1])$  for  $i \in \{2\}$  (resp.  $i \in \{2, 3\}$  and  $i \in \{2, 3, 4\}$ ) and  $\mu_i \sim \mathcal{U}([0, 1 - \varepsilon))$  for  $i \geq 3$  (resp.  $i \geq 4$  and  $i \geq 6$ ), hence  $\mathcal{I}_{\varepsilon}(\mu) = [2]$  (resp.  $\mathcal{I}_{\varepsilon}(\mu) = [3]$  and  $\mathcal{I}_{\varepsilon}(\mu) = [5]$ ). We display the boxplots of the empirical stopping time on 1000 runs.
In Figure 5, the tendencies glimpsed by inspecting Figure 4 are more apparent. Figure 5 shows that better empirical performance are obtained when using a slack  $\varepsilon_0$  which matches the error  $\varepsilon$ , i.e. choosing  $\varepsilon_0 = \varepsilon$  to tackle  $\varepsilon$ -BAI. We also see that the empirical gain of taking  $\varepsilon_0 = \varepsilon$  is increasing with the dimension  $K$ . Figure 5 shows that  $\varepsilon_0 > \varepsilon$  slightly damage the empirical performance: the larger the mismatch  $\varepsilon_0 - \varepsilon$  is, the worse the performances are. Moreover, Figure 5 highlights how choosing  $\varepsilon_0 < \varepsilon$  is highly detrimental to the empirical performance. Finally, we see that there is a mild advantage in using IDS over fixed  $\beta = 1/2$ . Note that the mild empirical gain of adaptive proportions is a known phenomenon which was studied empirically in [20], and larger gains can be observed for "two-groups" instances. Intuitively, the characteristic times  $T_{\varepsilon}(\mu)$  and  $T_{\varepsilon,1/2}(\mu)$  are very close to each other on average, and the difference is the largest when all the sub-optimal arms have the same mean. Therefore, on average, using IDS has mild gain compared to using fixed  $\beta = 1/2$ . This worst-case scenario of the "two-groups" instances is studied in Figure 7.
Table 3: Specific instances from [16] with corresponding  $\varepsilon$  .
ε	Instance	|Iε(μ)|	Arms
1	2	3	4	5	6
0.1	μ1	1	0.7	0.55	0.5	0.4	0.2	-
0.15	μ2	3	0.8	0.75	0.7	0.6	0.5	0.4
0.1	μ3	3	0.6	0.6	0.55	0.45	0.3	0.2
Specific instances We continue the experimental validation of our the phenomenon described above by considering the three specific instances from the experimental section of [16] which are described in Table 3. Instance  $\mu_{3}$  corresponds to the specific instances studied in Section 5. It is particularly interesting since it has two best arms, a third  $\varepsilon$ -good arm which is equally distant from  $\varepsilon$  than the bad arm with largest mean, and has two additional bad arms. Instance  $\mu_{2}$  has also three  $\varepsilon$ -good arms, which are equally spaced, and three bad arms. Instance  $\mu_{1}$  has only one  $\varepsilon$ -good arm, and four bad arms. We display the boxplots of the empirical stopping time on 1000 runs, and the empirical simple regret on 10000 runs (with associated standard deviation).
Figure 6: (Left) Empirical stopping time for the stopping rule (3) using  $\delta = 0.01$  and (right) empirical simple regret on instances (top)  $\mu_{1}$  with  $\varepsilon = 0.1$ , (middle)  $\mu_{2}$  with  $\varepsilon = 0.15$  and (bottom)  $\mu_{3}$  with  $\varepsilon = 0.1$ . "c-" denotes fixed  $\beta = 1/2$ , without refers to IDS.
The left column of Figure 6 confirms our previous observations: taking  $\varepsilon = \varepsilon_0$  yields the best performance, taking  $\varepsilon_0 > \varepsilon$  slightly damage performance and taking  $\varepsilon_0 < \varepsilon$  highly damage performance.
Figure 6(middle left) shows that the larger the missmatch  $\varepsilon -\varepsilon_0$  is, the worse the performances are.