SlowGuess/NeurIPS_2023_OCR huggingface dataset:
An  $\varepsilon$ -Best-Arm Identification Algorithm for Fixed-Confidence and Beyond
Marc Jourdan
marc.jourdan@inria.fr
Rémy Degenne
remy.degenne@inria.fr
Emilie Kaufmann
Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189-CRISTAL, F-59000 Lille, France
We propose EB-TC $_{\varepsilon}$ , a novel sampling rule for  $\varepsilon$ -best arm identification in stochastic bandits. It is the first instance of Top Two algorithm analyzed for approximate best arm identification. EB-TC $_{\varepsilon}$  is an anytime sampling rule that can therefore be employed without modification for fixed confidence or fixed budget identification (without prior knowledge of the budget). We provide three types of theoretical guarantees for EB-TC $_{\varepsilon}$ . First, we prove bounds on its expected sample complexity in the fixed confidence setting, notably showing its asymptotic optimality in combination with an adaptive tuning of its exploration parameter. We complement these findings with upper bounds on its probability of error at any time and for any error parameter, which further yield upper bounds on its simple regret at any time. Finally, we show through numerical simulations that EB-TC $_{\varepsilon}$  performs favorably compared to existing algorithms, in different settings.
In pure exploration problems, the goal is to answer a question about a set of unknown distributions (modelling for example the efficacy of a treatment) from which we can collect samples (measure its effect), and to provide guarantees on the candidate answer. Practitioners might have different pre-defined constraints, e.g. the maximal budget might be fixed in advance or the error made should be smaller than a fixed admissible error. However, in many cases, fixing such constraints in advance can be challenging since a "good" choice typically depends on unknown quantities. Moreover, while the budget is limited in clinical trials, it is often not fixed beforehand. The physicians can decide to stop earlier or might obtain additional fundings for their experiments. In light of those real-world constraints, regardless of its primal objective any strategy for choosing the next treatment should ideally come with guarantees on its current candidate answer that hold at any time.
We formalize our investigations in the well-studied stochastic bandit model [4, 29], in which a learner interacts sequentially with an environment composed of  $K \in \mathbb{N}$  arms, which are unknown distributions  $(\nu_{i})_{i \in [K]}$  with finite means  $(\mu_{i})_{i \in [K]}$ . At each stage  $n \in \mathbb{N}$ , the learner chooses an arm  $I_{n} \in [K]$  based on the samples previously observed and receives a sample  $X_{n,I_{n}}$ , random variable with conditional distribution  $\nu_{I_{n}}$  given  $I_{n}$ . It then proceeds to the next stage. An algorithm for the learner in this interaction is specified by a sampling rule, a procedure that determines  $I_{n}$  based on previously observed samples. Formally, the sampling rule defines for all  $n \in \mathbb{N}$  a function from  $([K] \times \mathbb{R})^{n-1}$  to the probability distribution on  $[K]$ , which is measurable with respect to the  $\sigma$ -algebra  $\mathcal{F}_{n} := \sigma(\{I_{t}, X_{t,I_{t}}\}_{t \in [n-1]})$ . We call that  $\sigma$ -algebra history before  $n$ .
Identification tasks We focus on best arm identification (BAI). In that task, the goal of the algorithm is to find which of the arms has the largest mean, and to do so with a small probability of error, as
quickly as possible. If several arms have means very close to the maximum, finding the one with the highest mean might be difficult. However in practice we are often satisfied by any good enough arm, in the sense that its mean is greater than  $\mu_{\star} - \varepsilon$ , where  $\mu_{\star} = \max_{i \in [K]} \mu_i$ . This is the  $\varepsilon$ -BAI task. Our results can also be adapted to the multiplicative  $\varepsilon$ -BAI task, in which all means are non-negative and we want to find an arm with mean  $\mu_i \geq (1 - \varepsilon) \mu_{\star}$  [19] (see Appendix I for details).
Now that we have a (for now informal) goal, we need to complement the sampling rule with a recommendation rule that specifies which arm is the candidate returned by the algorithm for the best arm. We follow [5] and define that rule for all stages: for all  $n \in \mathbb{N}$ , we denote by  $\hat{i}_n$  this  $\mathcal{F}_n$ -measurable function from  $([K] \times \mathbb{R})^{n-1}$  to  $[K]$ . We call algorithm the combination of a sampling and a recommendation rule.
Performance criteria There are several ways to evaluate the performance of an algorithm for  $\varepsilon$ -BAI. Let  $\mathcal{I}_{\varepsilon}(\mu) = \{i \in [K] \mid \mu_i \geq \mu_\star - \varepsilon\}$  be the set of  $\varepsilon$ -good arms. The probability of  $\varepsilon$ -error or the recommendation at  $n$  is defined as  $\mathbb{P}_{\nu}(\hat{i}_n \notin \mathcal{I}_{\varepsilon}(\mu))$ . Introduced in [1], the expected simple regret is defined as  $\mathbb{E}_{\nu}[\mu_\star - \mu_{\hat{i}_n}]$ , and is independent of any parameter  $\varepsilon$ . Based on those notions, several settings are studied in the bandit identification literature.
- Fixed confidence: we augment the algorithm with a stopping rule, a stopping time  $\tau_{\varepsilon,\delta}$  with respect to the history of samples and we impose that the algorithm should be  $(\varepsilon,\delta)$ -PAC. That is, its probability of  $\varepsilon$ -error at  $\tau_{\varepsilon,\delta}$  must satisfy  $\mathbb{P}_{\nu}(\tau_{\varepsilon,\delta} < +\infty, i_{\tau_{\varepsilon,\delta}} \notin \mathcal{I}_{\varepsilon}(\mu)) \leq \delta$ . The parameter  $\delta$  is known to the algorithm. An algorithm is judged based on its expected sample complexity  $\mathbb{E}[\tau_{\varepsilon,\delta}]$ , the expected number of samples it needs to collect before it can stop and return a good arm with the required confidence.
- Fixed budget: we run the algorithm until a predefined time  $T$  and we evaluate it based on the probability of error at  $T$ . This setting has been mostly studied for  $\varepsilon = 0$  [1, 25], but [41] present the first bounds for  $\varepsilon > 0$  for an algorithm that is actually agnostic to this value.
- Simple regret minimization: we evaluate the expected simple regret at  $T$  [5, 41].
Simple regret is typically studied in an anytime setting: [5] contains upper bounds on the simple regret at time  $n$  for any  $n \in \mathbb{N}^*$ . Similarly, [23] propose the anytime exploration setting, in which they control the error probability  $\mathbb{P}\left(\hat{i}_n \neq \hat{i}_{\star}\right)$  for exact best arm identification. Interestingly, the authors build on an algorithm for the fixed-confidence setting, LUCB [24], whose sampling rule depends on the risk parameter  $\delta$ , which they replace by a sequence  $\delta_n$ . The algorithm that we study in this paper, motivated by the fixed-confidence  $\varepsilon$ -BAI problem, will already be anytime, which means that it does not depend on a given final time  $T$  or a confidence level  $\delta$ . We shall analyze its sample complexity in the fixed confidence setting but thanks to the anytime property we will also be able to prove guarantees on its probability of  $\varepsilon$ -error for every  $\varepsilon \geq 0$  and its simple regret at any time.
Additional notation and assumption We denote by  $\mathcal{D}$  a set to which the distributions of the arms are known to belong. We suppose that all distributions in  $\mathcal{D}$  are 1-sub-Gaussian. A distribution  $\nu_{0}$  is 1-sub-Gaussian if it satisfies  $\mathbb{E}_{X\sim \nu_0}[e^{\lambda (X - \mathbb{E}_X\sim \nu_0[X])}]\leq e^{\lambda^2 /2}$  for all  $\lambda \in \mathbb{R}$ . For example, all distributions bounded in  $[-1,1]$  are 1-sub-Gaussian. Let us denote by  $i^{\star}(\mu)\coloneqq \arg \max_{i\in [K]}\mu_{i}$  the set of arms with largest mean (i.e.  $i^{\star}(\mu) = \mathcal{I}_0(\mu)$ ). Let  $\Delta_i\coloneqq \mu_\star -\mu_i$  denote the sub-optimality gap of arm  $i$ . We denote by  $\triangle_K\subset \mathbb{R}^K$  the simplex of dimension  $K - 1$ .
Fixed-confidence  $\varepsilon$ -best-arm identification Let  $\varepsilon \geq 0$  and  $\delta \in (0,1)$  be fixed error and confidence parameters. In the fixed-confidence  $\varepsilon$ -BAI setting [31, 13, 35, 16], the probability of error of an algorithm is required to be less than  $\delta$  for all instances  $\nu \in \mathcal{D}^{K}$ . That requirement leads to an asymptotic lower bound on the expected sample complexity on any instance.
Lemma 1 ([10]). For all  $(\varepsilon, \delta)$ -PAC algorithms and all instances  $\nu_i = \mathcal{N}(\mu_i, 1)$  with  $\mu \in \mathbb{R}^K$ ,  $\lim \inf_{\delta \to 0} \frac{\mathbb{E}_{\nu}[\tau_{\varepsilon, \delta}]}{\log(1 / \delta)} \geq T_{\varepsilon}(\mu)$  where  $T_{\varepsilon}(\mu) = \min_{i \in \mathcal{I}_{\varepsilon}(\mu)} \min_{\beta \in (0, 1)} T_{\varepsilon, \beta}(\mu, i)$  with
$$
T _ {\varepsilon , \beta} (\mu , i) ^ {- 1} = \max  _ {w \in \triangle_ {K}, w _ {i} = \beta} \min  _ {j \neq i} \frac {1}{2} \frac {\left(\mu_ {i} - \mu_ {j} + \varepsilon\right) ^ {2}}{1 / \beta + 1 / w _ {j}}. \tag {1}
$$
We say that an algorithm is asymptotically (resp.  $\beta$ -)optimal if its sample complexity matches that lower bound, that is if  $\lim \sup_{\delta \to 0} \frac{\mathbb{E}_{\nu}[\tau_{\varepsilon,\delta}]}{\log(1 / \delta)} \leq T_{\varepsilon}(\mu)$  (resp.  $T_{\varepsilon,\beta}(\mu) = \min_{i \in \mathcal{I}_{\varepsilon}(\mu)} T_{\varepsilon,\beta}(\mu,i)$ ). Note
that the expected sample complexity of an asymptotically  $1/2$ -optimal algorithm is at worst twice higher than that of any asymptotically optimal algorithm since  $T_{\varepsilon, 1/2}(\mu) \leq 2T_{\varepsilon}(\mu)$  [34].
The asymptotic characteristic time  $T_{\varepsilon}(\mu)$  is of order  $\sum_{i=1}^{K} \min \{\varepsilon^{-2}, \Delta_i^{-2}\}$ . It is computed as a minimum over all  $\varepsilon$ -good arms  $i \in \mathcal{I}_{\varepsilon}(\mu)$  of an arm-specific characteristic time, which can be interpreted as the time required to verify that  $i$  is  $\varepsilon$ -good. Each of the times  $\min_{\beta \in (0,1)} T_{\varepsilon,\beta}(\mu,i)$  correspond to the complexity of a BAI instance (i.e.  $\varepsilon$ -BAI with  $\varepsilon = 0$ ) in which the mean of arm  $i$  is increased by  $\varepsilon$  (Lemma 9). Let  $w_{\varepsilon,\beta}(\mu,i)$  be the maximizer of (1). In [16], they show that  $T_{\varepsilon}(\mu) = T_{\varepsilon,\beta^*(i^*)}(\mu,i^*)$  and  $T_{\varepsilon,\beta}(\mu) = T_{\varepsilon,\beta}(\mu,i^*)$ , where  $i^* \in i^*(\mu)$  and  $\beta^*(i^*) = \arg \min_{\beta \in (0,1)} T_{\varepsilon,\beta}(\mu,i^*)$ . For  $\varepsilon = 0$ , a similar lower bound to Lemma 1 holds for all  $\delta$  [15]. Lower bounds of order  $\sum_{i=1}^{K} \Delta_i^{-2} \log \log \Delta_i^{-2}$  (independent of  $\delta$ , but with a stronger dependence in the gaps) were also shown [17, 6, 38, 7]. Note that the characteristic time for  $\sigma$ -sub-Gaussian distributions (which does not have a form as "explicit" as (1)) is always smaller than the ones for Gaussian having the same means and variance  $\sigma^2$ .
A good algorithm should have an expected sample complexity as close as possible to these lower bounds. Several algorithms for  $(\varepsilon)$ -BAI are based on modifications of the UCB algorithm [24, 17, 14]. Others compute approximate solutions to the lower bound maximization problem and sample arms in order to approach the solution [15, 11, 39]. Our method belongs to the family of Top Two algorithms [34, 36, 21], which select at each time two arms called leader and challenger, and sample among them. It is the first Top Two algorithm for the  $\varepsilon$ -BAI problem (for  $\varepsilon > 0$ ).
Any time and uniform  $\varepsilon$ -error bound In addition to the fixed-confidence guarantees, we will prove a bound on the probability of error for any time  $n$  and any error  $\varepsilon$ , similarly to the results of [41]. That is, we bound  $\mathbb{P}_{\nu}(\hat{i}_n \notin \mathcal{I}_{\varepsilon}(\mu))$  for all  $n$  and all  $\varepsilon$ . This gives a bound on the probability of error in  $\varepsilon$ -BAI, and a bound on the simple regret of the sampling rule by integrating:  $\mathbb{E}_{\nu}[\mu_{\star} - \mu_{\hat{i}_n}] = \int \mathbb{P}_{\nu}(\hat{i}_n \notin \mathcal{I}_{\varepsilon}(\mu)) \mathrm{d}\varepsilon$ .
The literature mostly focuses on the fixed budget setting, where the time  $T$  at which we evaluate the error probability is known and can be used as a parameter of the algorithm. Notable algorithms are successive rejects (SR, [1]) and sequential halving (SH, [25]). These algorithms can be extended to not depend on  $T$  by using a doubling trick [23, 41]. That trick considers a sequence of algorithms that are run with budgets  $(T_k)_k$ , with  $T_{k+1} = 2T_k$  and  $T_1 = 2K\lceil \log_2 K \rceil$ . Past observations are dropped when reaching  $T_k$ , and the obtained recommendation is used until the budget  $T_{k+1}$  is reached.
We propose the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm for identification in bandits, with a slack parameter  $\varepsilon_0 > 0$  originally motivated by  $\varepsilon_0$ -BAI. We study its combination with a stopping rule for fixed confidence  $\varepsilon$ -BAI (possibly with  $\varepsilon_0 \neq \varepsilon$ ) and also its probability of error and simple regret at any time.
-  $\mathrm{EB - TC}_{\varepsilon_0}$  performs well empirically compared to existing methods, both for the expected sample complexity criterion in fixed confidence  $\varepsilon$ -BAI and for the anytime simple regret criterion. It is in addition easy to implement and computationally inexpensive in our regime.
- We prove upper bounds on the sample complexity of  $\mathrm{EB - TC}_{\varepsilon_0}$  in fixed confidence  $\varepsilon$ -BAI with sub-Gaussian distributions, both asymptotically (Theorem 1) as  $\delta \to 0$  and for any  $\delta$  (Theorem 2). In particular,  $\mathrm{EB - TC}_{\varepsilon}$  with  $\varepsilon > 0$  is asymptotically optimal for  $\varepsilon$ -BAI with Gaussian distributions.
- We prove a uniform  $\varepsilon$ -error bound valid for any time for  $\mathrm{EB - TC}_{\varepsilon_0}$ . This gives in particular a fixed budget error bound and a control of the expected simple regret of the algorithm (Theorem 3 and Corollary 1).
2 Anytime Top Two sampling rule
We propose an anytime Top Two algorithm, named  $\mathrm{EB - TC}_{\varepsilon_0}$  and summarized in Figure 1.
Recommendation rule Let  $N_{n,i} \coloneqq \sum_{t \in [n-1]} \mathbb{1}(I_t = i)$  be the number of pulls of arm  $i$  before time  $n$ , and  $\mu_{n,i} \coloneqq \frac{1}{N_{n,i}} \sum_{t \in [n-1]} X_{t,I_t} \mathbb{1}(I_t = i)$  be its empirical mean. At time  $n > K$ , we recommend the Empirical Best (EB) arm  $\hat{\imath}_n \in \arg \max_{i \in [K]} \mu_{n,i}$  (where ties are broken arbitrarily).
1: Input: slack  $\varepsilon_0 > 0$ , proportion  $\beta \in (0,1)$  (only for fixed proportions).
2: Set  $\hat{\iota}_n \in \arg \max_{i \in [K]} \mu_{n,i}$ ,  $B_n = \hat{\iota}_n$  and  $C_n \in \arg \min_{i \neq B_n} \frac{\mu_{n,B_n} - \mu_{n,i} + \varepsilon_0}{\sqrt{1 / N_{n,B_n} + 1 / N_{n,i}}}$ .
3: Set  $\bar{\beta}_{n+1}(B_n, C_n) = (T_n(B_n, C_n)\bar{\beta}_n(B_n, C_n) + \beta_n(B_n, C_n)) / T_{n+1}(B_n, C_n)$  with [fixed]  $\beta_n(i,j) = \beta$  or [IDS]  $\beta_n(i,j) = N_{n,j} / (N_{n,i} + N_{n,j})$  and  $T_{n+1}(B_n, C_n) = T_n(B_n, C_n) + 1$ .
4: Set  $I_{n} = C_{n}$  if  $N_{n,C_n}^{B_n} \leq (1 - \bar{\beta}_{n + 1}(B_n,C_n))T_{n + 1}(B_n,C_n)$ , otherwise set  $I_{n} = B_{n}$ .
5: Output: next arm to sample  $I_{n}$  and next recommendation  $\hat{v}_{n}$ .
Figure 1: EB-TC $_{\varepsilon_0}$  algorithm with fixed or IDS proportions.
2.1 Anytime Top Two sampling rule
We start by sampling each arm once. At time  $n > K$ , a Top Two sampling rule defines a leader  $B_{n} \in [K]$  and a challenger  $C_{n} \neq B_{n}$ . It then chooses the arm to pull among them. For the leader/challenger pair, we consider the Empirical Best (EB) leader  $B_{n}^{\mathrm{EB}} = \hat{i}_{n}$  and, given a slack  $\varepsilon_0 > 0$ , the Transportation Cost  $(\mathrm{TC}_{\varepsilon_0})$  challenger
$$
C _ {n} ^ {\mathrm {T C} \varepsilon_ {0}} \in \underset {i \neq B _ {n} ^ {\mathrm {E B}}} {\arg \min } \frac {\mu_ {n , B _ {n} ^ {\mathrm {E B}}} - \mu_ {n , i} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , B _ {n} ^ {\mathrm {E B}}} + 1 / N _ {n , i}}}. \tag {2}
$$
The algorithm then needs to choose between  $B_{n}$  and  $C_{n}$ . In order to do so, we use a so-called tracking procedure [15]. We define one tracking procedure per pair of leader/challenger  $(i,j) \in [K]^{2}$  such that  $i \neq j$ , hence we have  $K(K - 1)$  independent tracking procedures. For each pair  $(i,j)$  of leader and challenger, the associated tracking procedure will ensure that the proportion of times the algorithm pulled the leader  $i$  remains close to a target average proportion  $\bar{\beta}_{n}(i,j) \in (0,1)$ . At each round  $n$ , only one tracking rule is considered, i.e. the one of the pair  $(i,j) = (B_{n},C_{n})$ .
We define two variants of the algorithm that differ in the way they set the proportions  $\bar{\beta}_n(i,j)$ . Fixed proportions set  $\bar{\beta}_n(i,j) = \beta$  for all  $(n,i,j) \in \mathbb{N} \times [K]^2$ , where  $\beta \in (0,1)$  is fixed beforehand. Information-Directed Selection (IDS) [40] defines  $\beta_n(i,j) = N_{n,j} / (N_{n,i} + N_{n,j})$  and sets  $\bar{\beta}_n(i,j) := T_n(i,j)^{-1} \sum_{t \in [n-1]} \mathbb{1}((B_t, C_t) = (i,j)) \beta_t(i,j)$  where  $T_n(i,j) := \sum_{t \in [n-1]} \mathbb{1}((B_t, C_t) = (i,j))$  is the selection count of arms  $(i,j)$  as leader/challenger.
Let  $N_{n,j}^{i} \coloneqq \sum_{t \in [n-1]} \mathbb{1}((B_t, C_t) = (i, j), I_t = j)$  be the number of pulls of arm  $j$  at rounds in which  $i$  was the leader. We set  $I_n = C_n$  if  $N_{n,C_n}^{B_n} \leq (1 - \bar{\beta}_{n+1}(B_n, C_n)) T_{n+1}(B_n, C_n)$  and  $I_n = B_n$  otherwise. Using Theorem 6 in [12] for each tracking procedure (i.e. each pair  $(i, j)$ ) yields Lemma 2 (proved in Appendix H).
Lemma 2. For all  $n > K$ ,  $i \in [K]$ ,  $j \neq i$ , we have  $-1/2 \leq N_{n,j}^{i} - (1 - \bar{\beta}_{n}(i,j))T_{n}(i,j) \leq 1$ .
The  $\mathrm{TC}_{\varepsilon_0}$  challenger seeks to minimize an empirical version of a quantity that appears in the lower bound for  $\varepsilon_0$ -BAI (Lemma 1). As such, it is a natural extension of the TC challenger used in the T3C algorithm [36] for  $\varepsilon_0 = 0$ . In earlier works on Top Two methods [34, 32, 36], the choice between leader and challenger is randomized: given a fixed proportion  $\beta \in (0,1)$ , set  $I_n = B_n$  with probability  $\beta$ , otherwise  $I_n = C_n$ . [20] replaced randomization by tracking, and [40] proposed IDS to define adaptive proportions  $\beta_n(B_n,C_n) \in (0,1)$ . In this work we study both fixed proportions with  $\beta = 1/2$  and adaptive proportions with IDS. Empirically, we observe slightly better performances when using IDS (e.g. Figure 7 in Appendix J.2.1). While [20] tracked the leader with  $K$  procedures, we consider  $K(K - 1)$  independent tracking procedures depending on the current leader/challenger pair.
Choosing  $\varepsilon_0$  [21] shows that EB-TC (i.e.  $\mathrm{EB - TC}_{\varepsilon_0}$  with slack  $\varepsilon_0 = 0$ ) suffers from poor empirical performance for moderate  $\delta$  in BAI (see Appendix D.3 in [21] for a detailed discussion). Therefore, the choice of the slack  $\varepsilon_0 > 0$  is critical since it acts as a regularizer which naturally induces sufficient exploration. By setting  $\varepsilon_0$  too small, the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm will become as greedy as EB-TC and perform poorly. Having  $\varepsilon_0$  too large will flatten differences between sub-optimal arms, hence it will behave more uniformly. We observe from the theoretical guarantees and from our experiments that it is best to take  $\varepsilon_0 = \varepsilon$  for  $\varepsilon$ -BAI, but the empirical performance is only degrading slowly for  $\varepsilon_0 > \varepsilon$ . Taking  $\varepsilon_0 < \varepsilon$  leads to very poor performance. We discuss this trade-off in more details in our experiments (e.g. Figures 5, 6 and 7 in Appendix J.2.1). When tackling BAI, the limitation of EB-TC can be solved by adding an implicit exploration mechanism in the choice of the leader/challenger
pair. For the choice of leader, we can use randomization (TS leader [34, 32, 36]) or optimism (UCB leader [20]). For the choice of the challenger, we can use randomization (RS challenger [34]) or penalization (TCI challenger [21], KKT challenger [40] or EI challenger [32]).
Anytime sampling rule  $\mathrm{EB - TC}_{\varepsilon_0}$  is independent of a budget of samples  $T$  or a confidence parameter  $\delta$ . This anytime sampling rule can be regarded as a stream of empirical means/counts  $(\mu_n, N_n)_{n > K}$  (which could trigger stopping) and a stream of recommendations  $\hat{i}_n = i^{\star}(\mu_n)$ . These streams can be used by agents with different kinds of objectives. The fixed-confidence setting couples it with a stopping rule to be  $(\varepsilon, \delta)$ -PAC. It can also be used to get an  $\varepsilon$ -good recommendation with large probability at any given time  $n$ .
2.2 Stopping rule for fixed-confidence  $\varepsilon$ -best-arm identification
In addition to the sampling and recommendation rules, the fixed-confidence setting requires a stopping rule. Given an error/confidence pair, the  $\mathrm{GLR}_{\varepsilon}$  stopping rule [15] prescribes to stop at the time
$$
\tau_ {\varepsilon , \delta} = \inf  \left\{n > K \mid \min  _ {i \neq i _ {n}} \frac {\mu_ {n , \hat {i} _ {n}} - \mu_ {n , i} + \varepsilon}{\sqrt {1 / N _ {n , \hat {i} _ {n}} + 1 / N _ {n , i}}} \geq \sqrt {2 c (n - 1 , \delta)} \right\} \quad \text {w i t h} \quad \hat {i} _ {n} = i ^ {\star} (\mu_ {n}), \tag {3}
$$
where  $c:\mathbb{N}\times (0,1)\to \mathbb{R}_+$  is a threshold function. Lemma 3 gives a threshold ensuring that the  $\mathrm{GLR}_{\varepsilon}$  stopping rule is  $(\varepsilon ,\delta)$ -PAC for all  $\varepsilon \geq 0$  and  $\delta \in (0,1)$ , independently of the sampling rule.
Lemma 3 ([27]). Let  $\varepsilon \geq 0$  and  $\delta \in (0,1)$ . Given any sampling rule, using the threshold
$$
c (n, \delta) = 2 \mathcal {C} _ {G} (\log ((K - 1) / \delta) / 2) + 4 \log (4 + \log (n / 2)) \tag {4}
$$
with the stopping rule (3) with error/confidence pair  $(\varepsilon, \delta)$  yields a  $(\varepsilon, \delta)$ -PAC algorithm for sub-Gaussian distributions. The function  $\mathcal{C}_G$  is defined in (23). It satisfies  $\mathcal{C}_G(x) \approx x + \log(x)$ .
3 Fixed-confidence theoretical guarantees
To study  $\varepsilon$ -BAI in the fixed-confidence setting, we couple  $\mathrm{EB - TC}_{\varepsilon_0}$  with the  $\mathrm{GLR}_{\varepsilon}$  stopping rule (3) using error  $\varepsilon \geq 0$ , confidence  $\delta \in (0,1)$  and threshold (4). The resulting algorithm is  $(\varepsilon ,\delta)$ -PAC by Lemma 3. We derive upper bounds on the expected sample complexity  $\mathbb{E}_{\nu}[\tau_{\varepsilon ,\delta}]$  both in the asymptotic regime of  $\delta \to 0$  (Theorem 1) and for finite confidence when  $\varepsilon = \varepsilon_0$  (Theorem 2).
Theorem 1. Let  $\varepsilon \geq 0$ ,  $\varepsilon_0 > 0$  and  $(\beta, \delta) \in (0,1)^2$ . Combined with  $GLR_{\varepsilon}$  stopping (3), the  $EB-TC_{\varepsilon_0}$  algorithm is  $(\varepsilon, \delta)$ -PAC and it satisfies that, for all  $\nu \in \mathcal{D}^K$  with mean  $\mu$  such that  $|i^{\star}(\mu)| = 1$ ,