SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {P} _ {\boldsymbol {\mu}} \left[ X \geq t F _ {S} (\boldsymbol {\omega}), \boldsymbol {\omega} (t) \in W _ {n} ^ {N} \right] = \mathbb {P} _ {\boldsymbol {\mu}} \left[ \mathbb {1} \{\boldsymbol {\omega} (t) \in W _ {n} ^ {N} \} e ^ {\alpha (X - t F _ {S} (\boldsymbol {\omega} (t)))} \geq 1 \right] \\ \leq \mathbb {E} _ {\boldsymbol {\mu}} \left[ \mathbb {1} \left\{\boldsymbol {\omega} (t) \in W _ {n} ^ {N} \right\} e ^ {\alpha \left(X - t F _ {S} (\boldsymbol {\omega} (t))\right)} \right] \\ \leq \mathbb {E} _ {\boldsymbol {\mu}} \left[ \mathbb {1} \left\{\omega (t) \in W _ {n} ^ {N} \right\} e ^ {\alpha X} \right] e ^ {- \frac {\alpha u (n - 1)}{N}}, \tag {7} \\ \end{array}
$$
where the last inequality uses the definition of  $W_{n}^{N}$  (see (5)). By applying Hölder's inequality with  $p, q$ , where  $p \in [1, 1 / \alpha)$  and  $q = p / (p - 1)$  on r.h.s. of (7), we deduce that  $\log \mathbb{P}_{\mu}\left[\hat{\mu}(t) \in S, \omega(t) \in W_{n}^{N}\right]$  is at most
$$
\left. \left(\log \mathbb {E} _ {\boldsymbol {\mu}} \left[ e ^ {\alpha p X} \right]\right) / p + \left(\log \mathbb {E} _ {\boldsymbol {\mu}} \left[ \mathbb {1} \{\boldsymbol {\omega} (t) \in W _ {n} ^ {N} \} \right]\right) / q - \frac {\alpha u (n - 1)}{N}. \right.
$$
As  $\alpha p\in (0,1)$ , Lemma 2 in Appendix B shows that the first term above is  $o(t)$ . Using definition of the rate function, (3) with  $C = W_{n}^{N}$ , on the second term yields that  $\lim_{t\to \infty}\frac{1}{t}\log \frac{1}{\mathbb{P}_{\mu}[\hat{\mu}(t)\in S,\omega(t)\in W_{n}^{N}]}$  is lower bounded by
$$
(1 / q) \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} I (\boldsymbol {\omega}) + \frac {\alpha u (n - 1)}{N} = (1 - 1 / p) \inf  _ {\boldsymbol {\omega} \in W _ {n} ^ {N}} I (\boldsymbol {\omega}) + \frac {\alpha u (n - 1)}{N}.
$$
As  $p$  can be arbitrarily close to  $1 / \alpha$ , we get the lower bound  $(1 - \alpha)I(\omega) + \frac{\alpha u(n - 1)}{N}$ . Further choosing  $\alpha$  close to either 1 or 0, the proof is completed.
3.4 Improved analysis of the Successive Rejects algorithm
In SR, the set of candidate arms is initialized as  $\mathcal{C}_K = [K]$ . The budget of samples is partitioned into  $K - 1$  phases, and at the end of each phase, SR discards the empirical worst arm from the candidate set. In each phase, SR uniformly samples the arms in candidate set. The lengths of phases are set as follows. Define  $\overline{\log} K := \frac{1}{2} + \sum_{k=2}^{K} \frac{1}{k}$ . The candidate set is denoted by  $\mathcal{C}_j$  when it has  $j > 2$  arms. In the corresponding phase, (i) each arm in  $\mathcal{C}_j$  is sampled until the round  $t$  when  $\min_{k \in \mathcal{C}_j} N_k(t)$  reaches  $T / (j\overline{\log} K)$  (recall that  $N_k(t)$  is the number of times arm  $k$  has been sampled up to round  $t$ ); (ii) the empirical worst arm, denoted by  $\ell_j$ , is then discarded, i.e.,  $\mathcal{C}_{j-1} = \mathcal{C}_j \setminus \{\ell_j\}$ . During the last phase, the algorithm equally samples the two remaining arms and finally recommends  $\hat{i}$ , the arm with higher empirical mean in  $\mathcal{C}_2$ . The pseudo code is presented in Algorithm 1.
Algorithm 1: SR
initialization  $\mathcal{C}_K\gets [K],j\gets K;$
for  $(t = 1,\dots ,T)$  do
if  $(j > 2$  and  $\min_{k\in \mathcal{C}_j}N_k(t)\geq \frac{T}{j\log K})$  then
$\ell_{j}\gets \mathrm{argmin}_{k\in \mathcal{C}_{j}}\hat{\mu}_{k}(t)$  (tie broken arbitrarily),  $\mathcal{C}_{j - 1}\leftarrow \mathcal{C}_j\setminus \{\ell_j\}$  , and  $j\gets j - 1$
sample  $A_{t}\gets \mathrm{argmin}_{k\in \mathcal{C}_{j}}N_{k}(t)$  (tie broken arbitrarily), update  $\{N_k(t)\}_{k\in \mathcal{C}_j}$  and  $\hat{\pmb{\mu}} (t)$
$\ell_2 \gets \arg \min_{k \in \mathcal{C}_2} \hat{\mu}_k(T)$  and return  $\hat{\imath} \gets \arg \max_{k \in \mathcal{C}_2} \hat{\mu}_k(T)$  (tie broken arbitrarily).
We apply the corollary (c) in Section 3.2 to improve the existing performance guarantees of SR. To simplify the presentation, we assume wlog that  $\mu_1 > \mu_2 \geq \ldots \geq \mu_K$ . For  $j = 2, \ldots, K$ , define
$$
\Gamma_ {j} = \min  _ {J \in \mathcal {J}} \inf  \left\{\sum_ {k \in J} d \left(\lambda_ {k}, \mu_ {k}\right): \boldsymbol {\lambda} \in \mathbb {R} ^ {K}, \lambda_ {1} \leq \min  _ {k \in J} \lambda_ {k} \right\}, \tag {8}
$$
where  $\mathcal{J} = \{J\subseteq [K]:|J| = j,1\in J\}$
Theorem 2. Let  $\mu \in \Lambda$ . Under SR, we have: for  $j = 2,\dots ,K$ ,  $\varlimsup_{T\to \infty}\frac{1}{T}\log\frac{1}{\mathbb{P}_{\mu}[\ell_j=1]}\geq\frac{\Gamma_j}{j\log K}$ . Hence, the error probability of SR is upper bounded by  $\varliminf_{T\to \infty}\frac{1}{T}\log\frac{1}{\mathbb{P}_{\mu}[i\neq 1]}\geq\min_{j\neq 1}\frac{\Gamma_j}{j\log K}$ .
Proof of Theorem 2. Fix  $j\in \{2,\ldots ,K\}$ . Observe that
$$
\mathbb {P} _ {\boldsymbol {\mu}} \left[ \ell_ {j} = 1 \right] = \sum_ {J \in \mathcal {J}} \mathbb {P} _ {\boldsymbol {\mu}} \left[ \ell_ {j} = 1, \mathcal {C} _ {j} = J \right] \leq | \mathcal {J} | \max _ {J \in \mathcal {J}} \mathbb {P} _ {\boldsymbol {\mu}} \left[ \ell_ {j} = 1, \mathcal {C} _ {j} = J \right],
$$
$$
\lim  _ {T \rightarrow \infty} \frac {1}{T} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \ell_ {j} = 1 ]} \geq \min  _ {J \in \mathcal {J}} \lim  _ {T \rightarrow \infty} \frac {1}{T} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \ell_ {j} = 1 , \mathcal {C} _ {j} = J ]} \tag {9}
$$
as  $|\mathcal{J}| < \infty$ .
Since  $\ell_{j}$  is selected at the  $\theta T$ -th round<sup>1</sup> where  $\theta = (1 + \sum_{k = j + 1}^{K}\frac{1}{k}) / \overline{\log} K$ , the event  $\{\ell_{j} = 1, \mathcal{C}_{j} = J\}$  implies that  $\{\hat{\mu} (\theta T)\in \mathcal{S}, \omega (\theta T)\in W\}$ , where
$$
\mathcal {S} = \left\{\boldsymbol {\lambda} \in \mathbb {R} ^ {K}: \lambda_ {1} \leq \lambda_ {k}, \forall k \in J \right\} \text {a n d} W = \left\{\boldsymbol {\omega} \in \Sigma : \omega_ {k} = \frac {1}{\theta j \overline {{\log K}}}, \forall k \in J \right\}.
$$
In other words,  $\mathbb{P}_{\mu}[\ell_j = 1,\mathcal{C}_j = J]\leq \mathbb{P}_{\mu}[\hat{\mu} (\theta T)\in S,\omega (\theta T)\in W]$ . Applying (c) in Section 3.2 with the above  $S$  and  $W$  yields that  $\underline{\lim}_{T\to \infty}\frac{1}{\theta T}\log \frac{1}{\mathbb{P}_{\mu}[\hat{\mu} (\theta T)\in S,\omega(\theta T)\in W]}$  is larger than
$$
\inf  _ {\boldsymbol {\omega} \in W} \inf  _ {\boldsymbol {\lambda} \in \operatorname {c l} (S)} \Psi (\boldsymbol {\lambda}, \boldsymbol {\omega}) \geq \frac {1}{\theta j \log K} \inf  \left\{\sum_ {k \in J} d \left(\lambda_ {k}, \mu_ {k}\right): \lambda_ {1} \leq \min  _ {k \in J} \lambda_ {k} \right\} \geq \frac {\Gamma_ {j}}{\theta j \overline {{\log}} K}, \tag {10}
$$
where the first inequality uses the fact that KL-divergences and the components of  $\omega$  are nonnegative, and the second one is due to the definition (8) of  $\Gamma_{j}$ . Combining (9) and (10) completes the proof.
The upper bound derived in Theorem 2 is tighter than those recently derived in (Barrier et al., 2022). Indeed, for any  $J \in \mathcal{J}$ , since  $|J| = j$ , one can find at least one index in  $J$  at least larger than  $j$ , say  $k_{J}$ . Hence,
$$
\Gamma_ {j} \geq \min  _ {J \in \mathcal {I}} \inf  _ {\boldsymbol {\lambda} \in \mathbb {R} ^ {K}, \lambda_ {1} \leq \lambda_ {k _ {J}}} d (\lambda_ {1}, \mu_ {1}) + d (\lambda_ {k _ {J}}, \mu_ {k _ {J}}) \geq \inf  _ {\boldsymbol {\lambda} \in \mathbb {R} ^ {K}, \lambda_ {1} \leq \lambda_ {j}} d (\lambda_ {1}, \mu_ {1}) + d (\lambda_ {j}, \mu_ {j}).
$$
The r.h.s. in the previous inequality corresponds to the upper bounds derived by (Barrier et al., 2022).
To simplify the presentation and avoid rather intricate computations involving the KL-divergences, in the remaining of the paper, we restrict our attention to specific classes of reward distributions.
Assumption 1. The rewards are bounded with values in  $(0,1)$ . The reward distributions  $\nu_{1},\ldots ,\nu_{K}$  are Bernoulli distributions such that  $\nu_{a}$  is of mean  $a$ , and for any  $a\neq b$ ,  $d(a,b)\geq 2(a - b)^{2}$  (this is a consequence of Pinsker's inequality as rewards are in  $(0,1)$ ).
Under Assumption 1, we have  $\Gamma_j \geq 2\xi_j$  (a direct consequence of Proposition 3 in Appendix D.1), where for  $j = 2,\ldots ,K$ ,
$$
\xi_ {j} = \inf  _ {\boldsymbol {\lambda} \in [ 0, 1 ] ^ {j}} \left\{\sum_ {k = 1} ^ {j} (\lambda_ {k} - \mu_ {k}) ^ {2}: \lambda_ {1} \leq \min  _ {k = 1, \dots , j} \lambda_ {k} \right\}.
$$
We give an explicit expression of  $\xi_{j}$  in Proposition 1, presented in Appendix D.1. Moreover,  $2\xi_{j}$  is clearly larger than  $2\inf_{\lambda_1\leq \lambda_j}\left\{(\lambda_1 - \mu_1)^2 +(\lambda_j - \mu_j)^2\right\} = (\mu_1 - \mu_j)^2$  , and hence  $\min_{j\neq 1}\Gamma_j / (j\overline{\log} K)\geq \min_{j\neq 1}(\mu_1 - \mu_j)^2 /(j\overline{\log} K)$  . This implies that our error probability upper bound is better than that derived in (Audibert et al., 2010).
Example 1. To illustrate the improvement brought by Theorem 2 on the performance guarantees of SR, consider the simple example with 3 Bernoulli arms and  $\mu = (0.9, 0.1, 0.1)$ . Then  $\min_{j \neq 1} (\mu_1 - \mu_j)^2 / (j\overline{\log}3) = 0.16$  for the upper bound presented in (Audibert et al., 2010). From Proposition 1, instead we get  $\min_{j \neq 1} 2\xi_j / (j\overline{\log}3) = 0.21$ .
To simplify the presentation, we ignore cases where  $\theta T$  is not an integer. Refer to Appendix E for details.
4 Continuous Rejects Algorithms
In this section, we present CR, a truly adaptive algorithm that can discard an arm in any round. We propose two variants of the algorithm, CR-C using a conservative criterion to discard arms and CR-A discarding arms more aggressively. Using the Large Deviation results of Theorem 1, we establish error probability upper bounds for both CR-C and CR-A.
4.1 The CR-C and CR-A algorithms
As SR, CR initializes its candidate set as  $\mathcal{C}_K = [K]$ . For  $j \geq 2$ ,  $\mathcal{C}_j$  denotes the candidate set when it is reduced to  $j$  arms. When  $j > 2$ , the algorithm samples arms in the candidate set  $\mathcal{C}_j$  uniformly until a discarding condition is met. The algorithm then discards the empirically worst arm  $\ell_j \in \mathcal{C}_j$ , i.e.,  $\mathcal{C}_{j-1} \gets \mathcal{C}_j \setminus \{\ell_j\}$ . More precisely, in round  $t$ , if there are  $j$  candidate arms remaining and if  $\ell(t)$  denotes the empirically worst candidate arm, the discarding condition is  $N_{\ell(t)}(t) > \max_{k \notin \mathcal{C}_j} N_k(t)$ , ( $\forall k \in \mathcal{C}_j$ ,  $N_{\ell(t)}(t) = N_k(t)^2$ ), and
$$
\text {f o r} \mathbb {C R} - \mathbb {C}: \quad \min  _ {k \in \mathcal {C} _ {j}, k \neq \ell (t)} \hat {\mu} _ {k} (t) - \hat {\mu} _ {\ell (t)} (t) \geq G \left(\frac {\sum_ {k \in \mathcal {C} _ {j}} N _ {k} (t) \overline {{\log j}}}{T - \sum_ {k \notin \mathcal {C} _ {j}} N _ {k} (t)}\right), \tag {11}
$$
$$
\text {f o r} \mathrm {C R} - \mathrm {A}: \quad \frac {\sum_ {k \in \mathcal {C} _ {j} , k \neq \ell (t)} \hat {\mu} _ {k} (t)}{j - 1} - \hat {\mu} _ {\ell (t)} (t) \geq G \left(\frac {\sum_ {k \in \mathcal {C} _ {j}} N _ {k} (t) \overline {{\log}} j}{T - \sum_ {k \notin \mathcal {C} _ {j}} N _ {k} (t)}\right), \tag {12}
$$
where  $G(\beta) = 1 / \sqrt{\beta} - 1$  for all  $\beta > 0$ . The idea behind (11) is to keep the probability of discarding the best arm at most smaller than that of SR while using less budget. Note that (12) is easier to achieve than (11). CR-A is hence more aggressive than CR-C, and reduces the set of arms to  $C_2$  faster, but at the expense of a higher risk. After discarding  $\ell_3$ , CR will sample the arms in  $C_2$  evenly, and recommend the empirical best arm in  $C_2$  when the budget is exhausted. The pseudo-code of CR is presented in Algorithm 2.
Algorithm 2: CR-C and CR-A
Input:  $\theta_0\in (0,\frac{1}{\log K})\cap \mathbb{Q}$  independent of  $T$  (can be chosen as small as one wishes)   
initialization  $\begin{array}{rl} & {\mathcal{C}_K\gets [K],j\gets K,}\\ & {\mathrm{for~}t = K + 1,\ldots ,\lfloor \theta_0T\rfloor \mathrm{do}}\\ & {\mathrm{for~}A_t\gets \operatorname *{argmin}_{k\in \mathcal{C}_j}N_k(t)\mathrm{(tie~broken~arbitrarily)},}\\ & {\mathrm{end}} \end{array}$    
for  $\ell (t) = \lfloor \theta_0T\rfloor +1,\dots,T)$  do if  $j > 2,N_{\ell (t)}(t) > \max_{k\notin \mathcal{C}_j}N_k(t),(\forall k\in \mathcal{C}_j,N_{\ell (t)}(t) = N_k(t)),$  and (11) (resp. (12)) holds for CR-C (resp. CR-A) then  $\begin{array}{rl}{|\ell_j\leftarrow \ell (t),} & {\mathcal{C}_{j - 1}\leftarrow \mathcal{C}_j\setminus \{\ell_j\} ,j\leftarrow j - 1} \end{array}$  sample  $A_{t}\gets \operatorname {argmin}_{k\in \mathcal{C}_{j}}N_{k}(t)$  (tie broken arbitrarily), update  $\{N_k(t)\}_{k\in \mathcal{C}_j}$  and  $\hat{\mu} (t)$  .   
end   
 $\ell_2\gets \ell (T);$  return  $\hat{i}\gets \operatorname *{argmax}_{k\in \mathcal{C}_2}\hat{\mu}_k(T)$  (tie broken arbitrarily).
4.2 Analysis of CR-C and CR-A
As in Section 3.4,  $\mu_1 > \mu_2 \geq \ldots \geq \mu_K$  is assumed wlog and we further define  $\mu_{K+1} = 0$ . We introduce the following instance-specific quantities needed to state our error probability upper bounds. For  $j \in \{2, \ldots, K\}$ , define
$$
\psi_ {j} = \frac {j - 1}{j} \left(\mu_ {1} - \frac {\sum_ {k = 2} ^ {j} \mu_ {k}}{j - 1}\right) ^ {2}, \quad \bar {\psi} _ {j} = \frac {j - 1}{j} \left(\mu_ {1} - \frac {\sum_ {k = 2} ^ {j - 1} \mu_ {k} + \mu_ {j + 1}}{j - 1}\right) ^ {2}, \quad \zeta_ {j} = \mu_ {j} - \mu_ {j + 1},
$$
2This condition is only imposed to simplify our analysis. It may be removed.
$$
\varphi_ {j} = \frac {\sum_ {k = 1} ^ {j} \mu_ {k}}{j} - \mu_ {j + 1}, \quad \text {a n d} \quad \bar {\xi_ {j}} = \inf  \left\{\sum_ {k = 1} ^ {K} (\lambda_ {k} - \mu_ {k}) ^ {2}: \boldsymbol {\lambda} \in [ 0, 1 ] ^ {K},   \lambda_ {1} \leq \min  _ {k = 2, \dots , j - 1, j + 1} \lambda_ {k} \right\}.
$$
Here we remark  $\bar{\xi}_j\geq \xi_j$  and  $\bar{\psi}_j\geq \psi_j$ . These inequalities are proven in Proposition 3 and Proposition 5 in Appendix D.
Theorem 3. Let  $\pmb{\mu} \in [0,1]^K$ . Under CR-C,  $\varprojlim_{T \to \infty} \frac{1}{T} \log \frac{1}{\mathbb{P}_{\pmb{\mu}}[i \neq 1]}$  is larger than
$$
2 \min  _ {j = 2, \dots , K} \left\{\frac {\operatorname* {m i n} \left\{\max  \left\{\frac {\xi_ {j} \overline {{\log}} (j + 1) (1 - \alpha_ {j}) \mathbb {1} _ {\{j \neq K \}}}{\log j} , \xi_ {j} \right\} , \bar {\xi} _ {j} \right\}}{j \overline {{\log}} K} \right\},
$$
where  $\alpha_{j}\in \mathbb{R}$  is the real number such that  $\frac{2\xi_j(1 - \alpha_j)}{j\log j} = [(1 + \zeta_j)\sqrt{\alpha_j} -\sqrt{\frac{1}{(j + 1)\overline{\log(j + 1)}}}_+]^2.$
Theorem 4. Let  $\pmb{\mu} \in [0,1]^K$ . Under CR-A,  $\varlimsup_{T \to \infty} \frac{1}{T} \log \frac{1}{\mathbb{P}_{\pmb{\mu}}[\hat{i} \neq 1]}$  is larger than
$$
2\min_{j = 2,\ldots ,K}\left\{\frac{\min \left\{\max \left\{\frac{\psi_{j}\overline{\log}(j + 1)(1 - \alpha_{j})\mathbb{1}_{\{j\neq K\}}}{\log j},\psi_{j}\right\} ,\bar{\psi}_{j}\right\}}{j\overline{\log}K}\right\} ,
$$
where  $\alpha_{j}\in \mathbb{R}$  is the real number such that  $\frac{\psi_j(1 - \alpha_j)}{j\log j} = \frac{j}{j + 1} [((1 + \varphi_j)\sqrt{\alpha_j} -\sqrt{\frac{1}{(j + 1)\overline{\log}(j + 1)}})_+]^2.$
Note that Theorem 3 implies that CR-C enjoys better performance guarantees than SR, and hence has for now the best known error probability upper bounds.
Proof sketch. The complete proof of Theorems 3 and 4 are given in Appendices C.1 and C.2. We sketch that of Theorem 3. The proof consists in upper bounding  $\mathbb{P}_{\mu}[\ell_j = 1]$  for  $j \in \{2, \ldots, K\}$ . We focus here on the most challenging case where  $j \in \{3, \ldots, K - 1\}$  (the analysis is simpler when  $j = K$ , since the only possible allocation is uniform, and when  $j = 2$ , since the only possible round deciding  $\ell_2$  is the last round).
To upper bound  $\mathbb{P}_{\mu}[\ell_j = 1]$  using Theorem 1, we will show that it is enough to study the large deviations of the process  $\{\omega (\theta T)\}_{T\geq 1}$  for any fixed  $\theta \in [\theta_0,1]$  and to define a set  $S\subseteq [0,1]^K$  under which  $\ell_{j} = \ell (\theta T) = 1$ . We first observe that  $\ell_{j} = \ell (\theta T)$  restricts the possible values of  $\omega (\theta T)$ :  $\omega (\theta T)\in \mathcal{X}_j\coloneqq \left\{\pmb {x}\in \Sigma :\exists \sigma \in [K]^2\mathrm{s.t.}x_{\sigma (1)} = \dots = x_{\sigma (j)} > x_{\sigma (j + 1)} > \dots >x_{\sigma (K)} > 0\right\}$ . We can hence just derive the LDP satisfied by  $\{\omega (\theta T)\}_{T\geq 1}$  on  $\mathcal{X}_j$ . This is done in Appendix E, and we identify by  $I_{\theta}$  a rate function leading to an LDP upper bound. By defining
$$
\mathcal {X} _ {j, i} (\theta) = \left\{\boldsymbol {x} \in \mathcal {X} _ {j}: \theta x _ {\sigma (i)} i \overline {{\log i}} > 1 - \theta \sum_ {k = i + 1} ^ {K} x _ {\sigma (k)} \right\}, \forall i \in \{j, \dots , K \},
$$
As it is shown in Appendix E.1 that  $I_{\theta}(\pmb{x}) = \infty$  if  $\pmb{x} \in \mathcal{X}_{j,i}(\theta)$  for  $i \geq j$ , we may further restrict to  $\mathcal{X}_j \setminus \cup_{i = j}^K \mathcal{X}_{j,i}(\theta)$ .
Next, we explain how to apply Theorem 1 to upper bound  $\mathbb{P}_{\mu}[\ell_j = 1]$ . Let  $\mathcal{J} = \{J \subseteq [K] : |J| = j, 1 \in J\}$  as defined in Section 3.4. For all  $\beta, \theta \in (0,1]$  and  $J \in \mathcal{J}$ , we introduce the sets
$$
\mathcal {S} _ {J} (\beta) = \left\{\boldsymbol {\lambda} \in [ 0, 1 ] ^ {K}: \min  _ {k \in J, k \neq 1} \lambda_ {k} - \lambda_ {1} \geq G (\beta) \right\},
$$
$$
\mathcal {Z} _ {J} (\theta , \beta) = \left\{\boldsymbol {z} \in \mathcal {X} _ {j} \setminus \cup_ {i = j} ^ {K} \mathcal {X} _ {j, i} (\theta): (\forall k \in J, z _ {k} = \max  _ {k ^ {\prime} \in [ K ]} z _ {k ^ {\prime}}), \frac {\theta \sum_ {k \in J} z _ {k} \overline {{\log j}}}{1 - \theta \sum_ {k \notin J} z _ {k}} = \beta \right\}.
$$
Assume that in round  $t$ ,  $\ell_{j} = \ell(t) = 1$ ,  $\mathcal{C}_{j} = J$  and let  $\tau = \sum_{k \notin J} N_{k}(t) \leq t$  be the number of times arms outside  $J$  are pulled. While  $\omega(t) \notin \mathcal{X}_{j,j}(t / T)$ , we have  $\beta = \frac{(t - \tau) \log j}{T - \tau} \in (0,1]$ . Using the criteria (11), we observe that
$$
\begin{array}{l} \sum_ {t = K + 1} ^ {T} \sum_ {J \in \mathcal {J}} \mathbb {P} _ {\boldsymbol {\mu}} \left[ \ell_ {j} = \ell (t) = 1, \mathcal {C} _ {j} = J, \boldsymbol {\omega} (t) \in \mathcal {X} _ {j} \setminus \cup_ {i = j} ^ {K} \mathcal {X} _ {j, i} (\frac {t}{T}) \right] \\ \leq \sum_ {t = K + 1} ^ {T} \sum_ {J \in \mathcal {J}} \sum_ {\tau \leq t, \tau \in \mathbb {N}} \mathbb {P} _ {\boldsymbol {\mu}} \left[ \hat {\boldsymbol {\mu}} (t) \in \mathcal {S} _ {J} (\frac {(t - \tau) \overline {{\log}} j}{T - \tau}), \boldsymbol {\omega} (t) \in \mathcal {Z} _ {J} (\frac {t}{T}, \frac {(t - \tau) \overline {{\log}} j}{T - \tau}) \right]. \\ \end{array}
$$
To upper bound the r.h.s. in the above inequality, we combine the results of Theorem 1 and a partitioning technique (presented in Appendix G). This gives:
$$
\lim _ {T \to \infty} \frac {1}{T} \log \frac {1}{\mathbb {P} _ {\boldsymbol {\mu}} [ \hat {\boldsymbol {\mu}} (\theta T) \in \mathcal {S} _ {J} (\beta) , \boldsymbol {\omega} (\theta T) \in \mathcal {Z} _ {J} (\theta , \beta) ]} \geq \theta \inf _ {\boldsymbol {z} \in \operatorname {c l} (\mathcal {Z} _ {J} (\theta , \beta))} \max \{F _ {\mathcal {S} _ {J} (\beta)} (\boldsymbol {z}), I _ {\theta} (\boldsymbol {z}) \}.
$$
The proof is completed by providing lower bounds of  $\theta F_{S_J(\beta)}(z)$  and  $\theta I_{\theta}(z)$  for a fixed  $z \in \mathcal{Z}_J(\theta, \beta)$  with various  $J$ . Such bounds are derived in Appendix D.1 and E.3.1, respectively.
Example 2. To conclude this section, we just illustrate through a simple example the gain in terms of performance guarantees brought by CR compared to SR. Assume we have 50 Bernoulli arms with  $\mu_{1} = 0.95$ ,  $\mu_{2} = 0.85$ ,  $\mu_{3} = 0.2$ , and  $\mu_{k} = 0$  for  $k = 4,\ldots, 50$ . For SR, Theorem 2 states that with a budget of 5000 samples, the error probability of SR does not exceed  $1.93 \times 10^{-3}$ . With the same budget, Theorems 3 and 4 state that the error probabilities of CR-C and CR-A do not exceed  $6.40 \times 10^{-4}$  and  $6.36 \times 10^{-4}$ , respectively.
We note that in general, we cannot say that one of our two algorithms, CR-C or CR-A, has better guarantees than the other. This is demonstrated in the problem instances presented in Appendix K.1 and K.4.
We consider various problem instances to numerically evaluate the performance of CR. In these instances, we vary the number of arms from 5 to 55; we use Bernoulli distributed rewards, and vary the shape of the arm-to-reward mapping. For each instance, we compare CR to SR, SH, and UGapE.
Most of our numerical experiments are presented in Appendix K. Due to space constraints, we just provide an example of these results below. In this example, we have 55 arms with convex arm-to-reward mapping. The mapping has 10 steps, and the  $m$ -th step consists of  $m$  arms with same average reward, equal to  $\frac{3}{4} \cdot 3^{-\frac{m}{10}}$ . Table 1 presents the error probabilities averaged over 40,000 independent runs. Observe that CR-A performs better than CR-C (being aggressive when discarding arms has some benefits), and both versions of CR perform better than SR and all other algorithms.
Table 1: Error probability (in %).
T = 3,000	T = 4,000	T = 5,000
UGapE	(Gabillon et al., 2012)	24.7	21.3	18.9
SH	Karnin et al. (2013)	10.2	5.9	3.2
SR	Audibert et al. (2010)	5.5	2.8	1.3
CR-C	(this paper)	7.1	2.6	1.1
CR-A	(this paper)	4.7	1.6	0.6
In this paper, we have established, in MAB problems, a connection between the LDP satisfied by the sampling process (under any adaptive algorithm) and that satisfied by the empirical average rewards of the various arms. This connection has allowed us to improve the performance analysis of existing best arm identification algorithms, and to devise and analyze new algorithms with an increased level of adaptiveness. We show that one of these algorithms CR-C has better performance guarantees than existing algorithms and that it performs also better in practice in most cases.
Future research directions include: (i) developing algorithms with further improved performance guarantees - can the discarding conditions of CR be further optimized? (ii) Enhancing the Large Deviation analysis of adaptive algorithms - under which conditions, can we establish a complete LDP of the process  $\{Z(t)\}_{t\geq 1} = \{(\boldsymbol {\omega}(t),\hat{\boldsymbol{\mu}} (t))\}_{t\geq 1}$ ? Answering this question would constitute a strong step towards characterizing the minimal instance-specific error probability for best arm identification with fixed budget. (iii) Extending our approach to other pure exploration tasks: top- $m$  arm identification problems (Bubeck et al., 2013), best arm identification in structured bandits (Yang and Tan, 2022; Azizi et al., 2022), or best policy identification in reinforcement learning.
The authors would like to express their gratitude to Guo-Jhen Wu for his invaluable discussion during the initial stages of this project. R.-C Tzeng's research is supported by the ERC Advanced Grant REBOUND (834862), A. Proutiere is supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation, and Digital Futures.