SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Explicit upper bound Since  $g_{m}$  is itself a non-standard function (like logarithm), it is not straightforward to obtain an explicit upper bound on (11). For  $g_{u}$ , it was done in Lemma D.10 by using Lemma D.7.
Using Lemma D.7, we obtain that
$$
\begin{array}{l} x \geq 2 H (\mu) g _ {m} \left(x ^ {\alpha}\right) / \beta + K / \beta + 2 \\ \Longleftrightarrow \frac {x - c _ {0}}{c _ {\mu}} - \log \left(\frac {x - c _ {0}}{c _ {\mu}}\right) \geq 2 s \alpha^ {2} \log (x) + 2 \log (2 + \alpha^ {2} \log x) + 2 \\ \Longleftarrow y - \log (y) \geq \frac {1}{s \alpha^ {2} + 1 / 2} \left(\log (2 + \alpha^ {2} \log x) + \frac {c _ {0}}{2 c _ {\mu}}\right) + \frac {s \alpha^ {2}}{s \alpha^ {2} + 1 / 2} \log c _ {\mu} + c _ {1} \\ \iff x \geq h _ {2} \left(c _ {\mu}, x\right), \\ \end{array}
$$
where we used  $y = \frac{x}{c_{\mu}(2s\alpha^2 + 1)}$ ,  $c_{1} = \log (2s\alpha^{2} + 1) + \frac{1}{s\alpha^{2} + 1 / 2}$ ,  $c_{\mu} = 2H(\mu)$ ,  $c_{0} = K / \beta + 2$  and define
$$
h _ {2} (z, x) := z \left(2 s \alpha^ {2} + 1\right) \bar {W} _ {- 1} \left(\frac {1}{s \alpha^ {2} + 1 / 2} \left(s \alpha^ {2} \log z + \frac {c _ {0}}{2 z} + \log \left(2 + \alpha^ {2} \log x\right)\right) + c _ {1}\right). \tag {12}
$$
For both equivalences above, we used that we are interested in larger values of  $x$ , hence we used that  $\frac{x - c_0}{c_\mu} \geq 1$ ,  $\frac{x}{c_\mu(2s\alpha^2 + 1)} \geq 1$ ,  $2s\alpha^2\log (x) + 2\log (2 + \alpha^2\log x) + 2 \geq 1$  and
$\frac{1}{s\alpha^2 + 1 / 2}\left(s\alpha^2\log c_\mu +\frac{c_0}{2c_\mu} +\log (2 + \alpha^2\log x)\right) + c_1\geq 1.$  Since those conditions are implied by  $x\geq h_{2}(c_{\mu},x)$ , those conditions are neither restrictive nor informative.
Therefore, we have shown that  $\tilde{C}_{\mu}$  defined in (11) satisfies
$$
\tilde {C} _ {\mu} \leq \sup  \left\{x \in \mathbb {N} ^ {*} \mid x <   h _ {2} (2 H (\mu), x) \right\},
$$
where  $h_2(z,x)$  is defined in (12) is such that  $h_2(z,x)\approx_x 2z\log (2 + \alpha^2\log x)$  and  $h_2(z,x)\approx_z 2s\alpha^2 z\log z$ .
D.4 Uniform sampling
While the shortcomings of uniform sampling are well known for general bandit instances, it is also clear that uniform sampling performs well for highly symmetric instances  $\mu$  such that  $w^{\star}(\mu)\approx 1_{K} / K$ . Therefore, we derive a non-asymptotic upper bound for uniform sampling (Theorem D.6), which allows a clear comparison with Theorem D.4 (and Corollary D.5).
Theorem D.6. Let  $\delta \in (0,1)$ . Combining the stopping rule (1) with threshold (2) and the uniform sampling rule yields a  $\delta$ -correct algorithm such that, for all  $\mu \in \mathbb{R}^K$  with  $|i^{\star}(\mu)| = 1$ ,
$$
\mathbb {E} _ {\mu} \left[ \tau_ {\delta} \right] \leq \max  \left\{T _ {1} (\delta), h _ {3} \left(\frac {8 \alpha K (1 + s)}{\min  _ {i \neq i ^ {\star}} \left(\mu_ {i ^ {\star}} - \mu_ {i}\right) ^ {2}}\right) \right\} + 1 + (2 K - 1) \zeta (s), \tag {13}
$$
where  $h_3(x) = x\overline{W}_{-1}(\log (x))$  for all  $x \geq e$  and  $h_3(x) = x$  for all  $x \in (0,e)$  and
$$
T _ {1} (\delta) = \sup  \left\{n > K \mid n - 1 \leq \frac {4 K}{\min  _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}} \left(\sqrt {c (n - 1 , \delta)} + \sqrt {\alpha (2 + s) \log n}\right) ^ {2} \right\}.
$$
Proof. Let  $n > K$  such that  $\mathcal{E}_n$  holds true and the algorithm has not stop yet, i.e.  $\mathcal{E}_n \cap \{n < \tau_{\delta}\}$ . Let  $t := \sup \left\{t \in [n^{1 / \alpha}, n] \mid (t - 1) / K \in \mathbb{N}, \hat{t}_t = i^\star \right\}$ . Using the stopping rule (1) and  $t \leq n < \tau_{\delta}$ , we obtain
$$
\sqrt {2 c (n - 1 , \delta)} \geq \sqrt {2 c (t - 1 , \delta)} \geq \min _ {i \neq \hat {i} _ {t}} \frac {\mu_ {t , \hat {i} _ {t}} - \mu_ {t , i}}{\sqrt {1 / N _ {t , \hat {i} _ {t}} + 1 / N _ {t , i}}} = \frac {\mu_ {t , \hat {i} _ {t}} - \mu_ {t , k _ {t}}}{\sqrt {1 / N _ {t , \hat {i} _ {t}} + 1 / N _ {t , k _ {t}}}},
$$
where  $k_{t} = \arg \min_{i\neq \hat{t}_{t}}\frac{\mu_{t,i_{t}} - \mu_{t,i}}{\sqrt{1 / N_{t,i_{t}} + 1 / N_{t,i}}}$ . Since we sample uniformly, for all time  $t$  such that  $(t - 1) / K\in \mathbb{N}$ , we have  $N_{t,i} = (t - 1) / K$  for all  $i\in [K]$ . Combining the above with Lemma D.2, we obtain
$$
\sqrt {2 c (n - 1 , \delta)} + \sqrt {2 \alpha (2 + s) \log n} \geq \frac {\mu_ {i ^ {\star}} - \mu_ {k _ {t}}}{\sqrt {1 / N _ {t , i ^ {\star}} ^ {i ^ {\star}} + 1 / N _ {t , k _ {t}} ^ {i ^ {\star}}}} \geq \sqrt {\frac {t - 1}{2 K}} \min _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}),
$$
where the last inequality is obtained by taking the minimum over  $i \in [K]$ . To conclude the proof, we simply have to link  $t$  with  $n$ . More precisely, we will show that  $n - t \leq K - 1$  and  $\hat{t}_t = i^\star$  for  $n$  large enough. By concentration, we have
$$
\mu_ {i} - \sqrt {\frac {K g _ {u} (t)}{t - 1}} = \mu_ {i} - \sqrt {\frac {g _ {u} (t)}{N _ {t , i}}} \leq \mu_ {t, i} \leq \mu_ {i} + \sqrt {\frac {g _ {u} (t)}{N _ {t , i}}} = \mu_ {i} + \sqrt {\frac {K g _ {u} (t)}{t - 1}}.
$$
Therefore, we have  $\mu_{t,i^{\star}} > \max_{i\neq i^{\star}}\mu_{t,i}$ , i.e.  $\hat{t}_t = i^\star$ , for all  $t > N_3$  where
$$
N _ {3} := \sup \left\{n \in \mathbb {N} \mid n - 1 \leq \frac {4 K g _ {u} (n)}{\min _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}} \right\},
$$
which means that we have at worse  $n - t \leq K - 1$ . For  $n \geq N_3 + K$ , we obtain  $t \geq n - (K - 1) > N_3$ .
Let  $c_{\mu} = \frac{8\alpha K(1 + s)}{\min_{i \neq i^{\star}} (\mu_{i^{\star}} - \mu_{i})^{2}}$  and  $x = n / c_{\mu}$ . Assume that  $c_{\mu} \geq e$ . Using the definition of  $g_{u}$  and Lemma D.7, direct manipulations yields
$$
n > \frac {4 K g _ {u} (n)}{\min  _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}} \iff x - \log (x) > \log (c _ {\mu}) \iff x > \overline {{W}} _ {- 1} (\log (c _ {\mu})) ,
$$
where the last inequality uses that  $\log (c_{\mu})\geq 1$  and  $x\geq 1$ , which is not restrictive (or informative) since it is implied by  $x > \overline{W}_{-1}(\log (c_{\mu}))$ . When  $c_{\mu} < e$ , which means that the problem is easy, we have directly that  $x - \log (x) > 1 > \log (c_{\mu})$  for  $x > 1$ . Therefore, the condition becomes  $x > 1$ , i.e.  $n\geq c_{\mu}$ . Defining  $h_3(x) = x\overline{W}_{-1}(\log (x))$  for all  $x\geq e$  and  $h_3(x) = x$  for all  $x\in (0,e)$ , we have
$$
N _ {3} \leq h _ {3} \left(\frac {8 \alpha K (1 + s)}{\min _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}}\right).
$$
Using a similar argument as the one in Appendix D.2 which rely on D.8 and the definition of  $T_{1}(\delta)$ , we can conclude the proof.
The structure of the non-asymptotic upper bound of uniform sampling in (13) is similar to the one for the TTUCB sampling rule in Theorem D.4. Therefore, we can compare the dominating terms for both the asymptotic and the non-asymptotic regime.
First, we look at the asymptotic dominant term, namely we compare  $T_0(\delta)$  and  $T_1(\delta)$ . Taking  $w = 1_K / K$  instead of  $w^{\star}(\mu)$  in the definition of  $T^{\star}(\mu)$ , it is direct to see that  $T^{\star}(\mu) \leq \frac{4K}{\min_{i \neq i^{\star}} (\mu_i^{\star} - \mu_i)^2}$ .
Using Lemma C.1, we have  $T_{\beta}^{\star}(\mu) \leq T^{\star}(\mu) \max \left\{ \frac{\beta^{\star}}{\beta}, \frac{1 - \beta^{\star}}{1 - \beta} \right\}$  where  $\beta^{\star} = w_{i^{\star}}^{\star}(\mu)$ . Therefore, while we can't say that  $T_{\beta}^{\star}(\mu) \frac{(1 + \varepsilon)^2}{\beta(1 - w_0)^d\mu(w_0)} \leq \frac{4K}{\min_{i \neq i^{\star}} (\mu_i^{\star} - \mu_i)^2}$  for all instances  $\mu$ , empirical evidences suggest that the gap is significant for reasonable instances. It is even possible to design instances where the gap between both notion of complexity explodes.
Second, we examine the dominating  $\delta$ -independent term. Using Lemma D.7, we obtain by concavity that for all  $x > e$
$$
0 <   h _ {1} (x) - h _ {3} (x) \leq 2 \overline {{W}} _ {- 1} ^ {\prime} (\log (x)) = 2 \left(1 - \frac {1}{\overline {{W}} _ {- 1} (\log (x))}\right) ^ {- 1},
$$
which yields that  $\lim_{x\to +\infty}h_1(x) - h_3(x)\leq 2$ . While those two functions diverges when  $x\to +\infty$ , their difference remains bounded by a finite quantity. Therefore,  $h_1$  and  $h_3$  are qualitatively similar. It is direct to see that  $\frac{2}{\min_{i\neq i^{\star}}(\mu_i^{\star} - \mu_i)^2} < H(\mu)\leq \frac{2K}{\min_{i\neq i^{\star}}(\mu_i^{\star} - \mu_i)^2}$ . Therefore, while we can't say that  $\alpha H(\mu)\leq \frac{2K}{\min_{i\neq i^{\star}}(\mu_i^{\star} - \mu_i)^2}$  for all instances  $\mu$ , the gap can become significant for some instances.
D.5 Technicalities
Lemma D.7 gathers properties on the function  $\overline{W}_{-1}$  which we used in this work.
Lemma D.7 ([22]). Let  $\overline{W}_{-1}(x) = -W_{-1}(-e^{-x})$  for all  $x\geq 1$ , where  $W_{-1}$  is the negative branch of the Lambert  $W$  function. The function  $\overline{W}_{-1}$  is increasing on  $(1, + \infty)$  and strictly concave on  $(1, + \infty)$ . In particular,  $\overline{W}_{-1}'(x) = \left(1 - \frac{1}{\overline{W}_{-1}(x)}\right)^{-1}$  for all  $x > 1$ . Then, for all  $y\geq 1$  and  $x\geq 1$ ,
Lemma D.8 is a standard result to upper bound the expected sample complexity of an algorithm.
Lemma D.8. Let  $(\mathcal{E}_n)_{n > K}$  be a sequence of events and  $T(\delta) > K$  be such that for  $n\geq T(\delta)$ ,  $\mathcal{E}_n\subset \{\tau_\delta \leq n\}$ . Then,  $\mathbb{E}_{\mu}[\tau_{\delta}]\leq T(\delta) + \sum_{n > K}\mathbb{P}_{\mu}(\mathcal{E}_n^{\complement})$ .
$$
\mathbb {E} _ {\mu} \left[ \tau_ {\delta} \right] = \sum_ {n} \mathbb {P} _ {\mu} \left(\tau_ {\delta} > n\right) \leq \sum_ {n <   T (\delta)} \mathbb {P} _ {\mu} \left(\tau_ {\delta} > n\right) + \sum_ {n \geq T (\delta)} \mathbb {P} _ {\mu} \left(\mathcal {E} _ {n} ^ {\mathbb {C}}\right) \leq T (\delta) + \sum_ {n > K} \mathbb {P} _ {\mu} \left(\mathcal {E} _ {n} ^ {\mathbb {C}}\right).
$$
Technical results Lemma D.9 shows that a linear lower bound on the number of samples allocated to the challenger, i.e.  $L_{n,i^{\star}} - N_{n,i^{\star}}^{i^{\star}}\gtrsim (1 - \beta)(n - 1)$
Lemma D.9. Let  $w\in (0,1 - \beta)$  and  $(\mathcal{E}_{1,n})_n$  as in (17). For all  $n\in \mathbb{N}^{\star}$ , under  $\mathcal{E}_{1,n}$
$$
\frac {w}{1 - \beta} \left(L _ {n, i ^ {*}} - N _ {n, i ^ {*}} ^ {i ^ {*}}\right) - 1 \geq (n - 1) w - f _ {1} (n),
$$
$$
f _ {1} (n) = 2 H (\mu) g _ {u} (n) / \beta + K / \beta + 2. \tag {14}
$$
Proof. Using Lemma D.3, we obtain that, under  $\mathcal{E}_{1,n}$
$$
\frac {w}{1 - \beta} \left(L _ {n, i ^ {\star}} - N _ {n, i ^ {\star}} ^ {i ^ {\star}}\right) - 1 = w L _ {n, i ^ {\star}} - \frac {w}{1 - \beta} - 1 \geq w (n - 1) - 2 H (\mu) g _ {u} (n) / \beta - K / \beta - 2.
$$
where the last inequality uses Lemma D.1 for  $t = n$  and  $w\in (0,1 - \beta)$
Lemma D.10 gives an explicit upper bound on the constant  $C_0(w_{-})$  defined implicitly in Appendix D.2.
Lemma D.10. Let  $w_{-} > 0$  and  $C_0(w_{-})$  as in (8). Then, we have
$$
C _ {0} (w _ {-}) \leq \max  \left\{h _ {1} \left(4 \alpha^ {2} (1 + s) H (\mu) / \beta\right), \left(\frac {2}{w _ {-}} + 1\right) ^ {1 / (\alpha - 1)} \right\} ^ {\alpha}.
$$
where  $h_1: \mathbb{R}_+^\star \to \mathbb{R}_+^\star$  is an increasing function for  $x \geq 2 + K/\beta$  defined as
$$
h _ {1} (x) := x \bar {W} _ {- 1} \left(\log (x) + \frac {2 + K / \beta}{x}\right). \tag {15}
$$
Proof. Using the definition of  $g_{u}$  and Lemma D.7, direct manipulations yields
$$
n ^ {1 / \alpha} \geq f _ {1} (n) \iff n ^ {1 / \alpha} \geq c _ {\mu} \log (n ^ {1 / \alpha}) + 2 + K / \beta \iff x - \log (x) \geq d _ {\mu} \iff x \geq \overline {{W}} _ {- 1} (d _ {\mu}),
$$
where  $x = n^{1 / \alpha} / c_{\mu}$ ,  $d_{\mu} = \log (c_{\mu}) + (2 + K / \beta) / c_{\mu}$  and  $c_{\mu} = 4\alpha^2 (1 + s)H(\mu) / \beta$ . For the last equivalence we used that we are only interested in  $x \geq 1$  (small values are not relevant for upper bounds). Since  $\overline{W}_{-1}(d_{\mu}) \geq 1$ , this condition is neither restrictive nor informative as we obtain the final condition  $x \geq \overline{W}_{-1}(d_{\mu})$ . Moreover, we can show
$$
d _ {\mu} \geq 1 \iff c _ {\mu} (\log (c _ {\mu}) - 1) \geq - (2 + K / \beta) \iff c _ {\mu} (\log (c _ {\mu}) - 1) \geq - 1,
$$
where the last part is true since  $2 + K / \beta \geq 1$  and  $\min_{x\in \mathbb{R}}x(\log (x) - 1) = -1$  by direct computations. Therefore, for all  $n\geq h_1(c_\mu)^\alpha$ , we have
$$
n - 1 \geq \frac {1}{w _ {-}} \left(n ^ {1 / \alpha} + f _ {1} (n)\right) \iff n - 1 \geq \frac {2}{w _ {-}} n ^ {1 / \alpha} \iff n \geq \left(\frac {2}{w _ {-}} + 1\right) ^ {\alpha / (\alpha - 1)}.
$$
Given the definition of  $C_0(w_{-})$  as in (8), this concludes the proof.
Since  $x \to \overline{W}_{-1}(x)$  is a positive and increasing function, i.e.  $\overline{W}_{-1}(x) > 0$  and  $\overline{W}_{-1}'(x) > 0$ , a sufficient condition for  $h_1$  to be increasing is to have  $f: x \to \log(x) + \frac{2 + K / \beta}{x}$  increasing since
$$
h _ {1} ^ {\prime} (x) = \bar {W} _ {- 1} (f (x)) + x f ^ {\prime} (x) \bar {W} _ {- 1} ^ {\prime} (f (x)) > 0 \Longleftarrow \bar {W} _ {- 1} (f (x)) > 0 \text {a n d} f ^ {\prime} (x) \bar {W} _ {- 1} ^ {\prime} (f (x)) \geq 0.
$$
Since  $f'(x) = \frac{1}{x} - \frac{2 + K / \beta}{x^2}$ , we have that  $h_1$  is increasing for  $x \geq 2 + K / \beta$ .
Lemma D.11 gives an explicit upper bound on the constant  $C_1(w_-)$  defined implicitly in Appendix D.2.
Lemma D.11. Let  $w_{-} > 0$  and  $C_1(w_{-})$  as in (9). Then, we have
$$
C _ {1} (w _ {-}) \leq \max  \left\{h _ {1} \left(4 \alpha^ {2} (1 + s) H (\mu) / \beta\right), \left(\frac {1 + 1 / \varepsilon}{w _ {-}} + 1\right) ^ {1 / (\alpha - 1)} \right\} ^ {\alpha}.
$$
Proof. Using manipulations conducted in Lemma D.10, we obtain that, for all  $n \geq h_1\left(4\alpha^2 (1 + s)H(\mu) / \beta\right)^\alpha$ ,
$$
n - 1 \geq \frac {f _ {1} (n)}{w _ {-}} \left(1 + \frac {1}{\varepsilon}\right) \Longleftarrow n - 1 \geq \frac {1 + 1 / \varepsilon}{w _ {-}} n ^ {1 / \alpha} \Longleftarrow n \geq \left(\frac {1 + 1 / \varepsilon}{w _ {-}} + 1\right) ^ {\alpha / (\alpha - 1)}.
$$
Given the definition of  $C_1(w_-)$  as in (9), this concludes the proof.
Lemma D.12 gives an asymptotic upper bound for times that are defined implicitly. For example, it can be used with  $T_0(\delta)$  and  $T_1(\delta)$  defined in Theorem D.4 and Theorem D.6.
Lemma D.12. Let  $C > 0$ ,  $D > 0$ ,  $c(n, \delta)$  as in (2) and
$$
T (\delta) := \sup  \left\{n \in \mathbb {N} ^ {\star} \mid n - 1 \leq C \left(\sqrt {c (n - 1 , \delta)} + \sqrt {D \log n}\right) ^ {2} \right\}.
$$
Then, we have  $\lim \sup_{\delta \to 0} \frac{T(\delta, C)}{\log(1 / \delta)} \leq C$ .
Proof. Let  $\gamma > 0$ . Direct manipulations yield that
$$
\begin{array}{l} n - 1 \leq C \left(\sqrt {c (n - 1 , \delta)} + \sqrt {D \log n}\right) ^ {2} \\ \Longleftrightarrow \left(\sqrt {n - 1} - \sqrt {C D \log n}\right) ^ {2} - 4 \log \left(4 + \log \frac {n}{2}\right) \leq 2 C \mathcal {C} _ {G} \left(\frac {1}{2} \log \left(\frac {K - 1}{\delta}\right)\right) \\ \Longleftarrow n \leq \frac {2 C}{1 + \gamma} \mathcal {C} _ {G} \left(\frac {1}{2} \log \left(\frac {K - 1}{\delta}\right)\right) \quad \text {f o r} n > N _ {\gamma} \\ \end{array}
$$
where  $N_{\gamma} = \sup \left\{n\in \mathbb{N}^{\star}\mid \left(\sqrt{n - 1} -\sqrt{CD\log n}\right)^{2} - 4\log \left(4 + \log \frac{n}{2}\right) > (1 + \gamma)n\right\}$ . Therefore, we have
$$
T (\delta , C) \leq N _ {\gamma} + 1 + \frac {2 C}{1 + \gamma} \mathcal {C} _ {G} \left(\frac {1}{2} \log \left(\frac {K - 1}{\delta}\right)\right).
$$
Using that  $\mathcal{C}_G(x) \approx x + \log (x)$  (Lemma E.1), we obtain directly that, for all  $\gamma > 0$ ,
$$
\lim  _ {\delta \rightarrow 0} \sup  _ {\delta \rightarrow 0} \frac {T (\delta , C)}{\log (1 / \delta)} \leq \frac {C}{1 + \gamma},
$$
which yields the result by letting  $\gamma$  go to zero.
Lemma D.13 gives a non-asymptotic upper bound for times that are defined implicitly. For example, when considering the idealized choice  $c(n,\delta) = \log (1 / \delta)$ , it can be used as a first order approximation of  $T_{0}(\delta)$  and  $T_{1}(\delta)$  defined in Theorem D.4 and Theorem D.6.
Lemma D.13. Let  $C > 0$ ,  $D > 0$  and  $T(\delta) \coloneqq \sup \left\{n \in \mathbb{N}^{\star} \mid n - 1 \leq C \log(1 / \delta) + D \log n\right\}$ . Then, we have
$$
T (\delta) <   D \bar {W} _ {- 1} \left(\frac {C}{D} \log (1 / \delta) + 1 / D + \log D\right)
$$
for  $\log (1 / \delta)\geq \frac{D - D\log(D) - 1}{C}$ , else  $T(\delta) < D$
$$
n - 1 \geq C \log (1 / \delta) + D \log n \iff y - \log y \geq \frac {C}{D} \log (1 / \delta) + c _ {D} \iff n \geq D \overline {{W}} _ {- 1} \left(\frac {C}{D} \log (1 / \delta) + c _ {D}\right)
$$
where  $y = n / D$  and  $c_{D} = 1 / D + \log D$ . For the last equivalence, we used that  $\frac{C}{D}\log (1 / \delta) + c_{D}\geq 1$  if and only if  $\log (1 / \delta)\geq \frac{D}{C} (1 - c_D) = \frac{D - D\log(D) - 1}{C}$ . When  $\frac{C}{D}\log (1 / \delta) + c_{D} < 1$ , which means that the  $\delta$  parameter is large, we have directly that  $y - \log y\geq 1 > \frac{C}{D}\log (1 / \delta) + c_{D}$  for  $y\geq 1$ . Therefore, the condition becomes  $n\geq D$ .
E Concentration
The proof of Lemma 2.1 is given in Appendix E.1. In Appendix E.2, we show concentration results for the UCB leader (both with  $g_{u}$  and  $g_{m}$ ) and the TC challenger.
E.1 Proof of Lemma 2.1
Proving  $\delta$ -correctness of a GLR stopping rule is done by leveraging concentration results. In particular, we build upon Theorem 9 of [27], which is restated below.
Lemma E.1. Consider Gaussian bandits with means  $\mu \in \mathbb{R}^K$  and unit variance. Let  $S \subseteq [K]$  and  $x > 0$ .
$$
\mathbb {P} _ {\mu} \left[ \exists n \in \mathbb {N}, \sum_ {k \in S} \frac {N _ {n , k}}{2} (\mu_ {n, k} - \mu_ {k}) ^ {2} > \sum_ {k \in S} 2 \log \left(4 + \log (N _ {n, k})\right) + | S | \mathcal {C} _ {G} \left(\frac {x}{| S |}\right) \right] \leq e ^ {- x}
$$
$$
g _ {G} (\lambda) = 2 \lambda - 2 \lambda \log (4 \lambda) + \log \zeta (2 \lambda) - \frac {1}{2} \log (1 - \lambda), \tag {16}
$$
Since  $\hat{i}_n = i^*(\mu_n)$ , standard manipulations yield that for all  $k \neq \hat{i}_n$
$$
\frac {(\mu_ {n , \hat {\imath} _ {n}} - \mu_ {n , k}) ^ {2}}{\frac {1}{N _ {n , \hat {\imath} _ {n}}} + \frac {1}{N _ {n , k}}} = \inf _ {u \in \mathbb {R}} \left(N _ {n, \hat {\imath} _ {n}} (\mu_ {n, \hat {\imath} _ {n}} - u) ^ {2} + N _ {n, k} (\mu_ {n, k} - u) ^ {2}\right) = \inf _ {y \geq x} \left(N _ {n, \hat {\imath} _ {n}} (\mu_ {n, \hat {\imath} _ {n}} - x) ^ {2} + N _ {n, k} (\mu_ {n, k} - y) ^ {2}\right).
$$
Let  $i^{\star} = i^{\star}(\mu)$ . Using the stopping rule (1) and the above manipulations, we obtain
$$
\begin{array}{l} \mathbb {P} _ {\mu} \left(\tau_ {\delta} <   + \infty , \hat {i} _ {\tau_ {\delta}} \neq i ^ {\star}\right) \\ \leq \mathbb {P} _ {\mu} \left(\exists n \in \mathbb {N}, \exists i \neq i ^ {\star}, i = i ^ {\star} (\mu_ {n}), \min  _ {k \neq i} \inf  _ {y \geq x} \left(N _ {n, i} (\mu_ {n, i} - x) ^ {2} + N _ {n, k} (\mu_ {n, k} - y) ^ {2}\right) \geq 2 c (n - 1, \delta)\right) \\ \leq \mathbb {P} _ {\mu} \left(\exists n \in \mathbb {N}, \exists i \neq i ^ {\star}, i = i ^ {\star} \left(\mu_ {n}\right), \frac {N _ {n , i}}{2} \left(\mu_ {n, i} - \mu_ {i}\right) ^ {2} + \frac {N _ {n , i ^ {\star}}}{2} \left(\mu_ {n, i ^ {\star}} - \mu_ {i ^ {\star}}\right) ^ {2} \geq c (n - 1, \delta)\right) \\ \leq \sum_ {i \neq i ^ {\star}} \mathbb {P} _ {\mu} \left(\exists n \in \mathbb {N}, \frac {N _ {n , i}}{2} (\mu_ {n, i} - \mu_ {i}) ^ {2} + \frac {N _ {n , i ^ {\star}}}{2} (\mu_ {n, i ^ {\star}} - \mu_ {i ^ {\star}}) ^ {2} \geq c (n - 1, \delta)\right) \\ \end{array}
$$
where the second inequality is obtained with  $(k,x,y) = (i^{\star},\mu_i,\mu_{i^{\star}})$ , and the third by union bound. By concavity of  $x\mapsto \log (4 + \log (x))$  and  $N_{n,i^{\star}} + N_{n,i}\leq \sum_{k\in [K]}N_{n,k} = n - 1$ , we obtain
$$
\forall i \neq i ^ {\star}, \log (4 + \log N _ {n, i ^ {\star}}) + \log (4 + \log N _ {n, i}) \leq 2 \log (4 + \log ((n - 1) / 2))
$$
Combining the above with Lemma E.1 for all  $i \neq i^{\star}$ , we obtain  $\mathbb{P}_{\mu}(\tau_{\delta} < +\infty, \hat{i} = i^{\star}) \leq \sum_{i \neq i^{\star}} \frac{\delta}{K - 1} = \delta$ .
E.2 Sampling rule
In Lemmas E.2 and E.3, we prove that the UCB leader using  $g_{u}$  and  $g_{m}$  is truly an upper confidence bounds for the unknown mean parameters, when a certain concentration event occurs. Then, when another concentration event occurs, we show a lower bound on the "transportation" costs used by the TC challenger in Lemma E.5. Lemma E.6 upper bounds the probability of not being in the intersection of the two above sequence of concentration events.
UCB Leader Lemma E.2 proves that the bonus  $g_{u}$  is sufficient to have upper confidence bounds on the unknown mean  $\mu$  for Gaussian observations. The proof uses a simple union bound argument over the time.
Lemma E.2. Let  $\alpha > 1$  and  $s > 1$ . For all  $n > K$ , let  $g_u(n) = 2\alpha(1 + s)\log(n)$  and
$$
\mathcal {E} _ {1, n} := \left\{\forall k \in [ K ], \forall t \in \left[ n ^ {1 / \alpha}, n \right], \left| \mu_ {t, k} - \mu_ {k} \right| <   \sqrt {\frac {g _ {u} (t)}{N _ {t , k}}} \right\}. \tag {17}
$$
Then, for all  $n > K$ ,  $\mathbb{P}(\mathcal{E}_{1,n}^{\mathbb{C}}) \leq Kn^{-s}$ .
Proof. Let  $(X_{s})_{s\in [n]}$  be Gaussian observations from one distribution with unit variance. By union bound over  $[K]$  and using that  $n\leq t^{\alpha}$ , we obtain
$$
\begin{array}{l} \mathbb {P} \left(\exists k \in [ K ], \exists t \in [ n ^ {1 / \alpha}, n ], | \mu_ {t, k} - \mu_ {k} | \geq \sqrt {\frac {2 \alpha (1 + s) \log (t)}{N _ {t , k}}}\right) \\ \leq \sum_ {k \in [ K ]} \mathbb {P} \left(\exists t \in [ n ^ {1 / \alpha}, n ], | \mu_ {t, k} - \mu_ {k} | \geq \sqrt {\frac {2 (1 + s) \log (n)}{N _ {t , k}}}\right) \\ \leq \sum_ {k \in [ K ]} \mathbb {P} \left(\exists m \in [ n ], \left| \frac {1}{m} \sum_ {s \in [ m ]} X _ {s} \right| \geq \sqrt {\frac {2 (1 + s) \log (n)}{m}}\right) \\ \leq \sum_ {k \in [ K ]} \sum_ {m \in [ n ]} \mathbb {P} \left(\left| \frac {1}{m} \sum_ {s \in [ m ]} X _ {s} \right| \geq \sqrt {\frac {2 (1 + s) \log (n)}{m}}\right) \\ \leq \sum_ {k \in [ K ]} \sum_ {m \in [ n ]} n ^ {- (1 + s)} = K n ^ {- s}, \\ \end{array}
$$
where we used that  $\mu_{t,k} - \mu_k = \frac{1}{N_{t,k}}\sum_{s=1}^{t}\mathbb{1}(I_s = k)X_{s,k}$  and concentration results for Gaussian observations.
Lemma E.3 proves that the bonus  $g_{m}$  is sufficient to have upper confidence bounds on the unknown mean  $\mu$  for Gaussian observations. The proof uses a more sophisticated argument based on mixture of martingales.
Lemma E.3. Let  $\alpha > 1$  and  $s > 1$ . For all  $x \geq 1$ , let  $\overline{W}_{-1}(x) = -W_{-1}(-e^{-x})$  where  $W_{-1}$  is the negative branch of the Lambert  $W$  function. For all  $n > K$ , let  $g_m(n) = \overline{W}_{-1}\left(2s\alpha \log (n) + 2\log (2 + \alpha \log n) + 2\right)$  and
$$
\mathcal {E} _ {3, n} := \left\{\forall k \in [ K ], \forall t \in \left[ n ^ {1 / \alpha}, n \right], \left| \mu_ {t, k} - \mu_ {k} \right| <   \sqrt {\frac {g _ {m} (t)}{N _ {t , k}}} \right\}. \tag {18}
$$
Then, for all  $n > K$ ,  $\mathbb{P}(\mathcal{E}_{3,n}^{\mathsf{C}}) \leq Kn^{-s}$ .
Proof. Let  $(X_{s})_{s\in [t]}$  the observations from a standard normal distributions and denote  $S_{t} = \sum_{s\in [t]}X_{s}$ . To derived concentration result, we use peeling.
Let  $\eta > 0$  and  $D = \lceil \frac{\log(n)}{\log(1 + \eta)} \rceil$ . For all  $i \in [D]$ , let  $\gamma_i > 0$  and  $N_i = (1 + \eta)^{i-1}$ . For all  $i \in [D]$ , we define the family of priors  $f_{N_i, \gamma_i}(x) = \sqrt{\frac{\gamma_i N_i}{2\pi}} \exp\left(-\frac{x^2 \gamma_i N_i}{2}\right)$  with weights  $w_i = \frac{1}{D}$  and process
$$
\overline {{M}} (t) = \sum_ {i \in [ D ]} w _ {i} \int f _ {N _ {i}, \gamma_ {i}} (x) \exp \left(x S _ {t} - \frac {1}{2} x ^ {2} t\right) d x,
$$