SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We denote by  $N_{n,b}^{a}\triangleq \sum_{t\in [n - 1]}\mathbb{1}\left(B_{t} = a,I_{t} = C_{t} = b\right)$  the number of times the arm  $b$  was pulled while the arm  $a$  was the leader, and by  $L_{n,a}\triangleq \sum_{t\in [n - 1]}\mathbb{1}\left(B_{t} = a\right)$  the number of times arm  $a$  was the leader. At time  $n > K$ , the next arm to be pulled  $I_{n}$  is defined as
$$
I _ {n} = B _ {n} \quad \text {i f} N _ {n, B _ {n}} ^ {B _ {n}} \leq \beta L _ {n + 1, B _ {n}}, \text {o t h e r w i s e} I _ {n} = C _ {n}. \tag {19}
$$
In other words, we sample the leader if we have not yet sampled it a fraction  $\beta$  of the times it was leader. Those  $K$  independent tracking procedure satisfy the desired property (Lemma 6).
Lemma 6 (Lemma 2.2 in [JD22]). For all  $n > K$  and all  $a \in [K]$ , we have
$$
- 1 / 2 \leq N _ {n, a} ^ {a} - \beta L _ {n, a} \leq 1.
$$
To finish specifying a Top Two algorithm, we simply need to specify the choice of the leader/challenger pair. Intuitively, a good choice of the leader/challenger pair should ensure (1) sufficient exploration, (2) convergence of the leader towards the best arm  $a^{\star}$ , and (3) convergence of the global pulling proportions to the  $\beta$ -optimal allocation  $\omega_{\mathrm{KL},\beta}(\mu)$ , which is defined for Gaussian distributions as
$$
\omega_ {\mathrm {K L}, \beta} ^ {\star} (\boldsymbol {\mu}) \triangleq \underset {\omega \in \Sigma_ {K}, \omega_ {a} \star = \beta} {\arg \max } \underset {a \neq a ^ {\star}} {\min } \frac {\Delta_ {a} ^ {2}}{1 / \beta + 1 / \omega_ {a}}.
$$
While we consider TTUCB algorithm [JD22] in AdaP-TT, we emphasise that other Top Two algorithms could be used with the same type of guarantees. TTUCB is a Top Two algorithm which combines a UCB-based leader and a Transportation Cost (TC) challenger. Its key novelty lies in the use of  $K$  tracking procedures. Since it is deterministic, the analysis is less cumbersome.
Non-private leader/challenger pair. The non-private leader/challenger pair is inspired by the TTUCB algorithm [JD22]. At time  $n > K$ , the non-private UCB leader is defined as
$$
B _ {n} ^ {\mathrm {N P}} \triangleq \underset {a \in [ K ]} {\arg \max } \left\{\hat {\mu} _ {k _ {n, a}, a} + \sqrt {\frac {k _ {n , a}}{\tilde {N} _ {k _ {n , a} , a}}} \right\}, \tag {20}
$$
where  $\sqrt{k_{n,a} / \tilde{N}_{k_{n,a},a}}$  is a bonus to cope for the uncertainty which depends on the current phase  $k_{n,a}$ . Later, we show that  $\log (n) / k_{n,a} = \Theta (1)$  for all  $a\in [K]$ . Hence, it has the same scaling as the bonus used in standard UCB. Since it depends solely on the local counts  $(\tilde{N}_{k_{n,a},a})_{a\in [K]}$  and the empirical means  $(\hat{\mu}_{k_{n,a},a})_{a\in [K]}$ , the non-private UCB leader is fixed until we switch phase again.
At time  $n > K$ , the non-private TC challenger is defined as
$$
C _ {n} ^ {\mathrm {N P}} \triangleq \underset {a \neq B _ {n} ^ {\mathrm {N P}}} {\arg \min } \frac {\hat {\mu} _ {k _ {n , B _ {n} ^ {\mathrm {N P}} , B _ {n} ^ {\mathrm {N P}}} - \hat {\mu} _ {k _ {n , a} , a}}}{\sqrt {1 / N _ {n , B _ {n} ^ {\mathrm {N P}}} + 1 / N _ {n , a}}}. \tag {21}
$$
While it depends on the empirical means  $(\hat{\mu}_{k_{n,a},a})_{a\in [K]}$  that are fixed till we switch phase again, it also depends on the global counts  $(N_{n,a})_{a\in [K]}$ . Therefore, the non-private TC challenger is chosen in an adaptive manner. This is the key to obtain guarantees on the expected sample complexity of the non-private algorithm.
We derive upper bounds on the expected sample complexity of the non-private AdaP-TT algorithm in the asymptotic regime of  $\delta \rightarrow 0$  (Theorem 6). In particular, it shows that the cost of doubling and forgetting is multiplicative four-factor compared to the TTUCB algorithm, which achieves  $T_{\mathrm{KL},\beta}^{\star}(\mu)$  (see Theorem 2.3 in [JD22]). We defer its proof to Appendix E.8.
Theorem 6 (Asymptotic upper bound on expected sample complexity of non-private AdaP-TT). Let  $(\delta, \beta) \in (0,1)^2$ . Combined with the non-private GLR stopping rule (Equation (13)) using non-private threshold functions as in Equation (14), the non-private AdaP-TT algorithm is  $\delta$ -correct and satisfies that, for all 1-sub-Gaussian distributions  $\nu$  with means  $\mu \in \mathbb{R}^K$  such that  $\min_{a \neq b} |\mu_a - \mu_b| > 0$ ,
$$
\lim  _ {\delta \rightarrow 0} \sup  _ {\log (1 / \delta)} \frac {\mathbb {E} _ {\boldsymbol {\nu}} \left[ \tau_ {\delta} ^ {N P} \right]}{\log (1 / \delta)} \leq 4 T _ {\mathrm {K L}, \beta} ^ {\star} (\mu),
$$
where  $T_{\mathrm{KL},\beta}^{\star}(\mu)$  is the  $\beta$ -characteristic time for Gaussian distributions, such that
$$
2 T _ {\mathrm {K L}, \beta} ^ {\star} (\boldsymbol {\mu}) ^ {- 1} \triangleq \max  _ {\omega \in \Sigma_ {K}, \omega_ {a ^ {\star}} = \beta} \frac {(\mu_ {a ^ {\star}} - \mu_ {a}) ^ {2}}{1 / \beta + 1 / \omega_ {a}}.
$$
Private leader/challenger pair. Since we want to ensure privacy, the non-private leader/challenger pair cannot be used since it relies on the empirical means  $\hat{\mu}_{k_{n,a},a}$ , which are not private. To alleviate this problem, we propose a private leader/challenger pair which is based on the private empirical means  $\tilde{\mu}_{k_{n,a},a}$ .
At time  $n > K$ , the private UCB leader is defined as
$$
B _ {n} \triangleq \underset {a \in [ K ]} {\arg \max } \left\{\tilde {\mu} _ {k _ {n, a}, a} + \sqrt {\frac {k _ {n , a}}{\tilde {N} _ {k _ {n , a} , a}}} + \frac {k _ {n , a}}{\epsilon \tilde {N} _ {k _ {n , a} , a}} \right\}, \tag {22}
$$
where  $k_{n,a} / (\epsilon \tilde{N}_{k_{n,a},a})$  is a bonus to cope for the uncertainty due to the Laplace noise. It also depends on the current phase  $k_{n,a}$  and has the same scaling as the private UCB indices. Likewise, the private UCB leader is fixed until we switch phase again.
At time  $n > K$ , the private Transportation Cost (TC) challenger is defined as
$$
C _ {n} \triangleq \underset {a \neq B _ {n}} {\arg \min } \frac {\tilde {\mu} _ {k _ {n , B _ {n}} , B _ {n}} - \tilde {\mu} _ {k _ {n , a} , a}}{\sqrt {1 / N _ {n , B _ {n}} + 1 / N _ {n , a}}} . \tag {23}
$$
Likewise, the private TC challenger is chosen in an adaptive manner, which is key to obtain guarantees on the expected sample complexity of the private algorithm.
We derive upper bounds on the expected sample complexity of the private AdaP-TT algorithm in the asymptotic regime of  $\delta \rightarrow 0$  (Theorem 5). We defer its proof to Appendix E.
Theorem 5 (Asymptotic upper bound on expected sample complexity of AdaP-TT). Let  $(\delta, \beta) \in (0,1)^2$ . Combined with the non-private GLR stopping rule (Equation (13)) using non-private threshold functions as in Equation (14), the non-private AdaP-TT algorithm is  $\delta$ -correct and satisfies that, for all bandit instances  $\pmb{\nu}$  with 1-sub-Gaussian distributions and means  $\mu \in \mathbb{R}^K$  such that  $\min_{a \neq b} |\mu_a - \mu_b| > 0$ ,
$$
\lim _ {\delta \to 0} \sup _ {\mathrm {l o g} (1 / \delta)} \frac {\mathbb {E} _ {\boldsymbol {\nu}} [ \tau_ {\delta} ]}{\log (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\mathrm {l o g} (\operatorname * {l i m s u p}) \leq 4 T _ {\mathrm {K L}, \beta} ^ {\star} (\mu) \left(1 + \sqrt {1 + \frac {\Delta_ {\max} ^ {2}}{2 \epsilon^ {2}}}\right) ,
$$
where  $T_{\mathrm{KL},\beta}^{\star}(\mu)$  is the  $\beta$ -characteristic time for Gaussian distributions.
E Analysis of AdaP-TT: Proof of Theorem 5
Let  $\beta \in (0,1)$ ,  $\epsilon \in \mathbb{R}_{+}^{\star}$ , and  $\pmb{\nu}$  be a bandit instance consisting of  $K$ , 1-sub-Gaussian distributions with distinct means  $\pmb{\mu} \in \mathbb{R}^{K}$ , i.e.  $\min_{a \neq b} |\mu_a - \mu_b| > 0$ . For conciseness, we denote  $\Delta_{a} \triangleq \mu_{a^{\star}} - \mu_{a}$ ,  $\Delta_{\min} \triangleq \min_{a \neq a^{\star}} \Delta_{a}$ , and  $\Delta_{\max} \triangleq \max_{a \neq a^{\star}} \Delta_{a}$ .
For Gaussian distributions, the unique  $\beta$ -optimal allocation  $\omega_{\mathrm{KL},\beta}^{\star}(\pmb {\mu}) = \{\omega_{\beta ,a}^{\star}\}$  is defined as
$$
\omega_ {\mathrm {K L}, \beta} ^ {\star} (\boldsymbol {\mu}) \triangleq \underset {\omega \in \Sigma_ {K}, \omega_ {a ^ {\star}} = \beta} {\arg \max } \underset {a \neq a ^ {\star}} {\min } \frac {\Delta_ {a} ^ {2}}{1 / \beta + 1 / \omega_ {a}}. \tag {24}
$$
At equilibrium, we have equality of the transportation costs (see [JD22] for example), namely
$$
\forall a \neq a ^ {\star}, \quad \frac {\Delta_ {a} ^ {2}}{1 / \beta + 1 / \omega_ {\beta , a} ^ {\star}} = 2 T _ {\mathrm {K L}, \beta} ^ {\star} (\boldsymbol {\mu}) ^ {- 1}. \tag {25}
$$
Our proof follows the unified sample complexity analysis of Top Two algorithms from  $\left[\mathrm{JDB}^{+}22\right]$ .
Let  $\gamma > 0$ . We denote by  $T_{\mu, \gamma}$  the convergence time towards  $\omega_{\beta}^{\star}$ , which is a random variable quantifies the number of samples required for the global empirical allocations  $N_{n} / (n - 1)$  to be  $\gamma$ -close to  $\omega_{\beta}^{\star}$  for any subsequent time, namely
$$
T _ {\boldsymbol {\mu}, \gamma} \triangleq \inf  \left\{T \geq 1 \mid \forall n \geq T, \left\| \frac {N _ {n}}{n - 1} - \omega_ {\beta} ^ {\star} \right\| _ {\infty} \leq \gamma \right\}. \tag {26}
$$
The rest of Appendix E is organised as follows. First, we prove that AdaP-TT ensures sufficient exploration (Appendix E.2) Second, we prove that there is convergence towards the  $\beta$ -optimal
allocation (Appendix E.3) in finite time. Finally, we conclude the proof of Theorem 5 (Appendix E.5). In Appendix E.6, we compare our asymptotic upper bound with our asymptotic lower bound on the expected sample complexity (Theorem 2). In Appendix E.7, we discuss the limitation of our result and pose an open problem. Appendix E.8 will detail the slight modification that needs to be made in order to obtain Theorem 6.
E.1 Technical results
Before delving into the proofs, we first recall some useful technical results extracted from the literature.
Concentration results. In order to control the randomness of  $(\tilde{\mu}_{k_a,a})_{a\in [K]}$ , we use a standard concentration result on the empirical mean of sub-Gaussian random variables and on sub-exponential observations (Lemma 7). Since Bernoulli distributions are  $1/2$ -sub-Gaussian and the absolute value of a Laplace is an exponential distribution, Lemma 7 applies to our setting.
Lemma 7. There exists a sub-Gaussian random variable  $W_{\mu}$  such that, almost surely,
$$
\forall a \in [ K ], \forall k _ {a} \in \mathbb {N}, | \hat {\mu} _ {k _ {a}, a} - \mu_ {a} | \leq W _ {\mu} \sqrt {\frac {\log (e + \tilde {N} _ {k _ {a} , a})}{\tilde {N} _ {k _ {a} , a}}}.
$$
There exists a sub-exponential random variable  $W_{\epsilon}$  such that, almost surely,
$$
\forall a \in [ K ], \forall k _ {a} \in \mathbb {N}, | Y _ {k _ {a}, a} | \leq W _ {\epsilon} \frac {\log (e + k _ {a})}{\tilde {N} _ {k _ {a} , a}}.
$$
In particular, any random variable which is polynomial in  $(W_{\epsilon}, W_{\mu})$  has a finite expectation.
Proof. The first part is a known result, e.g. Appendix E.2 in  $\left[\mathrm{JDB}^{+}22\right]$ . Let us define
$$
W _ {\epsilon} \triangleq \sup  _ {a \in [ K ]} \sup  _ {k _ {a} \in \mathbb {N}} \frac {\tilde {N} _ {k _ {a} , a} | Y _ {k _ {a} , a} |}{\log (e + k _ {a})}.
$$
By definition, we have that, almost surely,
Since  $\tilde{N}_{k,i}|Y_{k,i}|\sim \mathcal{E}(\epsilon)$ , Lemma 72 in  $[\mathrm{JDB^{+}22}]$  yields that  $W_{\epsilon}$  is a sub-exponential random variable. Since  $W_{\mu}$  is sub-Gaussian and  $W_{\epsilon}$  is a sub-exponential, any random variable which is polynomial in  $(W_{\epsilon},W_{\mu})$  has a finite expectation.
Inversion results. Lemma 8 gathers properties on the function  $\overline{W}_{-1}$ , which is used in the literature to obtain concentration results.
Lemma 8 ([JDK23]). Let  $\overline{W}_{-1}(x) \triangleq -W_{-1}(-e^{-x})$  for all  $x \geq 1$ , where  $W_{-1}$  is the negative branch of the Lambert  $W$  function. The function  $\overline{W}_{-1}$  is increasing on  $(1, +\infty)$  and strictly concave on  $(1, +\infty)$ . In particular,  $\overline{W}_{-1}'(x) = \left(1 - \frac{1}{\overline{W}_{-1}(x)}\right)^{-1}$  for all  $x > 1$ . Then, for all  $y \geq 1$  and  $x \geq 1$ ,
Lemma 9 is an inversion result to upper bound a time, which is implicitly defined. It is a direct consequence of Lemma 8.
Lemma 9. Let  $\overline{W}_{-1}$  defined in Lemma 8. Let  $A > 0$ ,  $B > 0$  such that  $B / A + \log A > 1$  and
$$
C (A, B) = \sup  \left\{x \mid x <   A \log x + B \right\}.
$$
Then,  $C(A,B) < h_1(A,B)$  with  $h_1(z,y) = z\overline{W}_{-1}(y / z + \log z)$ .
Using Lemma 8 yields that
E.2 Sufficient exploration
The first step of in the generic analysis of Top Two algorithms  $\left[\mathrm{JDB}^{+}22\right]$  is to show that AdaP-TT ensures sufficient exploration. The main idea is to show that, if there are still undersampled arms, either the leader or the challenger will be among them. Therefore, after a long enough time, no arm can still be undersampled. We emphasise that there are multiple ways to select the leader/challenger pair in order to ensure sufficient exploration. Therefore, while we conduct the proof for AdaP-TT, other choices of leader/challenger pair would yield similar results.
Given an arbitrary phase  $p \in \mathbb{N}$ , we define the sampled enough set, i.e. the arms having reached phase  $p$ , and the arm with highest mean in this set (when not empty) as
$$
S _ {n} ^ {p} = \left\{a \in [ K ] \mid N _ {n, a} \geq 2 ^ {p - 1} \right\} \quad \text {a n d} \quad a _ {n} ^ {\star} = \underset {a \in S _ {n} ^ {p}} {\arg \max } \mu_ {a}. \tag {27}
$$
Since  $\min_{a\neq b}|\mu_a - \mu_b| > 0$ ,  $a_{n}^{\star}$  is unique.
Let  $p \in \mathbb{N}$  such that  $(p - 1) / 4 \in \mathbb{N}$ . We define the highly and the mildly under-sampled sets as
$$
U _ {n} ^ {p} \triangleq \left\{a \in [ K ] \mid N _ {n, a} <   2 ^ {(p - 1) / 2} \right\} \quad \text {a n d} \quad V _ {n} ^ {p} \triangleq \left\{a \in [ K ] \mid N _ {n, a} <   2 ^ {3 (p - 1) / 4} \right\}. \tag {28}
$$
They correspond to the arms having not reached phase  $(p - 1) / 2$  and phase  $3(p - 1) / 4$ , respectively.
Lemma 10 shows that, when the leader is sampled enough, it is the arm with highest true mean among the sampled enough arms.
Lemma 10. Let  $S_{n}^{p}$  and  $a_{n}^{\star}$  as in (27). There exists  $p_0$  with  $\mathbb{E}_{\nu}[\exp(\alpha p_0)] < +\infty$  for all  $\alpha > 0$  such that if  $p \geq p_0$ , for all  $n$  such that  $S_{n}^{p} \neq \emptyset$ ,  $B_{n} \in S_{n}^{p}$  implies that  $B_{n} = a_{n}^{\star} = \arg \max_{a \in S_{n}^{p}} \tilde{\mu}_{k_{n,a},a}$ .
Proof. Let  $p_0$  to be specified later. Let  $p \geq p_0$ . Let  $n \in \mathbb{N}$  such that  $S_n^p \neq \emptyset$ , where  $S_n^p$  and  $a_n^\star$  as in Equation (27). Let  $(k_{n,a})_{a \in [K]}$  be the phases indices for all arms. Since  $N_{n,a} \geq 2^{p-1}$  for all  $a \in S_n^p$ , we have  $k_{n,a} \geq p$  and  $\tilde{N}_{k_{n,a},a} \geq 2^{p-2}$  by using Lemma 3. Using Lemma 7, we obtain that
$$
\tilde {\mu} _ {k _ {n}, a _ {n} ^ {\star}, a _ {n} ^ {\star}} \geq \mu_ {a _ {n} ^ {\star}} - W _ {\mu} \sqrt {\frac {\log (e + 2 ^ {p - 2})}{2 ^ {p - 2}}} - W _ {\epsilon} \frac {\log (e + p)}{2 ^ {p - 2}},
$$
$$
\tilde {\mu} _ {k _ {n, a}, a} \leq \mu_ {a} + W _ {\mu} \sqrt {\frac {\log (e + 2 ^ {p - 2})}{2 ^ {p - 2}}} + W _ {\epsilon} \frac {\log (e + p)}{2 ^ {p - 2}}, \quad \forall a \in S _ {n} ^ {p} \setminus \left\{a _ {n} ^ {\star} \right\}.
$$
Here, we use that  $x \to \log (e + x) / x$  is decreasing.
Let  $\overline{\Delta}_{\mathrm{min}} = \min_{a\neq b}|\mu_a - \mu_b|$ . By assumption on the considered instances, we know that  $\overline{\Delta}_{\mathrm{min}} > 0$ . Let  $p_1 = \lceil \log_2(X_1 - e)\rceil +2$  and  $p_2 = \lceil \log_2((X_2 - e - 2)\log 2 + 1)\rceil +2$  with
$$
X _ {1} = \sup  \left\{x > 1 \mid x \leq 6 4 \overline {{\Delta}} _ {\min } ^ {- 2} W _ {\mu} ^ {2} \log x + e \right\} \leq h _ {1} (6 4 \overline {{\Delta}} _ {\min } ^ {- 2} W _ {\mu} ^ {2}, e),
$$
$$
X _ {2} = \sup  \left\{x > 1 \mid x \leq \frac {8}{\log 2} \overline {{\Delta}} _ {\min } ^ {- 1} W _ {\epsilon} \log x + e + 2 - 1 / \log 2 \right\} \leq h _ {1} \left(8 \overline {{\Delta}} _ {\min } ^ {- 1} W _ {\epsilon} / \log 2, 4\right),
$$
where we used Lemma 9, and  $h_1$  defined therein. Then, for all  $p \in \mathbb{N}$  such that  $p \geq \max\{p_1, p_2\} + 1$  and all  $n \in \mathbb{N}$  such that  $S_n^p \neq \emptyset$ , we have  $\tilde{\mu}_{k_{n,a_\star}}, a_n^\star \geq \mu_{a_n^\star} - \overline{\Delta}_{\min}/4$  and  $\tilde{\mu}_{k_{n,a},a} \leq \mu_a + \overline{\Delta}_{\min}/4$  for all  $a \in S_n^p \setminus \{a_n^\star\}$ , hence  $a_n^\star = \arg \max_{a \in [K]} \tilde{\mu}_{k_{n,a},a}$ .
We have, for all  $\alpha \in \mathbb{R}_{+}$
$$
\exp (\alpha p _ {1}) \leq e ^ {3 \alpha} (X _ {1} - e) ^ {\alpha / \log 2} \quad \text {h e n c e} \quad \mathbb {E} _ {\boldsymbol {\nu}} [ \exp (\alpha p _ {1}) ] <   + \infty ,
$$
where we used Lemma 7 and  $h_1(x, e) \sim_{x \to +\infty} x \log x$  to obtain that  $\exp(\alpha p_1)$  is at most polynomial in  $W_\mu$ . Likewise, we obtain that  $\mathbb{E}_\nu[\exp(\alpha p_2)] < +\infty$  for all  $\alpha \in \mathbb{R}_+$ .
Let us define the UCB indices by  $I_{k_{n,a},a} = \tilde{\mu}_{k_{n,a},a} + \sqrt{k_{n,a} / \tilde{N}_{k_{n,a},a}} + k_{n,a} / (\epsilon \tilde{N}_{k_{n,a},a})$ . Using the above, we have
$$
I _ {k _ {n, a _ {n} ^ {\star}}, a _ {n} ^ {\star}} \geq \mu a _ {n} ^ {\star} - W _ {\mu} \sqrt {\frac {\log (e + 2 ^ {p - 2})}{2 ^ {p - 2}}} - W _ {\epsilon} \frac {\log (e + p)}{2 ^ {p - 2}},
$$
$$
\forall a \in S _ {n} ^ {p} \setminus \{a _ {n} ^ {\star} \}, I _ {k _ {n, a}, a} \leq \mu_ {a} + W _ {\mu} \sqrt {\frac {\log (e + 2 ^ {p - 2})}{2 ^ {p - 2}}} + W _ {\epsilon} \frac {\log (e + p)}{2 ^ {p - 2}} + \sqrt {\frac {p}{2 ^ {p - 2}}} + \frac {p}{\epsilon 2 ^ {p - 2}},
$$
where we used Lemma 3 and the fact that  $x \to \log (e + x) / x$  and  $x \to x2^{2 - x}$  are decreasing functions for  $x \geq 2$ . Let  $p_3 = \lceil \log_2 X_3 \rceil + 2$  and  $p_4 = \lceil \log_2 X_4 \rceil + 2$  with
$$
X _ {3} = \sup  \left\{x > 1 \mid x \leq 6 4 \overline {{\Delta}} _ {\min } ^ {- 2} (\log_ {2} x + 2) \right\} \leq h _ {1} (6 4 \overline {{\Delta}} _ {\min } ^ {- 2} / \log 2, 1 2 8 \overline {{\Delta}} _ {\min } ^ {- 2}),
$$
$$
X _ {4} = \sup \left\{x > 1 \mid x \leq 8 \epsilon^ {- 1} \overline {{\Delta}} _ {\min } ^ {- 1} (\log_ {2} x + 2) \right\} \leq h _ {1} (8 \epsilon^ {- 1} \overline {{\Delta}} _ {\min } ^ {- 1} / \log 2, 1 6 \overline {{\Delta}} _ {\min } ^ {- 1} \epsilon^ {- 1}),
$$
where we used Lemma 9, and  $h_1$  defined therein. We highlight that  $(p_3, p_4)$  are deterministic values, hence their expectation is finite. Then, for all  $p \in \mathbb{N}$  such that  $p \geq p_0 = \max \{p_1, p_2, p_3, p_4\} + 1$  and all  $n \in \mathbb{N}$  such that  $S_n^p \neq \emptyset$ , we have  $I_{k_{n,a_n^\star},a_n^\star} \geq \mu_{a_n^\star} - \overline{\Delta}_{\min} / 4$  and  $I_{k_{n,a},a} \leq \mu_a + \overline{\Delta}_{\min} / 2$  for all  $a \in S_n^p \setminus \{a_n^\star\}$ , hence  $a_n^\star = B_n$  since we have  $B_n = \arg \max_{a \in [K]} I_{k_{n,a},a}$ .
Since we have  $\mathbb{E}_{\nu}[\exp (\alpha p_0)] < +\infty$  for all  $\alpha \in \mathbb{R}_+$ , this concludes the proof.
Lemma 11 shows that the transportation costs between the sampled enough arms with largest true means and the other sampled enough arms are increasing fast enough.
Lemma 11. Let  $S_{n}^{p}$  and  $a_{n}^{\star}$  are as in Equation (27). There exists  $p_{1}$  with  $\mathbb{E}_{\nu}[\exp (\alpha p_1)] < +\infty$  for all  $\alpha > 0$  such that if  $p \geq p_{1}$ , for all  $n$  such that  $S_{n}^{p} \neq \emptyset$ , for all  $b \in S_{n}^{p} \setminus \{a_{n}^{\star}\}$ , we have
$$
\frac {\tilde {\mu} _ {k _ {n , a _ {n} ^ {\star}} , a _ {n} ^ {\star}} - \tilde {\mu} _ {k _ {n , b} , b}}{\sqrt {1 / \tilde {N} _ {k _ {n , a _ {n} ^ {\star}} , a _ {n} ^ {\star}} + 1 / \tilde {N} _ {k _ {n , b} , b}}} \geq 2 ^ {p / 2} C _ {\mu},
$$
where  $C_\mu > 0$  is a problem dependent constant.
Proof. Let  $p_1$  to be specified later. Let  $p \geq p_1$ . Let  $n \in \mathbb{N}$  such that  $S_n^p \neq \emptyset$ , where  $S_n^p$  and  $a_n^\star$  as in Equation (27). Let  $(k_{n,a})_{a \in [K]}$  be the phases indices for all arms. Since  $N_{n,a} \geq 2^{p-1}$  for all  $a \in S_n^p$ , we have  $k_{n,a} \geq p$  and  $\tilde{N}_{k_{n,a},a} \geq 2^{p-2}$  by using Lemma 3. Let  $\overline{\Delta}_{\min} = \min_{a \neq b} |\mu_a - \mu_b|$ , which satisfies  $\overline{\Delta}_{\min} > 0$  by assumption on the instance considered.
Using Lemma 7, for all  $b \in S_n^p \setminus \{a_n^\star\}$ , we obtain
$$
\tilde {\mu} _ {k _ {n, a _ {n} ^ {\star}}, a _ {n} ^ {\star}} - \tilde {\mu} _ {k _ {n, b}, b} \geq \overline {{\Delta}} _ {\min } - W _ {\mu} \sqrt {\frac {\log (e + 2 ^ {p - 2})}{2 ^ {p - 4}}} - W _ {\epsilon} \frac {\log (e + p)}{2 ^ {p - 3}}.
$$
Let  $p_3 = \lceil \log_2((X_3 - e) / 4) \rceil + 4$  and  $p_2 = \lceil \log_2((X_2 - e - 3) \log 2 + 1) \rceil + 3$  with
$$
X _ {3} = \sup  \left\{x > 1 \mid x \leq 6 4 \overline {{\Delta}} _ {\min } ^ {- 2} W _ {\mu} ^ {2} \log x + e \right\} \leq h _ {1} (6 4 \overline {{\Delta}} _ {\min } ^ {- 2} W _ {\mu} ^ {2}, e),
$$
$$
X _ {2} = \sup  \left\{x > 1 \mid x \leq 4 \overline {{\Delta}} _ {\min } ^ {- 1} W _ {\epsilon} \log x + e + 3 - 1 / \log 2 \right\} \leq h _ {1} \left(4 \overline {{\Delta}} _ {\min } ^ {- 1} W _ {\epsilon}, 5\right),
$$
where we used Lemma 9, and  $h_1$  defined therein. Then, for all  $p \in \mathbb{N}$  such that  $p \geq p_1 = \max \{p_3, p_2\} + 1$  and all  $n \in \mathbb{N}$  such that  $S_n^p \neq \emptyset$ , we have, for all  $b \in S_n^p \setminus \{a_n^\star\}$ ,
$$
\tilde {\mu} _ {k _ {n, a _ {n} ^ {\star}}, a _ {n} ^ {\star}} - \tilde {\mu} _ {k _ {n, b}, b} \geq \overline {{\Delta}} _ {\min } / 2.
$$
As in the proof of Lemma 10, we obtain that  $\mathbb{E}_{\nu}[\exp (\alpha p_1)] < +\infty$  for all  $\alpha \in \mathbb{R}_+$ .
Then, for all  $b \in S_n^p \setminus \{a_n^\star\}$ , we have
$$
\frac {\tilde {\mu} _ {k _ {n , a _ {n} ^ {*}} , a _ {n} ^ {*}} - \tilde {\mu} _ {k _ {n , b} , b}}{\sqrt {1 / \tilde {N} _ {k _ {n , a _ {n} ^ {*}} , a _ {n} ^ {*}} + 1 / \tilde {N} _ {k _ {n , b} , b}}} \geq 2 ^ {p / 2} \frac {\overline {{\Delta}} _ {\mathrm {m i n}}}{2 ^ {5 / 2}},
$$
where we used that  $\min \{\tilde{N}_{k_{n,a_n^\star},\tilde{N}_{k_{n,b},b}}\} \geq 2^{p - 2}$ . Setting  $C_\mu = \overline{\Delta}_{\mathrm{min}} / 2^{5 / 2}$  yields the result.
Lemma 12 shows that the transportation costs between sampled enough arms and undersampled arms are not increasing too fast.
Lemma 12. Let  $S_{n}^{p}$  be as in Equation (27). For all  $p \geq 1$  and all  $n$  such that  $S_{n}^{p} \neq \emptyset$
$$
\forall a \in S _ {n} ^ {p}, \forall b \notin S _ {n} ^ {p}, \frac {\tilde {\mu} _ {k _ {n , a} , a} - \tilde {\mu} _ {k _ {n , b} , b}}{\sqrt {1 / \tilde {N} _ {k _ {n , a} , a} + 1 / \tilde {N} _ {k _ {n , b} , b}}} \leq 2 ^ {p / 2} D _ {\mu} + 2 W _ {\mu} \sqrt {\log (e + 2 ^ {p - 2})} + 2 W _ {\epsilon} \log (e + p),
$$
where  $D_{\mu} > 0$  is a problem dependent constant and  $(W_{\mu}, W_{\epsilon})$  are the random variables defined in Lemma 7.