SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} \in V _ {\lfloor L \rfloor} ^ {L}\right) \geq \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} \in V _ {t} ^ {L}\right) \geq \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(\left\{B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C}} \varepsilon_ {0} \right\} \subseteq V _ {t} ^ {L}\right) \geq K L ^ {3 / 4},
$$
Therefore, there exists  $i \in V_{\lfloor L \rfloor}^{L}$  which is sampled  $L^{3/4}$  times between  $\lfloor L \rfloor$  and  $\lfloor 2L \rfloor - 1$ .
In the following, we assume that
$$
\sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(\left\{B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \right\} \subseteq V _ {t} ^ {L}\right) <   K L ^ {3 / 4},
$$
Since  $B_t^{\mathrm{EB}} \in V_t^L$  or  $C_t^{\mathrm{TC}\varepsilon_0} \in V_t^L$ , we have
$$
\sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \left(\mathbb {1} \left(B _ {t} ^ {\mathrm {E B}} \in V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \notin V _ {t} ^ {L}\right) + \mathbb {1} \left(B _ {t} ^ {\mathrm {E B}} \notin V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \in V _ {t} ^ {L}\right)\right) > \lfloor 2 L \rfloor - \lfloor L \rfloor - K L ^ {3 / 4}.
$$
Case 1:  $\sum_{t = \lfloor L\rfloor}^{\lfloor 2L\rfloor -1}\mathbb{1}\left(B_t^{\mathrm{EB}}\in V_t^L,C_t^{\mathrm{TC}\varepsilon_0}\notin V_t^L\right) > \left(\lfloor 2L\rfloor -\lfloor L\rfloor -KL^{3 / 4}\right) / 2,$
or Case 2:  $\sum_{t = \lfloor L\rfloor}^{\lfloor 2L\rfloor -1}\mathbb{1}\left(B_t^{\mathrm{EB}}\notin V_t^L,C_t^{\mathrm{TC}\varepsilon_0}\in V_t^L\right) > \left(\lfloor 2L\rfloor -\lfloor L\rfloor -KL^{3 / 4}\right) / 2.$
Since  $\beta_{t}(i,j) = N_{t,j} / (N_{t,j} + N_{t,i})$ , we obtain
$$
1 / 2 \leq \left\{ \begin{array}{l l} \beta_ {t} (B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}) & \text {i f} \quad B _ {t} ^ {\mathrm {E B}} \in V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \notin V _ {t} ^ {L} \\ 1 - \beta_ {t} (B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}) & \text {i f} \quad B _ {t} ^ {\mathrm {E B}} \notin V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \in V _ {t} ^ {L} \end{array} \right..
$$
This lower bound will be crucial to argue that the challenger is sampled enough in case 2 and the leader is sampled enough in case 1.
Case 1. Using Lemma 46 and the above, we obtain
$$
\begin{array}{l} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} \in V _ {\lfloor L \rfloor} ^ {L}\right) \geq \sum_ {i \in V _ {\lfloor L \rfloor} ^ {L}} \sum_ {j \neq i} ^ {\lfloor 2 L \rfloor - 1} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} = i, (B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}) = (i, j)\right) \\ \geq \sum_ {i \in V _ {\lfloor L \rfloor} ^ {L}} \sum_ {j \neq i} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \beta_ {t} (i, j) \mathbb {1} \left(\left(B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}\right) = (i, j)\right) - K ^ {2} \\ \geq \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \beta_ {t} \left(B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}\right) \mathbb {1} \left(B _ {t} ^ {\mathrm {E B}} \in V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \notin V _ {t} ^ {L}\right) - K ^ {2} \\ \geq \left(\lfloor 2 L \rfloor - \lfloor L \rfloor - K L ^ {3 / 4}\right) / 4 - K ^ {2} \geq K L ^ {3 / 4}, \\ \end{array}
$$
where the last inequality is obtained for  $L \geq L_6 + 1$  with
$$
L _ {6} = \sup  \left\{L \in \mathbb {N} \mid \left(\lfloor 2 L \rfloor - \lfloor L \rfloor - K L ^ {3 / 4}\right) / 4 - K ^ {2} <   K L ^ {3 / 4} \right\}.
$$
Case 2. Using Lemma 46 and the above, we obtain
$$
\begin{array}{l} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} \in V _ {\lfloor L \rfloor} ^ {L}\right) \geq \sum_ {j \in V _ {\lfloor L \rfloor} ^ {L}} \sum_ {i \neq j} ^ {\lfloor 2 L \rfloor - 1} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} \mathbb {1} \left(I _ {t} = j, (B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}) = (i, j)\right) \\ \geq \sum_ {j \in V _ {\lfloor L \rfloor} ^ {L}} \sum_ {i \neq j} \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} (1 - \beta_ {t} (i, j)) \mathbb {1} \left(\left(B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}\right) = (i, j)\right) - K ^ {2} \\ \geq \sum_ {t = \lfloor L \rfloor} ^ {\lfloor 2 L \rfloor - 1} (1 - \beta_ {t} \left(B _ {t} ^ {\mathrm {E B}}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}\right)) \mathbb {1} \left(B _ {t} ^ {\mathrm {E B}} \notin V _ {t} ^ {L}, C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}} \in V _ {t} ^ {L}\right) - K ^ {2} \\ \geq \left(\lfloor 2 L \rfloor - \lfloor L \rfloor - K L ^ {3 / 4}\right) / 4 - K ^ {2} \geq K L ^ {3 / 4}, \\ \end{array}
$$
where the last inequality is obtained for  $L \geq L_6 + 1$ . Therefore, there exists  $i \in V_{[L]}^L$  which is sampled  $L^{3/4}$  times between  $[L]$  and  $[2L] - 1$ .
Summary. We have shown  $\left|V_{[2L]}^{L}\right| \leq K - 2$ . By induction, for any  $1 \leq k \leq K$ , we have  $\left|V_{[kL]}^{L}\right| \leq K - k$ , and finally  $U_{[KL]}^{L} = \emptyset$  for all  $L \geq L_7 \coloneqq \max\{L_5, L_4, L_6 + 1\}$ . Defining  $N_0 = KL_7$ , we have  $\mathbb{E}_{\nu}[N_0] < +\infty$ . For all  $n \geq N_0$ , we let  $L = n / K$ , hence we have  $U_n^{n / K} = U_{[KL]}^L = \emptyset$ . This concludes the proof of sufficient exploration.
Leader is best arm. Using Lemma 12, we obtain that for all  $n \geq N_0$ ,
$$
\begin{array}{l} \forall i \neq i ^ {\star}, \quad \mu_ {n, i} \leq \mu_ {i} + W _ {\mu} \sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}} \leq \mu_ {i} + W _ {\mu} \sqrt {\frac {\log (e + \sqrt {n / K})}{\sqrt {n / K}}}, \\ \mu_ {n, i ^ {\star}} \leq \mu_ {i ^ {\star}} - W _ {\mu} \sqrt {\frac {\log (e + N _ {n , i ^ {\star}})}{N _ {n , i ^ {\star}}}} \leq \mu_ {i ^ {\star}} - W _ {\mu} \sqrt {\frac {\log (e + \sqrt {n / K})}{\sqrt {n / K}}}. \\ \end{array}
$$
Therefore, we have  $B_n^{\mathrm{EB}} = i^\star$  for all  $n \geq N_1 \coloneqq \max \{N_0, K(X_0 - e)^2 + 1\}$  where
$$
X _ {0} = \sup  \left\{x > 1 \mid x <   \frac {1 6 W _ {\mu} ^ {2}}{\min  _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}} \log (x) + e \right\} \leq h _ {1} \left(\frac {1 6 W _ {\mu} ^ {2}}{\min  _ {i \neq i ^ {\star}} (\mu_ {i ^ {\star}} - \mu_ {i}) ^ {2}}, e\right),
$$
where the last inequality is obtained by using Lemma 51 with  $h_1$  defined therein. Since  $h_1(x, e) \sim_{x \to +\infty} x \log x$ , we have
$$
\mathbb {E} _ {\nu} [ N _ {1} ] <   \mathbb {E} _ {\nu} [ N _ {0} ] + \mathbb {E} _ {\nu} [ K (X _ {0} - e) ^ {2} + 1 ] <   + \infty .
$$
D.1.2 Empirical overall balance
As in [40], the key to obtain asymptotic optimality is to show that the empirical proportion satisfy the empirical overall balance equation. Compared to them, the novelty of Lemma 19 is that we use IDS proportions with  $K(K - 1)$  tracking procedures to select between the leader and the challenger instead of sampling.
Lemma 19. Let  $\gamma >0$ . There exists  $N_{2}$  with  $\mathbb{E}_{\nu}[N_2] < +\infty$  such that for all  $n\geq N_2$
$$
\left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \right| \leq \gamma .
$$
Proof. Let  $N_{1}$  as in Lemma 18. Then, we have for all  $n > N_{1}$
$$
\frac {N _ {n , i ^ {\star}}}{n - 1} = \frac {N _ {n , i ^ {\star}} ^ {i ^ {\star}}}{n - 1} + \frac {\sum_ {i \neq i ^ {\star}} N _ {N _ {1} , i ^ {\star}} ^ {i}}{n - 1}, \frac {N _ {n , i}}{n - 1} = \frac {N _ {n , i} ^ {i ^ {\star}}}{n - 1} + \frac {\sum_ {j \neq i ^ {\star}} N _ {N _ {1} , i} ^ {j}}{n - 1},
$$
$$
(1 - \bar {\beta} _ {n} (i ^ {\star}, j)) \frac {T _ {n} (i ^ {\star} , j)}{n - 1} = \frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = j\right) (1 - \beta_ {t} (i ^ {\star}, j)) + (1 - \bar {\beta} _ {N _ {1}} (i ^ {\star}, j)) \frac {T _ {N _ {1}} (i ^ {\star} , j)}{n - 1},
$$
$$
\sum_ {j \neq i ^ {\star}} \bar {\beta} _ {n} (i ^ {\star}, j) \frac {T _ {n} (i ^ {\star} , j)}{n - 1} = \frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t}) + \sum_ {j \neq i ^ {\star}} \bar {\beta} _ {N _ {1}} (i ^ {\star}, j) \frac {T _ {N _ {1}} (i ^ {\star} , j)}{n - 1}
$$
Chaining the inequalities with Lemma 46, we obtain that
$$
\begin{array}{l} - \frac {K}{n - 1} \leq \frac {N _ {n , i ^ {\star}}}{n - 1} - \frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t}) \leq \frac {2 N _ {1} + K}{n - 1}, \\ - \frac {1}{n - 1} \leq \frac {N _ {n , i}}{n - 1} - \frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = j\right) \left(1 - \beta_ {t} (i ^ {\star}, j)\right) \leq \frac {2 N _ {1} + 1}{n - 1}. \\ \end{array}
$$
Using that  $a^2 - b^2 = (a - b)(a + b) \leq 2(a - b)$  for  $(a, b) \in (0,1)^2$ , we obtain
$$
\begin{array}{l} - 2 \frac {K}{n - 1} \leq \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \left(\frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t})\right) ^ {2} \leq 2 \frac {2 N _ {1} + K}{n - 1}, \\ - 2 \frac {1}{n - 1} \leq \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} - \left(\frac {1}{n - 1} \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = j\right) \left(1 - \beta_ {t} (i ^ {\star}, j)\right)\right) ^ {2} \leq 2 \frac {2 N _ {1} + 1}{n - 1}. \\ \end{array}
$$
For all  $n > N_{1}$ , let us denote by
$$
G _ {n} = \left(\sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t})\right) ^ {2} - \sum_ {j \neq i ^ {\star}} \left(\sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = j\right) \left(1 - \beta_ {t} (i ^ {\star}, j)\right)\right) ^ {2}.
$$
$$
- 4 K \frac {N _ {1} + 1}{n - 1} \leq \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} - \frac {G _ {n}}{(n - 1) ^ {2}} \leq 4 \frac {N _ {1} + K}{n - 1}.
$$
Direct manipulations yield that
$$
\begin{array}{l} \frac {1}{2} \left(G _ {n + 1} - G _ {n}\right) = \beta_ {n} \left(i ^ {\star}, C _ {n}\right) \sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} \left(i ^ {\star}, C _ {t}\right) - \left(1 - \beta_ {n} \left(i ^ {\star}, C _ {n}\right)\right) \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = C _ {n}\right) \left(1 - \beta_ {t} \left(i ^ {\star}, C _ {n}\right)\right) \\ + \beta_ {n} \left(i ^ {\star}, C _ {n}\right) - 1 / 2. \\ \end{array}
$$
Therefore, we obtain by using the above inequalities that
$$
\begin{array}{l} \frac {1}{2} \left(G _ {n + 1} - G _ {n}\right) \leq \beta_ {n} \left(i ^ {\star}, C _ {n}\right) N _ {n, i ^ {\star}} - \left(1 - \beta_ {n} \left(i ^ {\star}, C _ {n}\right)\right) N _ {n, C _ {n}} + 2 N _ {1} + K + 3 / 2 \\ = 2 N _ {1} + K + 3 / 2, \\ \end{array}
$$
$$
\begin{array}{l} \frac {1}{2} \left(G _ {n + 1} - G _ {n}\right) \geq \beta_ {n} \left(i ^ {\star}, C _ {n}\right) N _ {n, i ^ {\star}} - \left(1 - \beta_ {n} \left(i ^ {\star}, C _ {n}\right)\right) N _ {n, C _ {n}} - 2 N _ {1} - K - 3 / 2 \\ = - 2 N _ {1} - K - 3 / 2, \\ \end{array}
$$
where we used that  $\beta_{n}(i^{\star},C_{n})N_{n,i^{\star}} = (1 - \beta_{n}(i^{\star},C_{n}))N_{n,C_{n}}$  since  $\beta_{n}(i^{\star},C_{n}) = \frac{N_{n,C_{n}}}{N_{n,i^{\star}} + N_{n,C_{n}}}$ . Summing those inequalities, we obtain
$$
\begin{array}{l} G _ {n} = \sum_ {t = N _ {1} + 1} ^ {n - 1} \left(G _ {t + 1} - G _ {t}\right) + G _ {N _ {1} + 1} \leq \left(4 N _ {1} + 2 K + 3\right) (n - 1 - N _ {1}) + 1 \\ \geq - (4 N _ {1} + 2 K + 3) (n - 1 - N _ {1}) - 1, \\ \end{array}
$$
where we used that  $G_{N_1 + 1} = 2\beta_{N_1}(i^\star, C_{N_1}) - 1 \in [-1, 1]$ . Combining everything together, we have shown that
$$
\begin{array}{l} \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \leq 4 \frac {N _ {1} + K}{n - 1} + \frac {(4 N _ {1} + 2 K + 3) (n - 1 - N _ {1}) + 1}{(n - 1) ^ {2}} \\ \geq - 4 K \frac {N _ {1} + 1}{n - 1} - \frac {(4 N _ {1} + 2 K + 3) (n - 1 - N _ {1}) + 1}{(n - 1) ^ {2}}. \\ \end{array}
$$
Therefore, we have shown the desired results for  $n \geq N_2$  defined as
$$
N _ {2} = \inf  \left\{n > 2 \mid 4 K \frac {N _ {1} + 1}{n - 1} + \frac {(4 N _ {1} + 2 K + 3) (n - 1 - N _ {1}) + 1}{(n - 1) ^ {2}} \leq \gamma \right\},
$$
which satisfies  $\mathbb{E}_{\nu}[N_2] < +\infty$  since it is a linear function of  $N_{1}$ .
As in [40], using Lemma 19 allows to bound the empirical proportion allocated to the unique best arm  $N_{n,i^{\star}} / (n - 1)$  away from 0 (Lemma 20).
Lemma 20. There exists  $N_{3}$  with  $\mathbb{E}_{\nu}[N_3] < +\infty$  such that for all  $n \geq N_3$
$$
\frac {1}{4 \sqrt {2 (K - 1)}} \leq \frac {N _ {n , i ^ {\star}}}{n - 1} \leq \frac {3}{4}.
$$
Proof. Let  $N_{2}$  as in Lemma 19 for  $\gamma = 1/2$ . Using Lemma 19, we obtain  $\frac{N_{n,i^{\star}}}{n-1} \leq \frac{3}{4}$  since
$$
1 / 2 \geq \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \geq \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \left(1 - \frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} = 2 \frac {N _ {n , i ^ {\star}}}{n - 1} - 1.
$$
Let  $\tilde{N}_2$  as in Lemma 19 for  $\gamma = \frac{1}{32(K - 1)}$ . Similarly by using Lemma 19, we obtain  $\frac{N_{n,i^{\star}}}{n - 1} \geq \frac{1}{4\sqrt{2(K - 1)}}$  since
$$
\begin{array}{l} \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} \geq - \frac {1}{3 2 (K - 1)} + \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \geq - \frac {1}{3 2 (K - 1)} + \frac {1}{K - 1} \left(1 - \frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} \\ \geq \frac {1}{3 2 (K - 1)}. \\ \end{array}
$$
Taking  $N_{3} = \max \{N_{2},\tilde{N}_{2}\}$  yields the result.
Lemma 21 is a rescaled version of the empirical overall balance equation which is proven by simply combining Lemma 19 and Lemma 20.
Lemma 21. Let  $\gamma >0$ . There exists  $N_{4}$  with  $\mathbb{E}_{\nu}[N_4] < +\infty$  such that for all  $n\geq N_4$
$$
\left| 1 - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{N _ {n , i ^ {\star}}}\right) ^ {2} \right| \leq \gamma .
$$
Proof. Let  $N_{2}$  as in Lemma 19 for  $\frac{\gamma}{32(K - 1)}$ . Let  $N_{3}$  as in Lemma 20. Direct manipulation shows that, for all  $n \geq N_{4} = \max \{N_{2}, N_{3}\}$ ,
$$
\left| 1 - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{N _ {n , i ^ {\star}}}\right) ^ {2} \right| = \left(\frac {n - 1}{N _ {n , i ^ {\star}}}\right) ^ {2} \left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \right| \leq 3 2 (K - 1) \frac {\gamma}{3 2 (K - 1)} = \gamma .
$$
This concludes the result.
D.1.3 Convergence towards optimal ratio of allocation
As in [40], we show that a challenger will not be sampled next when the ratio of its empirical proportion compared to the one of  $i^{\star}$  overshoots the ratio of the optimal allocations (Lemma 22).
Lemma 22. Let  $\gamma >0$ . There exists  $N_{5}$  with  $\mathbb{E}_{\nu}[N_5] < +\infty$  such that for all  $n\geq N_5$  and all  $i\neq i^{\star}$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma \quad \Longrightarrow \quad C _ {n} ^ {T C \varepsilon_ {0}} \neq i.
$$
Proof. Let  $\gamma > 0$ . Let  $i \neq i^{\star}$  such that
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma .
$$
Suppose towards contradiction that
$$
\forall j \neq i ^ {\star}, \frac {N _ {n , j}}{N _ {n , i ^ {\star}}} > \frac {w _ {j} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}.
$$
Let  $\tilde{\gamma} > 0$ . Let  $N_4$  as in Lemma 21 for  $\tilde{\gamma}$ . then using Lemma 21 and 8 yields that, for all  $n \geq N_4$ ,
$$
\tilde {\gamma} \geq \sum_ {j \neq i ^ {\star}} \left(\frac {N _ {n , j}}{N _ {n , i ^ {\star}}}\right) ^ {2} - 1 \geq \left(\frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma\right) ^ {2} - \left(\frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}\right) ^ {2} = \gamma \left(\gamma + 2 \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}\right).
$$
Therefore, we have a contradiction if we take  $\tilde{\gamma}$  small enough (e.g.  $\tilde{\gamma} < \gamma^2$ ), hence we have shown that, for all  $n \geq N_4$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma \quad \Longrightarrow \quad \exists j \notin \{i ^ {\star}, i \}, \quad \frac {N _ {n , j}}{N _ {n , i ^ {\star}}} \leq \frac {w _ {j} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}}.
$$
Let  $N_{1}$  as in Lemma 18. Using that  $B_{n}^{\mathrm{EB}} = i^{\star}$  for all  $n\geq N_1$  and the definition of  $C_n^{\mathrm{TC}\varepsilon_0}$ , we know that
$$
\frac {\mu_ {n , i ^ {\star}} - \mu_ {n , i} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i ^ {\star}} + 1 / N _ {n , i}}} > \frac {\mu_ {n , i ^ {\star}} - \mu_ {n , j} + \varepsilon_ {0}}{\sqrt {1 / N _ {n , i ^ {\star}} + 1 / N _ {n , j}}} \quad \Longrightarrow \quad C _ {n} ^ {\mathrm {T C} \varepsilon_ {0}} \neq i.
$$
To conclude the proof, it is sufficient to show that the ratio of the two quantities is strictly larger than one. For all  $n \geq \max \{N_1, N_4\}$ , we obtain
$$
\frac {\mu_ {n , i ^ {\star}} - \mu_ {n , i} + \varepsilon_ {0}}{\mu_ {n , i ^ {\star}} - \mu_ {n , j} + \varepsilon_ {0}} \sqrt {\frac {1 + N _ {n , i ^ {\star}} / N _ {n , j}}{1 + N _ {n , i ^ {\star}} / N _ {n , i}}} \geq \frac {\mu_ {n , i ^ {\star}} - \mu_ {n , i} + \varepsilon_ {0}}{\mu_ {n , i ^ {\star}} - \mu_ {n , j} + \varepsilon_ {0}} \sqrt {\frac {1 + w _ {i ^ {\star}} ^ {\star} / w _ {j} ^ {\star}}{1 + (w _ {i} ^ {\star} / w _ {i ^ {\star}} ^ {\star} + \gamma) ^ {- 1}}}.
$$
Let  $\tilde{\gamma} > 0$ . Using Lemmas 12 and 18, we have, for all  $n \geq N_1$  and all  $k \neq i^{\star}$ ,
$$
\begin{array}{l} \left| \frac {\mu_ {n , i ^ {\star}} - \mu_ {n , k} + \varepsilon_ {0}}{\mu_ {i ^ {\star}} - \mu_ {k} + \varepsilon_ {0}} - 1 \right| \leq \frac {W _ {\mu}}{\mu_ {i ^ {\star}} - \mu_ {k} + \varepsilon_ {0}} \left(\sqrt {\frac {\log (e + N _ {n , k})}{N _ {n , k}}} + \sqrt {\frac {\log (e + N _ {n , i ^ {\star}})}{N _ {n , i ^ {\star}}}}\right) \\ \leq \frac {2 W _ {\mu}}{\min  _ {k \neq i ^ {\star}} \left(\mu_ {i ^ {\star}} - \mu_ {k} + \varepsilon_ {0}\right)} \sqrt {\frac {\log (e + \sqrt {n / K})}{\sqrt {n / K}}} \leq \tilde {\gamma} \\ \end{array}
$$
for all  $n \geq N_6 = K(X_0 - e)^2 + 1$  which is defined as
$$
\begin{array}{l} X _ {0} = \sup  \left\{x \geq 1 \mid x <   \frac {4 W _ {\mu} ^ {2}}{\tilde {\gamma} ^ {2} \min  _ {k \neq i ^ {\star}} \left(\mu_ {i ^ {\star}} - \mu_ {k} + \varepsilon_ {0}\right) ^ {2}} \log x + e \right\} \\ \leq h _ {1} \left(\frac {4 W _ {\mu} ^ {2}}{\tilde {\gamma} ^ {2} \min  _ {k \neq i ^ {\star}} \left(\mu_ {i ^ {\star}} - \mu_ {k} + \varepsilon_ {0}\right) ^ {2}}, e\right). \\ \end{array}
$$
where the last inequality is obtained by using Lemma 51 with  $h_1$  defined therein. Since  $h_1(x, y) \sim_{x \to +\infty} x \log x$ , we have  $\mathbb{E}_{\nu}[N_6] < +\infty$  since it polynomial in  $W_\mu$  (by using Lemma 12). Let  $\kappa > 0$ . We have shown that, for all  $n \geq \max \{N_1, N_4, N_6\}$ ,
$$
\begin{array}{l} \frac {\mu_ {n , i ^ {\star}} - \mu_ {n , i} + \varepsilon_ {0}}{\mu_ {n , i ^ {\star}} - \mu_ {n , j} + \varepsilon_ {0}} \sqrt {\frac {1 + N _ {n , i ^ {\star}} / N _ {n , j}}{1 + N _ {n , i ^ {\star}} / N _ {n , i}}} \geq \frac {\mu_ {i ^ {\star}} - \mu_ {i} + \varepsilon_ {0}}{\mu_ {i ^ {\star}} - \mu_ {j} + \varepsilon_ {0}} \sqrt {\frac {1 + w _ {i ^ {\star}} ^ {\star} / w _ {j} ^ {\star}}{1 + (w _ {i} ^ {\star} / w _ {i ^ {\star}} ^ {\star} + \gamma) ^ {- 1}}} \frac {1 - \tilde {\gamma}}{1 + \tilde {\gamma}} \\ = \sqrt {\frac {1 + w _ {i ^ {*}} ^ {\star} / w _ {i} ^ {\star}}{1 + (w _ {i} ^ {\star} / w _ {i ^ {*}} ^ {\star} + \gamma) ^ {- 1}}} \frac {1 - \tilde {\gamma}}{1 + \tilde {\gamma}}. \\ \end{array}
$$
where the equality uses that the transportation costs are equal at the equilibrium, i.e. (9). Taking  $\tilde{\gamma}$  small enough, we have that shown that, for all  $n\geq \max \{N_1,N_4,N_6\}$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}}} + \gamma \quad \Longrightarrow \quad \frac {\mu_ {n , i ^ {\star}} - \mu_ {n , i} + \varepsilon_ {0}}{\mu_ {n , i ^ {\star}} - \mu_ {n , j} + \varepsilon_ {0}} \sqrt {\frac {1 + N _ {n , i ^ {\star}} / N _ {n , j}}{1 + N _ {n , i ^ {\star}} / N _ {n , i}}} > 1 \quad \Longrightarrow \quad C _ {n} ^ {\mathrm {T C} \varepsilon_ {0}} \neq i.
$$
Lemma 23 shows no sub-optimal arm is overshooting the ratio of its optimal allocation for  $n$  large enough.
Lemma 23. Let  $\gamma >0$ . There exists  $N_{6}$  with  $\mathbb{E}_{\nu}[N_6] < +\infty$  such that for all  $n\geq N_6$  and all  $i\neq i^{\star}$ ,
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \leq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma .
$$
Proof. Let  $\gamma > 0$ . Let  $N_{1}$  as in Lemma 18. Let  $N_{3}$  and  $N_{5}$  as in Lemmas 20 and 22. Let  $M \geq \max \{N_{3}, N_{5}, N_{1}\}$  and  $n > M$ . Let  $t_{n,i}(\gamma) = \max \left\{M, \max \left\{t \in \{M, \dots, n - 1\} \mid \frac{N_{t,i}}{N_{t,i^{\star}}} < \frac{w_{i^{\star}}^{*}}{w_{i^{\star}}} + \gamma / 2\right\} \right\}$  for all  $i \neq i^{\star}$ . In particular, using Lemma 22, for all  $n > M$
$$
\forall t > t _ {n, i} (\gamma), \quad \frac {N _ {t , i}}{N _ {t , i ^ {\star}}} \geq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma / 2 \quad \text {h e n c e} \quad i \neq C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}},
$$
$$
N _ {t _ {n, i} (\gamma), i} \leq \max  \left\{M, \left(\frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma / 2\right) N _ {t _ {n, i} (\gamma), i ^ {\star}} \right\}
$$
Then, we have for all  $n \geq M$ ,
$$
\begin{array}{l} N _ {n, i} = N _ {t _ {n, i} (\gamma), i} + \mathbb {1} (i = I _ {t _ {n, i} (\gamma)}) + \sum_ {t = t _ {n, i} (\gamma) + 1} ^ {n - 1} \mathbb {1} (i = I _ {t} = C _ {t} ^ {\mathrm {T C} \varepsilon_ {0}}) \\ \leq \max  \left\{M, \left(\frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma / 2\right) N _ {t _ {n, i (\gamma), i ^ {\star}}} \right\} + 1. \\ \end{array}
$$
Using that  $N_{t_{n,i}(\gamma),i^{\star}}\leq N_{n,i^{\star}},N_{n,i^{\star}}\geq \frac{n - 1}{4\sqrt{2(K - 1)}}$  (Lemma 20) and  $\max \{a,b\} \leq a + b$ , we obtain that
$$
\frac {N _ {n , i}}{N _ {n , i ^ {\star}}} \leq \frac {4 (M + 1) \sqrt {2 (K - 1)}}{n - 1} + \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma / 2 \leq \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} + \gamma ,
$$
where the last inequality holds for  $n \geq N_7 = 8(M + 1)\sqrt{2(K - 1)} / \gamma + 1$ . Taking  $N_6 = \max \{N_3, N_5, N_1, N_7\}$  yields the result since  $\mathbb{E}_{\nu}[N_6] < +\infty$ .
Finally, Lemma 24 shows that the sufficient condition (see Lemma 13) to obtain the asymptotic upper bound on the expected sample complexity is satisfied.
Lemma 24. Let  $\varepsilon_0 > 0$ ,  $\gamma > 0$  and  $T_{\varepsilon_0,\gamma}$  as in (10). Under the  $EB-TC_{\varepsilon_0}$  sampling rule with IDS proportions, we have  $\mathbb{E}_{\nu}[T_{\varepsilon_0,\gamma}] < +\infty$ .
Proof. Let  $\gamma > 0$  and  $\tilde{\gamma}$ . Let  $N_4$  and  $N_6$  as in Lemmas 21 and 23 for  $\tilde{\gamma}$ . Then, using Lemmas 21 and 23, we have for all  $n \geq \max\{N_4, N_6\}$ ,