SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
2 T _ {\varepsilon , \beta} ^ {\operatorname {m u l}} (\mu) ^ {- 1} (\mu , i) = \max  _ {w \in \triangle_ {K}, w _ {i} = \beta} \min  _ {j \neq i} \frac {\left(\mu_ {i} - (1 - \varepsilon) \mu_ {j}\right) ^ {2}}{1 / \beta + (1 - \varepsilon) ^ {2} / w _ {j}}, \tag {30}
$$
We denote by  $w_{\varepsilon}^{\mathrm{mul}}(\mu, i)$  and  $w_{\varepsilon, \beta}^{\mathrm{mul}}(\mu, i)$  their maximizer, which we refer to as the  $(\beta-$  )optimal allocation for multiplicative  $\varepsilon$ -BAI.
Even though the multiplicative  $\varepsilon$ -BAI is less studied, it is direct to see that it has similar properties as  $T_0(\mu)$ . Due to the factor  $(1 - \varepsilon)^2$  in the denominator, it is not possible to simply use a reduction to a BAI problem as we did for the additive  $\varepsilon$ -BAI problem (see Lemma 9). Since the properties listed in Appendix C are obtained by studying the optimization problem defining  $T_0(\mu)$ , we will have the same properties for  $T_{\varepsilon}^{\mathrm{mul}}(\mu)$  by accounting for the extra multiplicative factor  $(1 - \varepsilon)^2$ . We list here the properties on which our proof relies:
-  $T_{\varepsilon}^{\mathrm{mul}}(\mu) = \min_{\beta \in (0,1)} T_{\varepsilon,\beta}^{\mathrm{mul}}(\mu, i^{\star})$  and  $T_{\varepsilon,\beta}^{\mathrm{mul}}(\mu) = T_{\varepsilon,\beta}^{\mathrm{mul}}(\mu, i^{\star})$ , meaning the unique best arm corresponds to the arm which is the easiest to verify multiplicative  $\varepsilon$ -BAI.
-  $w_{\varepsilon}^{\mathrm{mul}}(\mu) = \{w_{\varepsilon}^{\mathrm{mul}}\}$  and  $w_{\varepsilon, \beta}^{\mathrm{mul}}(\mu) = \{w_{\varepsilon, \beta}^{\mathrm{mul}}\}$ , meaning there is a unique  $(\beta-)$  optimal allocation which satisfies  $\min_{i \in [K]} (w_{\varepsilon}^{\mathrm{mul}})_i > 0$  and  $\min_{i \in [K]} (w_{\varepsilon, \beta}^{\mathrm{mul}})_i > 0$ .
- There is equality at the equilibrium of the transportation costs, meaning for all  $i \neq i^{\star}$
$$
\frac {\left(\mu_ {i ^ {\star}} - (1 - \varepsilon) \mu_ {i}\right) ^ {2}}{\left(w _ {\varepsilon} ^ {\mathrm {m u l}}\right) _ {i ^ {\star}} ^ {- 1} + (1 - \varepsilon) ^ {2} \left(w _ {\varepsilon} ^ {\mathrm {m u l}}\right) _ {i} ^ {- 1}} = 2 T _ {\varepsilon} ^ {\mathrm {m u l}} (\mu) ^ {- 1} \text {a n d} \frac {\left(\mu_ {i ^ {\star}} - (1 - \varepsilon) \mu_ {i}\right) ^ {2}}{\beta^ {- 1} + (1 - \varepsilon) ^ {2} \left(w _ {\varepsilon , \beta} ^ {\mathrm {m u l}}\right) _ {i} ^ {- 1}} = 2 T _ {\varepsilon , \beta} ^ {\mathrm {m u l}} (\mu) ^ {- 1}, \tag {31}
$$
and the multiplicative overall balance equation is satisfied
$$
\sum_ {i \neq i ^ {\star}} \left(\frac {\left(w _ {\varepsilon} ^ {\operatorname {m u l}}\right) _ {i}}{\left(w _ {\varepsilon} ^ {\operatorname {m u l}}\right) _ {i ^ {\star}}}\right) ^ {2} = (1 - \varepsilon_ {0}) ^ {2}. \tag {32}
$$
For the sake of space, we omit the proofs and defer the reader to Appendix C for ideas on how to prove them.
I.2 Anytime Top Two algorithm
Let  $\varepsilon_0\in (0,1)$ . The  $\mathrm{EB - TC}_{\varepsilon_0}^{\mathrm{m}}$  algorithm, detailed in Figure 3, has the same structure as  $\mathrm{EB - TC}_{\varepsilon_0}$ . There are two differences. First, the IDS proportions are  $\beta_{n}(i,j) = N_{n,j} / ((1 - \varepsilon_{0})^{2}N_{n,i} + N_{n,j})$  when considering multiplicative  $\varepsilon$ -BAI. Second, the challenger will be based on transportation costs for multiplicative  $\varepsilon$ -BAI, namely
$$
C _ {n} ^ {\mathrm {T C} _ {\varepsilon_ {0}} ^ {\mathrm {m}}} \in \arg \min  _ {i \neq B _ {n} ^ {\mathrm {E B}}} \frac {\mu_ {n , B _ {n} ^ {\mathrm {E B}}} - \left(1 - \varepsilon_ {0}\right) \mu_ {n , i}}{\sqrt {1 / N _ {n , B _ {n} ^ {\mathrm {E B}}} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , i}}}. \tag {33}
$$
Stopping rule for fixed-confidence multiplicative  $\varepsilon$ -BAI Similarly, the fixed-confidence setting requires a stopping rule, which will be different since we tackle multiplicative  $\varepsilon$ -BAI. Given an error/confidence pair  $(\varepsilon, \delta) \in [0,1) \times (0,1)$ , the  $\mathrm{GLR}_{\varepsilon}^{\mathrm{m}}$  stopping rule [15] prescribes to stop at the time
$$
\tau_ {\varepsilon , \delta} ^ {\text {m u l}} = \inf  \left\{n > K \mid \min  _ {i \neq \hat {i} _ {n}} \frac {\mu_ {n , \hat {i} _ {n}} - (1 - \varepsilon) \mu_ {n , i}}{\sqrt {1 / N _ {n , \hat {i} _ {n}} + (1 - \varepsilon) ^ {2} / N _ {n , i}}} \geq \sqrt {2 c (n - 1 , \delta)} \right\}, \tag {34}
$$
where  $c:\mathbb{N}\times (0,1)\to \mathbb{R}_+$  is a threshold function. Lemma 53 gives a threshold ensuring that the  $\mathrm{GLR}_{\varepsilon}^{\mathrm{m}}$  stopping rule is  $(\varepsilon ,\delta)$ -PAC for all  $\varepsilon \in [0,1)$  and  $\delta \in (0,1)$ , independently of the sampling rule.
Figure 3: EB-TC $_{\varepsilon_0}$  algorithm with fixed or IDS proportions.
1: Input: slack  $\varepsilon_0\in (0,1)$ , proportion  $\beta \in (0,1)$  (only for fixed proportions).
2: Set  $\hat{\iota}_n \in \arg \max_{i \in [K]} \mu_{n,i}$ ,  $B_n^{\mathrm{EB}} = \hat{\iota}_n$  and  $C_n^{\mathrm{TC}^m_{\varepsilon_0}} \in \arg \max_{i \neq B_n^{\mathrm{EB}}} \frac{\mu_{n,B_n^{\mathrm{EB}} - (1 - \varepsilon_0)\mu_{n,i}}}{\sqrt{1 / N_{n,B_n^{\mathrm{EB}}} + (1 - \varepsilon_0)^2 / N_{n,i}}}$ .
3: Set [fixed]  $\bar{\beta}_{n+1}(i,j) = \beta$  or [IDS]  $\beta_n(i,j) = N_{n,j} / ((1 - \varepsilon_0)^2 N_{n,i} + N_{n,j})$  and update  $\bar{\beta}_{n+1}(i,j)$ .
4: Set  $I_{n} = C_{n}^{\mathrm{TC}_{\varepsilon_{0}}^{\mathrm{m}}}$  if  $N_{n,C_n^{\mathrm{TC}_{\varepsilon_0}^{\mathrm{m}}}}^{B_n^{\mathrm{EB}}}\leq (1 - \bar{\beta}_{n + 1}(B_n^{\mathrm{EB}},C_n^{\mathrm{TC}_{\varepsilon_0}^{\mathrm{m}}}))T_{n + 1}(B_n^{\mathrm{EB}},C_n^{\mathrm{TC}_{\varepsilon_0}^{\mathrm{m}}})$ , otherwise set  $I_{n} = B_{n}^{\mathrm{EB}}$ .
5: Output: next arm to sample  $I_{n}$  and next recommendation  $\hat{i}_n$ .
Lemma 53. Let  $\varepsilon \in [0,1)$  and  $\delta \in (0,1)$ . Given any sampling rule, using the threshold (4) with the stopping rule (34) with error/confidence pair  $(\varepsilon, \delta)$  yields a  $(\varepsilon, \delta)$ -PAC algorithm for sub-Gaussian distributions.
Proof. The proof of Lemma 53 is actually almost identical to the one of Lemma 3. The only change is at the beginning of the proof. Since  $\hat{i}_n = i^{\star}(\mu_n)$ , standard manipulations yield that for all  $i \neq \hat{i}_n$
$$
\begin{array}{l} \frac {\left(\mu_ {n , \hat {i} _ {n}} - (1 - \varepsilon) \mu_ {n , i}\right) ^ {2}}{1 / N _ {n , \hat {i} _ {n}} + (1 - \varepsilon) ^ {2} / N _ {n , i}} = \inf  _ {u \in \mathbb {R}} \left\{N _ {n, \hat {i} _ {n}} \left(\mu_ {n, \hat {i} _ {n}} - u\right) ^ {2} + N _ {n, i} \left(\mu_ {n, i} - u / (1 - \varepsilon)\right) ^ {2} \right\} \\ = \inf  _ {y \geq x / (1 - \varepsilon)} \left\{N _ {n, \hat {t} _ {n}} (\mu_ {n, \hat {t} _ {n}} - x) ^ {2} + N _ {n, i} (\mu_ {n, i} - y) ^ {2} \right\}. \\ \end{array}
$$
Let  $i^{\star} = i^{\star}(\mu)$ . Using the stopping rule (34) and the above manipulations, we obtain
$$
\begin{array}{l} \mathbb {P} _ {\nu} \left(\tau_ {\varepsilon , \delta} ^ {\mathrm {m u l}} <   + \infty , \hat {i} _ {\tau_ {\varepsilon , \delta} ^ {\mathrm {m u l}}} \notin \mathcal {I} _ {\varepsilon} ^ {\mathrm {m u l}} (\mu)\right) \\ \leq \mathbb {P} _ {\nu} (\exists n \in \mathbb {N}, \exists i \notin \mathcal {I} _ {\varepsilon} ^ {\mathrm {m u l}} (\mu), i = i ^ {\star} (\mu_ {n}), \\ \min  _ {k \neq i} \inf  _ {y \geq x / (1 - \varepsilon)} \left(N _ {n, \hat {i} _ {n}} \left(\mu_ {n, \hat {i} _ {n}} - x\right) ^ {2} + N _ {n, i} \left(\mu_ {n, i} - y\right) ^ {2}\right) \geq 2 c (n - 1, \delta)\Bigg) \\ \leq \mathbb {P} _ {\nu} (\exists n \in \mathbb {N}, \exists i \notin \mathcal {I} _ {\varepsilon} ^ {\mathrm {m u l}} (\mu), i = i ^ {\star} (\mu_ {n}), \\ \left. \frac {N _ {n , i}}{2} \left(\mu_ {n, i} - \mu_ {i}\right) ^ {2} + \frac {N _ {n , i ^ {\star}}}{2} \left(\mu_ {n, i ^ {\star}} - \mu_ {i ^ {\star}}\right) ^ {2} \geq c (n - 1, \delta)\right) \\ \leq \sum_ {i \notin \mathcal {I} _ {\varepsilon} ^ {\mathrm {m u l}} (\mu)} \mathbb {P} _ {\nu} \left(\exists n \in \mathbb {N}, \frac {N _ {n , i}}{2} (\mu_ {n, i} - \mu_ {i}) ^ {2} + \frac {N _ {n , i ^ {\star}}}{2} (\mu_ {n, i ^ {\star}} - \mu_ {i ^ {\star}}) ^ {2} \geq c (n - 1, \delta)\right), \\ \end{array}
$$
where the second inequality is obtained with  $(k,x,y) = (i^{\star},\mu_{i},\mu_{i^{\star}})$  since  $i\notin \mathcal{I}_{\varepsilon}^{\mathrm{mul}}(\mu)$ , and the third by union bound. Then, we can conclude as in Appendix G.1 that
$$
\mathbb {P} _ {\nu} \left(\tau_ {\varepsilon , \delta} ^ {\mathrm {m u l}} <   + \infty , \hat {i} _ {\tau_ {\varepsilon , \delta} ^ {\mathrm {m u l}}} \notin \mathcal {I} _ {\varepsilon} ^ {\mathrm {m u l}} (\mu)\right) \leq \delta .
$$
Asymptotic expected sample complexity While Theorem 8 holds for all sub-Gaussian distributions, it is particularly interesting for Gaussian ones, in light of Lemma 1. It shows that, for multiplicative  $\varepsilon$ -BAI,  $\mathrm{EB-TC}_{\varepsilon_0}^{\mathrm{m}}$  is asymptotically optimal for Gaussian bandits when using IDS proportions and asymptotically  $\beta$ -optimal when using fixed proportions  $\beta$ .
Theorem 8. Let  $\varepsilon_0\in (0,1)$ ,  $\beta \in (0,1)$  and  $\delta \in (0,1)$ . Using the threshold (4) in the stopping rule (34) with error/confidence  $(\varepsilon_0,\delta)$ , the  $EB-TC_{\varepsilon_0}^m$  algorithm is  $(\varepsilon_0,\delta)$ -PAC and it satisfies that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^{\star}(\mu)| = 1$ ,
$$
[ \mathbf {I D S} ] \quad \lim  _ {\delta \rightarrow 0} \sup  _ {\log (1 / \delta)} \frac {\mathbb {E} _ {\nu} [ \tau_ {\varepsilon_ {0} , \delta} ]}{\log (1 / \delta)} \leq T _ {\varepsilon_ {0}} ^ {\operatorname {m u l}} (\mu) \quad a n d \quad [ \mathbf {f i x e d} \beta ] \quad \lim  _ {\delta \rightarrow 0} \sup  _ {\log (1 / \delta)} \frac {\mathbb {E} _ {\nu} [ \tau_ {\varepsilon_ {0} , \delta} ]}{\log (1 / \delta)} \leq T _ {\varepsilon_ {0}, \beta} ^ {\operatorname {m u l}} (\mu).
$$
I.3 Proof of Theorem 8
The proof is very similar to the one of Theorem 1, hence we will only highlight key differences. We defer the reader to Appendix D for a detailed discussion on the proof method and intuition on the different steps of the proof, see also [36, 21, 40].
In the following, we consider a slack  $\varepsilon_0$  for  $\mathrm{EB - TC}_{\varepsilon_0}^{\mathrm{m}}$  which matches the error  $\varepsilon$  considered by the  $\mathrm{GLR}_{\varepsilon}^{\mathrm{m}}$  stopping rule (34), i.e.  $\varepsilon_0 = \varepsilon$ . For conciseness, we denote  $w^{\star} = w_{\varepsilon}^{\mathrm{mul}}$  and  $w_{\beta}^{\star} = w_{\varepsilon,\beta}^{\mathrm{mul}}$ .
Let  $\gamma > 0$ . Let us define the convergence times  $T_{\varepsilon_0,\gamma}$  and  $T_{\varepsilon_0,\beta,\gamma}$  as in (10) and (14), i.e.
$$
T _ {\varepsilon_ {0}, \gamma} := \inf  \left\{T \geq 1 \mid \forall n \geq T, \max  _ {i \neq i ^ {\star}} \left| \frac {N _ {n , i}}{N _ {n , i ^ {\star}}} - \frac {w _ {i} ^ {\star}}{w _ {i ^ {\star}} ^ {\star}} \right| \leq \gamma \right\},
$$
$$
T _ {\varepsilon_ {0}, \beta , \gamma} := \inf  \left\{T \geq 1 \mid \forall n \geq T, \left\| \frac {N _ {n}}{n - 1} - w _ {\beta} ^ {\star} \right\| _ {\infty} \leq \gamma \right\}.
$$
Lemma 54 gives a sufficient condition for asymptotic optimality and asymptotic  $\beta$ -optimality. The proof is the same as for Lemmas 13 and 25, hence we omit it.
Lemma 54. Let  $\varepsilon_0\in (0,1)$  and  $\delta \in (0,1)$ . Assume that there exists  $\gamma_{1}(\mu) > 0$  such that for all  $\gamma \in (0,\gamma_1(\mu)],\mathbb{E}_\nu [T_{\varepsilon_0,\gamma}] < + \infty$ . Using the threshold (4) in the stopping rule (3) with error  $\varepsilon_0$  yields an algorithm such that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^{\star}(\mu)| = 1$ ,
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\nu}[\tau_{\varepsilon_{0},\delta}]}{\log(1 / \delta)}\leq T_{\varepsilon_{0}}^{\mathrm{mul}}(\mu).
$$
Assume that there exists  $\gamma_{1}(\mu) > 0$  such that for all  $\gamma \in (0,\gamma_1(\mu)],\mathbb{E}_\nu [T_{\varepsilon_0,\beta ,\gamma}] < + \infty$  . Using the threshold (4) in the stopping rule (3) with error  $\varepsilon_0$  yields an algorithm such that, for all  $\nu \in \mathcal{D}^K$  such that  $|i^{\star}(\mu)| = 1$
$$
\limsup_{\delta \to 0}\frac{\mathbb{E}_{\nu}[\tau_{\varepsilon_{0},\delta}]}{\log(1 / \delta)}\leq T_{\varepsilon_{0},\beta}^{\mathrm{mul}}(\mu).
$$
I.3.1 Sufficient exploration
Lemma 55 shows that the transportation cost is strictly positive and increases linearly, it bares similarity with Lemma 14.
Lemma 55. Let  $S_{n}^{L}$  and  $\mathcal{I}_{n}^{\star}$  as in (11). There exists  $L_{0}$  with  $\mathbb{E}_{\mu}[(L_0)^{\alpha}] < +\infty$  for all  $\alpha > 0$  such that if  $L \geq L_{0}$ , for all  $n$  such that  $S_{n}^{L} \neq \emptyset$ , for all  $(i,j) \in \mathcal{I}_n^\star \times S_n^L$  such that  $i \neq j$ , we have
$$
\frac {\mu_ {n , i} - (1 - \varepsilon_ {0}) \mu_ {n , j}}{\sqrt {1 / N _ {n , i} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \geq \sqrt {L} D _ {\mu},
$$
where  $D_{\mu} > 0$  is a problem dependent constant.
$$
\begin{array}{l} \mu_ {n, i} - (1 - \varepsilon_ {0}) \mu_ {n, j} \geq \mu_ {i} - (1 - \varepsilon_ {0}) \mu_ {j} - W _ {\mu} \left(\sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}} + (1 - \varepsilon_ {0}) \sqrt {\frac {\log (e + N _ {n , j})}{N _ {n , j}}}\right) \\ \geq \varepsilon_ {0} \min  _ {i \in [ K ]} \mu_ {i} - (2 - \varepsilon_ {0}) W _ {\mu} \sqrt {\frac {\log (e + L)}{L}} \geq \varepsilon_ {0} \min  _ {i \in [ K ]} \mu_ {i} / 2, \\ \end{array}
$$
$$
L _ {1} = \sup  \left\{x \mid x <   \frac {4 (2 - \varepsilon_ {0}) ^ {2} W _ {\mu} ^ {2}}{\varepsilon_ {0} ^ {2} \min  _ {i \in [ K ]} \mu_ {i} ^ {2}} \log (x) + e \right\} <   h _ {1} \left(\frac {4 (2 - \varepsilon_ {0}) ^ {2} W _ {\mu} ^ {2}}{\varepsilon_ {0} ^ {2} \min  _ {i \in [ K ]} \mu_ {i} ^ {2}}, e\right),
$$
The last inequality is obtained by using Lemma 51, and we recall that  $h_1$  is defined in Lemma 51. Since  $h_1(x,e) \sim_{x \to +\infty} x \log x$ , we have  $\mathbb{E}_{\mu}[(L_0)^\alpha] < +\infty$  for all  $\alpha > 0$  by using Lemma 12 (polynomial in  $W_\mu$ ). Then, we have
$$
\frac {\mu_ {n , i} - (1 - \varepsilon_ {0}) \mu_ {n , j}}{\sqrt {1 / N _ {n , i} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \geq \frac {\varepsilon_ {0} \min _ {i \in [ K ]} \mu_ {i} / 2}{\sqrt {1 / N _ {n , i} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \geq \sqrt {L} \frac {\varepsilon_ {0} \min _ {i \in [ K ]} \mu_ {i}}{2 \sqrt {1 + (1 - \varepsilon_ {0}) ^ {2}}}.
$$
Setting  $D_{\mu} = \frac{\varepsilon_0\min_{i\in[K]}\mu_i}{2\sqrt{1 + (1 - \varepsilon_0)^2}}$  yields the result.
Lemma 56 gives an upper bound on the transportation costs between a sampled enough arm and an under-sampled one it bares similarity with Lemma 15.
Lemma 56. Let  $S_{n}^{L}$  as in (11). There exists  $L_{1}$  with  $\mathbb{E}_{\mu}[(L_1)^\alpha ] < +\infty$  for all  $\alpha >0$  such that for all  $L\geq L_{1}$  and all  $n\in \mathbb{N}$ ,
$$
\forall (i, j) \in S _ {n} ^ {L} \times \overline {{S _ {n} ^ {L}}}, \frac {\mu_ {n , i} - (1 - \varepsilon_ {0}) \mu_ {n , j}}{\sqrt {1 / N _ {n , i} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \leq \frac {\sqrt {L}}{1 - \varepsilon_ {0}} (D _ {1} + 4 W _ {\mu}),
$$
$$
\begin{array}{l} \frac {\mu_ {n , i} - (1 - \varepsilon_ {0}) \mu_ {n , j}}{\sqrt {1 / N _ {n , i} + (1 - \varepsilon_ {0}) ^ {2} / N _ {n , j}}} \leq \frac {\sqrt {N _ {n , j}}}{1 - \varepsilon_ {0}} | \mu_ {n, i} - (1 - \varepsilon_ {0}) \mu_ {n, j} | \\ \leq \frac {\sqrt {L}}{1 - \varepsilon_ {0}} \left(| \mu_ {i} - (1 - \varepsilon_ {0}) \mu_ {j} | + W _ {\mu} \left(\sqrt {\frac {\log (e + N _ {n , i})}{N _ {n , i}}} + (1 - \varepsilon_ {0}) \sqrt {\frac {\log (e + N _ {n , j})}{N _ {n , j}}}\right)\right) \\ \leq \frac {\sqrt {L}}{1 - \varepsilon_ {0}} \left(| \mu_ {i} - (1 - \varepsilon_ {0}) \mu_ {j} | + (2 - \varepsilon_ {0}) W _ {\mu} \sqrt {\log (e + 1)}\right) \leq \frac {\sqrt {L}}{1 - \varepsilon_ {0}} (D _ {1} + 4 W _ {\mu}), \\ \end{array}
$$
for  $D_{1} = \max_{i\neq j}|\mu_{i} - (1 - \varepsilon_{0})\mu_{j}|$
Lemma 57 show that the desired property for the TC challenger, we omit the proof since it is the same as for Lemma 17.
Lemma 57. There exists  $L_{1}$  with  $\mathbb{E}_{\nu}[L_3] < +\infty$  such that if  $L \geq L_{3}$ , for all  $n$  (at most polynomial in  $L$ ) such that  $U_{n}^{L} \neq \emptyset$ ,  $B_{n}^{EB} \notin V_{n}^{L}$  implies  $C_n^{TC_{\varepsilon_0}^m} \in V_n^L$ .
Combining Lemma 16 and Lemma 57 yields Lemma 58. Importantly, Lemma 58 holds for the  $\mathrm{EB - TC}_{\varepsilon_0}$  algorithm with fixed proportions  $\beta \in (0,1)$  and IDS proportions. Since the proof is the same as for Lemmas 18 and 26, we omit it.
Lemma 58. There exist  $N_{1}$  with  $\mathbb{E}_{\nu}[N_1] < +\infty$  such that for all  $n \geq N_{1}$  and all  $i \in [K]$ ,  $N_{n,i} \geq \sqrt{n / K}$  and  $B_{n}^{EB} = i^{\star}$ .
I.3.2 Empirical overall balance
As in [40], the key to obtain asymptotic optimality is to show that the empirical proportion satisfy the empirical overall balance equation. Compared to them, the novelty of Lemma 59 is that we use IDS proportions with  $K(K - 1)$  tracking procedures to select between the leader and the challenger instead of sampling. The proof of Lemma 59 is highly similar to the one of Lemma 19.
Lemma 59. Let  $\gamma >0$ . There exists  $N_{2}$  with  $\mathbb{E}_{\nu}[N_2] < +\infty$  such that for all  $n\geq N_2$
$$
\left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \frac {N _ {n , i}}{n - 1}\right) ^ {2} \right| \leq \gamma .
$$
Proof. Let  $N_{1}$  as in Lemma 58. We proceed as in Lemma 19. Let us define
$$
G _ {n} = \left(\sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t})\right) ^ {2} - \sum_ {j \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} \left(C _ {t} = j\right) \left(1 - \beta_ {t} (i ^ {\star}, j)\right)\right) ^ {2},
$$
$$
\begin{array}{l} \frac {1}{2} (G _ {n + 1} - G _ {n}) = \beta_ {n} (i ^ {\star}, C _ {n}) \sum_ {t = N _ {1}} ^ {n - 1} \beta_ {t} (i ^ {\star}, C _ {t}) - \frac {1 - \beta_ {n} (i ^ {\star} , C _ {n})}{(1 - \varepsilon_ {0}) ^ {2}} \sum_ {t = N _ {1}} ^ {n - 1} \mathbb {1} (C _ {t} = C _ {n}) (1 - \beta_ {t} (i ^ {\star}, C _ {n})) \\ + \beta_ {n} (i ^ {\star}, C _ {n}) ^ {2} - \frac {1}{(1 - \varepsilon_ {0}) ^ {2}} (1 - \beta_ {n} (i ^ {\star}, C _ {n})) ^ {2}. \\ \end{array}
$$
Using that  $\beta_{n}(i^{\star},C_{n})N_{n,i^{\star}} = (1 - \beta_{n}(i^{\star},C_{n}))N_{n,C_{n}} / (1 - \varepsilon_{0})^{2}$  since  $\beta_{n}(i^{\star},C_{n}) = N_{n,C_{n}} / ((1-\varepsilon_{0})^{2}N_{n,i^{\star}} + N_{n,C_{n}})$  and the inequalities proven in Lemma 19, we can exhibit deterministic constants  $A,B > 0$  (depending on  $K$  and  $\varepsilon_0$ ) such that
$$
\left| G _ {n + 1} - G _ {n} \right| \leq A N _ {1} + B \quad \text {h e n c e} \quad \left| G _ {n} \right| \leq (n - 1 - N _ {1}) (A N _ {1} + B) + (1 - \varepsilon_ {0}) ^ {2}.
$$
where the second inequality is obtained by telescopic sum (as in Lemma 19) and using that  $G_{N_1 + 1} = \beta_{N_1}(i^\star, C_{N_1})^2 - (1 - \beta_{N_1}(i^\star, C_{N_1}))^2 / (1 - \varepsilon_0)^2$ .
As in Lemma 19, it is possible to exhibit deterministic constants  $C, D > 0$  (depending on  $K$  and  $\varepsilon_0$ ) such that
$$
\left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \frac {N _ {n , i}}{n - 1}\right) ^ {2} - \frac {G _ {n}}{(n - 1) ^ {2}} \right| \leq \frac {C N _ {1} + D}{n - 1}.
$$
Combining both results, we can show
$$
\left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \frac {N _ {n , i}}{n - 1}\right) ^ {2} \right| \leq \frac {(A + C) N _ {1} + B + D}{n - 1} + \frac {(1 - \varepsilon_ {0}) ^ {2}}{(n - 1) ^ {2}}.
$$
Therefore, we have shown the desired result for  $n \geq N_2$  defined as
$$
N _ {2} = \inf  \left\{n > 2 \mid \frac {(A + C) N _ {1} + B + D}{n - 1} + \frac {(1 - \varepsilon_ {0}) ^ {2}}{(n - 1) ^ {2}} \leq \gamma \right\},
$$
As in [40], using Lemma 59 allows to bound the empirical proportion allocated to the unique best arm  $N_{n,i^*} / (n - 1)$  away from 0 (Lemma 60).
Lemma 60. There exists  $N_{3}$  with  $\mathbb{E}_{\nu}[N_3] < +\infty$  such that for all  $n \geq N_3$
$$
c _ {L} \leq \frac {N _ {n , i ^ {\star}}}{n - 1} \leq c _ {U} \quad \text {w h e r e} \quad c _ {U} = \sqrt {\frac {(1 - \varepsilon_ {0}) ^ {2} / 2 + 1}{(1 - \varepsilon_ {0}) ^ {2} + 1}} \quad \text {a n d} \quad c _ {L} = \frac {1 - c _ {U}}{(1 - \varepsilon_ {0}) \sqrt {2 (K - 1)}}.
$$
Proof. Let  $N_{2}$  as in Lemma 59 for  $\frac{\gamma}{(1 - \varepsilon_0)^4}$ . Using Lemma 59, we obtain for all  $n \geq N_{2}$
$$
\begin{array}{l} \gamma \geq (1 - \varepsilon_ {0}) ^ {2} \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \geq (1 - \varepsilon_ {0}) ^ {2} \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \left(1 - \frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} \\ = \left((1 - \varepsilon_ {0}) ^ {2} - 1\right) \left(\frac {N _ {n , i ^ {*}}}{n - 1}\right) ^ {2} + 2 \frac {N _ {n , i ^ {*}}}{n - 1} - 1 \\ \geq \left((1 - \varepsilon_ {0}) ^ {2} + 1\right) \left(\frac {N _ {n , i ^ {*}}}{n - 1}\right) ^ {2} - 1. \\ \end{array}
$$
where we used that  $\frac{N_{n,i^{\star}}}{n - 1} \leq 1$ . Taking  $\gamma = (1 - \varepsilon_0)^2 / 2$  yields the upper bound.
Let  $\tilde{N}_2$  as in Lemma 59 for  $\frac{\gamma}{(K - 1)(1 - \varepsilon_0)^2}$ . Using Lemma 59, we obtain for all  $n\geq \max \{N_2,\tilde{N}_2\}$
$$
\begin{array}{l} - \gamma \leq (K - 1) (1 - \varepsilon_ {0}) ^ {2} \left(\frac {N _ {n , i ^ {*}}}{n - 1}\right) ^ {2} - (K - 1) \sum_ {i \neq i ^ {*}} \left(\frac {N _ {n , i}}{n - 1}\right) ^ {2} \\ \leq (K - 1) \left(1 - \varepsilon_ {0}\right) ^ {2} \left(\frac {N _ {n , i ^ {*}}}{n - 1}\right) ^ {2} - \left(1 - \frac {N _ {n , i ^ {*}}}{n - 1}\right) ^ {2} \\ \leq (K - 1) \left(1 - \varepsilon_ {0}\right) ^ {2} \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \left(1 - c _ {U}\right) ^ {2} \\ \end{array}
$$
Taking  $\gamma = (1 - c_U)^2 / 2$  yields the lower bound. Note that
$$
c _ {L} \leq c _ {U} \quad \Longleftrightarrow \quad 1 \leq \frac {\left(1 - \varepsilon_ {0}\right) ^ {2} / 2 + 1}{\left(1 - \varepsilon_ {0}\right) ^ {2} + 1} \left(2 (K - 1) \left(1 - \varepsilon_ {0}\right) ^ {2} + 2 \sqrt {2 (K - 1)} \left(1 - \varepsilon_ {0}\right) + 1\right),
$$
where the last condition is trivially true. Taking  $N_{3} = \max \{N_{2},\tilde{N}_{2}\}$  yields the result.
Lemma 61 is a rescaled version of the empirical overall balance equation which is proven by simply combining Lemma 59 and Lemma 60.
Lemma 61. Let  $\gamma >0$ . There exists  $N_{4}$  with  $\mathbb{E}_{\nu}[N_4] < +\infty$  such that for all  $n\geq N_4$
$$
\left| (1 - \varepsilon_ {0}) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {N _ {n , i}}{N _ {n , i ^ {\star}}}\right) ^ {2} \right| \leq \gamma .
$$
Proof. Let  $N_{3}$  and  $c_{L}$  as in Lemma 60. Let  $N_{2}$  as in Lemma 59 for  $\gamma c_{L}^{2} / (1 - \varepsilon_{0})^{2}$ . Direct manipulation shows that, for all  $n\geq N_4 = \max \{N_2,N_3\}$ ,
$$
\begin{array}{l} \left| 1 - \sum_ {i \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \frac {N _ {n , i}}{N _ {n , i ^ {\star}}}\right) ^ {2} \right| = \left(\frac {n - 1}{N _ {n , i ^ {\star}}}\right) ^ {2} \left| \left(\frac {N _ {n , i ^ {\star}}}{n - 1}\right) ^ {2} - \sum_ {i \neq i ^ {\star}} \left(\frac {1}{1 - \varepsilon_ {0}} \frac {N _ {n , i}}{n - 1}\right) ^ {2} \right| \\ \leq \frac {1}{c _ {L} ^ {2}} \gamma c _ {L} ^ {2} = \frac {\gamma}{(1 - \varepsilon_ {0}) ^ {2}}. \\ \end{array}
$$
I.3.3 Convergence towards optimal allocation