SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \operatorname {R e g r e t} _ {p} \left(\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} _ {\boldsymbol {\pi}}\right) \geq \max  \left\{\operatorname {R e g r e t} _ {p} \left(\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} ^ {a}\right), \operatorname {R e g r e t} _ {p} \left(\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} ^ {b}\right) \right\} \\ \geq d \left(\boldsymbol {\sigma} ^ {a}, \boldsymbol {\sigma} ^ {b}\right) \exp \left(- \sum_ {g \in [ G ]} \mathbb {E} _ {\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} ^ {a}} \left[ n _ {g, t} \right] K L \left(\mathcal {D} _ {g} ^ {a} \| \mathcal {D} _ {g} ^ {b}\right)\right) \\ \geq \left(r _ {p} \left(\frac {1}{G} \boldsymbol {u}; \boldsymbol {\sigma} ^ {a}\right) - r _ {p} ^ {*} (\boldsymbol {\sigma} ^ {a})\right) \exp \left(\frac {- 1}{2}\right) \\ \end{array}
$$
where the first step stems from the definition of  $\mathcal{D}_{\pi}$ , the second step stems from Lemma 6, and the third step stems from a combination of both Lemma 19 and Lemma 15. The rest of the proof consists in deriving a lower bound on  $r_p\left(\frac{1}{G} u;\pmb{\sigma}^a\right) - r_p^* (\pmb{\sigma}^a)$ . Since we will now solely focus on  $\pmb{\sigma}^a$ , we can drop the dependencies in  $\pmb{\sigma}$ . A direct consequence of Inequality (19) is
$$
\frac {r _ {p} \left(\frac {1}{G} \boldsymbol {u}\right)}{r _ {p} ^ {*}} - 1 \geq \frac {p + 1}{2} \sum_ {g \in [ G ]} \frac {\left(\frac {1}{G} - \lambda_ {g} ^ {*}\right) ^ {2}}{\lambda_ {g} ^ {*}} - \frac {7 (p + 2) ^ {2}}{\left(\min _ {g} \lambda_ {g} ^ {*}\right) ^ {2}} \max  _ {g} \left(\frac {\lambda_ {g} ^ {*}}{1 / G}\right) ^ {3 p + 3} \| \boldsymbol {u} / G - \boldsymbol {\lambda} ^ {*} \| _ {\infty} ^ {3}. \tag {28}
$$
We will lower bound each of the terms  $\frac{p + 1}{2}\sum_{g\in [G]}\frac{\left(\frac{1}{G} - \lambda_g^*\right)^2}{\lambda_g^*}$  and
$-\frac{7(p + 2)^2}{(\min_g\lambda_g^*)^2}\max_g\left(\frac{\lambda_g^*}{1 / G}\right)^{3p + 3}\| \boldsymbol {u} / G - \boldsymbol {\lambda}^*\|_\infty^3.$  For convenience, we set
$$
f _ {T} := \lambda_ {1} ^ {*} - \frac {1}{G}.
$$
Since the first group has the highest variance,  $\lambda_1^*$  should also be the highest. In particular,  $\lambda_1^* \geq \frac{1}{G}$  and  $f_T \geq 0$ . By symmetry of  $\sigma^a$ ,  $\lambda_2^* = \ldots = \lambda_G^* = \frac{1}{G} - \frac{f_T}{G - 1}$ . Therefore,
$$
\sum_ {g \in [ G ]} \frac {\left(\frac {1}{G} - \lambda_ {g} ^ {*}\right) ^ {2}}{\lambda_ {g} ^ {*}} = \frac {f _ {T} ^ {2}}{\frac {1}{G} + f _ {T}} + \frac {(G - 1) \left(\frac {f _ {T}}{G - 1}\right) ^ {2}}{\frac {1}{G} - \frac {f _ {T}}{G - 1}} = G f _ {T} ^ {2} \left(\underbrace {\frac {1}{1 + G f _ {T}}} _ {\geq 1 - G f _ {T}} + \underbrace {\frac {1}{G - 1} \frac {1}{1 - \frac {G f _ {T}}{G - 1}}} _ {\geq 0}\right) \geq G f _ {T} ^ {2} - G ^ {2} f _ {T} ^ {3}.
$$
Next, notice that  $\lambda^{*} \leq 1$ , which implies that  $\max_g \left( \frac{\lambda_g^*}{1 / G} \right)^{3p + 3} \leq G^{3p + 3}$ , and that  $\| \boldsymbol{u} / G - \boldsymbol{\lambda}^* \|_{\infty} = \max (f_T, f_T / (G - 1)) = f_T$ , and  $\min_g \lambda_g^* = \frac{1}{G} - \frac{f_T}{G - 1}$ . Therefore,
$$
\frac {r _ {p} \left(\frac {1}{G} \boldsymbol {u}\right)}{r _ {p} ^ {*}} - 1 \geq \frac {p + 1}{2} \left(G f _ {T} ^ {2} - G ^ {2} f _ {T} ^ {3}\right) - \frac {7 (p + 2) ^ {2} G ^ {3 p + 3} f _ {T} ^ {3}}{\left(\frac {1}{G} - \frac {f _ {T}}{G - 1}\right) ^ {2}} = \frac {p + 1}{2} \left(G f _ {T} ^ {2} - G ^ {2} f _ {T} ^ {3}\right) - \frac {7 (p + 2) ^ {2} G ^ {3 p + 5} f _ {T} ^ {3}}{\left(1 - \frac {G}{G - 1} f _ {T}\right) ^ {2}}. \tag {29}
$$
It remains to bound  $f_{T}$ . We do so by deriving the first terms of its taylor expansion in and  $T$ :
$$
\begin{array}{l} f _ {T} = \frac {\left(1 + \frac {1}{\sqrt {T}}\right) ^ {\frac {2 p}{p + 1}}}{G - 1 + \left(1 + \frac {1}{\sqrt {T}}\right) ^ {\frac {2 p}{p + 1}}} - \frac {1}{G} \\ = \frac {1 + \frac {2 p}{p + 1} \frac {1}{\sqrt {T}} + o \left(\frac {1}{\sqrt {T}}\right)}{G + \frac {2 p}{p + 1} \frac {1}{\sqrt {T}} + o \left(\frac {1}{\sqrt {T}}\right)} - \frac {1}{G} \\ = \frac {1}{G} \left(\left(1 + \frac {2 p}{p + 1} \frac {1}{\sqrt {T}} + o \left(\frac {1}{\sqrt {T}}\right)\right) \left(1 - \frac {2 p}{p + 1} \frac {1}{G \sqrt {T}} + o \left(\frac {1}{\sqrt {T}}\right)\right) - 1\right) \\ = \frac {2 p}{p + 1} \frac {1}{G} \left(1 - \frac {1}{G}\right) \frac {1}{\sqrt {T}} + o \left(\frac {1}{\sqrt {T}}\right), \\ \end{array}
$$
where the first equality stems from Lemma 1, the second equality stems from the binomial Taylor expansion, and the third equality stems from the Taylor expansion of  $x \to \frac{1}{1 - x}$ . The last step implies that there exists universal constants  $q_{1}, q_{2}$  such that for  $T \geq 1$ ,
$$
\frac {1}{G} \left(1 - \frac {1}{G}\right) \frac {q _ {1}}{\sqrt {T}} \leq f _ {T} \leq \frac {1}{G} \left(1 - \frac {1}{G}\right) \frac {q _ {2}}{\sqrt {T}}.
$$
Replacing the previous bounding of  $f_{T}$  in Inequality (29) yields
$$
\frac {r _ {p} \left(\frac {1}{G} \boldsymbol {u}\right)}{r _ {p} ^ {*}} - 1 \geq \frac {2 q _ {1} ^ {2}}{G} \left(1 - \frac {1}{G}\right) ^ {2} \frac {p + 1}{T} + O \left(\frac {1}{T \sqrt {T}}\right),
$$
which in turn implies
$$
r _ {p} \left(\frac {1}{G} \boldsymbol {u}\right) - r _ {p} ^ {*} \geq \frac {q _ {1} ^ {2} (p + 1)}{4} \frac {r _ {p} ^ {*}}{G T} + O \left(\frac {1}{T ^ {2} \sqrt {T}}\right) \geq \frac {q _ {1} ^ {2}}{4} \frac {p + 1}{T ^ {2}} + O \left(\frac {1}{T ^ {2} \sqrt {T}}\right).
$$
where we used  $r_p^* = \Theta (T^{-1})$  and from Lemma 1 that
$$
r _ {p} ^ {*} = \left\| \frac {\pmb {\sigma} ^ {2}}{\pmb {n} _ {T} ^ {*}} \right\| _ {p} = \frac {1}{T} \Sigma_ {p} ^ {\frac {p + 1}{p}} = \frac {\left((G - 1) + \left(1 + \frac {1}{\sqrt {T}}\right) ^ {\frac {2 p}{p + 1}}\right) ^ {\frac {p + 1}{p}}}{T} \geq \frac {G}{T}.
$$
By setting  $\kappa = \frac{q_1^2}{4}$ , we get  $\mathrm{Regret}_T(\pmb{\pi},\pmb{D}_{\pmb{\pi}})\geq \kappa \frac{p + 1}{T^2} +O\left(\frac{1}{T^2\sqrt{T}}\right)$ , which completes the proof of Theorem 4.
C Upper and lower bounds when  $p = \infty$
The proof for Theorem 3 (upper bound when  $p = \infty$ ) follows the same high-level steps as the proof of Theorem 1 (upper bound when  $p \in \mathbb{R}$ ). However, some adjustments of the proofs are necessary. Table 2 summarizes the changes that are required.
In Appendix C.1, we introduce and prove Lemma 5, the replacement for Lemma 4. In Appendix C.3, we prove Theorem 4.
Result	Does it hold for p = +∞?
Lemma 1	Yes
Lemmas 2, 3	Yes
Lemma 4	No, replaced by Lemma 5
Lemmas 6, 19, 15	Yes
d(σa, σb) = Θ(T-2)	No, replaced by d(σa, σb) = Θ(T-1.5)
Table 2: Summary of the possible extensions to  $p =  + \infty$
C.1 Curvature of  $R_{\infty}$
The upper bound Lemma 4 goes to  $+\infty$  as  $p = +\infty$  and is no longer insightful. In this section, we provide a suitable bound:
$$
\frac {R _ {\infty} (\boldsymbol {n} _ {T} ^ {\prime}) - R _ {\infty} (\boldsymbol {n} _ {T} ^ {*})}{R _ {\infty} (\boldsymbol {n} _ {T} ^ {*})} \leq - \min _ {g} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) + \frac {1}{4} \max _ {g} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \max _ {g} \left(\frac {n _ {g , T} ^ {*}}{n _ {g} ^ {\prime}}\right) ^ {3}
$$
From Lemma 1,
$$
\frac {\sigma_ {1} ^ {2}}{n _ {1 , T} ^ {*}} = \ldots = \frac {\sigma_ {G} ^ {2}}{n _ {G , T} ^ {*}} = R _ {\infty} (\pmb {n} _ {T} ^ {*}) = R _ {\infty} ^ {*},
$$
so that for  $g\in [G]$  and  $\pmb{n}^{\prime}$
$$
\begin{array}{l} \frac {\sigma_ {g} ^ {2}}{n _ {g} ^ {\prime}} = R _ {\infty} ^ {*} + \sigma_ {g} ^ {2} \left(\frac {1}{n _ {g} ^ {\prime}} - \frac {1}{n _ {g , T} ^ {*}}\right) \\ \leq R _ {\infty} ^ {*} + \sigma_ {g} ^ {2} \left(\frac {- 1}{(n _ {g , T} ^ {*}) ^ {2}} \left(n _ {g} ^ {\prime} - n _ {g, T} ^ {*}\right) + \frac {1}{2} \left(n _ {g} ^ {\prime} - n _ {g, T} ^ {*}\right) ^ {2} \sup  _ {x \in \left\{n _ {g} ^ {\prime}, n _ {g, T} ^ {*} \right\}} \frac {1}{2 x ^ {3}}\right) \\ = R _ {\infty} ^ {*} \left(1 - \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) + \frac {1}{4} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \max \left(1, \left(\frac {n _ {g , T} ^ {*}}{n _ {g} ^ {\prime}}\right) ^ {3}\right)\right) \\ \end{array}
$$
where the first step follows from  $R_{\infty}^{*} = \frac{\sigma_{g}^{2}}{n_{g,T}^{*}}$ , the second step follows from Taylor's inequality applied on the function  $x \to \frac{1}{x}$  on  $n_{g,T}^{*}$ , and the first step follows from the definition of the infinite norm. By dividing both sides by  $R_{\infty}^{*}$ , rearranging the terms, and taking the max, we obtain
$$
\begin{array}{l} \frac {R _ {\infty} \left(\boldsymbol {n} _ {T} ^ {\prime}\right) - R _ {\infty} \left(\boldsymbol {n} _ {T} ^ {*}\right)}{R _ {\infty} \left(\boldsymbol {n} _ {T} ^ {*}\right)} \leq \max  _ {g} \left(- \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) + \frac {1}{4} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \max  \left(1, \left(\frac {n _ {g , T} ^ {*}}{n _ {g} ^ {\prime}}\right) ^ {3}\right)\right) \\ \leq - \min  _ {g} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) + \frac {1}{4} \max  _ {g} \left(\frac {n _ {g} ^ {\prime}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \max  _ {g} \left(\frac {n _ {g , T} ^ {*}}{n _ {g} ^ {\prime}}\right) ^ {3}. \\ \end{array}
$$
which completes the proof of Lemma 1.
C.2 Proof of Theorem 3
$$
R e g r e t _ {\infty , T} (V a r i a n c e - U C B, \boldsymbol {\mathcal {D}}) \leq \left(C _ {T} \sqrt {\Sigma_ {\infty}} + C _ {T} ^ {2}\right) G ^ {1. 5} T ^ {- 1. 5} + o (T ^ {- 1. 5}) = \tilde {O} (T ^ {- 1. 5}).
$$
Proof. Similarly to Appendix A.5, the total-expectation bounding still holds:
$$
\operatorname {R e g r e t} _ {\infty , T} \left(\text {V a r i a n c e - U C B}, \boldsymbol {\mathcal {D}}\right) \leq \mathbb {E} \left[ R _ {\infty} (\boldsymbol {n}) - R _ {\infty} ^ {*} \mid \mathcal {A} _ {T} \right] + \| \sigma^ {2} \| _ {\infty} \mathbb {P} _ {\pi} \left(\mathcal {A} _ {T} ^ {c}\right). \tag {30}
$$
We now upper bound each of the two terms in (30). First, similarly to the case where  $p < +\infty$  presented in Appendix A,  $\mathbb{P}_{\pi}(\mathcal{A}_T^c) \leq 2GT^{-2.5}$ , so that
$$
\left\| \boldsymbol {\sigma} ^ {2} \right\| _ {\infty} \mathbb {P} _ {\pi} \left(\mathcal {A} _ {T} ^ {c}\right) \leq 2 G \left\| \boldsymbol {\sigma} ^ {2} \right\| _ {\infty} T ^ {- 2. 5} = o \left(T ^ {- 1. 5}\right). \tag {31}
$$
Next, we apply the final lower bound derived in Section B.2 from [13]:
Lemma 17. Conditionally on  $\mathcal{A}_T$
$$
\frac {n _ {g , T}}{n _ {g , T} ^ {*}} - 1 \geq \frac {- G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right).
$$
By taking the min over  $g \in [G]$  in Lemma 17, the following inequality follows conditionally on  $\mathcal{A}_T$ :
$$
- \min  _ {g} \left(\frac {n _ {g , T}}{n _ {g , T} ^ {*}} - 1\right) \leq \frac {G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right). \tag {32}
$$
Moreover, since  $C_T = \tilde{O}(1)$ , the right hand side of the inequality above is in  $\tilde{O}(\sqrt{T})$ . Next, from Inequality (23), and from  $\min_g n_{g,T}^* = T \frac{\min_g \sigma_g^2}{\Sigma_\infty}$ ,
$$
\begin{array}{l} \max _ {g} \left(\frac {n _ {g , T}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \leq \frac {1}{(\min _ {h} n _ {h , T} ^ {*}) ^ {2}} \left(3 G + \frac {4 G C _ {T}}{\Sigma_ {\infty}} \left(\min _ {g} \sigma_ {g} + \frac {2 C _ {T}}{\sqrt {\min _ {h} n _ {h , T} ^ {*}}}\right) \sqrt {\frac {\min _ {h} n _ {h , T} ^ {*}}{n _ {g , T} ^ {*}}} - 1\right) \\ \leq \frac {1 8 G ^ {2}}{(\min _ {h} n _ {h , T} ^ {*}) ^ {2}} + \frac {3 2 G ^ {2} C _ {T} ^ {2}}{\Sigma_ {\infty} \min _ {h} n _ {h , T} ^ {*}} \left(2 (\min _ {g} \sigma_ {g}) ^ {2} + 4 C _ {T} ^ {2}\right) \\ = \frac {6 4 G ^ {2} C _ {T} ^ {2}}{T} \left(1 + \frac {2 C _ {T} ^ {2}}{\min _ {g} \sigma_ {g} ^ {2}}\right) + \frac {1 8 G ^ {2} \Sigma_ {\infty} ^ {2}}{T ^ {2} \min _ {g} \sigma_ {g} ^ {4}} \\ = \tilde {O} (T ^ {- 1}), \\ \end{array}
$$
$$
\begin{array}{l} \max  _ {g} \frac {n _ {g , T} ^ {*}}{n _ {g , T}} = \frac {1}{\min  _ {g} \frac {n _ {g , T}}{n _ {g , T} ^ {*}}} = \frac {1}{1 + \min  _ {g} \left(\frac {n _ {g , T}}{n _ {g , T} ^ {*}} - 1\right)} \\ \leq \frac {1}{\left(1 - \frac {G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right)\right) ^ {+}} \\ = O (1). \\ \end{array}
$$
Combining the previous two inequalities yields
$$
\frac {1}{4} \max  _ {g} \left(\frac {n _ {g , T}}{n _ {g , T} ^ {*}} - 1\right) ^ {2} \max  _ {g} \left(\frac {n _ {g , T} ^ {*}}{n _ {g ,}}\right) ^ {3} \leq \frac {\frac {6 4 G ^ {2} C _ {T} ^ {2}}{T} \left(1 + \frac {2 C _ {T} ^ {2}}{\min  _ {g} \sigma_ {g} ^ {2}}\right) + \frac {1 8 G ^ {2} \Sigma_ {\infty} ^ {2}}{T ^ {2} \min  _ {g} \sigma_ {g} ^ {4}}}{\left[ \left(1 - \frac {G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right)\right) ^ {+} \right] ^ {3}}. \tag {33}
$$
Recall that  $R_{\infty}^{*} = \frac{\Sigma_{\infty}}{T}$ . Applying Lemma 5 on  $n_{g,T}$  and using the upper bounds (32) and (33) implies that conditionally on  $\mathcal{A}_T$
$$
\begin{array}{l} R _ {\infty} (\boldsymbol {n} _ {T}) - R _ {\infty} (\boldsymbol {n} _ {T} ^ {*}) \leq \frac {G ^ {3 / 2} \Sigma_ {\infty}}{T ^ {3 / 2}} \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) \\ + \frac {G ^ {7 / 4} \Sigma_ {\infty}}{T ^ {9 / 4}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} \\ + \frac {\frac {6 4 G ^ {2} \Sigma_ {\infty} C _ {T} ^ {2}}{T ^ {2}} \left(1 + \frac {2 C _ {T} ^ {2}}{\min _ {g} \sigma_ {g} ^ {2}}\right)}{\left[ \left(1 - \frac {G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right)\right) ^ {+} \right] ^ {3}} \\ + \frac {\frac {1 8 G ^ {2} \Sigma_ {\infty} ^ {3}}{T ^ {3} \min _ {g} \sigma_ {g} ^ {4}}}{\left[ \left(1 - \frac {G \sqrt {G}}{\sqrt {T}} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) + 8 \sqrt {2} G ^ {1 / 4} \left(\frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) ^ {3 / 4} \sqrt {1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}} T ^ {- 3 / 4}\right)\right) ^ {+} \right] ^ {3}}. \\ \end{array}
$$
Since the right hand side is deterministic, the inequality above extends by taking the conditional expected value on  $\mathcal{A}_T$ . Moreover, the second, third, and fourth term are all in  $o(T^{-1.5})$ , so that:
$$
\mathbb {E} \left[ R _ {\infty} (\boldsymbol {n}) - R _ {\infty} ^ {*} \mid \mathcal {A} _ {T} \right] \leq C _ {T} \sqrt {\Sigma_ {\infty}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) G ^ {1. 5} T ^ {- 1. 5} + o \left(T ^ {- 1. 5}\right). \tag {34}
$$
Finally, combining both inequalities (31) and (34) in Inequality (30) yields
$$
\operatorname {R e g r e t} _ {\infty , T} (\text {V a r i a n c e - U C B}, \mathcal {D}) \leq C _ {T} \sqrt {\Sigma_ {\infty}} \left(1 + \frac {C _ {T}}{\sqrt {\Sigma_ {\infty}}}\right) G ^ {1. 5} T ^ {- 1. 5} + o (T ^ {- 1. 5}),
$$
which completes the proof of Theorem 3.
C.3 Proof of Theorem 4
Lemma 18. Let  $\sigma^a, \sigma^b$  be two vectors with  $\Sigma_{\infty}^a \coloneqq \sum_g (\sigma_g^a)^2$ ,  $\Sigma_{\infty}^b \coloneqq \sum_g (\sigma_g^b)^2$  and  $\Sigma_{\infty}^a \geq \Sigma_{\infty}^b$ . There exists a  $\Sigma_{\infty}^{a,b} \in [\Sigma_{\infty}^b, \Sigma_{\infty}^a]$  such that
$$
d (\boldsymbol {\sigma} ^ {a}, \boldsymbol {\sigma} ^ {b}) = \frac {\sum_ {\infty} ^ {a , b}}{2 T} \| \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {a}) - \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {b}) \| _ {1}
$$
Proof. Let  $\sigma^a, \sigma^b$  be two vectors with  $\Sigma_{\infty}^a := \sum_g (\sigma_g^a)^2$  and  $\Sigma_{\infty}^b := \sum_g (\sigma_g^b)^2$ ,  $\epsilon > 0$ , and  $\lambda \in [0,1]^G$  such that  $\sum_g \lambda_g = 1$ . We have:
$$
R _ {\infty} (T \pmb {\lambda}; \pmb {\sigma} ^ {a}) - R _ {\infty} ^ {*} (\pmb {\sigma} ^ {a}) = \frac {1}{T} \left(\max _ {g} \frac {\sigma_ {g} ^ {a}}{\lambda_ {g}} - \Sigma_ {\infty} ^ {a}\right) = \frac {\Sigma_ {\infty} ^ {a}}{T} \left(\max _ {g} \frac {\lambda_ {g} ^ {*} (\pmb {\sigma} ^ {a})}{\lambda_ {g}} - 1\right),
$$
therefore,
$$
\begin{array}{l} T \boldsymbol {\lambda} \in S _ {\epsilon} ^ {a} \iff R _ {\infty} (T \boldsymbol {\lambda}; \boldsymbol {\sigma} ^ {a}) - R _ {\infty} ^ {*} (\boldsymbol {\sigma} ^ {a}) \leq \epsilon \\ \Longleftrightarrow \frac {\Sigma_ {\infty} ^ {a}}{T} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {a})}{\lambda_ {g}} - 1\right) \leq \epsilon \\ \Longleftrightarrow \forall g \in [ G ], \quad \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1} \leq \lambda_ {g}. \\ \end{array}
$$
Hence, the allocation  $T\lambda$  is simultaneously in  $S_{\epsilon}^{a}$  and  $S_{\epsilon}^{b}$  if and only if
$$
\forall g \in [ G ], \quad \max  \left(\frac {\lambda_ {g} ^ {*} (\pmb {\sigma} ^ {\pmb {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\pmb {\sigma} ^ {\pmb {b}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}\right) \leq \lambda_ {g},
$$
In particular,  $S_{\epsilon}^{a}\cap S_{\epsilon}^{b}$  is the polytope
$$
\left\{T \boldsymbol {\lambda} | \boldsymbol {\lambda} \in [ 0, 1 ] ^ {G}, \sum_ {g} \lambda_ {g} = 1, \quad \forall g \in [ G ], \quad \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}\right) \leq \lambda_ {g} \right\}
$$
which is non-empty if and only if
$$
\sum_ {g} \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}\right) \leq 1.
$$
so that  $d(\pmb{\sigma}^a, \pmb{\sigma}^b)$  is the (unique) solution to the equation  $\sum_{g} \max \left( \frac{\lambda_g^*(\pmb{\sigma}^a)}{\Sigma_{\infty}^{a}} + 1, \frac{\lambda_g^*(\pmb{\sigma}^b)}{\Sigma_{\infty}^{b}} + 1 \right) = 1$ . To better understand the form of this solution, we introduce the following three decreasing functions:
$$
f: \epsilon > 0 \to \sum_ {g} \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {a})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {b})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}\right)
$$
$$
f _ {a}: \epsilon > 0 \rightarrow \sum_ {g} \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}\right)
$$
$$
f _ {a}: \epsilon > 0 \rightarrow \sum_ {g} \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {a})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {b})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {b}} + 1}\right)
$$
First, notice that  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) = f^{-1}(1)$ . Next, by assuming (WLOG) that  $\Sigma_{\infty}^a \geq \Sigma_{\infty}^b$ , we have  $f^a \geq f \geq f^b$ , so that  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) = f^{-1}(1) \in [(f^b)^{-1}(1), (f^a)^{-1}(1)]$ . Moreover,
$$
\begin{array}{l} f _ {a} (\epsilon) = \sum_ {g} \max  \left(\frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \frac {\lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}})}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}\right) \\ = \frac {\sum_ {g} \max \left(\lambda_ {g} ^ {*} (\pmb {\sigma} ^ {\pmb {a}}) , \lambda_ {g} ^ {*} (\pmb {\sigma} ^ {\pmb {b}})\right)}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1} \\ = \frac {\frac {1}{2} \sum_ {g} \lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}}) + \lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}}) + | \lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {a}}) - \lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {\boldsymbol {b}}) |}{\frac {T _ {\epsilon}}{\Sigma_ {\infty} ^ {\alpha}} + 1} \\ = \frac {1 + \frac {1}{2} \| \pmb {\lambda} ^ {*} (\pmb {\sigma} ^ {a}) - \pmb {\lambda} ^ {*} (\pmb {\sigma} ^ {b}) \| _ {1}}{\frac {T \epsilon}{\Sigma_ {\infty} ^ {a}} + 1}, \\ \end{array}
$$
where the first step stems from the definition of  $f_{a}$ , the second step stems from  $\max(x, y) = \frac{x + y + |x - y|}{2}$ , the third step stems from  $\sum_{g} \lambda_{g}^{*} = 1$ , and the fourth step stems from the definition of the 1-norm. The last equation implies
$$
f _ {a} ^ {- 1} (1) = \frac {\Sigma_ {\infty} ^ {a}}{2 T} \left\| \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {a}) - \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {b}) \right\| _ {1}.
$$
Similarly,  $f_{b}^{-1}(1) = \frac{\Sigma_{\infty}^{b}}{2T}\| \pmb{\lambda}^{*}(\pmb{\sigma}^{\pmb{a}}) - \pmb{\lambda}^{*}(\pmb{\sigma}^{\pmb{b}})\|_{1}$ . Therefore:
$$
\frac {\sum_ {\infty} ^ {b}}{2 T} \left\| \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {a}) - \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {b}) \right\| _ {1} \leq d (\boldsymbol {\sigma} ^ {a}, \boldsymbol {\sigma} ^ {b}) \leq \frac {\sum_ {\infty} ^ {a}}{2 T} \left\| \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {a}) - \boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {b}) \right\| _ {1},
$$
Proof. The proof consists of constructing two instances that optimize the trade-off stated in Lemma 6. The first instance, which is denoted  $\mathcal{D}^a$ , consists of  $G$  groups of standard normal distributions. Next, select a group  $h$  such that  $\mathbb{E}_{\pi,\mathcal{D}^a}[n_{g,T}]$  is minimal. Notice that by minimality of  $\mathbb{E}_{\pi,\mathcal{D}^a}[n_{h,T}]$ ,
$$
\mathbb {E} _ {\pi , \boldsymbol {\mathcal {D}} ^ {a}} [ n _ {h, T} ] \leq \frac {1}{G} \sum_ {g \in [ G ]} \mathbb {E} _ {\pi , \boldsymbol {\mathcal {D}} ^ {a}} [ n _ {g, T} ] = \frac {1}{G} \mathbb {E} _ {\pi , \boldsymbol {\mathcal {D}} ^ {a}} \left[ \sum_ {g \in [ G ]} n _ {g, T} \right] = \frac {T}{G}. \tag {35}
$$
The second instance, which is denotes  $\mathcal{D}^b$ , is defined as follows:
$$
\mathcal {D} _ {g} ^ {b} = \left\{ \begin{array}{l l} \mathcal {N} (0, 1 + \nu) & \text {F o r} g = h \\ \mathcal {N} (0, 1) & \text {O t h e r w i s e} \end{array} \right.
$$
$$
\left\{ \begin{array}{l l} \boldsymbol {\sigma} ^ {a} & = (1, \ldots , 1) \\ \boldsymbol {\sigma} ^ {b} & = \boldsymbol {\sigma} ^ {a} + \nu \boldsymbol {e} _ {h} \end{array} \right.
$$
so that  $\Sigma_{\infty}^{a} = G, \Sigma_{\infty}^{b} = G + \nu$  and
$$
\boldsymbol {\lambda} ^ {*} (\boldsymbol {\sigma} ^ {a}) = \frac {1}{G} (1, \dots , 1)
$$
$$
\forall g \in [ G ], \quad \lambda_ {g} ^ {*} (\boldsymbol {\sigma} ^ {b}) = \left\{ \begin{array}{l l} \frac {1}{G + \nu} & \text {F o r} g \neq h \\ \frac {1 + \nu}{G + \nu} & \text {F o r} g = h \end{array} \right.
$$