SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathcal {H} _ {g, h} = \frac {\partial^ {2}}{\partial \lambda_ {g} \partial \lambda_ {h}} r _ {p} \\ = \frac {\partial}{\partial \lambda_ {g}} \left(r _ {p} ^ {1 - p} H _ {h}\right) \\ = (1 - p) r _ {p} ^ {- p} \left(\frac {\partial}{\partial \lambda_ {g}} r _ {p}\right) H _ {h} + r _ {p} ^ {1 - p} \frac {\partial}{\partial \lambda_ {g}} H _ {h} \\ = (1 - p) r _ {p} ^ {1 - 2 p} H _ {g} H _ {h} + (p + 1) r _ {p} ^ {1 - p} G _ {g, h}. \\ \end{array}
$$
where the first equality is due to the definition of the Hessian, the second equality is due to the expression of the gradient from Eq. (26), the third equality applies the product rule to the derivative, and the fourth equality applies the definition of  $G_{g,h}$  in Lemma 13. This proves the second Equality of Lemma 13.
Third derivatives.
$$
\begin{array}{l} \frac {\partial^ {3}}{\partial \lambda_ {g} \partial \lambda_ {h} \partial_ {i}} r _ {p} = \frac {\partial}{\partial \lambda_ {i}} \mathcal {H} _ {g, h} \\ = \frac {\partial}{\partial \lambda_ {i}} \left\{(1 - p) r _ {p} ^ {1 - 2 p} H _ {g} H _ {h} + (p + 1) r _ {p} ^ {1 - p} G _ {g, h} \right\} \\ = (1 - p) \frac {\partial}{\partial \lambda_ {i}} \left\{r _ {p} ^ {1 - 2 p} H _ {g} H _ {h} \right\} + (p + 1) \frac {\partial}{\partial \lambda_ {i}} \left\{r _ {p} ^ {1 - p} G _ {g, h} \right\}, \\ \end{array}
$$
where the first equality is due to the definition of the Hessian, the second inequality is due to the Hessian expression established previously, and the third equality is due to the linearity of derivation. In a similar fashion to the previous case, we derive each of the products  $r_p^{1 - 2p}H_gH_h$  and  $r_p^{1 - p}G_{g,h}$  separately. On the one hand,
$$
\begin{array}{l} \frac {\partial}{\partial \lambda_ {i}} \left\{r _ {p} ^ {1 - 2 p} H _ {g} H _ {h} \right\} = H _ {g} H _ {h} \frac {\partial}{\partial \lambda_ {i}} \left\{r _ {p} ^ {1 - 2 p} \right\} + r _ {p} ^ {1 - 2 p} \left(H _ {g} \frac {\partial}{\partial \lambda_ {i}} H _ {h} + H _ {h} \frac {\partial}{\partial \lambda_ {i}} H _ {g}\right) \\ = H _ {g} H _ {h} (1 - 2 p) r _ {p} ^ {- 2 p} r _ {p} ^ {1 - p} H _ {i} + r _ {p} ^ {1 - 2 p} \left(H _ {g} (p + 1) G _ {h, i} + H _ {h} (p + 1) G _ {g, i}\right) \\ = (1 - 2 p) r _ {p} ^ {1 - 3 p} H _ {g} H _ {h} H _ {i} + (p + 1) r _ {p} ^ {1 - 2 p} \left(H _ {g} G _ {h, i} + H _ {h} G _ {g, i}\right), \\ \end{array}
$$
where the first equality is due to the derivation product rule, the second equality is due to the definition of  $G_{g,h}$  introduced in Lemma 13, and the third equality is due to a reordering of the terms. On the other hand, by following the exact same steps
$$
\begin{array}{l} \frac {\partial}{\partial \lambda_ {i}} \left\{r _ {p} ^ {1 - p} G _ {g, h} \right\} = (1 - p) r _ {p} ^ {- p} \frac {\partial}{\partial \lambda_ {i}} r _ {p} + r _ {p} ^ {1 - p} \frac {\partial}{\partial \lambda_ {i}} G _ {g, h} \\ = (1 - p) r _ {p} ^ {- p} r _ {p} ^ {1 - p} H _ {i} G _ {g, h} + (p + 2) r _ {p} ^ {1 - p} I _ {g, h, i} \\ = (1 - p) r _ {p} ^ {1 - 2 p} H _ {i} G _ {g, h} + (p + 2) r _ {p} ^ {1 - p} I _ {g, h, i}. \\ \end{array}
$$
Replacing the previous two expressions in the formula for  $\frac{\partial^3}{\partial\lambda_g\partial\lambda_h\partial_i} r_p$  yields
$$
\begin{array}{l} \frac {\partial^ {3}}{\partial \lambda_ {g} \partial \lambda_ {h} \partial_ {i}} r _ {p} = (1 - p) (1 - 2 p) H _ {g} H _ {h} H _ {i} r _ {p} ^ {1 - 3 p} \\ + (1 - 2 p) (1 + p) \left(H _ {g} G _ {h, i} + H _ {h} G _ {i, g} + H _ {g} G _ {h, i}\right) \\ + r _ {p} ^ {1 - 2 p} + (p + 1) (p + 2) I _ {g, h, i} r _ {p} ^ {1 - p}, \\ \end{array}
$$
where the symmetry of  $G_{g,h} = G_{h,g}$  is used. This derives the third Eq. of Lemma 13 and concludes the proof.
We are now ready to establish the proof of Lemma 11.
Lemma 11.  $\langle \mathcal{H}(\pmb{\lambda}^{*})(\pmb{\lambda} - \pmb{\lambda}^{*}), \pmb{\lambda} - \pmb{\lambda}^{*}\rangle = (p + 1)r_{p}(\pmb{\lambda}^{*})\sum_{g \in [G]}\frac{(\lambda_{g} - \lambda_{g}^{*})^{2}}{\lambda_{g}^{*}}.$
Proof. Setting the value  $\lambda^*$  in the value of the Hessian established in Lemma 13 implies that for  $g, h \in [G]$ :
$$
\begin{array}{l} \mathcal {H} _ {g, h} (\boldsymbol {\lambda} ^ {*}) = (1 - p) r _ {p} ^ {1 - 2 p} H _ {g} H _ {h} (\boldsymbol {\lambda} ^ {*}) + (p + 1) r _ {p} ^ {1 - p} (\boldsymbol {\lambda} ^ {*}) G _ {g, h} (\boldsymbol {\lambda} ^ {*}) \\ = (p + 1) r _ {p} ^ {1 - p} \left(\boldsymbol {\lambda} ^ {*}\right) G _ {g, h} \left(\boldsymbol {\lambda} ^ {*}\right) \\ = (p + 1) r _ {p} ^ {1 - p} \left(\boldsymbol {\lambda} ^ {*}\right) \Sigma_ {p} ^ {p + 1} \left(\frac {1}{\lambda_ {G} ^ {*}} + \frac {1}{\lambda_ {g} ^ {*}} \mathbb {1} (g = h)\right) \\ = (p + 1) r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) \left(\frac {1}{\lambda_ {G} ^ {*}} + \frac {1}{\lambda_ {g} ^ {*}} \mathbb {1} (g = h)\right) \\ \end{array}
$$
where the first equality stems from the definition of the Hessian, the second equality stems from  $H(\lambda^{*}) = 0$  due to the optimality of  $\lambda^{*}$ , the third equality stems from the definition of  $G_{g,h}$ , and the fourth equality stems from  $r_p(\lambda^*) = \Sigma_p^{\frac{1}{p} -1}$ . As a consequence,
$$
\begin{array}{l} \langle \mathcal {H} (\boldsymbol {\lambda} ^ {*}) (\boldsymbol {\lambda} - \boldsymbol {\lambda} ^ {*}), \boldsymbol {\lambda} - \boldsymbol {\lambda} ^ {*} \rangle = \sum_ {g, h \in [ G - 1 ]} \mathcal {H} _ {g, h} (\lambda - \lambda^ {*}) _ {g} (\lambda - \lambda^ {*}) _ {h} \\ = (p + 1) r _ {p} \left(\lambda^ {*}\right) \sum_ {g, h \in [ G - 1 ]} \left(\frac {1}{\lambda_ {G} ^ {*}} + \frac {1}{\lambda_ {g} ^ {*}} \mathbb {1} (g = h)\right) (\lambda - \lambda^ {*}) _ {g} (\lambda - \lambda^ {*}) _ {h} \\ = (p + 1) r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) \left\{\frac {\sum_ {g , h \in [ G - 1 ]} \left(\lambda - \lambda^ {*}\right) _ {g} \left(\lambda - \lambda^ {*}\right) _ {h}}{\lambda_ {G} ^ {*}} + \sum_ {g \in [ G - 1 ]} \frac {\left(\lambda - \lambda^ {*}\right) _ {g} ^ {2}}{\lambda_ {g} ^ {*}} \right\} \\ = (p + 1) r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) \left\{\frac {\left(\sum_ {g \in [ G - 1 ]} \left(\lambda - \lambda^ {*}\right) _ {g}\right) ^ {2}}{\lambda_ {G} ^ {*}} + \sum_ {g \in [ G - 1 ]} \frac {\left(\lambda - \lambda^ {*}\right) _ {g} ^ {2}}{\lambda_ {g} ^ {*}} \right\} \\ = (p + 1) r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) \left\{\frac {(\lambda - \lambda^ {*}) _ {G} ^ {2}}{\lambda_ {G} ^ {*}} + \sum_ {g \in [ G - 1 ]} \frac {(\lambda - \lambda^ {*}) _ {g} ^ {2}}{\lambda_ {g} ^ {*}} \right\} \\ = (p + 1) r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) \sum_ {g \in [ G ]} \frac {\left(\lambda_ {g} - \lambda_ {g} ^ {*}\right) ^ {2}}{\lambda_ {g} ^ {*}}, \\ \end{array}
$$
where the first step follows from the definition of the scalar product, and the second step stems from the expression of  $\mathcal{H}(\pmb{\lambda}^{*})$  derived previously in the proof. In the third step, the sum is distributed over the terms  $\frac{1}{\lambda_G^*}$  and  $\frac{1}{\lambda_g^*}\mathbb{1}(g = h)$ . In the fourth step, the first sum is factorized, and in the fifth step, the equality  $\sum_{g\in [G - 1]}(\lambda -\lambda^{*})_g = (1 - \lambda_G) - (1 - \lambda_G^*) = \lambda_G^* -\lambda_G$  is used. This completes the proof of Lemma 11.
Next, we establish the proof of Lemma 12.
Proof. From Lemma 1, the following equality holds for each  $g \in [G]$ :
$$
\sigma_ {g} ^ {2 p} = \left(\sum_ {h \in [ G ]} \sigma_ {h} ^ {\frac {2 p}{p + 1}}\right) ^ {p + 1} (\lambda_ {g} ^ {*}) ^ {p + 1} = \Sigma_ {p} ^ {p + 1} (\lambda_ {g} ^ {*}) ^ {p + 1}.
$$
For a fixed  $\pmb{u} \in [\pmb{\lambda}, \pmb{\lambda}^*]$ ,  $\sum_{g} u_{g} = \sum_{g} \lambda_{g} = \sum_{g} \lambda_{g}^{*} = 1$ , thus the coordinate  $g_{0}$  achieving the maximal  $\frac{\lambda_{g}^{*}}{u_{g}}$  must have  $u_{g_{0}} \leq \lambda_{g_{0}}^{*}$ . Since  $\pmb{u} \in [\pmb{\lambda}, \pmb{\lambda}^{*}]$  this implies that  $\lambda_{g_{0}} \leq u_{g_{0}} \leq \lambda_{g_{0}}^{*}$  and
consequently  $\max_g\frac{\lambda_g^*}{u_g}\leq \max_g\frac{\lambda_g^*}{\lambda_g}$ . The following upper bounds follow:
$$
\left| H _ {g} (\boldsymbol {u}) \right| = \Sigma_ {p} ^ {p + 1} \left| \left(\frac {\lambda_ {G} ^ {*}}{u _ {G}}\right) ^ {p + 1} - \left(\frac {\lambda_ {g} ^ {*}}{u _ {g}}\right) ^ {p + 1} \right| \leq \Sigma_ {p} ^ {p + 1} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 1},
$$
$$
| G _ {g, h} (\boldsymbol {u}) | = \Sigma_ {p} ^ {p + 1} \left| \frac {(\lambda_ {G} ^ {*}) ^ {p + 1}}{(u _ {G}) ^ {p + 2}} + \frac {(\lambda_ {g} ^ {*}) ^ {p + 1}}{(u _ {g}) ^ {p + 2}} \mathbb {1} (g = h) \right| \leq \frac {2 \Sigma_ {p} ^ {p + 1}}{\min _ {g} \lambda_ {g} ^ {*}} \left(\max _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 2},
$$
$$
| I _ {g, h, i} (\boldsymbol {u}) | = \Sigma_ {p} ^ {p + 1} \left| \frac {(\lambda_ {G} ^ {*}) ^ {p + 1}}{(u _ {G}) ^ {p + 3}} - \frac {(\lambda_ {g} ^ {*}) ^ {p + 1}}{(u _ {g}) ^ {p + 3}} \mathbb {1} (g = h = i) \right| \leq \frac {\Sigma_ {p} ^ {p + 1}}{(\min _ {g} \lambda_ {g} ^ {*}) ^ {2}} \left(\max _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 3}.
$$
Moreover, since  $p \geq 1$ , each  $j \in \{1 - p, 1 - 2p, 1 - 3p\}$  is non-positive, and by minimality of  $\lambda^*$ :
$$
r _ {p} (\boldsymbol {u}) ^ {j} \leq r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) ^ {j},
$$
so that:
$$
\left| H _ {g} H _ {h} H _ {i} r _ {p} ^ {1 - 3 p} (\boldsymbol {u}) \right| \leq \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {3 p + 3} \Sigma_ {p} ^ {3 p + 3} r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right) ^ {1 - 3 p},
$$
$$
\left| \left(H _ {g} G _ {h, i} + H _ {h} G _ {i, g} + H _ {g} G _ {h, i}\right) r _ {p} ^ {1 - 2 p} (\boldsymbol {u}) \right| \leq 3 \frac {2 \Sigma_ {p} ^ {p + 1}}{\min  _ {g} \lambda_ {g} ^ {*}} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 2} \left(\Sigma_ {p} ^ {p + 1} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 1}\right),
$$
$$
| I _ {g, h, i} r _ {p} ^ {1 - p} (\boldsymbol {u}) | \leq \frac {\sum_ {p} ^ {p + 1}}{(\min  _ {g} \lambda_ {g} ^ {*}) ^ {2}} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 3} r _ {p} ^ {1 - p} (\boldsymbol {\lambda} ^ {*}).
$$
Each of the previous three expressions can be simplified by using  $r_p(\boldsymbol{\lambda}^*) = \sum_{p}^{\frac{1}{p} - 1}$ :
$$
| H _ {g} H _ {h} H _ {i} r _ {p} ^ {1 - 3 p} (\boldsymbol {u}) | \leq \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {3 p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}),
$$
$$
\left| \left(H _ {g} G _ {h, i} + H _ {h} G _ {i, g} + H _ {g} G _ {h, i}\right) r _ {p} ^ {1 - 2 p} (\boldsymbol {u}) \right| \leq \frac {6}{\min _ {g} \lambda_ {g} ^ {*}} \left(\max _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {2 p + 3} r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right),
$$
$$
\left| I _ {g, h, i} r _ {p} ^ {1 - p} (\boldsymbol {u}) \right| \leq \frac {1}{(\min  _ {g} \lambda_ {g} ^ {*}) ^ {2}} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 3} r _ {p} \left(\boldsymbol {\lambda} ^ {*}\right).
$$
Hence by using the expression of the third derivatives established in Lemma 13:
$$
\begin{array}{l} \left| \frac {\partial^ {3} r _ {p} (\boldsymbol {u})}{\partial \lambda_ {g} \partial \lambda_ {h} \partial \lambda_ {i}} \right| \leq \left| (1 - p) (1 - 2 p) H _ {g} H _ {h} H _ {i} r _ {p} ^ {1 - 3 p} (\boldsymbol {u}) \right| \\ + \left| (1 - 2 p) (1 + p) \left(H _ {g} G _ {h, i} + H _ {h} G _ {i, g} + H _ {g} G _ {h, i}\right) r _ {p} ^ {1 - 2 p (\boldsymbol {u})} \right| \\ + \left| (p + 1) (p + 2) I _ {g, h, i} r _ {p} ^ {1 - p} (\boldsymbol {u}) \right| \\ \leq 2 p ^ {2} \left(\max _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {3 p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}) + \frac {1 2 (p + 1) ^ {2}}{\min _ {g} \lambda_ {g} ^ {*}} \left(\max _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {2 p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}) \\ + \frac {(p + 2) ^ {2}}{(\min _ {g} \lambda_ {g} ^ {*}) ^ {2}} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}) \\ \leq \frac {7 (p + 2) ^ {2}}{(\min  _ {g} \lambda_ {g} ^ {*}) ^ {2}} \left(\max  _ {g} \frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {3 p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}). \\ \end{array}
$$
As a consequence, by taking the sup over  $u,g,h,i,x,y,z$
$$
\sup  _ { \begin{array}{l} u \in [ \lambda , \lambda^ {*} ] \\ x, y, z \in \mathbb {N} \\ x + y + z = 3 \\ \{g, h, i \} \subset [ G - 1 ] \end{array} } \left| \frac {1}{x ! y ! z !} \frac {\partial^ {3} r _ {p} (\boldsymbol {u})}{\partial \lambda_ {g} \partial \lambda_ {h} \partial \lambda_ {i}} \right| \leq \frac {7 (p + 2) ^ {2}}{\left(\min  _ {g} \lambda_ {g} ^ {*}\right) ^ {2}} \max  _ {g} \left(\frac {\lambda_ {g} ^ {*}}{\lambda_ {g}}\right) ^ {3 p + 3} r _ {p} (\boldsymbol {\lambda} ^ {*}), \tag {27}
$$
which completes the proof of Lemma 12.
B Proof of Theorem 2
In this section, we prove Theorem 2. First, we establish an initial lower bound that captures the trade-off between how hard it is to distinguish two instances and how hard it is to optimize both under the same action (See Appendix B.1). Next, we provide a specific counter example which regret is at least  $\Theta(T^{-2})$  (See Appendix B.2).
B.1 Proof of Lemma 6
Proof. Let  $\mathcal{D}^a$  and  $\mathcal{D}^b$  be two instances with standard deviations  $\sigma^a$  and  $\sigma^b$ , and let  $\delta \geq d(\sigma^a, \sigma^b)$ . Let  $X$  be a random variable over the set  $\{a, b\}$ . The following inequalities hold
$\max (\operatorname{Regret}_{p,T}(\pi, \mathcal{D}^a), \operatorname{Regret}_{p,T}(\pi, \mathcal{D}^b)) \geq \mathbb{E}_X[\operatorname{Regret}_p(\pi, \mathcal{D}^X)]$
$$
\begin{array}{l} \geq \mathbb {E} \left[ R _ {p} \left(\boldsymbol {n}; \boldsymbol {\sigma} ^ {X}\right) - R _ {p} ^ {*} \left(\boldsymbol {\sigma} ^ {X}\right) \mid R _ {p} \left(\boldsymbol {n}; \boldsymbol {\sigma} ^ {X}\right) - R _ {p} ^ {*} \left(\boldsymbol {\sigma} ^ {X}\right) > \delta \right] \\ \times \mathbb {P} _ {\boldsymbol {\pi}, X} \left(R _ {p} (\boldsymbol {n}; \boldsymbol {\sigma} ^ {X}) - R _ {p} ^ {*} (\boldsymbol {\sigma} ^ {X}) > \delta\right) \\ \geq \delta \mathbb {P} _ {X, \boldsymbol {\pi}} \left(R _ {p} (\boldsymbol {n}; \boldsymbol {\sigma} ^ {X}) - R _ {p} ^ {*} (\boldsymbol {\sigma} ^ {X}) > \delta\right), \\ \end{array}
$$
where the first step stems from the support of  $X$  being  $\{a, b\}$ , and the second step stems from the law of total expectation. Let  $\hat{x}$  be the following (random) classifier:
$$
\hat {x} := \left\{ \begin{array}{l l} a & \text {I f} R _ {p} (\boldsymbol {n}; \boldsymbol {\sigma} ^ {a}) - R _ {p} ^ {*} (\boldsymbol {\sigma} ^ {a}) \leq \delta \\ b & \text {I f} R _ {p} (\boldsymbol {n}, \boldsymbol {\sigma} ^ {b}) - R _ {p} ^ {*} (\boldsymbol {\sigma} ^ {b}) \leq \delta \\ \text {I n d i f f e r e n t} & \text {O t h e r w i s e} \end{array} \right.
$$
Since  $\delta \geq d(\sigma^a,\sigma^b)$ ,  $\hat{x}$  is well defined. Moreover,  $\mathbb{P}_{\pi .X}\left(R_p(\pmb {n};\pmb{\sigma}^X) - R_p^* (\pmb {\sigma}^X) > \delta\right)\geq \mathbb{P}_X(\hat{x}\neq X)\geq \inf_{\hat{x}}\mathbb{P}_X(\hat{x}\neq X)$ , where the infimum is taken over all the classifiers of  $\{a,b\}$ . Moreover, by Pinsker's inequality,  $\inf_{\hat{x}}\mathbb{P}_X(\hat{x}\neq X)\geq \exp \left(-KL(\mathcal{D}^a ||\mathcal{D}^b)\right)$ . Moreover, from bandits feedback divergence properties (see [25]),  $KL(\mathcal{D}^a ||\mathcal{D}^b) = \sum_{g\in [G]}\mathbb{E}_{\pi ,\mathcal{D}^a}[n_{g,t}]KL(\mathcal{D}_g^a ||\mathcal{D}_g^b)$ , which concludes the proof of Lemma 6.
B.2 The counter-examples
We introduce the following instances for each  $g\in [G]$
$$
\mathcal {D} ^ {g}: \left\{ \begin{array}{l} \mathcal {D} _ {1} ^ {a} \sim \mathcal {N} \left(0, 1 + \frac {1}{\sqrt {T}}\right) \\ \mathcal {D} _ {g} ^ {a} \sim \mathcal {N} \left(0, 1\right), \quad \forall g \neq 1 \end{array} \right.
$$
$$
\boldsymbol {\mathcal {D}} ^ {b}: \left\{ \begin{array}{l} \mathcal {D} _ {2} ^ {b} \sim \mathcal {N} \left(0, 1 + \frac {1}{\sqrt {T}}\right) \\ \mathcal {D} _ {g} ^ {a} \sim \mathcal {N} \left(0, 1\right), \quad \forall g \neq 2 \end{array} \right.
$$
We start by upper bounding the  $KL$ -divergence between the two instances:
Lemma 14. The following inequality holds:  $\sum_{g\in [G]}\mathbb{E}_{\pi ,\mathcal{D}^a}[n_{g,T}]KL(\mathcal{D}_g^a ||\mathcal{D}_g^b)\leq \frac{1}{2}$
Proof. For convenience, we set  $\nu := \sqrt{\frac{1}{T}} < 1$ . The formula for the  $KL$ -divergence of two univariate normal distributions of zero mean implies
$$
K L (\mathcal {D} _ {h} ^ {a} | | \mathcal {D} _ {h} ^ {b}) = \frac {1}{2} \left(\log \left(\frac {\sigma^ {2} (1 + \nu)}{\sigma^ {2}}\right) + \frac {\sigma^ {2} - (\sigma^ {2} (1 + \nu))}{\sigma^ {2} (1 + \nu)}\right).
$$
The taylor expansion of the expression above can be derived by combining the expansions of both the functions  $x \to \log (1 + x)$  and  $x \to \frac{1}{1 + x}$  in the domain  $(0, 1)$ :
$$
\begin{array}{l} \frac {1}{2} \left(\log \left(\frac {\sigma^ {2} (1 + \nu)}{\sigma^ {2}}\right) + \frac {\sigma^ {2} - (\sigma^ {2} (1 + \nu))}{\sigma^ {2} (1 + \nu)}\right) = \frac {1}{2} \left(- \sum_ {k \geq 1} \frac {(- 1) ^ {k}}{k} \nu^ {k} - \nu \sum_ {k \geq 0} (- 1) ^ {k} \nu^ {k}\right) \\ = \frac {1}{2} \sum_ {k \geq 1} (- 1) ^ {k} \nu^ {k} \left(1 - \frac {1}{k}\right) \\ = \frac {\nu^ {2}}{2} \sum_ {k \geq 0} (- 1) ^ {k} \nu^ {k} \left(1 - \frac {1}{k + 2}\right) \\ \leq \frac {\nu^ {2}}{2}, \\ \end{array}
$$
Summing over all coordinates implies
$$
\sum_ {g \in [ G ]} \mathbb {E} _ {\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} ^ {a} \left[ n _ {g, T} \right]} K L (\mathcal {D} _ {g} ^ {a} | | \mathcal {D} _ {g} ^ {b}) = \mathbb {E} _ {\boldsymbol {\pi}, \boldsymbol {\mathcal {D}} ^ {a} \left[ n _ {h, T} \right]} K L (\mathcal {D} _ {h} ^ {a} | | \mathcal {D} _ {h} ^ {b}) \leq \frac {T \nu^ {2}}{2} = \frac {1}{2},
$$
where the inequality follows from  $\pmb{n}_T \leq T$ . This concludes the proof.
Next, we derive a simpler form for  $d(\sigma^a,\sigma^b)$ . The simplification exploits the symmetries in  $\sigma^a,\sigma^b$ :
Lemma 15. Let  $\mathbf{u}$  denote the unit vector  $(1,\dots ,1)^T$ . The following equality holds:
$$
d (\pmb {\sigma} ^ {a}, \pmb {\sigma} ^ {b}) = r _ {p} \left(\frac {1}{G} \pmb {u}; \pmb {\sigma} ^ {a}\right) - r _ {p} ^ {*} (\pmb {\sigma} ^ {a})
$$
where the function  $r_p$  is introduced in Appendix A.
Proof. For  $x \in \{a, b\}$ , let  $S_{\epsilon}^{x} \coloneqq \{\epsilon > 0 | r_{p}(\lambda; \sigma^{x}) - r_{p}^{*}(\sigma^{x}) \leq \epsilon\}$ . By definition of  $d$ ,
$$
d \left(\boldsymbol {\sigma} ^ {a}, \boldsymbol {\sigma} ^ {b}\right) = \inf  \left\{\delta \geq 0 \mid \mathcal {S} _ {\epsilon} ^ {a} \cap \mathcal {S} _ {\epsilon} ^ {b} \neq \emptyset \right\}.
$$
We prove  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) = r_p\left(\frac{1}{G}\pmb{u}; \pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$  by proving each of the inequalities  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) \leq r_p\left(\frac{1}{G}\pmb{u}; \pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$  and  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) \geq r_p\left(\frac{1}{G}\pmb{u}; \pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$ .
First, we prove  $d(\pmb{\sigma}^a,\pmb{\sigma}^b) \leq r_p\left(\frac{1}{G}\pmb{u};\pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$ . Since  $\pmb{\sigma}^a$  can be obtained by swapping the first two coordinates of  $\pmb{\sigma}^b$ , the symmetry of  $r_p$  implies  $r_p^*(\pmb{\sigma}^a) = r_p^*(\pmb{\sigma}^b)$ . Moreover, for each  $(\lambda_1,\lambda_2,\lambda') \in \mathcal{K}$ ,  $r_p((\lambda_1,\lambda_2,\lambda_{3:G});\pmb{\sigma}^a) = r_p((\lambda_2,\lambda_1,\lambda_{3:G});\pmb{\sigma}^b)$ . As a consequence,  $r_p\left(\frac{1}{G}\pmb{u};\pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a) = r_p\left(\frac{1}{G}\pmb{u};\pmb{\sigma}^b\right) - r_p^*(\pmb{\sigma}^b)$ , and any  $\epsilon > 0$  satisfying  $\epsilon \geq r_p\left(\frac{1}{G}\pmb{u};\pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$  must also satisfy  $S_\epsilon^a \cap S_\epsilon^b \neq \emptyset$ . In particular,  $d(\pmb{\sigma}^a,\pmb{\sigma}^b) \leq r_p\left(\frac{1}{G}\pmb{u};\pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$ .
To derive  $d(\pmb{\sigma}^a, \pmb{\sigma}^b) \geq r_p\left(\frac{1}{G}\pmb{u}; \pmb{\sigma}^a\right) - r_p^*(\pmb{\sigma}^a)$ , we use the following lemma, which proof is deferred later in the section:
Lemma 16. If  $\lambda = (\lambda_1, \dots, \lambda_G) \in S_\epsilon^a \cap S_\epsilon^b$ , and  $\tau$  a permutation of  $[G]$ .  $\lambda_\tau := (\lambda_{\tau(1)}, \dots, \lambda_{\tau(G)})$  is also in  $S_\epsilon^a \cap S_\epsilon^b$ .
Let  $\epsilon \geq 0$  satisfying  $S_{\epsilon}^{a}\cap S_{\epsilon}^{b}\neq \emptyset$ , and let  $\lambda \in S_{\epsilon}^{a}\cap S_{\epsilon}^{b}$ . For each permutation  $\tau$ ,  $\lambda_{\tau}$  is also in  $S_{\epsilon}^{a}\cap S_{\epsilon}^{b}$ . Each of  $S_{\epsilon}^{a}$  and  $S_{\epsilon}^{b}$  is convex and therefore the intersection  $S_{\epsilon}^{a}\cap S_{\epsilon}^{b}$  is also convex, which implies that
$$
\frac {1}{G} \boldsymbol {u} = \frac {1}{G !} \sum_ {\tau \text {p e r m u t a t i o n}} \lambda_ {\tau} \in \mathcal {S} _ {\epsilon} ^ {a} \cap \mathcal {S} _ {\epsilon} ^ {b},
$$
which in turn implies that  $\epsilon \geq r_p\left(\frac{1}{G}\mathbf{u},\pmb{\sigma}^a\right) - r^* (\sigma^a)$ . By taking the inf we get  $d_{p}(\pmb{\sigma}^{a},\pmb{\sigma}^{b})\geq$ $r_p\left(\frac{1}{G}\pmb {u},\pmb {\sigma}^a\right) - r_p^* (\pmb {\sigma}^a)$ , which completes the proof.
We now state the proof of Lemma 16:
Proof. Let  $\lambda = (\lambda_1, \dots, \lambda_G) \in S_\epsilon^a \cap S_\epsilon^b$ . Since every permutation can be written as a composition of transpositions (2-cycles), it suffices to prove the result for transpositions. Let  $\tau = (g, h)$  with  $g \neq h$ . We distinguish 3 cases:
1.  $g, h \geq 3$ : Since  $\sigma_{3:G}^a = \sigma_{3:G}^b$ ,  $r_p(\lambda; \sigma^x) = r_p(\lambda_\tau; \sigma^x)$  for each  $x \in \{a, b\}$ . The implication  $\lambda_\tau \in S_\epsilon^a \cap S_\epsilon^b$  follows.
2.  $g, h \leq 2$ : By symmetry of the problem,  $\sigma_1^a = \sigma_2^b = \sigma_{\tau(1)}^b$ . Similarly,  $\sigma_2^a = \sigma_{\tau(2)}^b$ . Therefore,  $\lambda \in S_\epsilon^a$  implies  $\lambda_\tau \in S_\epsilon^b$ . Similarly,  $\lambda \in S_\epsilon^b$  implies  $\lambda_\tau \in S_\epsilon^a$ . The implication  $\lambda_\tau \in S_\epsilon^a \cap S_\epsilon^b$  follows.
3.  $g \in \{1,2\}$  and  $h \geq 3$ . Without loss of generality, assume that  $g = 1$  and  $h = 3$ . Since  $\mathcal{D}_1^b \sim \mathcal{D}_3^b$ , the equality  $\sigma_1^b = \sigma_3^b$  holds and therefore  $r_p(\pmb{\lambda};\pmb{\sigma}^b) = r_p(\pmb{\lambda}_{\tau};\pmb{\sigma}^b)$ , therefore  $\pmb{\lambda}_{\tau} \in S_{\epsilon}^{b}$ . Moreover,  $r_p(\pmb{\lambda}_{\tau};\pmb{\sigma}^a) = r_p(\pmb{\lambda};\pmb{\sigma}^b)$ , therefore  $\pmb{\lambda}_{\tau} \in S_{\epsilon}^{a}$ . The implication  $\pmb{\lambda}_{\tau} \in S_{\epsilon}^{a} \cap S_{\epsilon}^{b}$  follows.
The three cases cover all possible 2-cycles, which completes the proof.
We are now ready to state the proof of Theorem 2. It remains to show that  $d_p(\pmb{\sigma}^a, \pmb{\sigma}^b) = r_p\left(\frac{1}{2}\mathbf{u}, \sigma^a\right) - r_p^*(\sigma^a) = \Theta(T^{-2})$ . We are ready to state the proof of Theorem 2.
Theorem 2. Let  $p$  be finite and  $\kappa$  be a universal constant. For any online policy  $\pi$ , there exists an instance  $\mathcal{D}_{\pi}$  such that for any  $T \geq 1$ ,
Proof. We set  $\mathcal{D}_{\pi} := \operatorname{argmax}\left\{\mathrm{Regret}_p(\pi, \mathcal{D}^a), \mathrm{Regret}_p(\pi, \mathcal{D}^b)\right\}$ . The following inequalities hold: