SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {1 / 2} \frac {\mathbb {E} ^ {t} \left[ \sum_ {i \in U} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2} \right]}{\gamma^ {t} \left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {3 / 2}} \\ \leq \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {1 / 2} \frac {\left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {2}\right) \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right)}{\gamma^ {t} \left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {\frac {3}{2}}} (58) \\ \leq \frac {1}{\gamma^ {t}} \sqrt {\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}} \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) (59) \\ \leq \frac {1}{\gamma^ {t}} \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) (60) \\ \leq \frac {K ^ {1 / 6}}{\sqrt {t}} \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right), (61) \\ \end{array}
$$
where Eq. (58) follows from
$$
\mathbb {E} ^ {t} \left[ \sum_ {i \in U} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2} \right] = \mathbb {E} ^ {t} \left[ \sum_ {i \in U} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \frac {\mathbb {I} \{j ^ {t} = i \} \ell_ {i} ^ {t}}{g _ {i} ^ {t}}\right) ^ {2} \right] \tag {byEq.(35)}
$$
$$
\begin{array}{l} \leq \sum_ {i \in U} \frac {\left(p _ {i} ^ {t}\right) ^ {8 / 3}}{g _ {i} ^ {t}} \quad (\ell_ {i} ^ {t} \leq 1 \text {a n d} \mathbb {E} ^ {t} [ \mathbb {I} \{j ^ {t} = i \} ] = g _ {i} ^ {t}) \\ = \left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {2}\right) \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right); \tag {byEq.(35)} \\ \end{array}
$$
Eq. (59) and Eq. (60) use the following, respectively:
$$
\left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {3 / 2} \geq \left(\sum_ {i \in U} \left(p _ {i} ^ {t}\right) ^ {2}\right) \text {s i n c e} \| x \| _ {2} \leq \| x \| _ {\frac {4}{3}}, \quad \text {a n d} \quad \sqrt {\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}} \leq \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right);
$$
and Eq. (61) holds by the facts  $\sum_{i\in [K]} \left(p_i^t\right)^{2/3} \leq K^{1/3}$  and  $\gamma^t = K^{1/6}\sqrt{t}$ . Thus, the cumulative regret of the term in Eq. (55) from round  $T_0 + 1$  to round  $T$  can be bounded as:
$$
\mathbb {E} \left[ \sum_ {t = T _ {0} + 1} ^ {T} \frac {\left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right)}{\gamma^ {t} \sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \lambda_ {U} ^ {2} \mathbb {I} \left\{\lambda_ {U} \geq 0 \right\} \right] \leq \mathcal {O} \left(\mathbb {E} \left[ \sum_ {t = T _ {0} + 1} ^ {T} \frac {K ^ {1 / 6}}{\sqrt {t}} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3} \right]\right). \tag {62}
$$
Bounding Eq. (56) For this term, we have
$$
\begin{array}{l} \mathbb {E} ^ {t} \left[ \frac {\left(\gamma^ {t + 1} - \gamma^ {t}\right) ^ {2}}{\gamma^ {t}} \frac {\left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}\right) ^ {2}}{\left(\sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right)} \right] \\ \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6}}{t ^ {3 / 2}} \frac {\sqrt {\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \left(\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}\right) ^ {2}}{\left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) ^ {3 / 2}} \right]\right) \\ \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6}}{t ^ {3 / 2}} \frac {\sqrt {\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}} \left(\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}\right) ^ {2}}{\left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) ^ {3 / 2}} \right]\right) \\ \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6}}{t ^ {3 / 2}} \frac {\left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) \left(\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}\right) ^ {2}}{\left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) ^ {3 / 2}} \right]\right) \\ = \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}}{t ^ {3 / 2} \left(\sum_ {i \in U} \left(\frac {\left(\bar {p} _ {i} ^ {t + 1}\right)}{\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}}\right) ^ {4 / 3}\right) ^ {3 / 2}} \right]\right), \\ \end{array}
$$
where the first step follows from  $\gamma^{t + 1} - \gamma^t\leq \frac{K^{1 / 6}}{\sqrt{t}}$ $\gamma^t = K^{1 / 6}\sqrt{t}$ , and Eq. (57); the second step applies Corollary B.5.2 to obtain the multiplicative relation; the third step applies  $\sqrt{\sum_{i\in V}(p_i^t)^{4 / 3}}\leq \sum_{i\in V}(p_i^t)^{2 / 3}$ ; the last step divides  $\left(\sum_{i\in U}\vec{p}_i^{t + 1}\right)^2 = \left(\left(\sum_{i\in U}\vec{p}_i^{t + 1}\right)^{4 / 3}\right)^{3 / 2}$  for both the numerator and the denominator. By using the bound above, we have
$$
\text {E q .} (5 6) \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}}{t ^ {3 / 2} \left(\sum_ {i \in U} \left(\frac {\left(\bar {p} _ {i} ^ {t + 1}\right)}{\sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}}\right) ^ {4 / 3}\right) ^ {3 / 2}} \right]\right)
$$
$$
\leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {2 / 3} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}}{t ^ {3 / 2}} \right]\right) \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {K ^ {1 / 6} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}}{\sqrt {t}} \right]\right),
$$
where the second step uses  $\sum_{i\in U}\left(\frac{p_i^t}{\sum_{j\in U}p_j^t}\right)^{4 / 3}\geq \frac{1}{|U|^{1 / 3}}\geq \frac{1}{K^{1 / 3}}$  and the last step follows from  $T_0\geq K\geq \sqrt{K}$ . Thus, the cumulative regret of this part, starting from  $t = T_0 + 1$  to  $t = T$ , can be bounded by
$$
\mathcal {O} \left(\mathbb {E} \left[ \sum_ {t = T _ {0} + 1} ^ {T} \frac {K ^ {1 / 6} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}}{\sqrt {t}} \right]\right). \tag {63}
$$
Bounding the  $\lambda_{V}$ -related term We now consider the regret of Eq. (54) associated with  $\lambda_{V}$ . Note that as  $\lambda_{V} \geq 0$  always holds, we do not need to consider two cases. Repeating the argument used to bound Eq. (55), we have
$$
\begin{array}{l} \mathbb {E} ^ {t} \left[ \frac {\left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right)}{\gamma^ {t} \sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \lambda_ {V} ^ {2} \right] \\ \leq \mathbb {E} ^ {t} \left[ \frac {\left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right)}{\gamma^ {t} \sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \frac {\sum_ {i \in V} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2}}{\left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {2}} \right] \\ \leq \mathbb {E} ^ {t} \left[ \left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \frac {\sum_ {i \in V} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2}}{\gamma^ {t} \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}\right) ^ {2}} \right] \leq \mathcal {O} \left(\mathbb {E} ^ {t} \left[ \frac {\sum_ {i \in V} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2}}{\gamma^ {t} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}} \right]\right), \\ \end{array}
$$
where the first step applies the upper bound of  $\lambda_{V}$  and uses the fact that  $\widehat{\ell}_i^t\cdot \widehat{\ell}_j^t = 0$  for any  $i\neq j$  together with the definition of  $\widehat{\ell}_i^t$ ; the second step bounds the fraction  $\sum_{i\in U}\left(\bar{p}_i^{t + 1}\right)^{4 / 3} / \sum_{i\in [K]}\left(\bar{p}_i^{t + 1}\right)^{4 / 3}$  by one; the last step applies Corollary B.5.2 to obtain the multiplicative relation. By a similar argument used for  $\lambda_U$ , we have
$$
\begin{array}{l} \mathbb {E} ^ {t} \left[ \frac {\sum_ {i \in V} \left(\left(p _ {i} ^ {t}\right) ^ {4 / 3} \widehat {\ell} _ {i} ^ {t}\right) ^ {2}}{\gamma^ {t} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}} \right] \leq \mathcal {O} \left(\frac {\left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2}\right) \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right)}{\gamma^ {t} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {4 / 3}}\right) \\ \leq \mathcal {O} \left(\frac {K ^ {1 / 6}}{\sqrt {t}} \left(\sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right)\right), \tag {64} \\ \end{array}
$$
where the last step uses  $\sum_{i\in V}\left(p_i^t\right)^2\leq \left(\sum_{i\in V}\left(p_i^t\right)^{2 / 3}\right)\left(\sum_{i\in V}\left(p_i^t\right)^{4 / 3}\right),\sum_{i\in [K]}\left(p_i^t\right)^{2 / 3}\leq K^{1 / 3},$  and  $\gamma^t = K^{1 / 6}\sqrt{t}$ . To conclude, the cumulative regret of the term related to  $\lambda_V$  is bounded as
$$
\mathbb {E} \left[ \sum_ {t = T _ {0} + 1} ^ {T} \frac {\left(\sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right) \left(\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}\right)}{\gamma^ {t} \sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \lambda_ {V} ^ {2} \right] \leq \mathcal {O} \left(\mathbb {E} \left[ \sum_ {t = T _ {0} + 1} ^ {T} \frac {K ^ {1 / 6}}{\sqrt {t}} \sum_ {i \in V} \left(p _ {i} ^ {t}\right) ^ {2 / 3} \right]\right). \tag {65}
$$
Finally, combining Eq. (62), Eq. (63), and Eq. (65) yields the result of Lemma B.4.1.
B.5 Auxiliary Lemmas for Analysis of Decoupled-Tsallis-INF
Since the regularizer of Decoupled-Tsallis-INF does not have an extra log-barrier, we cannot expect an entry-wise multiplicative relation as used in our MAB analysis. Hence, we follow Lemma 22 and Lemma 28 in [Ito, 2021] to show group multiplicative relation for  $\frac{2}{3}$ -Tsallis entropy.
Lemma B.5.1. For any given  $t$  and index set  $\mathcal{I} \subseteq [K]$ , if  $x, y \in [0, 1]^K$  satisfies  $\sum_{i \in \mathcal{I}} x_i = \sum_{i \in \mathcal{I}} y_i$  and for  $2/3$ -Tsallis entropy  $\phi$  and  $c \in \mathbb{R}$ , we have
$$
\phi_ {\mathcal {I}} ^ {t + 1} (y) = \phi_ {\mathcal {I}} ^ {t} (x) - \widehat {\ell} _ {\mathcal {I}} ^ {t} + c \cdot \mathbf {1} _ {\mathcal {I}}, \tag {66}
$$
then, the following holds.
$$
\sum_ {i \in \mathcal {I}} \left(y _ {i}\right) ^ {4 / 3} \leq 2 \left(1 + \frac {1}{t}\right) ^ {2} \sum_ {i \in \mathcal {I}} \left(x _ {i}\right) ^ {4 / 3} \leq 8 \sum_ {i \in \mathcal {I}} \left(x _ {i}\right) ^ {4 / 3}. \tag {67}
$$
Proof. From Eq. (66), we have
$$
\frac {3 \sqrt {t + 1}}{K ^ {- 1 / 6}} \left(y _ {i}\right) ^ {- 1 / 3} = \frac {3 \sqrt {t}}{K ^ {- 1 / 6}} \left(x _ {i}\right) ^ {- 1 / 3} + \widehat {\ell} _ {i} ^ {t} - c,
$$
$$
\sqrt {t + 1} \left(y _ {i}\right) ^ {- 1 / 3} = \sqrt {t} \left(x _ {i}\right) ^ {- 1 / 3} + \frac {K ^ {- 1 / 6}}{3} \left(\widehat {\ell} _ {i} ^ {t} - c\right).
$$
If  $c\leq 0$  , then, we have
$$
\sqrt {t + 1} \left(y _ {i}\right) ^ {- 1 / 3} \geq \sqrt {t} \left(x _ {i}\right) ^ {- 1 / 3}, \text {w h i c h i m p l i e s} \left(y _ {i}\right) ^ {4 / 3} \leq \left(\frac {t + 1}{t}\right) ^ {2} \left(x _ {i}\right) ^ {4 / 3}.
$$
Since the above holds for every  $i \in \mathcal{I}$ , the desired claim is immediate. Now, we consider  $c \geq 0$ . For all  $i \in \mathcal{I} \setminus \{i^t\}$ , we have  $\sqrt{t + 1}(y_i)^{-1/3} \leq \sqrt{t}(x_i)^{-1/3}$ , which gives  $\frac{(x_i)^{1/3}}{\sqrt{t}} \leq \frac{(y_i)^{1/3}}{\sqrt{t + 1}}$ . Rearranging it, we arrive at
$$
\frac {y _ {i}}{(t + 1) ^ {3 / 2}} \geq \frac {x _ {i}}{(t) ^ {3 / 2}}.
$$
$$
z _ {i} ^ {t} = \frac {x _ {i}}{\sum_ {i \in \mathcal {I}} x _ {i}}, \quad \text {a n d} \quad \bar {z} _ {i} ^ {t + 1} = \left(\frac {t}{t + 1}\right) ^ {3 / 2} \frac {y _ {i}}{\sum_ {i \in \mathcal {I}} y _ {i}}.
$$
By  $\overline{z}_i^{t + 1}\geq z_i^t$  for all  $i\in \mathcal{I}\backslash \{i^t\}$  and  $\sum_{i\in \mathcal{I}}\overline{z}_i^{t + 1}\leq \sum_{i\in \mathcal{I}}z_i^t = 1$  , we can show
$$
\frac {\sum_ {i \in \mathcal {I}} \left(y _ {i}\right) ^ {4 / 3}}{\sum_ {i \in \mathcal {I}} \left(x _ {i}\right) ^ {4 / 3}} = \left(\frac {t + 1}{t}\right) ^ {2} \frac {\sum_ {i \in \mathcal {I}} \left(\bar {z} _ {i} ^ {t + 1}\right) ^ {4 / 3}}{\sum_ {i \in \mathcal {I}} \left(z _ {i} ^ {t}\right) ^ {4 / 3}} \leq 2 \left(\frac {t + 1}{t}\right) ^ {2},
$$
where the last step uses Lemma 28 in [Ito, 2021] by replacing the power  $3/2$  by  $4/3$ .
Recall that  $\nabla \phi_U^{t + 1}(\bar{p}^{t + 1}) = \nabla \phi_U^t (p^t) - \widehat{\ell}_U^t +\lambda_U\cdot \mathbf{1}_U$ , which implies that Eq. (66) and  $\sum_{i\in \mathcal{I}}x_i = \sum_{i\in \mathcal{I}}y_i$  in Lemma B.5.1 hold by applying  $\mathcal{I} = U$ ,  $y = \bar{p}^{t + 1}$ , and  $x = p^t$ . Hence, we have  $\sum_{i\in U}\left(\bar{p}_i^{t + 1}\right)^{4 / 3}\leq 8\sum_{i\in U}\left(p_i^t\right)^{4 / 3}$ . By repeating the same reasoning in Lemma B.5.1 and changing  $\gamma^{t + 1}$  to  $\gamma^t$ , the following corollary is immediate.
Corollary B.5.2. For any given  $t$  and index set  $\mathcal{I} \subseteq [K]$ , if  $x, y \in [0,1]^K$  satisfies  $\sum_{i \in \mathcal{I}} x_i = \sum_{i \in \mathcal{I}} y_i$  and for  $2/3$ -Tsallis entropy  $\phi$  and  $c \in \mathbb{R}$ , we have  $\phi_{\mathcal{I}}^t(y) = \phi_{\mathcal{I}}^t(x) - \widehat{\ell}_{\mathcal{I}}^t + c \cdot \mathbf{1}_{\mathcal{I}}$ , then,  $\sum_{i \in \mathcal{I}} (y_i)^{4/3} \leq 8 \sum_{i \in \mathcal{I}} (x_i)^{4/3}$ .
Recall that as  $\bar{p}_V^{t + 1}$  is computed via  $\phi^t$  rather than  $\phi^{t + 1}$  and  $\nabla \phi_V^t (\bar{p}^{t + 1}) = \nabla \phi_V^t (p^t) - \widehat{\ell}_V^t +\lambda_V\cdot \mathbf{1}_V$  Corollary B.5.2 implies the multiplicative relation:  $\sum_{i\in V}\left(\bar{p}_i^{t + 1}\right)^{4 / 3}\leq 8\sum_{i\in V}\left(p_i^t\right)^{4 / 3}.$
Next, we present a lemma that is used to bound the regret for the first  $T_0$  rounds by simply  $\mathcal{O}(\sqrt{KT_0})$ .
Lemma B.5.3. For  $q^t$  defined in Eq. (5),  $p^t$  defined in Algorithm 1, and any  $t \in [T]$ , we have
$$
\mathbb {E} \left[ D ^ {t, t + 1} \left(p ^ {t}, p ^ {t + 1}\right) - D _ {U} ^ {t, t + 1} \left(q ^ {t}, q ^ {t + 1}\right) \right] \leq \mathcal {O} \left(\frac {\sqrt {K}}{\sqrt {t}}\right). \tag {68}
$$
Proof. By similar arguments of [Ito, 2021, Lemma 24], we arrive at
$$
D ^ {t, t + 1} \left(p ^ {t}, p ^ {t + 1}\right) - D _ {U} ^ {t, t + 1} \left(q ^ {t}, q ^ {t + 1}\right) \leq D ^ {t, t + 1} \left(p ^ {t}, z ^ {t + 1}\right),
$$
where  $z^{t + 1}\in \mathbb{R}^K$  is the unconstrained projection such that  $\nabla \phi^{t + 1}(z^{t + 1}) = \nabla \phi^t (p^t) - \widehat{\ell}^t$  (see more details in [Ito, 2021]). Then, we have for any  $i\in [K]$  and any  $t$ ,
$$
\frac {K ^ {1 / 6} \sqrt {t + 1}}{\left(z _ {i} ^ {t + 1}\right) ^ {1 / 3}} = \frac {K ^ {1 / 6} \sqrt {t}}{\left(p _ {i} ^ {t}\right) ^ {1 / 3}} + \widehat {\ell} _ {i} ^ {t} \geq \frac {K ^ {1 / 6} \sqrt {t}}{\left(p _ {i} ^ {t}\right) ^ {1 / 3}},
$$
which implies that for any  $i \in [K]$  and any  $t$ ,
$$
\left(z _ {i} ^ {t + 1}\right) ^ {2 / 3} \leq \left(\frac {1}{t} + 1\right) \left(p _ {i} ^ {t}\right) ^ {2 / 3} \leq 2 \left(p _ {i} ^ {t}\right) ^ {2 / 3}. \tag {69}
$$
By using  $\nabla \phi^{t + 1}(z^{t + 1}) = \nabla \phi^t (p^t) - \widehat{\ell}^t$ , we add and subtract  $\phi^t (z^{t + 1})$  to show
$$
\begin{array}{l} D ^ {t, t + 1} \left(p ^ {t}, z ^ {t + 1}\right) = \phi^ {t} \left(p ^ {t}\right) - \phi^ {t + 1} \left(z ^ {t + 1}\right) - \left\langle \nabla \phi^ {t + 1} \left(z ^ {t + 1}\right), p ^ {t} - z ^ {t + 1} \right\rangle + \left(\phi^ {t} \left(z ^ {t + 1}\right) - \phi^ {t} \left(z ^ {t + 1}\right)\right) \\ = \phi^ {t} (p ^ {t}) - \phi^ {t} (z ^ {t + 1}) - \left\langle \nabla \phi^ {t} (p ^ {t}) - \widehat {\ell} ^ {t}, p ^ {t} - z ^ {t + 1} \right\rangle + \phi^ {t} (z ^ {t + 1}) - \phi^ {t + 1} (z ^ {t + 1}) \\ = \left\langle p ^ {t} - z ^ {t + 1}, \widehat {\ell} ^ {t} \right\rangle - D ^ {t} \left(z ^ {t + 1}, p ^ {t}\right) + \phi^ {t} \left(z ^ {t + 1}\right) - \phi^ {t + 1} \left(z ^ {t + 1}\right). \\ \end{array}
$$
One can bound the stability term in conditional expectation by
$$
\mathbb {E} ^ {t} \left[ \left\langle p ^ {t} - z ^ {t + 1}, \widehat {\ell} ^ {t} \right\rangle - D ^ {t} \left(z ^ {t + 1}, p ^ {t}\right) \right] \leq \frac {1}{\gamma^ {t}} \left(\sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) ^ {2} \leq \frac {1}{\gamma^ {t}} \left(K ^ {1 / 3}\right) ^ {2} = \frac {K ^ {1 / 2}}{\sqrt {t}}, \tag {70}
$$
where the first step applies Lemma C.5.3 and uses a similar approach of Eq. (49) (changing  $V$  to  $[K]$ ); the second step uses the fact that when  $p_i^t = 1 / K$  for all  $i\in [K]$ , the square of sum is maximized.
The penalty term is bounded by
$$
\phi^ {t} \left(z ^ {t + 1}\right) - \phi^ {t + 1} \left(z ^ {t + 1}\right) \leq \mathcal {O} \left(\frac {K ^ {1 / 6}}{\sqrt {t}} \sum_ {i \in [ K ]} \left(z _ {i} ^ {t + 1}\right) ^ {2 / 3}\right) \leq \mathcal {O} \left(\frac {K ^ {1 / 6}}{\sqrt {t}} \sum_ {i \in [ K ]} \left(p _ {i} ^ {t}\right) ^ {2 / 3}\right) \leq \mathcal {O} \left(\frac {\sqrt {K}}{\sqrt {t}}\right), \tag {71}
$$
where the first step uses a similar argument of Eq. (51); the second step applies Eq. (69); the last step follows from the fact  $\sum_{i\in [K]}(p_i^t)^{2 / 3}\leq K^{1 / 3}$  since  $p_i^t = 1 / K$  maximizes the value of  $\sum_{i\in [K]}(p_i^t)^{2 / 3}$ . Combining Eq. (70) and Eq. (71), we complete the proof.
Sanity check for the condition in Lemma C.5.3 Recall that we defer the sanity check for the condition of Lemma C.5.3 in Appendix B.4. This condition requires us to check that  $\frac{\left(\bar{p}_j^{t+1}\right)^{1/3} (\lambda_V - c)}{\gamma^t}$  for any  $j \in V$  and  $\frac{\left(\bar{p}_j^{t+1}\right)^{1/3} (\lambda_U - c)}{\gamma^t}$  for any  $j \in U$  can be lower-bounded by a fixed negative constant (that is,  $\frac{\beta}{1-\beta}\left(e^{\frac{\beta-1}{\beta}} - 1\right) = 2\left(e^{-\frac{1}{2}} - 1\right) \approx -0.78693$  when  $\beta = 2/3$ ).
Because the lower bound of  $\lambda_{V}$  is zero, which is larger than that of  $\lambda_{U}$ , we will verify that  $\frac{\left(\bar{p}_j^{t + 1}\right)^{1 / 3}(\lambda_U - c)}{\gamma^t}$  can be lower-bounded for any  $j\in U$  and  $t\geq T_0 + 1$ , and the other one can be similarly bounded. For any  $j\in U$ , we first show that  $\frac{\left(\bar{p}_j^{t + 1}\right)^{1 / 3}\lambda_U}{\gamma^t}$  is lower bounded as:
$$
\begin{array}{l} \frac {\left(\bar {p} _ {j} ^ {t + 1}\right) ^ {1 / 3} \lambda_ {U}}{\gamma^ {t}} \geq - \frac {2 \left(\bar {p} _ {j} ^ {t + 1}\right) ^ {1 / 3}}{\gamma^ {t}} \frac {\left(\gamma^ {t + 1} - \gamma^ {t}\right) \sum_ {i \in U} \bar {p} _ {i} ^ {t + 1}}{\sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \\ \geq - \frac {2 \sum_ {i \in U} \left(\bar {p} _ {j} ^ {t + 1}\right) ^ {1 / 3} \left(\bar {p} _ {i} ^ {t + 1}\right)}{t \sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \\ \end{array}
$$
$$
\begin{array}{l} \geq - \frac {2 | U | \left(\bar {p} _ {j} ^ {t + 1}\right) ^ {4 / 3} + 2 \sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}}{t \sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \\ \geq - \frac {2 (| U | + 1)}{t} \geq - \frac {1}{4}, \\ \end{array}
$$
where the first step uses the lower bound of  $\lambda_U$ ; the second step follows from the definition of learning rate  $\gamma^t$ ; the third step follows from the fact that  $x^{1/3}y \leq x^{4/3} + y^{4/3}$  for any  $x, y \geq 0$ ; the last step uses  $t \geq T_0 + 1 \geq 16K + 1$ .
Then, we show that  $\frac{\left(\overline{p}_J^{t + 1}\right)^{1 / 3}c}{\gamma^t}$  is upper-bounded as:
$$
\begin{array}{l} \frac {\left(\bar {p} _ {j} ^ {t + 1}\right) ^ {1 / 3} c}{\gamma^ {t}} = \frac {\left(\bar {p} _ {j} ^ {t + 1}\right) ^ {1 / 3}}{\gamma^ {t}} \frac {\lambda_ {U} \sum_ {i \in U} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3} + \lambda_ {V} \sum_ {i \in V} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}}{\sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \\ \leq \frac {\left(p _ {i t} ^ {t}\right) ^ {1 / 3}}{\gamma^ {t} \sum_ {i \in [ K ]} \left(\bar {p} _ {i} ^ {t + 1}\right) ^ {4 / 3}} \leq \frac {K ^ {1 / 3} \left(p _ {i t} ^ {t}\right) ^ {1 / 3}}{\gamma^ {t}} = \frac {K ^ {1 / 6}}{\sqrt {t}} \leq \frac {1}{4}, \\ \end{array}
$$
where first step applies the choice of  $c$  (see Eq. (52)); the second step uses upper bounds of  $\lambda_U$  and  $\lambda_V$ ; the third step bounds  $\sum_{i\in [K]}\left(\bar{p}_i^{t + 1}\right)^{4 / 3}\geq K^{-1 / 3}$ ; the fourth step uses  $\gamma^t = K^{1 / 6}\sqrt{t}$  and bounds  $(p_{it}^{t})^{1 / 3}\leq 1$ ; the last step uses  $t\geq T_0 + 1\geq 16K + 1$ .
Finally, combining the bounds above, we have
$$
\frac {\left(\tilde {p} _ {j} ^ {t + 1}\right) ^ {1 / 3} \left(\lambda_ {U} - c\right)}{\gamma^ {t}} \geq - \frac {1}{8} - \frac {1}{4} \geq - 0. 5 \geq 2 \left(e ^ {- \frac {1}{2}} - 1\right),
$$
which satisfies the condition of Lemma C.5.3.
C Supplementary Lemmas
Proof. As  $D^{t,t+1}(p^t, p^{t+1}) = D_V^{t,t+1}(p^t, p^{t+1}) + D_U^{t,t+1}(p^t, p^{t+1})$ , we first decompose  $D_V^{t,t+1}(p^t, p^{t+1})$ . By the definition of skewed Bregman divergence, we have
$$
\begin{array}{l} D _ {V} ^ {t, t + 1} \left(p ^ {t}, p ^ {t + 1}\right) \\ = \phi_ {V} ^ {t} (p ^ {t}) - \phi_ {V} ^ {t + 1} (p ^ {t + 1}) - \left\langle \nabla \phi_ {V} ^ {t + 1} (p ^ {t + 1}), p _ {V} ^ {t} - p _ {V} ^ {t + 1} \right\rangle \\ = \phi_ {V} ^ {t} (p ^ {t}) - \phi_ {V} ^ {t + 1} (p ^ {t + 1}) - \left\langle \nabla \phi_ {V} ^ {t + 1} (p ^ {t + 1}), p _ {V} ^ {t} - \bar {p} _ {V} ^ {t + 1} \right\rangle - \left\langle \nabla \phi_ {V} ^ {t + 1} (p ^ {t + 1}), \bar {p} _ {V} ^ {t + 1} - p _ {V} ^ {t + 1} \right\rangle \\ = \phi_ {V} ^ {t} (p ^ {t}) - \phi_ {V} ^ {t + 1} (p ^ {t + 1}) - \left\langle \nabla \phi_ {V} ^ {t + 1} (\bar {p} ^ {t + 1}), p _ {V} ^ {t} - \bar {p} _ {V} ^ {t + 1} \right\rangle - \left\langle \nabla \phi_ {V} ^ {t + 1} (p ^ {t + 1}), \bar {p} _ {V} ^ {t + 1} - p _ {V} ^ {t + 1} \right\rangle \\ = \phi_ {V} ^ {t} (p ^ {t}) - \phi_ {V} ^ {t + 1} (\bar {p} ^ {t + 1}) - \left\langle \nabla \phi_ {V} ^ {t + 1} (\bar {p} ^ {t + 1}), p _ {V} ^ {t} - \bar {p} _ {V} ^ {t + 1} \right\rangle \\ + \phi_ {V} ^ {t + 1} (\bar {p} ^ {t + 1}) - \phi_ {V} ^ {t + 1} (p ^ {t + 1}) - \left\langle \nabla \phi_ {V} ^ {t + 1} (p ^ {t + 1}), \bar {p} _ {V} ^ {t + 1} - p _ {V} ^ {t + 1} \right\rangle \\ = D _ {V} ^ {t, t + 1} \left(p ^ {t}, \bar {p} ^ {t + 1}\right) + D _ {V} ^ {t + 1} \left(\bar {p} ^ {t + 1}, p ^ {t + 1}\right), \\ \end{array}
$$
where the third step follows from the fact  $\nabla \phi_V^{t + 1}(\vec{p}^{t + 1}) - \nabla \phi_V^{t + 1}(p^{t + 1}) = c\cdot \mathbf{1}_V$  for a Lagrange multiplier  $c\in \mathbb{R}$  and the fact  $\begin{array}{r}\sum_{i\in V}p_i^t = \sum_{i\in V}\bar{p}_i^{t + 1}. \end{array}$
By similar arguments, we have
$$
D _ {U} ^ {t, t + 1} \left(p ^ {t}, p ^ {t + 1}\right) = D _ {U} ^ {t, t + 1} \left(p ^ {t}, \bar {p} ^ {t + 1}\right) + D _ {U} ^ {t + 1} \left(\bar {p} ^ {t + 1}, p ^ {t + 1}\right).
$$
Combining these two parts together, we have
$$
D ^ {t, t + 1} (p ^ {t}, p ^ {t + 1}) = D _ {V} ^ {t, t + 1} \left(p ^ {t}, \bar {p} ^ {t + 1}\right) + D _ {U} ^ {t, t + 1} \left(p ^ {t}, \bar {p} ^ {t + 1}\right) + D ^ {t + 1} \left(\bar {p} ^ {t + 1}, p ^ {t + 1}\right).
$$
Subtracting  $D_U^{t,t + 1}(q^t,q^{t + 1})$  from both sides completes the proof.
C.2 Proof of Lemma 4.2