SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \frac {S _ {\delta^ {\prime}}}{V _ {\delta^ {\prime}}} \mathbb {E} \left[ \left\langle \mathbf {g} _ {t}, - \Gamma_ {\mathbf {y} _ {t}} ^ {\mathbf {x} _ {t}} \operatorname {E x p} _ {\mathbf {y} _ {t}} ^ {- 1} \mathbf {x} \right\rangle \right] \leq \frac {S _ {\delta^ {\prime}}}{2 \eta V _ {\delta^ {\prime}}} \left(d (\mathbf {y} _ {t}, \mathbf {x}) ^ {2} - d (\mathbf {y} _ {t + 1}, \mathbf {x}) ^ {2}\right) + \frac {\eta \zeta S _ {\delta^ {\prime}}}{2 V _ {\delta^ {\prime}}} \mathbb {E} \left[ \| \mathbf {g} _ {t} \| ^ {2} \right] \tag {31} \\ \leq \frac {1}{2 \eta} \left(\frac {n}{\delta^ {\prime}} + n | \kappa |\right) \left(d \left(\mathbf {y} _ {t}, \mathbf {x}\right) ^ {2} - d \left(\mathbf {y} _ {t + 1}, \mathbf {x}\right) ^ {2}\right) + \left(\frac {n}{2 \delta^ {\prime}} + \frac {n | \kappa |}{2}\right) \eta \zeta \cdot \delta^ {\prime 2} G ^ {2}. \\ \end{array}
$$
Combining Equations (28), (29), (30) and (31), and summing from  $t = s$  to  $e$ , we have
$$
\mathbb {E} \left[ \sum_ {t = s} ^ {e} \left(\hat {f} _ {t} (\mathbf {x} _ {t}) - \hat {f} _ {t} (\mathbf {x})\right) \right] = O \left(\frac {1}{\eta \delta^ {\prime}} + \delta^ {\prime} T + \eta \delta^ {\prime} T\right)
$$
holds for any  $\mathbf{x}\in (1 - \delta)\mathcal{K}$
Following Equations (22) and (23), and taking  $\mathbf{x} = \mathrm{Exp}_{\mathbf{p}}\left((1 - \delta)\mathrm{Exp}_{\mathbf{p}}^{-1}\mathbf{x}^{*}\right)$ , we have
$$
\mathbb {E} \left[ f _ {t} (\mathbf {z} _ {t}) - \hat {f} _ {t} (\mathbf {x} _ {t}) \right] = O \left(\delta^ {\prime}\right)
$$
$$
\mathbb {E} \left[ \hat {f} _ {t} (\mathbf {x}) - f _ {t} \left(\mathbf {x} ^ {*}\right) \right] = O \left(\delta + \delta^ {\prime}\right).
$$
In sum,
$$
\sum_ {t = s} ^ {e} \mathbb {E} \left[ f _ {t} (\mathbf {z} _ {t}) - f _ {t} (\mathbf {x} ^ {*}) \right] = O \left(\frac {1}{\eta \delta^ {\prime}} + (\delta + \delta^ {\prime}) T + \eta \delta^ {\prime} T\right).
$$
By choosing  $\eta = 1$ ,  $\delta = 1 - \beta = T^{-\frac{1}{2}}$ , then  $\delta' = O\left(T^{-\frac{1}{2}}\right)$  and we can get  $O(\sqrt{T})$  regret.
Denote  $\mathbf{x}_t^* \in \operatorname{argmin}_{\mathbf{x} \in (1 - \delta) \mathcal{K}} d(\mathbf{x}, \mathbf{y}_t)$  and we again bound the number of calls to the separation oracle. Due to  $\mathbf{y}_{t+1}' = \mathrm{Exp}_{\mathbf{y}_t}\left(-\eta \Gamma_{\mathbf{x}_t}^{\mathbf{y}_t} \mathbf{g}_t\right)$ ,
$$
\begin{array}{l} d \left(\mathbf {y} _ {t + 1} ^ {\prime}, (1 - \delta) \mathcal {K}\right) \leq d \left(\mathbf {x} _ {t} ^ {*}, \mathbf {y} _ {t + 1} ^ {\prime}\right) \leq d \left(\mathbf {x} _ {t} ^ {*}, \mathbf {y} _ {t}\right) + d \left(\mathbf {y} _ {t}, \mathbf {y} _ {t + 1} ^ {\prime}\right) \\ \leq d \left(\tilde {\mathbf {y}} _ {t}, (1 - \delta) \mathcal {K}\right) + \eta \| \mathbf {g} _ {t} \| \leq d \left(\tilde {\mathbf {y}} _ {t}, (1 - \delta) \mathcal {K}\right) + \eta \delta^ {\prime} G \\ \end{array}
$$
where we use  $\| \mathbf{g}_t\| = \frac{1}{2} |f_t(\mathbf{z}_t) - f_t(\tilde{\mathbf{z}}_t)|\leq \delta 'G$  by Assumption 4. Squaring both sides, we have
$$
\begin{array}{l} d \left(\mathbf {y} _ {t + 1} ^ {\prime}, (1 - \delta) \mathcal {K}\right) ^ {2} \leq d \left(\mathbf {y} _ {t}, (1 - \delta) \mathcal {K}\right) ^ {2} + 2 d \left(\mathbf {y} _ {t}, (1 - \delta) \mathcal {K}\right) \eta \delta^ {\prime} G + \eta^ {2} \delta^ {\prime 2} G ^ {2} \\ \leq d \left(\mathbf {y} _ {t}, (1 - \delta) \mathcal {K}\right) ^ {2} + 2 \delta R \eta \delta^ {\prime} G + \eta^ {2} \delta^ {\prime 2} G ^ {2}, \tag {32} \\ \end{array}
$$
where the second inequality is again due to  $\mathbf{y}_t\in \mathcal{K}$  and
$$
d \left(\mathbf {y} _ {t}, (1 - \delta) \mathcal {K}\right) \leq d \left(\mathbf {y} _ {t}, \operatorname {E x p} _ {\mathbf {p}} \left((1 - \delta) \operatorname {E x p} _ {\mathbf {p}} ^ {- 1} \mathbf {y} _ {t}\right)\right) = d \left(\mathbf {y} _ {t}, \operatorname {E x p} _ {\mathbf {y} _ {t}} \left(\delta \operatorname {E x p} _ {\mathbf {y} _ {t}} ^ {- 1} \mathbf {p}\right)\right) = \delta d \left(\mathbf {y} _ {t}, \mathbf {p}\right) \leq \delta R.
$$
Combining Equation (32) with Lemma 4, we can bound the number of separation oracle calls as:
$$
\begin{array}{l} N _ {c a l l s} \leq \sum_ {t = 1} ^ {T} \left(\zeta \frac {d (\mathbf {y} _ {t + 1} ^ {\prime} , (1 - \delta) \mathcal {K}) ^ {2} - d (\mathbf {y} _ {t + 1} , (1 - \delta) \mathcal {K}) ^ {2}}{\delta^ {2} \bar {r} ^ {2}} + 1\right) \\ \leq \sum_ {t = 1} ^ {T} \left(\zeta \frac {d \left(\mathbf {y} _ {t} , (1 - \delta) \mathcal {K}\right) ^ {2} + 2 R \delta \eta \delta^ {\prime} G + \eta^ {2} \delta^ {\prime 2} G ^ {2} - d \left(\mathbf {y} _ {t + 1} , (1 - \delta) \mathcal {K}\right) ^ {2}}{\delta^ {2} \bar {r} ^ {2}} + 1\right) \tag {33} \\ = \zeta \cdot \frac {2 \delta^ {\prime} \eta R G}{\delta \bar {r} ^ {2}} + \zeta \cdot \frac {\eta^ {2} \delta^ {\prime 2} G ^ {2}}{\delta^ {2} \bar {r} ^ {2}} T + T \\ \end{array}
$$
where the last inequality follows from  $\tilde{\mathbf{y}}_1 = \mathbf{p} \in (1 - \delta)\mathcal{K}$  and thus  $d(\tilde{\mathbf{y}}_1, (1 - \delta)\mathcal{K}) = 0$ . Since  $\eta = 1$ ,  $\delta = 1 - \beta = T^{-\frac{1}{2}}$  and
$$
\delta^ {\prime} = (1 - \beta) \frac {\sqrt {- \kappa} (R + r)}{\sinh (\sqrt {- \kappa} (R + r))} r = \delta \frac {\sqrt {- \kappa} (R + r)}{\sinh (\sqrt {- \kappa} (R + r))} r \leq \delta r.
$$
Plugging these parameters into Equation (33), we have
$$
N _ {c a l l s} \leq \zeta \cdot \frac {2 r R G T}{\bar {r} ^ {2}} + \zeta \cdot \frac {r ^ {2} G ^ {2} T}{\bar {r} ^ {2}} + T = O (T).
$$
C Omitted Proofs in Section 6
We have the following convergence guarantee for RFW.
C.1 Proof of Lemma 5
Proof. Since Algorithm 6 is indeed RFW with line-search (Algorithm 9) when  $f(\mathbf{x}) = d(\mathbf{x}, \mathbf{y})^2 / 2$ , which is  $\zeta$  gsc-smooth by Lemma 27 and  $\nabla f(\mathbf{x}) = -\mathrm{Exp}_{\mathbf{x}}^{-1}\mathbf{y}$ , the upper bound on the number of iterations follows from Lemma 26 with  $L = \zeta$  and  $D = 2R$ . Item 1 follows from the line-search because  $f(\mathbf{x}_i) = d(\mathbf{x}_i, \mathbf{y})^2 / 2$  does not increase as  $i$  increases.
For item 2, Algorithm 6 stops when either  $d(\tilde{\mathbf{x}},\mathbf{y})^2\leq 3\epsilon$  or  $d(\tilde{\mathbf{x}},\mathbf{y})^2 >3\epsilon$  and  $\langle \mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{y},\mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{x}\rangle \leq$ $\epsilon$ . For the first case, item 2 obviously holds. Now we consider the second case. We first show  $\langle \mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{z},\mathrm{Exp}_{\mathbf{y}}^{-1}\tilde{\mathbf{x}}\rangle +\langle \mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{z},\mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{y}\rangle \geq d(\tilde{\mathbf{x}},\mathbf{y})^2$  as follows. By Lemma 30, we have
$$
\begin{array}{l} \left\langle \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {z}, \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \tilde {\mathbf {x}} \right\rangle \geq \frac {1}{2} (d (\mathbf {y}, \mathbf {z}) ^ {2} + d (\mathbf {y}, \tilde {\mathbf {x}}) ^ {2} - d (\tilde {\mathbf {x}}, \mathbf {z}) ^ {2}) \\ \left\langle \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {z}, \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {y} \right\rangle \geq \frac {1}{2} \left(d (\tilde {\mathbf {x}}, \mathbf {y}) ^ {2} + d (\tilde {\mathbf {x}}, \mathbf {z}) ^ {2} - d (\mathbf {y}, \mathbf {z}) ^ {2}\right). \\ \end{array}
$$
Adding the above two inequalities we have  $\langle \mathrm{Exp}_{\mathbf{y}}^{-1}\mathbf{z},\mathrm{Exp}_{\mathbf{y}}^{-1}\tilde{\mathbf{x}}\rangle +\langle \mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{z},\mathrm{Exp}_{\tilde{\mathbf{x}}}^{-1}\mathbf{y}\rangle \geq d(\tilde{\mathbf{x}},\mathbf{y})^{2}$ . Now item 2 follows from
$$
\left\langle \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \mathbf {z}, \operatorname {E x p} _ {\mathbf {y}} ^ {- 1} \tilde {\mathbf {x}} \right\rangle \geq d (\tilde {\mathbf {x}}, \mathbf {y}) ^ {2} - \left\langle \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {z}, \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {y} \right\rangle > 3 \epsilon - \left\langle \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {v} _ {i}, \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {y} \right\rangle > 2 \epsilon .
$$
For the third item, denote  $\mathbf{x}^{*} = \operatorname{argmin}_{\mathbf{x} \in \mathcal{K}} d(\mathbf{x}, \mathbf{y})^{2}$ . Suppose  $d(\mathbf{y}, \mathcal{K})^{2} = d(\mathbf{x}^{*}, \mathbf{y})^{2} < \epsilon$  and  $d(\tilde{\mathbf{x}}, \mathbf{y})^{2} > 3\epsilon$ . Denote  $f(\mathbf{x}) = d(\mathbf{x}, \mathbf{y})^{2}/2$  and  $\nabla f(\mathbf{x}) = -\mathrm{Exp}_{\mathbf{x}}^{-1}\mathbf{y}$ . For the last iteration,
$$
\left\langle \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {v} _ {i}, \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {y} \right\rangle = \max _ {\mathbf {v} \in \mathcal {K}} \left\langle \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {v}, - \nabla f (\tilde {\mathbf {x}}) \right\rangle \leq \epsilon ,
$$
$$
\begin{array}{l} d (\tilde {\mathbf {x}}, \mathbf {y}) ^ {2} - d (\mathbf {y}, \mathcal {K}) ^ {2} = 2 f (\tilde {\mathbf {x}}) - 2 f \left(\mathbf {x} ^ {*}\right) \leq 2 \left\langle - \operatorname {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {x} ^ {*}, \nabla f (\tilde {\mathbf {x}}) \right\rangle \\ \leq 2 \max _ {\mathbf {v} \in \mathcal {K}} \left\langle - \mathrm {E x p} _ {\tilde {\mathbf {x}}} ^ {- 1} \mathbf {v}, \nabla f (\tilde {\mathbf {x}}) \right\rangle \leq 2 \epsilon . \\ \end{array}
$$
$$
d (\tilde {\mathbf {x}}, \mathbf {y}) ^ {2} \leq d (\mathbf {y}, \mathcal {K}) ^ {2} + 2 \epsilon \leq 3 \epsilon
$$
C.2 Proof of Lemma 6
Before proving Lemma 6, we need the following lemma.
Lemma 8. Consider Algorithm 7 and fix some  $\epsilon$  such that  $0 < 3\epsilon < d(\mathbf{x}_0,\mathbf{y}_0)^2$ . Setting  $\gamma = \frac{2\epsilon}{d(\mathbf{x}_0,\mathbf{y}_0)^2}$ , we have  $d(\mathbf{x}_i,\mathbf{y}_i) \leq d(\mathbf{x}_0,\mathbf{y}_0)$  for any  $i$ .
Proof. For all  $i > 1$ , we define the sequence  $\mathbf{y}_i$  by the relation  $\mathbf{y}_i = \mathrm{Exp}_{\mathbf{x}_{i-1}}((1 - \gamma)\mathrm{Exp}_{\mathbf{x}_{i-1}}^{-1}\mathbf{y}_{i-1})$  and  $\gamma \in [0,1)$ . From this, we can deduce the following sequence of inequalities:
$$
\begin{array}{l} d (\mathbf {x} _ {i - 1}, \mathbf {y} _ {i}) = \left\| \operatorname {E x p} _ {\mathbf {x} _ {i - 1}} ^ {- 1} \mathbf {y} _ {i} \right\| \\ = (1 - \gamma) \left\| \operatorname {E x p} _ {\mathbf {x} _ {i - 1}} ^ {- 1} \mathbf {y} _ {i - 1} \right\| \\ \leq d (\mathbf {x} _ {i - 1}, \mathbf {y} _ {i - 1}). \\ \end{array}
$$
The inequality follows since  $\gamma \in (0,1)$ .
By the first item of Lemma 5, for any  $i \geq 1$ , we have  $d(\mathbf{x}_i, \mathbf{y}_i) \leq d(\mathbf{x}_{i-1}, \mathbf{y}_i)$ . Combining this with the previous inequality, we get
$$
\begin{array}{l} d \left(\mathbf {x} _ {i}, \mathbf {y} _ {i}\right) \leq d \left(\mathbf {x} _ {i - 1}, \mathbf {y} _ {i - 1}\right) \\ \leq \dots \\ \leq d \left(\mathbf {x} _ {1}, \mathbf {y} _ {1}\right) \\ \leq d \left(\mathbf {x} _ {0}, \mathbf {y} _ {1}\right) \\ \leq d \left(\mathbf {x} _ {0}, \mathbf {y} _ {0}\right), \\ \end{array}
$$
where the last inequality follows from the fact that  $\mathbf{x}_0\in \mathcal{K}$  and  $\mathbf{y}_1$  is the projection of  $\mathbf{y}_0$  onto  $\mathbb{B}_{\mathbf{p}}(R)$ , while  $\mathcal{K}\subseteq \mathbb{B}_{\mathbf{p}}(R)$ .
Proof of Lemma 6. Given that  $\mathbf{y}_1$  is the projection of  $\mathbf{y}_0$  onto  $\mathbb{B}_{\mathbf{p}}(R)$  and  $\mathcal{K} \subseteq \mathbb{B}_{\mathbf{p}}(R)$ , we have  $\forall \mathbf{z} \in \mathcal{K}: d(\mathbf{y}_1, \mathbf{z})^2 \leq d(\mathbf{y}_0, \mathbf{z})^2$ . The lemma trivially holds when  $d(\mathbf{x}_0, \mathbf{y}_0)^2 \leq 3\epsilon$  or  $d(\mathbf{x}_1, \mathbf{y}_1)^2 \leq 3\epsilon$ .
Without loss of generality, we assume that  $d(\mathbf{x}_1,\mathbf{y}_1) > 3\epsilon$ . Let  $k > 1$  be the number of iterations in Algorithm 7. This means that  $d(\mathbf{x}_k,\mathbf{y}_k)^2\leq 3\epsilon$  and  $d(\mathbf{x}_i,\mathbf{y}_i)^2 >3\epsilon$  for any  $i < k$ . According to the second item of Lemma 5, we obtain  $\langle \mathrm{Exp}_{\mathbf{y}_i}^{-1}\mathbf{z},\mathrm{Exp}_{\mathbf{y}_i}^{-1}\mathbf{x}_i\rangle \geq 2\epsilon$  for all  $i < k$  and  $\mathbf{z}\in \mathcal{K}$ . Lemma 8 also gives us  $d(\mathbf{x}_i,\mathbf{y}_i)\leq d(\mathbf{x}_0,\mathbf{y}_0)$  for all  $i < k$ .
Applying Lemma 2 with  $\mathbf{g} = -\mathrm{Exp}_{\mathbf{y}_i}^{-1}\mathbf{x}_i$ ,  $C = d(\mathbf{x}_0,\mathbf{y}_0)$ , and  $Q = 2\epsilon$ , we deduce that for every  $1 \leq i < k$ ,
$$
\forall \mathbf {z} \in \mathcal {K}: \quad d (\mathbf {z}, \mathbf {y} _ {i + 1}) ^ {2} \leq d (\mathbf {z}, \mathbf {y} _ {i}) ^ {2} - \frac {4 \epsilon^ {2}}{\zeta d \left(\mathbf {x} _ {i} , \mathbf {y} _ {i}\right) ^ {2}} \leq d (\mathbf {z}, \mathbf {y} _ {i}) ^ {2} - \frac {4 \epsilon^ {2}}{\zeta d \left(\mathbf {x} _ {0} , \mathbf {y} _ {0}\right) ^ {2}}. \tag {34}
$$
Consequently, for any  $\mathbf{z} \in \mathcal{K}$ , the returned point  $\mathbf{y}$  satisfies  $d(\mathbf{y}, \mathbf{z})^2 \leq d(\mathbf{y}_1, \mathbf{z})^2 \leq d(\mathbf{y}_0, \mathbf{z})^2$ . Because  $\mathbf{p} \in \mathcal{K}$ , we have  $d(\mathbf{y}, \mathbf{p}) \leq d(\mathbf{y}_1, \mathbf{p}) \leq R$ , which implies  $\mathbf{y} \in \mathbb{B}_{\mathbf{p}}(R)$ . Note that  $\mathbf{x} \in \mathcal{K}$  as it is the output of Algorithm 6. Thus, we know Algorithm 7 returns  $(\mathbf{x}, \mathbf{y}) \in \mathcal{K} \times \mathbb{B}_{\mathbf{p}}(R)$ .
Now we bound the number of iterations in Algorithm 7. Denote  $\mathbf{x}_i^* = \operatorname{argmin}_{\mathbf{x} \in \mathcal{K}} d(\mathbf{x}, \mathbf{y}_i)^2$  for any  $i < k$ . Using Equation (34),
$$
\begin{array}{l} d \left(\mathbf {y} _ {i + 1}, \mathcal {K}\right) ^ {2} = d \left(\mathbf {x} _ {i + 1} ^ {*}, \mathbf {y} _ {i + 1}\right) ^ {2} \leq d \left(\mathbf {x} _ {i} ^ {*}, \mathbf {y} _ {i + 1}\right) ^ {2} \\ \leq d (\mathbf {x} _ {i} ^ {*}, \mathbf {y} _ {i}) ^ {2} - \frac {4 \epsilon^ {2}}{\zeta d (\mathbf {x} _ {0} , \mathbf {y} _ {0}) ^ {2}} = d (\mathbf {y} _ {i}, \mathcal {K}) ^ {2} - \frac {4 \epsilon^ {2}}{\zeta d (\mathbf {x} _ {0} , \mathbf {y} _ {0}) ^ {2}}. \\ \end{array}
$$
By applying the inequality  $d(\mathbf{y}_1,\mathcal{K})^2\leq d(\mathbf{y}_0,\mathcal{K})^2$  and subsequently performing a telescoping sum, we can demonstrate that
$$
d (\mathbf {y} _ {i + 1}, \mathcal {K}) ^ {2} \leq d (\mathbf {y} _ {1}, \mathcal {K}) ^ {2} - \frac {4 i \epsilon^ {2}}{\zeta d (\mathbf {x} _ {0} , \mathbf {y} _ {0}) ^ {2}} \leq d (\mathbf {y} _ {0}, \mathcal {K}) ^ {2} - \frac {4 i \epsilon^ {2}}{\zeta d (\mathbf {x} _ {0} , \mathbf {y} _ {0}) ^ {2}} \leq d (\mathbf {y} _ {0}, \mathbf {x} _ {0}) ^ {2} - \frac {4 i \epsilon^ {2}}{\zeta d (\mathbf {x} _ {0} , \mathbf {y} _ {0}) ^ {2}}.
$$
After at most  $k - 1 = (\zeta d(\mathbf{x}_0, \mathbf{y}_0)^2 (d(\mathbf{x}_0, \mathbf{y}_0)^2 - \epsilon)) / 4\epsilon^2$  iterations, we have  $d(\mathbf{y}_k, \mathcal{K})^2 \leq \epsilon$ . By the third item of Lemma 5, the next iteration will be the last one, and the returned points  $\mathbf{x}, \mathbf{y}$  satisfy  $d(\mathbf{x}, \mathbf{y})^2 \leq 3\epsilon$ , as claimed.
We first prove Lemma 9, which bounds the regret of the infeasible projection for gsc-convex losses.
Lemma 9. With some fixed block size  $B$ ,  $\eta_i = \eta$  and  $\epsilon_i = \epsilon$  for any  $i = 1, \dots, \frac{T}{B}$ . Assume all losses are gsc-convex. We use  $i(t)$  to denote the block index of round  $t$ . Then the regret of playing  $\tilde{\mathbf{y}}_{i(t)-1}$  as defined in Algorithm 8 can be bounded by
$$
\sup _ {I = [ s, e ] \subseteq [ T ]} \left\{\sum_ {t = s} ^ {e} f _ {t} (\tilde {\mathbf {y}} _ {i (t) - 1}) - \min _ {\mathbf {x} _ {I} \in \mathcal {K}} \sum_ {t = s} ^ {e} f _ {t} (\mathbf {x} _ {I}) \right\} \leq \frac {\zeta \eta B G ^ {2} T}{2} + 4 R K G + \frac {4 R ^ {2}}{\eta}.
$$
Proof. For convenience, we assume  $B$  divides  $T$ . Let  $\mathcal{T}_i = \{(i - 1)B + 1,\dots ,iB\}$  be the set of rounds in the  $i$ -th block. By Lemma 6,  $\tilde{\mathbf{y}}_{i + 1}$  is the infeasible projection of  $\mathbf{y}_{iB + 1}$  onto  $\mathcal{K}$ :
$$
d (\tilde {\mathbf {y}} _ {i + 1}, \mathbf {x}) ^ {2} \leq d (\mathbf {y} _ {i B + 1}, \mathbf {x}) ^ {2}
$$
holds for any  $\mathbf{x} \in \mathcal{K}$ . By Algorithm 8, we have
$$
\mathbf {y} _ {i B + 1} = \mathrm {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} \left(- \eta \sum_ {s = 1} ^ {B} \nabla f _ {(i - 1) B + s} (\tilde {\mathbf {y}} _ {i - 1})\right) = \mathrm {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} \left(- \eta \sum_ {t \in \mathcal {T} _ {i}} \nabla f _ {t} (\tilde {\mathbf {y}} _ {i (t) - 1})\right).
$$
$$
\begin{array}{l} d \left(\tilde {\mathbf {y}} _ {i + 1}, \mathbf {x}\right) ^ {2} \leq d \left(\mathbf {y} _ {i B + 1}, \mathbf {x}\right) ^ {2} \\ \leq d (\tilde {\mathbf {y}} _ {i - 1}, \mathbf {x}) ^ {2} + \zeta \eta^ {2} B ^ {2} G ^ {2} - 2 \eta \sum_ {t \in \mathcal {T} _ {i}} \left\langle \nabla f _ {t} (\tilde {\mathbf {y}} _ {i - 1}), - \mathrm {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} ^ {- 1} \mathbf {x} \right\rangle \\ \end{array}
$$
where the last inequality is due to Lemma 29 and  $\| \nabla f_t(\mathbf{x})\| \leq G$ . This is equivalent to
$$
\sum_ {t \in \mathcal {T} _ {i}} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} ^ {- 1} \mathbf {x} \right\rangle \leq \frac {d \left(\tilde {\mathbf {y}} _ {i - 1} , \mathbf {x}\right) ^ {2} - d \left(\tilde {\mathbf {y}} _ {i + 1} , \mathbf {x}\right) ^ {2}}{2 \eta} + \frac {\zeta \eta B ^ {2} G ^ {2}}{2}. \tag {35}
$$
For a certain interval  $[s, e] \subseteq [T]$ , let  $i_s$  and  $i_e$  be the smallest and the largest block indices such that  $\mathcal{T}_{i_s}$  and  $\mathcal{T}_{i_e}$  are fully contained in  $[s, e]$ . Then we have
$$
\begin{array}{l} \sum_ {t = s} ^ {e} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i (t) - 1}} ^ {- 1} \mathbf {x} \right\rangle \leq \sum_ {t = s} ^ {(i _ {s} - 1) B} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i _ {s} - 2}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i _ {s} - 2}} ^ {- 1} \mathbf {x} \right\rangle \tag {36} \\ + \sum_ {i = i _ {s}} ^ {i _ {e}} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} ^ {- 1} \mathbf {x} \right\rangle + \sum_ {t = i _ {e} B + 1} ^ {e} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i _ {e}}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i _ {e}}} ^ {- 1} \right\rangle . \\ \end{array}
$$
There are some undesirable terms in blocks  $i_s - 1$  and  $i_e + 1$ , and we can bound them by noticing
$$
\left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i (t) - 1}} ^ {- 1} \mathbf {x} \right\rangle \leq \| \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right) \| \cdot \| \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i (t) - 1}} ^ {- 1} \mathbf {x} \| \leq G \cdot 2 R = 2 G R. \tag {37}
$$
Combining Equations (35), (36) and (37), we have
$$
\begin{array}{l} \sum_ {t = s} ^ {e} \left(f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right) - f _ {t} (\mathbf {x})\right) \leq \sum_ {t = s} ^ {e} \left\langle \nabla f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right), - \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i (t) - 1}} ^ {- 1} \mathbf {x} \right\rangle \\ \leq \sum_ {i = i _ {s}} ^ {i _ {e}} \left(\frac {d \left(\tilde {\mathbf {y}} _ {i - 1} , \mathbf {x}\right) ^ {2} - d \left(\tilde {\mathbf {y}} _ {i + 1} , \mathbf {x}\right) ^ {2}}{2 \eta} + \frac {\zeta \eta B ^ {2} G ^ {2}}{2}\right) + 2 G R \cdot 2 B \tag {38} \\ \leq \frac {4 R ^ {2}}{\eta} + \frac {\zeta \eta B G ^ {2} T}{2} + 4 B G R. \\ \end{array}
$$
where the last inequality is due to the number of blocks in  $[s,e]$  is upper bounded by  $\frac{T}{B}$ .
Now we give the proof of Theorem 4.
Proof of Theorem 4. We still use  $i(t)$  to denote the block index of the  $t$ -th round. For an interval  $[s, e] \subseteq [T]$ , we can do the following decomposition
$$
\begin{array}{l} \sum_ {t = s} ^ {e} \left(f _ {t} (\mathbf {x} _ {i (t) - 1}) - f _ {t} (\mathbf {x})\right) \\ = \sum_ {t = s} ^ {e} \left(f _ {t} \left(\mathbf {x} _ {i (t) - 1}\right) - f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right)\right) + \sum_ {t = s} ^ {e} \left(f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right) - f _ {t} (\mathbf {x})\right) \tag {39} \\ \leq \sum_ {t = s} ^ {e} \left\langle \nabla f _ {t} \left(\mathbf {x} _ {i (t) - 1}\right), - \operatorname {E x p} _ {\mathbf {x} _ {i (t) - 1}} ^ {- 1} \tilde {\mathbf {y}} _ {i (t) - 1} \right\rangle + \sum_ {t = s} ^ {e} \left(f _ {t} \left(\tilde {\mathbf {y}} _ {i (t) - 1}\right) - f _ {t} (\mathbf {x})\right) \\ \end{array}
$$
Algorithm 8 calls Algorithm 7 in each block to compute an infeasible projection. By Lemma 6 and Algorithm 8,  $(\mathbf{x}_i,\tilde{\mathbf{y}}_i)\in \mathcal{K}\times \mathbb{B}_{\mathbf{p}}(R)$ , which satisfies  $d(\mathbf{x}_i,\tilde{\mathbf{y}}_i)^2\leq 3\epsilon_i = 3\epsilon$ . Thus
$$
\left\langle \nabla f _ {t} \left(\mathbf {x} _ {i (t) - 1}\right), - \operatorname {E x p} _ {\mathbf {x} _ {i (t) - 1}} ^ {- 1} \tilde {\mathbf {y}} _ {i (t) - 1} \right\rangle \leq G d \left(\mathbf {x} _ {i (t) - 1}, \tilde {\mathbf {y}} _ {i (t)}\right) \leq \sqrt {3 \epsilon} G. \tag {40}
$$
Combining Equations (39), (40) and Lemma 9, we have
$$
\sum_ {t = s} ^ {e} \left(f _ {t} \left(\mathbf {x} _ {i (t) - 1}\right) - f _ {t} (\mathbf {x})\right) \leq 4 R G B + \frac {4 R ^ {2}}{\eta} + \frac {\zeta \eta B G ^ {2} T}{2} + G T \sqrt {3 \epsilon}
$$
holds for any  $\mathbf{x} \in \mathcal{K}$ . By plugging in  $\eta, \epsilon$ , and  $B$ , we get the claimed regret bound.
We also need to bound the number of calls to the linear optimization oracle. Note that  $d(\mathbf{x}_1, \tilde{\mathbf{y}}_1) = 0$  due to the initialization, and for any  $i \geq 2$ , we have  $d(\mathbf{x}_i, \tilde{\mathbf{y}}_i)^2 \leq 3\epsilon$  due to Lemma 6. By Algorithm 8, we have
$$
\mathbf {y} _ {i B + 1} = \operatorname {E x p} _ {\tilde {\mathbf {y}} _ {i - 1}} \left(- \eta \sum_ {t = (i - 1) B + 1} ^ {i B} \nabla f _ {t} (\tilde {\mathbf {y}} _ {i - 1})\right) \quad \forall i = 1, \dots , \frac {T}{B}.
$$
$$
d \left(\mathbf {x} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) \leq d \left(\mathbf {x} _ {i - 1}, \tilde {\mathbf {y}} _ {i - 1}\right) + d \left(\tilde {\mathbf {y}} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) \leq \sqrt {3 \epsilon} + \eta B G. \tag {41}
$$
Squaring on both sides, we have
$$
d \left(\mathbf {x} _ {i - 1}, \mathbf {y} _ {i B + 1}\right) ^ {2} \leq \left(\sqrt {3 \epsilon} + \eta B G\right) ^ {2} \leq 6 \epsilon + 2 B ^ {2} G ^ {2} \eta^ {2}, \tag {42}
$$
where  $(a + b)^2\leq 2(a^2 +b^2)$  is used.
By Lemma 6, on the  $i$ -th block, Algorithm 7 stops after at most
$$
\max  \left\{\frac {\zeta d \left(\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}\right) ^ {2} \left(d \left(\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}\right) ^ {2} - \epsilon\right)}{4 \epsilon^ {2}} + 1, 1 \right\}
$$
iterations. According to Lemma 5, each iteration calls the linear optimization oracle for at most  $\zeta \left\lceil \frac{27R^2}{\epsilon^2} - 2 \right\rceil$  times. Thus in the  $i$ -th block, the number of calls to the linear optimization oracle is bounded by
$$
N _ {c a l l s} := \max \left\{\frac {\zeta d (\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}) ^ {2} (d (\mathbf {x} _ {i - 1} , \mathbf {y} _ {i B + 1}) ^ {2} - \epsilon)}{4 \epsilon^ {2}} + 1, 1 \right\} \cdot \frac {2 7 \zeta R ^ {2}}{\epsilon}.
$$
There are  $\frac{T}{B}$  blocks in total, so the total number of calls to the linear optimization oracle is at most