SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\sum_ {k = 0} ^ {N - 1} \frac {1}{\hat {\eta} _ {k} ^ {2}} \leq \frac {2 - \beta^ {2}}{(1 - \beta^ {2}) \sigma_ {0} ^ {2}} + \frac {2 - \beta^ {2}}{(1 - \beta^ {2}) \alpha_ {2} ^ {2} \beta^ {2}} \sum_ {k \in \mathcal {B}} \frac {\| \mathbf {w} _ {k} - \mathbf {B} _ {k} \mathbf {s} _ {k} \| ^ {2}}{\| \mathbf {s} _ {k} \| ^ {2}} \leq \frac {2 - \beta^ {2}}{(1 - \beta^ {2}) \sigma_ {0} ^ {2}} + \frac {(2 - \beta^ {2}) L _ {1} ^ {2}}{(1 - \beta^ {2}) \alpha_ {2} ^ {2} \beta^ {2}} N.
$$
By further using (14) and the elementary inequality that  $\sqrt{a + b} \leq \sqrt{a} + \sqrt{b}$ , we obtain
$$
f (\mathbf {x} _ {N}) - f (\mathbf {x} ^ {*}) \leq \frac {C _ {4} L _ {1} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2}}{N ^ {2}} + \frac {C _ {5} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2}}{\sigma_ {0} N ^ {2 . 5}},
$$
where  $C_4 = C_1\sqrt{\frac{2 - \beta^2}{(1 - \beta^2)\sigma_0^2} + \frac{(2 - \beta^2)}{(1 - \beta^2)\alpha_2^2\beta^2}}$  and  $C_5 = C_1\sqrt{\frac{2 - \beta^2}{(1 - \beta^2)\sigma_0^2}}$ .
Next, we divide the proof of Part (b) of Theorem 1 into the following steps.
Step 1: We first use regret analysis to control the cumulative loss  $\sum_{t=0}^{T-1} \ell_t(\mathbf{B}_t)$  incurred by our online learning algorithm in Subroutine 2. In particular, we prove a dynamic regret bound, where we compare the cumulative loss of our algorithm against the one achieved by the sequence  $\{\mathbf{H}_t\}_{t=0}^{T-1}$ .
Lemma 12. We have
$$
\sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {B} _ {t}) \leq 2 5 6 \| \mathbf {B} _ {0} - \mathbf {H} _ {0} \| _ {F} ^ {2} + 4 \sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {H} _ {t}) + 2 L _ {1} ^ {2} \sum_ {t = 0} ^ {T - 1} \delta_ {t} ^ {2} + 5 1 2 L _ {1} \sqrt {d} \sum_ {t = 0} ^ {T - 1} \| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F},
$$
where  $\mathbf{H}_t\triangleq \nabla^2 f(\mathbf{y}_t)$
Step 2: In light of Lemma 12, it suffices to upper bound the cumulative loss  $\sum_{t=0}^{T-1} \ell_t(\mathbf{H}_t)$  and the path-length  $\sum_{t=0}^{T-1} \|\mathbf{H}_{t+1} - \mathbf{H}_t\|_F$  in the following lemma. To achieve this, we use the stability properties of our algorithm in (21) and Lemma 8, which is most technical part of the proof.
Lemma 13. We have
$$
\sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {H} _ {t}) \leq \frac {C _ {3}}{4} L _ {2} ^ {2} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2} \quad a n d \quad \sum_ {t = 0} ^ {T - 1} \| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F} \leq C _ {2} \sqrt {d} L _ {2} \left(1 + \log \frac {A _ {N}}{A _ {1}}\right) \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \|, \tag {72}
$$
where  $C_2$  is defined in (49) and  $C_3 = \frac{(1 + \alpha_1)^2}{\beta^2(1 - \alpha_1)^2(1 - \sigma^2)}$ .
Step 3: Thus, we obtain an upper bound on  $\sum_{t=0}^{T-1} \ell_t(\mathbf{B}_t)$  by combining Lemma 12 and Lemma 13. Finally, in the following lemma, we prove an upper bound on  $\frac{1}{A_N}$  by further using Lemma 2 and Proposition 1.
Lemma 14. We have
$$
\frac {1}{A _ {N}} \leq \frac {1}{N ^ {2 . 5}} \left(M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log^ {+} \left(\frac {\operatorname* {m a x} \{\frac {L _ {1}}{\alpha_ {2} \beta} , \frac {1}{\sigma_ {0}} \} N ^ {2 . 5}}{\sqrt {M}}\right)\right) ^ {\frac {1}{2}},
$$
where we define  $\log^{+}(x) \triangleq \max \{\log(x), 0\}$ ,
$$
M = \frac {C _ {6}}{\sigma_ {0} ^ {2}} + C _ {7} L _ {1} ^ {2} + C _ {8} \| \mathbf {B} _ {0} - \mathbf {H} _ {0} \| _ {F} ^ {2} + C _ {9} L _ {2} ^ {2} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2} + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \|,
$$
and  $C_i$  ( $i = 6, \ldots, 10$ ) are absolute constants given by
$$
C _ {6} = \frac {4 C _ {1} ^ {2} (2 - \beta^ {2})}{1 - \beta^ {2}}, C _ {7} = \frac {5 C _ {6}}{\alpha_ {2} ^ {2} \beta^ {2}}, C _ {8} = \frac {2 5 6 C _ {6}}{\alpha_ {2} ^ {2} \beta^ {2}}, C _ {9} = \frac {C _ {3} C _ {6}}{\alpha_ {2} ^ {2} \beta^ {2}}, C _ {1 0} = \frac {5 1 2 C _ {2} C _ {6}}{\alpha_ {2} ^ {2} \beta^ {2}}.
$$
Therefore, Part (b) of Theorem 1 immediately follows from Proposition 1.
In the remaining of this section, we present the proofs for the above lemmas that we used to prove the results in Theorem 1.
D.1 Proof of Lemma 11
Recall that  $\mathbf{w}_k \triangleq \nabla f(\tilde{\mathbf{x}}_{k+1}) - \nabla f(\mathbf{y}_k)$  and  $\mathbf{s}_k \triangleq \tilde{\mathbf{x}}_{k+1} - \mathbf{y}_k$  for  $k \in \mathcal{B}$ . We can write  $\nabla f(\tilde{\mathbf{x}}_{k+1}) - \nabla f(\mathbf{y}_k) = \bar{\mathbf{H}}_k(\tilde{\mathbf{x}}_{k+1} - \mathbf{y}_k)$  by using the fundamental theorem of calculus, where  $\bar{\mathbf{H}}_k = \int_0^1 \nabla^2 f(t\tilde{\mathbf{x}}_{k+1} + (1 - t)\mathbf{y}_k) dt$ . Since we have  $0 \preceq \nabla^2 f(\mathbf{x}) \preceq L_1\mathbf{I}$  for all  $\mathbf{x} \in \mathbb{R}^d$  by Assumption 1, it implies that  $0 \preceq \bar{\mathbf{H}}_k \preceq L_1\mathbf{I}$ . Moreover, since  $0 \preceq \mathbf{B}_k \preceq L_1\mathbf{I}$ , we further have  $-L_1\mathbf{I} \preceq \bar{\mathbf{H}}_k - \mathbf{B}_k \preceq L_1\mathbf{I}$ , which yields  $\| \bar{\mathbf{H}}_k - \mathbf{B}_k \|_{\mathrm{op}} \leq L_1$ . Thus, we have
$$
\left\| \mathbf {w} _ {k} - \mathbf {B} _ {k} \mathbf {s} _ {k} \right\| = \left\| \left(\bar {\mathbf {H}} _ {k} - \mathbf {B} _ {k}\right) \left(\tilde {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k}\right) \right\| \leq L _ {1} \| \tilde {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k} \|,
$$
which proves that  $\ell_k(\mathbf{B}_k) \leq L_1^2$ .
D.2 Proof of Lemma 12
To prove Lemma 12, we first present the following lemma showing a smooth property of the loss function  $\ell_{k}$ . The proof is similar to [35, Lemma 15].
Lemma 15. For  $k\in \mathcal{B}$  we have
$$
\nabla \ell_ {k} (\mathbf {B}) = \frac {1}{\left\| \mathbf {s} _ {k} \right\| ^ {2}} \left(- \mathbf {s} _ {k} \left(\mathbf {w} _ {k} - \mathbf {B} \mathbf {s} _ {k}\right) ^ {\top} - \left(\mathbf {w} _ {k} - \mathbf {B} \mathbf {s} _ {k}\right) \mathbf {s} _ {k} ^ {\top}\right). \tag {73}
$$
Moreover, for any  $\mathbf{B} \in \mathbb{S}^d$ , it holds that
$$
\left\| \nabla \ell_ {k} (\mathbf {B}) \right\| _ {F} \leq \left\| \nabla \ell_ {k} (\mathbf {B}) \right\| _ {*} \leq 2 \sqrt {\ell_ {k} (\mathbf {B})}, \tag {74}
$$
where  $\| \cdot \| _F$  and  $\| \cdot \|_*$  denote the Frobenius norm and the nuclear norm, respectively.
Proof. It is straightforward to verify the expression in (73). The first inequality in (74) follows from the fact that  $\| \mathbf{A}\| _F\leq \| \mathbf{A}\| _*$  for any matrix  $\mathbf{A}\in \mathbb{S}^d$ . For the second inequality, note that
$$
\begin{array}{l} \| \nabla \ell_ {k} (\mathbf {B}) \| _ {*} \leq \frac {1}{\| \mathbf {s} _ {k} \| ^ {2}} \left(\| \mathbf {s} _ {k} (\mathbf {w} _ {k} - \mathbf {B s} _ {k}) ^ {\top} \| _ {*} + \| (\mathbf {w} _ {k} - \mathbf {B s} _ {k}) \mathbf {s} _ {k} ^ {\top} \| _ {*}\right) \\ \leq \frac {2}{\| \mathbf {s} _ {k} \| ^ {2}} \| \mathbf {w} _ {k} - \mathbf {B s} _ {k} \| \| \mathbf {s} _ {k} \| = \frac {2 \| \mathbf {w} _ {k} - \mathbf {B s} _ {k} \|}{\| \mathbf {s} _ {k} \|} = 2 \sqrt {\ell_ {k} (\mathbf {B})}, \\ \end{array}
$$
where in the first inequality we used the triangle inequality, and in the second inequality we used the fact that the rank-one matrix  $\mathbf{u}\mathbf{v}^{\top}$  has only one nonzero singular value  $\| \mathbf{u}\| \| \mathbf{v}\|$ .
We will also need the following helper lemma.
Lemma 16. If the real number  $x$  satisfies  $x \leq A + B\sqrt{x}$ , then we have  $x \leq 2A + B^2$ .
Proof. From the assumption, we have
$$
\left(\sqrt {x} - \frac {B}{2}\right) ^ {2} \leq A + \frac {B ^ {2}}{4}.
$$
Hence, we obtain
$$
x \leq \left(\sqrt {A + \frac {B ^ {2}}{4}} + \frac {B}{2}\right) ^ {2} \leq 2 A + B ^ {2}.
$$
Before proving Lemma 12, we also present the following lemma that bounds the loss in each round.
Lemma 17. For any  $\mathbf{H} \in \mathcal{Z}$ , we have
$$
\ell_ {t} (\mathbf {B} _ {t}) \leq 4 \ell_ {t} (\mathbf {H}) + 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} \| _ {F} ^ {2} - 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} \| _ {F} ^ {2} + 2 L _ {1} ^ {2} \delta_ {t} ^ {2}.
$$
Proof. By letting  $\mathbf{x}_t = \hat{\mathbf{B}}_t$ ,  $\mathbf{x} = \hat{\mathbf{H}} \triangleq \frac{2}{L_1} (\mathbf{H} - \frac{L_1}{2}\mathbf{I})$ ,  $\mathbf{g}_t = \mathbf{G}_t \triangleq \frac{2}{L_1}\nabla \ell_t(\mathbf{B}_t)$ ,  $\tilde{\mathbf{g}}_t = \tilde{\mathbf{G}}_t$ ,  $\mathbf{w}_t = \mathbf{W}_t$  in Lemma 10, we obtain:
(i)  $\hat{\mathbf{B}}_t\in \hat{\mathcal{Z}}$  , which means that  $\| \hat{\mathbf{B}}_t\|_{\mathrm{op}}\leq 1$
(ii) It holds that
$$
\langle \mathbf {G} _ {t}, \hat {\mathbf {B}} _ {t} - \hat {\mathbf {H}} \rangle \leq \frac {1}{2 \rho} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} \| _ {F} ^ {2} - \frac {1}{2 \rho} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} \| _ {F} ^ {2} + \frac {\rho}{2} \| \tilde {\mathbf {G}} _ {t} \| _ {F} ^ {2} + \max  \{0, - \langle \mathbf {G} _ {t}, \hat {\mathbf {B}} _ {t} \rangle \} \delta_ {t}, \tag {75}
$$
$$
\left\| \tilde {\mathbf {G}} _ {t} \right\| _ {F} \leq \left\| \mathbf {G} _ {t} \right\| _ {F} + | \langle \mathbf {G} _ {t}, \hat {\mathbf {B}} _ {t} \rangle | \| \mathbf {S} _ {t} \| _ {F}. \tag {76}
$$
First, note that  $\| \mathbf{S}_t\| _F\leq 3$  by Definition 2 and  $|\langle \mathbf{G}_t,\hat{\mathbf{B}}_t\rangle |\leq \| \mathbf{G}_t\|_*\| \hat{\mathbf{B}}_t\|_{\mathrm{op}}\leq \| \mathbf{G}_t\|_*$ . Together with (76), we get
$$
\left\| \tilde {\mathbf {G}} _ {t} \right\| _ {F} \leq \left\| \mathbf {G} _ {t} \right\| _ {F} + 3 \left\| \mathbf {G} _ {t} \right\| _ {*} \leq 4 \left\| \mathbf {G} _ {t} \right\| _ {*} \leq \frac {1 6}{L _ {1}} \sqrt {\ell_ {t} (\mathbf {B} _ {t})}, \tag {77}
$$
where we used the fact that  $\mathbf{G}_t = \frac{2}{L_1}\nabla \ell_t(\mathbf{B}_t)$  and Lemma 15 in the last inequality. Furthermore, since  $\ell_t$  is convex, we have
$$
\ell_ {t} (\mathbf {B} _ {t}) - \ell_ {t} (\mathbf {H}) \leq \langle \nabla \ell_ {t} (\mathbf {B} _ {t}), \mathbf {B} _ {t} - \mathbf {H} \rangle = \left(\frac {L _ {1}}{2}\right) ^ {2} \langle \mathbf {G} _ {t}, \hat {\mathbf {B}} _ {t} - \hat {\mathbf {H}} \rangle ,
$$
where we used  $\mathbf{G}_t = \frac{2}{L_1}\nabla \ell_t(\mathbf{B}_t)$ ,  $\hat{\mathbf{B}}_t \triangleq \frac{2}{L_1} (\mathbf{B}_t - \frac{L_1}{2}\mathbf{I})$ , and  $\hat{\mathbf{H}} \triangleq \frac{2}{L_1} (\mathbf{H} - \frac{L_1}{2}\mathbf{I})$ . Therefore, by combining (75) and (77) we get
$$
\begin{array}{l} \ell_ {t} \left(\mathbf {B} _ {t}\right) - \ell_ {t} (\mathbf {H}) \leq \frac {L _ {1} ^ {2}}{8 \rho} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} \| _ {F} ^ {2} - \frac {L _ {1} ^ {2}}{8 \rho} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} \| _ {F} ^ {2} + \frac {\rho}{8} L _ {1} ^ {2} \| \tilde {\mathbf {G}} _ {t} \| _ {F} ^ {2} + \frac {L _ {1} ^ {2}}{4} \| \mathbf {G} _ {t} \| _ {*} \delta_ {t} (78) \\ \leq \frac {L _ {1} ^ {2}}{8 \rho} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} \| _ {F} ^ {2} - \frac {L _ {1} ^ {2}}{8 \rho} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} \| _ {F} ^ {2} + 3 2 \rho \ell_ {t} (\mathbf {B} _ {t}) + L _ {1} \sqrt {\ell_ {t} (\mathbf {B} _ {t})} \delta_ {t}. (79) \\ \end{array}
$$
Note that  $\ell_t(\mathbf{B}_t)$  appears on both sides of (79). By further applying Lemma 16, we obtain
$$
\ell_ {t} (\mathbf {B} _ {t}) \leq 2 \ell_ {t} (\mathbf {H}) + \frac {L _ {1} ^ {2}}{4 \rho} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} \| _ {F} ^ {2} - \frac {L _ {1} ^ {2}}{4 \rho} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} \| _ {F} ^ {2} + 6 4 \rho \ell_ {t} (\mathbf {B} _ {t}) + L _ {1} ^ {2} \delta_ {t} ^ {2}.
$$
Since  $\rho = 1 / 128$ , by rearranging and simplifying terms in the above inequality, we obtain
Proof of Lemma 12. We let  $\mathbf{H}_t = \nabla^2 f(\mathbf{y}_t)$  for  $t = 0,1,\ldots ,T - 1$ . Thus, we get
$$
\begin{array}{l} \ell_ {t} (\mathbf {B} _ {t}) \leq 4 \ell_ {t} (\mathbf {H} _ {t}) + 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} _ {t} \| _ {F} ^ {2} - 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t} \| _ {F} ^ {2} + 2 L _ {1} ^ {2} \delta_ {t} ^ {2} \\ = 4 \ell_ {t} (\mathbf {H} _ {t}) + 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} _ {t} \| _ {F} ^ {2} - 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \| _ {F} ^ {2} + 2 L _ {1} ^ {2} \delta_ {t} ^ {2} \\ + 6 4 L _ {1} ^ {2} \left(\| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \| _ {F} ^ {2} - \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t} \| _ {F} ^ {2}\right). \\ \end{array}
$$
Furthermore, note that
$$
\begin{array}{l} \left\| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \right\| _ {F} ^ {2} - \left\| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t} \right\| _ {F} ^ {2} \\ = \left(\| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \| _ {F} + \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t} \| _ {F}\right) \left(\| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \| _ {F} - \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t} \| _ {F}\right) \\ \leq 4 \sqrt {d} \| \hat {\mathbf {H}} _ {t + 1} - \hat {\mathbf {H}} _ {t} \| _ {F} = \frac {8 \sqrt {d}}{L _ {1}} \| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F}, \\ \end{array}
$$
where in the last inequality we used the fact that  $\hat{\mathbf{H}}_t, \hat{\mathbf{H}}_{t+1}, \mathbf{W}_{t+1} \in \mathcal{B}_{\sqrt{d}}(0)$  and the triangle inequality. Therefore, we get
$$
\ell_ {t} (\mathbf {B} _ {t}) \leq 4 \ell_ {t} (\mathbf {H} _ {t}) + 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t} - \hat {\mathbf {H}} _ {t} \| _ {F} ^ {2} - 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {t + 1} - \hat {\mathbf {H}} _ {t + 1} \| _ {F} ^ {2} + 2 L _ {1} ^ {2} \delta_ {t} ^ {2} + 5 1 2 L _ {1} \sqrt {d} \| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F}.
$$
By summing the above inequality from  $t = 0$  to  $T - 1$ , we get
$$
\sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {B} _ {t}) \leq 6 4 L _ {1} ^ {2} \| \mathbf {W} _ {0} - \hat {\mathbf {H}} _ {0} \| _ {F} ^ {2} + 4 \sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {H} _ {t}) + 2 L _ {1} ^ {2} \sum_ {t = 0} ^ {T - 1} \delta_ {t} ^ {2} + 5 1 2 L _ {1} \sqrt {d} \sum_ {t = 0} ^ {T - 1} \| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F}.
$$
Finally, we use the fact that  $\mathbf{W}_0 \triangleq \frac{2}{L_1} (\mathbf{B}_0 - \frac{L_1}{2}\mathbf{I})$ , and  $\hat{\mathbf{H}}_0 \triangleq \frac{2}{L_1} (\mathbf{H}_0 - \frac{L_1}{2}\mathbf{I})$  to obtain Lemma 12.
D.3 Proof of Lemma 13
By Assumption 2, we have  $\| \mathbf{w}_t - \mathbf{H}_t\mathbf{s}_t\| = \| \nabla f(\tilde{\mathbf{x}}_{t + 1}) - \nabla f(\mathbf{y}_t) - \nabla f(\mathbf{y}_t)(\tilde{\mathbf{x}}_{t + 1} - \mathbf{y}_t)\| \leq \frac{L_2}{2}\|\tilde{\mathbf{x}}_{t + 1} - \mathbf{y}_t\|^2$ . Thus,
$$
\ell_ {t} (\mathbf {H} _ {t}) = \frac {\| \mathbf {w} _ {t} - \mathbf {H} _ {t} \mathbf {s} _ {t} \| ^ {2}}{\| \mathbf {s} _ {t} \| ^ {2}} \leq \frac {L _ {2} ^ {2}}{4} \| \tilde {\mathbf {x}} _ {t + 1} - \mathbf {y} _ {t} \| ^ {2} \leq \frac {(1 + \alpha_ {1}) ^ {2} L _ {2} ^ {2}}{4 \beta^ {2} (1 - \alpha_ {1}) ^ {2}} \| \hat {\mathbf {x}} _ {t + 1} - \mathbf {y} _ {t} \| ^ {2},
$$
where we used Lemma 1 in the last inequality. Also, Since  $a_{k} \geq \eta_{k}$  for all  $k \geq 0$ , by (21) we get
$$
\sum_ {k = 0} ^ {N - 1} \left\| \hat {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k} \right\| ^ {2} \leq \sum_ {k = 0} ^ {N - 1} \frac {a _ {k} ^ {2}}{\eta_ {k} ^ {2}} \left\| \hat {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k} \right\| ^ {2} \leq \frac {1}{1 - \sigma^ {2}} \left\| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \right\| ^ {2}.
$$
Hence, we have
$$
\begin{array}{l} \sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {H} _ {t}) \leq \frac {(1 + \alpha_ {1}) ^ {2} L _ {2} ^ {2}}{4 \beta^ {2} (1 - \alpha_ {1}) ^ {2}} \sum_ {k \in \mathcal {B}} \| \hat {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k} \| ^ {2} \leq \frac {(1 + \alpha_ {1}) ^ {2} L _ {2} ^ {2}}{4 \beta^ {2} (1 - \alpha_ {1}) ^ {2}} \sum_ {k = 0} ^ {N - 1} \| \hat {\mathbf {x}} _ {k + 1} - \mathbf {y} _ {k} \| ^ {2} \\ \leq \frac {\left(1 + \alpha_ {1}\right) ^ {2} L _ {2} ^ {2} \left\| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \right\| ^ {2}}{4 \beta^ {2} \left(1 - \alpha_ {1}\right) ^ {2} \left(1 - \sigma^ {2}\right)}, \\ \end{array}
$$
which proves the first inequality in (72).
Furthermore, by Assumption 2, we have
$$
\| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \| _ {F} = \| \nabla^ {2} f (\mathbf {y} _ {t + 1}) - \nabla^ {2} f (\mathbf {y} _ {t}) \| _ {F} \leq \sqrt {d} \| \nabla^ {2} f (\mathbf {y} _ {t + 1}) - \nabla^ {2} f (\mathbf {y} _ {t}) \| _ {\mathrm {o p}} \leq \sqrt {d} L _ {2} \| \mathbf {y} _ {t + 1} - \mathbf {y} _ {t} \|.
$$
Hence, by using the triangle inequality, we can bound
$$
\sum_ {t = 0} ^ {T - 1} \left\| \mathbf {H} _ {t + 1} - \mathbf {H} _ {t} \right\| _ {F} \leq \sqrt {d} L _ {2} \sum_ {k = 0} ^ {N - 1} \left\| \mathbf {y} _ {k + 1} - \mathbf {y} _ {k} \right\| \leq \sqrt {d} L _ {2} C _ {2} \left(1 + \log \frac {A _ {N}}{A _ {1}}\right) \left\| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \right\|,
$$
where we used Lemma 8 in the last inequality.
D.4 Proof of Lemma 14
Before presenting the proof of Lemma 14, we start with a helper lemma that shows a lower bound on  $A_{1}$ .
Lemma 18. We have  $A_{1} = \hat{\eta}_{0}\geq \min \{\sigma_{0},\frac{\alpha_{2}\beta}{L_{1}}\}$
Proof. The equality  $A_{1} = \hat{\eta}_{0}$  is shown in the proof of Lemma 3. To show the lower bound on  $\hat{\eta}_{0}$ , we use Lemma 1 and separate two cases. If  $0 \notin \mathcal{B}$ , then we have  $\hat{\eta}_{0} = \eta_{0} = \sigma_{0}$ . Otherwise, if  $0 \in \mathcal{B}$ , then we have
$$
\hat {\eta} _ {0} \geq \frac {\alpha_ {2} \beta \| \tilde {\mathbf {x}} _ {1} - \mathbf {y} _ {0} \|}{\| \nabla f (\tilde {\mathbf {x}} _ {1}) - \nabla f (\mathbf {y} _ {0}) - \mathbf {B} _ {k} (\tilde {\mathbf {x}} _ {1} - \mathbf {y} _ {0}) \|}.
$$
Moreover, as shown in the proof of Lemma 11, we have  $\| \nabla f(\tilde{\mathbf{x}}_1) - \nabla f(\mathbf{y}_0) - \mathbf{B}_k(\tilde{\mathbf{x}}_1 - \mathbf{y}_0)\| \leq L_1\| \tilde{\mathbf{x}}_1 - \mathbf{y}_0\|$ , which further implies that  $\hat{\eta}_0 \geq \frac{\alpha_2\beta}{L_1}$ . This completes the proof.
We combine Lemma 12 and Lemma 13 to get
$$
\begin{array}{l} \sum_ {k \in \mathcal {B}} \frac {\| \mathbf {w} _ {k} - \mathbf {B} _ {k} \mathbf {s} _ {k} \| ^ {2}}{\| \mathbf {s} _ {k} \| ^ {2}} = \sum_ {t = 0} ^ {T - 1} \ell_ {t} (\mathbf {B} _ {t}) \leq 2 5 6 \| \mathbf {B} _ {0} - \mathbf {H} _ {0} \| _ {F} ^ {2} + C _ {3} L _ {2} ^ {2} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2} + 2 L _ {1} ^ {2} \sum_ {t = 0} ^ {T - 1} \delta_ {t} ^ {2} \\ + 5 1 2 C _ {2} L _ {1} L _ {2} d \left(1 + \log \frac {A _ {N}}{A _ {1}}\right) \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \|. \\ \end{array}
$$
Since  $\delta_t = 1 / (\sqrt{t + 2}\ln (t + 2))$ , we have
$$
\sum_ {t = 0} ^ {T - 1} \delta_ {t} ^ {2} = \sum_ {t = 2} ^ {T + 1} \frac {1}{t \ln^ {2} t} \leq \frac {1}{2 \ln^ {2} 2} + \int_ {2} ^ {T + 1} \frac {1}{t \ln^ {2} t} d t = \frac {1}{2 \ln^ {2} 2} + \frac {1}{\ln 2} - \frac {1}{\ln (T + 1)} \leq 2. 5.
$$
Hence, it further follows from (14) and Lemma 2 that
$$
\begin{array}{l} \frac {N ^ {5}}{A _ {N} ^ {2}} \leq 4 C _ {1} ^ {2} \sum_ {k = 0} ^ {N - 1} \frac {1}{\hat {\eta} _ {k} ^ {2}} \\ \leq \frac {4 C _ {1} ^ {2} (2 - \beta^ {2})}{(1 - \beta^ {2}) \sigma_ {0} ^ {2}} + \frac {4 C _ {1} ^ {2} (2 - \beta^ {2})}{(1 - \beta^ {2}) \alpha_ {2} ^ {2} \beta^ {2}} \sum_ {k \in \mathcal {B}} \frac {\| \mathbf {w} _ {k} - \mathbf {B} _ {k} \mathbf {s} _ {k} \| ^ {2}}{\| \mathbf {s} _ {k} \| ^ {2}} \\ \leq \frac {C _ {6}}{\sigma_ {0} ^ {2}} + C _ {7} L _ {1} ^ {2} + C _ {8} \| \mathbf {B} _ {0} - \mathbf {H} _ {0} \| _ {F} ^ {2} + C _ {9} L _ {2} ^ {2} \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| ^ {2} \\ + C _ {1 0} L _ {1} L _ {2} d \left(1 + \log \frac {A _ {N}}{A _ {1}}\right) \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \|. \tag {80} \\ \end{array}
$$
To simplify the notation, define
and the inequality in (80) becomes  $\frac{N^5}{A_N^2} \leq M + C_{10}L_1L_2d\| \mathbf{z}_0 - \mathbf{x}^* \| \log \frac{A_N}{A_1}$ . Let  $A_N^*$  be the number that achieves the equality
$$
\frac {N ^ {5}}{(A _ {N} ^ {*}) ^ {2}} = M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log \frac {A _ {N} ^ {*}}{A _ {1}},
$$
and we can see that  $A_N \geq A_N^*$ . Thus, we instead try to construct a lower bound on  $A_N^*$ . If  $A_N^* \leq A_1$ , then  $\log (A_N^* / A_1) \leq 0$  and furthermore
$$
\frac {N ^ {5}}{\left(A _ {N} ^ {*}\right) ^ {2}} \leq M \quad \Rightarrow \quad \frac {1}{A _ {N}} \leq \frac {\sqrt {M}}{N ^ {2 . 5}}. \tag {81}
$$
Otherwise, assume that  $A_N^* > A_1$ . Then  $\log (A_N^* / A_1) > 0$  and we first show an upper bound on  $A_N^*$ :
$$
\frac {N ^ {5}}{(A _ {N} ^ {*}) ^ {2}} = M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log \frac {A _ {N} ^ {*}}{A _ {1}} \geq M \quad \Rightarrow \quad A _ {N} ^ {*} \leq \frac {1}{\sqrt {M}} N ^ {2. 5}.
$$
This in turn leads to a lower bound on  $A_N^*$ :
$$
\frac {N ^ {5}}{(A _ {N} ^ {*}) ^ {2}} = M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log \frac {A _ {N} ^ {*}}{A _ {1}} \leq M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log \left(\frac {\max \{\frac {L _ {1}}{\alpha_ {2} \beta} , \frac {1}{\sigma_ {0}} \} N ^ {2 . 5}}{\sqrt {M}}\right),
$$
where we also used the fact that  $A_{1} \geq \min \left\{\sigma_{0}, \frac{\alpha_{2}\beta}{L_{1}}\right\}$  (cf. Lemma 18). Thus, we get
$$
\frac {1}{A _ {N}} \leq \frac {1}{A _ {N} ^ {*}} \leq \frac {1}{N ^ {2 . 5}} \left(M + C _ {1 0} L _ {1} L _ {2} d \| \mathbf {z} _ {0} - \mathbf {x} ^ {*} \| \log \left(\frac {\operatorname* {m a x} \left\{\frac {L _ {1}}{\alpha_ {2} \beta} , \frac {1}{\sigma_ {0}} \right\} N ^ {2 . 5}}{\sqrt {M}}\right)\right) ^ {\frac {1}{2}}. \tag {82}
$$
Combining both cases in (81) and (82), we conclude the proof of Lemma 14.
Subroutine 3 LinearSolver(A, b;  $\alpha$ )
1: Input:  $\mathbf{A} \in \mathbb{S}_+^d$ ,  $\mathbf{b} \in \mathbb{R}^d$ ,  $0 < \alpha < 1$   
2: Initialize:  $\mathbf{s}_0 \gets 0$ ,  $\mathbf{r}_0 \gets \mathbf{b} - \mathbf{A}\mathbf{s}_0$ ,  $\mathbf{p}_0 \gets \mathbf{r}_0$   
3: for  $k = 0, 1, \ldots$  do  
4: if  $\|\mathbf{r}_k\|_2 \leq \alpha\|\mathbf{s}_k\|_2$  then  
5: Return  $\mathbf{s}_k$   
6: end if  
7:  $\alpha_k \gets \langle \mathbf{r}_k, \mathbf{A}\mathbf{r}_k \rangle / \langle \mathbf{A}\mathbf{p}_k, \mathbf{A}\mathbf{p}_k \rangle$   
8:  $\mathbf{s}_{k+1} \gets \mathbf{s}_k + \alpha_k\mathbf{p}_k$   
9:  $\mathbf{r}_{k+1} \gets \mathbf{r}_k - \alpha_k\mathbf{A}\mathbf{p}_k$   
10: Compute and store  $\mathbf{A}\mathbf{r}_{k+1}$   
11:  $\beta_k \gets \langle \mathbf{r}_{k+1}, \mathbf{A}\mathbf{r}_{k+1} \rangle / \langle \mathbf{r}_k, \mathbf{A}\mathbf{r}_k \rangle$   
12:  $\mathbf{p}_{k+1} \gets \mathbf{r}_{k+1} + \beta_k\mathbf{p}_k$   
13: Compute and store  $\mathbf{A}\mathbf{p}_{k+1} \gets \mathbf{A}\mathbf{r}_{k+1} + \beta_k\mathbf{A}\mathbf{p}_k$   
14: end for
E Characterizing the Computational Cost
In this section, we first specify the implementation details of the LinearSolver oracle in Definition 1 and the SEP oracle in Definition 2. Then in Section E.3, we present the proof of Theorem 2.
E.1 Implementation of the LinearSolver Oracle
We implement the LinearSolver oracle by running the conjugate residual (CR) method [48] to solve the linear system  $\mathbf{A}\mathbf{s} = \mathbf{b}$ . In particular, we initialize the CR method with  $\mathbf{s}_0 = 0$  and returns the iterate  $\mathbf{s}_k$  once we achieve  $\| \mathbf{A}\mathbf{s}_k - \mathbf{b}\| \leq \alpha \| \mathbf{s}_k\|$ . The following lemma provides the convergence guarantee of the CR method, which will be later used in the proof of Theorem 2.
Lemma 19 ([56, Chapter 12.4]). Let  $\mathbf{s}^*$  be any optimal solution of  $\mathbf{A}\mathbf{s}^{*} = \mathbf{b}$  and let  $\{\mathbf{s}_k\}$  be the iterates generated by Subroutine 3. Then we have
$$
\| \mathbf {r} _ {k} \| _ {2} = \| \mathbf {A s} _ {k} - \mathbf {b} \| _ {2} \leq \frac {\lambda_ {\max } (\mathbf {A}) \| \mathbf {s} ^ {*} \| _ {2}}{(k + 1) ^ {2}}.
$$