SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\eta \left(\| \widehat {③} \| _ {2} + \| \widehat {④} \| _ {2} + \| \widehat {⑥} \| _ {2} + \| \widehat {⑦} \| _ {2}\right) \leq \frac {1}{2} c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}}.
$$
if  $n^2 p \gtrsim \mu^5 n \log^{20} n$ . Combining all, we have
$$
\begin{array}{l} \left\| \boldsymbol {x} ^ {(s + 1)} - \boldsymbol {x} ^ {(s + 1, l)} \right\| _ {2} \leq \left(1 - \eta \left\| \widetilde {\boldsymbol {x}} ^ {(s)} \right\| _ {2} ^ {2} + \frac {c _ {4}}{\log^ {2} n}\right) \left\| \boldsymbol {x} ^ {(s)} - \boldsymbol {x} ^ {(s, l)} \right\| _ {2} + \frac {1}{2} c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} \\ \leq \left(1 + \frac {c _ {4}}{\log^ {2} n}\right) \left\| \boldsymbol {x} ^ {(s)} - \boldsymbol {x} ^ {(s, l)} \right\| _ {2} + \frac {1}{2} c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}}. \\ \end{array}
$$
for all  $s \leq t$ . An analysis on the recursive equation
$$
x _ {s + 1} = \left(1 + \frac {c _ {4}}{\log^ {2} n}\right) x _ {s} + \frac {1}{2} c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}}, x _ {0} = 0
$$
gives the desired bound
$$
\begin{array}{l} \left\| \boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)} \right\| _ {2} \leq \frac {\log^ {2} n}{c _ {4}} \left(\left(1 + \frac {c _ {4}}{\log^ {2} n}\right) ^ {t + 1} - 1\right) \frac {1}{2} c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} \\ \leq c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (t + 1), \\ \end{array}
$$
where we used basic inequalities  $(1 + x)^{a} \leq e^{ax}$  and  $e^{ax} - 1 \leq 2ax$  which hold if  $x$  is small and  $ax$  is small, respectively.
(D.9) at  $(t + 1)$  We decompose  $x^{(t + 1)} - x^{(t + 1,l)}$  as
$$
\begin{array}{l} \boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)} \\ = (\boldsymbol {x} - \eta \nabla f (\boldsymbol {x})) - \left(\boldsymbol {x} ^ {(l)} - \eta \nabla f ^ {(l)} \left(\boldsymbol {x} ^ {(l)}\right)\right) \\ = (\boldsymbol {x} - \eta \nabla g (\boldsymbol {x})) - \left(\boldsymbol {x} ^ {(l)} - \eta \nabla g ^ {(l)} (\boldsymbol {x} ^ {(l)})\right) + \eta \left(\boldsymbol {M} ^ {\circ} \boldsymbol {x} - \boldsymbol {M} ^ {(l)} \boldsymbol {x} ^ {(l)}\right) \\ = (\boldsymbol {x} - \eta \nabla g (\boldsymbol {x})) - \left(\boldsymbol {x} ^ {(l)} - \eta \nabla g (\boldsymbol {x} ^ {(l)})\right) - \eta \left(\nabla g (\boldsymbol {x} ^ {(l)}) - \nabla g ^ {(l)} (\boldsymbol {x} ^ {(l)})\right) \\ + \eta (\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}) \boldsymbol {x} + \eta \boldsymbol {M} ^ {(l)} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \\ = \int_ {0} ^ {1} (\boldsymbol {I} - \eta \nabla^ {2} g (\boldsymbol {x} ^ {(l)} (\tau)) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \mathrm {d} \tau - \eta \left(\frac {1}{p} \mathcal {P} _ {\Omega_ {l}} (\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) \top}) - \mathcal {P} _ {l} (\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) \top})\right) \boldsymbol {x} ^ {(l)} \\ + \eta \boldsymbol {M} ^ {(l)} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \eta \left(\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \eta \left(\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}\right) \boldsymbol {x} ^ {(l)} \\ = \left(1 - \eta \| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2}\right) \left(\boldsymbol {x} - \boldsymbol {x} ^ {(l)}\right) - 2 \eta \widetilde {\boldsymbol {x}} \widetilde {\boldsymbol {x}} ^ {\top} \left(\boldsymbol {x} - \boldsymbol {x} ^ {(l)}\right) \\ - \eta \int_ {0} ^ {1} \left(\nabla^ {2} g (\boldsymbol {x} ^ {(l)} (\tau)) - \left(\| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2} \boldsymbol {I} + 2 \widetilde {\boldsymbol {x}} \widetilde {\boldsymbol {x}} ^ {\top}\right)\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \mathrm {d} \tau \\ - \eta \left(\frac {1}{p} \mathcal {P} _ {\Omega_ {l}} \left(\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) \top}\right) - \mathcal {P} _ {l} \left(\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) \top}\right)\right) \boldsymbol {x} ^ {(l)} \\ + \eta \boldsymbol {M} ^ {(l)} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \eta \left(\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \eta \left(\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}\right) \boldsymbol {x} ^ {(l)}, \\ \end{array}
$$
where  $\pmb{x}^{(l)}(\tau) = \pmb{x}^{(l)} + \tau (\pmb{x} - \pmb{x}^{(l)})$ . Then, we take inner product with  $\pmb{u}^{(l)}$  on both sides.
$$
\begin{array}{l} \boldsymbol {u} ^ {(l) \top} \left(\boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)}\right) = \left(1 - \eta \| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2}\right) \boldsymbol {u} ^ {(l) \top} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) - 2 \eta \underbrace {\boldsymbol {u} ^ {(l) \top} \widetilde {\boldsymbol {x}} \widetilde {\boldsymbol {x}} ^ {\top} \left(\boldsymbol {x} - \boldsymbol {x} ^ {(l)}\right)} _ {①} \\ - \eta \underbrace {\int_ {0} ^ {1} \boldsymbol {u} ^ {(l) \top} \left(\nabla^ {2} g (\boldsymbol {x} (\tau)) - \left(\| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2} \boldsymbol {I} + 2 \widetilde {\boldsymbol {x}} \widetilde {\boldsymbol {x}} ^ {\top}\right)\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \mathrm {d} \tau} _ {②} \\ - \eta \underbrace {\boldsymbol {u} ^ {(l) ^ {\top}} \left(\frac {1}{p} \mathcal {P} _ {\Omega_ {l}} \left(\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) ^ {\top}}\right) - \mathcal {P} _ {l} \left(\boldsymbol {x} ^ {(l)} \boldsymbol {x} ^ {(l) ^ {\top}}\right)\right) \boldsymbol {x} ^ {(l)}} _ {③} \\ + \eta \lambda^ {(l)} \boldsymbol {u} ^ {(l) \top} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \eta \underbrace {\boldsymbol {u} ^ {(l) \top} \left(M ^ {\circ} - M ^ {(l)}\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)})} _ {④} \\ + \eta \underbrace {\boldsymbol {u} ^ {(l) \top} \left(\boldsymbol {M} ^ {\circ} - \boldsymbol {M} ^ {(l)}\right) \boldsymbol {x} ^ {(l)}} _ {⑤} \\ \end{array}
$$
For the term ①, we have
$$
\left| \boldsymbol {u} ^ {(l) \top} \widetilde {\boldsymbol {x}} \right| \leq \left| \boldsymbol {u} ^ {(l) \top} \boldsymbol {x} ^ {(0)} \right| + (1 + \eta \lambda^ {\star}) ^ {t} \left| \boldsymbol {u} ^ {\star \top} \boldsymbol {x} ^ {(0)} \right| \left| \boldsymbol {u} ^ {(l) \top} \boldsymbol {u} ^ {\star} \right| \lesssim \sqrt {\frac {\log n}{n}} (1 + \eta \lambda^ {\star}) ^ {t} \beta_ {0}
$$
by Lemma A.5, and thus,
$$
\begin{array}{l} \left| ① \right| \leq \left| \boldsymbol {u} ^ {(l) \top} \widetilde {\boldsymbol {x}} \right| \| \widetilde {\boldsymbol {x}} \| _ {2} \left\| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \right\| _ {2} \lesssim \sqrt {\frac {\log n}{n}} (1 + \eta \lambda^ {\star}) ^ {t} \beta_ {0} ^ {2} \left\| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \right\| _ {2} \\ \lesssim \mu \sqrt {\frac {\log^ {3} n}{n p}} \frac {\beta_ {0} ^ {3}}{n} (1 + \eta \lambda^ {\star}) ^ {t} t \\ \lesssim \lambda^ {\star} \sqrt {\frac {\mu}{n p}} \frac {\beta_ {0}}{n}. \\ \end{array}
$$
The definition of Phase I was used to bound  $(1 + \eta \lambda^{\star})^{t}t$  in deriving the last line. We use (D.13) to get
$$
\begin{array}{l} | ② | \lesssim \int_ {0} ^ {1} \| \nabla^ {2} g (\boldsymbol {x} (\tau)) - \left(\| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2} \boldsymbol {I} + 2 \widetilde {\boldsymbol {x}} \widetilde {\boldsymbol {x}} ^ {\top}\right) \| \| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \| _ {2} \| \boldsymbol {u} ^ {(l)} \| _ {2} d \tau \\ \lesssim \sqrt {\frac {\mu^ {3} \log^ {7} n}{n p}} \beta_ {0} ^ {2} \cdot \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} t \lesssim \lambda^ {\star} \sqrt {\frac {\mu}{n p \log^ {1 5} n}} \frac {\beta_ {0}}{n}. \\ \end{array}
$$
We apply Lemma G.12 to ③ to yield
$$
\begin{array}{l} \left| ③ \right| \lesssim \left\| \boldsymbol {x} ^ {(l)} \right\| _ {\infty} ^ {2} \sqrt {\frac {\log n}{p}} \left(\left\| \boldsymbol {u} ^ {(l)} \right\| _ {2} \left\| \boldsymbol {x} ^ {(l)} \right\| _ {\infty} + \left\| \boldsymbol {x} ^ {(l)} \right\| _ {2} \left\| \boldsymbol {u} ^ {(l)} \right\| _ {\infty}\right) \\ \lesssim \sqrt {\frac {\mu \log^ {4} n}{n p}} \frac {\beta_ {0} ^ {3}}{n} \lesssim \lambda^ {\star} \sqrt {\frac {1}{\mu^ {3} n \log^ {2 2} n}} \frac {\beta_ {0}}{n}. \\ \end{array}
$$
We divide ④ into two terms that are related to sampling and noise, respectively.
$$
④ = \boldsymbol {u} ^ {(l) \top} \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {M} ^ {\star}) - \mathcal {P} _ {\Omega} ^ {(l)} (\boldsymbol {M} ^ {\star})\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) + \boldsymbol {u} ^ {(l) \top} \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {E}) - \boldsymbol {E} ^ {(l)}\right) (\boldsymbol {x} - \boldsymbol {x} ^ {(l)})
$$
Then, Cauchy-Schwartz inequality is applied to yield
$$
\left| ④ \right| \leq \left\| \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {M} ^ {\star}) - \mathcal {P} _ {\Omega} ^ {(l)} (\boldsymbol {M} ^ {\star})\right) \boldsymbol {u} ^ {(l)} \right\| _ {2} \left\| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \right\| _ {2} + \left\| \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {E}) - \boldsymbol {E} ^ {(l)}\right) \boldsymbol {u} ^ {(l)} \right\| _ {2} \left\| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \right\| _ {2}.
$$
Applying Lemmas G.11 and G.13 to the two terms, respectively, we get
$$
| ④ | \lesssim \lambda^ {\star} \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {1}{\sqrt {n}} \left\| \boldsymbol {x} - \boldsymbol {x} ^ {(l)} \right\| _ {2} \lesssim \lambda^ {\star} \frac {\sqrt {\mu^ {5} \log^ {5} n}}{n p} \frac {\beta_ {0}}{n}.
$$
For the term  $⑤$  , we decompose it into two terms as for  $④$
$$
\left| \widehat {⑤} \right| \leq \left| \boldsymbol {u} ^ {(l) \top} \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {M} ^ {\star}) - \mathcal {P} _ {\Omega} ^ {(l)} (\boldsymbol {M} ^ {\star})\right) \boldsymbol {x} ^ {(l)} \right| + \left| \boldsymbol {u} ^ {(l) \top} \left(\frac {1}{p} \mathcal {P} _ {\Omega} (\boldsymbol {E}) - \boldsymbol {E} ^ {(l)}\right) \boldsymbol {x} ^ {(l)} \right|
$$
Then, we apply Lemmas G.12 and G.14 to each term to obtain
$$
| \overline {{⑤}} | \lesssim \lambda^ {\star} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \frac {\beta_ {0}}{n}.
$$
Combining all, there exists a universal constant  $c_6 > 0$  such that
$$
\eta \left(2 | ① | + | ② | + | ③ | + | ④ | + | ⑤ |\right) \leq \frac {\eta \lambda^ {\star}}{2} c _ {6} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \frac {\beta_ {0}}{n}
$$
if  $n^2 p \gtrsim \mu^2 n\log^3 n$ , and there exists a universal constant  $c_7 > 0$  such that
$$
\eta \lambda^ {(l)} \leq \eta \lambda^ {\star} + \frac {c _ {7}}{\log^ {2} n}
$$
by (A.2) if  $n^2 p \gtrsim \mu^2 n \log^5 n$ . Finally, we have
$$
\begin{array}{l} \left| \boldsymbol {u} ^ {(l) \top} \left(\boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)}\right) \right| \leq \left(1 - \eta \| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2} + \eta \lambda^ {(l)}\right) \left| \boldsymbol {u} ^ {(l) \top} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \right| + \frac {\eta \lambda^ {\star}}{2} c _ {6} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \frac {\beta_ {0}}{n} \\ \leq \left(1 + \eta \lambda^ {\star} + \frac {c _ {7}}{\log^ {2} n}\right) \left| \boldsymbol {u} ^ {(l) \top} (\boldsymbol {x} - \boldsymbol {x} ^ {(l)}) \right| + \frac {\eta \lambda^ {\star}}{2} c _ {6} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \frac {\beta_ {0}}{n}. \\ \end{array}
$$
$$
x _ {t + 1} = \left(1 + \eta \lambda^ {\star} + \frac {c _ {7}}{\log^ {2} n}\right) x _ {t} + \frac {\eta \lambda^ {\star}}{2} c _ {6} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \frac {\beta_ {0}}{n}, \quad x _ {0} = 0
$$
givesthebound
$$
\left| \boldsymbol {u} ^ {(l) \top} \left(\boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)}\right) \right| \leq c _ {6} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} (1 + \eta \lambda^ {\star}) ^ {t + 1} \frac {\beta_ {0}}{n}.
$$
(D.10) at  $(t + 1)$  We decompose  $(\pmb{x}^{(t + 1,l)} - \widetilde{\pmb{x}}^{(t + 1)})_l$  as
$$
(\boldsymbol {x} ^ {(t + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}) _ {l} = \left(1 - \eta \| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2}\right) (\boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}}) _ {l} + \eta \lambda^ {\star} \boldsymbol {u} ^ {\star \top} (\boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}}) u _ {l} ^ {\star} + \eta \left(\| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2} - \left\| \boldsymbol {x} ^ {(l)} \right\| _ {2} ^ {2}\right) x _ {l} ^ {(l)},
$$
$$
\begin{array}{l} \left| \left(\boldsymbol {x} ^ {(t + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}\right) _ {l} \right| \leq \left(1 - \eta \| \widetilde {\boldsymbol {x}} \| _ {2} ^ {2}\right) \left| \left(\boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}}\right) _ {l} \right| \\ + \eta \lambda^ {\star} \left\| \boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}} \right\| _ {2} \| \boldsymbol {u} ^ {\star} \| _ {\infty} + \eta \| \widetilde {\boldsymbol {x}} \| _ {2} \left\| \boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}} \right\| _ {2} \| \widetilde {\boldsymbol {x}} \| _ {\infty}. \\ \end{array}
$$
From (D.5) and (D.8), we have
$$
\left\| \boldsymbol {x} ^ {(l)} - \widetilde {\boldsymbol {x}} \right\| _ {2} \leq \left\| \boldsymbol {x} ^ {(l)} - \boldsymbol {x} \right\| _ {2} + \| \boldsymbol {x} - \widetilde {\boldsymbol {x}} \| _ {2} \leq 3 c _ {0} \mu \sqrt {\frac {\log n}{n p}} \beta_ {0} t.
$$
If  $n$  is sufficiently large, we have
$$
\eta \lambda^ {\star} \| \boldsymbol {u} ^ {\star} \| _ {\infty} + \eta \| \widetilde {\boldsymbol {x}} \| _ {2} \| \widetilde {\boldsymbol {x}} \| _ {\infty} \leq \sqrt {\frac {\mu \log n}{n}}
$$
for all  $t \leq T_1$  because
$$
\eta \lambda^ {\star} \| \pmb {u} ^ {\star} \| _ {\infty} + \eta \| \widetilde {\pmb {x}} \| _ {2} \| \widetilde {\pmb {x}} \| _ {\infty} \lesssim \eta \lambda^ {\star} \sqrt {\frac {\mu}{n}} + \eta \sqrt {\frac {\log^ {2} n}{n}} \beta_ {0} ^ {2} \lesssim \sqrt {\frac {\mu}{n}}.
$$
$$
\left| (\boldsymbol {x} ^ {(s + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(s + 1)}) _ {l} \right| \leq \left| (\boldsymbol {x} ^ {(s, l)} - \widetilde {\boldsymbol {x}} ^ {(s)}) _ {l} \right| + 3 c _ {0} \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} s
$$
for all  $s \leq t$ . Finally, we have
$$
\begin{array}{l} \left| \left(\boldsymbol {x} ^ {(t + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}\right) _ {l} \right| \leq 3 c _ {0} \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} \sum_ {s = 1} ^ {t} s \\ \leq 3 c _ {0} \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (t + 1) ^ {2}. \\ \end{array}
$$
(D.5) at  $(t + 1)$  We can obtain this through the combination of (D.7) and Lemma 5.2.
$$
\begin{array}{l} \left\| \boldsymbol {x} ^ {(t + 1)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)} \right\| _ {2} \leq \left\| \boldsymbol {x} ^ {(t + 1)} - \widehat {\boldsymbol {x}} ^ {(t + 1)} \right\| _ {2} + \left\| \widehat {\boldsymbol {x}} ^ {(t + 1)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)} \right\| _ {2} \\ \leq c _ {2} \left(3 c _ {0} + c _ {5} + 1\right) \frac {1}{\lambda^ {\star}} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \left(1 + \eta \lambda^ {\star}\right) ^ {t + 1} \beta_ {0} ^ {3} + c _ {0} \mu \sqrt {\frac {\log n}{n p}} \beta_ {0} (t + 1) \\ \leq c _ {2} \left(3 c _ {0} + c _ {5} + 1\right) \frac {1}{\lambda^ {\star}} \sqrt {\frac {\mu^ {3} \log^ {3} n}{n p}} \left(1 + \eta \lambda^ {\star}\right) ^ {T _ {1}} \beta_ {0} ^ {3} + c _ {0} \mu \sqrt {\frac {\log n}{n p}} \beta_ {0} (t + 1) \\ \leq c _ {1} c _ {2} (3 c _ {0} + c _ {5} + 1) \mu \sqrt {\frac {1}{n p \log^ {2} n}} \beta_ {0} + c _ {0} \mu \sqrt {\frac {\log n}{n p}} \beta_ {0} (t + 1) \\ \leq 2 c _ {0} \mu \sqrt {\frac {\log n}{n p}} \beta_ {0} (t + 1) \\ \end{array}
$$
(D.6) at  $(t + 1)$  The  $l$ th component of  $\pmb{x}^{(t + 1)} - \widetilde{\pmb{x}}^{(t + 1)}$  is bounded by
$$
\begin{array}{l} \left| \left(\boldsymbol {x} ^ {(t + 1)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}\right) _ {l} \right| \leq \left\| \boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)} \right\| _ {\infty} + \left| \left(\boldsymbol {x} ^ {(t + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}\right) _ {l} \right| \\ \leq \left\| \boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1, l)} \right\| _ {2} + \left| \left(\boldsymbol {x} ^ {(t + 1, l)} - \widetilde {\boldsymbol {x}} ^ {(t + 1)}\right) _ {l} \right| \\ \leq c _ {5} \mu \sqrt {\frac {\log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (t + 1) + 3 c _ {0} \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (t + 1) ^ {2} \\ \leq (3 c _ {0} + c _ {5}) \sqrt {\frac {\mu^ {3} \log^ {2} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (t + 1) ^ {2}. \\ \end{array}
$$
$\mathbf{At} t = T_1$  Because  $T_{1} \lesssim \log n$ , it is implied from Lemma D.2 that at  $t = T_{1}$ , there exists a constant  $c_{7} > 0$  such that
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \leq c _ {7} \mu \sqrt {\frac {\log^ {3} n}{n p}} \beta_ {0},
$$
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \leq c _ {7} \sqrt {\frac {\mu^ {3} \log^ {8} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}}, \tag {D.14}
$$
$$
\left\| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(t, l)} \right\| _ {2} \leq c _ {7} \mu \sqrt {\frac {\log^ {4} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}},
$$
$$
\left| \left(\boldsymbol {x} ^ {(t, l)} - \widetilde {\boldsymbol {x}} ^ {(t)}\right) _ {l} \right| \leq c _ {7} \sqrt {\frac {\mu^ {3} \log^ {8} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}}.
$$
These bounds serve as a base case for the induction of the next part.
E Phase II
This section is mostly devoted to the proof of Lemma E.1 which is a formal version of Lemma 6.1.
Lemma E.1. Suppose that (D.14) holds at  $t = T_1$  and the initialization point  $\pmb{x}^{(0)}$  satisfies (B.1) to (B.6). Then, for all  $T_1 < t \leq T_2$ , we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \leq 2 c _ {7} \mu \sqrt {\frac {\log^ {3} n}{n p}} \beta_ {0} (1 + \eta \lambda^ {\star}) ^ {t - T _ {1}}, \tag {E.1}
$$
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \leq c _ {1 3} \sqrt {\frac {\mu^ {3} \log^ {8} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (1 + \eta \lambda^ {\star}) ^ {t - T _ {1}}, \tag {E.2}
$$
$$
\left\| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(t, l)} \right\| _ {2} \leq 2 c _ {7} \mu \sqrt {\frac {\log^ {5} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (1 + \eta \lambda^ {\star}) ^ {t - T _ {1}}, \tag {E.3}
$$
$$
\left| \left(\boldsymbol {x} ^ {(t, l)} - \widetilde {\boldsymbol {x}} ^ {(t)}\right) _ {l} \right| \leq 3 c _ {7} \sqrt {\frac {\mu^ {3} \log^ {8} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} (1 + \eta \lambda^ {\star}) ^ {t - T _ {1}}, \tag {E.4}
$$
with high probability, where  $T_{2}$  is the largest  $t$  such that  $\widetilde{\beta}_{t}^{2} \leq \lambda^{\star}\left(1 - \frac{1}{\log n}\right)$ , and  $c_{13} > 0$  is a constant.
Proof of Theorems 3.1 and 3.3 We first explain how Theorems 3.1 and 3.3 are derived from Lemmas D.2 and E.1. We first focus on  $\left\| \pmb{x}^{(t)} - \widetilde{\pmb{x}}^{(t)}\right\|_2$ . For  $t \leq T_1$ , from Lemma D.2 and (C.3), we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \mu \sqrt {\frac {\log^ {3} n}{n p}} \beta_ {0} \lesssim \frac {1}{\log n} \beta_ {0} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2}
$$
provided that  $n^2 p \gtrsim \mu^2 n \log^7 n$ . For  $T_1 < t \leq T_2$ , from the definition of  $T_1$  and Lemma E.1, we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \sqrt {\frac {1}{\log^ {1 8} n}} \frac {\beta_ {0}}{\sqrt {n}} (1 + \eta \lambda^ {\star}) ^ {t}.
$$
From the lower bound of Lemma C.1, for all  $t \leq T_2'$ , we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \sqrt {\frac {1}{\log^ {1 8} n}} \left(1 + \frac {(1 + \eta \lambda^ {\star}) ^ {t}}{\sqrt {n}}\right) \beta_ {0} \lesssim \sqrt {\frac {1}{\log^ {1 7} n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2}.
$$
Now, for  $T_2' < t \leq T_2$ , we have
$$
\begin{array}{l} \left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \sqrt {\frac {1}{\log^ {1 8} n}} \left(1 + \frac {(1 + \eta \lambda^ {\star}) ^ {T _ {2} ^ {\prime}}}{\sqrt {n}}\right) \beta_ {0} (1 + \eta \lambda^ {\star}) ^ {t - T _ {2} ^ {\prime}} \\ \lesssim \sqrt {\frac {1}{\log^ {1 7} n}} \left\| \widetilde {\boldsymbol {x}} ^ {(T _ {2} ^ {\prime})} \right\| _ {2} (1 + \eta \lambda^ {\star}) ^ {t - T _ {2} ^ {\prime}}. \\ \end{array}
$$
For any  $T_2' < t \leq T_2$ , it is Lemma C.2 implied from Lemma C.2 that  $(1 + \eta \lambda^{\star})^{t - T_2'} \leq \log^6 n$ , and we have  $\left\| \widetilde{\boldsymbol{x}}^{(T_2')} \right\|_2 \leq \| \widetilde{\boldsymbol{x}} \|_2$ . Hence, we get
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \sqrt {\frac {1}{\log^ {1 7} n}} \sqrt {\log^ {1 2} n} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2}, \tag {E.5}
$$
and the proof for (11) of Theorem 3.3 is completed. If we combine this with (C.7), we are able to prove (4) of Theorem 3.1.
We move on to  $\left\| \pmb{x}^{(t)} - \widetilde{\pmb{x}}^{(t)}\right\|_{\infty}$ . For  $t \leq T_1$ , from Lemma D.2 and (C.4), we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \lesssim \sqrt {\frac {\mu^ {3} \log^ {8} n}{n p}} \frac {\beta_ {0}}{\sqrt {n}} \lesssim \frac {1}{\log n} \frac {\beta_ {0}}{\sqrt {n}} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty}
$$
provided that  $n^2 p \gtrsim \mu^3 n \log^{10} n$ . For  $T_1 < t \leq T_2'$ , from the definition of  $T_1$  and Lemma E.1, we have
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \lesssim \sqrt {\frac {1}{\log^ {1 3} n}} \frac {\beta_ {0}}{\sqrt {n}} (1 + \eta \lambda^ {\star}) ^ {t}.
$$
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \lesssim \sqrt {\frac {1}{\log^ {1 3} n}} \left(1 + \frac {(1 + \eta \lambda^ {\star}) ^ {t}}{\sqrt {n}}\right) \frac {\beta_ {0}}{\sqrt {n}} \lesssim \sqrt {\frac {1}{\log^ {1 2} n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty}.
$$
Now, for  $T_2' < t \leq T_2$ , if we do the same as before, we get
$$
\left\| \boldsymbol {x} ^ {(t)} - \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty} \lesssim \sqrt {\frac {1}{\log^ {1 3} n}} \sqrt {\log^ {1 2} n} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {2} \frac {1}{\sqrt {n}} \lesssim \frac {1}{\sqrt {\log n}} \left\| \widetilde {\boldsymbol {x}} ^ {(t)} \right\| _ {\infty},
$$
and the proof for (12) of Theorem 3.3 is completed. If we combine this with (C.8), we are able to prove (5) of Theorem 3.1.
Going through a similar way with (22) and (24), we can complete the proof of Theorems 3.1 and 3.3.
Proof of Lemma E.1 Before we start the proof, we define a function  $G$  as
$$
G (\boldsymbol {x}) = \frac {1}{4} \left\| \boldsymbol {x x} ^ {\top} \right\| _ {\mathrm {F}} ^ {2}.
$$
The gradient of  $G$  satisfies
$$
\nabla F (\boldsymbol {x}) = \nabla G (\boldsymbol {x}) - M ^ {\star} \boldsymbol {x}.
$$
Now, we assume that the hypotheses hold up to the  $t$ th iteration and show that they hold at the  $(t + 1)$ st iteration. For brevity, we drop the superscript  $(t)$  from  $\pmb{x}^{(t)},\pmb{x}^{(t,l)},\widetilde{\pmb{x}}^{(t)}$  and denote them as  $\pmb {x},\pmb{x}^{(l)},\widetilde{\pmb{x}}$ , respectively.