SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} J(K) = \max_{\left[ \begin{array}{c}\mathbf{w}\\ \mathbf{n} \end{array} \right]:\| \left[ \begin{array}{c}\mathbf{w}\\ \mathbf{n} \end{array} \right]\| \leq 1}}\sum_{t = 0}^{\infty}x_{t}^{\mathsf{T}}Qx_{t} + u_{t}Ru_{t} \\ = \max  _ {\left[ \begin{array}{c} \mathbf {w} \\ \mathbf {n} \end{array} \right]: \left\| \left[ \begin{array}{c} \mathbf {w} \\ \mathbf {n} \end{array} \right] \right\| \leq 1} \sum_ {t = 0} ^ {\infty} \left[ \begin{array}{c} x _ {t} \\ n _ {t - 1} \end{array} \right] ^ {\top} \left[ \begin{array}{c c} Q & 0 \\ 0 & 0 \end{array} \right] \left[ \begin{array}{c} x _ {t} \\ n _ {t - 1} \end{array} \right] + (K C x _ {t} + K n _ {t - 1}) ^ {\top} R (K C x _ {t} + K n _ {t - 1}) \\ = \max _ {\left[ \begin{array}{c} \mathbf {w} \\ \mathbf {n} \end{array} \right]: \| \left[ \begin{array}{c} \mathbf {w} \\ \mathbf {n} \end{array} \right] \| \leq 1} \sum_ {t = 0} ^ {\infty} \left[ \begin{array}{c} x _ {t} \\ n _ {t - 1} \end{array} \right] ^ {\top} \left[ \begin{array}{c c} Q + C ^ {\top} K ^ {\top} R K C & C ^ {\top} K ^ {\top} R K \\ K ^ {\top} R K C & K ^ {\top} R K \end{array} \right] \left[ \begin{array}{c} x _ {t} \\ n _ {t - 1} \end{array} \right]. \\ \end{array}
$$
If we define  $z_{t} = \begin{bmatrix} Q + C^{\top}K^{\top}RK\\ K^{\top}RKC \end{bmatrix} \begin{bmatrix} C^{\top}K^{\top}RK\\ K^{\top}RK \end{bmatrix} ^{\frac{1}{2}}\left[ \begin{array}{c}x_{t}\\ n_{t - 1} \end{array} \right]$ , in frequency domain, we can rewrite the above cost function as
$$
J (K) = \sup  _ {\omega \in [ 0, 2 \pi ]} \sigma_ {\max } \left(\left[ \begin{array}{c c} Q + C ^ {\top} K ^ {\top} R K C & C ^ {\top} K ^ {\top} R K \\ K ^ {\top} R K C & K ^ {\top} R K \end{array} \right] ^ {\frac {1}{2}} \left(e ^ {j \omega} I - \left[ \begin{array}{c c} A - B K C & - B K \\ 0 & 0 \end{array} \right]\right) ^ {- 1}\right). \tag {A.5}
$$
We can slightly modify our proof of Lemma 3.2 to show that (A.5) remains coercive. Therefore, all the results presented in the paper follow.
B Detailed Proofs
B.1 Proof of Lemma 3.2
Suppose we have a sequence  $\{K^l\}$  satisfying  $\| K^l \|_F \to +\infty$ . Let  $\mathbf{w}^l = \{w_0^l, 0, 0, \dots\}$  with  $\| w_0^l \| = 1$  such that  $\sigma_{\max}(K^l C) = \| K^l C w_0^l \|$ . Then we have:
$$
\begin{array}{l} J (K ^ {l}) = \max _ {\mathbf {w} ^ {l}: \| \mathbf {w} ^ {l} \| \leq 1} \sum_ {t = 0} ^ {\infty} x _ {t} ^ {\top} (Q + (K ^ {l} C) ^ {\top} R K ^ {l} C) x _ {t} \\ \geq_ {(i)} w _ {0} ^ {l} ^ {\top} (Q + (K ^ {l} C) ^ {\top} R K ^ {l} C) w _ {0} ^ {l} \\ \geq_ {(i i)} \lambda_ {\min } (R) \| K ^ {l} C w _ {0} ^ {l} \| ^ {2} \\ \geq \lambda_ {\min } (R) \lambda_ {\min } ^ {\frac {1}{2}} \left(C C ^ {\top}\right) \sigma_ {\max } \left(K ^ {l}\right) \\ \end{array}
$$
where inequality  $(i)$  holds since we plugged into a specific  $\mathbf{w}^l$  over the max operation and the matrix  $Q + (K^{l}C)^{\top}RK^{l}C$  is positive definite. Inequality  $(ii)$  uses the fact that  $R\geq \lambda_{\min}(R)I$ , where  $\lambda_{\mathrm{min}}(R)$  is the minimum eigenvalue of  $R$ . Since  $\| K^l\| _F\to +\infty$  by equivalence of norms  $\sigma_{\max}(K^l)\rightarrow +\infty$ , and knowing  $C$  is a full row rank matrix guarantees that  $\lambda_{\mathrm{min}}(CC^{\top}) > 0$ , therefore  $J(K^{l})\rightarrow +\infty$  as  $\| K^l\| _F\to +\infty$ .
Next, we assume  $K^l \to K$  where  $K$  is on the boundary  $\partial \mathcal{K}$ . Clearly we have  $\rho(A - BKC) = 1$ . We will use a frequency-domain argument to prove  $J(K^l) \to +\infty$ . Since  $\rho(A - BKC) = 1$ , there exists some  $\omega_0$  such that the matrix  $(e^{j\omega_0}I - A + BKC)$  becomes singular. Obviously, for the same  $\omega_0$ , the matrix  $(e^{j\omega_0}I - A + BKC)$  is also singular. Therefore, we have:
$$
\begin{array}{l} J (K ^ {l}) = \sup  _ {\omega \in [ 0, 2 \pi ]} \sigma_ {\max } \left((Q + (K ^ {l} C) ^ {\top} R K ^ {l} C) ^ {1 / 2} \left(e ^ {j \omega} I - A + B K ^ {l} C\right) ^ {- 1}\right) \\ \geq \sup _ {\omega \in [ 0; 2 \pi ]} \sigma_ {\min } \left((Q + (K ^ {l} C) ^ {\top} R K ^ {l} C) ^ {1 / 2}\right) \sigma_ {\max } \left((e ^ {j \omega} I - A + B K ^ {l} C) ^ {- 1}\right) \\ \geq \lambda_ {\min } ^ {1 / 2} (Q) \sigma_ {\max } \left(\left(e ^ {j \omega_ {0}} I - A + B K ^ {l} C\right) ^ {- 1}\right). \\ \end{array}
$$
Clearly, the above argument relies on the fact that  $Q$  is positive definite and  $Q \geq \lambda_{\min}(Q)I$ . Notice that we have  $\rho(A - BK^l C) < 1$  for each  $l$ , and hence we have  $\sigma_{\min}\left((e^{j\omega_0}I - A + BK^l C)\right) > 0$ ,
i.e. the smallest singular values of  $(e^{j\omega_0}I - A + BK^l C)$  are strictly positive for all  $l$ . By the continuity of the  $\sigma_{\mathrm{min}}(\cdot)$ , we must have  $\sigma_{\mathrm{min}}\left((e^{j\omega_0}I - A + BK^l C)\right) \to 0$  as  $\lim_{l \to \infty} K^l \in \partial \mathcal{K}$ . Hence we have  $\sigma_{\mathrm{max}}\left((e^{j\omega_0}I - A + BK^l C)^{-1}\right) \to +\infty$  as  $l \to \infty$ . Therefore, we have  $J(K^l) \to +\infty$  as  $K^l \to K \in \partial \mathcal{K}$ . This completes the proof.
B.2 Proof of Lemma 3.3
We first prove Lemma 3.3. Since  $J$  is continuous, for any  $\gamma > J^*$ , we know  $\mathcal{S}_{\gamma} = \{K \in \mathcal{K} : J(K) \leq \gamma\}$  is a closed set. It remains to show  $\mathcal{S}_{\gamma}$  is bounded. Suppose that  $\mathcal{S}_{\gamma}$  is unbounded. Then there exists a sequence  $\{K^l\}_{l=1}^{\infty} \subset \mathcal{S}$  such that  $\|K^l\|_2 \to +\infty$  as  $l \to \infty$ . But by coerciveness of  $J(K)$ , we must have  $J(K^l) \to +\infty$  as well, which contradicts that  $J(K^l) \leq \gamma$  for all  $l$ . Hence  $\mathcal{S}_{\gamma}$  is bounded. Therefore,  $\mathcal{S}_{\gamma}$  is compact.
B.3 Proof of Theorem 3.6
We first use induction to show that the iterative method (8) stays in  $\mathcal{K}$  for all  $t$ . Since  $K^0 \in S^0$  and  $\delta \leq \Delta_1$ , we know that  $J_{\delta}(K^0)$  is well defined. In addition, we have:
$$
\left\| K ^ {- 1} - K ^ {0} \right\| _ {F} = \eta \| \nabla J _ {\delta} \left(K ^ {0}\right) \| _ {F} \leq \eta L _ {1} \leq \xi , \tag {B.1}
$$
where we use the fact that  $J_{\delta}$  is  $L_{1}$ -Lipschitz on the sublevel set  $S^1$  and  $\eta \leq \frac{\xi}{L_1}$ . Therefore we have  $K^1 \in S^1$  by Remark 3.5 and  $J_{\delta}(K^{1})$  is well defined. Since  $J_{\delta}$  is  $L_{1}$ -Lipschitz with the  $\frac{cL_1\sqrt{d}}{\delta}$ -Lipschitz gradient, we have:
$$
\begin{array}{l} J _ {\delta} (K ^ {1}) - J _ {\delta} (K ^ {0}) \leq \langle \nabla J _ {\delta} (K ^ {0}), K ^ {1} - K ^ {0} \rangle + \eta^ {2} \frac {c L _ {1} \sqrt {d}}{2 \delta} \| \nabla J _ {\delta} (K ^ {0}) \| _ {F} ^ {2} \\ = (\eta^ {2} \frac {c L _ {1} \sqrt {d}}{2 \delta} - \eta) \| \nabla J _ {\delta} (K ^ {0}) \| _ {F} ^ {2} \\ \leq - \frac {\eta}{2} \| \nabla J _ {\delta} (K ^ {0}) \| _ {F} ^ {2}, \\ \end{array}
$$
where the last inequality holds since we have  $0 < \eta = \min \left\{\frac{\xi}{L_1}, \frac{\delta}{cL_1\sqrt{d}}\right\}$ . Therefore, we have  $J_{\delta}(K^1) \leq J_{\delta}(K^0)$ . This implies:
$$
J \left(K ^ {1}\right) \leq J _ {\delta} \left(K ^ {1}\right) + \delta L _ {1} \leq J _ {\delta} \left(K ^ {0}\right) + \delta L _ {1} \leq J \left(K ^ {0}\right) + 2 \delta L _ {1} \leq 5 0 J \left(K ^ {0}\right). \tag {B.2}
$$
Hence we have  $K^1 \in S^0$ . Repeating this argument, leads to the fact that  $K^t \in S^0$  for all  $t$ . Thus:
$$
\begin{array}{l} J _ {\delta} \left(K ^ {t + 1}\right) \leq J _ {\delta} \left(K ^ {t}\right) + \left\langle \nabla J _ {\delta} \left(K ^ {t}\right), K ^ {t + 1} - K ^ {t} \right\rangle + \eta^ {2} \frac {c L _ {1} \sqrt {d}}{2 \delta} \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \\ \leq J _ {\delta} \left(K ^ {t}\right) - \frac {\eta}{2} \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2}. \\ \end{array}
$$
Rearranging terms of the above inequality yields:
$$
\frac {\eta}{2} \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \leq J _ {\delta} (K ^ {t}) - J _ {\delta} (K ^ {t + 1}).
$$
Summing up the above inequality over  $t = 0,1,\dots ,T - 1$  gives:
$$
\frac {\eta}{2 T} \sum_ {t = 0} ^ {T - 1} \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \leq \frac {J _ {\delta} (K ^ {0}) - J _ {\delta} (K ^ {T})}{T} \leq \frac {J _ {\delta} (K ^ {0}) - J ^ {*}}{T},
$$
where we use the inequality  $J_{\delta}(K^T) \geq J^{*}$  by definition of  $J_{\delta}(K)$ . Therefore, we have:
$$
\min  _ {t = 0, 1, \dots , T - 1} \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \leq \frac {2 (J _ {\delta} (K ^ {0}) - J ^ {*})}{\eta T},
$$
$$
\min  _ {t = 0, 1, \dots , T - 1} \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} \leq \sqrt {\frac {2 \left(J _ {\delta} \left(K ^ {0}\right) - J ^ {*}\right)}{\eta T}}
$$
and the sample complexity result follows. This completes the proof.
B.4 Proof of Theorem 3.7
We first provide several technical lemmas which will be useful for proving Theorem 3.7. To this end, let  $\mathcal{F}_t$  denote the filtration  $\sigma(K^s, s \leq t)$  for each  $t = 1, 2, \dots, T$ .
Lemma B.1. Suppose  $J_{\delta}(K) \leq 49J(K^0)$  and  $\delta \leq \frac{J(K^0)}{L_1}$ . Then we have  $K \in S^0$ .
Proof. Since  $J_{\delta}(K)$  is well defined, we must have  $K \in \mathcal{K}$  by definition of  $J_{\delta}$ . Then Proposition 2.3 (i) implies that  $J(K) \leq J_{\delta}(K) + \delta L_1 \leq 49J(K^0) + \frac{J(K^0)}{L_1} L_1 = 50J(K^0)$ . Hence we have  $K \in S^0$ .
Lemma B.2. Suppose that  $J(K)$  is  $L_{1}$ -Lipschitz on the sublevel set  $S^1$  and let  $\{g^t\}_{t=0}^{T-1}$  and  $\{K^t\}_{t=0}^{T-1}$  be generated by Algorithm 1 such that  $\{K^t\}_{t=0}^{T-1}$  are feasible and  $\{J_\delta(K^t)\}_{t=0}^{T-1}$  are well defined. Then, we have
$$
\mathbb {E} \left[ g ^ {t} \mid \mathcal {F} _ {t} \right] = \nabla J _ {\delta} \left(K ^ {t}\right), \tag {B.3}
$$
$$
\mathbb {E} \left[ \| g ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq 1 6 \sqrt {2 \pi} d L _ {1} ^ {2}. \tag {B.4}
$$
Proof. The proof can be found in [46, Lemma D.1]. We omit it here.
Lemma B.3.  $J(K)$  is  $L_{0}$ -Lipschitz on the sublevel set  $\mathcal{S}^0$ , let  $\eta \leq \frac{2\delta\xi}{d(100J(K^0) - J^*)}$  and  $\delta \leq \min \{\Delta_1, \Delta\}$ , then as long as  $K^t \in \mathcal{S}^0$ , we will have  $K^{t+1} \in \mathcal{S}^1$  and
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mid \mathcal {F} _ {t} \right] \leq J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z, \tag {B.5}
$$
where  $Z = \eta (8\sqrt{2\pi})cd^{3 / 2}L_1^3\delta^{-1}$
Proof. Since  $\| W^t\| _F = 1$ , we have:
$$
\begin{array}{l} \left\| K ^ {t + 1} - K ^ {t} \right\| _ {F} = \eta \| \frac {d}{2 \delta} (J (K ^ {t} + \delta W ^ {t}) - J (K ^ {t} - \delta W ^ {t})) W ^ {t} \| _ {F} \\ \leq \frac {\eta d}{2 \delta} | (J (K ^ {t} + \delta W ^ {t}) - J (K ^ {t} - \delta W ^ {t}) | \\ \leq \frac {\eta d}{2 \delta} \cdot (1 0 0 J (K ^ {0}) - J ^ {*}) \\ \leq \xi , \\ \end{array}
$$
where the second inequality holds since we have  $K^t \pm \delta W^t \in S^1$  when  $\delta \leq \Delta$ . This implies that  $K^{t + 1} \in S^1$  by Remark 3.5. In addition,  $K^{t + 1} \in S^1$  and  $\delta \leq \Delta_1$  ensures that  $J_{\delta}(K^{t + 1})$  is well defined. By Proposition 2.3 (ii), we know that  $J_{\delta}(K)$  is differentiable and  $L_{1}$ -Lipschitz with  $\frac{cL_1\sqrt{d}}{\delta}$ -Lipschitz gradient on the sublevel set  $S^1$ . Then we have:
$$
J _ {\delta} \left(K ^ {t + 1}\right) \leq J _ {\delta} \left(K ^ {t}\right) - \eta \left\langle \nabla J _ {\delta} \left(K ^ {t}\right), g ^ {t} \right\rangle + \frac {c \eta^ {2} L _ {1} \sqrt {d}}{2 \delta} \| g ^ {t} \| _ {F} ^ {2}. \tag {B.6}
$$
Taking the expectation on both sides conditioned on  $\mathcal{F}_t$  and using Lemma B.2 yields
$$
\begin{array}{l} \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mid \mathcal {F} _ {t} \right] \leq J _ {\delta} \left(K ^ {t}\right) - \eta \langle \nabla J _ {\delta} \left(K ^ {t}\right), \mathbb {E} \left[ g ^ {t} \mid \mathcal {F} _ {t} \right] \rangle + \frac {c \eta^ {2} L _ {1} \sqrt {d}}{2 \delta} \mathbb {E} [ \| g ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] \\ \leq J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \frac {c \eta^ {2} L _ {1} \sqrt {d}}{2 \delta} 1 6 \sqrt {2 \pi} d L _ {1} ^ {2} \\ = J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z, \\ \end{array}
$$
with  $Z = \eta (8\sqrt{2\pi})cd^{3 / 2}L_1^3\delta^{-1}$ . This completes the proof.
Now we are ready to prove the Theorem 3.7. We will first prove Statement 1 in Theorem 3.7: all the generated controllers are within the feasible set with high probability. Then we will show Statement 2: the Algorithm 1 returns a  $(\delta, \epsilon)$ -stationary point with high probability.
Proof of Statement 1 We first define a stopping time  $\tau$  as below:
$$
\tau := \min  \left\{t \in \{0, 1, 2, \dots , T - 1 \}: J _ {\delta} \left(K ^ {t}\right) > 4 9 J \left(K ^ {0}\right) \right\}. \tag {B.7}
$$
Based on Lemma B.1, it can be seen that as long as  $\tau \geq T - 1$ , the iterates  $\{K^t\}_{t=0}^{T-1}$  generated by Algorithm 1 are feasible. Therefore our goal becomes bounding the probability  $\operatorname*{Pr}(\tau \leq T - 1)$ . To this end, we can define a nonnegative supermartingale  $Y(t)$  as below
$$
Y (t) := J _ {\delta} \left(K ^ {\min  \{t, \tau \}}\right) + (T - t) \eta Z. \tag {B.8}
$$
To show it is a supermartingale, noticing that we have
$$
\begin{array}{l} \mathbb {E} [ Y (t + 1) \mid \mathcal {F} _ {t} ] = \mathbb {E} [ J _ {\delta} (K ^ {\tau}) \mathbb {1} _ {\tau \leq t} \mid \mathcal {F} _ {t} ] + \mathbb {E} [ J _ {\delta} (K ^ {t + 1}) \mathbb {1} _ {\tau > t} \mid \mathcal {F} _ {t} ] + (T - t - 1) \eta Z \\ = _ {(i)} J _ {\delta} \left(K ^ {\tau}\right) \mathbb {1} _ {\tau \leq t} + \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t} \mid \mathcal {F} _ {t} \right] + (T - t - 1) \eta Z \\ \leq_ {(i i)} J _ {\delta} \left(K ^ {\tau}\right) \mathbb {1} _ {\tau \leq t} + J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z + (T - t - 1) \eta Z \\ \leq J _ {\delta} \left(K ^ {\min  \{t, \tau \}}\right) + (T - t) \eta Z = Y (t), \\ \end{array}
$$
where in equality (i),  $J_{\delta}(K^{t + 1})$  is well defined by the definition of the  $\tau$ ; in particular, we have  $J_{\delta}(K^t)\leq 49J(K^0)$ , hence  $K^t\in S^0$  by Lemma B.1 and we can apply Lemma B.3 to obtain inequality (ii). Then Doob's maximal inequality for super-martingales gives
$$
\begin{array}{l} \Pr (\tau \leq T - 1)\leq \Pr \bigl(\max_{t = 0,1,\dots ,T - 1}Y(t) > 49J(K^{0})\bigr) \\ \leq \frac {\mathbb {E} [ Y (0) ]}{4 9 J \left(K ^ {0}\right)} = \frac {J _ {\delta} \left(K ^ {0}\right) + T \eta Z}{4 9 J \left(K ^ {0}\right)} \leq_ {(i)} \frac {J \left(K ^ {0}\right) + \delta L _ {1}}{4 9 J \left(K ^ {0}\right)} + \frac {T \eta Z}{4 9 J \left(K ^ {0}\right)} \\ \leq_ {(i i)} \frac {2}{4 9} + \frac {T \eta Z}{4 9 J (K ^ {0})}, \\ \end{array}
$$
where inequality (i) uses Lemma 2.3 and (ii) is true since  $\delta \leq \frac{J(K^0)}{L_1}$ . For sufficiently small  $\epsilon$ , we have  $\eta = \frac{\delta c^2}{3500cd^{3/2}L_1^3}$ , then one can verify that  $T\eta Z = \frac{4\sqrt{2\pi}v}{25} J(K^0)$ . Therefore, we have
$$
\operatorname * {P r} (\tau \leq T - 1) \leq \frac {2}{4 9} + \frac {T \eta Z}{4 9 J (K ^ {0})} \leq \frac {2}{4 9} + \frac {4 \sqrt {2 \pi} v}{1 2 2 5} \leq \frac {1}{2 4} + \frac {v}{1 0 0} \leq 0. 0 5 + 0. 0 1 v.
$$
This implies that all the iterates  $K^t$  are stabilizing with probability at least  $1 - (0.05 + 0.01v) = 0.95 - 0.01v$ . This completes the proof for Statement 1.
Proof of Statement 2 We first show that we can extend (B.5) as
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} \right] \leq J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z, \tag {B.9}
$$
If  $\tau > t$ , then we know  $J_{\delta}(K^t) \leq 49 J(K^0)$ , hence  $J(K^t) \leq J_{\delta}(K^t) + \delta L_1 \leq 50 J(K^0)$ , we have  $K^t \in S^0$  and  $K^{t+1} \in S^1$ . Therefore, we can apply Lemma B.3 to show that:
$$
\begin{array}{l} \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} \right] \leq \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mid \mathcal {F} _ {t} \right] \\ \leq J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z \\ = J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z. \\ \end{array}
$$
On the other hand, if  $\tau \leq t$ , we have
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} \right] = 0 \leq J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z \tag {B.10}
$$
since  $Z \geq 0$ . Therefore, (B.9) holds. Taking the expectation and rearranging the terms of (B.9) gives:
$$
\mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} ] \leq \frac {\mathbb {E} [ J _ {\delta} (K ^ {t}) \mathbb {1} _ {\tau > t} ] - \mathbb {E} [ J _ {\delta} (K ^ {t + 1}) \mathbb {1} _ {\tau > t + 1} ]}{\eta} + Z. \tag {B.11}
$$
Summing up the above inequality over  $t = 0,1,\dots ,T - 1$  yields
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} ] \leq \frac {J _ {\delta} (K ^ {0}) - \mathbb {E} [ J _ {\delta} (K ^ {T}) \mathbb {1} _ {\tau > T} ]}{\eta T} + Z.
$$
Since the cost function  $J(K)$  is nonnegative, we have  $\mathbb{E}[J_{\delta}(K^T)\mathbb{1}_{\tau > T}] \geq 0$  and:
$$
J _ {\delta} \left(K ^ {0}\right) - \mathbb {E} \left[ J _ {\delta} \left(K ^ {T}\right) \mathbb {1} _ {\tau > T} \right] \leq J _ {\delta} \left(K ^ {0}\right) \leq J \left(K ^ {0}\right) + \delta L _ {1} = 2 J \left(K ^ {0}\right).
$$
Hence we conclude that
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {2 J (K ^ {0})}{\eta T} + Z.
$$
By the choice of  $\eta$  and  $T$ , the above inequality becomes:
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {2 J (K ^ {0})}{\eta T} + Z \leq (\frac {5}{v} + 1) Z \leq (\frac {5}{v} + 1) \frac {8 \sqrt {2 \pi}}{3 5 0 0} \epsilon^ {2}.
$$
Since the random count  $R \in \{0, 1, 2, \dots, T - 1\}$  is uniformly sampled, we have
$$
\mathbb {E} [ \| \nabla J _ {\delta} (K ^ {R}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] = \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq (\frac {5}{\upsilon} + 1) \frac {8 \sqrt {2 \pi}}{3 5 0 0} \epsilon^ {2},
$$
$$
\mathbb {E} \left[ \| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \mathbb {1} _ {\tau > T - 1} \right] \leq (0. 1 7 v ^ {- \frac {1}{2}} + 0. 0 8) \epsilon .
$$
$$
\begin{array}{l} \Pr (\| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \geq \epsilon) = \Pr (\| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \geq \epsilon , \tau > T - 1) + \Pr (\| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \geq \epsilon , \tau \leq T - 1) \\ \leq \Pr (\| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \geq \epsilon , \tau > T - 1) + \Pr (\tau \leq T - 1) \\ \leq \frac {1}{\epsilon} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \mathbb {1} _ {\tau > T - 1} ] + \Pr (\tau \leq T - 1) \\ \leq 0. 1 7 v ^ {- \frac {1}{2}} + 0. 0 8 + \frac {1}{2 4} + \frac {v}{1 0 0} \\ \leq 0. 1 7 v ^ {- \frac {1}{2}} + 0. 1 3 + 0. 0 1 v, \\ \end{array}
$$
where the second inequality uses the Markov's inequality. By Proposition 2.3 (iii), we have  $\nabla J_{\delta}(K^{R}) \in \partial_{\delta}J(K^{R})$ . This implies that
$$
\Pr \left(\min  \left\{\| H \| _ {F}: H \in \partial_ {\delta} J \left(K ^ {R}\right) \geq \epsilon \right\}\right) \leq \Pr \left(\| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \geq \epsilon\right) \leq 0. 1 7 v ^ {- \frac {1}{2}} + 0. 1 3 + 0. 0 1 v.
$$
Therefore our algorithm can find a  $(\delta, \epsilon)$ -stationary point with probability at least  $1 - (0.17v^{-\frac{1}{2}} + 0.13 + 0.01v) = 0.87 - 0.17v^{-\frac{1}{2}} - 0.01v$ . This completes the proof for Statement 2.
Remark B.4. It is worth mentioning that the choice of sublevels in sets in (9) is WLOG. Tuning these two numbers entails trade-offs. On the one hand, a larger sublevel in set  $S^1$  yields a higher probability as observed from the proofs of Theorem 3.7 where we are bounding the term  $\operatorname*{Pr}(\tau \leq T - 1)$  as an example. On the other hand, a larger sublevel set also results in a larger Lipschitz constant  $L_1$  and a smaller smooth radius  $\delta$ , thereby worsening the complexity bounds.
B.5 Proof of Theorem 4.1
We first state several technical lemmas which will be useful in proving of Theorem 4.1.
Lemma B.5. Let  $P(z) = \sum_{k=0}^{\infty} P_k z^{-k}$  be a stable, discrete-time MIMO LTI system, where  $P = \{P_k \in \mathbb{C}^{n_x \times n_x}\}_{k \in \mathbb{Z}_+}$  is the impulse response of  $P(z)$  and  $T(P) = (P_{i-j})_{i,j=0}^{\infty,i}$  denote the Toeplitz operator associated with system  $P(z)$ . Suppose for any given Euclidean unit vectors  $y \in \mathbb{C}^{n_x}$  and  $x \in \mathbb{C}^{n_x}$ , we define a scalar sequence  $g = \{g_k = y^* P_k x\}_{k \in \mathbb{Z}_+}$ . Let  $G(z) = \sum_{k=0}^{\infty} g_k z^{-k}$  denote the  $z$ -transform of the impulse response  $g$  and  $T(g) = (g_{i-j})_{i,j=0}^{\infty,i}$  be the corresponding Toeplitz matrix. For any given Euclidean unit vectors  $x$  and  $y$ , we can show:
$$
\left\| T _ {N} (g) \right\| \leq \left\| T _ {N} (P) \right\| \tag {B.12}
$$
where  $T_{N}(P)$  and  $T_{N}(g)$  are  $Nn_{x} \times Nn_{x}$  and  $N \times N$  upper-left submatrix of  $T(P)$  and  $T(g)$ , respectively,  $\| \cdot \|$  is the spectral norm of a given matrix.
Proof. Using the definition of  $g_{k} = y^{*}P_{k}x$ , we can write  $T_{N}(g)$  as:
$$
\begin{array}{l} T _ {N} (g) = \left[ \begin{array}{c c c c} g _ {0} & 0 & \dots & 0 \\ g _ {1} & g _ {0} & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ g _ {N - 1} & g _ {N - 2} & \dots & g _ {0} \end{array} \right] = \left[ \begin{array}{c c c c} y ^ {*} P _ {0} x & 0 & \dots & 0 \\ y ^ {*} P _ {1} x & y ^ {*} P _ {0} x & \dots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ y ^ {*} P _ {N - 1} x & y ^ {*} P _ {N - 2} x & \dots & y ^ {*} P _ {0} x \end{array} \right] \\ = \underbrace {\left[ \begin{array}{l l l} y ^ {*} & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & y ^ {*} \end{array} \right]} _ {Y ^ {*}} \underbrace {\left[ \begin{array}{c c c c} P _ {0} & 0 & \cdots & 0 \\ P _ {1} & P _ {0} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ P _ {N - 1} & P _ {N - 2} & \cdots & P _ {0} \end{array} \right]} _ {T _ {N} (P)} \underbrace {\left[ \begin{array}{l l l} x & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & x \end{array} \right]} _ {X} \tag {B.13} \\ \end{array}
$$
Since  $\| x\| = \| y\| = 1$ , we have  $\sigma_{\max}(Y^{*}) = \sigma_{\max}(X) = 1$ .
$$
\begin{array}{l} \left\| T _ {N} (g) \right\| = \sigma_ {\max } \left(T _ {N} (g)\right) = \sigma_ {\max } \left(Y ^ {*} T _ {N} (P) X\right) \\ \leq \sigma_ {\max } \left(Y ^ {*}\right) \sigma_ {\max } \left(T _ {N} (P)\right) \sigma_ {\max } (X) \tag {B.14} \\ \leq \sigma_ {\max } \left(T _ {N} (P)\right) = \| T _ {N} (P) \| \\ \end{array}
$$