SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Corollary B.6. Let  $P(z)$  and  $G(z)$  be defined as lemma B.5, fix a  $\gamma \in (\iota, 1)$  and define the systems  $P^{\gamma}(z) \coloneqq \gamma P(\gamma z)$  and  $G^{\gamma}(z) \coloneqq \gamma G(\gamma z)$ . We can show that:
$$
\| G ^ {\gamma} \| _ {\infty} \leq \| P ^ {\gamma} \| _ {\infty} \tag {B.15}
$$
Proof. This result can be shown by applying lemma B.5 to  $G^{\gamma}(z)$  and  $P^{\gamma}(z)$  and considering this fact that  $\| T_{N}(g^{\gamma})\|$  and  $\| T_{N}(P^{\gamma})\|$  converge to  $\| G^{\gamma}\|_{\infty}$  and  $\| P^{\gamma}\|_{\infty}$  as  $N\to \infty$ , respectively.
Now we are ready to extend [68] results of  $\mathcal{H}_{\infty}$ -norm approximation of SISO system using the corresponding truncated Toeplitz matrix to more general MIMO systems.
Lemma B.7. Let  $P(z) = \sum_{k=0}^{\infty} P_k z^{-k}$  be a stable, discrete-time MIMO LTI system with stability radius  $\iota \in (0,1)$ . Fix a  $\gamma \in (\iota,1)$  and suppose that  $\sigma_{\max}(P_0) \leq D$ . For all  $N \geq 3$ , we have that
$$
\begin{array}{l} \| P (z) \| _ {\infty} - \| T _ {N} (P) \| \leq C _ {1} \frac {D \| P ^ {\gamma} (z) \| _ {\infty} \left(1 - \gamma^ {2}\right) + \| P ^ {\gamma} (z) \| _ {\infty} ^ {2} \gamma}{\| P (z) \| _ {\infty} (1 - \gamma) ^ {4}} \frac {1}{N ^ {2}} \tag {B.16} \\ + C _ {2} \frac {\| P ^ {\gamma} (z) \| _ {\infty} ^ {2}}{\| P (z) \| _ {\infty} (1 + \gamma) (1 - \gamma) ^ {5}} \frac {1}{N ^ {3}} \\ \end{array}
$$
where  $\| P^{\gamma}(z)\|_{\infty}$  denotes the  $\mathcal{H}_{\infty}$ -norm of the system  $P^{\gamma}(z) \coloneqq \gamma P(\gamma z)$ ,  $C_1 = 3\sqrt{2}\pi (2 + 3\pi^4)$ ,  $C_2 = 9\sqrt{2}\pi^2$  are universal constants.
Proof. We can write  $\| P\|_{\infty}$  as follows
$$
\| P \| _ {\infty} = \max  _ {\omega \in [ 0, 2 \pi ]} \sigma_ {\max } \left(P \left(e ^ {i \omega}\right)\right) = \sigma_ {\max } \left(P \left(e ^ {i \omega_ {\mathrm {o p t}}}\right)\right) \tag {B.17}
$$
This means that there exists Euclidean unit vectors  $x$  and  $y$  such that  $\| P\|_{\infty} = y^{*}P(e^{i\omega_{\mathrm{opt}}})x$ . Now define a SISO LTI system  $G(z) = y^{*}P(z)x$ , which has the impulse response  $g = \{g_k = y^* P_kx\}_{k\in \mathbb{Z}_+}$ . Since  $P(z)$  is a stable function, so is  $G(z)$  and we have  $\| P\|_{\infty} = \| G\|_{\infty}$ . Additionally, regarding the stability radius of  $G$ , it can be derived that,  $\iota_g \leq \iota$  and  $\gamma \in (\iota_g, 1)$ . Consequently:
$$
\left\| g _ {0} \right\| = \left\| y ^ {*} P _ {0} x \right\| \leq \sigma_ {\max } \left(P _ {0}\right) \leq D \tag {B.18}
$$
Now we invoke Theorem 4.1 from [68]; for a stable, SISO LTI system  $G(z) = \sum_{k=0}^{\infty} g_k z^{-k}$ , with a fix  $\gamma \in (\iota_g, 1)$  and  $\| g_0 \| \leq D$ , we have:
$$
\| G \| _ {\infty} - \| T _ {N} (g) \| \leq C _ {1} \frac {D \| G ^ {\gamma} \| _ {\infty} (1 - \gamma^ {2}) + \| G ^ {\gamma} \| _ {\infty} ^ {2} \gamma}{\| G \| _ {\infty} (1 - \gamma) ^ {4}} \frac {1}{N ^ {2}} + C _ {2} \frac {\| G ^ {\gamma} \| _ {\infty} ^ {2}}{\| G \| _ {\infty} (1 + \gamma) (1 - \gamma) ^ {5}} \frac {1}{N ^ {3}}
$$
From lemma B.5 and corollary B.6, we know that  $\| T_N(g)\| \leq \| T_N(P)\|$  and  $\| G^{\gamma}\|_{\infty}\leq \| P^{\gamma}\|_{\infty}$ . Also using this fact that  $\| G\|_{\infty} = \| P\|_{\infty}$ , we have
$$
\left| \left| \right| P \right| _ {\infty} - \left| \left| T _ {N} (P) \right| \right| \leq \zeta_ {1} \tag {B.19}
$$
where  $\zeta_1 = C_1\frac{D\|P^\gamma\|_\infty(1 - \gamma^2) + \|P^\gamma\|_\infty^2\gamma}{\|P\|_\infty(1 - \gamma)^4}\frac{1}{N^2} +C_2\frac{\|P^\gamma\|_\infty^2}{\|P\|_\infty(1 + \gamma)(1 - \gamma)^5}\frac{1}{N^3}.$
Algorithm 2: Power iteration method to find the maximum eigenvalue of a matrix
Require: Given a positive semi-definite matrix  $M \in \mathcal{R}^{Nn_x \times Nn_x}$  and an initial vector
$$
v ^ {(0)} \in \mathcal {R} ^ {N n _ {x}}
$$
for  $i = 0,1,\dots ,n$  do
$$
z ^ {(i)} = M v ^ {(i - 1)}.
$$
$$
v ^ {(i)} = z ^ {(i)} / \left\| z ^ {(i)} \right\| _ {2}.
$$
$$
\lambda^ {(i)} = \left[ v ^ {(i)} \right] ^ {*} M v ^ {(i)}.
$$
Output:  $\lambda^{(n)}$ .
Lemma B.8. Given a positive semi-definite matrix  $M \in \mathbb{R}^{Nn_x \times Nn_x}$  with spectral factorization
$$
V ^ {*} M V = \operatorname {d i a g} \left(\lambda_ {1}, \dots , \lambda_ {N n _ {x}}\right)
$$
Where  $V = [v_{1}|\ldots |v_{Nn_{x}}]$  is orthogonal,  $\lambda_1\geq \lambda_2\geq \dots \geq \lambda_{Nn_x}\geq 0$  and  $\lambda_{1} > 0$ . Suppose we choose a  $\psi$  such that  $\lambda_1,\dots ,\lambda_j\in [\lambda_1 - \psi ,\lambda_1]$ . Let the vectors  $v^{(i)}$  be specified by Algorithm 2 and define  $\theta_0\in [0,\pi /2)$  by  $\cos (\theta_0) = |v_1^T v^{(0)}|$ . For  $i = 0,1,\ldots$  we have
$$
\left| \lambda^ {(i)} - \lambda_ {1} \right| \leq \tan \left(\theta_ {0}\right) ^ {2} \left(\psi + \lambda_ {1} \left(\frac {\lambda_ {1} - \psi}{\lambda_ {1}}\right) ^ {2 i}\right) \tag {B.20}
$$
Proof. The idea of our proof is based on the results in section 8.2.1 of [29]. Suppose  $v^{(0)}$  has the eigenvector expansion  $v^{(0)} = a_1v_1 + a_2v_2 + \ldots + a_{Nn_x}v_{Nn_x}$ , then
$$
\left| a _ {1} \right| = \left| v _ {1} ^ {T} v ^ {(0)} \right| = \cos \left(\theta_ {0}\right) \neq 0
$$
$$
a _ {1} ^ {2} + a _ {2} ^ {2} + \ldots + a _ {N n x} ^ {2} = 1
$$
$$
\lambda^ {(i)} = [ v ^ {(i)} ] ^ {T} M v ^ {(i)} = \frac {[ v ^ {(0)} ] ^ {T} M ^ {2 i + 1} v ^ {(0)}}{[ v ^ {(0)} ] ^ {T} M ^ {2 i} v ^ {(0)}} = \frac {\sum_ {k = 1} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i + 1}}{\sum_ {k = 1} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i}}
$$
$$
\begin{array}{l} | \lambda^ {(i)} - \lambda_ {1} | = \left| \frac {\sum_ {k = 2} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i} (\lambda_ {k} - \lambda_ {1})}{\sum_ {k = 1} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i}} \right| \leq \left| \frac {\sum_ {k = 2} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i} (\lambda_ {k} - \lambda_ {1})}{a _ {1} ^ {2} \lambda_ {1} ^ {2 i}} \right| \\ = \left| \frac {\sum_ {k = 2} ^ {j} a _ {k} ^ {2} \lambda_ {k} ^ {2 i} (\lambda_ {k} - \lambda_ {1})}{a _ {1} ^ {2} \lambda_ {1} ^ {2 i}} \right| + \left| \frac {\sum_ {k = j + 1} ^ {N n _ {x}} a _ {k} ^ {2} \lambda_ {k} ^ {2 i} (\lambda_ {k} - \lambda_ {1})}{a _ {1} ^ {2} \lambda_ {1} ^ {2 i}} \right| \\ \leq \psi \frac {1}{a _ {1} ^ {2}} \sum_ {k = 2} ^ {j} a _ {k} ^ {2} \left(\frac {\lambda_ {k}}{\lambda_ {1}}\right) ^ {2 i} + \frac {1}{a _ {1} ^ {2}} \Bigg | \sum_ {k = j + 1} ^ {N n _ {x}} a _ {k} ^ {2} (\lambda_ {k} - \lambda_ {1}) \left(\frac {\lambda_ {k}}{\lambda_ {1}}\right) ^ {2 i} \\ \leq \psi \frac {1}{a _ {1} ^ {2}} \sum_ {k = 2} ^ {j} a _ {k} ^ {2} \left(\frac {\lambda_ {2}}{\lambda_ {1}}\right) ^ {2 i} + \lambda_ {1} \frac {1}{a _ {1} ^ {2}} \sum_ {k = j + 1} ^ {N n _ {x}} a _ {k} ^ {2} \left(\frac {\lambda_ {j + 1}}{\lambda_ {1}}\right) ^ {2 i} \\ \le_ {(i)} \tan (\theta_ {0}) ^ {2} \left(\psi \left(\frac {\lambda_ {2}}{\lambda_ {1}}\right) ^ {2 i} + \lambda_ {1} \left(\frac {\lambda_ {j + 1}}{\lambda_ {1}}\right) ^ {2 i}\right) \\ \leq \tan (\theta_ {0}) ^ {2} \left(\psi + \lambda_ {1} \left(\frac {\lambda_ {1} - \psi}{\lambda_ {1}}\right) ^ {2 i}\right) \\ \end{array}
$$
where the inequality  $(i)$  follows from the fact that  $\tan (\theta_0)^2 = \frac{1 - a_1^2}{a_1^2}$  and this completes the proof.
Corollary B.9. Let matrix  $M$  be given as lemma B.8, we can show that for each iteration  $i$ ,  $\psi_{\min}^{(i)} = \lambda_1 \left( 1 - \frac{1}{\sqrt[2i-1]{2i}} \right)$  minimize the upper bound in power iteration algorithm (B.20), and (B.20) can be written as:
$$
\left| \lambda^ {(i)} - \lambda_ {1} \right| \leq \tan \left(\theta_ {0}\right) ^ {2} \lambda_ {1} \left(1 - \frac {1}{\sqrt [ 2 i - 1 ]{2 i}} \left(1 - \frac {1}{2 i}\right)\right) \tag {B.21}
$$
Proof. Since the right hand side of inequality (B.20) is a convex function of  $\psi$ , we can minimize it by setting its derivative with respect to  $\psi$  equal to zero. Define  $f(\psi) = \tan (\theta_0)^2\left(\psi + \lambda_1\left(\frac{\lambda_1 - \psi}{\lambda_1}\right)^{2i}\right)$ , we have:
$$
f ^ {\prime} (\psi) = \tan (\theta_ {0}) ^ {2} \left(1 - 2 i \left(\frac {\lambda_ {1} - \psi}{\lambda_ {1}}\right) ^ {2 i - 1}\right) = 0
$$
It is straightforward to show that  $\psi_{\mathrm{min}}^{(i)} = \lambda_1\left(1 - \frac{1}{2^i - 1 / 2i}\right)$  makes  $f'(\psi) = 0$  and substitute  $\psi_{\mathrm{min}}^{(i)}$  in  $f(\psi)$ , gives  $f(\psi_{\mathrm{min}}^{(i)}) = \tan (\theta_0)^2\lambda_1\left(1 - \frac{1}{2^i - 1 / 2i}\left(1 - \frac{1}{2i}\right)\right)$ , which completes the proof.
Now we are ready to prove Theorem 4.1. As mentioned before, cost function  $J(K)$  is equal to  $\mathcal{H}_{\infty}$ -norm of  $P(z)$ ,  $\| P \|_{\infty}$ . From lemma B.7, We can find the bound  $\zeta_1(K)$  for  $|J(K) - \| T_N(P) \|| \leq \zeta_1(K)$ . Also, using Algorithm 2 we can obtain the model-free cost function  $J(K)$  as  $\hat{J}(K) \coloneqq \sqrt{\lambda^{(n)}}$ . It is also well-known that  $\sqrt{\lambda_1} \leq \| P \|_{\infty}$  and  $\sqrt{\lambda_1} \to \| P \|_{\infty}$  as  $N \to \infty$ . Now invoke lemma B.8, corollary B.9 and use  $M \coloneqq T_N(P)^* T_N(P)$ , we have
$$
\begin{array}{l} | \hat {J} (K) ^ {2} - \| T _ {N} (P) \| ^ {2} | \leq \tan (\theta_ {0}) ^ {2} \lambda_ {1} \left(1 - \frac {1}{\sqrt [ 2 n - 1 ]{2 n}} \left(1 - \frac {1}{2 n}\right)\right) \\ \leq \tan (\theta_ {0}) ^ {2} \| P \| _ {\infty} ^ {2} \left(1 - \frac {1}{\sqrt [ 2 n - 1 ]{2 n}} \left(1 - \frac {1}{2 n}\right)\right) \\ \end{array}
$$
It is easy to show that
$$
\left| \hat {J} (K) - \| T _ {N} (P) \| \right| \leq \tan \left(\theta_ {0}\right) ^ {2} \| P \| _ {\infty} \left(1 - \frac {1}{\sqrt [ 2 n - 1 ]{2 n}} \left(1 - \frac {1}{2 n}\right)\right).
$$
In addition, for  $n \geq 1$ , one can verify that
$$
1 - \frac {1}{\sqrt [ 2 n - 1 ]{2 n}} \left(1 - \frac {1}{2 n}\right) \leq n ^ {- \frac {2}{3}}
$$
and hence  $|\hat{J}(K) - \|T_N(P)\|| \leq C_3 \|P\|_{\infty} \frac{1}{\sqrt{n}} = \zeta_2(K)$  with  $C_3 = \tan(\theta_0)^2$ . This gives us sample complexity (16). Finally, applying triangle inequality, we have
$$
| \hat {J} (K) - J (K) | \leq | \hat {J} (K) - \| T _ {N} (P) \| | + \| \| T _ {N} (P) - J (K) \| \leq \zeta_ {1} (K) + \zeta_ {2} (K) \tag {B.22}
$$
B.6 Proof of Theorem 4.2
We consider that we have an inaccurate function value estimation  $\hat{J} (K) = J(K) + \zeta (K)$ , here we assume that the estimation error  $\zeta (K)$  is uniformly upper bounded for all  $K\in S^1$  as stated in (18). Then we have:
$$
\begin{array}{l} \hat {g} ^ {t} = \frac {d}{2 \delta} (\hat {J} (K ^ {t} + \delta W ^ {t}) - \hat {J} (K ^ {t} - \delta W ^ {t})) W ^ {t} \\ = \frac {d}{2 \delta} \left(J \left(K ^ {t} + \delta W ^ {t}\right) - J \left(K ^ {t} - \delta W ^ {t}\right)\right) W ^ {t} + \frac {d}{2 \delta} \left(\zeta \left(K ^ {t} + \delta W ^ {t}\right) - \zeta \left(K ^ {t} - \delta W ^ {t}\right)\right) W ^ {t} \\ = g ^ {t} + \frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta} W ^ {t}, \\ \end{array}
$$
where we denote  $\zeta_1^t = \zeta (K^t +\delta W^t)$  and  $\zeta_2^t = \zeta (K^t -\delta W^t)$  for simplicity. To prove Theorem 4.2, we first present some technical lemmas.
Lemma B.10. Suppose that  $J(K)$  is  $L_{1}$ -Lipschitz on the sublevel set  $S^{1}$  and let  $\{g^t\}_{t=0}^{T-1}$  and  $\{K^t\}_{t=0}^{T-1}$  be generated by Algorithm 1 such that  $\{K^t\}_{t=0}^{T-1}$  are feasible and  $\{J_{\delta}(K^t)\}_{t=0}^{T-1}$  are well defined. Then, we have
$$
\left\| \mathbb {E} \left[ \hat {g} ^ {t} \mid \mathcal {F} _ {t} \right] - \nabla_ {\delta} J \left(K ^ {t}\right) \right\| _ {F} \leq \frac {d \kappa}{\delta}, \tag {B.23}
$$
$$
\mathbb {E} \left[ \| \hat {g} ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \leq 3 2 \sqrt {2 \pi} d L _ {1} ^ {2} + 2 \left(\frac {d \kappa}{\delta}\right) ^ {2}. \tag {B.24}
$$
Proof. By definition of  $\hat{g}^t$ , we have:
$$
\begin{array}{l} \| \mathbb {E} [ \hat {g} ^ {t} \mid \mathcal {F} _ {t} ] - \nabla_ {\delta} J (K ^ {t}) \| _ {F} = \| \mathbb {E} \left[ g ^ {t} + \frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta} W ^ {t} \mid \mathcal {F} _ {t} \right] - \nabla_ {\delta} J (K ^ {t}) \| _ {F} \\ = \| \mathbb {E} [ g ^ {t} \mid \mathcal {F} _ {t} ] + \mathbb {E} \left[ \frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta} W ^ {t} \mid \mathcal {F} _ {t} \right] - \nabla_ {\delta} J (K ^ {t}) \| _ {F} \\ = \| \mathbb {E} \left[ \frac {d \left(\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t}\right)}{2 \delta} W ^ {t} \mid \mathcal {F} _ {t} \right] \| _ {F} \\ \leq \mathbb {E} \left[ \| \frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta} \| _ {F} \| W ^ {t} \| _ {F} \mid \mathcal {F} _ {t} \right] \leq \frac {d \kappa}{\delta}. \\ \mathbb {E} [ \| \hat {g} ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] = \mathbb {E} \left[ \| g ^ {t} + \frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta} W ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} \right] \\ \leq 2 \mathbb {E} [ \| g ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] + 2 \mathbb {E} [ \| (\frac {d (\zeta_ {1} ^ {t} - \zeta_ {2} ^ {t})}{2 \delta}) W ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] \\ \leq 3 2 \sqrt {2 \pi} d L _ {1} ^ {2} + 2 \left(\frac {d \kappa}{\delta}\right) ^ {2}. \\ \end{array}
$$
Lemma B.11.  $J(K)$  is  $L_{1}$ -Lipschitz on the sublevel set  $\mathcal{S}^1$ , let  $\eta \leq \frac{\delta\xi}{d\kappa(100J(K^0) - J^*)}$  and  $\delta \leq \min \{\Delta_1, \Delta\}$ , then as long as  $K^t \in \mathcal{S}^0$ , we will have  $K^{t+1} \in \mathcal{S}^1$  and
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mid \mathcal {F} _ {t} \right] \leq J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta \left(Z _ {1} + Z _ {2}\right), \tag {B.25}
$$
where  $Z_{1} = \frac{L_{1}d\kappa}{\delta}$  and  $Z_{2} = \frac{\eta}{\delta}\Gamma$  with  $\Gamma = cL_1\sqrt{d}\cdot (16\sqrt{2\pi} dL_1^2 +(\frac{d\kappa}{\delta})^2)$ .
$$
\begin{array}{l} \| K ^ {t + 1} - K ^ {t} \| _ {F} = \eta \| \frac {d}{2 \delta} (\hat {J} (K ^ {t} + \delta W ^ {t}) - \hat {J} (K ^ {t} - \delta W ^ {t})) W ^ {t} \| _ {F} \\ \leq \frac {\eta d}{2 \delta} | (J (K ^ {t} + \delta W ^ {t}) - J (K ^ {t} - \delta W ^ {t}) | \cdot | \zeta_ {1} ^ {t} - \zeta_ {2} ^ {t} | \\ \leq \frac {\eta d \kappa}{\delta} \cdot (1 0 0 J \left(K ^ {0}\right) - J ^ {*}) \\ \leq \xi , \\ \end{array}
$$
where the second inequality holds since we have  $K^t \pm \delta W^t \in S^1$  when  $\delta \leq \Delta$  and  $|\zeta_1^t - \zeta_2^t| \leq 2\kappa$ . This implies that  $K^{t+1} \in S^1$  by Remark 3.5. In addition,  $K^{t+1} \in S^1$  and  $\delta \leq \Delta_1$  ensures that  $J_{\delta}(K^{t+1})$  is well defined. By Proposition 2.3 (ii), we know that  $J_{\delta}(K)$  is differentiable and  $L_1$ -Lipschitz with  $\frac{cL_1\sqrt{d}}{\delta}$ -Lipschitz gradient on the sublevel set  $S^1$ . Then we have:
$$
J _ {\delta} (K ^ {t + 1}) \leq J _ {\delta} (K ^ {t}) - \eta \langle \nabla J _ {\delta} (K ^ {t}), \hat {g} ^ {t} \rangle + \frac {c \eta^ {2} L _ {1} \sqrt {\hat {d}}}{2 \delta} \| \hat {g} ^ {t} \| _ {F} ^ {2}.
$$
Taking the expectation of both sides conditioned on  $K^t$ , we have
$$
\begin{array}{l} \mathbb {E} [ J _ {\delta} (K ^ {t + 1}) \mid \mathcal {F} _ {t} ] \leq J _ {\delta} (K ^ {t}) - \eta \langle \nabla J _ {\delta} (K ^ {t}), \mathbb {E} [ \hat {g} ^ {t} \mid \mathcal {F} _ {t} ] \rangle + \frac {c \eta^ {2} L _ {1} \sqrt {d}}{2 \delta} \mathbb {E} [ \| \hat {g} ^ {t} \| _ {F} ^ {2} \mid \mathcal {F} _ {t} ] \\ \le_ {(i)} J _ {\delta} (K ^ {t}) - \eta \langle \nabla J _ {\delta} (K ^ {t}), \nabla J _ {\delta} (K ^ {t}) - \nabla J _ {\delta} (K ^ {t}) + \mathbb {E} [ \hat {g} ^ {t} \mid \mathcal {F} _ {t} ] \rangle + \frac {\eta^ {2}}{\delta} \Gamma \\ \le_ {(i i)} J _ {\delta} (K ^ {t}) - \eta \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} + \eta \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} \| \nabla J _ {\delta} (K ^ {t}) - \mathbb {E} [ \hat {g} ^ {t} \mid \mathcal {F} _ {t} ] \| _ {F} + \frac {\eta^ {2}}{\delta} \Gamma \\ \leq_ {(i i i)} J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta \frac {L _ {1} d \kappa}{\delta} + \frac {\eta^ {2}}{\delta} \Gamma \\ = J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z _ {1} + \eta Z _ {2}, \\ \end{array}
$$
where inequality (i) and (iii) use the bounds in Lemma B.10, and (ii) holds by Cauchy-Schwarz inequality. This completes the proof.
Now we are ready to prove the Theorem 4.2. We will first prove Statement 1 in Theorem 4.2: all the generated controllers are within the feasible set with high probability. Then we will show Statement 2: the Algorithm 1 returns a  $(\delta, \epsilon)$ -stationary point with high probability.
$$
\tau := \min  \left\{t \in \left\{0, 1, 2, \dots , T - 1 \right\}: J _ {\delta} \left(K ^ {t}\right) > 4 9 J \left(K ^ {0}\right) \right\}. \tag {B.26}
$$
Based on Lemma 3.2, it can be seen that as long as  $\tau \geq T - 1$ , the iterates  $\{K^t\}_{t=0}^{T-1}$  generated by Algorithm 1 are feasible. Therefore our goal becomes bounding the probability  $\operatorname*{Pr}(\tau \leq T - 1)$ . To this end, we can define a nonnegative supermartingale  $Y(t)$  as below
$$
Y (t) := J _ {\delta} \left(K ^ {\min  \{t, \tau \}}\right) + \eta (T - t) \left(Z _ {1} + Z _ {2}\right). \tag {B.27}
$$
$$
\begin{array}{l} \mathbb {E} [ Y (t + 1) \mid \mathcal {F} _ {t} ] = \mathbb {E} \left[ J _ {\delta} \left(K ^ {\tau}\right) \mathbb {1} _ {\tau \leq t} \mid \mathcal {F} _ {t} \right] + \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t} \mid \mathcal {F} _ {t} \right] + \eta (T - t - 1) \left(Z _ {1} + Z _ {2}\right) \\ = J _ {\delta} \left(K ^ {\tau}\right) \mathbb {1} _ {\tau \leq t} + \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t} \mid \mathcal {F} _ {t} \right] + \eta (T - t - 1) \left(Z _ {1} + Z _ {2}\right) \\ \leq^ {(i)} J _ {\delta} \left(K ^ {\tau}\right) \mathbb {1} _ {\tau \leq t} + J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta \left(Z _ {1} + Z _ {2}\right) + \eta (T - t - 1) \left(Z _ {1} + Z _ {2}\right) \\ \leq J _ {\delta} \left(K ^ {\min  \{t, \tau \}}\right) + \eta (T - t) \left(Z _ {1} + Z _ {2}\right) = Y (t), \\ \end{array}
$$
where the inequality  $(i)$  uses (B.25). This is valid since  $\tau > t$  implies  $J_{\delta}(K^t) \leq 49J(K^0)$  by the definition of  $\tau$ . Then we have  $J(K^t) \leq J_{\delta}(K^t) + \delta L_1 \leq 50J(K^0)$ . Hence we have  $K^t \in S^0$ . The choice of  $\eta$  guarantees that  $K^{t+1} \in S^1$ . Hence  $K^{t+1}$  is feasible and  $J_{\delta}(K^{t+1})$  is well defined when  $\delta \leq \Delta_1$ . Then Doob's maximal inequality for super-martingales gives
$$
\begin{array}{l} \Pr (\tau \leq T - 1)\leq \Pr \bigl(\max_{t = 0,1,\dots ,T - 1}Y(t) > 49J(K^{0})\bigr) \\ \leq \frac {\mathbb {E} [ Y (0) ]}{4 9 J \left(K ^ {0}\right)} = \frac {J _ {\delta} \left(K ^ {0}\right) + T \eta \left(Z _ {1} + Z _ {2}\right)}{4 9 J \left(K ^ {0}\right)} \leq \frac {J \left(K ^ {0}\right) + \delta L _ {1}}{4 9 J \left(K ^ {0}\right)} + \frac {T \eta \left(Z _ {1} + Z _ {2}\right)}{4 9 J \left(K ^ {0}\right)} \\ \leq \frac {2}{4 9} + \frac {T \eta \left(Z _ {1} + Z _ {2}\right)}{4 9 J \left(K ^ {0}\right)} \\ \end{array}
$$
For sufficiently small  $\epsilon$ , we have  $\eta = \frac{\delta \epsilon^2}{100 \Gamma}$ , then one can verify that  $T \eta(Z_1 + Z_2) \leq v J(K^0)$  by the choice of  $T, \eta$ , and  $\kappa$ . Therefore, we have
$$
\Pr (\tau \leq T - 1) \leq \frac {2}{4 9} + \frac {T \eta Z}{4 9 J (K ^ {0})} \leq \frac {2}{4 9} + \frac {v}{4 9} \leq 0. 0 5 + 0. 0 3 v.
$$
This implies that all the iterates  $K^t$  are stabilizing with probability at least  $1 - (0.05 + 0.03v) = 0.95 - 0.03v$ . This completes the proof for Statement 1.
Proof of Statement 2 We first show that we can extend (B.25) as
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} \right] \leq J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z _ {1} + \eta Z _ {2}, \tag {B.28}
$$
$$
\begin{array}{l} \mathbb {E} [ J _ {\delta} (K ^ {t + 1}) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} ] \leq \mathbb {E} [ J _ {\delta} (K ^ {t + 1}) \mid \mathcal {F} _ {t} ] \\ \leq J _ {\delta} \left(K ^ {t}\right) - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} + \eta Z _ {1} + \eta Z _ {2} \\ = J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z _ {1} + \eta Z _ {2}. \\ \end{array}
$$
$$
\mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \mid \mathcal {F} _ {t} \right] = 0 \leq J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} - \eta \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} + \eta Z _ {1} + \eta Z _ {2} \tag {B.29}
$$
since  $Z_{1} \geq 0$  and  $Z_{2} \geq 0$ . Therefore, (B.28) holds. Taking the expectation and rearranging the terms of (B.28) gives:
$$
\mathbb {E} \left[ \| \nabla J _ {\delta} \left(K ^ {t}\right) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} \right] \leq \frac {\mathbb {E} \left[ J _ {\delta} \left(K ^ {t}\right) \mathbb {1} _ {\tau > t} \right] - \mathbb {E} \left[ J _ {\delta} \left(K ^ {t + 1}\right) \mathbb {1} _ {\tau > t + 1} \right]}{\eta} + Z _ {1} + Z _ {2}. \tag {B.30}
$$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > t} ] \\ \leq \frac {J _ {\delta} (K ^ {0}) - \mathbb {E} [ J _ {\delta} (K ^ {T}) \mathbb {1} _ {\tau > T} ]}{\eta T} + Z _ {1} + Z _ {2}. \\ \end{array}
$$
Since the cost function  $J(K)$  is nonnegative, we have  $\mathbb{E}[J_{\delta}(K^T)\mathbb{1}_{\tau > T}]\geq 0$  and:
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {J (K ^ {0})}{\eta T} + Z _ {1} + Z _ {2}.
$$
$$
\frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq \frac {J (K ^ {0})}{\eta T} + Z _ {1} + Z _ {2} \leq (\frac {2}{v} + 1) Z _ {2} + Z _ {1} \leq (\frac {2}{v} + 2) \frac {\epsilon^ {2}}{1 0 0}.
$$
$$
\mathbb {E} [ \| \nabla J _ {\delta} (K ^ {R}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] = \frac {1}{T} \sum_ {t = 0} ^ {T - 1} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {t}) \| _ {F} ^ {2} \mathbb {1} _ {\tau > T - 1} ] \leq (\frac {2}{v} + 2) \frac {\epsilon^ {2}}{1 0 0},
$$
$$
\mathbb {E} [ \| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \mathbb {1} _ {\tau > T - 1} ] \leq (0. 1 5 v ^ {- \frac {1}{2}} + 0. 1 5) \epsilon .
$$
$$
\begin{array}{l} \Pr \left(\| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \geq \epsilon\right) = \Pr \left(\| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \geq \epsilon , \tau > T - 1\right) + \Pr \left(\| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \geq \epsilon , \tau \leq T - 1\right) \\ \leq \Pr (\| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \geq \epsilon , \tau > T - 1) + \Pr (\tau \leq T - 1) \\ \leq \frac {1}{\epsilon} \mathbb {E} [ \| \nabla J _ {\delta} (K ^ {R}) \| _ {F} \mathbb {1} _ {\tau > T - 1} ] + \Pr (\tau \leq T - 1) \\ \leq 0. 1 5 v ^ {- \frac {1}{2}} + 0. 1 5 + 0. 0 5 + 0. 0 3 v \\ = 0. 1 5 v ^ {- \frac {1}{2}} + 0. 2 + 0. 0 3 v, \\ \end{array}
$$
where the second inequality uses the Markov's inequality. By Proposition 2.3 (iii), we have  $\nabla J_{\delta}(K^R) \in \partial_{\delta}J(K^{R})$ . This implies that