SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Theorem 1. The first step of the proof is to show that the assumption of Proposition 2 is satisfied for  $E = \nabla \widehat{J}(L)$  for all  $L \in S_{\alpha} \setminus \mathcal{C}_{\gamma/2}$ . This is true because
$$
\begin{array}{l} \left\| \nabla \widehat {J} (L) - \nabla J (L) \right\| \leq s \| \nabla J (L) \| + s _ {0} \\ \leq s \| \nabla J (L) \| + \frac {\gamma}{2} \| \nabla J (L) \| \\ \leq \gamma \| \nabla J (L) \| \\ \end{array}
$$
where the first inequality follows from Assumption 3, the second inequality follows from  $L \notin C_{\gamma /2}$  (i.e.  $s_0\| \nabla J(L)\| \geq \gamma /2$ ), and the last step follows from the assumption  $s \leq \gamma /2$ .
The rest of the proof relies on repeated application of Proposition 2. In particular, starting from  $L_0 \in S_\alpha \setminus \mathcal{C}_{\gamma/2}$ , the application of Proposition 2 implies that  $L_1 \coloneqq L_0 - \bar{\eta} \nabla \hat{J}(L_0)$  remains in the same sublevel set, i.e.,  $L_1 \in S_\alpha$ , and we obtain the following linear decay of the cost value:
Now, if  $L_{1} \in \mathcal{C}_{\gamma / 2}$  then we stop; otherwise  $L_{1} \in S_{\alpha} \setminus \mathcal{C}_{\gamma / 2}$  and we can repeat the above process to arrive at  $L_{2} \coloneqq L_{1} - \bar{\eta} \nabla \widehat{J}(L_{1}) \in S_{\alpha}$ , with a guaranteed linear decay
$$
J \left(L _ {2}\right) - J \left(L ^ {*}\right) \leq \left[ 1 - c _ {1} (\alpha) \bar {\eta} (1 - \gamma) / 2 \right] \left[ J \left(L _ {1}\right) - J \left(L ^ {*}\right) \right].
$$
Combining the last two linear decays yields
$$
J (L _ {2}) - J (L ^ {*}) \leq [ 1 - c _ {1} (\alpha) \bar {\eta} (1 - \gamma) / 2 ] ^ {2} [ J (L _ {0}) - J (L ^ {*}) ].
$$
Repeating the process generates a sequence of policies  $L_0, L_1, L_2 \ldots$  with a combined linear decay of
$$
J (L _ {k}) - J (L ^ {*}) \leq [ 1 - c _ {1} (\alpha) \bar {\eta} (1 - \gamma) / 2 ] ^ {k} [ J (L _ {0}) - J (L ^ {*}) ],
$$
unless at some iteration  $j$ , we arrive at a policy  $L_{j}$  such that  $L_{j} \in \mathcal{C}_{\gamma /2}$ . This completes the proof.
Appendix F Proofs of the result for observation model and sample complexity
F.1 Preliminary lemmas and their proofs
First, we provide the proof for the complete version of Lemma 6:4
Lemma 6' (Uniform Bounds for Stable Systems). Suppose  $L \in \mathcal{S}$ , then there exit a constant  $C_L > 0$  such that
$$
\left\| A _ {L} ^ {k} \right\| \leq C _ {L} \left(\sqrt {\rho (A _ {L})}\right) ^ {k + 1}, \quad \forall k \geq 0,
$$
whenever  $\rho(A_L) > 0$ , and otherwise  $\sqrt{\rho(A_L)}$  is replaced with any arbitrarily small  $r \in (0,1)$ . Additionally,
$$
\sum_ {i = 0} ^ {\infty} \left\| A _ {L} ^ {i} \right\| \leq \frac {C _ {L}}{1 - \sqrt {\rho (A _ {L})}}
$$
$$
\sum_ {i = 0} ^ {\infty} \| M _ {i} [ E ] \| \leq \frac {1 + 2 C _ {L} ^ {2} \rho (A _ {L}) ^ {3 / 2}}{\left[ 1 - \sqrt {\rho (A _ {L})} \right] ^ {2}} \| E H \|
$$
Furthermore, consider  $S_{\alpha}$  for some  $\alpha > 0$ , then there exist constants  $D_{\alpha} > 0$ ,  $C_{\alpha} > 0$  and  $\rho_{\alpha} \in (0,1)$  such that  $\|L\| \leq D_{\alpha}$ ,  $C_{L} \leq C_{\alpha}$  and  $\rho(A_{L}) \leq \rho_{\alpha}$ ,  $\forall L \in S_{\alpha}$ .
Proof of Lemma 6. Recall the Cauchy Integral formula for matrix functions [53, Theorem 1.12]: for any matrix  $M \in \mathbb{C}^{n \times n}$ ,
$$
f (M) = \frac {1}{2 \pi i} \oint_ {\Gamma} f (z) (z I - M) ^ {- 1} d z,
$$
whenever  $f$  is real analytic on and inside a closed contour  $\Gamma$  that encloses spectrum of  $M$ . Note that  $L \in S$  implying that  $\rho(A_L) < 1$ . Now, fix some  $r \in (\rho(A_L), 1)$  and define  $\Gamma(\theta) = r e^{i\theta}$  with  $\theta$  ranging on  $[0, 2\pi]$ . Therefore, Cauchy Integral formula applies to  $f(z) = z^k$  for any positive integer  $k$  and the contour  $\Gamma$  defined above. So, for matrix  $A_L$ , we obtain
$$
A _ {L} ^ {k} = \frac {1}{2 \pi i} \oint_ {\Gamma} z ^ {k} (z I - A _ {L}) ^ {- 1} d z = \frac {1}{2 \pi i} \int_ {0} ^ {2 \pi} r ^ {k} e ^ {i k \theta} (r e ^ {i \theta} I - A) ^ {- 1} r d e ^ {i \theta},
$$
implying that
$$
\| A _ {L} ^ {k} \| \leq \frac {r ^ {k + 1}}{2 \pi} \int_ {0} ^ {2 \pi} \| (r e ^ {i \theta} I - A _ {L}) ^ {- 1} \| d \theta \leq r ^ {k + 1} \max  _ {\theta \in [ 0, 2 \pi ]} \| (r e ^ {i \theta} I - A _ {L}) ^ {- 1} \|.
$$
Finally, the first claim follows by choosing  $r = \sqrt{\rho(A_L)}$  (whenever  $\rho(A_L) > 0$ , otherwise  $r \in (0,1)$  can be chosen arbitrarily small) and defining
$$
C _ {L} := \max  _ {\theta \in [ 0, 2 \pi ]} \| (\sqrt {\rho (A _ {L})} e ^ {i \theta} I - A _ {L}) ^ {- 1} \|
$$
which is attained and bounded.
4See [52, Definition 3.1] for an alternative notation; however, we prefer the explicit and simple form of expressing spectral radius in the bounds established in our work, which also facilitates the comparison to literature on first order methods for stabilizing policies.
Next, by applying the first claim, we have
$$
\sum_ {i = 0} ^ {T} \left\| A _ {L} ^ {i} \right\| \leq C _ {L} \frac {1 - \sqrt {\rho (A _ {L})} ^ {T}}{1 - \sqrt {\rho (A _ {L})}},
$$
implying the second bound. For the third claim, note that for  $i = 1,2,\dots$  we obtain
$$
\begin{array}{l} \| M _ {i + 1} [ E ] \| \leq \sum_ {k = 0} ^ {i} \| A _ {L} ^ {i - k} \| \| A _ {L} ^ {k} \| \| E H \| \leq \| E H \| C _ {L} ^ {2} \sum_ {k = 0} ^ {i} \left[ \sqrt {\rho (A _ {L})} \right] ^ {(i - k + 1) + (k + 1)} \\ \leq \| E H \| C _ {L} ^ {2} \sqrt {\rho \left(A _ {L}\right)} \left[ (i + 1) \cdot \rho \left(A _ {L}\right) ^ {(i + 1) / 2} \right]. \tag {33} \\ \end{array}
$$
But, then by recalling that  $M_0[E] = 0$  and  $\| M_1[E] \| = \| EH \|$  we have
$$
\sum_ {i = 0} ^ {\infty} \| M _ {i} [ E ] \| \leq \| E H \| + \| E H \| \left[ \frac {2 C _ {L} ^ {2} \rho (A _ {L}) ^ {3 / 2}}{\left[ 1 - \sqrt {\rho (A _ {L})} \right] ^ {2}} \right]
$$
where we used the following convergent sum for any  $\rho \in (0,1)$ :
$$
\sum_ {i = 1} ^ {\infty} (i + 1) \cdot \rho^ {i + 1} = \frac {(2 - \rho) \rho^ {2}}{(1 - \rho) ^ {2}} \leq \frac {2 \rho^ {2}}{(1 - \rho) ^ {2}}.
$$
This implies the third bound.
The final claim follows directly from compactness of sublevel set  $S_{\alpha}$  (Lemma 1) and continuity of the mappings  $(L, \theta) \mapsto (\rho(A_L), \theta) \mapsto \|( \sqrt{\rho(A_L)} e^{i\theta} I - A_L)^{-1} \|$  on  $S_{\alpha} \times [0, 2\pi]$  whenever  $\rho(A_L) > 0$  (and otherwise considering the mapping  $(L, \theta) \mapsto \| (re^{i\theta} I - A_L)^{-1} \|$  for arbitrarily small and fixed  $r \in (0, 1)$ ).
As mentioned in Section 4.2, a key idea behind these error bounds that scale well with respect to the length  $T$  is the following consequence of von Neumann Trace Inequality [49, Theorem 8.7.6]:
$$
\left| \operatorname {t r} [ M N ] \right| \leq \sum_ {i = 1} ^ {T} \sigma_ {i} (M) \sigma_ {i} (N) \leq \| M \| \| N \| _ {*},
$$
with  $\| \cdot \|_*$  denoting the nuclear norm. Additionally, as a direct consequence of Courant-Fischer Theorem, one can also show that nuclear norm is submultiplicative. More precisely,
$$
\left\| A B \right\| _ {*} \leq \left\| A \right\| \left\| B \right\| _ {*} \leq \left\| A \right\| _ {*} \left\| B \right\| _ {*}.
$$
Next, we require the following lemma to bound these errors.
Lemma 8. For any  $L \in S_{\alpha}$ , we have
$$
\left\| \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right\| _ {*} \leq \frac {C _ {L} ^ {2} \left\| H ^ {\intercal} H \right\| _ {*}}{1 - \rho (A _ {L})},
$$
$$
\| \mathcal {N} _ {L} [ E ] \| _ {*} \leq \frac {\left[ 2 C _ {L} + 4 C _ {L} ^ {3} \rho (A _ {L}) ^ {3 / 2} \right] \| H \| \| H ^ {\intercal} H \| _ {*}}{\left[ 1 - \rho (A _ {L}) \right] ^ {2}} \| E \|.
$$
Proof of Lemma 8. For the first claim, note that  $\mathcal{A}_L^\top H^\top H\mathcal{A}_L$  is positive semi-definite, so
$$
\begin{array}{l} \left\| \mathcal {A} _ {L} ^ {\top} H ^ {\top} H \mathcal {A} _ {L} \right\| _ {*} = \operatorname {t r} \left[ \mathcal {A} _ {L} ^ {\top} H ^ {\top} H \mathcal {A} _ {L} \right] \\ \leq \operatorname {t r} \left[ H ^ {\mathsf {T}} H \right] \left\| \mathcal {A} _ {L} \mathcal {A} _ {L} ^ {\mathsf {T}} \right\| \\ \leq \| H ^ {\intercal} H \| _ {*} \left\| \sum_ {i = 0} ^ {T} A _ {L} ^ {i} \left(A _ {L} ^ {\intercal}\right) ^ {i} \right\| \\ \leq \| H ^ {\intercal} H \| _ {*} \sum_ {i = 0} ^ {T} \| A _ {L} ^ {i} \| ^ {2} \\ \leq \| H ^ {\intercal} H \| _ {*} \frac {C _ {L} ^ {2}}{1 - \rho (A _ {L})} \\ \end{array}
$$
where the last inequality follows by Lemma 6. Next, we have
$$
\begin{array}{l} \| \mathcal {M} _ {L} [ E ] \| = \| \mathcal {M} _ {L} [ E ] \mathcal {M} _ {L} [ E ] ^ {\intercal} \| ^ {1 / 2} \\ \leq \left[ \sum_ {i = 0} ^ {T} \| M _ {i} [ E ] \| ^ {2} \right] ^ {1 / 2} \\ \leq \| E H \| + \| E H \| C _ {L} ^ {2} \sqrt {\rho (A _ {L})} \left[ \sum_ {i = 0} ^ {T} (i + 1) ^ {2} \cdot \rho (A _ {L}) ^ {(i + 1)} \right] ^ {1 / 2} \\ \leq \| E H \| + \| E H \| C _ {L} ^ {2} \sqrt {\rho (A _ {L})} \frac {2 \rho (A _ {L})}{[ 1 - \rho (A _ {L}) ] ^ {3 / 2}} \\ \leq \| E H \| \left[ \frac {1 + 2 C _ {L} ^ {2} \rho (A _ {L}) ^ {3 / 2}}{\left[ 1 - \rho (A _ {L}) \right] ^ {3 / 2}} \right] \\ \end{array}
$$
where the second inequality follows by (33) and the third one by the following convergent sum for any  $\rho \in (0,1)$ :
$$
\sum_ {i = 1} ^ {\infty} (i + 1) ^ {2} \cdot \rho^ {i + 1} = \frac {\rho^ {2} (\rho^ {2} - 3 \rho + 4)}{(1 - \rho) ^ {3}} \leq \frac {4 \rho^ {2}}{(1 - \rho) ^ {3}}.
$$
Also, by the properties of nuclear norm
$$
\begin{array}{l} \left\| H ^ {\intercal} H \mathcal {A} _ {L} \right\| _ {*} = \operatorname {t r} \left[ \sqrt {H ^ {\intercal} H \mathcal {A} _ {L} \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H} \right] \\ \leq \left\| \mathcal {A} _ {L} \mathcal {A} _ {L} ^ {\top} \right\| ^ {1 / 2} \left\| H ^ {\top} H \right\| _ {*} \\ \leq \left[ \sum_ {i = 0} ^ {\infty} \| A _ {L} ^ {i} \| ^ {2} \right] ^ {1 / 2} \| H ^ {\intercal} H \| _ {*} \\ \leq \left[ \frac {C _ {L} ^ {2}}{1 - \rho (A _ {L})} \right] ^ {1 / 2} \| H ^ {\intercal} H \| _ {*}, \\ \end{array}
$$
where the last inequality follows by Lemma 6. Finally, notice that
$$
\begin{array}{l} \| \mathcal {N} _ {L} [ E ] \| _ {*} \leq 2 \| \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {M} _ {L} [ E ] \| _ {*} \\ \leq 2 \| \mathcal {M} _ {L} [ E ] \| \| H ^ {\intercal} H \mathcal {A} _ {L} \| _ {*} \\ \end{array}
$$
and thus combining the last three bounds implies the second claim. This completes the proof.
The next tool we will be using is the following famous bound on random matrices which is a variant of Bernstein inequality:
Lemma 9 (Matrix Bernstein Inequality [54, Corollary 6.2.1]). Let  $Z$  be a  $d_1 \times d_2$  random matrices such that  $\mathbb{E}[Z] = \bar{Z}$  and  $\| Z \| \leq K$  almost surely. Consider  $M$  independent copy of  $Z$  as  $Z_1, \dots, Z_M$ , then for every  $t \geq 0$ , we have
$$
\mathbb {P} \left[ \left\| \frac {1}{M} \sum_ {i} Z _ {i} - \bar {Z} \right\| \geq t \right] \leq (d _ {1} + d _ {2}) \exp \left\{\frac {- M t ^ {2} / 2}{\sigma^ {2} + 2 K t / 3} \right\}
$$
where  $\sigma^2 = \max \left\{\| \mathbb{E}\left[ZZ^\top \right]\right\|$ ,  $\| \mathbb{E}\left[Z^\top Z\right]\|$  is the per-sample second moment. This bound can be expressed as the mixture of sub-gaussian and sub-exponential tail as  $(d_1 + d_2)\exp \left\{-c\min \left\{\frac{t^2}{\sigma^2},\frac{t}{2K}\right\} \right\}$  for some  $c$ .
We are now well-equipped to provide the main proofs.
F.2 Proof of Proposition 3: The Observation model
$$
\varepsilon (L, \mathcal {Y} _ {T}) = \| H x (T) - H \hat {x} (T) \| ^ {2} = \sum_ {i = 1} ^ {m} | H _ {i} ^ {\intercal} x (T) - H _ {i} ^ {\intercal} \hat {x} (T) | ^ {2}
$$
where  $H_{i}^{\top}$  is the  $i$ -th row of  $H$ . Also, by duality, if  $z(t) = (A_L^\top)^T - tH_i$  is the adjoint dynamics' closed-loop trajectory with control signal  $u(t) = L^\top z(t)$  then
$$
H _ {i} ^ {\intercal} x (T) - H _ {i} ^ {\intercal} \hat {x} (T) = \vec {z} _ {i} ^ {\intercal} \vec {\xi} - \vec {u} ^ {\intercal} \vec {\omega}
$$
$$
\vec {z} _ {i} ^ {\mathsf {T}} = (z (T) ^ {\mathsf {T}} \quad z (T - 1) ^ {\mathsf {T}} \quad \ldots \quad z (1) ^ {\mathsf {T}} \quad z (0) ^ {\mathsf {T}}),
$$
$$
\vec {u} ^ {\intercal} = (u (T) ^ {\intercal} \quad u (T - 1) ^ {\intercal} \quad \ldots \quad u (1) ^ {\intercal} \quad 0 _ {m} ^ {\intercal}).
$$
But  $\vec{u} = (I\otimes L^{\intercal})\vec{z}_i$  and then  $\vec{z}_i = \mathcal{A}_L^\intercal H_i$ . Therefore,
$$
\begin{array}{l} \varepsilon (L, \mathcal {Y} _ {T}) = \sum_ {i = 1} ^ {m} \left| H _ {i} ^ {\intercal} x (T) - H _ {i} ^ {\intercal} \hat {x} (T) \right| ^ {2} \\ = \sum_ {i = 1} ^ {m} \operatorname {t r} \left[ \vec {\xi} \vec {\xi} ^ {\mathsf {T}} \vec {z} _ {i} \vec {z} _ {i} ^ {\mathsf {T}} \right] - \operatorname {t r} \left[ \left(\vec {\xi} \vec {\omega} ^ {\mathsf {T}} (I \otimes L ^ {\mathsf {T}}) + (I \otimes L) \vec {\omega} \vec {\xi} ^ {\mathsf {T}}\right) \vec {z} _ {i} \vec {z} _ {i} ^ {\mathsf {T}} \right] \\ + \operatorname {t r} \left[ \vec {\omega} \vec {\omega} ^ {\mathsf {T}} (I \otimes L ^ {\mathsf {T}}) \vec {z} _ {i} \vec {z} _ {i} ^ {\mathsf {T}} (I \otimes L) \right] \\ \end{array}
$$
Then, by using the fact that  $\sum_{i=1}^{m} H_{i}H_{i}^{\mathsf{T}} = H^{\mathsf{T}}H$ , we obtain that
$$
\varepsilon (L, \mathcal {Y} _ {T}) = \operatorname {t r} \left[ \mathcal {X} _ {L} \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right].
$$
Thus, we can rewrite the estimation error as
$$
\varepsilon (L, \mathcal {Y} _ {T}) = \left\langle \vec {\xi}, \vec {\xi} \right\rangle_ {\mathcal {A} _ {L}} + \left\langle (I \otimes L) \vec {\omega}, (I \otimes L) \vec {\omega} \right\rangle_ {\mathcal {A} _ {L}} - 2 \left\langle \vec {\xi}, (I \otimes L) \vec {\omega} \right\rangle_ {\mathcal {A} _ {L}} = \| \vec {\eta} _ {L} \| _ {\mathcal {A} _ {L}} ^ {2}
$$
Next, we can compute that for small enough  $E$
$$
\mathcal {A} _ {L + E} - \mathcal {A} _ {L} = \mathcal {M} _ {L} [ E ] + o (\| E \|).
$$
$$
\left. \mathrm {d} (\mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L}) \right| _ {L} [ E ] = \mathcal {M} _ {L} [ E ] ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} + \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {M} _ {L} [ E ]
$$
$$
\begin{array}{l} \mathcal {X} _ {L + E} - \mathcal {X} _ {L} = (I \otimes E) \vec {\omega} \vec {\omega} ^ {\intercal} (I \otimes L ^ {\intercal}) \\ + (I \otimes L) \vec {\omega} \vec {\omega} ^ {T} (I \otimes E ^ {T}) \\ - \vec {\xi} \vec {\omega} ^ {\intercal} (I \otimes E ^ {\intercal}) + (I \otimes E) \vec {\omega} \vec {\xi} ^ {\intercal} + o (\| E \|). \\ \end{array}
$$
Therefore, the second claim follows by the chain rule.
F.3 Proof of Proposition 4: Concentration bounds
We provide the proof for a detailed version of Proposition 4:
Proposition 4' (Concentration independent of length  $T$ ). Consider length  $T$  trajectories  $\{\mathcal{Y}_{[t_0,t_0 + T]}^i\}_{i = 1}^M$  and let  $\widehat{J}_T(L)\coloneqq \frac{1}{M}\sum_{i = 1}^{M}\varepsilon (L,\mathcal{Y}_T^i)$ . Then, under Assumption 4, for any  $s > 0$
$$
\begin{array}{l} \mathbb {P} \left[ | \widehat {J} _ {T} (L) - J _ {T} (L) | \leq s \right] \geq 1 - 2 n \exp \left[ \frac {- M s ^ {2} / 2}{\mu_ {L} ^ {2} + 2 \mu_ {L} s / 3} \right], \\ \mathbb {P} \left[ \| \nabla \widehat {J} _ {T} (L) - \nabla J _ {T} (L) \| \leq s \right] \geq 1 - 2 n \exp \left[ \frac {- M s ^ {2} / 2}{\nu_ {L} ^ {2} + 2 \nu_ {L} s / 3} \right] \\ \end{array}
$$
where  $\kappa_{L} = \kappa_{\xi} + \| L\| \kappa_{\omega}$  and
$$
\mu_ {L} := \frac {\kappa_ {L} ^ {2} C _ {L} ^ {2}}{\left[ 1 - \sqrt {\rho \left(A _ {L}\right)} \right] ^ {2}} \| H ^ {\intercal} H \| _ {*}
$$
$$
\nu_ {L} := \frac {2 \kappa_ {L} \kappa_ {\omega} C _ {L} ^ {2} + \left[ C _ {L} + 2 C _ {L} ^ {3} \rho (A _ {L}) ^ {3 / 2} \right] \| H \| \kappa_ {L} ^ {2}}{\left[ 1 - \sqrt {\rho (A _ {L})} \right] ^ {3}} \| H ^ {\intercal} H \| _ {*}.
$$
Proof of Proposition 4. Note that  $\mathbb{E}\left[\vec{\xi}\vec{\xi}^{\intercal}\right] = \begin{pmatrix} I\otimes Q & 0\\ 0 & P_0 \end{pmatrix} =: \mathcal{Q}, \mathbb{E}\left[\vec{\omega}\vec{\omega}^{\intercal}\right] = \begin{pmatrix} I\otimes R & 0\\ 0 & 0_m \end{pmatrix} =:$ $\mathcal{R}$ , and  $\mathbb{E}\left[\vec{\xi}\vec{\omega}^{\intercal}\right] = 0$ . Assume  $m_0 = 0$  and recall that  $\langle \nabla \varepsilon (L,\mathcal{Y}_T),E\rangle = \mathrm{d}\varepsilon (\cdot ,\mathcal{Y}_T)\big|_L(E)$  thus, using Proposition 3, we can rewrite the  $J_{T}(L)$  and its gradient as
$$
J _ {T} (L) = \mathbb {E} \left[ \varepsilon (L, \mathcal {Y} _ {T}) \right] = \mathbb {E} \left[ \| \vec {\eta} _ {L} \| _ {\mathcal {A} _ {L}} ^ {2} \right] = \mathrm {t r} \left[ \mathbb {E} \left[ \mathcal {X} _ {L} \right] \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right]
$$
where  $\mathbb{E}\left[\mathcal{X}_L\right] = \mathcal{Q} + (I\otimes L)\mathcal{R}(I\otimes L^{\intercal})$ . Therefore, by definition of  $\widehat{J}_T(L)$  we obtain
$$
\widehat {J} _ {T} (L) = \mathrm {t r} \left[ \mathcal {Z} _ {L} \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right]
$$
with  $\mathcal{Z}_L = \frac{1}{M}\sum_{i = 1}^{M}\mathcal{X}_L(\mathcal{Y}^i)$  which can be expanded as
$$
\mathcal {Z} _ {L} = \mathcal {Z} _ {1} + (I \otimes L) \mathcal {Z} _ {2} (I \otimes L ^ {\intercal}) - \mathcal {Z} _ {3} (I \otimes L ^ {\intercal}) - (I \otimes L) \mathcal {Z} _ {3} ^ {\intercal},
$$
$$
\mathcal {Z} _ {1} = \frac {1}{M} \sum_ {i = 1} ^ {M} \vec {\xi} _ {i} \vec {\xi} _ {i} ^ {\intercal}, \quad \mathcal {Z} _ {2} = \frac {1}{M} \sum_ {i = 1} ^ {M} \vec {\omega} _ {i} \vec {\omega} _ {i} ^ {\intercal}, \quad \mathcal {Z} _ {3} = \frac {1}{M} \sum_ {i = 1} ^ {M} \xi_ {i} \vec {\omega} _ {i} ^ {\intercal},
$$
and  $\mathbb{E}\left[\mathcal{Z}_L\right] = \mathcal{Q} + (I\otimes L)\mathcal{R}(I\otimes L^{\intercal})$  . Therefore,
$$
\widehat {J} _ {T} (L) - J _ {T} (L) = \mathrm {t r} \left[ \left(\mathcal {Z} _ {L} - \mathbb {E} \left[ \mathcal {Z} _ {L} \right]\right) \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right].
$$
Thus, by cyclic permutation property of trace and (12) we obtain
$$
\left| \widehat {J} _ {T} (L) - J _ {T} (L) \right| \leq \| \mathcal {A} _ {L} \left(\mathcal {Z} _ {L} - \mathbb {E} \left[ \mathcal {Z} _ {L} \right]\right) \mathcal {A} _ {L} ^ {\intercal} \| \| H ^ {\intercal} H \| _ {*}. \tag {34}
$$
Next, we consider the symmetric random matrix  $\mathcal{A}_L\left(\mathcal{Z}_L - \mathbb{E}\left[\mathcal{Z}_L\right]\right)\mathcal{A}_L^\intercal$ . Note that  $\| \xi (t) - L\omega (t)\| \leq \kappa_L$  almost surely and thus
$$
\| \mathcal {A} _ {L} \mathcal {X} _ {L} \mathcal {A} _ {L} ^ {\intercal} \| = \| \mathcal {A} _ {L} \vec {\eta} _ {L} \| ^ {2} \leq \kappa_ {L} ^ {2} \left[ \sum_ {i = 0} ^ {\infty} \| A _ {L} ^ {i} \| \right] ^ {2} \leq \mu_ {L} / \| H ^ {\intercal} H \| _ {*}.
$$
It then follows that
$$
\left| \left| \mathbb {E} \left[ (\mathcal {A} _ {L} \mathcal {X} _ {L} \mathcal {A} _ {L} ^ {\intercal}) ^ {2} \right] \right| \right| \leq \mathbb {E} \left[ \| \mathcal {A} _ {L} \mathcal {X} _ {L} \mathcal {A} _ {L} ^ {\intercal} \| ^ {2} \right] \leq \mu_ {L} ^ {2} / \| H ^ {\intercal} H \| _ {*} ^ {2}.
$$
Therefore, by Lemma 9 we obtain that
$$
\mathbb {P} \left[ \| \mathcal {A} _ {L} \left(\mathcal {Z} _ {L} - \mathbb {E} \left[ \mathcal {Z} _ {L} \right]\right) \mathcal {A} _ {L} ^ {\intercal} \| \geq t \right] \leq 2 n \exp \left[ \frac {- M \| H ^ {\intercal} H \| _ {*} ^ {2} t ^ {2} / 2}{\mu_ {L} ^ {2} + 2 \mu_ {L} \| H ^ {\intercal} H \| _ {*} t / 3} \right].
$$
Substituting  $t$  with  $t / \| H^{\intercal}H\|_{*}$  together with (34) implies the first claim.
Similarly, we can compute that
$$
\begin{array}{l} \langle \nabla J _ {T} (L), E \rangle = \left\langle \mathbb {E} \left[ \nabla \varepsilon (L, \mathcal {Y} _ {T}) \right], E \right\rangle \\ = 2 \operatorname {t r} \left[ (I \otimes L) \mathcal {R} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right] + \operatorname {t r} \left[ (\mathcal {Q} + (I \otimes L) \mathcal {R} (I \otimes L ^ {\intercal})) \mathcal {N} _ {L} [ E ] \right], \\ \end{array}
$$
$$
\begin{array}{l} \langle \nabla \widehat {J} _ {T} (L) - \nabla J _ {T} (L), E \rangle = - 2 \operatorname {t r} \left[ \mathcal {Z} _ {3} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} H ^ {\intercal} H \mathcal {A} _ {L} \right] \\ + 2 \operatorname {t r} \left[ (I \otimes L) (\mathcal {Z} _ {2} - \mathcal {R}) (I \otimes E ^ {\top}) \mathcal {A} _ {L} ^ {\top} H ^ {\top} H \mathcal {A} _ {L} \right] \\ + \operatorname {t r} \left[ \left(\mathcal {Z} _ {L} - \mathbb {E} \left[ \mathcal {Z} _ {L} \right]\right) \mathcal {N} _ {L} [ E ] \right]. \\ \end{array}
$$
Thus, by cyclic permutation property of trace and (12) we obtain that
$$
\left| \left\langle \nabla \widehat {J} _ {T} (L) - \nabla J _ {T} (L), E \right\rangle \right| \leq \| \frac {1}{M} \sum_ {i = 1} ^ {M} S _ {L} \left(E, \mathcal {Y} _ {T} ^ {i}\right) - \mathbb {E} \left[ S _ {L} \left(E, \mathcal {Y} _ {T} ^ {i}\right) \right] \| \| H ^ {\intercal} H \| _ {*} \tag {35}
$$
where  $S_{L}(E,\mathcal{Y})$  is the symmetric part of the following random matrix
$$
- 2 \mathcal {A} _ {L} \vec {\xi} \vec {\omega} ^ {\intercal} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} + 2 \mathcal {A} _ {L} (I \otimes L) \vec {\omega} \vec {\omega} ^ {\intercal} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} + 2 \mathcal {A} _ {L} \mathcal {X} _ {L} \mathcal {M} _ {L} [ E ] ^ {\intercal}.
$$
Next, we provide the following almost sure bounds for each term: first,
$$
\begin{array}{l} \left\| \mathcal {A} _ {L} \mathcal {Z} _ {3} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} \right\| \leq \left\| \mathcal {A} _ {L} \vec {\xi} \right\| \left\| \mathcal {A} _ {L} (I \otimes E) \vec {\omega} \right\| \\ \leq \kappa_ {\xi} \left[ \sum_ {i = 0} ^ {T} \| A _ {L} ^ {i} \| \right] \kappa_ {\omega} \| E \| \left[ \sum_ {i = 0} ^ {T} \| A _ {L} ^ {i} \| \right] \\ \leq \kappa_ {\xi} \kappa_ {\omega} \frac {C _ {L} ^ {2}}{[ 1 - \sqrt {\rho (A _ {L})} ] ^ {2}} \| E \| \\ \end{array}
$$
where the last equality follows by Lemma 6; second, similarly
$$
\begin{array}{l} \left\| \mathcal {A} _ {L} (I \otimes L) \vec {\omega} \vec {\omega} ^ {\intercal} (I \otimes E ^ {\intercal}) \mathcal {A} _ {L} ^ {\intercal} \right\| \\ \leq \left\| \mathcal {A} _ {L} (I \otimes L) \vec {\omega} \right\| \left\| \mathcal {A} _ {L} (I \otimes E) \vec {\omega} \right\| \\ \leq \kappa_ {\omega} ^ {2} \| L \| \| E \| \left[ \sum_ {i = 0} ^ {T} \| A _ {L} ^ {i} \| \right] \\ \leq \kappa_ {\omega} ^ {2} \frac {C _ {L} ^ {2}}{\left[ 1 - \sqrt {\rho (A _ {L})} \right] ^ {2}} \| L \| \| E \|; \\ \end{array}
$$
$$
\begin{array}{l} \left\| \mathcal {A} _ {L} \mathcal {X} _ {L} \mathcal {M} _ {L} [ E ] ^ {\mathsf {T}} \right\| \leq \left\| \mathcal {A} _ {L} \vec {\eta} _ {L} \right\| \left\| \mathcal {M} _ {L} [ E ] \vec {\eta} _ {L} \right\| \\ \leq \kappa_ {L} ^ {2} \left[ \sum_ {i = 0} ^ {T} \| A _ {L} ^ {i} \| \right] \left[ \sum_ {i = 0} ^ {T} \| M _ {i} [ E ] \| \right] \\ \leq \kappa_ {L} ^ {2} \left[ \frac {C _ {L} + 2 C _ {L} ^ {3} \rho (A _ {L}) ^ {3 / 2}}{\left[ 1 - \sqrt {\rho (A _ {L})} \right] ^ {3}} \right] \| E H \| \\ \end{array}
$$
where the last inequality follows by Lemma 6. Now, by combining the last three bounds we can claim that almost surely
$$
\| S _ {L} (E, \mathcal {Y}) \| \leq \frac {\nu_ {L}}{\| H ^ {\tau} H \| _ {*}} \| E \|.
$$
This also implies that
$$
\left\| \mathbb {E} \left[ S _ {L} (E, \mathcal {Y}) ^ {2} \right] \right\| \leq \mathbb {E} \left[ \left\| S _ {L} (E, \mathcal {Y}) \right\| ^ {2} \right] \leq \frac {\nu_ {L} ^ {2}}{\left\| H ^ {\intercal} H \right\| _ {*} ^ {2}} \| E \| ^ {2}.
$$