SlowGuess/NeurIPS_2023_OCR huggingface dataset:
E.1 Proof of lemma 3: Derivation of stochastic gradient formula 24
E.2 Proof of Lemma 4: Robustness of the policy with respect to perturbation 24
E.3 Proof of Lemma 5: Uniform lower-bound on stepsize 24
E.4 Proof of Proposition 2: Linear decay in cost value 25
E.5 Proof of Theorem 1: Convergence of SGD algorithm 26
F Proofs of the result for observation model and sample complexity 27
F.1 Preliminary lemmas and their proofs 27
F.2 Proof of Proposition 3: The Observation model 30
F.3 Proof of Proposition 4: Concentration bounds 30
F.4 Proof of the Proposition 5: Truncation error bound 33
F.5 Complete version of Theorem 2: Sample complexity bounds for the stochastic oracle 34
F.6 Additional concentration bound results 35
G Numerical Results 37
H Nomenclature 37
Appendix A Discussion of additional related works
After the initial submission of this manuscript, we encountered the following related works, and we will now briefly discuss their connections and differences in relation to our work: [36] establishes an end-to-end sample complexity bound on learning a robust LQG controller (which is different than our filtering problem). As the system parameters are also unknown, this work only considers the open-loop stable systems. Additionally, it establishes a nice trade-off between optimality and robustness characterized by a suboptimality performance bound as a result of the robust LQG design. While our filtering design is based on the knowledge of system parameters, we do not consider the open-loop stability assumption; and the robust synthesis is included as one of the main future directions of this work. Furthermore, the complexity bounds in [36] depends on the length of trajectory, whereas ours does not. Also, their suboptimality bound scale as  $O(1 / \sqrt{N})$  in the number of trajectories, whereas ours scales as  $O(1 / N)$ . Next, [37] similarly considers the problem of learning the steady-state Kalman gain but in a different setup: The model is assumed to be completely unknown. However, the algorithm requires access to a simulator that provides noisy measurement of the MSE  $\mathbb{E}[\| X(t) - \hat{X}(t)\|^2]$  which requires generation of ground-truth state trajectories  $X(t)$  (see Assumption 3.2 therein). The proposed approaches are different: zeroth-order global optimization (theirs) vs first-order stochastic gradient descent (ours). The analysis reports a  $O(\varepsilon^{-2})$  sample complexity on optimal policy error which aligns with our result. However, it is difficult to provide more detailed comparison as explicit dependence of the error terms on problem dimension is not provided. Finally, [38] considers the problem of simultaneously learning the optimal Kalman filter and linear feedback control policy in the LQG setup. Their approach involves solving Semidefinite Programmings (SDP) using input-state-output trajectories. Their result, for the case when trajectories involve noise, relies on the assumption that the magnitude of the noise can be made arbitrary small. This is in contrast to our setup where we only assume a bound on the noise level and do not require access to state trajectories.
Appendix B Summary of Assumptions
Herein, we provide a summary of all assumptions considered in our work and their relation to our results.
Note that if the given pair  $(A, H)$  of system parameters is not detectable, then the estimation problem is not well-posed, simply because there may not exist any stabilizing policy  $L_0$ . Therefore, we consider the minimum required assumption for well-posedness of the problem as follows:
It is indeed possible to work with the weaker conditions compared with Assumption 1 and obtain similar results presented here—by incorporating more system theoretic tools. However, in order to improve the clarity of the learning problem with less system theoretic technicalities, we work with the following stronger assumption in lieu of Assumption 1.
Assumption 2. The pair  $(A, H)$  is observable, and the noise covariances  $Q \succ 0$  and  $R \succ 0$ .
It is clear that Assumption 2 implies Assumption 1.
Next, in order to provide an independent analysis of the SGD algorithm for locally Lipschitz functions in presence of gradient bias, we consider the following assumption on the gradient oracle.
Assumption 3. Suppose, for some  $\alpha >0$ , we have access to a biased estimate of the gradient  $\nabla \widehat{J} (L)$  such that, there exists constants  $s,s_0 > 0$  implying  $\| \nabla \widehat{J} (L) - \nabla J(L)\| _F\leq s\| \nabla J(L)\| _F + s_0$  for all  $L\in S_{\alpha}\setminus C_{\tau}$ .
Next, in order to utilize the complexity analysis of SGD algorithm provided in Theorem 1, we need to build an oracle that satisfies Assumption 3. This is oracle is built in Section 4.2, building on the following assumptions on the process and observation noise.
Then, based on this assumption, Theorem 2 provides lower bounds on the trajectory length and the batch-size in order for the oracle to satisfy Assumption 3. Finally, the combined version of our results is presented in Theorem 3.
Appendix C Proof of the duality relationship: Proposition 1
1. By pairing the original state dynamics (1) and its dual (6):
$$
z (t + 1) ^ {\intercal} x (t + 1) - z (t) ^ {\intercal} x (t) = z (t + 1) ^ {\intercal} \xi (t) + u (t + 1) ^ {\intercal} H x (t).
$$
Summing this relationship from  $t = 0$  to  $t = T - 1$  yields,
$$
z (T) ^ {\intercal} x (T) = z (0) ^ {\intercal} x (0) + \sum_ {t = 0} ^ {T - 1} z (t + 1) ^ {\intercal} \xi (t) + u (t + 1) ^ {\intercal} H x (t).
$$
Upon subtracting the estimate  $a^{\mathsf{T}}\hat{x}_{\mathcal{L}}(T)$ , using the adjoint relationship
$$
\sum_ {t = 0} ^ {T - 1} u (t + 1) ^ {\intercal} y (t) = a ^ {\intercal} \hat {x} _ {\mathcal {L}} (T). \tag {14}
$$
and  $z(T) = a$ , lead to
$$
a ^ {\intercal} x (T) - a ^ {\intercal} \hat {x} _ {\mathcal {L}} (T) = z (0) ^ {\intercal} x (0) + \sum_ {t = 0} ^ {T - 1} z (t + 1) ^ {\intercal} \xi (t) - u (t + 1) ^ {\intercal} w (t).
$$
Squaring both sides and taking the expectation concludes the duality result in (8).
2. Consider the adjoint system (6) with the linear feedback law  $u(t) = L^{\intercal}z(t)$ . Then,
$$
z (t) = \left(A _ {L} ^ {\intercal}\right) ^ {T - t} a, \quad \text {f o r} \quad t = 0, 1, \dots , T. \tag {15}
$$
Therefore, as a function of  $a$ ,  $u(t) = L^{\intercal}(A_L^{\intercal})^{T - t}a$ . These relationships are used to identify the control policy
$$
\mathcal {L} ^ {\dagger} (a) = (u (1), \dots , u (T)) = \left(L ^ {\intercal} \left(A _ {L} ^ {\intercal}\right) ^ {T - 1} a, \dots , L ^ {\intercal} a\right).
$$
This control policy corresponds to an estimation policy by the adjoint relationship (14):
$$
a ^ {\intercal} \hat {x} _ {\mathcal {L}} (T) = \sum_ {t = 0} ^ {T - 1} a ^ {\intercal} A _ {L} ^ {T - t - 1} L y (t), \quad \forall a \in \mathbb {R} ^ {n}.
$$
This relationship holds for all  $a \in \mathbb{R}^n$ . Therefore,
$$
\hat {x} _ {\mathcal {L}} (T) = \sum_ {t = 0} ^ {T - 1} A _ {L} ^ {T - t - 1} L y (t),
$$
which coincides with the Kalman filter estimate with constant gain  $L$  given by the formula (4) (with  $m_0 = 0$ ). Therefore, the adjoint relationship (14) relates the control policy with constant gain  $L^{\intercal}$  to the Kalman filter with constant gain  $L$ . The result (9) follows from the identity
$$
\begin{array}{l} \mathbb {E} \left[ \| y (T) - \hat {y} _ {L} (T) \| ^ {2} \right] = \mathbb {E} \left[ \| H x (T) - H \hat {x} _ {L} (T) \| ^ {2} + \| w (T) \| ^ {2} \right] \\ = \sum_ {i = 1} ^ {m} \mathbb {E} \left[ \left| H _ {i} ^ {\mathsf {T}} x (T) - H _ {i} ^ {\mathsf {T}} \hat {x} _ {L} (T) \right| ^ {2} \right] + \operatorname {t r} [ R ], \\ \end{array}
$$
and the application of the first result (9) with  $a = H_{i}$ .
Appendix D Proofs for the results for the analysis of the optimization landscape
D.1 Preliminary lemma
The following lemmas are a direct consequence of duality and useful for our subsequent analysis.
Lemma 7. The set of Schur stabilizing gains  $S$  is regular open, contractible, and unbounded when  $m \geq 2$  and the boundary  $\partial S$  coincides with the set  $\{L \in \mathbb{R}^{n \times m} : \rho(A - LH) = 1\}$ . Furthermore,  $J(.)$  is real analytic on  $S$ .
Proof. Consider the duality described in Remark 3. The proof then follows identical to [23, Lemmas 3.5 and 3.6] by noting that the spectrum of a matrix is identical to the spectrum of its transpose.  $\square$
Next, we present the proof of Lemma 1 that provides sufficient conditions to recover the coercive property of  $J(\cdot)$  which resembles Lemma 3.7 in [23] (but extended for the time-varying parameters). The proof is removed on the account of space and appears in the extended version of this work.
Remark 9. This approach recovers the claimed coercivity also in the control setting with weaker assumptions. In particular, using this result, one can replace the positive definite condition on the covariance of the initial condition in [23], i.e.,  $\Sigma \succ 0$ , with just the controllability of  $(A,\Sigma^{1 / 2})$ .
Proof. Consider any  $L \in S$  and note that the right eigenvectors of  $A$  and  $A_L$  that are annihilated by  $H$  are identical. Thus, by Popov-Belevitch-Hautus (PBH) test, observability of  $(A, H)$  is equivalent to observability of  $(A_L, H)$ . Therefore, there exists a positive integer  $n_0 \leq n$  such that
$$
H _ {n _ {0}} ^ {\intercal} (L) := \left( \begin{array}{c c c c} H ^ {\intercal} & A _ {L} ^ {\intercal} H ^ {\intercal} & \ldots & (A _ {L} ^ {\intercal}) ^ {n _ {0} - 1} H ^ {\intercal} \end{array} \right)
$$
is full-rank, implying that  $H_{n_0}^\intercal (L)H_{n_0}(L)$  is positive definite. Now, recall that for any such stabilizing gain  $L$ , we compute
$$
\begin{array}{l} J (L) = \operatorname {t r} \left[ \sum_ {t = 0} ^ {\infty} \left(A _ {L}\right) ^ {t} \left(Q + L R L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \right] \\ = \operatorname {t r} \left[ \sum_ {t = 0} ^ {\infty} \sum_ {k = 0} ^ {n _ {0} - 1} \left(A _ {L}\right) ^ {n _ {0} t + k} \left(Q + L R L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0} t + k} H ^ {\intercal} H \right] \\ = \operatorname {t r} \left[ \sum_ {t = 0} ^ {\infty} \left(A _ {L}\right) ^ {n _ {0} t} \left(Q + L R L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0} t} H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L) \right] \\ =: \operatorname {t r} \left[ X _ {n _ {0}} (L) H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L) \right], \\ \end{array}
$$
where we used the cyclic property of trace and the inequality follows because for any PSD matrices  $P_{1}, P_{2} \succeq 0$  we have
$$
\operatorname {t r} \left[ P _ {1} P _ {2} \right] = \operatorname {t r} \left[ P _ {2} ^ {\frac {1}{2}} P _ {1} P _ {2} ^ {\frac {1}{2}} \right] \geq 0. \tag {16}
$$
Also,  $X_{n_0}(L)$  is well defined because  $A_{L}$  is Schur stable if and only if  $(A_L)^{n_0}$  is. Moreover,  $X_{n_0}(L)$  coincides with the unique solution to the following Lyapunov equation
$$
X _ {n _ {0}} (L) = \left(A _ {L}\right) ^ {n _ {0}} X _ {n _ {0}} (L) \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0}} + Q + L R L ^ {\intercal}.
$$
Next, as  $R \succeq 0$ ,
$$
\begin{array}{l} J (L) \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)\right) \operatorname {t r} \left[ X _ {n _ {0}} (L) \right] \\ \geq \underline {{\lambda}} (H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)) \mathrm {t r} \left[ \sum_ {t = 0} ^ {\infty} (A _ {L}) ^ {n _ {0} t} Q (A _ {L} ^ {\intercal}) ^ {n _ {0} t} \right] \\ \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)\right) \underline {{\lambda}} (Q) \sum_ {t = 0} ^ {\infty} \operatorname {t r} \left[ \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0} t} \left(A _ {L}\right) ^ {n _ {0} t} \right] \\ \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)\right) \underline {{\lambda}} (Q) \sum_ {t = 0} ^ {\infty} \rho \left(A _ {L}\right) ^ {2 n _ {0} t}, \tag {17} \\ \end{array}
$$
where the last inequality follows by the fact that
$$
\mathrm {t r} \left[ (A _ {L} ^ {\mathsf {T}}) ^ {n _ {0} t} (A _ {L}) ^ {n _ {0} t} \right] = \| (A _ {L}) ^ {n _ {0} t} \| _ {F} ^ {2} \geq \| (A _ {L}) ^ {n _ {0} t} \| _ {\mathrm {o p}} ^ {2} \geq \rho ((A _ {L}) ^ {n _ {0} t}) ^ {2} = \rho (A _ {L}) ^ {2 n _ {0} t},
$$
with  $\| \cdot \|_{\mathrm{op}}$  denoting the operator norm induced by 2-norm. Now, by Lemma 7 and continuity of the spectral radius, as  $L_{k}\to \partial S$  we observe that  $\rho (A_{L_k})\rightarrow 1$ . But then, the obtained lowerbound implies that  $J(L_{k})\rightarrow \infty$ . On the other hand, as  $Q\succ 0$ ,  $R\succ 0$  are both time-independent, by using a similar technique we also provide the following lowerbound
$$
\begin{array}{l} J (L) \geq \operatorname {t r} \left[ (Q + L R L ^ {\intercal}) \sum_ {t = 0} ^ {\infty} (A _ {L} ^ {\intercal}) ^ {n _ {0} t} H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L) (A _ {L}) ^ {n _ {0} t} \right] \\ \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)\right) \operatorname {t r} \left[ Q + L R L ^ {\intercal} \right] \\ \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\mathsf {T}} (L) H _ {n _ {0}} (L)\right) \operatorname {t r} \left[ R L L ^ {\mathsf {T}} \right] \\ \geq \underline {{\lambda}} \left(H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L)\right) \underline {{\lambda}} (R) \| L \| _ {F} ^ {2}, \\ \end{array}
$$
where  $\| \cdot \| _F$  denotes the Frobenius norm. Therefore, by equivalency of norms on finite dimensional spaces,  $\| L_k\| \to \infty$  implies that  $J(L_{k})\rightarrow \infty$  which concludes that  $J(.)$  is coercive on  $\mathcal{S}$ . Finally, note that for any  $L\notin S$ , by (17) we can argue that  $J(L) = \infty$ , therefore the sublevel sets  $S_{\alpha}\subset S$  whenever  $\alpha$  is finite. The compactness of  $S_{\alpha}$  is then a direct consequence of the coercive property and continuity of  $J(.)$  (Lemma 7).
D.3 Derivation of the gradient formula in Lemma 2
Next, we aim to compute the gradient of  $J$  in a more general format. We do the derivation for time-varying  $R$  and  $Q$  with specialization to time-invariant setting at the end. For any admissible  $\Delta$ , we have
$$
\begin{array}{l} X _ {\infty} (L + \Delta) - X _ {\infty} (L) = \sum_ {t = 1} ^ {\infty} \left(A _ {L}\right) ^ {t} \left(Q _ {t} + L R _ {t} L ^ {\intercal}\right) (\star) ^ {\intercal} + (\star) \left(Q _ {t} + L R _ {t} L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {t} \\ - \sum_ {t = 0} ^ {\infty} \left(A _ {L}\right) ^ {t} \left(\Delta R _ {t} L ^ {\intercal} + L R _ {t} \Delta^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {t} + o (\| \Delta \|), \\ \end{array}
$$
where the  $\star$  is hiding the following term
$$
\sum_ {k = 1} ^ {t} (A _ {L}) ^ {t - k} \Delta H (A _ {L}) ^ {k - 1}.
$$
Therefore, by linearity and cyclic permutation property of trace, we get that
$$
\begin{array}{l} J (L + \Delta) - J (L) = \operatorname {t r} \left[ \Delta H \sum_ {t = 1} ^ {\infty} \sum_ {k = 1} ^ {t} 2 \left(A _ {L}\right) ^ {k - 1} \left(Q _ {t} + L R _ {t} L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \left(A _ {L}\right) ^ {t - k} \right] \\ - \mathrm {t r} \left[ \Delta \sum_ {t = 0} ^ {\infty} 2 R _ {t} L ^ {\intercal} (A _ {L} ^ {\intercal}) ^ {t} H ^ {\intercal} H (A _ {L}) ^ {t} \right] + o (\| \Delta \|). \\ \end{array}
$$
Finally, by considering the Euclidean metric on real matrices induced by the inner product  $\langle Q,P\rangle =$  tr  $[Q^{\mathrm{T}}P]$ , we obtain the gradient of  $J$  as follows
$$
\begin{array}{l} \nabla J (L) = - 2 \sum_ {t = 0} ^ {\infty} \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \left(A _ {L}\right) ^ {t} L R _ {t} \\ + 2 \sum_ {t = 1} ^ {\infty} \sum_ {k = 1} ^ {t} \left(A _ {L} ^ {\intercal}\right) ^ {t - k} H ^ {\intercal} H \left(A _ {L}\right) ^ {t} \left(Q _ {t} + L R _ {t} L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {k - 1} H ^ {\intercal}, \\ \end{array}
$$
whenever the series are convergent! And, by switching the order of the sums it simplifies to
$$
\begin{array}{l} \nabla J (L) = - 2 \sum_ {t = 0} ^ {\infty} \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \left(A _ {L}\right) ^ {t} L R _ {t} \\ + 2 \sum_ {k = 1} ^ {\infty} \sum_ {t = k} ^ {\infty} \left[ \left(A _ {L} ^ {\intercal}\right) ^ {t - k} H ^ {\intercal} H \left(A _ {L}\right) ^ {t - k} \right] A _ {L} \left[ \left(A _ {L}\right) ^ {k - 1} \left(Q _ {t} + L R _ {t} L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {k - 1} \right] H ^ {\intercal}. \\ = - 2 \sum_ {t = 0} ^ {\infty} \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \left(A _ {L}\right) ^ {t} L R _ {t} + 2 \sum_ {t = 0} ^ {\infty} \left[ \left(A _ {L} ^ {\intercal}\right) ^ {t} H ^ {\intercal} H \left(A _ {L}\right) ^ {t} \right] \\ \cdot A _ {L} \left[ \sum_ {k = 0} ^ {\infty} \left(A _ {L}\right) ^ {k} \left(Q _ {t + k + 1} + L R _ {t + k + 1} L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {k} \right] H ^ {\intercal}. \\ \end{array}
$$
For the case of time-independent  $Q$  and  $R$ , this reduces to
$$
\begin{array}{l} \nabla J (L) = - 2 Y _ {(L)} L R + 2 Y _ {(L)} A _ {L} \left[ \sum_ {k = 0} ^ {\infty} \left(A _ {L}\right) ^ {k} \left(Q + L R L ^ {\intercal}\right) \left(A _ {L} ^ {\intercal}\right) ^ {k} \right] H ^ {\intercal} \\ = 2 Y _ {(L)} \left[ - L R + A _ {L} X _ {(L)} H ^ {\intercal} \right]. \\ \end{array}
$$
where  $Y_{(L)} = Y$  is the unique solution of
$$
Y = A _ {L} ^ {\intercal} Y A _ {L} + H ^ {\intercal} H.
$$
D.4 Proof of existence of global minimizer in Lemma 2
The domain  $S$  is non-empty whenever  $(A, H)$  is observable. Thus, by continuity of  $L \to J(L)$ , there exists some finite  $\alpha > 0$  such that the sublevel set  $S_{\alpha}$  is non-empty and compact. Therefore, the minimizer is an interior point and thus must satisfy the first-order optimality condition  $\nabla J(L^{*}) = 0$ . Therefore, by coercivity, it is stabilizing and unique which satisfies
$$
L ^ {*} = A X ^ {*} H ^ {\intercal} \left(R + H X ^ {*} H ^ {\intercal}\right) ^ {- 1},
$$
with  $X^{*}$  being the unique solution of
$$
X ^ {*} = A _ {L ^ {*}} X ^ {*} A _ {L ^ {*}} ^ {\intercal} + Q + L ^ {*} R \left(L ^ {*}\right) ^ {\intercal}. \tag {18}
$$
As expected, the global minimizer  $L^{*}$  is equal to the steady-state Kalman gain, but explicitly dependent on the noise covariances  $Q$  and  $R$ .
D.5 Proof of gradient dominance property in Lemma 2
Note that  $X = X_{(L)}$  satisfies
$$
X = A _ {L} X A _ {L} ^ {\intercal} + Q + L R L ^ {\intercal}. \tag {19}
$$
Then, by combining (18) and (19), and some algebraic manipulation, we recover part of the gradient information, i.e.,  $(-LR + A_{L}XH^{\intercal})$ , in the gap of cost matrices by arriving at the following identity
$$
\begin{array}{l} X - X ^ {*} - A _ {L ^ {*}} (X - X ^ {*}) A _ {L ^ {*}} ^ {\intercal} \\ = \left(L R - A _ {L} X H ^ {\intercal}\right) \left(L - L ^ {*}\right) ^ {\intercal} + \left(L - L ^ {*}\right) \left(R L ^ {\intercal} - H X A _ {L} ^ {\intercal}\right) \\ - \left(L - L ^ {*}\right) R \left(L - L ^ {*}\right) ^ {\intercal} - \left(L - L ^ {*}\right) H X H ^ {\intercal} \left(L - L ^ {*}\right) ^ {\intercal} \tag {20} \\ \preceq \frac {1}{a} \left(L R - A _ {L} X H ^ {\intercal}\right) \left(R L ^ {\intercal} - H X A _ {L} ^ {\intercal}\right) + a \left(L - L ^ {*}\right) \left(L - L ^ {*}\right) ^ {\intercal} \\ - (L - L ^ {*}) (R + H X H ^ {\intercal}) (L - L ^ {*}) ^ {\intercal} \\ \end{array}
$$
where the upperbound is valid for any choice of  $a > 0$ . Now, as  $R \succ 0$ , we choose  $a = \underline{\lambda}(R)/2$ . As  $X \succeq 0$ , it further upperbounds
$$
\begin{array}{l} X - X ^ {*} - A _ {L ^ {*}} \left(X - X ^ {*}\right) A _ {L ^ {*}} ^ {\intercal} \\ \preceq \frac {2}{\underline {{\lambda}} (R)} (- L R + A _ {L} X H ^ {\intercal}) (- R L ^ {\intercal} + H X A _ {L} ^ {\intercal}) \\ - \frac {\lambda (R)}{2} (L - L ^ {*}) (L - L ^ {*}) ^ {\intercal}. \\ \end{array}
$$
Now, let  $\hat{X}$  and  $\hat{X}$  be, respectively, the unique solution of the following Lyapunov equations
$$
\tilde {X} = A _ {L ^ {*}} \tilde {X} A _ {L ^ {*}} ^ {\intercal} + (- L R + A _ {L} X H ^ {\intercal}) (- R L ^ {\intercal} + H X A _ {L} ^ {\intercal}),
$$
$$
\widehat {X} = A _ {L ^ {*}} \widehat {X} A _ {L ^ {*}} ^ {\intercal} + (L - L ^ {*}) (L - L ^ {*}) ^ {\intercal}.
$$
Then by comparison, we conclude that
$$
X - X ^ {*} \preceq \frac {2}{\underline {{\lambda}} (R)} \tilde {X} - \frac {\underline {{\lambda}} (R)}{2} \widehat {X}.
$$
Recall that by the fact in (16),
$$
J (L) - J \left(L ^ {*}\right) = \operatorname {t r} \left[ \left(X - X ^ {*}\right) H ^ {\intercal} H \right] \leq \frac {2}{\underline {{\lambda}} (R)} \operatorname {t r} \left[ \tilde {X} H ^ {\intercal} H \right] - \frac {\underline {{\lambda}} (R)}{2} \operatorname {t r} \left[ \widehat {X} H ^ {\intercal} H \right]. \tag {21}
$$
Let  $Y^{*} \succ 0$  be the unique solution of
$$
Y ^ {*} = A _ {L ^ {*}} ^ {\intercal} Y ^ {*} A _ {L ^ {*}} + H ^ {\intercal} H,
$$
then, by cyclic permutation property
$$
\begin{array}{l} \operatorname {t r} \left[ \tilde {X} H ^ {\intercal} H \right] = \operatorname {t r} \left[ (- L R + A _ {L} X H ^ {\intercal}) (- R L ^ {\intercal} + H X A _ {L} ^ {\intercal}) Y ^ {*} \right] \\ \leq \frac {\bar {\lambda} (Y ^ {*})}{\underline {{\lambda}} ^ {2} (Y _ {(L)})} \operatorname {t r} \left[ Y _ {(L)} (- L R + A _ {L} X H ^ {\intercal}) (- R L ^ {\intercal} + H X A _ {L} ^ {\intercal}) Y _ {(L)} \right] \\ = \frac {\bar {\lambda} \left(Y ^ {*}\right)}{4 \underline {{\lambda}} ^ {2} \left(Y _ {(L)}\right)} \langle \nabla J (L), \nabla J (L) \rangle \tag {22} \\ \end{array}
$$
where the last equality follows by the obtained formula for the gradient  $\nabla J(L)$ . Similarly, we obtain that
$$
\operatorname {t r} \left[ \widehat {X} H ^ {\intercal} H \right] = \operatorname {t r} \left[ (L - L ^ {*}) (L - L ^ {*}) ^ {\intercal} Y ^ {*} \right] \geq \underline {{\lambda}} (Y ^ {*}) \| L - L ^ {*} \| _ {F} ^ {2}. \tag {23}
$$
Notice that the mapping  $L \to Y_{(L)}$  is continuous on  $S \supseteq S_{\alpha}$ , and also by observability of  $(A, H)$ ,  $Y_{(L)} \succ 0$  for any  $L \in S$ . To see this, let  $H_{n_0}(L) \succ 0$  be as defined in Lemma 1. Then,
$$
\begin{array}{l} Y _ {(L)} = \sum_ {t = 0} ^ {\infty} (A _ {L} ^ {\intercal}) ^ {t} (H ^ {\intercal} H) (A _ {L}) ^ {t} \\ = \sum_ {t = 0} ^ {\infty} \sum_ {k = 0} ^ {n _ {0} - 1} \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0} t + k} \left(H ^ {\intercal} H\right) \left(A _ {L}\right) ^ {n _ {0} t + k} \\ = \sum_ {t = 0} ^ {\infty} \left(A _ {L} ^ {\intercal}\right) ^ {n _ {0} t} H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L) \left(A _ {L}\right) ^ {n _ {0} t} \\ \succeq H _ {n _ {0}} ^ {\intercal} (L) H _ {n _ {0}} (L) \succ 0. \\ \end{array}
$$
Now, by Lemma 1,  $S_{\alpha}$  is compact and therefore we claim that the following infimum is attained with some positive value  $\kappa_{\alpha}$ :
$$
\inf  _ {L \in \mathcal {S} _ {\alpha}} \underline {{\lambda}} (Y _ {(L)}) =: \kappa_ {\alpha} > 0. \tag {24}
$$
Finally, the first claimed inequality follows by combining the inequalities (21), (22) and (23), with the following choice of
$$
c _ {1} (\alpha) := \frac {2   \underline {{\lambda}} (R)}{\overline {{\lambda}} (Y ^ {*})} \kappa_ {\alpha} ^ {2}, \quad \text {a n d} \quad c _ {2} (\alpha) := \frac {\underline {{\lambda}} (Y ^ {*})   \underline {{\lambda}} (R) ^ {2}}{\overline {{\lambda}} (Y ^ {*})} \kappa_ {\alpha} ^ {2}.
$$
For the second claimed inequality, one arrive at the following identity by similar computation to (20):