SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The supplementary material is structured as follows. Appendix §A presents the well-known solution to the classic LQG problem using dynamic programming and Kalman Filter estimation. Appendix §B provides the definitions of the stacked system matrices utilized in the compact formulation (5) of the distributionally robust LQG problem. Appendix §C contains the proofs of the formal statements in the main text and provides additional technical results. Appendix §D derives the SDP reformulation of the dual problem (11). Appendix §E, finally, elaborates on the bisection algorithm used for solving the linearization oracle of the Frank-Wolfe algorithm.
A. Solution of the LQG Problem
The classic LQG problem can be solved efficiently via dynamic programming; see, e.g., [8]. That is, the unique optimal control inputs satisfy  $u_{t}^{\star} = K_{t}\hat{x}_{t}$  for every  $t\in [T - 1]$ , where  $K_{t}\in \mathbb{R}^{n\times n}$  is the optimal feedback gain matrix, and  $\hat{x}_t = \mathbb{E}_{\mathbb{P}}[x_t|y_0,\dots ,y_t]$  is the minimum mean-squared-error estimator of  $x_{t}$  given the observation history up to time  $t$ . Thanks to the celebrated separation principle,  $K_{t}$  can be computed by pretending that the system is deterministic and allows for perfect state observations, and  $\hat{x}_t$  can be computed while ignoring the control problem.
To compute  $K_{t}$ , one first solves the deterministic LQR problem corresponding to the LQG problem at hand. Its value function  $x_{t}^{\top} P_{t} x_{t}$  at time  $t$  is quadratic in  $x_{t}$ , and  $P_{t}$  obeys the backward recursion
$$
P _ {t} = A _ {t} ^ {\top} P _ {t + 1} A _ {t} + Q _ {t} - A _ {t} ^ {\top} P _ {t + 1} B _ {t} \left(R _ {t} + B _ {t} ^ {\top} P _ {t + 1} B _ {t}\right) ^ {- 1} B _ {t} ^ {\top} P _ {t + 1} A _ {t} \quad \forall t \in [ T - 1 ] \tag {A.15a}
$$
initialized by  $P_{T} = Q_{T}$ . The optimal feedback gain matrix  $K_{t}$  can then be computed from  $P_{t + 1}$  as
$$
K _ {t} = - \left(R _ {t} + B _ {t} ^ {\top} P _ {t + 1} B _ {t}\right) ^ {- 1} B _ {t} ^ {\top} P _ {t + 1} A _ {t} \quad \forall t \in [ T - 1 ]. \tag {A.15b}
$$
Since  $x_{t}$  and  $(y_0, \ldots, y_t)$  are jointly normally distributed, the minimum mean-squared-error estimator  $\hat{x}_t$  can be calculated directly using the formula for the mean of a conditional normal distribution. Alternatively, however, one can use the Kalman filter to compute  $\hat{x}_t$  recursively, which is significantly more insightful and efficient. The Kalman filter also recursively computes the covariance matrix  $\Sigma_t$  of  $x_{t}$  conditional on  $y_0, \ldots, y_t$  and the covariance matrix  $\Sigma_{t+1|t}$  of  $x_{t+1}$  conditional on  $y_0, \ldots, y_t$  evaluated under  $\mathbb{P}$ . Specifically, these covariance matrices obey the forward recursion
$$
\left. \begin{array}{l} \Sigma_ {t} = \Sigma_ {t | t - 1} - \Sigma_ {t | t - 1} C _ {t} ^ {\top} \left(C _ {t} \Sigma_ {t | t - 1} C _ {t} ^ {\top} + V _ {t}\right) ^ {- 1} C _ {t} \Sigma_ {t | t - 1} \\ \Sigma_ {t + 1 | t} = A _ {t} \Sigma_ {t} A _ {t} ^ {\top} + W _ {t} \end{array} \right\} \forall t \in [ T - 1 ] \tag {A.16}
$$
initialized by  $\Sigma_{0| - 1} = X_0$ . Using  $\Sigma_{t|t - 1}$ , we then define the Kalman filter gain as
$$
L _ {t} = \Sigma_ {t} C _ {t} ^ {\top} V _ {t} ^ {- 1} \quad \forall t \in [ T - 1 ]
$$
which allows us to compute the minimum mean-squared-error estimator via the forward recursion
$$
\hat {x} _ {t + 1} = A _ {t} \hat {x} _ {t} + B _ {t} u _ {t} + L _ {t + 1} \left(y _ {t + 1} - C _ {t + 1} \left(A _ {t} \hat {x} _ {t} + B _ {t} u _ {t}\right)\right) \quad \forall t \in [ T - 1 ]
$$
initialized by  $\hat{x}_0 = L_0y_0$ . One can also show that the optimal value of the LQG problem amounts to
$$
\sum_ {t = 0} ^ {T - 1} \operatorname {T r} \left(\left(Q _ {t} - P _ {t}\right) \Sigma_ {t}\right) + \sum_ {t = 1} ^ {T} \operatorname {T r} \left(P _ {t} \left(A _ {t - 1} \Sigma_ {t - 1} A _ {t - 1} ^ {\top} + W _ {t - 1}\right)\right) + \operatorname {T r} \left(P _ {0} X _ {0}\right). \tag {A.17}
$$
B. Definitions of Stacked System Matrices
The stacked system matrices appearing in the distributionally robust LQG problem (5) are defined as follows. First, the stacked state and input cost matrices  $Q \in \mathbb{S}^{n(T + 1)}$  and  $R \in \mathbb{S}^{mT}$  are set to
$$
Q = \left[ \begin{array}{c c c c} Q _ {0} & & & \\ & Q _ {1} & & \\ & & \ddots & \\ & & & Q _ {T} \end{array} \right] \quad \text {a n d} \quad R = \left[ \begin{array}{c c c c} R _ {0} & & & \\ & R _ {1} & & \\ & & \ddots & \\ & & & R _ {T - 1} \end{array} \right],
$$
respectively. Similarly, the stacked matrices appearing in the linear dynamics and the measurement equations  $C \in \mathbb{R}^{pT \times n(T + 1)}$ ,  $G \in \mathbb{R}^{n(T + 1) \times n(T + 1)}$  and  $H \in \mathbb{R}^{n(T + 1) \times mT}$  are defined as
$$
C = \left[ \begin{array}{c c c c c} C _ {0} & 0 & & & \\ & C _ {1} & 0 & & \\ & & \ddots & \ddots & \\ & & & C _ {T - 1} & 0 \end{array} \right], \quad G = \left[ \begin{array}{c c c c} A _ {0} ^ {0} & & & \\ A _ {0} ^ {1} & A _ {1} ^ {1} & & \\ \vdots & & \ddots & \\ A _ {0} ^ {T} & A _ {1} ^ {T} & \ldots & A _ {T} ^ {T} \end{array} \right]
$$
$$
H = \left[ \begin{array}{c c c c c} 0 & & & & \\ A _ {1} ^ {1} B _ {0} & 0 & & & \\ A _ {1} ^ {2} B _ {0} & A _ {2} ^ {2} B _ {1} & 0 & & \\ \vdots & & & \ddots & \\ \vdots & & & & 0 \\ A _ {1} ^ {T} B _ {0} & A _ {2} ^ {T} B _ {1} & \ldots & \ldots & A _ {T} ^ {T} B _ {T - 1} \end{array} \right],
$$
respectively, where  $A_{s}^{t} = \prod_{k=s}^{t-1} A_{k}$  for every  $s < t$  and  $A_{s}^{t} = I_{n}$  for  $s = t$ .
Using the stacked system matrices, we can now express the purified observation process  $\eta$  as a linear function of the exogenous uncertainties  $w$  and  $v$  that is not impacted by  $u$ ; see also [5, 46]
Lemma B.1. We have  $\eta = Dw + v$ , where  $D = CG$ .
Proof of Lemma B.1. The purified observation process is defined as  $\eta = y - \hat{y}$ . Recall now that the observations of the original system satisfy  $y = Cx + v$ . Similarly, one readily verifies that the observations of the fictitious noise-free system satisfy  $\hat{y} = C\hat{x}$ . Thus, we have  $\eta = C(x - \hat{x}) + v$ . Next, recall that the state of the original system satisfies  $x = H u + G w$ , and note that the state of the fictitious noise-free system satisfies  $\hat{x} = H u$ . Combining all of these linear equations finally shows that  $u$  cancels out and that  $\eta = C G w + v = D w + v$ .
C. Proofs
C.1. Additional Technical Results
It is well known that every causal controller that is linear in the original observations  $y$  can be reformulated as a causal controller that is linear in the purified observations  $\eta$  and vice versa [5, 46]. Perhaps surprisingly, however, the one-to-one transformation between the respective coefficients of  $y$  and  $\eta$  is not linear. To keep this paper self-contained, we review these insights in the next lemma.
Lemma C.1. If  $u = U\eta + q$  for some  $U \in \mathcal{U}$  and  $q \in \mathbb{R}^{pT}$ , then  $u = U' y + q'$  for  $U' = (I + UCH)^{-1}U$  and  $q' = (I + UCH)^{-1}q$ . Conversely, if  $u = U'y + q'$  for some  $U' \in \mathcal{U}$  and  $q' \in \mathbb{R}^{pT}$ , then  $u = U\eta + q$  for  $U = (I - U'CH)^{-1}U'$  and  $q = (I - U'CH)^{-1}q'$ .
Proof of Lemma C.1. If  $u = U\eta + q$  for some  $U \in \mathcal{U}$  and  $q \in \mathbb{R}^{pT}$ , then we have
$$
u = U \eta + q = U (y - \hat {y}) + q = U y - U C \hat {x} + q = U y - U C H u + q,
$$
where the second equality follows from the definition of  $\eta$ , the third equality holds because  $y = Cx + v$ , and the last equality exploits our earlier insight that  $\hat{y} = C\hat{x}$ . The last expression depends only on  $y$  and  $u$ . Solving for  $u$  yields  $u = U'y + q'$ , where  $U' = (I + UCH)^{-1}U$  and  $q' = (I + UCH)^{-1}q$ . Note that  $(I + UCH)$  is indeed invertible because  $I + UCH$  is a lower triangular matrix with all diagonal entries equal to one, ensuring a determinant of one.
Similarly, if  $u = U' y + q'$  for some  $U' \in \mathcal{U}$  and  $q' \in \mathbb{R}^{pT}$ , then we have
$$
u = U ^ {\prime} y + q ^ {\prime} = U ^ {\prime} (\eta + \hat {y}) + q ^ {\prime} = U ^ {\prime} \eta + U ^ {\prime} C \hat {x} + q ^ {\prime} = U ^ {\prime} \eta + U ^ {\prime} C H u + q ^ {\prime}.
$$
Solving for  $u$  yields  $u = U\eta + q$ , where  $U = (I - U'CH)^{-1}U'$  and  $q = (I - U'CH)^{-1}q'$ . Note again that  $(I - U'CH)$  is indeed invertible because  $(I - U'CH)$  is a lower triangular matrix with all diagonal entries equal to one.
C.2. Proofs of Section 3
Proof of Proposition 3.2. In problem (8), both  $u$  and  $x$  are linear in  $w$  and  $v$ , i.e.,  $u = q + UDw + Uv$  and  $x = H_u + Gw = Hq + HUDw + HUv + Gw$ . By substituting the linear representations of  $u$  and  $x$  into the objective function of problem (8), we obtain the following equivalent reformulation.
$$
\min_{\substack{q\in \mathbb{R}^{pT}\\ U\in \mathcal{U}}}\max_{\mathbb{P}\in \mathcal{G}}\mathbb{E}_{\mathbb{P}}\left[w^{\top}\left(D^{\top}U^{\top}(R + H^{\top}QH)UD + 2D^{\top}U^{\top}H^{\top}QG + G^{\top}QG\right)w\right] + \mathbb{E}_{\mathbb{P}}\left[v^{\top}\left(U^{\top}(R + H^{\top}QH)U\right)v\right] + q^{\top}(R + H^{\top}QH)q
$$
For any fixed  $\mathbb{P} \in \mathcal{G}$ , we can express the expression in the objective function of the above problem in terms of the covariance matrices  $W = \mathbb{E}_{\mathbb{P}}[vw^{\top}]$  and  $V = \mathbb{E}_{\mathbb{P}}[vv^{\top}]$ . Thus, the problem becomes
$$
\min  _ {\substack {q \in \mathbb {R} ^ {p T} \\ U \in \mathcal {U}}} \quad \max  _ {W, V, \mathbb {P}} \quad \operatorname {Tr} \left(\left(D ^ {\top} U ^ {\top} (R + H ^ {\top} Q H) U D + 2 G ^ {\top} Q H U D + G ^ {\top} Q G\right) W\right) + \operatorname {Tr} \left(\left(U ^ {\top} (R + H ^ {\top} Q H) U\right) V\right) + q ^ {\top} (R + H ^ {\top} Q H) q \tag{A.18}
$$
$$
\begin{array}{l} \text {s . t .} \quad \mathbb {P} \in \mathcal {G}, \quad W = \mathbb {E} _ {\mathbb {P}} [ w w ^ {\top} ], \quad V = \mathbb {E} _ {\mathbb {P}} [ v v ^ {\top} ]. \end{array}
$$
Recall now the definition of  $\mathcal{G}$ , and note that the requirements  $\mathbb{G}(X_0,\hat{X}_0)\leq \rho_{x_0}$ ,  $\mathbb{G}(W_t,\hat{W}_t)\leq \rho_{w_t}$  and  $\mathbb{G}(V_t,\hat{V}_t)\leq \rho_{v_t}$  are equivalent to the convex constraints  $\mathbb{G}(X_0,\hat{X}_0)^2\leq \rho_{x_0}^2$ ,  $\mathbb{G}(W_t,\hat{W}_t)^2\leq \rho_{w_t}^2$  and  $\mathbb{G}(V_t,\hat{V}_t)^2\leq \rho_{v_t}^2$ , respectively, for all  $t\in [T - 1]$ . The definition of  $\mathcal{G}$  also implies that
$$
W = \mathbb {E} _ {\mathbb {P}} [ w w ^ {\top} ] = \operatorname {d i a g} (X _ {0}, W _ {0}, \dots , W _ {T - 1}) \quad \text {a n d} \quad V = \mathbb {E} _ {\mathbb {P}} [ v v ^ {\top} ] = \operatorname {d i a g} (V _ {0}, \dots , V _ {T - 1}).
$$
Problem (A.18) thus constitutes a relaxation of problem (9). Indeed, the feasible set of the inner maximization problem in (A.18) is a subset of the feasible set of the inner maximization problem in (9). Moreover, for any  $W$  and  $V$  feasible in the inner maximization problem in (9), the distribution  $\mathbb{P} = \mathbb{P}_{x_0}\otimes (\otimes_{t = 0}^{T - 1}\mathbb{P}_{w_t})\otimes (\otimes_{t = 0}^T\mathbb{P}_{v_t})$  defined through  $\mathbb{P}_{x_0} = \mathcal{N}(0,X_0)$ ,  $\mathbb{P}_{w_t} = \mathcal{N}(0,W_t)$  and  $\mathbb{P}_{v_t} = \mathcal{N}(0,V_t)$ ,  $t\in [T - 1]$ , is feasible in the inner maximization problem in (A.18) with the same objective value. The relaxation is thus exact, and the optimal values of (8), (9) and (A.18) coincide.
Proof of Proposition 3.4. Recall that the space  $\mathcal{U}_y$  of all causal output-feedback controllers coincides with the space  $\mathcal{U}_{\eta}$  of all causal purified output-feedback controllers. We can thus replace the feasible set  $\mathcal{U}_{\eta}$  of the inner minimization problem in (10) with  $\mathcal{U}_y$ . Hence, for any fixed  $\mathbb{P} \in \mathcal{W}_{\mathcal{N}}$ , the inner minimization problem in (10) constitutes a classic LQG problem. By standard LQG theory [8], it is solved by a linear output-feedback controller of the form  $u = U'y + q'$  for some  $U' \in \mathcal{U}$  and  $q' \in \mathbb{R}^{pT}$ ; see also Appendix §A. Lemma C.1 shows, however, that any linear output-feedback controller can be equivalently expressed as a linear purified-output feedback controller of the form  $u = U\eta + q$  for some  $U \in \mathcal{U}$  and  $q \in \mathbb{R}^{pT}$ . In summary, the above reasoning shows that the feasible set of the inner minimization problem in (10) can be reduced to the family of all linear purified-output feedback controllers without sacrificing optimality. Thus, problem (10) is equivalent to
$$
\begin{array}{l l l} \max  _ {\mathbb {P} \in \mathcal {W} _ {\mathcal {N}}} & \min  _ {q, U, x, u} & \mathbb {E} _ {\mathbb {P}} \left[ u ^ {\top} R u + x ^ {\top} Q x \right] \\ & \text {s . t .} & U \in \mathcal {U}, u = q + U \eta , x = H u + G w. \end{array}
$$
Using a similar reasoning as in the proof of Proposition 3.2, we can now substitute the linear representations of  $u$  and  $x$  into the objective function and reformulate the above problem as
$$
\max_{W,V,\mathbb{P}}\quad \min_{\substack{q\in \mathbb{R}^{pT}\\ U\in \mathcal{U}}}\quad \operatorname {Tr}\left(\left(D^{\top}U^{\top}(R + H^{\top}QH)UD + 2G^{\top}QHUD + G^{\top}QG\right)W\right)\\ +\operatorname {Tr}\left(\left(U^{\top}(R + H^{\top}QH)U\right)V\right) + q^{\top}(R + H^{\top}QH)q
$$
$$
\begin{array}{l} \text {s . t .} \quad \mathbb {P} \in \mathcal {W} _ {\mathcal {N}}, W = \mathbb {E} _ {\mathbb {P}} [ w w ^ {\top} ], V = \mathbb {E} _ {\mathbb {P}} [ v v ^ {\top} ]. \end{array}
$$
As  $\mathcal{W}_{\mathcal{N}}$  contains only normal distributions, Proposition 3.3 implies that  $\mathbb{W}(\mathbb{P}_{x_0},\hat{\mathbb{P}}_{x_0}) = \mathbb{G}(X_0,\hat{X}_0)$ ,  $\mathbb{W}(\mathbb{P}_{w_t},\hat{\mathbb{P}}_{w_t}) = \mathbb{G}(W_t,\hat{W}_t)$  and  $\mathbb{W}(\mathbb{P}_{v_t},\hat{\mathbb{P}}_{v_t}) = \mathbb{G}(V_t,\hat{V}_t)$  for all  $t\in [T - 1]$ . We may thus replace the requirement  $\mathbb{W}(\mathbb{P}_{x_0},\hat{\mathbb{P}}_{x_0})\leq \rho_{x_0}$  in the definition of  $\mathcal{W}_{\mathcal{N}}$  by  $\mathbb{G}(X_0,\hat{X}_0)\leq \rho_{x_0}$ , which is equivalent to the convex constraint  $\mathbb{G}(X_0,\hat{X}_0)^2\leq \rho_{x_0}^2$ . The conditions on the marginal distributions of  $w_{t}$  and  $v_{t}$ ,  $t\in [T - 1]$ , admit similar reformulations. The definition of  $\mathcal{W}_{\mathcal{N}}$  also implies that
Thus, the feasible set of the outer maximization problem in (11) constitutes a relaxation of that in (10). One readily verifies that the relaxation is exact by using similar arguments as in the proof of Proposition 3.2. Thus, the claim follows.
Proof of Theorem 3.5. By Proposition 3.2,  $\overline{p}^{\star}$  coincides with the minimum of (9). Similarly, by Proposition  $3.4\underline{d}^{\star}$  coincides with the maximum of (11). Note that problems (9) and (11) only differ by the order of minimization and maximization. Note also that  $\mathcal{U}$  is convex and closed,  $\mathcal{G}_W$  and  $\mathcal{G}_V$  are convex and compact by virtue of [38, Lemma A.6], and the (identical) trace terms in (9) and (11) are bilinear in  $(W,V)$  and  $(U,q)$ . The claim thus follows from Sion's minimax theorem [45].
C.3. Proofs of Section 4
Note that Proposition 4.1 is consistent with Corollary 3 because the optimal LQG controller corresponding to  $\mathbb{P}^{\star}$  is linear in the past observations.
Proof of Proposition 4.1. By [38, Lemma A.3], the inner problem in (9) admits a maximizer  $(W^{\star},V^{\star})$  with  $X_0^\star\succeq\lambda_{\min}(\hat{X}_0)$  as well as  $W_{t}^{\star}\succeq\lambda_{\min}(\hat{W}_{t})$  and  $V_{t}^{\star}\succeq\lambda_{\min}(\hat{V}_{t})$  for all  $t\in [T - 1]$ . Thus, the optimal value of problem (9) and its strong dual (11) does not change if we restrict  $\mathcal{G}_W$  and  $\mathcal{G}_V$  to  $\mathcal{G}_W^+$  and  $\mathcal{G}_V^+$ , respectively. We may thus conclude that problem (11) has a maximizer  $(W^{\star},V^{\star})$  with  $V_{t}^{\star}\succeq\lambda_{\min}(\hat{V}_{t})\succ 0$  for all  $t\in [T - 1]$ . This in turn implies that problem (6) is solved by a normal distribution  $\mathbb{P}^\star$  under which the covariance matrix of the observation noise  $v_{t}$  satisfies  $V_{t}^{\star}\succ 0$  for every  $t\in [T - 1]$ . As (5) and (6) are strong duals, the optimal solution  $u^{\star}$  of problem (5) forms a Nash equilibrium with  $\mathbb{P}^{\star}$ , i.e.,  $u^{\star}$  is a best response to  $\mathbb{P}^{\star}$  and thus solves the classic LQG problem corresponding to  $\mathbb{P}^{\star}$ . As  $R_{t}\succ 0$  for every  $t\in [T - 1]$ , this best response  $u^{\star}$  is unique, and as  $V_{T}^{\star}\succ 0$  for every  $t\in [T - 1]$ ,  $u^{\star}$  is in fact the Kalman filter-based optimal output-feedback strategy corresponding to  $\mathbb{P}^{\star}$  (which can be obtained using the techniques highlighted in Appendix §A).
Before proving Proposition 4.2, recall that  $f(W,V)$  is called  $\beta$ -smooth for some  $\beta >0$  if
$$
\left| \nabla f (W, V) - \nabla f \left(W ^ {\prime}, V ^ {\prime}\right) \right| \leq \beta \left(\left\| W - W ^ {\prime} \right\| _ {F} ^ {2} + \left\| V - V ^ {\prime} \right\| _ {F} ^ {2}\right) ^ {\frac {1}{2}} \quad \forall W, W ^ {\prime} \in \mathcal {G} _ {W} ^ {+}, V, V ^ {\prime} \in \mathcal {G} _ {V} ^ {+},
$$
where  $\| \cdot \| _F$  denotes the Frobenius norm.
Proof of Proposition 4.2. The function  $f(W, V)$  is concave because the objective function of the inner minimization problem in (11) is linear (and hence concave) in  $W$  and  $V$  and because concavity is preserved under minimization. To prove that  $f(W, V)$  is  $\beta$ -smooth, we first recall from Proposition 3.3 that it coincides with the optimal value of the inner minimization problem in (10). As  $\mathcal{U}_{\eta} = \mathcal{U}_y$ ,  $f(W, V)$  can thus be viewed as the optimal value of the classic LQG problem corresponding to the normal distribution  $\mathbb{P}$  determined by the covariance matrices  $W$  and  $V$ . Hence,  $f(W, V)$  coincides with (A.17), where  $\Sigma_t$ , for  $t \in [T - 1]$ , is a function of  $(W, V)$  defined recursively through the Kalman filter equations (A.16). Note that all inverse matrices in (A.16) are well-defined because any  $V \in \mathcal{G}_V^+$  is strictly positive definite. Therefore,  $\Sigma_t$  constitutes a proper rational function (that is, a ratio of two polynomials with the polynomial in the denominator being strictly positive) for every  $t \in [T - 1]$ . Thus,  $f(W, V)$  is infinitely often continuously differentiable on a neighborhood of  $\mathcal{G}_W^+ \times \mathcal{G}_V^+$ .
As  $f(W, V)$  is concave and (at least) twice continuously differentiable, it is  $\beta$ -smooth on  $\mathcal{G}_W^+ \times \mathcal{G}_V^+$  if and only if the largest eigenvalue of the Hessian matrix of  $-f(W, V)$  is bounded above by  $\beta$  throughout  $\mathcal{G}_W^+ \times \mathcal{G}_V^+$ . Also, the largest eigenvalue of the positive semidefinite Hessian matrix  $\nabla^2(-f(W, V))$  coincides with the spectral norm of  $\nabla^2 f(W, V)$ . We may thus set
$$
\beta = \sup  _ {W \in \mathcal {G} _ {W} ^ {+}, V \in \mathcal {G} _ {V} ^ {+}} \| \nabla^ {2} f (W, V) \| _ {2}, \tag {A.19}
$$
where  $\| \cdot \|_2$  denotes the spectral norm. The supremum in the above maximization problem is finite and attained thanks to Weierstrass' theorem, which applies because  $f(W, V)$  is twice continuously differentiable and the spectral norm is continuous, while the sets  $\mathcal{G}_W^+$  and  $\mathcal{G}_V^+$  are compact by virtue of [38, Lemma A.6]. This observation completes the proof.
D. SDP Reformulation of the Lower Problem (11)
Instead of solving the dual problem (11) with the customized Frank-Wolfe algorithm of Section 4, it can be reformulated as an SDP amenable to off-the-shelf solvers. This reformulation is obtained by dualizing the inner minimization problem and by exploiting the following preliminary lemma.
Lemma D.1. For any  $\hat{Z} \in \mathbb{S}_{+}^{d}$  and  $\rho_{z} \geq 0$ , the set  $\mathcal{G}_Z = \{Z \in \mathbb{S}_{+}^{d} : \mathbb{G}(Z, \hat{Z}) \leq \rho_{z}\}$  coincides with
$$
\left\{Z \in \mathbb {S} _ {+} ^ {d}: \exists E _ {z} \in \mathbb {S} _ {+} ^ {d} \text {w i t h} \operatorname {T r} (Z + \hat {Z} - 2 E _ {z}) \leq \rho_ {z} ^ {2}, \left[ \begin{array}{c c} \hat {Z} ^ {\frac {1}{2}} Z \hat {Z} ^ {\frac {1}{2}} & E _ {z} \\ E _ {z} & I \end{array} \right] \succeq 0 \right\}.
$$
Proof of Lemma D.1. By Definition 2, we have
$$
\mathcal {G} _ {Z} = \{Z \in \mathbb {S} _ {+} ^ {d}: \mathrm {T r} (Z + \hat {Z} - 2 (\hat {Z} ^ {\frac {1}{2}} Z \hat {Z} ^ {\frac {1}{2}}) ^ {\frac {1}{2}}) \leq \rho_ {z} ^ {2} \}.
$$
Next, introduce an auxiliary variable  $E_z \in \mathbb{S}_+^d$  subject to the matrix inequality  $E_z^2 \preceq (\hat{Z}^{\frac{1}{2}} Z \hat{Z}^{\frac{1}{2}})$ . By [4, Theorem 1], this inequality can be recast as  $E_z \preceq (\hat{Z}^{\frac{1}{2}} Z \hat{Z}^{\frac{1}{2}})^{\frac{1}{2}}$ . Hence, we can reformulate the nonlinear matrix inequality in the above representation of  $\mathcal{G}_Z$  as  $\mathrm{Tr}(Z + \hat{Z} - 2E_z) \leq \rho_z^2$ . A standard Schur complement argument reveals that the inequality  $E_z^2 \preceq (\hat{Z}^{\frac{1}{2}} Z \hat{Z}^{\frac{1}{2}})$  is also equivalent to
$$
\left[ \begin{array}{c c} \hat {Z} ^ {\frac {1}{2}} Z \hat {Z} ^ {\frac {1}{2}} & E _ {z} \\ E _ {z} & I \end{array} \right] \succeq 0.
$$
The claim then follows by combining all of these insights.
We are now ready to derive the desired SDP reformulation of problem (11).
Proposition D.2. If  $\hat{V} \succ 0$ , then problem (11) is equivalent to the SDP
max  $\operatorname {Tr}(G^{\top}QGW) - \operatorname {Tr}(F(R + H^{\top}QH)^{-1})$
s.t.  $W\in \mathbb{S}_{+}^{n(T + 1)},V\in \mathbb{S}_{+}^{pT},M\in \mathcal{M},F\in \mathbb{S}_{+}^{Tm}$
$$
E _ {x _ {0}} \in \mathbb {S} _ {+} ^ {n}, E _ {w _ {t}} \in \mathbb {S} _ {+} ^ {n}, E _ {v _ {t}} \in \mathbb {S} _ {+} ^ {p} \forall t \in [ T - 1 ]
$$
$$
\operatorname {T r} \left(W _ {0} + \hat {X} _ {0} - 2 E _ {x _ {0}}\right) \leq \rho_ {x _ {0}} ^ {2},
$$
$$
\operatorname {T r} \left(W _ {t + 1} + \hat {W} _ {t} - 2 E _ {w _ {t}}\right) \leq \rho_ {w _ {t}} ^ {2}, \operatorname {T r} \left(V _ {t} + \hat {V} _ {t} - 2 E _ {v _ {t}}\right) \leq \rho_ {v _ {t}} ^ {2} \forall t \in [ T - 1 ]
$$
$$
\left[ \begin{array}{c c} \hat {X} _ {0} ^ {\frac {1}{2}} X _ {0} \hat {X} _ {0} ^ {\frac {1}{2}} & E _ {x _ {0}} \\ E _ {x _ {0}} & I _ {n} \end{array} \right] \succeq 0, \tag {A.20}
$$
$$
\left[ \begin{array}{c c} \hat {W} _ {t} ^ {\frac {1}{2}} W _ {t + 1} \hat {W} _ {t} ^ {\frac {1}{2}} & E _ {w _ {t}} \\ E _ {w _ {t}} & I _ {n} \end{array} \right] \succeq 0, \left[ \begin{array}{c c} \hat {V} _ {t} ^ {\frac {1}{2}} V _ {t} \hat {V} _ {t} ^ {\frac {1}{2}} & E _ {v _ {t}} \\ E _ {v _ {t}} & I _ {p} \end{array} \right] \succeq 0 \quad \forall t \in [ T - 1 ]
$$
$$
\left[ \begin{array}{c c} F & H ^ {\top} Q G W D ^ {\top} + M / 2) ^ {\top} \\ (H ^ {\top} Q G W D ^ {\top} + M / 2) ^ {\top} & D W D ^ {\top} + V \end{array} \right] \succeq 0
$$
$$
W _ {0} \succeq \lambda_ {\min } (\hat {X} _ {0}) I, \quad W _ {t + 1} \succeq \lambda_ {\min } (\hat {W} _ {t}) I, \quad V _ {t} \succeq \lambda_ {\min } (\hat {V} _ {t}) I \quad \forall t \in [ T - 1 ].
$$
Here,  $\mathcal{M}$  denotes the set of all strictly upper block triangular matrices of the form
$$
\left[ \begin{array}{c c c c c} 0 & M _ {1, 2} & M _ {1, 3} & \ldots & M _ {1, T} \\ & 0 & M _ {2, 3} & & M _ {2, T} \\ & & \ddots & & \vdots \\ & & & 0 & M _ {T - 1, T} \\ & & & & 0 \end{array} \right] \in \mathbb {R} ^ {T m \times T p},
$$
where  $M_{t,s}\in \mathbb{R}^{m\times p}$  for every  $t,s\in \mathbb{Z}$  with  $1\leq t < s\leq T$