SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By construction, the random vectors  $x_0, \{w_t\}_{t=0}^{T-1}$  and  $\{v_t\}_{t=0}^{T-1}$  are thus mutually independent under any  $\mathbb{P} \in \mathcal{G}$ . In addition and as a direct consequence of Proposition 3.1,  $\mathcal{G}$  constitutes an outer approximation for the Wasserstein ambiguity set  $\mathcal{W}$ , as summarized in the next result.
Corollary 1 (Gelbrich hull). We have  $\mathcal{W} \subseteq \mathcal{G}$
Because  $\mathcal{G}$  covers  $\mathcal{W}$ , we henceforth refer to it as the Gelbrich hull of the Wasserstein ambiguity set  $\mathcal{W}$ . To finalize our construction of the upper bound on  $p^{\star}$ , we focus on linear policies<sup>2</sup> of the form
2Technically, the policies are affine because they include a constant term, but we retain the more common terminology that focuses on the dependencies.
$u = q + U\eta = q + U(Dw + v)$ , where  $q = (q_0, \dots, q_{T-1})$ , and  $U$  is a block lower triangular matrix
$$
U = \left[ \begin{array}{c c c c} U _ {0, 0} & & & \\ U _ {1, 0} & U _ {1, 1} & & \\ \vdots & & \ddots & \\ U _ {T - 1, 0} & \dots & \dots & U _ {T - 1, T - 1} \end{array} \right]. \tag {7}
$$
The block lower triangularity of  $U$  ensures that the corresponding controller is causal, which in turn ensures that  $u \in \mathcal{U}_{\eta}$ . In the following, we denote by  $\mathcal{U}$  the set of all block lower triangular matrices of the form (7). An upper bound on problem (5) can now be obtained by restricting the controller's feasible set to causal controllers that are linear in the purified observations  $\eta$  and by relaxing nature's feasible set to the Gelbrich hull  $\mathcal{G}$  of  $\mathcal{W}$ . The resulting bounding problem is given by
$$
\bar {p} ^ {\star} = \left\{ \begin{array}{c l} \min  _ {U, q, x, u} & \max  _ {\mathbb {P} \in \mathcal {G}} \mathbb {E} _ {\mathbb {P}} \left[ u ^ {\top} R u + x ^ {\top} Q x \right] \\ \text {s . t .} & U \in \mathcal {U}, u = q + U (D w + v), x = H u + G w. \end{array} \right. \tag {8}
$$
As we obtained (8) by restricting the feasible set of the outer minimization problem and relaxing the feasible set of the inner maximization problem in (5), it is clear that  $\overline{p}^{\star} \geq p^{\star}$ . Recall also that problem (5) constitutes an infinite-dimensional zero-sum game, where the agents optimize over measurable policies and non-parametric distributions, respectively. In contrast, the next proposition shows that problem (8) is equivalent to a finite-dimensional zero-sum game.
Proposition 3.2. Problem (8) is equivalent to the optimization problem
$$
\bar {p} ^ {\star} = \left\{ \begin{array}{l} \min  _ {\substack {q \in \mathbb {R} ^ {p T} \\ U \in \mathcal {U}}} \max  _ {\substack {W \in \mathcal {G} _ {W} \\ V \in \mathcal {G} _ {V}}} \operatorname {Tr} \left(\left(D ^ {\top} U ^ {\top} (R + H ^ {\top} Q H) U D + 2 G ^ {\top} Q H U D + G ^ {\top} Q G\right) W\right) \\ + \operatorname {Tr} \left(\left(U ^ {\top} (R + H ^ {\top} Q H) U\right) V\right) + q ^ {\top} (R + H ^ {\top} Q H) q, \end{array} \right. \tag{9}
$$
$$
\mathcal {G} _ {W} = \left\{W \in \mathbb {S} _ {+} ^ {n (T + 1)}: \begin{array}{l} W = \operatorname {d i a g} (X _ {0}, W _ {0}, \ldots , W _ {T - 1}), X _ {0} \in \mathbb {S} _ {+} ^ {n}, W _ {t} \in \mathbb {S} _ {+} ^ {n} \forall t \in [ T - 1 ] \\ \mathbb {G} (X _ {0}, \hat {X} _ {0}) ^ {2} \leq \rho_ {x _ {0}} ^ {2}, \mathbb {G} (W _ {t}, \hat {W} _ {t}) ^ {2} \leq \rho_ {w _ {t}} ^ {2} \forall t \in [ T - 1 ] \end{array} \right\}
$$
$$
\mathcal {G} _ {V} = \left\{V \in \mathbb {S} _ {+} ^ {p T}: V = \mathrm {d i a g} (V _ {0}, \ldots , V _ {T - 1}), V _ {t} \in \mathbb {S} _ {+} ^ {p}, \mathbb {G} (V _ {t}, \hat {V} _ {t}) ^ {2} \leq \rho_ {v _ {t}} ^ {2} \forall t \in [ T - 1 ] \right\}.
$$
We emphasize that Proposition 3.2 remains valid even if the nominal distribution  $\hat{\mathbb{P}}$  fails to be normal. Note also that, while nature's feasible set in (8) is non-convex due to the independence conditions, the sets  $\mathcal{G}_W$  and  $\mathcal{G}_V$  are convex and even semidefinite representable thanks to the properties of the squared Gelbrich distance.<sup>3</sup> By dualizing the inner maximization problem, one can therefore reformulate the minimax problem (9) as a convex semidefinite program (SDP). Even though this SDP is computationally tractable in theory, it involves  $\mathcal{O}(T(mp + n^2 + p^2))$  decision variables. For practically interesting problem dimensions, it thus quickly exceeds the capabilities of existing solvers.
Lower Bound for  $d^{\star}$ . To derive a tractable lower bound on  $d^{\star}$ , we restrict nature's feasible set to the family  $\mathcal{W}_{\mathcal{N}}$  of all normal distributions in the Wasserstein ambiguity set  $\mathcal{W}$ . The resulting bounding problem is thus given by
$$
\underline {{d}} ^ {\star} = \left\{ \begin{array}{l l} \max  _ {\mathbb {P} \in \mathcal {W} _ {\mathcal {N}}} & \min  _ {x, u} \\ & \text {s . t .} \end{array} \quad \begin{array}{l l} \mathbb {E} _ {\mathbb {P}} [ u ^ {\top} R u + x ^ {\top} Q x ] \\ u \in \mathcal {U} _ {\eta}, \quad x = H u + G w. \end{array} \right. \tag {10}
$$
As we obtained (10) by restricting the feasible set of the outer maximization problem in (6), it is clear that  $\underline{d}^{\star} \leq d^{\star}$ . Next, we show that (10) can be recast as a finite-dimensional zero-sum game. This result critically relies on the following known fact regarding the 2-Wasserstein distance between two normal distributions, which coincides with the Gelbrich distance between their covariance matrices.
Proposition 3.3 (Tightness for normal distributions [26, Proposition 7]). For any two normal distributions  $\mathbb{P}_1 = \mathcal{N}(0,\Sigma_1)$  and  $\mathbb{P}_2 = \mathcal{N}(0,\Sigma_2)$  with zero means we have  $\mathbb{W}(\mathbb{P}_1,\mathbb{P}_2) = \mathbb{G}(\Sigma_1,\Sigma_2)$ .
With this, we can provide a finite-dimensional reformulation, as summarized in the next result.
Note that the ambiguity sets  $\mathcal{G}_W$  and  $\mathcal{G}_V$  appearing in (9) involve the squared Gelbrich distance,  $\mathrm{G}(\Sigma_1,\Sigma_2)^2$ . The reason is that  $\mathrm{G}(\Sigma_1,\Sigma_2)^2$  is known to be jointly convex in  $\Sigma_1,\Sigma_2$  and semidefinite representable [38, Proposition 2.3], unlike the Gelbrich distance  $\mathrm{G}(\Sigma_1,\Sigma_2)$  itself, which is generally non-convex.
Proposition 3.4. Problem (10) is equivalent to the optimization problem
$$
\underline {{d}} ^ {\star} = \left\{ \begin{array}{l l} \max  _ {W \in \mathcal {G} _ {W}} & \min  _ {q \in \mathbb {R} ^ {p T}} \operatorname {T r} \left(\left(D ^ {\top} U ^ {\top} (R + H ^ {\top} Q H) U D + 2 G ^ {\top} Q H U D + G ^ {\top} Q G\right) W\right) \\ V \in \mathcal {G} _ {V} & U \in \mathcal {U} \end{array} + \operatorname {T r} \left(\left(U ^ {\top} (R + H ^ {\top} Q H) U\right) V\right) + q ^ {\top} (R + H ^ {\top} Q H) q, \right. \tag {11}
$$
where  $\mathcal{G}_W$  and  $\mathcal{G}_V$  are defined exactly as in Proposition 3.2.
Proposition 3.4 relies on Proposition 3.3 and thus fails to hold unless  $\hat{\mathbb{P}}$  is normal. Also, one can again reformulate (11) as a tractable SDP by dualizing the inner minimization problem.
Conclusions. Propositions 3.2 and 3.4 reveal that problems (9) and (11) are dual to each other, that is, they can be transformed into one another by interchanging minimization and maximization. The following main theorem shows that strong duality holds irrespective of the problem data.
Theorem 3.5 (Strong duality of (9) and (11)). We have  $\overline{p}^{\star} = \underline{d}^{\star}$ .
Theorem 3.5 follows immediately from Sion's classic minimax theorem [45], which applies because  $\mathcal{G}_W$  and  $\mathcal{G}_V$  are convex as well as compact thanks to [38, Lemma A.6].
By weak duality and the construction of the bounding problems (9) and (11), we trivially have  $\underline{d}^{\star} \leq d^{\star} \leq p^{\star} \leq \overline{p}^{\star}$ . Theorem 3.5 reveals that all of these inequalities are in fact equalities, each of which gives rise to a non-trivial insight. The first key insight is that (5) and (6) are strong duals.
Corollary 2 (Strong duality of (5) and (6)). We have  $p^{\star} = d^{\star}$ .
We stress that, unlike Theorem 3.5, Corollary 2 establishes strong duality between two infinite-dimensional zero-sum games. The second key implication of Theorem 3.5 is that the distributionally robust LQG problem (5) is solved by a linear output-feedback controller.
Corollary 3 (The controller's Nash strategy is linear in the observations). There exist  $U^{\star} \in \mathcal{U}$  and  $q^{\star} \in \mathbb{R}^{m}$  such that the distributionally robust LQG problem (5) is solved by  $u^{\star} = q^{\star} + U^{\star}y$ .
The identity  $p^{\star} = \overline{p}^{\star}$  readily implies that (5) is solved by a causal controller that is linear in the purified observations. However, any causal controller that is linear in the purified observations  $\eta$  can be reformulated exactly as a causal controller that is linear in the original observations  $y$  and vice-versa [6, Proposition 3]. Thus, Corollary 3 follows. The third key implication of Theorem 3.5 is that the dual distributionally robust LQG problem is solved by a normal distribution.
Corollary 4 (Nature's Nash strategy is a normal distribution). The dual distributionally robust LQG problem (6) is solved by a distribution  $\mathbb{P}^{\star}\in \mathcal{W}_{\mathcal{N}}$
Corollary 4 is a direct consequence of the identity  $\underline{d}^{\star} = d^{\star}$ . Note that the optimal normal distribution  $\mathbb{P}^{\star}$  is uniquely determined by the covariance matrices  $W^{\star}$  and  $V^{\star}$  of the exogenous uncertain parameters, which can be computed by solving problem (11). That the worst-case distribution is actually Gaussian is not a-priori expected and is surprising given that the Wasserstein ball contains many non-Gaussian distributions.
4. Efficient Numerical Solution of Distributionally Robust LQG Problems
Having proven these structural results, we next turn attention to the problem of finding the optimal strategies. Our next result shows that, under a mild regularity condition, the optimal controller  $u^{\star}$  of the distributionally robust LQG problem (5) can be computed efficiently from  $\mathbb{P}^{\star}$ .
Proposition 4.1 (Optimality of Kalman filter-based feedback controllers). If  $\hat{V}_t\succ 0$  for all  $t\in [T - 1]$  then problem (6) is solved by a Gaussian distribution  $\mathbb{P}^{\star}$  under which  $v_{t}$  has a covariance matrix  $V_{t}^{\star}\succ 0$  for every  $t\in [T - 1]$ , and (5) is solved by the optimal LQG controller corresponding to  $\mathbb{P}^{\star}$ . Additionally, the optimal value of problem (9) and its strong dual (11) does not change if we restrict  $\mathcal{G}_W$  and  $\mathcal{G}_V$  to  $\mathcal{G}_W^+$  and  $\mathcal{G}_V^+$ , respectively, where
$$
\mathcal {G} _ {W} ^ {+} = \left\{W \in \mathcal {G} _ {W}: X _ {0} \succeq \lambda_ {\min } (\hat {X} _ {0}) I, W _ {t} \succeq \lambda_ {\min } (\hat {W} _ {t}) I \forall t \in [ T - 1 ] \right\},
$$
$$
\mathcal {G} _ {V} ^ {+} = \left\{V \in \mathcal {G} _ {V}: V _ {t} \succeq \lambda_ {\min } (\hat {V} _ {t}) I \forall t \in [ T - 1 ] \right\}.
$$
This implies that the optimal controller can be computed by solving a classic LQG problem corresponding to nature's optimal strategy  $\mathbb{P}^{\star}$ , which can be done very efficiently through Kalman filtering and dynamic programming (see Appendix §A for details). It thus suffices to design an efficient algorithm for computing  $\mathbb{P}^{\star}$ , which is uniquely determined by the covariance matrices  $(W^{\star}, V^{\star})$  that solve problem (11). To this end, we first reformulate (11) as
$$
\max  _ {W \in \mathcal {G} _ {W} ^ {+}, V \in \mathcal {G} _ {V} ^ {+}} f (W, V), \tag {12}
$$
where we restrict  $\mathcal{G}_W$  and  $\mathcal{G}_V$  to  $\mathcal{G}_W^+$  and  $\mathcal{G}_V^+$ , respectively, due to Proposition 4.1, and where  $f(W,V)$  denotes the optimal value function of the inner minimization problem in (11). As (11) is a reformulation of (10) and as the family of all causal purified output-feedback controllers matches the family of causal output-feedback controllers,  $f(W,V)$  can also be viewed as the optimal value of the classic LQG problem corresponding to the normal distribution  $\mathbb{P}$  determined by the covariance matrices  $W$  and  $V$ . These insights lead to the following structural result.
Proposition 4.2.  $f(W,V)$  is concave and  $\beta$ -smooth in  $(W,V) \in \mathcal{G}_W^+ \times \mathcal{G}_V^+$  for some  $\beta > 0$ .
By Proposition 4.2, it is possible to address problem (12) with a Frank-Wolfe algorithm [13, 18, 19, 20, 23, 35]. Each iteration of this algorithm solves a direction-finding subproblem, that is, a variant of problem (12) that maximizes the first-order Taylor expansion of  $f(W,V)$  around the current iterates.
$$
\max  _ {L _ {W} \in \mathcal {G} _ {W} ^ {+}, L _ {V} \in \mathcal {G} _ {V} ^ {+}} \left\langle \nabla_ {W} f (W, V), L _ {W} - W \right\rangle + \left\langle \nabla_ {V} f (W, V), L _ {V} - V \right\rangle \tag {13}
$$
The next iterates are then obtained by moving towards a maximizer  $(L_W^{\star}, L_V^{\star})$  of (13), i.e., we update
$$
(W, V) \leftarrow (W, V) + \alpha \cdot \left(L _ {W} ^ {\star} - W, L _ {v} ^ {\star} - V\right),
$$
where  $\alpha$  is an appropriate step size. The proposed Frank-Wolfe algorithm enjoys a very low per-iteration complexity because problem (13) is separable. To see this, we reformulate (13) as
$$
\max  _ {L _ {W}, L _ {V}} \left\langle \nabla_ {X _ {0}} f (W, V), L _ {X _ {0}} - X _ {0} \right\rangle + \sum_ {t = 0} ^ {T - 1} \left\langle \nabla_ {W _ {t}} f (W, V), L _ {W _ {t}} - W _ {t} \right\rangle + \left\langle \nabla_ {V _ {t}} f (W, V), L _ {V _ {t}} - V _ {t} \right\rangle
$$
$$
\text {s . t .} \quad \mathbb {G} \left(L _ {X _ {0}}, \hat {X} _ {0}\right) ^ {2} \leq \rho_ {x _ {0}} ^ {2}, \quad \mathbb {G} \left(L _ {W _ {t}}, \hat {W} _ {t}\right) ^ {2} \leq \rho_ {w _ {t}} ^ {2}, \quad \mathbb {G} \left(L _ {V _ {t}}, \hat {V} _ {t}\right) ^ {2} \leq \rho_ {v _ {t}} ^ {2} \quad \forall t \in [ T - 1 ]
$$
$$
L _ {X _ {0}} \succeq \lambda_ {\min } (\hat {X} _ {0}) I, \quad L _ {W _ {t}} \succeq \lambda_ {\min } (\hat {W} _ {t}) I, \quad L _ {V _ {t}} \succeq \lambda_ {\min } (\hat {V} _ {t}) I \forall t \in [ T - 1 ].
$$
Hence, (13) decomposes into  $2T + 1$  separate subproblems that can be solved in parallel. That is, for any matrix  $Z \in \{X_0, W_0, \ldots, W_{T-1}, V_0, \ldots, V_{T-1}\}$  we solve a separate subproblem of the form
$$
\max  _ {L _ {Z} \succeq \lambda_ {\min } (\hat {Z})} \left\{\langle \nabla_ {Z} f (W, V), L _ {Z} - Z \rangle : \mathbb {G} \left(L _ {Z}, \hat {Z}\right) ^ {2} \leq \rho_ {z} ^ {2} \right\}. \tag {14}
$$
These subproblems can be reformulated as tractable SDPs and are thus amenable to efficient off-the-shelf solvers. By [38, Theorem 6.2], however, one can exploit the structure of the Gelbrich distance in order to reduce (14) to a univariate algebraic equation that can be solved to any desired accuracy  $\delta >0$  by a highly efficient bisection algorithm. We say that  $L_{Z}^{\delta}$  is a  $\delta$ -approximate solution of problem (14) for some  $\delta \in (0,1)$  if  $L_{Z}^{\delta}$  is feasible in (14) and if
$$
\langle \nabla_ {Z} f (W, V), L _ {Z} ^ {\delta} - Z \rangle \geq \delta \langle \nabla_ {Z} f (W, V), L _ {Z} ^ {*} - Z \rangle ,
$$
where  $L_Z^\star$  is an exact maximizer of (14). Note that, by the concavity of  $f(W,V)$ , the inner product on the right-hand side is nonnegative and vanishes if and only if  $Z$  maximizes  $f(W,V)$  over the feasible set of (14). For further details we refer to Appendix §E in the supplementary material.
Remark 1 (Automatic differentiation). Recall that  $f(W, V)$  coincides with the optimal value of the LQG problem corresponding to the normal distribution  $\mathbb{P}$  determined by the covariance matrices  $W$  and  $V$ . By using the underlying dynamic programming equations,  $f(W, V)$  can thus be expressed in closed form as a serial composition of  $\mathcal{O}(T)$  rational functions (see Appendix §A for details). Hence,  $\nabla_Z f(W, V)$  can be calculated symbolically for any  $Z \in \{X_0, W_0, \ldots, W_{T-1}, V_0, \ldots, V_{T-1}\}$  by repeatedly applying the chain and product rules. However, the resulting formulas are lengthy and cumbersome. We thus compute the gradients numerically using backpropagation. The cost of evaluating  $\nabla_Z f(W, V)$  is then of the same order of magnitude as the cost of evaluating  $f(W, V)$ .
A detailed description of the proposed Frank-Wolfe method is given in Algorithm 1 below.
By [31, Theorem 1 and Lemma 7], which applies thanks to the structural properties of  $f(W,V)$  established in Proposition 4.2, Algorithm 1 attains a suboptimality gap of  $\epsilon$  within  $\mathcal{O}(1 / \epsilon)$  iterations.
5. Numerical Experiments
All experiments are run on an Intel i7-8700 CPU (3.2 GHz) machine with 16GB RAM. All linear SDP problems are modeled in Python 3.8.6 using CVXPY [1, 14] and solved with MOSEK [37]. The gradients of  $f(W, V)$  are computed via Pymanopt [48] with PyTorch's automated differentiation module [39, 40].
Consider a class of distributionally robust LQG problems with  $n = m = p = 10$ . We set  $A_{t} = 0.1 \times A$  to have ones on the main diagonal and the superdiagonal and zeroes everywhere else ( $A_{i,j} = 1$  if  $i = j$  or  $i = j - 1$  and  $A_{i,j} = 0$  otherwise), and the other matrices to  $B_{t} = C_{t} = Q_{t} = R_{t} = I_{d}$ . The Wasserstein radii are set to  $\rho_{x_0} = \rho_{w_t} = \rho_{v_t} = 10^{-1}$ . The nominal covariance matrices of the exogenous uncertainties are constructed randomly and with eigenvalues in the interval [1, 2] (so as to ensure they are positive definite). The code is publicly available in the Github repository https://github.com/RAO-EPFL/DR-Control.
The optimal value of the distributionally robust LQG problem (5) can be computed by directly solving the SDP reformulation of (11) with MOSEK or by solving the nonlinear SDP (12) with our Frank-Wolfe method detailed in Algorithm 1. We next compare these two approaches in 10 independent simulation runs, where we set a stopping criterion corresponding to an optimality gap below  $10^{-3}$  and we run the Frank-Wolfe method with  $\delta = 0.95$ . Figure 1a illustrates the execution time for both approaches as a function of the planning horizon  $T$ ; runs where MOSEK exceeds 100s are not reported. Figure 1b visualizes the empirical convergence behavior of the Frank-Wolfe algorithm. The results highlight that the Frank-Wolfe algorithm achieves running times that are uniformly lower than MOSEK across all problem horizons and is able to find highly accurate solutions already after a small number of iterations (50 iterations for problem instances of time horizon  $T = 10$ ).
6. Concluding Remarks and Limitations
In view of the popularity of LQG models, the results in this work carry important theoretical and practical implications. Despite considering a generalization of the classic LQG setting where the noise affecting the system dynamics and the observations follows unknown (and potentially non-Gaussian) distributions, our findings suggest that certain classic structural results continue to hold and that highly efficient methods can be adapted to tackle this more realistic (and more challenging) problem. Specifically, that control policies depending linearly on observations continue to be optimal and that the worst-case distribution turns out to be Gaussian is surprising from a theoretical angle and also has direct practical implications, because it allows leveraging the highly efficient Kalman filter in conjunction with dynamic programming and a Frank-Wolfe method to design an efficient computational procedure for solving the problem.
The results also raise several important questions that warrant future exploration. First, it would be highly relevant to consider extensions where the system matrices are also affected by uncertainty, as this captures many applications of practical interest in, e.g., reinforcement learning or revenue management. Second, it would be worth exploring an infinite horizon setting or relaxing the assumption that the nominal distribution is Gaussian, as both assumptions may be limiting the practical appeal of the framework. Third, one could also attempt to prove structural optimality results or design novel algorithms for generating high-quality suboptimal solutions for the more general setting involving constraints on states and/or control inputs. Lastly, one could improve the present algorithmic proposal by exploiting topological properties of the objective so as to guarantee linear convergence rates in the Frank-Wolfe procedure.
Acknowledgements. This research was supported by the Swiss National Science Foundation under the NCCR Automation, grant agreement 51NF40_180545. Dan A. Iancu would like to acknowledge INSEAD for financial support during the duration of the project.
[1] Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42-60, 2018.
[2] Rohit Arora and Rui Gao. Data-driven multistage distributionally robust optimization with nested distance: Time consistency and tractable dynamic reformulations. Available at Optimization Online, 2022.
[3] François Auger, Mickael Hilairet, Josep M. Guerrero, Eric Monmasson, Teresa Orlowska-Kowalska, and Seiichiro Katsura. Industrial applications of the Kalman filter: A review. IEEE Transactions on Industrial Electronics, 60(12):5458-5471, 2013.
[4] Richard Bellman. Some inequalities for the square root of a positive definite matrix. Linear Algebra and its Applications, 1(3):321-324, 1968.
[5] Aharon Ben-Tal, Stephen Boyd, and Arkadi Nemirovski. Control of uncertainty-affected discrete time linear systems via convex programming. Available at Optimization Online, 2005.
[6] Aharon Ben-Tal, Stephen Boyd, and Arkadi Nemirovski. Extending scope of robust optimization: Comprehensive robust counterparts of uncertain problems. Mathematical Programming, 107(1):63-89, 2006.
[7] Denis S. Bernstein and Wassim M. Haddad. LQG control with an  $\mathcal{H}_{\infty}$  performance bound: A Riccati equation approach. In American Control Conference, pages 796-802, 1988.
[8] Dimitri Bertsekas. Dynamic Programming and Optimal Control, volume I. Athena Scientific, 2017.
[9] Dimitris Bertsimas and Vineet Goyal. On the power and limitations of affine policies in two-stage adaptive optimization. Mathematical Programming, 134(2):491-531, 2012.
[10] Dimitris Bertsimas, Dan A. Iancu, and Pablo A. Parrilo. Optimality of affine policies in multistage robust optimization. Mathematics of Operations Research, 35(2):363-394, 2010.
[49] Bart P. G. Van Parys, Paul J. Goulart, and Manfred Morari. Infinite horizon performance bounds for uncertain constrained systems. IEEE Transactions on Automatic Control, 58(11):2803-2817, 2013.
[50] Bart P. G. Van Parys, Daniel Kuhn, Paul J. Goulart, and Manfred Morari. Distributionally robust control of constrained stochastic systems. IEEE Transactions on Automatic Control, 61(2):430-442, 2016.
[51] Peter Whittle. Risk-sensitive linear/quadratic/Gaussian control. Advances in Applied Probability, 13(4):764-777, 1981.
[52] Insoon Yang. Wasserstein distributionally robust stochastic control: A data-driven approach. IEEE Transactions on Automatic Control, 66(8):3863-3870, 2021.
[53] Kemin Zhou and John C. Doyle. Essentials of Robust Control. Prentice Hall, 1998.