SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof of Proposition D.2. The proof relies on dualizing the inner minimization problem in (11). Note that strong duality holds because the primal problem is trivially feasible and involves only equality constraints, which implies that any feasible point is in fact a Slater point. In the following we use  $M \in \mathcal{M}$  to denote the Lagrange multiplier of the constraint  $U \in \mathcal{U}$ , which requires all blocks of
the matrix  $U$  above the main diagonal to vanish. The Lagrangian function of the inner minimization problem in (11) can therefore be represented as
$$
\begin{array}{l} \mathcal {L} (q, U, M) = \operatorname {T r} \left(\left(D ^ {\top} U ^ {\top} (R + H ^ {\top} Q H) U D + G ^ {\top} Q G\right) W\right) + 2 \operatorname {T r} \left(G ^ {\top} Q H U D W\right) \\ + \operatorname {T r} \left(\left(U ^ {\top} (R + H ^ {\top} Q H) U\right) V\right) + q ^ {\top} (R + H ^ {\top} Q H) q + \operatorname {T r} (U M ^ {\top}). \\ \end{array}
$$
Recall now that  $R \succ 0$  and  $Q \succeq 0$ , and thus  $R + H^{\top}QH \succ 0$ . Consequently,  $\mathcal{L}$  is minimized by  $q^{\star} = 0$  for any fixed  $U$  and  $M$ . In addition, the partial gradient of  $\mathcal{L}$  with respect to  $U$  is given by
$$
\frac {\partial \mathcal {L}}{\partial U} = 2 (R + H ^ {\top} Q H) U D W D ^ {\top} + 2 (R + H ^ {\top} Q H) U V + 2 H ^ {\top} Q G W D ^ {\top} + M.
$$
Recall also that  $V \in \mathcal{G}_V^+$  is strictly positive, which implies that  $DWD^\top + V \succ 0$  is invertible. As we already know that  $R + H^\top QH \succ 0$  is invertible, as well,  $\mathcal{L}$  is minimized by
$$
U ^ {\star} = - (R + H ^ {\top} Q H) ^ {- 1} \left(H ^ {\top} Q G W D ^ {\top} + M / 2\right) \left(D W D ^ {\top} + V\right) ^ {- 1}
$$
for any fixed  $M$ . Substituting both  $q^{\star}$  and  $U^{\star}$  into  $\mathcal{L}$  yields the dual objective function
$$
\begin{array}{l} g (M) = \mathcal {L} \left(q ^ {\star}, U ^ {\star}, M\right) = \operatorname {T r} \left(G ^ {\top} Q G W\right) \\ - \operatorname {T r} \left((R + H ^ {\top} Q H) ^ {- 1} (H ^ {\top} Q G W D ^ {\top} + M / 2) (D W D ^ {\top} + V) ^ {- 1} (H ^ {\top} Q G W D ^ {\top} + M / 2) ^ {\top}\right). \\ \end{array}
$$
The dual of the inner minimization problem in (11) is thus given by  $\max_{M\in \mathcal{M}}g(M)$ . To linearize the dual objective function, we next introduce an auxiliary variable  $F\in \mathbb{S}_{+}^{mT}$  subject to the matrix inequality  $F\succeq (H^{\top}QGWD^{\top} + M / 2)(DWD^{\top} + V)^{-1}(H^{\top}QGWD^{\top} + M / 2)^{\top}$ . By using a standard Schur complement reformulation, we can then rewrite the dual problem as
$$
\begin{array}{l} \max  \operatorname {T r} \left(G ^ {\top} Q G W\right) - \operatorname {T r} \left(\left(R + H ^ {\top} Q H\right) ^ {- 1} F\right) \\ \begin{array}{l l} \text {s . t .} & M \in \mathcal {M}, F \in \mathbb {S} _ {+} ^ {m T} \\ & \left[ \begin{array}{c c} F & H ^ {\top} Q G W D ^ {\top} + M / 2) ^ {\top} & D W D ^ {\top} + V \end{array} \right] \succeq 0. \end{array} \tag {A.21} \\ \end{array}
$$
Next, by replacing the inner problem in (11) with its strong dual (A.21), we can reformulate (11) as
$$
\max  \operatorname {T r} \left(G ^ {\top} Q G W\right) - \operatorname {T r} \left(\left(R + H ^ {\top} Q H\right) ^ {- 1} F\right)
$$
$$
\begin{array}{l} \text {s . t .} \quad M \in \mathcal {M}, F \in \mathbb {S} _ {+} ^ {m T}, W \in \mathbb {S} _ {+} ^ {n (T + 1)}, V \in \mathbb {S} _ {+} ^ {p T} \\ \left[ \begin{array}{c c} F & H ^ {\top} Q G W D ^ {\top} + M / 2 \\ (H ^ {\top} Q G W D ^ {\top} + M / 2) ^ {\top} & D W D ^ {\top} + V \end{array} \right] \succeq 0 \tag {A.22} \\ \mathbb {G} (X _ {0}, \hat {X} _ {0}) ^ {2} \leq \rho_ {x _ {0}} ^ {2}, \mathbb {G} (W _ {t}, \hat {W} _ {t}) \leq \rho_ {w _ {t}} ^ {2}, \mathbb {G} (V _ {t}, \hat {V} _ {t}) \leq \rho_ {v _ {t}} ^ {2} \quad \forall t \in [ T - 1 ]. \\ \end{array}
$$
By Proposition 4.1, the inclusion of the constraints  $X_0 \succeq \lambda_{\min}(\hat{X}_0)I$ ,  $W_t \succeq \lambda_{\min}(\hat{W}_t)I$  and  $V_t \succeq \lambda_{\min}(\hat{V}_t)I$  for all  $t \in [T - 1]$  has no effect on the solution to problem (A.22). In addition, by Lemma D.1, each (non-linear) Gelbrich constraint in (A.22) can be reformulated as an equivalent (linear) SDP constraint. Thus, problem (A.22) reduces to (A.20), and the claim follows.
E. Bisection Algorithm for the Linearization Oracle
We now show that the direction-finding subproblem (14) can be solved efficiently via bisection. To this end, we first establish that (14) can be reduced to the solution of a univariate algebraic equation.
Proposition E.1 ([38, Proposition A.4 (iii)]). If  $\hat{Z} \in \mathbb{S}_{++}^{d}$ ,  $\Gamma_{Z} \in \mathbb{S}_{+}^{d}$ ,  $\Gamma_{Z} \neq 0$  and  $\rho_{z} \in \mathbb{R}_{++}$ , then
$$
\begin{array}{l} \max  \quad \langle \Gamma_ {Z}, L - Z \rangle \\ \mathrm {s . t .} \quad \mathbb {G} (L, \hat {Z}) \leq \rho_ {z} \tag {A.23} \\ L \succeq \lambda_ {\min } (\hat {Z}) I \\ \end{array}
$$
is uniquely solved by  $L^{\star} = (\gamma^{\star})^{2}(\gamma^{\star}I - \Gamma_{Z})^{-1}\hat{Z} (\gamma^{\star}I - \Gamma_{Z})^{-1}$ , where  $\gamma^{\star}$  is the unique solution of
$$
\rho_ {z} ^ {2} - \left\langle \hat {Z}, (I - \gamma^ {\star} (\gamma^ {\star} I - \Gamma_ {Z}) ^ {- 1}) ^ {2} \right\rangle = 0 \tag {A.24}
$$
in the interval  $(\lambda_{\max}(\Gamma_Z),\infty)$
In practice, we need to solve the algebraic equation (A.24) numerically. The numerical error in approximating  $\gamma^{\star}$  should be contained to ensure that  $L^{\star}$  approximates the exact maximizer of problem (A.23). The next proposition shows that, for any tolerance  $\delta \in (0,1)$ , a  $\delta$ -approximate solution of (A.23) can be computed with an efficient bisection algorithm.
Proposition E.2 ([38, Theorem 6.4]). For any fixed  $\rho_z\in \mathbb{R}_{+ + }$ $\hat{Z}\in \mathbb{S}_{++}^{d}$  and  $\Gamma_Z\in \mathbb{S}_+^d$ $\Gamma_{Z}\neq 0$  define  $\mathcal{G}_Z^+ = \{Z\in \mathbb{S}_+^d:\mathbb{G}(Z,\hat{Z})\leq \rho_z,Z\succeq \lambda_{\min}(\hat{Z})\}$  as the feasible set of problem (A.23), and let  $Z\in \mathcal{G}_Z^+$  be any reference covariance matrix. Additionally, let  $\delta \in (0,1)$  be the desired oracle precision, and define  $\varphi (\gamma) = \gamma (\rho^{2} + \langle \gamma (\gamma I - \Gamma_{Z})^{-1} - I,\hat{Z}\rangle) - \langle Z,\Gamma_{Z}\rangle$  for any  $\gamma >\lambda_{\max}(\Gamma_Z)$ . Then, Algorithm A.2 returns in finite time a matrix  $L_{Z}^{\delta}\in \mathbb{S}_{+}^{d}$  with the following properties. (i) Feasibility:  $L_{Z}^{\delta}\in \mathcal{G}_{Z}^{+}$  (ii)  $\delta$  Suboptimality:  $\langle L_Z^\delta -Z,\Gamma_Z\rangle \geq \delta \max_{L\in \mathcal{G}_Z^+}\langle \Gamma_Z,L - Z\rangle$
In summary, for any  $Z \in \{X_0, W_0, \ldots, W_{T-1}, V_0, \ldots, V_{T-1}\}$ , Algorithm A.2 computes a  $\delta$ -approximate solutions to the direction-finding subproblem (14) with  $\Gamma_Z = \nabla_Z f(W, V)$ .
F. Additional Information on Experiments
Generation of Nominal Covariance Matrices. The nominal covariance matrices of the exogenous uncertainties are constructed randomly using the following procedure. For each exogenous uncertainty  $z \in \{x_0, w_0, \dots, w_{T-1}, v_0, \dots, v_{T-1}\}$ , we denote the dimension of  $z$  by  $d$  and sample a matrix  $M_Z \in \mathbb{R}^{d \times d}$  from the uniform distribution on the hypercube  $[0, 1]^{d \times d}$ . Next, we define  $\Xi_Z \in \mathbb{R}^{d \times d}$  as the orthogonal matrix whose columns represent the orthonormal eigenvectors of the symmetric matrix  $M_Z + M_Z^\top$ . Finally, we set  $\hat{Z} = \Xi_Z \Lambda_Z \Xi_Z^\top$ , where  $\Lambda_Z$  is a diagonal matrix whose main diagonal is sampled uniformly from the interval  $[1, 2]^d$ . The rationale for adopting this cumbersome procedure is to ensure that the covariance matrix  $\hat{Z}$  is positive definite.
Optimality Gap. The optimality gap of the Frank-Wolfe algorithm visualized in Figure 1b is calculated as the sum of the surrogate optimality gaps  $\langle L_Z^\delta - Z, \nabla_Z f(W, V) \rangle$  across all  $Z \in \{X_0, W_0, \ldots, W_{T-1}, V_0, \ldots, V_{T-1}\}$ . For more information on the surrogate optimality gaps see [31].