SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\Pr \left(\min  \left\{\| H \| _ {F}: H \in \partial_ {\delta} J \left(K ^ {R}\right) \geq \epsilon \right\}\right) \leq \Pr \left(\| \nabla J _ {\delta} \left(K ^ {R}\right) \| _ {F} \geq \epsilon\right) \leq 0. 1 5 v ^ {- \frac {1}{2}} + 0. 2 + 0. 0 3 v.
$$
Therefore our algorithm can find a  $(\delta, \epsilon)$ -stationary point with probability at least  $1 - (0.15v^{-\frac{1}{2}} + 0.2 + 0.03v) = 0.8 - 0.15v^{-\frac{1}{2}} - 0.03v$ . This completes the proof for Statement 2.
C Numerical Experiments
In this section, we provide more details on the numerical experiments. All the experiments are performed on a desktop computer with a 3.7 GHz Intel i5-9600K processor.
C.1 Two-dimension example
In this subsection, we first present an example where the iterates of Algorithm 1 can be visualized in controller space directly. In particular, we consider the following system
$$
A = \left[ \begin{array}{c c c} 0. 5 & 0 & - 1 \\ - 0. 5 & 0. 5 & 0 \\ 0 & 0 & 0. 5 \end{array} \right], B = \left[ \begin{array}{l} 1 \\ 1 \\ 0 \end{array} \right], C = \left[ \begin{array}{c c c} 1 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right]. \tag {C.1}
$$
The weight matrices  $Q$  and  $R$  are set as identity matrices with proper dimensions. In this case, we have  $K = [k_{1} \quad k_{2}] \in \mathbb{R}^{1 \times 2}$ . Since  $A$  is a stable matrix, we initialize  $K^0 = [0 \quad 0]$ . For Algorithm 1, we set  $\eta = 1 \times 10^{-3}$ ,  $\delta = 1 \times 10^{-4}$ , and  $\epsilon = 1 \times 10^{-3}$ . The iterates and contour lines of the  $\mathcal{H}_{\infty}$  norm are drawn in Figure 2. It can be seen that Algorithm 1 yields to a local minimum and after 5000 iterations, it converges to a controller  $K^{*} = [-0.1429 \quad -0.6425]$ .
Figure 2: Algorithm 1 iterates in controller space for (C.1).
C.2 Experiments for Figure 1
In this subsection, we discuss more details about the three different experiments displayed in Figure 1 to show the effectiveness of Algorithm 1 with exact and inexact oracle.
Left plot of Figure 1 For the left plot of Figure 1, we show the relative error of the Algorithm 1 with exact oracle for different system dimensions  $n_x = \{10,50,100\}$ . The system is of form (1) with parameters  $(A,B,C)$ . The entries of  $(A,B,C)$  are sampled from a standard normal distribution  $\mathcal{N}(0,1)$ . The system matrix  $A$  is scaled when necessary to make  $\rho(A) < 1$  such that we can set  $K^0$  as zero matrices. The cost matrices  $Q$  and  $R$  are set to be identities with appropriate dimensions. Table 2 shows the detailed algorithm parameters.
Table 2: Algorithm 1 parameters
(nx, nu, ny)	η	δ	ε
(10,5,5)	1 × 10-6	1 × 10-4	1 × 10-3
(50,10,10)	1 × 10-7	1 × 10-4	1 × 10-3
(100,20,20)	1 × 10-9	1 × 10-4	1 × 10-3
Middle plot of Figure 1 For the second experiment, we perform the Algorithm 1 with both exact and inexact zeroth-order oracles. In particular, we consider the following specific MIMO system:
$$
A = \left[ \begin{array}{c c c} 0. 5 & 0 & - 1 \\ - 0. 5 & 0. 5 & 0 \\ 0 & 0 & 0. 5 \end{array} \right], B = \left[ \begin{array}{c c} 1 & 0 \\ 0 & 1 \\ - 1 & 0 \end{array} \right], C = \left[ \begin{array}{c c c} 1 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right]. \tag {C.2}
$$
The weight matrices  $Q$  and  $R$  are set as:
$$
Q = \left[ \begin{array}{c c c} 2 & - 1 & 0 \\ - 1 & 2 & - 1 \\ 0 & - 1 & 2 \end{array} \right], R = \left[ \begin{array}{c c} 1 & 0 \\ 0 & 1 \end{array} \right].
$$
Since the system matrix  $A$  is Schur stable, we set the initial point  $K^0$  as a zero matrix. We also set the smooth radius  $\delta = 0.001$  and the stepsize  $\eta = 0.0001$ . It can be seen that the inexact oracle case works well and tracks the exact oracle scenario.
Right plot of Figure 1 Finally, we examine how the sample complexity varies with different  $\epsilon$  for the exact oracle setting with system matrices (C.2). It is worth mentioning that in Algorithm 1, we cannot really determine whether an iterate  $K^t$  is a  $(\delta, \epsilon)$ -stationary point or not since we cannot calculate the exact  $\nabla J_{\delta}(K^t)$ . In practice, we use the magnitude of  $g^t$  as a stationarity criterion. In particular, we terminate the algorithm when the norm of  $g^t$  is less than  $\epsilon$ .
Figure 3: Algorithm 1 trajectories for selected the library examples.
C.3 Comparison with model-based methods
In this subsection, we compare our derivative-free methods to model-based packages HIFOO and Hinfstruct, which are available in MATLAB. In particular, we select several benchmark models from COMPl<sub>e</sub>ib including aircraft model AC15, 2D heat flow model HF2D11, second-order model DLR2, and helicopter models HE4 [41]. Table 3 summarized more details on the selected examples.
It is worth mentioning that the models in  $\mathrm{COMPl}_e\mathrm{ib}$  are all continuous time models. In addition, the implementations of HIFOO and Hinfstruct are also based on continuous-time models. Therefore, we
Table 3: Selected examples form  ${\mathrm{{COMPl}}}_{e}\mathrm{{ib}}$
Physical model	Example	(nx, nu, ny)	Structure of A	Stability of A
Air craft model	AC15	(4, 2, 3)	dense	stable
2-D heat flow model	HF2D11	(5, 2, 3)	dense	unstable
second-order model	DLR2	(40, 2, 2)	sparse	stable
helicopter model	HE4	(8, 4, 6)	dense	unstable
implemented the continuous-time version of Algorithm 1. In other words, when we compute the  $\mathcal{H}_{\infty}$  norm of the closed-loop system, we are computing the  $\mathcal{H}_{\infty}$  norm of a continuous-time system instead of a discrete-time system. Figure 3 shows the trajectories of the four selected examples when we run Algorithm 1 for 5000 iterations. It can be seen that our algorithm converges to the ones computed by the model-based methods HIFOO and Hinfstruct with proper initialization. In practice, one can use multiple initial points and run Algorithm 1 multiple times then just report the best case with the lowest  $J(K)$ . Such a strategy is used in HIFOO and Hinfstruct. It can be seen that our derivative-free method can achieve comparable results even without the knowledge of the system dynamics.