[64] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015.
[65] D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al. Mastering the game of Go without human knowledge. Nature, 550(7676):354-359, 2017.
[66] Y. Tang and Y. Zheng. On the global optimality of direct policy search for nonsmooth  $h_{\infty}$  output-feedback control. arXiv preprint arXiv:2304.00753, 2023.
[67] H. Toivonen. A globally convergent algorithm for the optimal constant output feedback problem. International Journal of Control, 41(6):1589-1599, 1985.
[68] S. Tu, R. Boczar, and B. Recht. On the approximation of Toeplitz operators for nonparametric  $\mathcal{H}_{\infty}$ -norm estimation. In 2018 Annual American Control Conference (ACC), pages 1867-1872. IEEE, 2018.
[69] J. Umenberger, M. Simchowitz, J. C. Perdomo, K. Zhang, and R. Tedrake. Globally convergent policy search over dynamic filters for output estimation. arXiv preprint arXiv:2202.11659, 2022.
[70] B. Wahlberg, M. Syberg, and H. Hjalmarsson. Non-parametric methods for  $l_{2}$ -gain estimation using iterative experiments. Automatica, 46(8):1376-1381, 2010.
[71] B. Wahlberg, M. B. Syberg, and H. Hjalmarsson. Non-parametric methods for  $l_{2}$ -gain estimation using iterative experiments. Automatica, 46(8):1376-1381, 2010.
[72] Z. Yang, Y. Chen, M. Hong, and Z. Wang. Provably global convergence of actor-critic: A case for linear quadratic regulator with ergodic cost. Advances in neural information processing systems, 32, 2019.
[73] H. Zhang, Y. Shi, and A. Mehr. Robust static output feedback control and remote pid design for networked motor systems. IEEE Transactions on Industrial Electronics, 58(12):5396-5405, 2011.
[74] K. Zhang, B. Hu, and T. Basar. On the stability and convergence of robust adversarial reinforcement learning: A case study on linear quadratic systems. Advances in Neural Information Processing Systems, 33, 2020.
[75] K. Zhang, B. Hu, and T. Basar. Policy optimization for  $\mathcal{H}_2$  linear control with  $\mathcal{H}_{\infty}$  robustness guarantee: Implicit regularization and global convergence. SIAM Journal on Control and Optimization, 59(6):4081-4109, 2021.
[76] K. Zhang, X. Zhang, B. Hu, and T. Başar. Derivative-free policy optimization for linear risk-sensitive and robust control design: Implicit regularization and sample complexity. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
[77] F. Zhao and K. You. Primal-dual learning for the model-free risk-constrained linear quadratic regulator. In Learning for Dynamics and Control, pages 702-714, 2021.
[78] Y. Zheng, Y. Sun, M. Fazel, and N. Li. Escaping high-order saddles in policy optimization for linear quadratic gaussian (lqg) control. In IEEE 61st Conference on Decision and Control (CDC), pages 5329-5334, 2022.
[79] Y. Zheng, Y. Tang, and N. Li. Analysis of the optimization landscape of linear quadratic gaussian (LQG) control. arXiv preprint arXiv:2102.04393, 2021.
[80] K. Zhou, J. Doyle, and K. Glover. Robust and Optimal Control. Prentice-Hall, 1996.