[43] J. Li, A. So, and W. Ma. Understanding notions of stationarity in nonsmooth optimization: A guided tour of various constructions of subdifferential for nonsmooth functions. IEEE Signal Processing Magazine, 37(5):18-31, 2020.
[44] Y. Li, Y. Tang, R. Zhang, and N. Li. Distributed reinforcement learning for decentralized linear quadratic control: A derivative-free policy optimization approach. IEEE Transactions on Automatic Control, 2021.
[45] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.
[46] T. Lin, Z. Zheng, and M. Jordan. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. In Conference on Neural Information Processing Systems, 2022.
[47] D. Malik, A. Pananjady, K. Bhatia, K. Khamaru, P. Bartlett, and M. Wainwright. Derivative-free methods for policy optimization: Guarantees for linear quadratic systems. In International Conference on Artificial Intelligence and Statistics, pages 2916-2925, 2019.
[48] V. Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, and S. Petersen. Human-level control through deep reinforcement learning. Nature, 518:529-533, 2015.
[49] D. Moerder and A. Calise. Convergence of a numerical algorithm for calculating optimal output feedback gains. IEEE Transactions on Automatic Control, 30(9):900-903, 1985.
[50] H. Mohammadi, A. Zare, M. Soltanolkotabi, and M. R. Jovanovic. Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem. IEEE Transactions on Automatic Control, 2021.
[51] M. I. Müller, P. E. Valenzuela, A. Proutiere, and C. R. Rojas. A stochastic multi-armed bandit approach to nonparametric  $\mathcal{H}_{\infty}$ -norm estimation. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 4632-4637. IEEE, 2017.
[52] D. Noll and P. Apkarian. Spectral bundle methods for non-convex maximum eigenvalue functions: second-order methods. Mathematical Programming, 104(2):729-747, 2005.
[53] T. Oomen, R. van der Maas, C. R. Rojas, and H. Hjalmarsson. Iteratively learning the  $\mathcal{H}_{\infty}$ -norm of multivariable systems applied to model-error-modeling of a vibration isolation system. In 2013 American Control Conference, pages 6703-6708. IEEE, 2013.
[54] F. Oustry. A second-order bundle method to minimize the maximum eigenvalue function. Mathematical Programming, 89:1-33, 2000.
[55] S. Oymak and N. Ozay. Non-asymptotic identification of lti systems from a single trajectory. In 2019 American control conference (ACC), pages 5655-5661. IEEE, 2019.
[56] I. K. Ozaslan, H. Mohammadi, and M. R. Jovanović. Computing stabilizing feedback gains via a model-free policy gradient method. IEEE Control Systems Letters, 7:407-412, 2022.
[57] J. Perdomo, J. Umenberger, and M. Simchowitz. Stabilizing dynamical systems via policy gradient methods. Advances in Neural Information Processing Systems, 34, 2021.
[58] G. Qu, C. Yu, S. Low, and A. Wierman. Exploiting linear models for model-free nonlinear control: A provably convergent policy gradient approach. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 6539-6546, 2021.
[59] S. Rathod, M. Bhadu, and A. De. Global convergence using policy gradient methods for model-free Markovian jump linear quadratic control. arXiv preprint arXiv:2111.15228, 2021.
[60] T. Rautert and E. W. Sachs. Computational design of optimal output feedback controllers. SIAM Journal on Optimization, 7(3):837-852, 1997.
[61] T. Sarkar and A. Rakhlin. Near optimal finite time identification of arbitrary linear dynamical systems. In International Conference on Machine Learning, pages 5610-5618, 2019.
[62] C. Scherer, P. Gahinet, and M. Chilali. Multiobjective output-feedback control via lmi optimization. IEEE Transactions on automatic control, 42(7):896-911, 1997.
[63] C. Scherer and S. Wieland. Linear matrix inequalities in control. Lecture notes for a course of the dutch institute of systems and control, Delft University of Technology, 2004.