[26] N. Hansen. The CMA Evolution Strategy: A Comparing Review, pages 75-102. Springer Berlin Heidelberg, Berlin, Heidelberg, 2006.
[27] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournaapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. Fernández del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant. Array programming with NumPy. Nature, 585(7825):357–362, September 2020.
[28] A. Hedayat and W. D. Wallis. Hadamard matrices and their applications. The Annals of Statistics, 6(6):1184-1238, 1978.
[29] J. D. Hunter. Matplotlib: A 2d graphics environment. Computing in Science & Engineering, 9(3):90-95, 2007.
[30] N. K. Jain, U. Nangia, and J. Jain. A review of particle swarm optimization. Journal of The Institution of Engineers (India): Series B, 99(4):407-411, Aug 2018.
[31] J. Kiefer and J. Wolfowitz. Stochastic estimation of the maximum of a regression function. Annals of Mathematical Statistics, 23:462-466, 1952.
[32] D. Kozak, C. Molinari, L. Rosasco, L. Tenorio, and S. Villa. Zeroth order optimization with orthogonal random directions, 2021.
[33] R. M. Lewis, V. Torczon, and M. W. Trosset. Direct search methods: then and now. Journal of Computational and Applied Mathematics, 124(1):191-207, 2000. Numerical Analysis 2000. Vol. IV: Optimization and Nonlinear Equations.
[34] T. Lin, Z. Zheng, and M. Jordan. Gradient-free methods for deterministic and stochastic nonsmooth nonconvex optimization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 26160-26175. Curran Associates, Inc., 2022.
[35] S. Liu, P. Y. Chen, B. Kailkhura, G. Zhang, A. O. Hero III, and P. K. Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37(5):43-54, 2020.
[36] H. Mania, A. Guy, and B. Recht. Simple random search of static linear policies is competitive for reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[37] P. Mattila. Geometry of Sets and Measures in Euclidean Spaces: Fractals and Rectifiability. Cambridge Studies in Advanced Mathematics. Cambridge University Press, 1995.
[38] F. Mezzadri. How to generate random matrices from the classical compact groups. arXiv preprint math-ph/0609050, 2006.
[39] Y. Nesterov and V. Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17:527-566, 2017.
[40] Z. Opial. Weak convergence of the sequence of successive approximations for nonexpansive mappings. Bulletin of the American Mathematical Society, 73(4):591 - 597, 1967.
[41] B. T. Polyak. Introduction to optimization. Optimization Software Inc., Publications Division, New York, 1:32, 1987.
[42] M. Rando, C. Molinari, S. Villa, and L. Rosasco. Stochastic zeroth order descent with structured directions, 2022.
[43] H. Robbins and D. Siegmund. A convergence theorem for non-negative almost supermartingales and some applications. In J. S. Rustagi, editor, Optimizing Methods in Statistics, pages 233-257. Academic Press, 1971.