[28] Hou, T. Y., Li, Z., and Zhang, Z. Analysis of asymptotic escape of strict saddle sets in manifold optimization. SIAM Journal on Mathematics of Data Science, 2(3):840-871, 2020.
[29] Hsieh, Y.-P., Mertikopoulos, P., and Cevher, V. The limits of min-max optimization algorithms: Convergence to spurious non-critical sets. In International Conference on Machine Learning, pp. 4337-4348. PMLR, 2021.
[30] Jin, C., Ge, R., Netrapalli, P., Kakade, S. M., and Jordan, M. I. How to escape saddle points efficiently. In International conference on machine learning, pp. 1724-1732. PMLR, 2017.
[31] Juditsky, A., Nemirovski, A. S., and Tauvel, C. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1):17-58, 2011.
[32] Kakade, S. M. A natural policy gradient. Advances in neural information processing systems, 14, 2001.
[33] Karimi, M. R., Hsieh, Y.-P., Mertikopoulos, P., and Krause, A. The dynamics of Riemannian Robbins-Monro algorithms. In COLT '22: Proceedings of the 35th Annual Conference on Learning Theory, 2022.
[34] Karimi, M. R., Hsieh, Y.-P., Mertikopoulos, P., and Krause, A. The dynamics of Riemannian Robbins-Monro algorithms. In COLT 2022-35th Annual Conference on Learning Theory, pp. 1-31, 2022.
[35] Kawaguchi, K. Deep learning without poor local minima. In NIPS '16: Proceedings of the 30th International Conference on Neural Information Processing Systems, 2016.
[36] Kiefer, J. and Wolfowitz, J. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics, 23(3):462-466, 1952.
[37] Kobayashi, S. and Nomizu, K. Foundations of Differential Geometry. Wiley Classics Library. Wiley, 2 edition, 1996.
[38] Korpelevich, G. M. The extragradient method for finding saddle points and other problems. *Ekonom. i Mat. Metody*, 12:747-756, 1976.
[39] Kushner, H. J. and Clark, D. S. Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, 1978.
[40] Kushner, H. J. and Yin, G. G. Stochastic approximation algorithms and applications. Springer-Verlag, New York, NY, 1997.
[41] Lee, J. D., Simchowitz, M., Jordan, M. I., and Recht, B. Gradient descent only converges to minimizers. In 29th Annual Conference on Learning Theory, pp. 1246-1257, 2016.
[42] Lee, J. D., Panageas, I., Piliouras, G., Simchowitz, M., Jordan, M. I., and Recht, B. First-order methods almost always avoid strict saddle points. Mathematical programming, 176:311-337, 2019.
[43] Lee, J. M. Riemannian Manifolds: an Introduction to Curvature. Number 176 in Graduate Texts in Mathematics. Springer, 1997.
[44] Lee, J. M. Introduction to Smooth Manifolds. Number 218 in Graduate Texts in Mathematics. Springer-Verlag, New York, NY, 2003.
[45] Mertikopoulos, P. and Sandholm, W. H. Riemannian game dynamics. Journal of Economic Theory, 177: 315-364, September 2018.
[46] Mertikopoulos, P., Hallak, N., Kavis, A., and Cevher, V. On the almost sure convergence of stochastic gradient descent in non-convex problems. In NeurIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020.
[47] Mertikopoulos, P., Hsieh, Y.-P., and Cevher, V. A unified stochastic approximation framework for learning in games. Mathematical Programming, forthcoming, 2023.
[48] Meyer, G., Bonnabel, S., and Sepulchre, R. Linear regression under fixed-rank constraints: a riemannian approach. In 28th International Conference on Machine Learning, 2011.
[49] Nemirovski, A. S. and Yudin, D. B. Problem Complexity and Method Efficiency in Optimization. Wiley, New York, NY, 1983.
[50] Nesterov, Y. Introductory Lectures on Convex Optimization: A Basic Course. Number 87 in Applied Optimization. Kluwer Academic Publishers, 2004.
[51] Neto, J. C., Santos, P., and Soares, P. An extragradient method for equilibrium problems on Hadamard manifolds. Optimization Letters, 10(6):1327-1336, 2016.
[52] Oja, E. Principal components, minor components, and linear neural networks. Neural networks, 5(6): 927-935, 1992.
[53] Panageas, I., Piliouras, G., and Wang, X. First-order methods almost always avoid saddle points: The case of vanishing step-sizes. In Advances in Neural Information Processing Systems, pp. 6471-6480, 2019.
[54] Pemantle, R. Nonconvergence to unstable points in urn models and stochastic approximations. Annals of Probability, 18(2):698-712, April 1990.
[55] Pemantle, R. Vertex-reinforced random walk. Probability Theory and Related Fields, 92:117-136, 1992.