[2] Raef Bassily, Mikhail Belkin, and Siyuan Ma. "On exponential convergence of sgd in nonconvex over-parametrized learning". In: arXiv preprint arXiv:1811.02564 (2018).
[3] Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. "Reconciling modern machine-learning practice and the classical bias-variance trade-off". In: Proceedings of the National Academy of Sciences 116.32 (2019), pp. 15849-15854.
[4] Samuel Burer and Renato DC Monteiro. “A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization”. In: Mathematical Programming 95.2 (2003), pp. 329–357.
[5] Jeremy M Cohen, Simran Kaur, Yuanzhi Li, J Zico Kolter, and Ameet Talwalkar. "Gradient descent on neural networks typically occurs at the edge of stability". In: arXiv preprint arXiv:2103.00065 (2021).
[6] Dmitriy Drusvyatskiy, Alexander D Ioffe, and Adrian S Lewis. "Curves of descent". In: SIAM Journal on Control and Optimization 53.1 (2015), pp. 114-138.
[7] Dmitriy Drusvyatskiy and Adrian S. Lewis. "Error Bounds, Quadratic Growth, and Linear Convergence of Proximal Methods". In: Math. of Oper. Res. 43.3 (2018), pp. 919-948. DOI: 10.1287/moor.2017.0889.eprint: https://doi.org/10.1287/moor.2017.0889. URL: https://doi.org/10.1287/moor.2017.0889.
[8] Simon Du, Jason Lee, Haochuan Li, Liwei Wang, and Xiyu Zhai. "Gradient Descent Finds Global Minima of Deep Neural Networks". In: International Conference on Machine Learning. 2019, pp. 1675-1685.
[9] Simon S Du, Wei Hu, and Jason D Lee. "Algorithms regularization in learning deep homogeneous models: Layers are automatically balanced". In: Advances in neural information processing systems 31 (2018).
[10] Simon S Du, Xiyu Zhai, Barnabas Poczos, and Aarti Singh. "Gradient Descent Provably Optimizes Over-parameterized Neural Networks". In: International Conference on Learning Representations. 2018.
[11] Qiang Fu, Dongchu Xu, and Ashia Wilson. "Accelerated Stochastic Optimization Methods under Quasar-convexity". In: arXiv preprint arXiv:2305.04736 (2023).
[12] Guillaume Garrigos and Robert M Gower. "Handbook of convergence theorems for (stochastic) gradient methods". In: arXiv preprint arXiv:2301.11235 (2023).
[13] Robert Gower, Othmane Sebbouh, and Nicolas Loizou. "Sgd for structured nonconvex functions: Learning rates, minibatching and interpolation". In: International Conference on Artificial Intelligence and Statistics. PMLR. 2021, pp. 1315-1323.
[14] Moritz Hardt, Tengyu Ma, and Benjamin Recht. "Gradient descent learns linear dynamical systems". In: arXiv preprint arXiv:1609.05191 (2016).
[15] Oliver Hinder, Aaron Sidford, and Nimit Sohoni. "Near-Optimal Methods for Minimizing Star-Convex Functions and Beyond". In: Proceedings of Thirty Third Conference on Learning Theory. Ed. by Jacob Abernethy and Shivani Agarwal. Vol. 125. Proceedings of Machine Learning Research. PMLR, Sept. 2020, pp. 1894–1938. URL: https://proceedings.mlr.org/press/v125/hinder20a.html.
[16] Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan First, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al. "Gpipe: Efficient training of giant neural networks using pipeline parallelism". In: Advances in neural information processing systems 32 (2019).
[17] Aleksandr Davidovich Ioffe. "Metric regularity and subdifferential calculus". In: Russian Mathematical Surveys 55.3 (2000), p. 501.
[18] Arthur Jacot, Franck Gabriel, and Clément Hongler. "Neural tangent kernel: Convergence and generalization in neural networks". In: Advances in neural information processing systems 31 (2018).
[19] Stanisław Jastrzebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. "Three factors influencing minima in sgd". In: arXiv preprint arXiv:1711.04623 (2017).
[20] Jikai Jin. "On the convergence of first order methods for quasar-convex optimization". In: arXiv preprint arXiv:2010.04937 (2020).