[1] Farshed Abdukhakimov, Chulu Xiang, Dmitry Kamzolov, and Martin Takáč. Stochastic gradient descent with preconditioned polyak step-size. arXiv preprint arXiv:2310.02093, 2023.
[2] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. arXiv preprint arXiv:2111.04263, 2021.
[3] Larry Armijo. Minimization of functions having lipschitz continuous first partial derivatives. Pacific Journal of Mathematics, 16(1):1-3, 1 1966.
[4] Hilal Asi and John C Duchi. Stochastic (approximate) proximal point methods: Convergence, optimality, and adaptivity. SIAM Journal on Optimization, 29(3):2257-2290, 2019.
[5] Hilal Asi, Daniel Lévy, and John C Duchi. Adapting to function difficulty and growth conditions in private optimization. Advances in Neural Information Processing Systems, 34:19069-19081, 2021.
[6] Leonard Berrada, Andrew Zisserman, and M. Pawan Kumar. Training neural networks for and by interpolation. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020.
[7] Léon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM Review, 60(2):223-311, 2018. doi: 10.1137/16M1080173. URL https://doi.org/10.1137/16M1080173.
[8] Sébastien Bubeck. Convex optimization: Algorithms and complexity. Found. Trends Mach. Learn., 8(3-4): 231-357, nov 2015. ISSN 1935-8237. doi: 10.1561/2200000050. URL https://doi.org/10.1561/2200000050.
[9] Karan Chadha, Gary Cheng, and John Duchi. Accelerated, optimal and parallel: Some results on model-based stochastic optimization. In International Conference on Machine Learning, pages 2811-2827. PMLR, 2022.
[10] Chih-Chung Chang and Chih-Jen Lin. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2(3), may 2011. ISSN 2157-6904. doi: 10.1145/1961189.1961199. URL https://doi.org/10.1145/1961189.1961199.
[11] Gary Cheng and John Duchi. adastar: A method for adapting to interpolation. In OPT 2022: Optimization for Machine Learning (NeurIPS 2022 Workshop), 2022. URL https://openreview.net/forum?id=dmbczvwk6Jw.
[12] Ashok Cutkosky and Kwabena Boahen. Online convex optimization with unconstrained domains and losses. In Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS'16, page 748-756, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
[13] Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. Saga: A fast incremental gradient method with support for non-strongly convex composite objectives. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL https://proceedings.neurips.cc/paper_files/paper/2014/file/ede7e2b6d13a41ddb9f4bdef84fdc737-Paper.pdf.
[14] Benjamin Dubois-Taine, Sharan Vaswani, Reza Babanezhad, Mark Schmidt, and Simon Lacoste-Julien. SVRG meets adagrad: painless variance reduction. Mach. Learn., 111(12):4359-4409, 2022. doi: 10.1007/s10994-022-06265-x. URL https://doi.org/10.1007/s10994-022-06265-x.
[15] John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res., 12(null):2121-2159, jul 2011. ISSN 1532-4435.
[16] Leonardo Galli, Holger Rauhut, and Mark Schmidt. Don't be so monotone: Relaxing stochastic line search in over-parameterized models, 2023.
[17] Robert M. Gower, Aaron Defazio, and Michael G. Rabbat. Stochastic polyak stepsize with a moving target. CoRR, abs/2106.11851, 2021. URL https://arxiv.org/abs/2106.11851.
[18] Robert M Gower, Mathieu Blondel, Nidham Gazagnadou, and Fabian Pedregosa. Cutting some slack for sgd with adaptive polyak stepsizes. arXiv preprint arXiv:2202.12328, 2022.