Sai Praneeth Karimireddy, Sebastian Stich, and Martin Jaggi. Adaptive balancing of gradient and update computation times using global geometry and approximate subproblems. Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS, 2018).
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U. Stich, and Martin Jaggi. Error feedback fixes SignSGD and other gradient compression schemes. In International Conference on Machine Learning (ICML), volume 97, pages 3252-3261, 2019.
Ahmed Khaled and Peter Rictarik. Better theory for SGD in the nonconvex world. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum? id=AU4qHN2VkS. Survey Certification.
Sarit Khirirat, Hamid Reza Feyzmahdavian, and Mikael Johansson. Distributed learning with compressed gradients. arXiv preprint arXiv:1806.06573, 2018a.
Sarit Khirirat, Mikael Johansson, and Dan Alistarh. Gradient compression for communication-limited convex optimization. In 2018 IEEE Conference on Decision and Control (CDC), pages 166-171. IEEE, 2018b.
Sarit Khirirat, Sindri Magnússon, and Mikael Johansson. Compressed gradient methods with Hessian-aided error compensation. IEEE Transactions on Signal Processing, 69:998-1011, 2020.
Sarit Khirirat, Sindri Magnusson, and Mikael Johansson. Eco-Fedsplit: Federated learning with error-compensated compression. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5952-5956. IEEE, 2022.
Yunwei Lei, Ting Hu, Guiying Li, and Ke Tang. Stochastic Gradient Descent for nonconvex learning without bounded gradient assumptions. In IEEE Transactions on Neural Networks and Learning Systems, pages 1-7. ISSN 2162-2388, 2019.
Rémi Leluc and François Portier. SGD with coordinate sampling: Theory and practice. Journal of Machine Learning Research, 23(342):1-47, 2022.
Daniel Levy, Yair Carmon, John C. Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. In Proceedings of the 34th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Sijia Liu, Bhavya Kailkhura, Pin-Yu Chen, Paishun Ting, Shiyu Chang, and Lisa Amini. Zeroth-order stochastic variance reduction for nonconvex optimization. Advances in Neural Information Processing Systems (NeurIPS), 31:3727-3737, 2018.
Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev, and Peter Richtárik. IntSGD: Adaptive floatless compression of stochastic gradients. arXiv preprint arXiv:2102.08374, 2021.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Arkadi Nemirovsky and David Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley, New York, 1983.
Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 17(2):527-566, 2017.
Feng Niu, Benjamin Recht, Christopher Re, and Stephen Wright. Hogwild: A lock-free approach to parallelizing Stochastic Gradient Descent. In Advances in Neural Information Processing Systems (NeurIPS), volume 24, pages 693-701, 2011.
Boris Polyak. Gradient methods for minimizing functionals. U.S.S.R. Computational Mathematics and Mathematical Physics, 3(4):864-878, 1963.
Boris Polyak. Introduction to Optimization. Optimization Software, Inc., 1987.