[23] X. Huang, K. Yuan, X. Mao, and W. Yin. Improved analysis and rates for variance reduction under without-replacement sampling orders. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[24] S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning, 2020.
[25] S. P. Karimireddy, Q. Rebjock, S. U. Stich, and M. Jaggi. Error feedback fixes signsgd and other gradient compression schemes. In International Conference on Machine Learning, 2019.
[26] A. Khaled, O. Sebbouh, N. Loizou, R. M. Gower, and P. Richtárik. Unified analysis of stochastic gradient methods for composite convex and smooth optimization. arXiv preprint arXiv:2006.11573, 2020.
[27] S. Khirirat, H. R. Feyzmahdavian, and M. Johansson. Distributed learning with compressed gradients. arXiv preprint arXiv:1806.06573, 2018.
[28] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2015.
[29] A. Koloskova, T. Lin, and S. U. Stich. An improved analysis of gradient tracking for decentralized machine learning. In Advances in Neural Information Processing Systems, 2021.
[30] A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, and S. Stich. A unified theory of decentralized sgd with changing topology and local updates. In International Conference on Machine Learning, pages 5381-5393. PMLR, 2020.
[31] D. Kovalev, E. Shulgin, P. Richtarik, A. V. Rogozin, and A. Gasnikov. Adom: Accelerated decentralized optimization method for time-varying networks. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 5784-5793. PMLR, 18-24 Jul 2021.
[32] Z. Li, D. Kovalev, X. Qian, and P. Richtárik. Acceleration for compressed gradient descent in distributed and federated optimization. arXiv preprint arXiv:2002.11364, 2020.
[33] Z. Li and P. Richtárik. Canita: Faster rates for distributed convex optimization with communication compression. Advances in Neural Information Processing Systems, 34:13770-13781, 2021.
[34] X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems, 2017.
[35] T. Lin, S. P. Karimireddy, S. U. Stich, and M. Jaggi. Quasi-global momentum: Accelerating decentralized deep learning on heterogeneous data. In International Conference on Machine Learning, 2021.
[36] Y. Liu, W. Xu, G. Wu, Z. Tian, and Q. Ling. Communication-censored admm for decentralized consensus optimization. IEEE Transactions on Signal Processing, 67(10):2565-2579, 2019.
[37] P. Mayekar and H. Tyagi. Ratq: A universal fixed-length quantizer for stochastic optimization. In International Conference on Artificial Intelligence and Statistics, pages 1399-1409. PMLR, 2020.
[38] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, 2017.
[39] P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, et al. Mixed precision training. In International Conference on Learning Representations, 2018.
[40] K. Mishchenko, E. A. Gorbunov, M. Takac, and P. Richtárik. Distributed learning with compressed gradient differences. ArXiv, abs/1901.09269, 2019.
[41] K. Mishchenko, G. Malinovsky, S. Stich, and P. Richtárik. Proxskip: Yes! local gradient steps provably lead to communication acceleration! finally! In International Conference on Machine Learning, pages 15750-15769. PMLR, 2022.
[42] Y. Nesterov. A method for unconstrained convex minimization problem with the rate of convergence o(1/k2). In Doklady an ussr, volume 29, 1983.
[43] Y. Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2003.