[1] A. Agarwal and L. Bottou. A lower bound for the optimization of finite sums. In International Conference on Machine Learning, 2015.
[2] S. A. Alghunaim and K. Yuan. A unified and refined convergence analysis for non-convex decentralized learning. arXiv preprint arXiv:2110.09993, 2021.
[3] D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. Vojnovic. QSGD: Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems, 2017.
[4] D. Alistarh, T. Hoefler, M. Johansson, S. Khirirat, N. Konstantinov, and C. Renggli. The convergence of sparsified gradient methods. In Advances in Neural Information Processing Systems, 2018.
[5] Y. Arjevani, Y. Carmon, J. C. Duchi, D. J. Foster, N. Srebro, and B. E. Woodworth. Lower bounds for non-convex stochastic optimization. *ArXiv*, abs/1912.02365, 2019.
[6] Y. Arjevani and O. Shamir. Communication complexity of distributed convex learning and optimization. In Advances in Neural Information Processing Systems, 2015.
[7] E. Balkanski and Y. Singer. Parallelization does not accelerate convex optimization: Adaptivity lower bounds for non-smooth convex minimization. ArXiv, abs/1808.03880, 2018.
[8] J. Bernstein, Y.-X. Wang, K. Azizzadenesheli, and A. Anandkumar. Signsgd: compressed optimisation for non-convex problems. In International Conference on Machine Learning, 2018.
[9] Y. Carmon, J. C. Duchi, O. Hinder, and A. Sidford. Lower bounds for finding stationary points i. Mathematical Programming, 184(1):71-120, 2020.
[10] Y. Carmon, J. C. Duchi, O. Hinder, and A. Sidford. Lower bounds for finding stationary points ii: first-order methods. Mathematical Programming, 185:315-355, 2021.
[11] C.-C. Chang and C.-J. Lin. Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST), 2(3):1-27, 2011.
[12] J. Chen and A. H. Sayed. Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Transactions on Signal Processing, 60(8):4289-4305, 2012.
[13] T. Chen, G. Giannakis, T. Sun, and W. Yin. LAG: Lazily aggregated gradient for communication-efficient distributed learning. In Advances in Neural Information Processing Systems, pages 5050-5060, 2018.
[14] Z. Cheng, X. Huang, and K. Yuan. Momentum benefits non-iid federated learning simply and provably. arXiv preprint arXiv:2306.16504, 2023.
[15] J. Diakonikolas and C. Guzmán. Lower bounds for parallel and randomized convex optimization. In Conference on Learning Theory, 2019.
[16] P. Elias. Universal codeword sets and representations of the integers. IEEE transactions on information theory, 21(2):194-203, 1975.
[17] D. J. Foster, A. Sekhari, O. Shamir, N. Srebro, K. Sridharan, and B. E. Woodworth. The complexity of making the gradient small in stochastic convex optimization. In Conference on Learning Theory, 2019.
[18] V. Gandikota, D. Kane, R. K. Maity, and A. Mazumdar. vqsgd: Vector quantized stochastic gradient descent. In International Conference on Artificial Intelligence and Statistics, pages 2197-2205. PMLR, 2021.
[19] Y. He, X. Huang, Y. Chen, W. Yin, and K. Yuan. Lower bounds and accelerated algorithms in distributed stochastic optimization with communication compression. arXiv preprint arXiv:2305.07612, 2023.
[20] S. Horvath, C.-Y. Ho, L. Horvath, A. N. Sahu, M. Canini, and P. Richtárik. Natural compression for distributed deep learning. ArXiv, abs/1905.10988, 2019.
[21] X. Huang, Y. Chen, W. Yin, and K. Yuan. Lower bounds and nearly optimal algorithms in distributed learning with communication compression. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[22] X. Huang, P. Li, and X. Li. Stochastic controlled averaging for federated learning with communication compression. arXiv preprint arXiv:2308.08165, 2023.