SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Federated learning has assumed growing relevance in ML. However, most practical problems are characterized by the presence of local objectives, jointly afflicted by nonconvexity and nondifferentiability, precluding resolution by most FL schemes, which can cope with nonconvexity in only smooth settings. We resolve this gap via a zeroth-order communication-efficient FL framework that can contend with both nondifferentiability and nonsmoothness with rate and complexity guarantees for computing approximate Clarke-stationary points. Extensions to nonconvex bilevel and nonconvex-strongly concave minimax settings are developed via inexact generalizations.
We acknowledge the funding support from the U.S. Department of Energy under grant #DE-SC0023303, and the U.S. Office of Naval Research under grants #N00014-22-1-2757 and #N00014-22-1-2589. We also would like to thank the four anonymous referees for their constructive suggestions.
[52] F. Yousefian, A. Nedic, and U. V. Shanbhag. On stochastic gradient and subgradient methods with adaptive steplength sequences. Automatica, 48(1):56-67, 2012.
[53] H. Yu, S. Yang, and S. Zhu. Parallel restarted SGD with faster convergence and less communication: demystifying why model averaging works for deep learning. In AAAI Conference on Artificial Intelligence, 2019.
[54] H. Yuan and T. Ma. Federated accelerated stochastic gradient descent. Advances in Neural Information Processing Systems, 33:5332-5344, 2020.
[55] X. Yuan and P. Li. On convergence of fedprox: Local dissimilarity invariant bounds, nonsmoothness and beyond. Advances in Neural Information Processing Systems, 35:10752-10765, 2022.
[56] J. Zhang, H. Lin, S. Jegelka, S. Sra, and A. Jadbabaie. Complexity of finding stationary points of nonconvex nonsmooth functions. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 11173-11182. PMLR, 13-18 Jul 2020.
[57] L. Zhang, D. Xu, S. Yuan, and X. Wu. FairGAN: Fairness-aware generative adversarial networks. In CoRR, 2018.
[58] F. Zhou and G. Cong. On the convergence properties of a K-step averaging stochastic gradient descent algorithm for nonconvex optimization. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2018.