Federated learning with stochastic quantization

International Journal of Intelligent SystemsVolume 37, Issue 12 p. 11600-11621 RESEARCH ARTICLE Federated learning with stochastic quantization Yawen Li, Yawen Li Management Science and Engineering Department, School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorWenling Li, Corresponding Author Wenling Li lwlmath@buaa.edu.cn orcid.org/0000-0002-9130-5736 Department of Intelligent Systems and Control Engineering, School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing, People's Republic of China Correspondence Wenling Li, Department of Intelligent Systems and Control Engineering, School of Automation Science and Electrical Engineering, Beihang University (BUAA), 100191 Beijing, People's Republic of China. Email: lwlmath@buaa.edu.cnSearch for more papers by this authorZhe Xue, Zhe Xue Department of Computer Science and Technology, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this author Yawen Li, Yawen Li Management Science and Engineering Department, School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this authorWenling Li, Corresponding Author Wenling Li lwlmath@buaa.edu.cn orcid.org/0000-0002-9130-5736 Department of Intelligent Systems and Control Engineering, School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing, People's Republic of China Correspondence Wenling Li, Department of Intelligent Systems and Control Engineering, School of Automation Science and Electrical Engineering, Beihang University (BUAA), 100191 Beijing, People's Republic of China. Email: lwlmath@buaa.edu.cnSearch for more papers by this authorZhe Xue, Zhe Xue Department of Computer Science and Technology, School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, People's Republic of ChinaSearch for more papers by this author First published: 02 September 2022 https://doi.org/10.1002/int.23056Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract This paper studies the distributed federated learning problem when the exchanged information between the server and the workers is quantized. A novel quantized federated averaging algorithm is developed by applying stochastic quantization scheme to the local and global model parameters. Specifically, the server broadcasts the quantized global model parameter to the workers; the workers update local model parameters using their own data sets and upload the quantized version to the server; then the server updates the global model parameter by aggregating all the quantized local model parameters and its previous global model parameter. This algorithm can be interpreted as a quantized variant of the federated averaging algorithm. The convergence is analyzed theoretically for both convex and strongly convex loss functions with Lipschitz gradient. Extensive experiments using realistic data are provided to show the effectiveness of the proposed algorithm. Open Research DATA AVAILABILITY STATEMENT Data sharing is not applicable to this article as no new data were created or analyzed in this study. Volume37, Issue12December 2022Pages 11600-11621 RelatedInformation