text: 372,166,1279060276189581313,1661813766,Mehdi Kamani,"Our new paper is out! We investigate the impacts of Compression on #FederatedLearning and present FedCOMGATE, which improves SOTA. We provide sharp guarantees under different settings in FL with Compression. <LINK> @Farzinhaddadpou @AryanMokhtari @mehrdadmahdavi <LINK> We show that even without compression, our algorithm matches the SOTA results with no extra control variable. We provide guarantees for general nonconvex, PL/strongly convex, and general convex objective functions, as well as, homogeneous and heterogenous data distributions. We accompanied our theoretical results with extensive experimental results. We investigate both Quantization and Sparsification as the compression method in our algorithm. #Quantization #Sparsification #Compression <LINK> We will have a major code release in the coming weeks for our Distributed Learning and Federated Learning setups, including FedCOMGATE and many more. Stay tuned. For getting updates you can follow my GitHub handle: <LINK> #MachineLearning #DistributedOptimization @aminkarbasi @AryanMokhtari @Farzinhaddadpou @mehrdadmahdavi Thanks 🙏",https://arxiv.org/abs/2007.01154,"In federated learning, communication cost is often a critical bottleneck to scale up distributed optimization algorithms to collaboratively learn a model from millions of devices with potentially unreliable or limited communication and heterogeneous data distributions. Two notable trends to deal with the communication overhead of federated algorithms are gradient compression and local computation with periodic communication. Despite many attempts, characterizing the relationship between these two approaches has proven elusive. We address this by proposing a set of algorithms with periodical compressed (quantized or sparsified) communication and analyze their convergence properties in both homogeneous and heterogeneous local data distribution settings. For the homogeneous setting, our analysis improves existing bounds by providing tighter convergence rates for both strongly convex and non-convex objective functions. To mitigate data heterogeneity, we introduce a local gradient tracking scheme and obtain sharp convergence rates that match the best-known communication complexities without compression for convex, strongly convex, and nonconvex settings. We complement our theoretical results and demonstrate the effectiveness of our proposed methods by several experiments on real-world datasets. ","Federated Learning with Compression: Unified Analysis and Sharp