  Federated learning (FL) has attracted tremendous attentions in recent years
due to its privacy preserving measures and great potentials in some distributed
but privacy-sensitive applications like finance and health. However, high
communication overloads for transmitting high-dimensional networks and extra
security masks remains a bottleneck of FL. This paper proposes a
communication-efficient FL framework with Adaptive Quantized Gradient (AQG)
which adaptively adjusts the quantization level based on local gradient's
update to fully utilize the heterogeneousness of local data distribution for
reducing unnecessary transmissions. Besides, the client dropout issues are
taken into account and the Augmented AQG is developed, which could limit the
dropout noise with an appropriate amplification mechanism for transmitted
gradients. Theoretical analysis and experiment results show that the proposed
AQG leads to 25%-50% of additional transmission reduction as compared to
existing popular methods including Quantized Gradient Descent (QGD) and Lazily
Aggregated Quantized (LAQ) gradient-based method without deteriorating
convergence properties. Particularly, experiments with heterogenous data
distributions corroborate a more significant transmission reduction compared
with independent identical data distributions. Meanwhile, the proposed AQG is
robust to a client dropping rate up to 90% empirically, and the Augmented AQG
manages to further improve the FL system's communication efficiency with the
presence of moderate-scale client dropouts commonly seen in practical FL
scenarios.
