  Federated averaging (FedAvg) is a communication efficient algorithm for the
distributed training with an enormous number of clients. In FedAvg, clients
keep their data locally for privacy protection; a central parameter server is
used to communicate between clients. This central server distributes the
parameters to each client and collects the updated parameters from clients.
FedAvg is mostly studied in centralized fashions, which requires massive
communication between server and clients in each communication. Moreover,
attacking the central server can break the whole system's privacy. In this
paper, we study the decentralized FedAvg with momentum (DFedAvgM), which is
implemented on clients that are connected by an undirected graph. In DFedAvgM,
all clients perform stochastic gradient descent with momentum and communicate
with their neighbors only. To further reduce the communication cost, we also
consider the quantized DFedAvgM. We prove convergence of the (quantized)
DFedAvgM under trivial assumptions; the convergence rate can be improved when
the loss function satisfies the P{\L} property. Finally, we numerically verify
the efficacy of DFedAvgM.
