  In this paper, we develop two new algorithms, called, \textbf{FedDR} and
\textbf{asyncFedDR}, for solving a fundamental nonconvex optimization problem
in federated learning. Our algorithms rely on a novel combination between a
nonconvex Douglas-Rachford splitting method, randomized block-coordinate
strategies, and asynchronous implementation. Unlike recent methods in the
literature, e.g., FedSplit and FedPD, our algorithms update only a subset of
users at each communication round, and possibly in an asynchronous mode, making
them more practical. These new algorithms also achieve communication efficiency
and more importantly can handle statistical and system heterogeneity, which are
the two main challenges in federated learning. Our convergence analysis shows
that the new algorithms match the communication complexity lower bound up to a
constant factor under standard assumptions. Our numerical experiments
illustrate the advantages of the proposed methods compared to existing ones
using both synthetic and real datasets.
