We consider the problem where n clients transmit d-dimensional real-valued vectors using only d(1+o(1)) bits each, in a manner that allows a receiver to approximately reconstruct their mean. Such compression problems arise in federated and distributed learning, as well as in other domains . We provide novel mathematical results and derive corresponding new algorithms that outperform previous compression algorithms in accuracy and computational efficiency . We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.
We consider the problem where n clients transmit d-dimensional real-valued vectors using d(1+o(1)) bits each, in a manner that allows the receiver to approximately reconstruct their mean. Such compression problems naturally arise in distributed and federated learning . We provide novel mathematical results and derive computationally efficient algorithms that are more accurate than previous compression techniques . We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.
D
only
R
a
the
R
arise in federated and distributed learning, as well as in other domains
naturally arise in distributed and federated learning
R
corresponding new algorithms that outperform previous compression algorithms in accuracy and computational efficiency
computationally efficient algorithms that are more accurate than previous compression techniques