A computer-implemented method for communication efficient machine learning, the method comprising: obtaining, by a client computing device, global values for a set of parameters of a machine-learned model; training, by the client computing device, the machine-learned model based at least in part on a local dataset to obtain an update matrix that is descriptive of updated values for the set of parameters of the machine-learned model, wherein the update matrix is restricted to be a low-rank matrix, and wherein the local dataset is stored locally by the client computing device; and communicating, by the client computing device, information descriptive of the update matrix to a server computing device, wherein: training, by the client computing device, the machine-learned model based at least in part on the local dataset to obtain the update matrix comprises: defining, by the client computing device, the update matrix as a product of a first matrix and a second matrix, wherein the first matrix comprises fixed values and the second matrix comprises optimizable variables; and training, by the client computing device, machine-learned model based at least in part on the local dataset to obtain the second matrix; and communicating, by the client computing device, information descriptive of the update matrix to the server computing device comprises communicating, by the client computing device, information descriptive of the second matrix to the server computing device without sending the first matrix from the client computing device to the server computing device. obtaining, by a client computing device, global values for a set of parameters of a machine-learned model; training, by the client computing device, the machine-learned model based at least in part on a local dataset to obtain an update matrix that is descriptive of updated values for the set of parameters of the machine-learned model, wherein the update matrix is restricted to be a low-rank matrix, and wherein the local dataset is stored locally by the client computing device; and communicating, by the client computing device, information descriptive of the update matrix to a server computing device, wherein: training, by the client computing device, the machine-learned model based at least in part on the local dataset to obtain the update matrix comprises: defining, by the client computing device, the update matrix as a product of a first matrix and a second matrix, wherein the first matrix comprises fixed values and the second matrix comprises optimizable variables; and training, by the client computing device, machine-learned model based at least in part on the local dataset to obtain the second matrix; and communicating, by the client computing device, information descriptive of the update matrix to the server computing device comprises communicating, by the client computing device, information descriptive of the second matrix to the server computing device without sending the first matrix from the client computing device to the server computing device. training, by the client computing device, the machine-learned model based at least in part on the local dataset to obtain the update matrix comprises: defining, by the client computing device, the update matrix as a product of a first matrix and a second matrix, wherein the first matrix comprises fixed values and the second matrix comprises optimizable variables; and training, by the client computing device, machine-learned model based at least in part on the local dataset to obtain the second matrix; and defining, by the client computing device, the update matrix as a product of a first matrix and a second matrix, wherein the first matrix comprises fixed values and the second matrix comprises optimizable variables; and training, by the client computing device, machine-learned model based at least in part on the local dataset to obtain the second matrix; and communicating, by the client computing device, information descriptive of the update matrix to the server computing device comprises communicating, by the client computing device, information descriptive of the second matrix to the server computing device without sending the first matrix from the client computing device to the server computing device.