  We consider training models on private data that is distributed across user
devices. To ensure privacy, we add on-device noise and use secure aggregation
so that only the noisy sum is revealed to the server. We present a
comprehensive end-to-end system, which appropriately discretizes the data and
adds discrete Gaussian noise before performing secure aggregation. We provide a
novel privacy analysis for sums of discrete Gaussians. We also analyze the
effect of rounding the input data and the modular summation arithmetic. Our
theoretical guarantees highlight the complex tension between communication,
privacy, and accuracy. Our extensive experimental results demonstrate that our
solution is essentially able to achieve a comparable accuracy to central
differential privacy with 16 bits of precision per value.
