Measuring the Effects of Data Parallelism on Neural Network Training
Don't Use Large Mini-Batches, Use Local SGD