Measuring the Effects of Data Parallelism on Neural Network Training
Revisiting Distributed Synchronous SGD