Measuring the Effects of Data Parallelism on Neural Network Training
Deep Learning Scaling is Predictable, Empirically