Mesh-TensorFlow: Deep Learning for Supercomputers
Bandwidth Optimal All-reduce Algorithms for Clusters of Workstations