Horovod: fast and easy distributed deep learning in TensorFlow
Bandwidth Optimal All-reduce Algorithms for Clusters of Workstations