Optimized Broadcast for Deep Learning Workloads on Dense-GPU InfiniBand Clusters: MPI or NCCL?
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters