Before revision:  Our proposed designs in HyPar-Flow offer up to 3.1x speedup over sequential training for ResNet-110 and up to 1.6x speedup over Horovod-based data-parallel training for ResNet-1001;
After revision:  Our proposed designs in HyPar-Flow provides: 1) Up to 1.6x speedup over Horovod-based data-parallel training for ResNet-1001;