Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering
1-bit stochastic gradient descent and its application to data-parallel distributed training of speech DNNs.