Before revision:  In this work, we  develop a system for communication scheduling which realizes near-optimal overlap of communication and computation in graph-based models.
After revision:  In this work, we identify a shortcoming in systems with graph representation for computation, such as TensorFlow and PyTorch, that result in high variance in iteration time --- random order of received parameters across workers. We develop a system for communication scheduling which realizes near-optimal overlap of communication and computation in graph-based models.