Before revision:  In this work, we  develop a system for communication scheduling which realizes near-optimal overlap of communication and computation in graph-based models.
After revision:  In this work, we develop a system , TicTac, to improve the iteration time by fixing this issue in distributed deep learning with Parameter Servers while guaranteeing near-optimal overlap of communication and computation in graph-based models.