Before revision:  In this paper, we propose HyPar-Flow : a framework for scalable and user-transparent parallel training of very large DNNs (up to 5, 000 layers).
After revision:  In this paper, we propose HyPar-Flow --- a model-size/-type agnostic, scalable, practical, and user-transparent parallel training of very large DNNs (up to 5, 000 layers).