Before revision:  While data-parallelism has been extensively studied and developed,  model-parallelism has received less attention as it is non-trivial to split the model across processes .
After revision:  While data-parallelism has been extensively studied and developed, model-parallelism and hybrid-parallelism efficiently. Four major problems we focus on are: 1) defining a notion of a distributed model across processes .