Before revision:  We discuss various implementations of pipeline parallelism and propose a novel  schedule that can improve throughput by more than 10\% with comparable memory footprint compared to previously-proposed approaches.
After revision:  We survey techniques for pipeline parallelism and propose a novel schedule that can improve throughput by more than 10\% with comparable memory footprint compared to previously-proposed approaches.