Update pipeline functionality and introduce the idea of completed jobs etc