Making sure that we always clean up the pipeline. Closes #1783