Properly clean up after jenkins run