Make deploy suite clean up after itself