Do teardown in create-environment

If executed with retry, create-environment will fail its sanity
checks.