Fixes for multi-tenant teardown