Fix handling of models during provisioning deletes (CO-953)