Don't remove layer coordinate transform info if layer is no longer present (layer could be switched off but still present)