remove by id, checking against the set of cloud user ids we just verified are in the group in LDAP