Fixes a bug where leaving the last group a user was a member of would throw an
exception and not remove the group in ldap