Refine default group checking

We don't want to just ignore the default group -- we just don't want to
try to remove it if it's in the user's group list and not in the state's
group list