Remove de-registered users from the registration hash
Switch to unordered set since we never walk the list