Disregard server list

If the user changes, we remove and readd them regardless. This fixes
the behavior of sorting changes in user list not applying when not
within the active window.