A little cleaner in _remove_channel_user_on_quit()