Revert "UserModel: keep the users while parted"

Works in case of manual disconnect, but not manual part. It looks
visually confusing and breaks the join/part menu item, for instance...

This reverts commit b07c9b28a825cb0edc85de76e55983bdbb7f9ecd.