Handle lobby disconnects

Also, we don't need to explicitly pass the member count to the client.
They can just get that from the full list of members that we're passing.