Remove players from the lobby scoreboard when they leave.