Remove players from hashset on quit