Remove players who can see a FrozenSand if they disconnect