Fixed a buffer overflow on Linux due to the double removal of a client socket.

The SFML Socket selector doesn't handle removing the same socket twice on that platform.

Fixes #38