Stop trying to accept connections when socket is closed >_>