More explicitly ensure sockets aren't removed twice

If the socket closes while we are waiting on a checkAuthToken, that'll fire
twice.