Pinger properly unreference socket on handshake