Interrupt the ConnectionListener thread too