Stop listening to the socket before disconnect