Finish listen thread in client end