Fix race condition accessing the client socket fd.