Only retry connect if errno == EAGAIN