Repository: urllib3/urllib3

Issue:
DNS lookup error exception
I am using `requests` to check domain status for `k8s.io` domains (https://github.com/kubernetes/k8s.io/pull/30) and I can't find a good way to check for DNS lookup error like `ERR_NAME_NOT_RESOLVED` in Chrome. Exceptions output seems platform specific:

```
    Traceback (most recent call last):
      File "check.py", line 11, in <module>
        status = requests.head('http://' + site)
      File "C:\Python27\lib\site-packages\requests\api.py", line 93, in head
        return request('head', url, **kwargs)
      File "C:\Python27\lib\site-packages\requests\api.py", line 53, in request
        return session.request(method=method, url=url, **kwargs)
      File "C:\Python27\lib\site-packages\requests\sessions.py", line 468, in request
        resp = self.send(prep, **send_kwargs)
      File "C:\Python27\lib\site-packages\requests\sessions.py", line 576, in send
        r = adapter.send(request, **kwargs)
      File "C:\Python27\lib\site-packages\requests\adapters.py", line 437, in send
        raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='jenkins.k8s.io', port=80):
        Max retries exceeded with url: / (Caused by
        NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at
        0x029E43B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
```

So I was redirected here from https://github.com/kennethreitz/requests/issues/3630#issuecomment-255031285 to see if `urllib3` could expose more fine-grained DNSLookupError exception and maybe remove MaxRetryError wrapper.
## 



Discussion:
Ok, so.

We're certainly not going to remove the `MaxRetryError`: that's not a winner. But we can probably be more clear about the specific error by hoisting the `getaddrinfo` call on line 75 of `urllib3/util/connection.py` out of the `for` statement and shoving it in a `try...except` that raises an appropriate urllib3 exception.

That would at least give you a specific exception to try to haul out. `MaxRetryError` has a `reason` attribute which you can use to fetch the underlying exception, so you can from that point take the exception.

Note, however, that this change will take a while to propagate into a released version of Requests. In the interim, you may find it more helpful to check for DNS errors by simply calling `socket.getaddrinfo` yourself before making the request.

Too late. =) I've already done exception handling through string parsing - https://github.com/kennethreitz/requests/issues/3630#issuecomment-255100951

So how about reusing Chrome's `ERR_NAME_NOT_RESOLVED` constant which seems to have cross-platform value -105? https://cs.chromium.org/chromium/src/net/base/net_error_list.h?sq=package:chromium&dr=C&rcl=1477077889&l=130

Or it could be `NXDOMAIN` error code 3 - https://support.opendns.com/hc/en-us/articles/227986827-FAQ-What-are-common-DNS-return-or-response-codes-

I don't know that we gain anything from using a constant: it's just not that Pythonic an approach. Right now we're propagating the error code returned from getaddrinfo, which is the bare minimum of effort. If we're going to put in more effort we should do it Pythonically, which means instead of using an error code we should use a different exception type.

So, is `DNSLookupError` a good name for this extension?

Maybe, or perhaps `NameResolutionError`. Either would be reasonable.

The PR is closed automatically, because I am MIA, but if anyone wants to continue fixing this issue, the code is still here - https://github.com/urllib3/urllib3/pull/1008

I came here, because I got another upvote on Stack Overflow.
A few comments on this feature as implemented by @euri10:

---

IIUC, previously if DNS resolution failed, `NewConnectionError` (subclass of `ConnectTimeoutError`, `TimeoutError`, `OSError`, `HTTPError`) was raised, now `NameResolutionError` (subclass of `socket.gaierror`, `OSError`, `HTTPError`) is raised.

My main concern here is that users who previously handles `NameResolutionError` for DNS errors would now need to change/add `NameResolutionError`. I think that's OK and makes sense, but should be documented as a breaking change.

Another concern is is how the `NameResolutionError` is classified - for example it is possible the DNS failed due to a timeout but it would no longer be caught by `TimeoutError` (correct me if I'm wrong).

This can get quite complicated because DNS resolution is its own tiny request-response transaction, with connect and read phases and timeouts. It's hard to say how the exception hierarchy should be laid out.

---

In `HTTPConnectionPool.urlopen`

https://github.com/urllib3/urllib3/blob/1831327b881880ed871f96f56a6977d360042e1b/src/urllib3/connectionpool.py#L749-L752

I wonder what happens when using e.g. `socks5h://` and the proxy is down. For most requests DNS resolution is the first thing that happens. Will this go to the `ProxyError` branch of the `NameResolutionError` branch?

---

Regarding `Retry` and PR #2312, how are DNS errors classified - `connect`, `read` or `other`? I think it's `connect` now (and previously), but want to make sure.

It also might be worth thinking about adding a separate limit for `resolve`, seems like a useful thing to have control over apart from connection to the actual resolved host.
I'll humbly pass on responding properly since I'm too new to have a good idea on this, but all seem to be very valid points to me
@bluetech Your thoughful comments are deeply appreciated, thank you! I answered point by point, but the main takeaway is that we just have two things to do to close this issue:

 * Change the subclass from `gaierror` to `NewConnectionError`
 * Add retry (started #2312)

> IIUC, previously if DNS resolution failed, `NewConnectionError` (subclass of `ConnectTimeoutError`, `TimeoutError`, `OSError`, `HTTPError`) was raised, now `NameResolutionError` (subclass of `socket.gaierror`, `OSError`, `HTTPError`) is raised.

You're right. I thought `gaierror` was raised, but it's not the case. I think it would be better to make `NameResolutionError` a subclass of `NewConnectionError`.

> Another concern is is how the `NameResolutionError` is classified - for example it is possible the DNS failed due to a timeout but it would no longer be caught by `TimeoutError` (correct me if I'm wrong).

We were already raising `TimeoutError` separately, and that's still the case. This isn't affected by the change, and not a concern. (But I had to check, so thanks for raising this!)

> I wonder what happens when using e.g. `socks5h://` and the proxy is down. For most requests DNS resolution is the first thing that happens. Will this go to the `ProxyError` branch of the `NameResolutionError` branch?

I assume you meant `or` instead of `of`. If we change the parent class to `NewConnectionError`, this will go to the `ProxyError` branch again, as it should. Nice catch!

> Regarding `Retry` and PR #2312, how are DNS errors classified - `connect`, `read` or `other`? I think it's `connect` now (and previously), but want to make sure.
> 
> It also might be worth thinking about adding a separate limit for `resolve`, seems like a useful thing to have control over apart from connection to the actual resolved host.

Right, it's `connect`. I guess `resolve` could make sense, but I'd rather wait until someone has an actual use case.
Actually, if we change the parent class to `NewConnectionError`, that will fix the retry issue! So that's the only needed change.
> Actually, if we change the parent class to `NewConnectionError`, that will fix the retry issue! So that's the only needed change.

do you want it done as a seperate PR @pquentin or part of https://github.com/urllib3/urllib3/pull/2312 is fine for you ?

Resolving patch:
diff --git a/src/urllib3/connection.py b/src/urllib3/connection.py
--- a/src/urllib3/connection.py
+++ b/src/urllib3/connection.py
@@ -207,7 +207,7 @@ def _new_conn(self) -> socket.socket:
                 socket_options=self.socket_options,
             )
         except socket.gaierror as e:
-            raise NameResolutionError(self.host, e)
+            raise NameResolutionError(self.host, self, e)
         except SocketTimeout:
             raise ConnectTimeoutError(
                 self,
diff --git a/src/urllib3/connectionpool.py b/src/urllib3/connectionpool.py
--- a/src/urllib3/connectionpool.py
+++ b/src/urllib3/connectionpool.py
@@ -30,7 +30,6 @@
     InsecureRequestWarning,
     LocationValueError,
     MaxRetryError,
-    NameResolutionError,
     NewConnectionError,
     ProtocolError,
     ProxyError,
@@ -739,7 +738,6 @@ def urlopen(  # type: ignore
             SSLError,
             CertificateError,
             HTTPSProxyError,
-            NameResolutionError,
         ) as e:
             # Discard the connection for these exceptions. It will be
             # replaced during the next _get_conn() call.
@@ -747,8 +745,6 @@ def urlopen(  # type: ignore
             new_e: Exception = e
             if isinstance(e, (BaseSSLError, CertificateError)):
                 new_e = SSLError(e)
-            elif isinstance(e, NameResolutionError):
-                pass
             elif isinstance(e, (OSError, NewConnectionError)) and self.proxy:
                 new_e = ProxyError("Cannot connect to proxy.", e)
             elif isinstance(e, (OSError, HTTPException)):
diff --git a/src/urllib3/exceptions.py b/src/urllib3/exceptions.py
--- a/src/urllib3/exceptions.py
+++ b/src/urllib3/exceptions.py
@@ -184,6 +184,14 @@ def pool(self) -> "HTTPConnection":
         return self.conn
 
 
+class NameResolutionError(NewConnectionError):
+    """Raised when host name resolution fails."""
+
+    def __init__(self, host: str, conn: "HTTPConnection", reason: socket.gaierror):
+        message = f"Failed to resolve '{host}' ({reason})"
+        super().__init__(conn, message)
+
+
 class EmptyPoolError(PoolError):
     """Raised when a pool runs out of connections and no more are allowed."""
 
@@ -379,11 +387,3 @@ class UnrewindableBodyError(HTTPError):
     """urllib3 encountered an error when trying to rewind a body"""
 
     pass
-
-
-class NameResolutionError(HTTPError, socket.gaierror):
-    """Raised when host name resolution fails."""
-
-    def __init__(self, host: str, reason: socket.gaierror):
-        message = f"Failed to resolve '{host}' ({reason})"
-        HTTPError.__init__(self, message)


Test patch:
diff --git a/test/with_dummyserver/test_proxy_poolmanager.py b/test/with_dummyserver/test_proxy_poolmanager.py
--- a/test/with_dummyserver/test_proxy_poolmanager.py
+++ b/test/with_dummyserver/test_proxy_poolmanager.py
@@ -16,7 +16,7 @@
     ConnectTimeoutError,
     HTTPSProxyError,
     MaxRetryError,
-    NameResolutionError,
+    ProxyError,
     ProxySchemeUnknown,
     ProxySchemeUnsupported,
     SSLError,
@@ -146,7 +146,7 @@ def test_proxy_conn_fail(self):
 
             with pytest.raises(MaxRetryError) as e:
                 http.request("GET", f"{self.http_url}/")
-            assert type(e.value.reason) == NameResolutionError
+            assert type(e.value.reason) == ProxyError
 
     def test_https_conn_failed(self):
         """