Repository: encode/httpx

Issue:
Considering non env-var 'no_proxy' support?
### Checklist

<!-- Please make sure you check all these items before submitting your feature request. -->

- [x] There are no similar issues or pull requests for this yet.
- [ ] I discussed this idea on the [community chat](https://gitter.im/encode/community) and feedback is positive.

### Is your feature related to a problem? Please describe.

Hey all,

So while looking into an issues with `no_proxy` in Requests (https://github.com/psf/requests/issues/5000) I tried one of the failing cases with HTTPX, and ended up realizing that we don't take into account `no_proxy` in the `proxies={...}` usage. We only account for the `no_proxy`/`NO_PROXY` environment variable.

Eg this will fail as the request will be routed through the proxy:

```python
import httpx

proxies = {'https': 'http://no.where.com', 'no_proxy': 'google.com'}
client = httpx.Client(proxies=proxies)
client.get('https://google.com')  # ConnectError
```

However, this use case is covered by Requests (although with rough edges - see https://github.com/psf/requests/issues/5000).

```python
import os
import requests

os.environ['HTTPS_PROXY']='https://b.r.o.k.e.n.com'
proxies = {'no_proxy': 'google.com'}
r = requests.get('https://google.com', proxies=proxies)
```


## Describe the solution you would like.

The Requests way of passing a `no_proxy` key in the dict is somewhat clunky tbh, and also not compatible with us allowing users to pass an `httpx.Proxy` config object, or an URL, instead of a dict.

So I'd be thinking about a dedicated `no_proxy` option on clients, which accepts a list of domains or IPs (similar to how the env variable accepts a comma-separated string):

```python
httpx.Client(proxies=..., no_proxy=['<domain>', '<ip>', ...])
```

In terms of implementation, we could easily store this as an extra attribute, and then pass it when checking for "should we use proxies for this URL", without interfering with the proxy map building logic:

https://github.com/encode/httpx/blob/620b0670db07ae844aa6fae40d9209a856440c9d/httpx/_client.py#L555-L556

The priority order compared to env vars would be:

1. `no_proxy=[...]`
2. Environment variables (`no_proxy`/`NO_PROXY`)

Happy to discuss use cases and API details further!

## Describe alternatives you considered

<!-- Please describe any alternative solutions or features you've considered to solve
your problem and why they wouldn't solve it. -->

## Additional context

<!-- Provide any additional context, screenshots, tracebacks, etc. about the feature here. -->



Discussion:
So a consistent way to approach this would be to allow the values in the `proxies=...` dictionary to be `None`, so eg...

`proxies={'all': 'http://127.0.0.1:1234', 'all://google.com': None}`
Related to https://github.com/encode/httpx/pull/1059
Incidentally, I'm not super keen on `all://` since it *actually* means "http" or "https", but I think we need to treat any conversations around that separately.

Resolving patch:
diff --git a/httpx/_client.py b/httpx/_client.py
--- a/httpx/_client.py
+++ b/httpx/_client.py
@@ -86,7 +86,7 @@ def __init__(
 
     def _get_proxy_map(
         self, proxies: typing.Optional[ProxiesTypes], trust_env: bool,
-    ) -> typing.Dict[str, Proxy]:
+    ) -> typing.Dict[str, typing.Optional[Proxy]]:
         if proxies is None:
             if trust_env:
                 return {
@@ -94,15 +94,15 @@ def _get_proxy_map(
                     for key, url in get_environment_proxies().items()
                 }
             return {}
-        elif isinstance(proxies, (str, URL, Proxy)):
-            proxy = Proxy(url=proxies) if isinstance(proxies, (str, URL)) else proxies
-            return {"all": proxy}
-        else:
+        if isinstance(proxies, dict):
             new_proxies = {}
             for key, value in proxies.items():
                 proxy = Proxy(url=value) if isinstance(value, (str, URL)) else value
                 new_proxies[str(key)] = proxy
             return new_proxies
+        else:
+            proxy = Proxy(url=proxies) if isinstance(proxies, (str, URL)) else proxies
+            return {"all": proxy}
 
     @property
     def headers(self) -> Headers:
@@ -472,8 +472,12 @@ def __init__(
             app=app,
             trust_env=trust_env,
         )
-        self._proxies: typing.Dict[URLMatcher, httpcore.SyncHTTPTransport] = {
-            URLMatcher(key): self._init_proxy_transport(
+        self._proxies: typing.Dict[
+            URLMatcher, typing.Optional[httpcore.SyncHTTPTransport]
+        ] = {
+            URLMatcher(key): None
+            if proxy is None
+            else self._init_proxy_transport(
                 proxy,
                 verify=verify,
                 cert=cert,
@@ -543,7 +547,7 @@ def _transport_for_url(self, url: URL) -> httpcore.SyncHTTPTransport:
         if self._proxies and not should_not_be_proxied(url):
             for matcher, transport in self._proxies.items():
                 if matcher.matches(url):
-                    return transport
+                    return self._transport if transport is None else transport
 
         return self._transport
 
@@ -885,7 +889,8 @@ def delete(
     def close(self) -> None:
         self._transport.close()
         for proxy in self._proxies.values():
-            proxy.close()
+            if proxy is not None:
+                proxy.close()
 
     def __enter__(self) -> "Client":
         return self
@@ -989,8 +994,12 @@ def __init__(
             app=app,
             trust_env=trust_env,
         )
-        self._proxies: typing.Dict[URLMatcher, httpcore.AsyncHTTPTransport] = {
-            URLMatcher(key): self._init_proxy_transport(
+        self._proxies: typing.Dict[
+            URLMatcher, typing.Optional[httpcore.AsyncHTTPTransport]
+        ] = {
+            URLMatcher(key): None
+            if proxy is None
+            else self._init_proxy_transport(
                 proxy,
                 verify=verify,
                 cert=cert,
@@ -1060,7 +1069,7 @@ def _transport_for_url(self, url: URL) -> httpcore.AsyncHTTPTransport:
         if self._proxies and not should_not_be_proxied(url):
             for matcher, transport in self._proxies.items():
                 if matcher.matches(url):
-                    return transport
+                    return self._transport if transport is None else transport
 
         return self._transport
 
@@ -1402,7 +1411,8 @@ async def delete(
     async def aclose(self) -> None:
         await self._transport.aclose()
         for proxy in self._proxies.values():
-            await proxy.aclose()
+            if proxy is not None:
+                await proxy.aclose()
 
     async def __aenter__(self) -> "AsyncClient":
         return self
diff --git a/httpx/_types.py b/httpx/_types.py
--- a/httpx/_types.py
+++ b/httpx/_types.py
@@ -54,7 +54,7 @@
     Tuple[Optional[float], Optional[float], Optional[float], Optional[float]],
     "Timeout",
 ]
-ProxiesTypes = Union[URLTypes, "Proxy", Dict[URLTypes, Union[URLTypes, "Proxy"]]]
+ProxiesTypes = Union[URLTypes, "Proxy", Dict[URLTypes, Union[None, URLTypes, "Proxy"]]]
 
 AuthTypes = Union[
     Tuple[Union[str, bytes], Union[str, bytes]],


Test patch:
diff --git a/tests/client/test_proxies.py b/tests/client/test_proxies.py
--- a/tests/client/test_proxies.py
+++ b/tests/client/test_proxies.py
@@ -58,6 +58,7 @@ def test_proxies_parameter(proxies, expected_proxies):
         ("http://example.com", {"http://example.net": PROXY_URL}, None),
         ("http://example.com:443", {"http://example.com": PROXY_URL}, PROXY_URL),
         ("http://example.com", {"all": PROXY_URL}, PROXY_URL),
+        ("http://example.com", {"all": PROXY_URL, "http://example.com": None}, None),
         ("http://example.com", {"http": PROXY_URL}, PROXY_URL),
         ("http://example.com", {"all://example.com": PROXY_URL}, PROXY_URL),
         ("http://example.com", {"all://example.com:80": PROXY_URL}, None),