Repository: encode/httpx (Python)

Issue:
On `proxies={"http": ...}` vs. `proxies={"http://": ...}`
Currently we're in line with `requests` in supporting "all", "http", and "https" as valid proxy keys.

Internally we handle those as URL patterns that we can match against, and eg. `"http"` is *actually* mapped to `"http://"`.

I'm wondering if we might have better internal consistency by...

* Raising a warning on base usage of "all", "http", "https", later escalating this to an error.
* Instead requiring an explicit "all://", "http://", "https://", all of which properly parse directly as URLs.

My feeling is that doing so would create a nicer consistency, and also has a subtle indication to developers about what is actually going on here.

For example...

`proxies={"http://": httpx.Proxies(...)}` neatly implies that `proxies={"http://www.example.com": httpx.Proxies(...)}` might also be a valid thing to do, which indeed it is.

Allowing `proxies={"http": httpx.Proxies(...)}` makes it less obvious that the key is a general purpose URL pattern.

~~Additionally this change means that other usages will end up with more expected behaviors, eg. `proxies={"www.example.com": ...}` will do what you'd expect, rather than mapping to `proxies={"www.example.com://": ...}` which is what we currently do.~~ - Not true, that'd parse as scheme='', host='', path='www.example.com'. Instead we'd probably want to be strict and ensure that only absolute URLs are allowed as pattern keys. (While still including a graceful deprecation for plain "http", "https", "all")

This becomes doubly relevant once we introduce a mount API, which will use the same URL pattern style for it's API, as current `proxies=...` argument.



Discussion:
Hello @tomchristie ,

Are you talking about the changes in `httpx._client.BaseClient._get_proxy_map` (here: [link](https://github.com/encode/httpx/blob/master/httpx/_client.py#L101))
Also, what should we do with `get_environment_proxies`? ([link](https://github.com/encode/httpx/blob/master/httpx/_utils.py#L302)) This function requires envs like `HTTPS_PROXY` etc.

UPD: @tomchristie I've created a PR just to show my idea. If I'm wrong, feel free to discard it



Patch:
diff --git a/docs/advanced.md b/docs/advanced.md
index 51544e5547..cf9ab94bc3 100644
--- a/docs/advanced.md
+++ b/docs/advanced.md
@@ -284,8 +284,8 @@ For more advanced use cases, pass a proxies `dict`. For example, to route HTTP a
 
 ```python
 proxies = {
-    "http": "http://localhost:8030",
-    "https": "http://localhost:8031",
+    "http://": "http://localhost:8030",
+    "https://": "http://localhost:8031",
 }
 
 with httpx.Client(proxies=proxies) as client:
@@ -295,7 +295,7 @@ with httpx.Client(proxies=proxies) as client:
 For detailed information about proxy routing, see the [Routing](#routing) section.
 
 !!! tip "Gotcha"
-    In most cases, the proxy URL for the `https` key _should_ use the `http://` scheme (that's not a typo!).
+    In most cases, the proxy URL for the `https://` key _should_ use the `http://` scheme (that's not a typo!).
 
     This is because HTTP proxying requires initiating a connection with the proxy server. While it's possible that your proxy supports doing it via HTTPS, most proxies only support doing it via HTTP.
 
@@ -307,7 +307,7 @@ Proxy credentials can be passed as the `userinfo` section of the proxy URL. For
 
 ```python
 proxies = {
-    "http": "http://username:password@localhost:8030",
+    "http://": "http://username:password@localhost:8030",
     # ...
 }
 ```
@@ -316,7 +316,7 @@ proxies = {
 
 HTTPX provides fine-grained controls for deciding which requests should go through a proxy, and which shouldn't. This process is known as proxy routing.
 
-The `proxies` dictionary maps URL patterns ("proxy keys") to proxy URLs. HTTPX matches requested URLs against proxy keys to decide which proxy should be used, if any. Matching is done from most specific proxy keys (e.g. `https://<domain>:<port>`) to least specific ones (e.g. `https`).
+The `proxies` dictionary maps URL patterns ("proxy keys") to proxy URLs. HTTPX matches requested URLs against proxy keys to decide which proxy should be used, if any. Matching is done from most specific proxy keys (e.g. `https://<domain>:<port>`) to least specific ones (e.g. `https://`).
 
 HTTPX supports routing proxies based on **scheme**, **domain**, **port**, or a combination of these.
 
@@ -326,7 +326,7 @@ Route everything through a proxy...
 
 ```python
 proxies = {
-    "all": "http://localhost:8030",
+    "all://": "http://localhost:8030",
 }
 ```
 
@@ -336,8 +336,8 @@ Route HTTP requests through one proxy, and HTTPS requests through another...
 
 ```python
 proxies = {
-    "http": "http://localhost:8030",
-    "https": "http://localhost:8031",
+    "http://": "http://localhost:8030",
+    "https://": "http://localhost:8031",
 }
 ```
 
@@ -402,7 +402,7 @@ To do so, pass `None` as the proxy URL. For example...
 ```python
 proxies = {
     # Route requests through a proxy by default...
-    "all": "http://localhost:8031",
+    "all://": "http://localhost:8031",
     # Except those for "example.com".
     "all://example.com": None,
 }
@@ -415,7 +415,7 @@ You can combine the routing features outlined above to build complex proxy routi
 ```python
 proxies = {
     # Route all traffic through a proxy by default...
-    "all": "http://localhost:8030",
+    "all://": "http://localhost:8030",
     # But don't use proxies for HTTPS requests to "domain.io"...
     "https://domain.io": None,
     # And use another proxy for requests to "example.com" and its subdomains...
diff --git a/httpx/_utils.py b/httpx/_utils.py
index b09aebcfd7..67f1c9105f 100644
--- a/httpx/_utils.py
+++ b/httpx/_utils.py
@@ -478,6 +478,11 @@ def __init__(self, pattern: str) -> None:
         from ._models import URL
 
         if pattern and ":" not in pattern:
+            warn_deprecated(
+                f"Proxy keys should use proper URL forms rather "
+                f"than plain scheme strings. "
+                f'Instead of "{pattern}", use "{pattern}://"'
+            )
             pattern += "://"
 
         url = URL(pattern)


Test patch:
diff --git a/tests/client/test_proxies.py b/tests/client/test_proxies.py
index 3f014baa96..45ff01743c 100644
--- a/tests/client/test_proxies.py
+++ b/tests/client/test_proxies.py
@@ -20,16 +20,19 @@ def url_to_origin(url: str):
 @pytest.mark.parametrize(
     ["proxies", "expected_proxies"],
     [
-        ("http://127.0.0.1", [("all", "http://127.0.0.1")]),
-        ({"all": "http://127.0.0.1"}, [("all", "http://127.0.0.1")]),
+        ("http://127.0.0.1", [("all://", "http://127.0.0.1")]),
+        ({"all://": "http://127.0.0.1"}, [("all://", "http://127.0.0.1")]),
         (
-            {"http": "http://127.0.0.1", "https": "https://127.0.0.1"},
-            [("http", "http://127.0.0.1"), ("https", "https://127.0.0.1")],
+            {"http://": "http://127.0.0.1", "https://": "https://127.0.0.1"},
+            [("http://", "http://127.0.0.1"), ("https://", "https://127.0.0.1")],
         ),
-        (httpx.Proxy("http://127.0.0.1"), [("all", "http://127.0.0.1")]),
+        (httpx.Proxy("http://127.0.0.1"), [("all://", "http://127.0.0.1")]),
         (
-            {"https": httpx.Proxy("https://127.0.0.1"), "all": "http://127.0.0.1"},
-            [("all", "http://127.0.0.1"), ("https", "https://127.0.0.1")],
+            {
+                "https://": httpx.Proxy("https://127.0.0.1"),
+                "all://": "http://127.0.0.1",
+            },
+            [("all://", "http://127.0.0.1"), ("https://", "https://127.0.0.1")],
         ),
     ],
 )
@@ -54,7 +57,7 @@ def test_proxies_parameter(proxies, expected_proxies):
     [
         ("http://example.com", None, None),
         ("http://example.com", {}, None),
-        ("http://example.com", {"https": PROXY_URL}, None),
+        ("http://example.com", {"https://": PROXY_URL}, None),
         ("http://example.com", {"http://example.net": PROXY_URL}, None),
         # Using "*" should match any domain name.
         ("http://example.com", {"http://*": PROXY_URL}, PROXY_URL),
@@ -71,9 +74,9 @@ def test_proxies_parameter(proxies, expected_proxies):
         ("http://wwwexample.com", {"http://*example.com": PROXY_URL}, None),
         # ...
         ("http://example.com:443", {"http://example.com": PROXY_URL}, PROXY_URL),
-        ("http://example.com", {"all": PROXY_URL}, PROXY_URL),
-        ("http://example.com", {"all": PROXY_URL, "http://example.com": None}, None),
-        ("http://example.com", {"http": PROXY_URL}, PROXY_URL),
+        ("http://example.com", {"all://": PROXY_URL}, PROXY_URL),
+        ("http://example.com", {"all://": PROXY_URL, "http://example.com": None}, None),
+        ("http://example.com", {"http://": PROXY_URL}, PROXY_URL),
         ("http://example.com", {"all://example.com": PROXY_URL}, PROXY_URL),
         ("http://example.com", {"all://example.com:80": PROXY_URL}, None),
         ("http://example.com", {"http://example.com": PROXY_URL}, PROXY_URL),
@@ -83,8 +86,8 @@ def test_proxies_parameter(proxies, expected_proxies):
         (
             "http://example.com",
             {
-                "all": PROXY_URL + ":1",
-                "http": PROXY_URL + ":2",
+                "all://": PROXY_URL + ":1",
+                "http://": PROXY_URL + ":2",
                 "all://example.com": PROXY_URL + ":3",
                 "http://example.com": PROXY_URL + ":4",
             },
@@ -93,15 +96,15 @@ def test_proxies_parameter(proxies, expected_proxies):
         (
             "http://example.com",
             {
-                "all": PROXY_URL + ":1",
-                "http": PROXY_URL + ":2",
+                "all://": PROXY_URL + ":1",
+                "http://": PROXY_URL + ":2",
                 "all://example.com": PROXY_URL + ":3",
             },
             PROXY_URL + ":3",
         ),
         (
             "http://example.com",
-            {"all": PROXY_URL + ":1", "http": PROXY_URL + ":2"},
+            {"all://": PROXY_URL + ":1", "http://": PROXY_URL + ":2"},
             PROXY_URL + ":2",
         ),
     ],
@@ -120,12 +123,12 @@ def test_transport_for_request(url, proxies, expected):
 
 @pytest.mark.asyncio
 async def test_async_proxy_close():
-    client = httpx.AsyncClient(proxies={"all": PROXY_URL})
+    client = httpx.AsyncClient(proxies={"all://": PROXY_URL})
     await client.aclose()
 
 
 def test_sync_proxy_close():
-    client = httpx.Client(proxies={"all": PROXY_URL})
+    client = httpx.Client(proxies={"all://": PROXY_URL})
     client.close()
 
 
@@ -244,3 +247,20 @@ def test_proxies_environ(monkeypatch, client_class, url, env, expected):
         assert transport == client._transport
     else:
         assert transport.proxy_origin == url_to_origin(expected)
+
+
+@pytest.mark.parametrize(
+    ["proxies", "expected_scheme"],
+    [
+        ({"http": "http://127.0.0.1"}, "http://"),
+        ({"https": "http://127.0.0.1"}, "https://"),
+        ({"all": "http://127.0.0.1"}, "all://"),
+    ],
+)
+def test_for_deprecated_proxy_params(proxies, expected_scheme):
+    with pytest.deprecated_call() as block:
+        httpx.AsyncClient(proxies=proxies)
+
+    warning_message = str(block.pop(DeprecationWarning))
+
+    assert expected_scheme in warning_message