Repository: encode/httpx

Issue:
base_url strips relative path
Hi,
I've noticed that if I do something like:
```
http_client = https.AsyncClient(base_url="https://my-service/api/v1")
```
for further requests (e.g. `/messages`) `/api/v1` will be stripped.
Is it expected behavior?
Tested on httpx 0.11.1
Thanks!


Discussion:
The docs are missing a specific reference to that, but you need to make sure any subpaths you're using don't have a leading slash, else they will be interpreted as absolute paths. There's a PR to update the docs for that: https://github.com/encode/httpx/pull/843

Personally I generally use `path.lstrip("/")` before I do `http_client.verb(path, **kwargs)` in my calls.
I’d personally like HTTPX to be smart enough to not strip any prefix path too, _and_ allow using a leading `/` on subsequent requests. Mostly because that’s the UX that test clients in web frameworks have been acustoming folks to. I’d consider the current behavior a bug, somewhat.
@florimondmanca Yeah I think that's a reasonable observation actually. It's not that our `.join(...)` behaviour is broken, but that we *shouldn't quite* be using URL joinging semantics for clients with a base URL.
But what of the case where someone _wants_ to clear the prefix? Should they create a new client, or perhaps pass an `absolute_path=True` arg to the `client.verb` call?
@StephenBrown2 Note that that sounds like a different use case/feature to me (that we don’t even support in the current state of things).
Just tested a client with a base_url will append without a leading slash, and replace with one, and was surprised that it was not what either of us were talking about:
```py
Python 3.8.1 (default, Jan 10 2020, 09:36:37)
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
>>> import httpx
>>> http_client = httpx.Client(base_url="https://example.com/api/v1")
>>> append = http_client.get("endpoint")
>>> append.url
URL('https://example.com/api/endpoint')
>>> replace = http_client.get("/api/v2/new")
>>> replace.url
URL('https://example.com/api/v2/new')
>>> httpx.__version__
'0.11.1'
```
And confirmed in iPython for AsyncClient:
```py
Python 3.8.1 (default, Jan 10 2020, 09:36:37)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import httpx
In [2]: async with httpx.AsyncClient(base_url="https://example.com/api/v1") as http_client:
   ...:     append = await http_client.get("endpoint")
   ...:     print(f"{append.url=}")
   ...:     replace = await http_client.get("/api/v2/new")
   ...:     print(f"{replace.url=}")
   ...: 
append.url=URL('https://example.com/api/endpoint')
replace.url=URL('https://example.com/api/v2/new')
```
where I expected `append.url==URL('https://example.com/api/v1/endpoint')`, `replace.url` was as expected. So, a bug? This is not the case if the `base_url` has a trailing slash.

```py
In [5]: http_client_ts = httpx.Client(base_url="https://example.com/api/v1/")
   ...: append = http_client_ts.get("endpoint")
   ...: append.url
   ...:
Out[5]: URL('https://example.com/api/v1/endpoint')
In [6]: replace = http_client_ts.get("/api/v2/new")
   ...: replace.url
   ...:
Out[6]: URL('https://example.com/api/v2/new')
```

Here's the result table:

|  | base=`/api/v1/` | base=`/api/v1` |
| --: | :-: | :-: |
| **path=`/endpoint`** | `/endpoint` | `/endpoint` |
| **path=`endpoint`** | `/api/v1/endpoint` | `/api/endpoint` |

So either we document this (which I think is the proper solution), or "fix" it in some way.
To be clear, the _only_ usage I'd be expecting from `base_url` is…

```python
>>> base_url = "https://example.com/api/v1"  # Trailing slash is irrelevant
>>> client = httpx.Client(base_url=base_url)
>>> response = client.get("/endpoint")  # Leading slash is *always* required
>>> response.url
URL("https://example.com/api/v1/endpoint")
```

(Right now this is wrongly yielding

```python
URL("https://example.com/endpoint")
```

as you noted too.)

Formally: `request.url = client.base_url.rstrip("/") + url` — no need for any other smarts or advanced URL semantics.

From an UX perspective, this is the most sensible thing to do in my POV as it is an extension of the default case of using `base_url="https://example.com"` and eg `.get("/api/v1/endpoint")` (which we currently correctly implement).
Personally, now that I see the results, I think that "correct" trumps "surprising" in this case. There are three results currently:

- If you want to use a new base path, prepend the request path with a slash.
- If you want to append path segments to the `base_url`, be sure the base_url ends with a slash and the request path does not begin with one.
- If you want to replace the _last segment_ of the path with a new one, don't put a slash at the end or beginning of the base_url or request path.

The only case that is "surprising" to me at first glance is the last one, while the first two make intuitive sense to me and are what I would want. Though the last one makes sense after consideration, I currently work around it in the way I said above, with:

```py
client = httpx.Client(base_url="http://example.com/prefix/") # Use a trailing slash
client.verb(path.lstrip("/")) # Enforce no leading slash
```

This also works with no base_url prefix:
```py
path = "/endpoint"
client = httpx.Client(base_url="http://example.com") # Use no trailing slash
request = client.get(path.lstrip("/")) # Enforce no leading slash
request.url == URL('http://example.com/endpoint')
```
I'm just saying we shouldn't even have to _think_ about "what are the possible cases?". Set a `base_url`, it'll be prepended to all requested URL paths, with automatic handling of trailing slash madness. Wouldn't that be ideal? Also in line with what @dmfigol originally posted.
I'd expect to see just merge, no magic required:
* `base_url="https://my-service/api/v1"` and `.get("/messages")` will result in `https://my-service/api/v1/messages`
* `base_url="https://my-service/api/v1/"` and `.get("messages")` will result in `https://my-service/api/v1/messages`
* `base_url="https://my-service/api/v1/"` and `.get("https://my-other-service/api/v1/me")` will result in `https://my-other-service/api/v1/me`
So then to follow:
- `base_url="https://my-service/api/v1"` and `.get("v2/messages")` will result in `https://my-service/api/v1/v2/messages`
- `base_url="https://my-service/api/v1"` and `.get("/api/v2/messages")` will result in `https://my-service/api/v1/api/v2/messages`

requiring a `v1_client = httpx.Client(base_url=".../v1")` and `v2_client = httpx.Client(base_url=".../v2")` is probably an acceptable compromise.
@StephenBrown2 

- IMO cases when the requested URL doesn't start with either a scheme or `/` (as in your first example) should be forbidden. Or naively result in `https://my-service/api/v1v2/messages`.
- Agreed, if we need multiple `base_url` it's a sign we are talking to multiple services and so we should use multiple separate clients.
As long as those restrictions are documented with proper error messages (and the naive case prevented), I'm good with that.
Milestoning to review for 1.0, since there's a potential behavioural change for us to introduce here.

Resolving patch:
diff --git a/httpx/_client.py b/httpx/_client.py
--- a/httpx/_client.py
+++ b/httpx/_client.py
@@ -66,13 +66,10 @@ def __init__(
         cookies: CookieTypes = None,
         timeout: TimeoutTypes = DEFAULT_TIMEOUT_CONFIG,
         max_redirects: int = DEFAULT_MAX_REDIRECTS,
-        base_url: URLTypes = None,
+        base_url: URLTypes = "",
         trust_env: bool = True,
     ):
-        if base_url is None:
-            self.base_url = URL("")
-        else:
-            self.base_url = URL(base_url)
+        self._base_url = self._enforce_trailing_slash(URL(base_url))
 
         self.auth = auth
         self._params = QueryParams(params)
@@ -87,6 +84,11 @@ def __init__(
     def trust_env(self) -> bool:
         return self._trust_env
 
+    def _enforce_trailing_slash(self, url: URL) -> URL:
+        if url.path.endswith("/"):
+            return url
+        return url.copy_with(path=url.path + "/")
+
     def _get_proxy_map(
         self, proxies: typing.Optional[ProxiesTypes], allow_env_proxies: bool,
     ) -> typing.Dict[str, typing.Optional[Proxy]]:
@@ -107,6 +109,17 @@ def _get_proxy_map(
             proxy = Proxy(url=proxies) if isinstance(proxies, (str, URL)) else proxies
             return {"all": proxy}
 
+    @property
+    def base_url(self) -> URL:
+        """
+        Base URL to use when sending requests with relative URLs.
+        """
+        return self._base_url
+
+    @base_url.setter
+    def base_url(self, url: URLTypes) -> None:
+        self._base_url = self._enforce_trailing_slash(URL(url))
+
     @property
     def headers(self) -> Headers:
         """
@@ -208,7 +221,13 @@ def _merge_url(self, url: URLTypes) -> URL:
         Merge a URL argument together with any 'base_url' on the client,
         to create the URL used for the outgoing request.
         """
-        return self.base_url.join(url)
+        merge_url = URL(url)
+        if merge_url.is_relative_url:
+            # We always ensure the base_url paths include the trailing '/',
+            # and always strip any leading '/' from the merge URL.
+            merge_url = merge_url.copy_with(path=merge_url.path.lstrip("/"))
+            return self.base_url.join(merge_url)
+        return merge_url
 
     def _merge_cookies(
         self, cookies: CookieTypes = None
@@ -441,7 +460,7 @@ def __init__(
         limits: Limits = DEFAULT_LIMITS,
         pool_limits: Limits = None,
         max_redirects: int = DEFAULT_MAX_REDIRECTS,
-        base_url: URLTypes = None,
+        base_url: URLTypes = "",
         transport: httpcore.SyncHTTPTransport = None,
         app: typing.Callable = None,
         trust_env: bool = True,
@@ -972,7 +991,7 @@ def __init__(
         limits: Limits = DEFAULT_LIMITS,
         pool_limits: Limits = None,
         max_redirects: int = DEFAULT_MAX_REDIRECTS,
-        base_url: URLTypes = None,
+        base_url: URLTypes = "",
         transport: httpcore.AsyncHTTPTransport = None,
         app: typing.Callable = None,
         trust_env: bool = True,
diff --git a/httpx/_models.py b/httpx/_models.py
--- a/httpx/_models.py
+++ b/httpx/_models.py
@@ -55,7 +55,7 @@
 
 
 class URL:
-    def __init__(self, url: URLTypes, params: QueryParamTypes = None) -> None:
+    def __init__(self, url: URLTypes = "", params: QueryParamTypes = None) -> None:
         if isinstance(url, str):
             self._uri_reference = rfc3986.api.iri_reference(url).encode()
         else:


Test patch:
diff --git a/tests/client/test_client.py b/tests/client/test_client.py
--- a/tests/client/test_client.py
+++ b/tests/client/test_client.py
@@ -174,13 +174,31 @@ def test_base_url(server):
     assert response.url == base_url
 
 
-def test_merge_url():
+def test_merge_absolute_url():
     client = httpx.Client(base_url="https://www.example.com/")
-    request = client.build_request("GET", "http://www.example.com")
-    assert request.url.scheme == "http"
+    request = client.build_request("GET", "http://www.example.com/")
+    assert request.url == httpx.URL("http://www.example.com/")
     assert not request.url.is_ssl
 
 
+def test_merge_relative_url():
+    client = httpx.Client(base_url="https://www.example.com/")
+    request = client.build_request("GET", "/testing/123")
+    assert request.url == httpx.URL("https://www.example.com/testing/123")
+
+
+def test_merge_relative_url_with_path():
+    client = httpx.Client(base_url="https://www.example.com/some/path")
+    request = client.build_request("GET", "/testing/123")
+    assert request.url == httpx.URL("https://www.example.com/some/path/testing/123")
+
+
+def test_merge_relative_url_with_dotted_path():
+    client = httpx.Client(base_url="https://www.example.com/some/path")
+    request = client.build_request("GET", "../testing/123")
+    assert request.url == httpx.URL("https://www.example.com/some/testing/123")
+
+
 def test_pool_limits_deprecated():
     limits = httpx.Limits()
 
diff --git a/tests/client/test_properties.py b/tests/client/test_properties.py
--- a/tests/client/test_properties.py
+++ b/tests/client/test_properties.py
@@ -1,4 +1,25 @@
-from httpx import AsyncClient, Cookies, Headers
+from httpx import URL, AsyncClient, Cookies, Headers
+
+
+def test_client_base_url():
+    client = AsyncClient()
+    client.base_url = "https://www.example.org/"  # type: ignore
+    assert isinstance(client.base_url, URL)
+    assert client.base_url == URL("https://www.example.org/")
+
+
+def test_client_base_url_without_trailing_slash():
+    client = AsyncClient()
+    client.base_url = "https://www.example.org/path"  # type: ignore
+    assert isinstance(client.base_url, URL)
+    assert client.base_url == URL("https://www.example.org/path/")
+
+
+def test_client_base_url_with_trailing_slash():
+    client = AsyncClient()
+    client.base_url = "https://www.example.org/path/"  # type: ignore
+    assert isinstance(client.base_url, URL)
+    assert client.base_url == URL("https://www.example.org/path/")
 
 
 def test_client_headers():