Repository: aio-libs/yarl

Issue:
No way to replace path without replacing query
The following is unexpected behavior:

```
url1 = URL("http://www.example.com/foo?bar=10")
url2 = url1.with_path("/bar")

assert url1.query = url2.query  # Assertion fails
```

While I understand the design decision to treat the query and fragment components as part of the path, this has left us without an (efficient) way to preserve the query and fragment components when replacing just the path. The only way to do it is:

```
url2 = url1.with_path(new_path).with_query(url1.query_string).with_fragment(url1.fragment)
```

This is a total of 4 extra method calls, at least of which does a lot of unnecessary processing. I'm assuming that there's code out there that depends on this weird behavior, so I'd like to propose and, with permission, implement a new method, `with_just_path`, which replaces the path but leaves the fragment and query the same.

Discussion:
codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/aio-libs/yarl/branches/paul-nameless:with-path-keep-query)

### Merging #1421 will **not alter performance**

<sub>Comparing <code>paul-nameless:with-path-keep-query</code> (f85a084) with <code>master</code> (b31eaa1)</sub>



### Summary

`✅ 85` untouched benchmarks





codecov[bot]: ## [Codecov](https://app.codecov.io/gh/aio-libs/yarl/pull/1421?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 96.08%. Comparing base [(`b31eaa1`)](https://app.codecov.io/gh/aio-libs/yarl/commit/b31eaa17956cb756f9493a541d57c17420b4baed?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) to head [(`f85a084`)](https://app.codecov.io/gh/aio-libs/yarl/commit/f85a0845b37d7c0029fa4c2bff09aa1f4c67a93f?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1421      +/-   ##
==========================================
+ Coverage   96.06%   96.08%   +0.01%     
==========================================
  Files          31       31              
  Lines        5774     5795      +21     
  Branches      348      350       +2     
==========================================
+ Hits         5547     5568      +21     
  Misses        201      201              
  Partials       26       26              
```

| [Flag](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | Coverage Δ | |
|---|---|---|
| [CI-GHA](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `96.08% <100.00%> (+0.01%)` | :arrow_up: |
| [MyPy](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `48.67% <47.82%> (-0.03%)` | :arrow_down: |
| [OS-Linux](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.55% <100.00%> (+<0.01%)` | :arrow_up: |
| [OS-Windows](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.62% <100.00%> (+<0.01%)` | :arrow_up: |
| [OS-macOS](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.30% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.10.11](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.28% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.10.15](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.51% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.11.10](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.51% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.11.9](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.28% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.12.7](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.51% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.13.0](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.51% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.9.13](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.24% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.9.20](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.47% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-pypy7.3.16](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `?` | |
| [Py-pypy7.3.17](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.55% <100.00%> (+<0.01%)` | :arrow_up: |
| [VM-macos-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.30% <100.00%> (+<0.01%)` | :arrow_up: |
| [VM-ubuntu-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.55% <100.00%> (+<0.01%)` | :arrow_up: |
| [VM-windows-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.62% <100.00%> (+<0.01%)` | :arrow_up: |
| [pytest](https://app.codecov.io/gh/aio-libs/yarl/pull/1421/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.55% <100.00%> (+<0.01%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aio-libs/yarl/pull/1421?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).

bdraco: reframed this as a new feature since the current behavior doesn't change and its by design. All the functionality here is new.
bdraco: <img width="697" alt="Screenshot 2024-11-18 at 12 36 02 PM" src="https://github.com/user-attachments/assets/1ca7d9dc-1921-4415-acab-838fd980a3af">


bdraco: One last thing, we also need to update the docs for `with_path` with the new params

https://yarl--1421.org.readthedocs.build/en/1421/api/#yarl.URL.with_path
asvetlov: We also have `.with_name()` and `.with_suffix()`.
Maybe these methods need proposed skip options as well?
paul-nameless: Updated the documentation
bdraco: We will also need to bump the version to 1.18.0.dev0 after merging this PR
asvetlov: > We will also need to bump the version to 1.18.0.dev0 after merging this PR

Technically, we can do it right now. 

Patch:
diff --git a/CHANGES/111.feature.rst b/CHANGES/111.feature.rst
new file mode 120000
index 0000000..96c7efc
--- /dev/null
+++ b/CHANGES/111.feature.rst
@@ -0,0 +1,1 @@
+1421.feature.rst
\ No newline at end of file
diff --git a/CHANGES/1421.feature.rst b/CHANGES/1421.feature.rst
new file mode 100644
index 0000000..bb8f7de
--- /dev/null
+++ b/CHANGES/1421.feature.rst
@@ -0,0 +1,1 @@
+Added ``keep_query`` and ``keep_fragment`` flags in the :py:meth:`yarl.URL.with_path`, :py:meth:`yarl.URL.with_name` and :py:meth:`yarl.URL.with_suffix` methods, allowing users to optionally retain the query string and fragment in the resulting URL when replacing the path -- by :user:`paul-nameless`.
diff --git a/docs/api.rst b/docs/api.rst
index f9698fb..69417bd 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -652,10 +652,16 @@ section generates a new :class:`URL` instance.
       >>> URL('http://example.com:8888').with_port(None)
       URL('http://example.com')
 
-.. method:: URL.with_path(path)
+.. method:: URL.with_path(path, *, keep_query=False, keep_fragment=False)
 
    Return a new URL with *path* replaced, encode *path* if needed.
 
+   If ``keep_query=True`` or ``keep_fragment=True`` it retains the existing query or fragment in the URL.
+
+   .. versionchanged:: 1.18
+
+      Added *keep_query* and *keep_fragment* parameters.
+
    .. doctest::
 
       >>> URL('http://example.com/').with_path('/path/to')
@@ -857,13 +863,19 @@ section generates a new :class:`URL` instance.
       >>> URL('http://example.com/path#frag').with_fragment(None)
       URL('http://example.com/path')
 
-.. method:: URL.with_name(name)
+.. method:: URL.with_name(name, *, keep_query=False, keep_fragment=False)
 
    Return a new URL with *name* (last part of *path*) replaced and
    cleaned up *query* and *fragment* parts.
 
    Name is encoded if needed.
 
+   If ``keep_query=True`` or ``keep_fragment=True`` it retains the existing query or fragment in the URL.
+
+   .. versionchanged:: 1.18
+
+      Added *keep_query* and *keep_fragment* parameters.
+
    .. doctest::
 
       >>> URL('http://example.com/path/to?arg#frag').with_name('new')
@@ -871,13 +883,19 @@ section generates a new :class:`URL` instance.
       >>> URL('http://example.com/path/to').with_name("ім'я")
       URL('http://example.com/path/%D1%96%D0%BC%27%D1%8F')
 
-.. method:: URL.with_suffix(suffix)
+.. method:: URL.with_suffix(suffix, *, keep_query=False, keep_fragment=False)
 
    Return a new URL with *suffix* (file extension of *name*) replaced and
    cleaned up *query* and *fragment* parts.
 
    Name is encoded if needed.
 
+   If ``keep_query=True`` or ``keep_fragment=True`` it retains the existing query or fragment in the URL.
+
+   .. versionchanged:: 1.18
+
+      Added *keep_query* and *keep_fragment* parameters.
+
    .. doctest::
 
       >>> URL('http://example.com/path/to?arg#frag').with_suffix('.doc')
diff --git a/yarl/_url.py b/yarl/_url.py
index d20a610..4bf3c2d 100644
--- a/yarl/_url.py
+++ b/yarl/_url.py
@@ -1114,7 +1114,14 @@ class URL:
             self._scheme, netloc, self._path, self._query, self._fragment
         )
 
-    def with_path(self, path: str, *, encoded: bool = False) -> "URL":
+    def with_path(
+        self,
+        path: str,
+        *,
+        encoded: bool = False,
+        keep_query: bool = False,
+        keep_fragment: bool = False,
+    ) -> "URL":
         """Return a new URL with path replaced."""
         netloc = self._netloc
         if not encoded:
@@ -1123,7 +1130,9 @@ class URL:
                 path = normalize_path(path) if "." in path else path
         if path and path[0] != "/":
             path = f"/{path}"
-        return self._from_parts(self._scheme, netloc, path, "", "")
+        query = self._query if keep_query else ""
+        fragment = self._fragment if keep_fragment else ""
+        return self._from_parts(self._scheme, netloc, path, query, fragment)
 
     @overload
     def with_query(self, query: Query) -> "URL": ...
@@ -1271,7 +1280,13 @@ class URL:
             self._scheme, self._netloc, self._path, self._query, raw_fragment
         )
 
-    def with_name(self, name: str) -> "URL":
+    def with_name(
+        self,
+        name: str,
+        *,
+        keep_query: bool = False,
+        keep_fragment: bool = False,
+    ) -> "URL":
         """Return a new URL with name (last part of path) replaced.
 
         Query and fragment parts are cleaned up.
@@ -1298,9 +1313,18 @@ class URL:
             parts[-1] = name
             if parts[0] == "/":
                 parts[0] = ""  # replace leading '/'
-        return self._from_parts(self._scheme, netloc, "/".join(parts), "", "")
 
-    def with_suffix(self, suffix: str) -> "URL":
+        query = self._query if keep_query else ""
+        fragment = self._fragment if keep_fragment else ""
+        return self._from_parts(self._scheme, netloc, "/".join(parts), query, fragment)
+
+    def with_suffix(
+        self,
+        suffix: str,
+        *,
+        keep_query: bool = False,
+        keep_fragment: bool = False,
+    ) -> "URL":
         """Return a new URL with suffix (file extension of name) replaced.
 
         Query and fragment parts are cleaned up.
@@ -1316,7 +1340,8 @@ class URL:
             raise ValueError(f"{self!r} has an empty name")
         old_suffix = self.raw_suffix
         name = name + suffix if not old_suffix else name[: -len(old_suffix)] + suffix
-        return self.with_name(name)
+
+        return self.with_name(name, keep_query=keep_query, keep_fragment=keep_fragment)
 
     def join(self, url: "URL") -> "URL":
         """Join URLs


Test patch:
diff --git a/tests/test_url.py b/tests/test_url.py
index 0a8ed90..da68c2f 100644
--- a/tests/test_url.py
+++ b/tests/test_url.py
@@ -10,7 +10,7 @@ _WHATWG_C0_CONTROL_OR_SPACE = (
     "\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f "
 )
 _VERTICAL_COLON = "\ufe13"  # normalizes to ":"
-_FULL_WITH_NUMBER_SIGN = "\uFF03"  # normalizes to "#"
+_FULL_WITH_NUMBER_SIGN = "\uff03"  # normalizes to "#"
 _ACCOUNT_OF = "\u2100"  # normalizes to "a/c"
 
 
@@ -1240,6 +1240,47 @@ def test_with_path_fragment():
     assert str(url.with_path("/test")) == "http://example.com/test"
 
 
+@pytest.mark.parametrize(
+    ("original_url", "keep_query", "keep_fragment", "expected_url"),
+    [
+        pytest.param(
+            "http://example.com?a=b#frag",
+            True,
+            False,
+            "http://example.com/test?a=b",
+            id="query-only",
+        ),
+        pytest.param(
+            "http://example.com?a=b#frag",
+            False,
+            True,
+            "http://example.com/test#frag",
+            id="fragment-only",
+        ),
+        pytest.param(
+            "http://example.com?a=b#frag",
+            True,
+            True,
+            "http://example.com/test?a=b#frag",
+            id="all",
+        ),
+        pytest.param(
+            "http://example.com?a=b#frag",
+            False,
+            False,
+            "http://example.com/test",
+            id="none",
+        ),
+    ],
+)
+def test_with_path_keep_query_keep_fragment_flags(
+    original_url, keep_query, keep_fragment, expected_url
+):
+    url = URL(original_url)
+    url2 = url.with_path("/test", keep_query=keep_query, keep_fragment=keep_fragment)
+    assert str(url2) == expected_url
+
+
 def test_with_path_empty():
     url = URL("http://example.com/test")
     assert str(url.with_path("")) == "http://example.com"
@@ -1319,6 +1360,47 @@ def test_with_name():
     assert url2.path == "/a/c"
 
 
+@pytest.mark.parametrize(
+    ("original_url", "keep_query", "keep_fragment", "expected_url"),
+    [
+        pytest.param(
+            "http://example.com/path/to?a=b#frag",
+            True,
+            False,
+            "http://example.com/path/newname?a=b",
+            id="query-only",
+        ),
+        pytest.param(
+            "http://example.com/path/to?a=b#frag",
+            False,
+            True,
+            "http://example.com/path/newname#frag",
+            id="fragment-only",
+        ),
+        pytest.param(
+            "http://example.com/path/to?a=b#frag",
+            True,
+            True,
+            "http://example.com/path/newname?a=b#frag",
+            id="all",
+        ),
+        pytest.param(
+            "http://example.com/path/to?a=b#frag",
+            False,
+            False,
+            "http://example.com/path/newname",
+            id="none",
+        ),
+    ],
+)
+def test_with_name_keep_query_keep_fragment_flags(
+    original_url, keep_query, keep_fragment, expected_url
+):
+    url = URL(original_url)
+    url2 = url.with_name("newname", keep_query=keep_query, keep_fragment=keep_fragment)
+    assert str(url2) == expected_url
+
+
 def test_with_name_for_naked_path():
     url = URL("http://example.com")
     url2 = url.with_name("a")
@@ -1409,6 +1491,47 @@ def test_with_suffix():
     assert url2.path == "/a/b.c"
 
 
+@pytest.mark.parametrize(
+    ("original_url", "keep_query", "keep_fragment", "expected_url"),
+    [
+        pytest.param(
+            "http://example.com/path/to.txt?a=b#frag",
+            True,
+            False,
+            "http://example.com/path/to.md?a=b",
+            id="query-only",
+        ),
+        pytest.param(
+            "http://example.com/path/to.txt?a=b#frag",
+            False,
+            True,
+            "http://example.com/path/to.md#frag",
+            id="fragment-only",
+        ),
+        pytest.param(
+            "http://example.com/path/to.txt?a=b#frag",
+            True,
+            True,
+            "http://example.com/path/to.md?a=b#frag",
+            id="all",
+        ),
+        pytest.param(
+            "http://example.com/path/to.txt?a=b#frag",
+            False,
+            False,
+            "http://example.com/path/to.md",
+            id="none",
+        ),
+    ],
+)
+def test_with_suffix_keep_query_keep_fragment_flags(
+    original_url, keep_query, keep_fragment, expected_url
+):
+    url = URL(original_url)
+    url2 = url.with_suffix(".md", keep_query=keep_query, keep_fragment=keep_fragment)
+    assert str(url2) == expected_url
+
+
 def test_with_suffix_for_naked_path():
     url = URL("http://example.com")
     with pytest.raises(ValueError) as excinfo: