Repository: pallets/werkzeug (python)

Issue:
Use of warnings.catch_warnings resets the global warnings registry
There is a long-standing bug in Python python/cpython#73858 whereby any use of `warnings.catch_warnings` causes all memory of which warnings have already been displayed to be lost.  So warnings whose [action](https://docs.python.org/3/library/warnings.html#the-warnings-filter) is configured as "default" (meaning show once for each module/line-number) get displayed multiple (possibly many, many) times.

Note that use of `catch_warnings` resets memory of what warnings have been displayed globally — use of `catch_warnings` by werkzeug can cause redisplay of unrelated warnings from other modules.

PR #2608 (19d55c1e01d628d9a256526b573f478248d4bc5a) introduced a lot of new uses of `catch_warnings`.

I'm not sure what the clean way is to work-around the issue, but it would be nice not to break the "default" warning action so drastically.

<!--
This issue tracker is a tool to address bugs in Werkzeug itself. Please
use Pallets Discord or Stack Overflow for questions about your own code.

Replace this comment with a clear outline of what the bug is.
-->

<!--
Describe how to replicate the bug.

Include a minimal reproducible example that demonstrates the bug.
Include the full traceback if there was an exception.
-->
Example: `too-verbose.py`:
```py
from werkzeug.urls import url_parse

for _ in range(10):
    url_parse("x")
```

Running
```sh
python too-verbose.py
```
will elicit 10 `url_parse` deprecation warnings.  (The default warnings filter specifies an action of "default" for DeprecationWarnings, which means that they should be displayed just once per location.)

<!--
Describe the expected behavior that should have happened but didn't.
-->

Environment:

- Python version: I've tested under 3.9 and 3.11.  I think it affects all 3.x pythons.
- Werkzeug version: 2.3.3


Pull request:
Remove uses of warnings.catch_warnings (#2690)
This removes the use of `warnings.catch_warnings`, thus preventing the rest of the global warnings registry (and the ensuing repetition of warnings filtered with the "default" action)  as discussed at #2690.


<!--
Before opening a PR, open a ticket describing the issue or feature the
PR will address. Follow the steps in CONTRIBUTING.rst.

Replace this comment with a description of the change. Describe how it
addresses the linked ticket.
-->

<!--
Link to relevant issues or previous PRs, one per line. Use "fixes" to
automatically close an issue.
-->

- fixes #2690

<!--
Ensure each step in CONTRIBUTING.rst is complete by adding an "x" to
each box below.

If only docs were changed, these aren't relevant and can be removed.
-->

Checklist:

- [x] Add tests that demonstrate the correct behavior of the change. Tests should fail without the change.
- [x] Add an entry in `CHANGES.rst` summarizing the change and linking to the issue.
- [x] Run `pre-commit` hooks and fix any issues.
- [x] Run `pytest` and `tox`, no tests failed.
- [ ] Question (see below): Does `werkzeug.wsgi._make_chunk_iter` really have to issue a deprecation warning?

#### Question

Since `_make_chunk_iter` is not document API,  can the [deprecation warning](https://github.com/pallets/werkzeug/blob/2051469a2be722121369b416a5c14435c9e82f90/src/werkzeug/wsgi.py#L464-L468) it emits just be deleted?  Doing so would remove several cases where `catch_warnings` was suppressing warnings from internal calls.


Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 2d83ee59..fa8d36a8 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -7,6 +7,7 @@ Unreleased
 
 -   ``Authorization.from_header`` and ``WWWAuthenticate.from_header`` detects tokens
     that end with base64 padding (``=``). :issue:`2685`
+-   Remove usage of ``warnings.catch_warnings``. :issue:`2690`
 
 
 Version 2.3.3
diff --git a/src/werkzeug/urls.py b/src/werkzeug/urls.py
index b0e62d0b..ea60690b 100644
--- a/src/werkzeug/urls.py
+++ b/src/werkzeug/urls.py
@@ -371,15 +371,13 @@ class URL(BaseURL):
         """Encodes the URL to a tuple made out of bytes.  The charset is
         only being used for the path, query and fragment.
         """
-        with warnings.catch_warnings():
-            warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-            return BytesURL(
-                self.scheme.encode("ascii"),
-                self.encode_netloc(),
-                self.path.encode(charset, errors),
-                self.query.encode(charset, errors),
-                self.fragment.encode(charset, errors),
-            )
+        return BytesURL(
+            self.scheme.encode("ascii"),
+            self.encode_netloc(),
+            self.path.encode(charset, errors),
+            self.query.encode(charset, errors),
+            self.fragment.encode(charset, errors),
+        )
 
 
 class BytesURL(BaseURL):
@@ -406,15 +404,13 @@ class BytesURL(BaseURL):
         """Decodes the URL to a tuple made out of strings.  The charset is
         only being used for the path, query and fragment.
         """
-        with warnings.catch_warnings():
-            warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-            return URL(
-                self.scheme.decode("ascii"),  # type: ignore
-                self.decode_netloc(),
-                self.path.decode(charset, errors),  # type: ignore
-                self.query.decode(charset, errors),  # type: ignore
-                self.fragment.decode(charset, errors),  # type: ignore
-            )
+        return URL(
+            self.scheme.decode("ascii"),  # type: ignore
+            self.decode_netloc(),
+            self.path.decode(charset, errors),  # type: ignore
+            self.query.decode(charset, errors),  # type: ignore
+            self.fragment.decode(charset, errors),  # type: ignore
+        )
 
 
 _unquote_maps: dict[frozenset[int], dict[bytes, int]] = {frozenset(): _hextobyte}
@@ -546,9 +542,7 @@ def url_parse(
 
     result_type = URL if is_text_based else BytesURL
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return result_type(scheme, netloc, url, query, fragment)
+    return result_type(scheme, netloc, url, query, fragment)
 
 
 def _make_fast_url_quote(
@@ -655,9 +649,7 @@ def url_quote_plus(
         stacklevel=2,
     )
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return url_quote(string, charset, errors, safe + " ", "+").replace(" ", "+")
+    return url_quote(string, charset, errors, safe + " ", "+").replace(" ", "+")
 
 
 def url_unparse(components: tuple[str, str, str, str, str]) -> str:
@@ -759,9 +751,7 @@ def url_unquote_plus(
     else:
         s = s.replace(b"+", b" ")
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return url_unquote(s, charset, errors)
+    return url_unquote(s, charset, errors)
 
 
 def url_fix(s: str, charset: str = "utf-8") -> str:
@@ -795,13 +785,11 @@ def url_fix(s: str, charset: str = "utf-8") -> str:
     if s.startswith("file://") and s[7:8].isalpha() and s[8:10] in (":/", "|/"):
         s = f"file:///{s[7:]}"
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        url = url_parse(s)
-        path = url_quote(url.path, charset, safe="/%+$!*'(),")
-        qs = url_quote_plus(url.query, charset, safe=":&%=+$!*'(),")
-        anchor = url_quote_plus(url.fragment, charset, safe=":&%=+$!*'(),")
-        return url_unparse((url.scheme, url.encode_netloc(), path, qs, anchor))
+    url = url_parse(s)
+    path = url_quote(url.path, charset, safe="/%+$!*'(),")
+    qs = url_quote_plus(url.query, charset, safe=":&%=+$!*'(),")
+    anchor = url_quote_plus(url.fragment, charset, safe=":&%=+$!*'(),")
+    return url_unparse((url.scheme, url.encode_netloc(), path, qs, anchor))
 
 
 def _codec_error_url_quote(e: UnicodeError) -> tuple[str, int]:
@@ -1171,9 +1159,7 @@ def url_decode_stream(
 
         cls = MultiDict
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'make_chunk_iter", DeprecationWarning)
-        return cls(decoder)
+    return cls(decoder)
 
 
 def _url_decode_impl(
diff --git a/src/werkzeug/wsgi.py b/src/werkzeug/wsgi.py
index a3a2fbb5..b555f512 100644
--- a/src/werkzeug/wsgi.py
+++ b/src/werkzeug/wsgi.py
@@ -534,9 +534,7 @@ def make_line_iter(
     )
     _iter = _make_chunk_iter(stream, limit, buffer_size)
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'_make_chunk_iter", DeprecationWarning)
-        first_item = next(_iter, "")
+    first_item = next(_iter, "")
 
     if not first_item:
         return
@@ -631,9 +629,7 @@ def make_chunk_iter(
     )
     _iter = _make_chunk_iter(stream, limit, buffer_size)
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'_make_chunk_iter", DeprecationWarning)
-        first_item = next(_iter, b"")
+    first_item = next(_iter, b"")
 
     if not first_item:
         return


Test patch:
diff --git a/tests/test_urls.py b/tests/test_urls.py
index 8baf0ad6..56bca8e9 100644
--- a/tests/test_urls.py
+++ b/tests/test_urls.py
@@ -1,11 +1,15 @@
 import io
+import warnings
 
 import pytest
 
 from werkzeug import urls
 from werkzeug.datastructures import OrderedMultiDict
 
-pytestmark = [pytest.mark.filterwarnings("ignore:'werkzeug:DeprecationWarning")]
+pytestmark = [
+    pytest.mark.filterwarnings("ignore:'werkzeug:DeprecationWarning"),
+    pytest.mark.filterwarnings("ignore:'_?make_chunk_iter':DeprecationWarning"),
+]
 
 
 def test_parsing():
@@ -382,3 +386,12 @@ def test_iri_to_uri_dont_quote_valid_code_points():
     # [] are not valid URL code points according to WhatWG URL Standard
     # https://url.spec.whatwg.org/#url-code-points
     assert urls.iri_to_uri("/path[bracket]?(paren)") == "/path%5Bbracket%5D?(paren)"
+
+
+def test_url_parse_does_not_clear_warnings_registry(recwarn):
+    warnings.simplefilter("default")
+    warnings.simplefilter("ignore", DeprecationWarning)
+    for _ in range(2):
+        urls.url_parse("http://example.org/")
+        warnings.warn("test warning")
+    assert len(recwarn) == 1
diff --git a/tests/test_wsgi.py b/tests/test_wsgi.py
index d8335e5a..5f37aca9 100644
--- a/tests/test_wsgi.py
+++ b/tests/test_wsgi.py
@@ -257,6 +257,7 @@ def test_get_current_url_invalid_utf8():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks():
     data = b"abcdef\r\nghijkl\r\nmnopqrstuvwxyz\r\nABCDEFGHIJK"
     test_stream = io.BytesIO(data)
@@ -279,6 +280,7 @@ def test_multi_part_line_breaks():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks_bytes():
     data = b"abcdef\r\nghijkl\r\nmnopqrstuvwxyz\r\nABCDEFGHIJK"
     test_stream = io.BytesIO(data)
@@ -301,6 +303,7 @@ def test_multi_part_line_breaks_bytes():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks_problematic():
     data = b"abc\rdef\r\nghi"
     for _ in range(1, 10):
@@ -310,13 +313,14 @@ def test_multi_part_line_breaks_problematic():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_iter_functions_support_iterators():
     data = ["abcdef\r\nghi", "jkl\r\nmnopqrstuvwxyz\r", "\nABCDEFGHIJK"]
     lines = list(wsgi.make_line_iter(data))
     assert lines == ["abcdef\r\n", "ghijkl\r\n", "mnopqrstuvwxyz\r\n", "ABCDEFGHIJK"]
 
 
-@pytest.mark.filterwarnings("ignore:'make_chunk_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_?make_chunk_iter:DeprecationWarning")
 def test_make_chunk_iter():
     data = [b"abcdefXghi", b"jklXmnopqrstuvwxyzX", b"ABCDEFGHIJK"]
     rv = list(wsgi.make_chunk_iter(data, b"X"))
@@ -328,7 +332,7 @@ def test_make_chunk_iter():
     assert rv == [b"abcdef", b"ghijkl", b"mnopqrstuvwxyz", b"ABCDEFGHIJK"]
 
 
-@pytest.mark.filterwarnings("ignore:'make_chunk_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_?make_chunk_iter:DeprecationWarning")
 def test_make_chunk_iter_bytes():
     data = [b"abcdefXghi", b"jklXmnopqrstuvwxyzX", b"ABCDEFGHIJK"]
     rv = list(wsgi.make_chunk_iter(data, "X"))
@@ -362,6 +366,7 @@ def test_make_chunk_iter_bytes():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_lines_longer_buffer_size():
     data = b"1234567890\n1234567890\n"
     for bufsize in range(1, 15):
@@ -372,6 +377,7 @@ def test_lines_longer_buffer_size():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_lines_longer_buffer_size_cap():
     data = b"1234567890\n1234567890\n"
     for bufsize in range(1, 15):