Repository: pallets/werkzeug

Issue:
Use of warnings.catch_warnings resets the global warnings registry
There is a long-standing bug in Python python/cpython#73858 whereby any use of `warnings.catch_warnings` causes all memory of which warnings have already been displayed to be lost.  So warnings whose [action](https://docs.python.org/3/library/warnings.html#the-warnings-filter) is configured as "default" (meaning show once for each module/line-number) get displayed multiple (possibly many, many) times.

Note that use of `catch_warnings` resets memory of what warnings have been displayed globally — use of `catch_warnings` by werkzeug can cause redisplay of unrelated warnings from other modules.

PR #2608 (19d55c1e01d628d9a256526b573f478248d4bc5a) introduced a lot of new uses of `catch_warnings`.

I'm not sure what the clean way is to work-around the issue, but it would be nice not to break the "default" warning action so drastically.

<!--
This issue tracker is a tool to address bugs in Werkzeug itself. Please
use Pallets Discord or Stack Overflow for questions about your own code.

Replace this comment with a clear outline of what the bug is.
-->

<!--
Describe how to replicate the bug.

Include a minimal reproducible example that demonstrates the bug.
Include the full traceback if there was an exception.
-->
Example: `too-verbose.py`:
```py
from werkzeug.urls import url_parse

for _ in range(10):
    url_parse("x")
```

Running
```sh
python too-verbose.py
```
will elicit 10 `url_parse` deprecation warnings.  (The default warnings filter specifies an action of "default" for DeprecationWarnings, which means that they should be displayed just once per location.)

<!--
Describe the expected behavior that should have happened but didn't.
-->

Environment:

- Python version: I've tested under 3.9 and 3.11.  I think it affects all 3.x pythons.
- Werkzeug version: 2.3.3



Discussion:
The answer is to stop using the code that's showing the warnings, or report and ignore them if they're in other libraries. "Warnings are even more visible" isn't really a bad thing. 
The reason it's used is to prevent functions that call other functions that take the same deprecated arguments from showing two warnings one of which is not addressable by users. 
> The reason it's used is to prevent functions that call other functions that take the same deprecated arguments from showing two warnings one of which is not addressable by users.

That I realized, which is why I don't have a real clean way to work-around the issue.  One way would be to use a thread-local flag or some such to disable the generation of the non-user-addressable warnings — admittedly, a messy solution. 


I don't plan to add more complexity to the warnings, they're meant to be low overhead. I can remove `catch_warnings` and let the extra warnings pop up, if we think it won't be too confusing.
> I can remove `catch_warnings` and let the extra warnings pop up, if we think it won't be too confusing.

Standby.  Let me try (removing the `catch_warnings`, that is) with my use case and see how confusing the resulting extra warnings are.  I'll report back later today.

Resolving patch:
diff --git a/src/werkzeug/urls.py b/src/werkzeug/urls.py
--- a/src/werkzeug/urls.py
+++ b/src/werkzeug/urls.py
@@ -371,15 +371,13 @@ def encode(self, charset: str = "utf-8", errors: str = "replace") -> BytesURL:
         """Encodes the URL to a tuple made out of bytes.  The charset is
         only being used for the path, query and fragment.
         """
-        with warnings.catch_warnings():
-            warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-            return BytesURL(
-                self.scheme.encode("ascii"),
-                self.encode_netloc(),
-                self.path.encode(charset, errors),
-                self.query.encode(charset, errors),
-                self.fragment.encode(charset, errors),
-            )
+        return BytesURL(
+            self.scheme.encode("ascii"),
+            self.encode_netloc(),
+            self.path.encode(charset, errors),
+            self.query.encode(charset, errors),
+            self.fragment.encode(charset, errors),
+        )
 
 
 class BytesURL(BaseURL):
@@ -406,15 +404,13 @@ def decode(self, charset: str = "utf-8", errors: str = "replace") -> URL:
         """Decodes the URL to a tuple made out of strings.  The charset is
         only being used for the path, query and fragment.
         """
-        with warnings.catch_warnings():
-            warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-            return URL(
-                self.scheme.decode("ascii"),  # type: ignore
-                self.decode_netloc(),
-                self.path.decode(charset, errors),  # type: ignore
-                self.query.decode(charset, errors),  # type: ignore
-                self.fragment.decode(charset, errors),  # type: ignore
-            )
+        return URL(
+            self.scheme.decode("ascii"),  # type: ignore
+            self.decode_netloc(),
+            self.path.decode(charset, errors),  # type: ignore
+            self.query.decode(charset, errors),  # type: ignore
+            self.fragment.decode(charset, errors),  # type: ignore
+        )
 
 
 _unquote_maps: dict[frozenset[int], dict[bytes, int]] = {frozenset(): _hextobyte}
@@ -546,9 +542,7 @@ def url_parse(
 
     result_type = URL if is_text_based else BytesURL
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return result_type(scheme, netloc, url, query, fragment)
+    return result_type(scheme, netloc, url, query, fragment)
 
 
 def _make_fast_url_quote(
@@ -655,9 +649,7 @@ def url_quote_plus(
         stacklevel=2,
     )
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return url_quote(string, charset, errors, safe + " ", "+").replace(" ", "+")
+    return url_quote(string, charset, errors, safe + " ", "+").replace(" ", "+")
 
 
 def url_unparse(components: tuple[str, str, str, str, str]) -> str:
@@ -759,9 +751,7 @@ def url_unquote_plus(
     else:
         s = s.replace(b"+", b" ")
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        return url_unquote(s, charset, errors)
+    return url_unquote(s, charset, errors)
 
 
 def url_fix(s: str, charset: str = "utf-8") -> str:
@@ -795,13 +785,11 @@ def url_fix(s: str, charset: str = "utf-8") -> str:
     if s.startswith("file://") and s[7:8].isalpha() and s[8:10] in (":/", "|/"):
         s = f"file:///{s[7:]}"
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'werkzeug", DeprecationWarning)
-        url = url_parse(s)
-        path = url_quote(url.path, charset, safe="/%+$!*'(),")
-        qs = url_quote_plus(url.query, charset, safe=":&%=+$!*'(),")
-        anchor = url_quote_plus(url.fragment, charset, safe=":&%=+$!*'(),")
-        return url_unparse((url.scheme, url.encode_netloc(), path, qs, anchor))
+    url = url_parse(s)
+    path = url_quote(url.path, charset, safe="/%+$!*'(),")
+    qs = url_quote_plus(url.query, charset, safe=":&%=+$!*'(),")
+    anchor = url_quote_plus(url.fragment, charset, safe=":&%=+$!*'(),")
+    return url_unparse((url.scheme, url.encode_netloc(), path, qs, anchor))
 
 
 def _codec_error_url_quote(e: UnicodeError) -> tuple[str, int]:
@@ -1171,9 +1159,7 @@ def url_decode_stream(
 
         cls = MultiDict
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'make_chunk_iter", DeprecationWarning)
-        return cls(decoder)
+    return cls(decoder)
 
 
 def _url_decode_impl(
diff --git a/src/werkzeug/wsgi.py b/src/werkzeug/wsgi.py
--- a/src/werkzeug/wsgi.py
+++ b/src/werkzeug/wsgi.py
@@ -534,9 +534,7 @@ def make_line_iter(
     )
     _iter = _make_chunk_iter(stream, limit, buffer_size)
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'_make_chunk_iter", DeprecationWarning)
-        first_item = next(_iter, "")
+    first_item = next(_iter, "")
 
     if not first_item:
         return
@@ -631,9 +629,7 @@ def make_chunk_iter(
     )
     _iter = _make_chunk_iter(stream, limit, buffer_size)
 
-    with warnings.catch_warnings():
-        warnings.filterwarnings("ignore", "'_make_chunk_iter", DeprecationWarning)
-        first_item = next(_iter, b"")
+    first_item = next(_iter, b"")
 
     if not first_item:
         return


Test patch:
diff --git a/tests/test_urls.py b/tests/test_urls.py
--- a/tests/test_urls.py
+++ b/tests/test_urls.py
@@ -1,11 +1,15 @@
 import io
+import warnings
 
 import pytest
 
 from werkzeug import urls
 from werkzeug.datastructures import OrderedMultiDict
 
-pytestmark = [pytest.mark.filterwarnings("ignore:'werkzeug:DeprecationWarning")]
+pytestmark = [
+    pytest.mark.filterwarnings("ignore:'werkzeug:DeprecationWarning"),
+    pytest.mark.filterwarnings("ignore:'_?make_chunk_iter':DeprecationWarning"),
+]
 
 
 def test_parsing():
@@ -382,3 +386,12 @@ def test_iri_to_uri_dont_quote_valid_code_points():
     # [] are not valid URL code points according to WhatWG URL Standard
     # https://url.spec.whatwg.org/#url-code-points
     assert urls.iri_to_uri("/path[bracket]?(paren)") == "/path%5Bbracket%5D?(paren)"
+
+
+def test_url_parse_does_not_clear_warnings_registry(recwarn):
+    warnings.simplefilter("default")
+    warnings.simplefilter("ignore", DeprecationWarning)
+    for _ in range(2):
+        urls.url_parse("http://example.org/")
+        warnings.warn("test warning")
+    assert len(recwarn) == 1
diff --git a/tests/test_wsgi.py b/tests/test_wsgi.py
--- a/tests/test_wsgi.py
+++ b/tests/test_wsgi.py
@@ -257,6 +257,7 @@ def test_get_current_url_invalid_utf8():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks():
     data = b"abcdef\r\nghijkl\r\nmnopqrstuvwxyz\r\nABCDEFGHIJK"
     test_stream = io.BytesIO(data)
@@ -279,6 +280,7 @@ def test_multi_part_line_breaks():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks_bytes():
     data = b"abcdef\r\nghijkl\r\nmnopqrstuvwxyz\r\nABCDEFGHIJK"
     test_stream = io.BytesIO(data)
@@ -301,6 +303,7 @@ def test_multi_part_line_breaks_bytes():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_multi_part_line_breaks_problematic():
     data = b"abc\rdef\r\nghi"
     for _ in range(1, 10):
@@ -310,13 +313,14 @@ def test_multi_part_line_breaks_problematic():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_iter_functions_support_iterators():
     data = ["abcdef\r\nghi", "jkl\r\nmnopqrstuvwxyz\r", "\nABCDEFGHIJK"]
     lines = list(wsgi.make_line_iter(data))
     assert lines == ["abcdef\r\n", "ghijkl\r\n", "mnopqrstuvwxyz\r\n", "ABCDEFGHIJK"]
 
 
-@pytest.mark.filterwarnings("ignore:'make_chunk_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_?make_chunk_iter:DeprecationWarning")
 def test_make_chunk_iter():
     data = [b"abcdefXghi", b"jklXmnopqrstuvwxyzX", b"ABCDEFGHIJK"]
     rv = list(wsgi.make_chunk_iter(data, b"X"))
@@ -328,7 +332,7 @@ def test_make_chunk_iter():
     assert rv == [b"abcdef", b"ghijkl", b"mnopqrstuvwxyz", b"ABCDEFGHIJK"]
 
 
-@pytest.mark.filterwarnings("ignore:'make_chunk_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_?make_chunk_iter:DeprecationWarning")
 def test_make_chunk_iter_bytes():
     data = [b"abcdefXghi", b"jklXmnopqrstuvwxyzX", b"ABCDEFGHIJK"]
     rv = list(wsgi.make_chunk_iter(data, "X"))
@@ -362,6 +366,7 @@ def test_make_chunk_iter_bytes():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_lines_longer_buffer_size():
     data = b"1234567890\n1234567890\n"
     for bufsize in range(1, 15):
@@ -372,6 +377,7 @@ def test_lines_longer_buffer_size():
 
 
 @pytest.mark.filterwarnings("ignore:'make_line_iter:DeprecationWarning")
+@pytest.mark.filterwarnings("ignore:'_make_chunk_iter:DeprecationWarning")
 def test_lines_longer_buffer_size_cap():
     data = b"1234567890\n1234567890\n"
     for bufsize in range(1, 15):