Repository: pallets/werkzeug (python)

Issue:
Some relative redirect strings incorrectly bypass iri_to_uri normalization. 
## Summary 
It appears logic introduced in https://github.com/pallets/werkzeug/pull/2695 to work-around the `itms-services:` schema oddity has an overly broad assumption that is results in invalid URIs due to not percent-encoding relative IRIs as intended. 

Ultimately, this can trick browsers into behaving badly when faced with invalid relative URL locations. But I think it is ours to fix in werkzeug because the current code is emitting an invalid location header per RFCs (and the presumed intention of the code.) 

RFC 3986 for URIs has a [section 2](https://datatracker.ietf.org/doc/html/rfc3986#section-2.3) that essentially says URI components should be percent-encoded if they aren't part of the "reserved" or explicitly "unreserved" lists of characters. As an case-study: backslashes should be percent-encoded. Indeed, werkzeug seems to intent to encode them, but this issue describes the case where the typical encoding logic is not being applied.

## Replication
As an example consider an app expecting to use `redirect(relative_uri)` and expecting werkzeug to properly encode the `Location:` header (as it historically would before the mentioned work-around)

```python
from werkzeug.utils import redirect
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
response = redirect(problem_relative_uri)
response.location
# '/\\\\github.com?extra=data&blank=#fragment'
```

Note the backslashes (`\`) in this URI are /not/ being encoded as `%5C` . 

As a smaller test case:
```python
from werkzeug.urls import iri_to_uri, _invalid_iri_to_uri
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
_invalid_iri_to_uri(problem_relative_uri)
# '/\\\\github.com?extra=data&blank=#fragment'
# :-(
```
## Expected Behavior
The previous encoding behavior before the wrapper was desirable
```python
iri_to_uri(problem_relative_uri)
# '/%5C%5Cgithub.com?extra=data&blank=#fragment'
# :-)
```

Environment:

- Python version: 3.12.1
- Werkzeug version: 3.0.1


## Relevant Code
https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/wrappers/response.py#L482-L483

https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/urls.py#L167-L185



Pull request:
remove iri_to_uri redirect workaround
Tell Python < 3.12 to handle the `itms-services` scheme correctly, rather than using a workaround in Werkzeug. The workaround was too broad and would pass through other ASCII URIs that should have been percent encoded. See https://github.com/python/cpython/issues/104139#issuecomment-1539176043 for how to add other schemes. If you find a scheme that needs special handling, please report it to Python.

fixes #2828


Patch:
diff --git a/CHANGES.rst b/CHANGES.rst
index 279fd3f7..367cfb66 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -7,6 +7,9 @@ Unreleased
 
 -   Make reloader more robust when ``""`` is in ``sys.path``. :pr:`2823`
 -   Better TLS cert format with ``adhoc`` dev certs. :pr:`2891`
+-   Inform Python < 3.12 how to handle ``itms-services`` URIs correctly, rather
+    than using an overly-broad workaround in Werkzeug that caused some redirect
+    URIs to be passed on without encoding. :issue:`2828`
 
 
 Version 3.0.2
diff --git a/src/werkzeug/urls.py b/src/werkzeug/urls.py
index 4d61e600..5bffe392 100644
--- a/src/werkzeug/urls.py
+++ b/src/werkzeug/urls.py
@@ -3,6 +3,7 @@ from __future__ import annotations
 import codecs
 import re
 import typing as t
+import urllib.parse
 from urllib.parse import quote
 from urllib.parse import unquote
 from urllib.parse import urlencode
@@ -164,25 +165,11 @@ def iri_to_uri(iri: str) -> str:
     return urlunsplit((parts.scheme, netloc, path, query, fragment))
 
 
-def _invalid_iri_to_uri(iri: str) -> str:
-    """The URL scheme ``itms-services://`` must contain the ``//`` even though it does
-    not have a host component. There may be other invalid schemes as well. Currently,
-    responses will always call ``iri_to_uri`` on the redirect ``Location`` header, which
-    removes the ``//``. For now, if the IRI only contains ASCII and does not contain
-    spaces, pass it on as-is. In Werkzeug 3.0, this should become a
-    ``response.process_location`` flag.
-
-    :meta private:
-    """
-    try:
-        iri.encode("ascii")
-    except UnicodeError:
-        pass
-    else:
-        if len(iri.split(None, 1)) == 1:
-            return iri
-
-    return iri_to_uri(iri)
+# Python < 3.12
+# itms-services was worked around in previous iri_to_uri implementations, but
+# we can tell Python directly that it needs to preserve the //.
+if "itms-services" not in urllib.parse.uses_netloc:
+    urllib.parse.uses_netloc.append("itms-services")
 
 
 def _decode_idna(domain: str) -> str:
diff --git a/src/werkzeug/wrappers/response.py b/src/werkzeug/wrappers/response.py
index 7b666e3e..7f01287c 100644
--- a/src/werkzeug/wrappers/response.py
+++ b/src/werkzeug/wrappers/response.py
@@ -14,7 +14,6 @@ from ..http import parse_etags
 from ..http import parse_range_header
 from ..http import remove_entity_headers
 from ..sansio.response import Response as _SansIOResponse
-from ..urls import _invalid_iri_to_uri
 from ..urls import iri_to_uri
 from ..utils import cached_property
 from ..wsgi import _RangeWrapper
@@ -479,7 +478,7 @@ class Response(_SansIOResponse):
                 content_length = value
 
         if location is not None:
-            location = _invalid_iri_to_uri(location)
+            location = iri_to_uri(location)
 
             if self.autocorrect_location_header:
                 # Make the location header an absolute URL.


Test patch:
diff --git a/tests/test_urls.py b/tests/test_urls.py
index fdaa913a..101b886e 100644
--- a/tests/test_urls.py
+++ b/tests/test_urls.py
@@ -98,3 +98,9 @@ def test_iri_to_uri_dont_quote_valid_code_points():
     # [] are not valid URL code points according to WhatWG URL Standard
     # https://url.spec.whatwg.org/#url-code-points
     assert urls.iri_to_uri("/path[bracket]?(paren)") == "/path%5Bbracket%5D?(paren)"
+
+
+# Python < 3.12
+def test_itms_services() -> None:
+    url = "itms-services://?action=download-manifest&url=https://test.example/path"
+    assert urls.iri_to_uri(url) == url
diff --git a/tests/test_wrappers.py b/tests/test_wrappers.py
index d7bc12b9..f7569445 100644
--- a/tests/test_wrappers.py
+++ b/tests/test_wrappers.py
@@ -1154,6 +1154,7 @@ def test_disabled_auto_content_length():
     ("auto", "location", "expect"),
     (
         (False, "/test", "/test"),
+        (False, "/\\\\test.example?q", "/%5C%5Ctest.example?q"),
         (True, "/test", "http://localhost/test"),
         (True, "test", "http://localhost/a/b/test"),
         (True, "./test", "http://localhost/a/b/test"),