Repository: pallets/werkzeug

Issue:
Some relative redirect strings incorrectly bypass iri_to_uri normalization. 
## Summary 
It appears logic introduced in https://github.com/pallets/werkzeug/pull/2695 to work-around the `itms-services:` schema oddity has an overly broad assumption that is results in invalid URIs due to not percent-encoding relative IRIs as intended. 

Ultimately, this can trick browsers into behaving badly when faced with invalid relative URL locations. But I think it is ours to fix in werkzeug because the current code is emitting an invalid location header per RFCs (and the presumed intention of the code.) 

RFC 3986 for URIs has a [section 2](https://datatracker.ietf.org/doc/html/rfc3986#section-2.3) that essentially says URI components should be percent-encoded if they aren't part of the "reserved" or explicitly "unreserved" lists of characters. As an case-study: backslashes should be percent-encoded. Indeed, werkzeug seems to intent to encode them, but this issue describes the case where the typical encoding logic is not being applied.

## Replication
As an example consider an app expecting to use `redirect(relative_uri)` and expecting werkzeug to properly encode the `Location:` header (as it historically would before the mentioned work-around)

```python
from werkzeug.utils import redirect
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
response = redirect(problem_relative_uri)
response.location
# '/\\\\github.com?extra=data&blank=#fragment'
```

Note the backslashes (`\`) in this URI are /not/ being encoded as `%5C` . 

As a smaller test case:
```python
from werkzeug.urls import iri_to_uri, _invalid_iri_to_uri
problem_relative_uri = '/\\\\github.com?extra=data&blank=#fragment'
_invalid_iri_to_uri(problem_relative_uri)
# '/\\\\github.com?extra=data&blank=#fragment'
# :-(
```
## Expected Behavior
The previous encoding behavior before the wrapper was desirable
```python
iri_to_uri(problem_relative_uri)
# '/%5C%5Cgithub.com?extra=data&blank=#fragment'
# :-)
```

Environment:

- Python version: 3.12.1
- Werkzeug version: 3.0.1


## Relevant Code
https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/wrappers/response.py#L482-L483

https://github.com/pallets/werkzeug/blob/d3dd65a27388fbd39d146caacf2563639ba622f0/src/werkzeug/urls.py#L167-L185




Discussion:
After @davidism 's prompt in the original issue to also file an upstream Python issue (https://github.com/python/cpython/issues/104139#issuecomment-1539176043),  Python's `urlib.parse` was improved with the `itms-services:` url schema knowledge in Python 3.12. 

The cpython comment thread also mentions a potentially more tightly scoped workaround for the "_invalid_iri" issue on older python versions:
```python
if "itms-services" not in urllib.parse.uses_netloc:
    urllib.parse.uses_netloc.append("itms-services")
```

Should we adopt this? Or otherwise rethink the current conditionals bypassing `iri_to_uri(location_iri)`?

Yes, I like that solution, since it allows users to tell Python in general about any special scheme they need. Note that you won't see `response.location` encoded in your example, that only happens at the last moment when converting the `Response` to HTTP.

Resolving patch:
diff --git a/src/werkzeug/urls.py b/src/werkzeug/urls.py
--- a/src/werkzeug/urls.py
+++ b/src/werkzeug/urls.py
@@ -3,6 +3,7 @@
 import codecs
 import re
 import typing as t
+import urllib.parse
 from urllib.parse import quote
 from urllib.parse import unquote
 from urllib.parse import urlencode
@@ -164,25 +165,11 @@ def iri_to_uri(iri: str) -> str:
     return urlunsplit((parts.scheme, netloc, path, query, fragment))
 
 
-def _invalid_iri_to_uri(iri: str) -> str:
-    """The URL scheme ``itms-services://`` must contain the ``//`` even though it does
-    not have a host component. There may be other invalid schemes as well. Currently,
-    responses will always call ``iri_to_uri`` on the redirect ``Location`` header, which
-    removes the ``//``. For now, if the IRI only contains ASCII and does not contain
-    spaces, pass it on as-is. In Werkzeug 3.0, this should become a
-    ``response.process_location`` flag.
-
-    :meta private:
-    """
-    try:
-        iri.encode("ascii")
-    except UnicodeError:
-        pass
-    else:
-        if len(iri.split(None, 1)) == 1:
-            return iri
-
-    return iri_to_uri(iri)
+# Python < 3.12
+# itms-services was worked around in previous iri_to_uri implementations, but
+# we can tell Python directly that it needs to preserve the //.
+if "itms-services" not in urllib.parse.uses_netloc:
+    urllib.parse.uses_netloc.append("itms-services")
 
 
 def _decode_idna(domain: str) -> str:
diff --git a/src/werkzeug/wrappers/response.py b/src/werkzeug/wrappers/response.py
--- a/src/werkzeug/wrappers/response.py
+++ b/src/werkzeug/wrappers/response.py
@@ -14,7 +14,6 @@
 from ..http import parse_range_header
 from ..http import remove_entity_headers
 from ..sansio.response import Response as _SansIOResponse
-from ..urls import _invalid_iri_to_uri
 from ..urls import iri_to_uri
 from ..utils import cached_property
 from ..wsgi import _RangeWrapper
@@ -479,7 +478,7 @@ def get_wsgi_headers(self, environ: WSGIEnvironment) -> Headers:
                 content_length = value
 
         if location is not None:
-            location = _invalid_iri_to_uri(location)
+            location = iri_to_uri(location)
 
             if self.autocorrect_location_header:
                 # Make the location header an absolute URL.


Test patch:
diff --git a/tests/test_urls.py b/tests/test_urls.py
--- a/tests/test_urls.py
+++ b/tests/test_urls.py
@@ -98,3 +98,9 @@ def test_iri_to_uri_dont_quote_valid_code_points():
     # [] are not valid URL code points according to WhatWG URL Standard
     # https://url.spec.whatwg.org/#url-code-points
     assert urls.iri_to_uri("/path[bracket]?(paren)") == "/path%5Bbracket%5D?(paren)"
+
+
+# Python < 3.12
+def test_itms_services() -> None:
+    url = "itms-services://?action=download-manifest&url=https://test.example/path"
+    assert urls.iri_to_uri(url) == url
diff --git a/tests/test_wrappers.py b/tests/test_wrappers.py
--- a/tests/test_wrappers.py
+++ b/tests/test_wrappers.py
@@ -1154,6 +1154,7 @@ class MyResponse(wrappers.Response):
     ("auto", "location", "expect"),
     (
         (False, "/test", "/test"),
+        (False, "/\\\\test.example?q", "/%5C%5Ctest.example?q"),
         (True, "/test", "http://localhost/test"),
         (True, "test", "http://localhost/a/b/test"),
         (True, "./test", "http://localhost/a/b/test"),