Repository: pallets/werkzeug

Issue:
deprecation of `safe_conversion` in `iri_to_uri` breaks `itms-services:` URIs
itms-services whilst undocumented is a validly used scheme and is used to install Enterprise or Ad-Hoc iOS Apps.

The fix for issue #2609 breaks this.

- Python version: 3.9.16
- Werkzeug version: 2.3.3



Discussion:
Please provide official documentation about the format of the `itms-services:` URI scheme. Additionally, in my searches that did not turn up anything, it seemed like Apple may have switched to other URI schemes that did not have the same issue; please verify that `itms-services` is not superseded by another scheme.
You can also use direct links instead of redirects:

```html
<a href="itms-services:...">Click here to install the app.</a>
```

Or a native UI button, etc.  This does not require a redirect to navigate to.
And as the linked PR mentions, please report this as a bug in `urlunsplit` to Python.
Here is a link to the Apple documentation (look for the "Use a website to distribute the app" section).
https://support.apple.com/en-gb/guide/deployment/depce7cefc4d/web

The main issue is that I generate a unique link with a short lived token in order to allow the download. This is generated by the backend and a redirect is done over to the itms-services link.
At the end of the day though, if you're using the same value for the `itms-services` URI each time, then someone could copy the redirect location instead and not have to go through that redirect next time. Why not make the unique URL part of the original link:

```html
<a href="itms-services://?action=download-manifest&url=https://example.com/manifest.plist&token=abcde">Download</a>
```

```python
token = request.args["token"]

if signer.loads(token, max_age=600):
    return send_file("manifest.plist")
else:
    abort(401)
```

Now the direct link is guarded by a [token](https://itsdangerous.palletsprojects.com), rather than only protecting the initial link but not the real URI.
Not quite - I generate a random token which expires after say 10 seconds -  just enough to redirect and validate it.
Something like 
itms-services://?action=download-manifest&url=https://theacmeinc.com/abcdefeg

Thats currently generated when you click an install button, and its currently a simple redirect over to that link which that installs the app associated with the token.

Any attempt to reuse that token or use it after its expired returns invalid.

I can of course re-write it and do it totally differently but this has been working nicely since about 2010.
Did you report this to Python?
Basically, this one (invalid?) URI scheme that's unlikely to be used in most applications requires extra code on every single response, which is not great. It requires a call to `encode` (which will require exception handling overhead if it actually is an IRI with non-ASCII chars), and a call to `str.split` to look for unlikely spaces. This is unecessary work in the vast, vast majority of cases, and part of the motivation of refactoring our URL parsing was improving speed.

I'm open to fixing this, but it could either be fixed in Python for everyone, or we could figure out a lower impact solution. I don't plan to undeprecate the original solution.
Would it be possible to implement a flag on the response to just pass the redirect url through unchanged - would be more than happy to default that to no and just have an exception path which may well be slower for that as as you mentioned, its only a very tiny number of urls that would be affected.
That's the other idea I mention in the linked issue. It would be nice if `autocorrect_location_header` could be refactored rather than adding a new flag, but it's not clear how to do that.
It looks like you haven't reported this to Python yet. Please create an issue https://github.com/python/cpython/issues/ and link it here so I can see that it's been reported upstream and track that.
For now, I will add back in the special case for the location header, but keep the parameter itself deprecated. In 2.4 I will add in an attribute like `response.normalize_location`, and either get rid of or rename `response.autocorrect_location_header` (`absolute_location`?).

Resolving patch:
diff --git a/src/werkzeug/urls.py b/src/werkzeug/urls.py
--- a/src/werkzeug/urls.py
+++ b/src/werkzeug/urls.py
@@ -1053,6 +1053,27 @@ def iri_to_uri(
     return urlunsplit((parts.scheme, netloc, path, query, fragment))
 
 
+def _invalid_iri_to_uri(iri: str) -> str:
+    """The URL scheme ``itms-services://`` must contain the ``//`` even though it does
+    not have a host component. There may be other invalid schemes as well. Currently,
+    responses will always call ``iri_to_uri`` on the redirect ``Location`` header, which
+    removes the ``//``. For now, if the IRI only contains ASCII and does not contain
+    spaces, pass it on as-is. In Werkzeug 2.4, this should become a
+    ``response.process_location`` flag.
+
+    :meta private:
+    """
+    try:
+        iri.encode("ascii")
+    except UnicodeError:
+        pass
+    else:
+        if len(iri.split(None, 1)) == 1:
+            return iri
+
+    return iri_to_uri(iri)
+
+
 def url_decode(
     s: t.AnyStr,
     charset: str = "utf-8",
diff --git a/src/werkzeug/utils.py b/src/werkzeug/utils.py
--- a/src/werkzeug/utils.py
+++ b/src/werkzeug/utils.py
@@ -260,21 +260,17 @@ def redirect(
         response. The default is :class:`werkzeug.wrappers.Response` if
         unspecified.
     """
-    from .urls import iri_to_uri
-
     if Response is None:
         from .wrappers import Response
 
-    display_location = escape(location)
-    location = iri_to_uri(location)
+    html_location = escape(location)
     response = Response(  # type: ignore[misc]
         "<!doctype html>\n"
         "<html lang=en>\n"
         "<title>Redirecting...</title>\n"
         "<h1>Redirecting...</h1>\n"
         "<p>You should be redirected automatically to the target URL: "
-        f'<a href="{escape(location)}">{display_location}</a>. If'
-        " not, click the link.\n",
+        f'<a href="{html_location}">{html_location}</a>. If not, click the link.\n',
         code,
         mimetype="text/html",
     )
diff --git a/src/werkzeug/wrappers/response.py b/src/werkzeug/wrappers/response.py
--- a/src/werkzeug/wrappers/response.py
+++ b/src/werkzeug/wrappers/response.py
@@ -8,6 +8,7 @@
 from ..datastructures import Headers
 from ..http import remove_entity_headers
 from ..sansio.response import Response as _SansIOResponse
+from ..urls import _invalid_iri_to_uri
 from ..urls import iri_to_uri
 from ..utils import cached_property
 from ..wsgi import ClosingIterator
@@ -478,11 +479,11 @@ def get_wsgi_headers(self, environ: WSGIEnvironment) -> Headers:
             elif ikey == "content-length":
                 content_length = value
 
-        # make sure the location header is an absolute URL
         if location is not None:
-            location = iri_to_uri(location)
+            location = _invalid_iri_to_uri(location)
 
             if self.autocorrect_location_header:
+                # Make the location header an absolute URL.
                 current_url = get_current_url(environ, strip_querystring=True)
                 current_url = iri_to_uri(current_url)
                 location = urljoin(current_url, location)


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -1,3 +1,5 @@
+from __future__ import annotations
+
 import inspect
 from datetime import datetime
 
@@ -9,48 +11,32 @@
 from werkzeug.http import http_date
 from werkzeug.http import parse_date
 from werkzeug.test import Client
+from werkzeug.test import EnvironBuilder
 from werkzeug.wrappers import Response
 
 
-def test_redirect():
-    resp = utils.redirect("/füübär")
-    assert resp.headers["Location"] == "/f%C3%BC%C3%BCb%C3%A4r"
-    assert resp.status_code == 302
-    assert resp.get_data() == (
-        b"<!doctype html>\n"
-        b"<html lang=en>\n"
-        b"<title>Redirecting...</title>\n"
-        b"<h1>Redirecting...</h1>\n"
-        b"<p>You should be redirected automatically to the target URL: "
-        b'<a href="/f%C3%BC%C3%BCb%C3%A4r">/f\xc3\xbc\xc3\xbcb\xc3\xa4r</a>. '
-        b"If not, click the link.\n"
-    )
+@pytest.mark.parametrize(
+    ("url", "code", "expect"),
+    [
+        ("http://example.com", None, "http://example.com"),
+        ("/füübär", 305, "/f%C3%BC%C3%BCb%C3%A4r"),
+        ("http://☃.example.com/", 307, "http://xn--n3h.example.com/"),
+        ("itms-services://?url=abc", None, "itms-services://?url=abc"),
+    ],
+)
+def test_redirect(url: str, code: int | None, expect: str) -> None:
+    environ = EnvironBuilder().get_environ()
 
-    resp = utils.redirect("http://☃.net/", 307)
-    assert resp.headers["Location"] == "http://xn--n3h.net/"
-    assert resp.status_code == 307
-    assert resp.get_data() == (
-        b"<!doctype html>\n"
-        b"<html lang=en>\n"
-        b"<title>Redirecting...</title>\n"
-        b"<h1>Redirecting...</h1>\n"
-        b"<p>You should be redirected automatically to the target URL: "
-        b'<a href="http://xn--n3h.net/">http://\xe2\x98\x83.net/</a>. '
-        b"If not, click the link.\n"
-    )
+    if code is None:
+        resp = utils.redirect(url)
+        assert resp.status_code == 302
+    else:
+        resp = utils.redirect(url, code)
+        assert resp.status_code == code
 
-    resp = utils.redirect("http://example.com/", 305)
-    assert resp.headers["Location"] == "http://example.com/"
-    assert resp.status_code == 305
-    assert resp.get_data() == (
-        b"<!doctype html>\n"
-        b"<html lang=en>\n"
-        b"<title>Redirecting...</title>\n"
-        b"<h1>Redirecting...</h1>\n"
-        b"<p>You should be redirected automatically to the target URL: "
-        b'<a href="http://example.com/">http://example.com/</a>. '
-        b"If not, click the link.\n"
-    )
+    assert resp.headers["Location"] == url
+    assert resp.get_wsgi_headers(environ)["Location"] == expect
+    assert resp.get_data(as_text=True).count(url) == 2
 
 
 def test_redirect_xss():