Repository: pallets/werkzeug

Issue:
app.route mixes up parameters passed when multiple regex patterns are used
We recently upgraded from Flask v1.1.2 to Flask v2.2.2 and were surprised when many of our project unit tests started failing because Flask v2.2.2 is not passing the correct parameters to a route function.  This worked correctly in the previous v1.1.2 which we were using but no longer works properly in v2.2.2.

The following python code example of a route which passes 3 parameters with regex expressions will replicate the bug:

```python
ALLOWED_METHODS = ["GET", "HEAD"]
DATE_PATTERN = "((\d{8}T\d{6}([.,]\d{1,3})?)|(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}([.,]\d{1,3})?))Z"
# Note that 2 date formats are supported using this date pattern
# if you want to see what this regex does, paste it into https://regexper.com/

@app.route('/<regex("\d+"):source_id>/'
        + '<regex("' + DATE_PATTERN + '"):start_time>/'
        + '<regex("' + DATE_PATTERN + '"):end_time>', methods=ALLOWED_METHODS)
def clipslist_id(source_id, start_time, end_time):
    print("clipslist_id parameters source_id {} start_time {} and end time {}".format(source_id, start_time, end_time))
    print("Valid parameters {}".format(end_time > start_time))
    return
```

Call the app.get function and pass in a uri such one of the following (two date formats are supported):

```
/22/2023-02-16T23:36:36.266Z/2023-02-16T23:46:36.266Z
/22/20230217T002724.998Z/20230217T003724.998Z
```

Note that in both of the example uri's above the 3rd datetime parameter value is 10 minutes after the 2nd datetime parameter value
When the uri gets routed to clipslist_id the function receives 3 parameters, but the 3rd parameter is not correct.  Instead of passing through the correct 3rd parameter the route code passes the 2nd parameter twice, once as the 2nd parameter, and again as the 3rd parameter.

The three parameters received by the function should have had the values for: source_id, start_time, and end_time
Instead the three parameters that the function received were: source_id, start_time, start_time
The end_time value was not passed to the function

Environment:
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
Flask 2.2.2

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.5 LTS
Release:	20.04
Codename:	focal
5.15.0-58-generic



Discussion:
Probably solved by https://github.com/pallets/werkzeug/issues/2582 which will be in 2.3 when it's released. It's also a duplicate of #2491, #2518, #2519, and possibly some others.

Quoting from my comment on #2491:

> Note that the `regex` converter is not provided by Werkzeug. You should report an issue to the library providing that converter.


I looked at the code change you referred to in #2582 and I see that it checks for a / character in the regex expression and then changes a `part_isolating` flag to false.

In the example I gave above none of the regex expressions contain a / character.  I am not sure how the `part_isolating` flag alters the processing and it was not obvious to me whether it should be set to False or True when all the regex expressions do *not* contain slashes.

I tried to test this in our code by attempting to explicitly modify the `part_isolating` flag in the `__init__` function.  I experimented by setting it to both False and True, but the app.route code continued to pass the wrong parameter values into the route handing function for every variation that I tried.

This is the code where I attempted to change the `part_isolating` flag to work around this issue. 

```
 class RegexConverter(BaseConverter):
    """
    This class is used to covert a Regex pattern to a Flask Regex pattern
    """
    def __init__(self, url_map, *items):
        super(RegexConverter, self).__init__(url_map)
        RegexConverter.part_isolating = False
        self.part_isolating = False
        self.regex = items[0]

app.url_map.converters["regex"] = RegexConverter
```

I also tried the opposite

```
class RegexConverter(BaseConverter):
    """
    This class is used to covert a Regex pattern to a Flask Regex pattern
    """
    def __init__(self, url_map, *items):
        super(RegexConverter, self).__init__(url_map)
        RegexConverter.part_isolating = True
        self.part_isolating = True
        self.regex = items[0]
```

Is there a different way that I should have attempted to change the `part_isolating` flag to work around this issue?  Or is this issue not resolved by changing the value of the `part_isolating` flag?
I believe this is because your regex has additional matching groups within it - could you rewrite your regex to remove these (so that there is only one matching group)?
https://regexper.com/ shows that the date pattern regex expression we are using has 5 matching groups.

![image](https://user-images.githubusercontent.com/60823338/220483729-6f32d3d9-5d2c-4cf5-9727-5c10f6d610d3.png)

It does not appear that the date pattern regex could be rewritten to have only one matching group.  We need to support two different date formats.  Even supporting only one date format would still require a regex pattern with multiple matching groups.

This route logic has been working properly for us for a long time while using the previous version of Flask.  It is only the new version of Flask which exhibits the incorrect parameter passing issue to the route handler function.

The regex pattern matching portion of the logic seems to work correctly with the new Flask version.  We have many routes registered and the proper route gets matched correctly and the proper route handling function is getting called each time.

The bug seems to be in the logic which passes the parameter values to the route handling function.  For some reason this logic is now passing incorrect parameter values to the route handling function with the new Flask version.

In our code base I have identified 5 separate routes so far where the new version of Flask is passing the parameters incorrectly to the route handling function.  In each of these 5 routes at least one of the parameters uses the regex date pattern to match the expected uri data for that parameter.  

As you suggested the date pattern regex we are using does seem to be a consistent factor which triggers the bug that causes incorrect parameter values to be passed to the route handling function.

Resolving patch:
diff --git a/src/werkzeug/routing/matcher.py b/src/werkzeug/routing/matcher.py
--- a/src/werkzeug/routing/matcher.py
+++ b/src/werkzeug/routing/matcher.py
@@ -127,13 +127,21 @@ def _match(
                     remaining = []
                 match = re.compile(test_part.content).match(target)
                 if match is not None:
-                    groups = list(match.groups())
                     if test_part.suffixed:
                         # If a part_isolating=False part has a slash suffix, remove the
                         # suffix from the match and check for the slash redirect next.
-                        suffix = groups.pop()
+                        suffix = match.groups()[-1]
                         if suffix == "/":
                             remaining = [""]
+
+                    converter_groups = sorted(
+                        match.groupdict().items(), key=lambda entry: entry[0]
+                    )
+                    groups = [
+                        value
+                        for key, value in converter_groups
+                        if key[:11] == "__werkzeug_"
+                    ]
                     rv = _match(new_state, remaining, values + groups)
                     if rv is not None:
                         return rv
diff --git a/src/werkzeug/routing/rules.py b/src/werkzeug/routing/rules.py
--- a/src/werkzeug/routing/rules.py
+++ b/src/werkzeug/routing/rules.py
@@ -588,6 +588,7 @@ def _parse_rule(self, rule: str) -> t.Iterable[RulePart]:
         argument_weights = []
         static_weights: t.List[t.Tuple[int, int]] = []
         final = False
+        convertor_number = 0
 
         pos = 0
         while pos < len(rule):
@@ -614,7 +615,8 @@ def _parse_rule(self, rule: str) -> t.Iterable[RulePart]:
                 self.arguments.add(data["variable"])
                 if not convobj.part_isolating:
                     final = True
-                content += f"({convobj.regex})"
+                content += f"(?P<__werkzeug_{convertor_number}>{convobj.regex})"
+                convertor_number += 1
                 argument_weights.append(convobj.weight)
                 self._trace.append((True, data["variable"]))
 
@@ -643,6 +645,7 @@ def _parse_rule(self, rule: str) -> t.Iterable[RulePart]:
                     argument_weights = []
                     static_weights = []
                     final = False
+                    convertor_number = 0
 
             pos = match.end()
 


Test patch:
diff --git a/tests/test_routing.py b/tests/test_routing.py
--- a/tests/test_routing.py
+++ b/tests/test_routing.py
@@ -778,6 +778,35 @@ def to_url(self, values):
     assert kwargs["foo"] == ("qwert", "yuiop")
 
 
+def test_nested_regex_groups():
+    """
+    Regression test for https://github.com/pallets/werkzeug/issues/2590
+    """
+
+    class RegexConverter(r.BaseConverter):
+        def __init__(self, url_map, *items):
+            super().__init__(url_map)
+            self.part_isolating = False
+            self.regex = items[0]
+
+    # This is a regex pattern with nested groups
+    DATE_PATTERN = r"((\d{8}T\d{6}([.,]\d{1,3})?)|(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}([.,]\d{1,3})?))Z"  # noqa: B950
+
+    map = r.Map(
+        [
+            r.Rule(
+                f"/<regex('{DATE_PATTERN}'):start>/<regex('{DATE_PATTERN}'):end>/",
+                endpoint="handler",
+            )
+        ],
+        converters={"regex": RegexConverter},
+    )
+    a = map.bind("example.org", "/")
+    route, kwargs = a.match("/2023-02-16T23:36:36.266Z/2023-02-16T23:46:36.266Z/")
+    assert kwargs["start"] == "2023-02-16T23:36:36.266Z"
+    assert kwargs["end"] == "2023-02-16T23:46:36.266Z"
+
+
 def test_anyconverter():
     m = r.Map(
         [