Repository: falconry/falcon

Issue:
RequestOptions.auto_parse_qs_csv documentation is misleading for the True case
[`uri.parse_query_string`](https://falcon.readthedocs.io/en/stable/api/util.html#falcon.uri.parse_query_string) supports splitting parameter values on comma, however, only the first occurrence is actually split:
```python
>>> uri.parse_query_string('interesting&stuff=1,2&key=value&stuff=3,4,5', csv=True)
{'stuff': ['1', '2', '3,4,5'], 'key': 'value'}
```

This is a known and documented limitation of `uri.parse_query_string`:
> The two different ways of specifying lists may not be mixed in a single query string for the same parameter.

However, documentation for [`RequestOptions.auto_parse_qs_csv`](https://falcon.readthedocs.io/en/stable/api/api.html#falcon.RequestOptions.auto_parse_qs_csv) mentions neither this limitation nor the fact that `uri.parse_query_string` is used for parsing. Moreover, it provides a misleading (although the example values are technically correct since only the first occurrence of `t` has commas) example:
> When `auto_parse_qs_csv` is set to ``True``, the query string value
> is also split on non-percent-encoded commas and these items
> are added to the final list (i.e. ``/?t=1,2,3&t=4``
> becomes ``['1', '2', '3', '4']``).


Discussion:
Hi :wave:,

Thanks for using Falcon. The large amount of time and effort needed to
maintain the project and develop new features is not sustainable without
the generous financial support of community members like you.

Please consider helping us secure the future of the Falcon framework with a
one-time or recurring donation.

Thank you for your support!

Also I found another bug with `auto_parse_qs_csv`

If a http client send quoted query string it doesn't work

for example
```python
print(req.query_string)
print(req.params)
t=1%2C2%2C3
{'t': ['1,2,3']}
```
but
```python
print(parse_query_string(unquote(req.query_string), csv=True))
{'t': ['1', '2', '3']}
```

@vytas7 I can help in this issue, both adding the warning for the limitation of `parse_query_string` and taking a look at the bug @frad00r4 brought up to see if it makes sense to address it in this issue.
@danilito19 Awesome! That would be much appreciated :+1: 

Re the issue @frad00r4 brought up, I'm not sure whether this is a bug or a feature. If using csv params, quoting is an easy way to escape commas, so that could be a massively breaking change for people relying on this.
See also discussions on https://github.com/falconry/falcon/issues/1547 , and my last comment therein: https://github.com/falconry/falcon/issues/1547#issuecomment-668753529
Ah yes I see what you mean. I'll stick to the original intent of the issue.
> When `auto_parse_qs_csv` is set to `True`, the query string value
> is also split on non-percent-encoded commas and these items
> are added to the final list (i.e. `/?t=1,2,3&t=4`
> becomes `['1', '2', '3', '4']`).

@vytas7 I think the above example is also incorrect. In the REPL I tested out
```
>>> parse_query_string('/?t=1,2,3&t=4', csv=True)
{'/?t': ['1', '2', '3'], 't': '4'}
```
so it's clearly not joining the `4` in
@danilito19 [`parse_query_string`](https://falcon.readthedocs.io/en/latest/api/util.html#falcon.uri.parse_query_string) is supposed to only be used for the query string _after_ the question mark character, i.e.,
```python
>>> from falcon.uri import parse_query_string
>>> parse_query_string('t=1,2,3&t=4', csv=True)
{'t': ['1', '2', '3', '4']}
>>> # However...
... 
>>> parse_query_string('t=1,2&t=3,4', csv=True)
{'t': ['1', '2', '3,4']}
```

But maybe the docs are not 100% technically correct either in this case, if they managed to confuse you :thinking:
> within the query string (i.e. /?t=1,2,3&t=4 ...

The query string is `t=1,2,3&t=4`, not `/?t=1,2,3&t=4`, in this case.

While at it, we could clean up the text to use the correct terms that are consistent with, for instance, the [`Request` object](https://falcon.readthedocs.io/en/latest/api/request_and_response_wsgi.html#request). E.g., given the URL https://foo.bar/some/thing?t=1,2,3&t=4, we should be referring to
* `/some/thing?t=1,2,3&t=4` -- the relative URI
* `/some/thing` -- the request path
* `t=1,2,3&t=4` -- the query string

Resolving patch:
diff --git a/falcon/request.py b/falcon/request.py
--- a/falcon/request.py
+++ b/falcon/request.py
@@ -1957,14 +1957,17 @@ class RequestOptions:
                 also http://goo.gl/6rlcux).
 
         auto_parse_qs_csv: Set to ``True`` to split query string values on
-            any non-percent-encoded commas (default ``False``). When ``False``,
+            any non-percent-encoded commas (default ``False``).
+
+            When ``False``,
             values containing commas are left as-is. In this mode, list items
             are taken only from multiples of the same parameter name within the
-            query string (i.e. ``/?t=1,2,3&t=4`` becomes ``['1,2,3', '4']``).
+            query string (i.e. ``t=1,2,3&t=4`` becomes ``['1,2,3', '4']``).
+
             When `auto_parse_qs_csv` is set to ``True``, the query string value
             is also split on non-percent-encoded commas and these items
-            are added to the final list (i.e. ``/?t=1,2,3&t=4``
-            becomes ``['1', '2', '3', '4']``).
+            are added to the final list (i.e. ``t=1,2,3&t=4,5``
+            becomes ``['1', '2', '3', '4', '5']``).
 
             Warning:
                 Enabling this option will cause the framework to misinterpret
diff --git a/falcon/util/uri.py b/falcon/util/uri.py
--- a/falcon/util/uri.py
+++ b/falcon/util/uri.py
@@ -346,13 +346,36 @@ def parse_query_string(query_string, keep_blank=False, csv=True):
             # a list, or append the next value to the list.
             old_value = params[k]
 
-            if is_encoded:
-                v = decode(v)
+            if csv and ',' in v:
+                # NOTE(kgriffs): Falcon supports a more compact form of
+                # lists, in which the elements are comma-separated and
+                # assigned to a single param instance. If it turns out that
+                # very few people use this, it can be deprecated at some
+                # point.
+                v = v.split(',')
+
+                if not keep_blank:
+                    # NOTE(kgriffs): Normalize the result in the case that
+                    # some elements are empty strings, such that the result
+                    # will be the same for 'foo=1,,3' as 'foo=1&foo=&foo=3'.
+                    additional_values = [decode(element) for element in v if element]
+                else:
+                    additional_values = [decode(element) for element in v]
+
+                if isinstance(old_value, list):
+                    old_value.extend(additional_values)
+                else:
+                    additional_values.insert(0, old_value)
+                    params[k] = additional_values
 
-            if isinstance(old_value, list):
-                old_value.append(v)
             else:
-                params[k] = [old_value, v]
+                if is_encoded:
+                    v = decode(v)
+
+                if isinstance(old_value, list):
+                    old_value.append(v)
+                else:
+                    params[k] = [old_value, v]
 
         else:
             if csv and ',' in v:


Test patch:
diff --git a/tests/test_query_params.py b/tests/test_query_params.py
--- a/tests/test_query_params.py
+++ b/tests/test_query_params.py
@@ -182,6 +182,60 @@ def test_option_auto_parse_qs_csv_simple_true(self, simulate_request, client, re
         assert req.get_param('id') in ['23', '42', '2']
         assert req.get_param_as_list('id', int) == [23, 42, 2]
 
+    def test_option_auto_parse_qs_csv_multiple_fields_false(
+        self,
+        simulate_request,
+        client,
+        resource
+    ):
+        client.app.add_route('/', resource)
+
+        query_string = 't=1,2&t=3,4'
+        simulate_request(client=client, path='/', query_string=query_string)
+
+        req = resource.captured_req
+
+        assert req.params['t'] == ['1,2', '3,4']
+        assert req.get_param('t') in ['1,2', '3,4']
+        assert req.get_param_as_list('t') == ['1,2', '3,4']
+
+    @pytest.mark.parametrize(
+        'qs, keep_blank, expected',
+        [
+            ('t=1&t=3,4', False, ['1', '3', '4']),
+            ('t=1&t=2&t=3,4', False, ['1', '2', '3', '4']),
+            ('t=1,2&t=3,4', False, ['1', '2', '3', '4']),
+            ('t=1,,2&t=3,4', False, ['1', '2', '3', '4']),
+            ('t=1,,2&t=3,4', True, ['1', '', '2', '3', '4']),
+            ('t=1,2&t=3,4,,5', False, ['1', '2', '3', '4', '5']),
+            ('t=1&t=,1,4,,5', False, ['1', '1', '4', '5']),
+            ('t=1&t=,1,4,,5', True, ['1', '', '1', '4', '', '5']),
+            ('t=1&t=,1,4,,5&t=a,b,c', True, ['1', '', '1', '4', '', '5', 'a', 'b', 'c']),
+        ]
+    )
+    def test_option_auto_parse_qs_csv_multiple_fields_true(
+        self,
+        simulate_request,
+        client,
+        resource,
+
+        qs,
+        keep_blank,
+        expected,
+    ):
+        client.app.add_route('/', resource)
+        client.app.req_options.auto_parse_qs_csv = True
+        client.app.req_options.keep_blank_qs_values = keep_blank
+
+        query_string = qs
+        simulate_request(client=client, path='/', query_string=query_string)
+
+        req = resource.captured_req
+
+        assert req.params['t'] == expected
+        assert req.get_param('t') in expected
+        assert req.get_param_as_list('t') == expected
+
     def test_option_auto_parse_qs_csv_complex_false(self, simulate_request, client, resource):
         client.app.add_route('/', resource)
         client.app.req_options.auto_parse_qs_csv = False