Repository: marshmallow-code/webargs

Issue:
'Not a valid tuple.'  when trying to use marshmallow fields.Tuple for argument validation
I'm trying to use the marshmallow fields.Tuple for querystring argument validation on a GET request using Flask.  The issue I'm running into is that no matter what type of object I declare and no matter what I use in the request, I always get the default 'Not a valid tuple.' response.  I have tried using a tuple of size 1 and 2; using fields.String and/or fields.Integer, etc with the same result.

- I'm using Python 3.6.9 with these dependencies:
anyjson==0.3.3
apipkg==1.5
arrow==0.15.5
attrs==19.3.0
backports.functools-lru-cache==1.6.1
cassandra-driver==3.22.0
Cerberus==1.3.2
certifi==2019.11.28
cffi==1.13.2
chardet==3.0.4
click==7.1.1
execnet==1.7.1
Flask==1.1.1
Flask-Cors==3.0.8
funcsigs==1.0.2
futures==3.1.1
geomet==0.1.2
gevent==1.4.0
greenlet==0.4.13
gunicorn==20.0.4
idna==2.9
importlib-metadata==1.6.0
itsdangerous==1.1.0
Jinja2==2.11.1
jsonklog==0.15.0
MarkupSafe==1.1.1
marshmallow==3.5.1
neurolab==0.3.5
numpy==1.18.1
pluggy==0.13.1
py==1.8.1
pyaml==20.3.1
pymongo==3.10.1
pytest==3.3.0
pytest-forked==0.2
pytest-xdist==1.20.1
python-dateutil==2.8.1
PyYAML==5.3.1
readline==6.2.4.1
requests==2.23.0
six==1.14.0
urllib3==1.25.8
webargs==6.0.0
Werkzeug==1.0.0
zipp==3.1.0

- Here is an example of what I'm trying to do:
```
from flask import Flask
from webargs.flaskparser import parser, use_kwargs
from marshmallow import EXCLUDE, fields, Schema


app = Flask(__name__)


@app.errorhandler(422)
def custom_handler(error):
    errors = []
    if 'query' in error.data['messages']:
        for arg in error.data['messages']['query']:
            for item in error.data['messages']['query'][arg]:
                errors.append(item)
    return str(errors), 400


class test_schema(Schema):
    class Meta:
        unknown = EXCLUDE
        strict = True
    
    test_tup = fields.Tuple((fields.Integer(required=True), fields.Integer(required=True)), required=True)


@app.route('/test/', strict_slashes=False)
@parser.use_kwargs(test_schema, location='query')
def test_the_mallow(**kwargs):
    return "True"
```

- Finally, here are a couple example url's I've tried:
  localhost:2300/test/?test_tup=[0,0]
  localhost:2300/test/?test_tup=(0,0)
  localhost:2300/test/?test_tup=0,0


Discussion:
sirosen: > I suppose you added a new commit to help me reviewing but you'd like to squash before merging.

Yep! I'm happy to squash things together now. I just figured the added diff would make it easier.

> I can do the squash / update changelog / release later when I get the time.

I'm also happy to do this. As a relatively new contributor, I don't want to be too pushy about my changes or doing releases -- let me know if I'm being too tentative.

I'll do a rebase and add a changelog entry, so this should simply be a merge + tweak to the changelog (to add the release date) when the time comes.

Patch:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 1e1bfda..d5e3c65 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -1,6 +1,17 @@
 Changelog
 ---------
 
+6.1.0 (Unreleased)
+******************
+
+Features:
+
+* Add ``fields.DelimitedTuple`` when using marshmallow 3. This behaves as a
+  combination of ``fields.DelimitedList`` and ``marshmallow.fields.Tuple``. It
+  takes an iterable of fields, plus a delimiter (defaults to ``,``), and parses
+  delimiter-separated strings into tuples.
+
+
 6.0.0 (2020-02-27)
 ******************
 
diff --git a/docs/upgrading.rst b/docs/upgrading.rst
index 9879692..147759f 100644
--- a/docs/upgrading.rst
+++ b/docs/upgrading.rst
@@ -11,15 +11,15 @@ Multiple Locations Are No Longer Supported In A Single Call
 
 The default location is JSON/body.
 
-Under webargs 5.x, code often did not have to specify a location.
+Under webargs 5.x, you may have written code which did not specify a location.
 
-Because webargs would parse data from multiple locations automatically, users
-did not need to specify where a parameter, call it `q`, was passed.
+Because webargs would parse data from multiple locations automatically, you did
+not need to specify where a parameter, call it `q`, was passed.
 `q` could be in a query parameter or in a JSON or form-post body.
 
-Now, webargs requires that users specify only one location for data loading per
+Now, webargs requires that you specify only one location for data loading per
 `use_args` call, and `"json"` is the default. If `q` is intended to be a query
-parameter, the developer must be explicit and rewrite like so:
+parameter, you must be explicit and rewrite like so:
 
 .. code-block:: python
 
@@ -34,9 +34,9 @@ parameter, the developer must be explicit and rewrite like so:
     def foo(args):
         return some_function(user_query=args.get("q"))
 
-This also means that another usage from 5.x is not supported. Code with
-multiple locations in a single `use_args`, `use_kwargs`, or `parse` call
-must be rewritten in multiple separate `use_args` or `use_kwargs` invocations,
+This also means that another usage from 5.x is not supported. If you had code
+with multiple locations in a single `use_args`, `use_kwargs`, or `parse` call,
+you must write it in multiple separate `use_args` or `use_kwargs` invocations,
 like so:
 
 .. code-block:: python
@@ -87,11 +87,9 @@ location_handler Has Been Replaced With location_loader
 This is not just a name change. The expected signature of a `location_loader`
 is slightly different from the signature for a `location_handler`.
 
-Where previously a `location_handler` code took the incoming request data and
-details of a single field being loaded, a `location_loader` takes the request
-and the schema as a pair. It does not return a specific field's data, but data
-for the whole location.
-
+Where previously your code took the incoming request data and details of a
+single field being loaded, it now takes the request and the schema as a pair.
+It does not return a specific field's data, but data for the whole location.
 Rewrite code like this:
 
 .. code-block:: python
@@ -107,8 +105,8 @@ Rewrite code like this:
     def load_data(request, schema):
         return request.data
 
-Data Is Not Filtered Before Being Passed To Schemas, And It May Be Proxified
-----------------------------------------------------------------------------
+Data Is Not Filtered Before Being Passed To Your Schema, And It May Be Proxified
+--------------------------------------------------------------------------------
 
 In webargs 5.x, the deserialization schema was used to pull data out of the
 request object. That data was compiled into a dictionary which was then passed
@@ -119,8 +117,16 @@ on schemas. This lets a schema decide what to do with fields not specified in
 the schema. In order to achieve this, webargs now passes the full data from
 the specified location to the schema.
 
-Therefore, users should specify `unknown=marshmallow.EXCLUDE` on their schemas in
-order to filter out unknown fields. Like so:
+However, many types of request data are so-called "multidicts" -- dictionary-like
+types which can return one or multiple values as you prefer. To handle
+`marshmallow.fields.List` and `webargs.fields.DelimitedList` fields correctly,
+passing list data, webargs must combine schema information with the raw request
+data. This is done in the
+:class:`MultiDictProxy <webargs.multidictproxy.MultiDictProxy>` type, which
+will often be passed to your schemas.
+
+Therefore, you should specify `unknown=marshmallow.EXCLUDE` on your schemas if
+you want to filter out unknown fields. Like so:
 
 .. code-block:: python
 
@@ -155,7 +161,8 @@ order to filter out unknown fields. Like so:
         ...
 
 
-This also allows usage which passes the unknown parameters through, like so:
+This also allows you to write code which passes the unknown parameters through,
+like so:
 
 .. code-block:: python
 
@@ -170,17 +177,11 @@ This also allows usage which passes the unknown parameters through, like so:
         ...
 
 
-However, many types of request data are so-called "multidicts" -- dictionary-like
-types which can return one or multiple values. To handle `marshmallow.fields.List`
-and `webargs.fields.DelimitedList` fields correctly, passing list data, webargs
-must combine schema information with the raw request data. This is done in the
-:class:`MultiDictProxy <webargs.multidictproxy.MultiDictProxy>` type, which
-will often be passed to schemas.
-
-This means that if a schema has a `pre_load` hook which interacts with the data,
+Finally, this change passes a proxy object where you once saw a dict. This
+means that if your schema has a `pre_load` hook which interacts with the data,
 it may need modifications. For example, a `flask` query string will be parsed
 into an `ImmutableMultiDict` type, which will break pre-load hooks which modify
-the data in-place. Such usages need rewrites like so:
+the data in-place. You may need to apply rewrites like so:
 
 .. code-block:: python
 
@@ -230,8 +231,8 @@ DelimitedList Now Only Takes A String Input
 -------------------------------------------
 
 Combining `List` and string parsing functionality in a single type had some
-messy corner cases. For the most part, this should not require rewrites. But
-for APIs which need to allow both usages, rewrites are possible like so:
+messy corner cases. For the most part, this should not require rewrites. But if
+you need to allow both usages in your API, you can do a rewrite like so:
 
 .. code-block:: python
 
@@ -273,14 +274,14 @@ for APIs which need to allow both usages, rewrites are possible like so:
 ValidationError Messages Are Namespaced Under The Location
 ----------------------------------------------------------
 
-Code parsing ValidationError messages will notice a change in the messages
-produced by webargs.
+If you were parsing ValidationError messages, you will notice a change in the
+messages produced by webargs.
 What would previously have come back with messages like `{"foo":["Not a valid integer."]}`
 will now have messages nested one layer deeper, like
 `{"json":{"foo":["Not a valid integer."]}}`.
 
-To rewrite code which was handling these errors, the handler will need to be
-prepared to traverse messages by one additional level. For example:
+To rewrite code which was handling these errors, you will need to be prepared
+to traverse messages by one additional level. For example:
 
 .. code-block:: python
 
diff --git a/src/webargs/fields.py b/src/webargs/fields.py
index 851756b..2227f3f 100644
--- a/src/webargs/fields.py
+++ b/src/webargs/fields.py
@@ -43,26 +43,24 @@ class Nested(ma.fields.Nested):
         super().__init__(nested, *args, **kwargs)
 
 
-class DelimitedList(ma.fields.List):
-    """A field which is similar to a List, but takes its input as a delimited
-    string (e.g. "foo,bar,baz").
+class DelimitedFieldMixin:
+    """
+    This is a mixin class for subclasses of ma.fields.List and ma.fields.Tuple
+    which split on a pre-specified delimiter. By default, the delimiter will be ","
 
-    Like List, it can be given a nested field type which it will use to
-    de/serialize each element of the list.
+    Because we want the MRO to reach this class before the List or Tuple class,
+    it must be listed first in the superclasses
 
-    :param Field cls_or_instance: A field class or instance.
-    :param str delimiter: Delimiter between values.
+    For example, a DelimitedList-like type can be defined like so:
+
+    >>> class MyDelimitedList(DelimitedFieldMixin, ma.fields.List):
+    >>>     pass
     """
 
-    default_error_messages = {"invalid": "Not a valid delimited list."}
     delimiter = ","
 
-    def __init__(self, cls_or_instance, *, delimiter=None, **kwargs):
-        self.delimiter = delimiter or self.delimiter
-        super().__init__(cls_or_instance, **kwargs)
-
     def _serialize(self, value, attr, obj):
-        # serializing will start with List serialization, so that we correctly
+        # serializing will start with parent-class serialization, so that we correctly
         # output lists of non-primitive types, e.g. DelimitedList(DateTime)
         return self.delimiter.join(
             format(each) for each in super()._serialize(value, attr, obj)
@@ -76,3 +74,45 @@ class DelimitedList(ma.fields.List):
             else:
                 raise self.make_error("invalid")
         return super()._deserialize(value.split(self.delimiter), attr, data, **kwargs)
+
+
+class DelimitedList(DelimitedFieldMixin, ma.fields.List):
+    """A field which is similar to a List, but takes its input as a delimited
+    string (e.g. "foo,bar,baz").
+
+    Like List, it can be given a nested field type which it will use to
+    de/serialize each element of the list.
+
+    :param Field cls_or_instance: A field class or instance.
+    :param str delimiter: Delimiter between values.
+    """
+
+    default_error_messages = {"invalid": "Not a valid delimited list."}
+    delimiter = ","
+
+    def __init__(self, cls_or_instance, *, delimiter=None, **kwargs):
+        self.delimiter = delimiter or self.delimiter
+        super().__init__(cls_or_instance, **kwargs)
+
+
+# DelimitedTuple can only be defined when using marshmallow3, when Tuple was
+# added
+if MARSHMALLOW_VERSION_INFO[0] >= 3:
+
+    class DelimitedTuple(DelimitedFieldMixin, ma.fields.Tuple):
+        """A field which is similar to a Tuple, but takes its input as a delimited
+        string (e.g. "foo,bar,baz").
+
+        Like Tuple, it can be given a tuple of nested field types which it will use to
+        de/serialize each element of the tuple.
+
+        :param Iterable[Field] tuple_fields: An iterable of field classes or instances.
+        :param str delimiter: Delimiter between values.
+        """
+
+        default_error_messages = {"invalid": "Not a valid delimited tuple."}
+        delimiter = ","
+
+        def __init__(self, tuple_fields, *, delimiter=None, **kwargs):
+            self.delimiter = delimiter or self.delimiter
+            super().__init__(tuple_fields, **kwargs)


Test patch:
diff --git a/tests/test_core.py b/tests/test_core.py
index fef3c65..a9a32e9 100644
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -851,7 +851,53 @@ def test_delimited_list_default_delimiter(web_request, parser):
     assert data["ids"] == "1,2,3"
 
 
-def test_delimited_list_as_string_v2(web_request, parser):
+@pytest.mark.skipif(
+    MARSHMALLOW_VERSION_INFO[0] < 3, reason="fields.Tuple added in marshmallow3"
+)
+def test_delimited_tuple_default_delimiter(web_request, parser):
+    """
+    Test load and dump from DelimitedTuple, including the use of a datetime
+    type (similar to a DelimitedList test below) which confirms that we aren't
+    relying on __str__, but are properly de/serializing the included fields
+    """
+    web_request.json = {"ids": "1,2,2020-05-04"}
+    schema_cls = dict2schema(
+        {
+            "ids": fields.DelimitedTuple(
+                (fields.Int, fields.Int, fields.DateTime(format="%Y-%m-%d"))
+            )
+        }
+    )
+    schema = schema_cls()
+
+    parsed = parser.parse(schema, web_request)
+    assert parsed["ids"] == (1, 2, datetime.datetime(2020, 5, 4))
+
+    data = schema.dump(parsed)
+    assert data["ids"] == "1,2,2020-05-04"
+
+
+@pytest.mark.skipif(
+    MARSHMALLOW_VERSION_INFO[0] < 3, reason="fields.Tuple added in marshmallow3"
+)
+def test_delimited_tuple_incorrect_arity(web_request, parser):
+    web_request.json = {"ids": "1,2"}
+    schema_cls = dict2schema(
+        {"ids": fields.DelimitedTuple((fields.Int, fields.Int, fields.Int))}
+    )
+    schema = schema_cls()
+
+    with pytest.raises(ValidationError):
+        parser.parse(schema, web_request)
+
+
+def test_delimited_list_with_datetime(web_request, parser):
+    """
+    Test that DelimitedList(DateTime(format=...)) correctly parses and dumps
+    dates to and from strings -- indicates that we're doing proper
+    serialization of values in dump() and not just relying on __str__ producing
+    correct results
+    """
     web_request.json = {"dates": "2018-11-01,2018-11-02"}
     schema_cls = dict2schema(
         {"dates": fields.DelimitedList(fields.DateTime(format="%Y-%m-%d"))}
@@ -877,6 +923,27 @@ def test_delimited_list_custom_delimiter(web_request, parser):
     parsed = parser.parse(schema, web_request)
     assert parsed["ids"] == [1, 2, 3]
 
+    dumped = schema.dump(parsed)
+    data = dumped.data if MARSHMALLOW_VERSION_INFO[0] < 3 else dumped
+    assert data["ids"] == "1|2|3"
+
+
+@pytest.mark.skipif(
+    MARSHMALLOW_VERSION_INFO[0] < 3, reason="fields.Tuple added in marshmallow3"
+)
+def test_delimited_tuple_custom_delimiter(web_request, parser):
+    web_request.json = {"ids": "1|2"}
+    schema_cls = dict2schema(
+        {"ids": fields.DelimitedTuple((fields.Int, fields.Int), delimiter="|")}
+    )
+    schema = schema_cls()
+
+    parsed = parser.parse(schema, web_request)
+    assert parsed["ids"] == (1, 2)
+
+    data = schema.dump(parsed)
+    assert data["ids"] == "1|2"
+
 
 def test_delimited_list_load_list_errors(web_request, parser):
     web_request.json = {"ids": [1, 2, 3]}
@@ -891,6 +958,22 @@ def test_delimited_list_load_list_errors(web_request, parser):
     assert errors["ids"] == ["Not a valid delimited list."]
 
 
+@pytest.mark.skipif(
+    MARSHMALLOW_VERSION_INFO[0] < 3, reason="fields.Tuple added in marshmallow3"
+)
+def test_delimited_tuple_load_list_errors(web_request, parser):
+    web_request.json = {"ids": [1, 2]}
+    schema_cls = dict2schema({"ids": fields.DelimitedTuple((fields.Int, fields.Int))})
+    schema = schema_cls()
+
+    with pytest.raises(ValidationError) as excinfo:
+        parser.parse(schema, web_request)
+    exc = excinfo.value
+    assert isinstance(exc, ValidationError)
+    errors = exc.args[0]
+    assert errors["ids"] == ["Not a valid delimited tuple."]
+
+
 # Regresion test for https://github.com/marshmallow-code/webargs/issues/149
 def test_delimited_list_passed_invalid_type(web_request, parser):
     web_request.json = {"ids": 1}
@@ -902,6 +985,19 @@ def test_delimited_list_passed_invalid_type(web_request, parser):
     assert excinfo.value.messages == {"json": {"ids": ["Not a valid delimited list."]}}
 
 
+@pytest.mark.skipif(
+    MARSHMALLOW_VERSION_INFO[0] < 3, reason="fields.Tuple added in marshmallow3"
+)
+def test_delimited_tuple_passed_invalid_type(web_request, parser):
+    web_request.json = {"ids": 1}
+    schema_cls = dict2schema({"ids": fields.DelimitedTuple((fields.Int,))})
+    schema = schema_cls()
+
+    with pytest.raises(ValidationError) as excinfo:
+        parser.parse(schema, web_request)
+    assert excinfo.value.messages == {"json": {"ids": ["Not a valid delimited tuple."]}}
+
+
 def test_missing_list_argument_not_in_parsed_result(web_request, parser):
     # arg missing in request
     web_request.json = {}