Repository: marshmallow-code/webargs

Issue:
Dealing with empty values in `DelimitedFieldMixin`
`DelimitedList(String())` deserializes "a,,c" as `["a", "", "c"]`.

I guess this meets user expectations.

My expectation with integers would be that

`DelimitedList(Integer(allow_none=True))` deserializes `"1,,3"` as `[1,None,3]`

but it errors.

The reason for this is that when the string is split, it is turned into `["1", "", "3"]`. This is why it works in the string case.

I'm not sure this was really intended. It may be a side effect of the `split` function that happens to do well with strings.

We could change that to replace empty values with `None`. But it would break the string use case, unless the user overloads `String` field to deserialize `None` as `""`.

Likewise, users may already overload `Integer` to deserialize `""` as `None` and no modification is required to `DelimitedFieldMixin`.

Just been caught by this and wondering out loud. Advice welcome.

In any case, there is an intrinsic limitation in the delimited string format: one can't distinguish empty string from missing value (as opposed to a JSON payload). It is not clear to me how OpenAPI (for instance) deals with the case of an empty element in an array (in a query argument).


Discussion:
We could add another argument/instance variable just for the empty case. I don't know if I like it yet as an idea, but we'd get...

```python
DelimitedList(String(), empty="")  # default
DelimitedList(Integer(), empty=0)
DelimitedList(Integer(allow_none=True), empty=None)
```

For the record, I think allowing for None has utility. I've seen APIs do things like
```
range=0,100
range=1,  # no upper bound
```

Having written that, I'm starting to think that I like adding an argument for this. It's not much new API surface and it ought to work. Are there other ideas we should consider (e.g. a classvar)?
Indeed, adding an argument here makes total sense. Thanks.

Argument should be enough IMHO. Class var would allow someone to subclass to avoid the argument duplication but I don't think it is worth it.

I don't need this right now. But it is nice to keep this in the backlog so that anyone needing it can work on it. It is a pretty easy contribution.

It would be interesting to wonder how this can be documented in OpenAPI via apispec.



Resolving patch:
diff --git a/src/webargs/fields.py b/src/webargs/fields.py
--- a/src/webargs/fields.py
+++ b/src/webargs/fields.py
@@ -15,6 +15,8 @@
 """
 from __future__ import annotations
 
+import typing
+
 import marshmallow as ma
 
 # Expose all fields from marshmallow.fields.
@@ -64,6 +66,8 @@ class DelimitedFieldMixin:
     delimiter: str = ","
     # delimited fields set is_multiple=False for webargs.core.is_multiple
     is_multiple: bool = False
+    # NOTE: in 8.x this defaults to "" but in 9.x it will be 'missing'
+    empty_value: typing.Any = ""
 
     def _serialize(self, value, attr, obj, **kwargs):
         # serializing will start with parent-class serialization, so that we correctly
@@ -77,6 +81,8 @@ def _deserialize(self, value, attr, data, **kwargs):
         if not isinstance(value, (str, bytes)):
             raise self.make_error("invalid")
         values = value.split(self.delimiter) if value else []
+        # convert empty strings to the empty value; typically "" and therefore a no-op
+        values = [v or self.empty_value for v in values]
         return super()._deserialize(values, attr, data, **kwargs)
 
 
@@ -117,6 +123,12 @@ class DelimitedTuple(DelimitedFieldMixin, ma.fields.Tuple):
 
     default_error_messages = {"invalid": "Not a valid delimited tuple."}
 
-    def __init__(self, tuple_fields, *, delimiter: str | None = None, **kwargs):
+    def __init__(
+        self,
+        tuple_fields,
+        *,
+        delimiter: str | None = None,
+        **kwargs,
+    ):
         self.delimiter = delimiter or self.delimiter
         super().__init__(tuple_fields, **kwargs)


Test patch:
diff --git a/tests/test_core.py b/tests/test_core.py
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -11,6 +11,7 @@
     INCLUDE,
     RAISE,
     Schema,
+    missing,
     post_load,
     pre_load,
     validates_schema,
@@ -1106,6 +1107,47 @@ def test_delimited_tuple_passed_invalid_type(web_request, parser):
     assert excinfo.value.messages == {"json": {"ids": ["Not a valid delimited tuple."]}}
 
 
+def test_delimited_list_custom_empty_value(web_request, parser):
+    class ZeroList(fields.DelimitedList):
+        empty_value = 0
+
+    web_request.json = {"ids": "1,,3"}
+    schema_cls = Schema.from_dict({"ids": ZeroList(fields.Int())})
+    schema = schema_cls()
+
+    parsed = parser.parse(schema, web_request)
+    assert parsed["ids"] == [1, 0, 3]
+
+
+def test_delimited_tuple_custom_empty_value(web_request, parser):
+    class ZeroTuple(fields.DelimitedTuple):
+        empty_value = 0
+
+    web_request.json = {"ids": "1,,3"}
+    schema_cls = Schema.from_dict(
+        {"ids": ZeroTuple((fields.Int, fields.Int, fields.Int))}
+    )
+    schema = schema_cls()
+
+    parsed = parser.parse(schema, web_request)
+    assert parsed["ids"] == (1, 0, 3)
+
+
+def test_delimited_list_using_missing_for_empty(web_request, parser):
+    # this is "future" because we plan to make this the default for webargs v9.0
+    class FutureList(fields.DelimitedList):
+        empty_value = missing
+
+    web_request.json = {"ids": "foo,,bar"}
+    schema_cls = Schema.from_dict(
+        {"ids": FutureList(fields.String(load_default="nil"))}
+    )
+    schema = schema_cls()
+
+    parsed = parser.parse(schema, web_request)
+    assert parsed["ids"] == ["foo", "nil", "bar"]
+
+
 def test_missing_list_argument_not_in_parsed_result(web_request, parser):
     # arg missing in request
     web_request.json = {}