Repository: encode/django-rest-framework

Issue:
allow_null=True is not being passed to ListSerializer constructor in many_init
Hello,

We have a use case where we accept either a list of objects or null. The problem is that `BaseSerializer.many_init` is not passing the `allow_null` argument to the `ListSerializer` constructor. because of that I have validation errors even if the field was specified with `allow_null=True`.

He's an example:

``` python
class PaymentInfoSerializer(serializers.Serializer):
    url = serializer.CharField()
    amount = serializers.DecimalField()

    def validate(self, attrs):
        """
        This is never being called
        """


class Request(serializers.Serializer):
    payment_info = PaymentInfoSerializer(
        many=True, 
        required=False,
        allow_null=True)
```

Basically `LIST_SERIALIZER_KWARGS` (defined [here](https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/serializers.py#L48)) does not include `allow_null` and therefore the arguments is not passed to the `ListSerializer` constructor (details [here](https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/serializers.py#L118)).

Is there a reason why `allow_null` should not be sent to the list serializer?

Thanks!



Discussion:
Our current fix is to create a custom `ListSerializer` that passes `allow_null=True` to the default constructor:

``` python
class NullableListSerializer(serializers.ListSerializer):
    def __init__(self, *args, **kwargs):
        kwargs['allow_null'] = True
        super(NullableListSerializer, self).__init__(*args, **kwargs)


class PaymentInfoSerializer(serializers.Serializer):
    class Meta:
        list_serializer_class = NullableListSerializer
```

It just feels hacky, since the default implementation of `ListSerializer` work well if you pass it the `allow_null=True` keyword argument, and this is the only required changed to make it work.

Not convinced that we should be supporting this by default.
I'd assume the current behavior will be to allow this:

```
[{'child': 'object'}, {'child': 'object'}, None, {'child': 'object'}]
```

Which I think is reasonable. If you want an empty list, use an empty list `[]` rather than `None`, or use a custom list serializer as you've done. We could discuss this further and might be open to a change, but for now I'm going to consider this as closed.

I'm not saying it's necessarily a bug that needs to be addressed. You point regarding empty lists is perfectly valid. I'm not sure though that a list containing a `null` value at some index is more plausible use-case than specifying a `null` as an input for the `many` field.

But there are a few aspects that make sense regarding this:
1. It costs almost nothing to support this functionality (just add the `allow_null` value to `LIST_SERIALIZER_KWARGS`). It will not break backwards compatibility and whoever will want to use this behavior will have to just define the field as `WhateverSerializer(many=True, allow_null=True)`. This will also be consistent with the other serializer usage `OtherSerializer(allow_null=True)`.
2. There might be cases when an empty list might mean something and a null value might means something else for the same field and the API needs to handle that.
3. In my case (and it's possible in other cases) I'm not building a new API, I'm upgrading a DRF 2.x API to the latest version and I need to keep backwards compatibility with my mobile apps

If you don't consider this important enough, and I understand if you don't, we should at least document that this is not support, or even raise an exception if someone tries to use `WhateverSerializer(many=True, allow_null=True)`. Currently it just silently drops the keyword arguments that are not supported and it's somehow confusing (it took me a while to understand why the null value is not being passed to the serializer)

Thanks for the great work you are doing!

> if someone tries to use WhateverSerializer(many=True, allow_null=True)

I think it's probably more a case of trying to figure out which of [{}, None, {}, {}] vs None is more intuitive - perhaps worth taking to the discussion group?

Sounds good to me. I posted a question to the discussion group.

:+1:

@tomchristie There was not much of a discussion on the forum about this issue. The only comment was in favor of supporting None as  a value for a list serializer field (instead of [{}, None, {}]). 

Did you get a chance to think about this more throughly?

Perhaps the best thing to move this forward would be to write the _most minimal possible_ test case that demonstrates the behavior change that you're suggesting.

Then submit that as a pull request, along with the suggested fix.

Not a garuntee that it'll get pulled in, but that'd remove any blockers aside from the design decision.

Sounds good. I'll have a PR in the next day or two.


Resolving patch:
diff --git a/rest_framework/serializers.py b/rest_framework/serializers.py
--- a/rest_framework/serializers.py
+++ b/rest_framework/serializers.py
@@ -50,7 +50,7 @@
 LIST_SERIALIZER_KWARGS = (
     'read_only', 'write_only', 'required', 'default', 'initial', 'source',
     'label', 'help_text', 'style', 'error_messages', 'allow_empty',
-    'instance', 'data', 'partial', 'context'
+    'instance', 'data', 'partial', 'context', 'allow_null'
 )
 
 


Test patch:
diff --git a/tests/test_serializer_nested.py b/tests/test_serializer_nested.py
--- a/tests/test_serializer_nested.py
+++ b/tests/test_serializer_nested.py
@@ -69,3 +69,59 @@ def test_multipart_validate(self):
         input_data = QueryDict('nested[one]=1')
         serializer = self.Serializer(data=input_data)
         assert serializer.is_valid()
+
+
+class TestNestedSerializerWithMany:
+    def setup(self):
+        class NestedSerializer(serializers.Serializer):
+            example = serializers.IntegerField(max_value=10)
+
+        class TestSerializer(serializers.Serializer):
+            allow_null = NestedSerializer(many=True, allow_null=True)
+            not_allow_null = NestedSerializer(many=True)
+
+        self.Serializer = TestSerializer
+
+    def test_null_allowed_if_allow_null_is_set(self):
+        input_data = {
+            'allow_null': None,
+            'not_allow_null': [{'example': '2'}, {'example': '3'}]
+        }
+        expected_data = {
+            'allow_null': None,
+            'not_allow_null': [{'example': 2}, {'example': 3}]
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert serializer.is_valid(), serializer.errors
+        assert serializer.validated_data == expected_data
+
+    def test_null_is_not_allowed_if_allow_null_is_not_set(self):
+        input_data = {
+            'allow_null': None,
+            'not_allow_null': None
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert not serializer.is_valid()
+
+        expected_errors = {'not_allow_null': [serializer.error_messages['null']]}
+        assert serializer.errors == expected_errors
+
+    def test_run_the_field_validation_even_if_the_field_is_null(self):
+        class TestSerializer(self.Serializer):
+            validation_was_run = False
+
+            def validate_allow_null(self, value):
+                TestSerializer.validation_was_run = True
+                return value
+
+        input_data = {
+            'allow_null': None,
+            'not_allow_null': [{'example': 2}]
+        }
+        serializer = TestSerializer(data=input_data)
+
+        assert serializer.is_valid()
+        assert serializer.validated_data == input_data
+        assert TestSerializer.validation_was_run