Repository: encode/django-rest-framework

Issue:
Better catching of error cases in `.update` and `.create`.
When I do a PUT through the API browser to a view whose serializer contains a source property:

``` python
last_name = serializers.CharField(source='user.last_name')
```

I receive the following error:

```
Cannot assign "{'first_name': u'first', 'last_name': u'last'}": "ContactInfo.user" must be a "User" instance
```

I've created a [bare-bones django project that reproduces the error](https://github.com/brianhv/drfissue). Of particular note are the [serializers](https://github.com/brianhv/drfissue/blob/master/drftest/drftest/readonly/serializers.py) and the [model](https://github.com/brianhv/drfissue/blob/master/drftest/drftest/readonly/models.py).

Relevant excerpt of IRC conversation leading to this ticket:                   

```
<BrianHV> if you clone the repo, install the stuff in the requirements.txt (just django
          1.6 and DRF 3), set up a user, and use admin to create a ContactInfo for the
          user, you should be able to reproduce
<BrianHV> I'm testing through the API browser, btw.
<kevin-brown> Yeah, I think I know what is happening
<kevin-brown> `validated_data` is being (correctly?) generated as a nested dictionary,
              and that's being assigned to the `user` parameter directly
<kevin-brown> If you can open a ticket explaining that it works in DRF 2.4, that'd be
              helpful
<kevin-brown> In DRF 2.4, it would traverse the relation and assign the properties
              directly
```



Discussion:
Possible that we should make this case raise an explicit error explaining why it can't be dealt with automatically. You need to write an explicit create and/or update method here. Eg.

There _isnt_ any way of automatically handling this case: What _are_ you expecting to happen on create? Create a new user instance? If so how would you determine a username? You probably only want to support updates on this serializer and override create to raise 'NotImplemented' but if that's not the case then you'll need to figure out what it is you want to happen instead.

I've no doubt that 2.x would present some horrible buggy behaviour here.

Incidentally what type was the exception you got? Would be useful to know so we can improve the error messaging.

> Incidentally what type was the exception you got?

[Pretty sure](http://stackoverflow.com/q/15337636/359284) this is a `ValueError`.

We could expand [on the exception handling then](https://github.com/tomchristie/django-rest-framework/blob/f221b737a1c329277ac90b8e2049882e0dde3e6f/rest_framework/serializers.py#L658-673)

Yep, ValueError.

```
Environment:


Request Method: POST
Request URL: http://localhost:8765/api/users/1/

Django Version: 1.6.8
Python Version: 2.7.5
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'rest_framework',
 'drftest.readonly')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')


Traceback:
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  112.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
  57.         return view_func(*args, **kwargs)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/viewsets.py" in view
  79.             return self.dispatch(request, *args, **kwargs)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  406.             response = self.handle_exception(exc)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  403.             response = handler(request, *args, **kwargs)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/mixins.py" in update
  68.         self.perform_update(serializer)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/mixins.py" in perform_update
  72.         serializer.save()
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/serializers.py" in save
  136.             self.instance = self.update(self.instance, validated_data)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/rest_framework/serializers.py" in update
  669.             setattr(instance, attr, value)
File "/Users/bhv1/irccode/drfvirt/lib/python2.7/site-packages/django/db/models/fields/related.py" in __set__
  339.                                  self.field.name, self.field.rel.to._meta.object_name))

Exception Type: ValueError at /api/users/1/
Exception Value: Cannot assign "{'first_name': u'asdf', 'last_name': u'asdf'}": "ContactInfo.user" must be a "User" instance.
```

How about the following?...

> Got a `ValueError` when calling `%s.objects.create()`. This may be because you have a field on the serializer class that is is returning the incorrect type for '`%s.objects.create()`. You probably need to either change the field type, or override the %s.create() method to handle this correctly.\nOriginal exception text was: %s.'

Even more comprehensive would be to check for dotted source notation in the same way we do for writable nested fields.

Here's my perspective as a newbie (and assuming `TypeError` was meant to be `ValueError`).

My first response to that was similar to my response to the error I got: "Guess I can't do that, then." As I thought about it while finishing breakfast, my response shifted towards: "But if I really wanted to do it maybe that would tell me how if I were more familiar with the framework."

At the moment, though, I've settled on, "Why is it trying to create something when I'm trying to update an existing record? I must really not be doing this the right way."

If those are reasonable things to think in this situation, then that message would have the correct effect.

> Why is it trying to create something

Oops my error above is incorrect of course - this is not during `.objects.create()` it's during assignment in `update()`. Anyways, upshot is that we should look into better error reporting here.

As it happens for "to one" relationships we _could_ try to handle this automatically _for updates only_ but there would still be error cases to deal with there (eg nullable relationship that does not exist) so not sure if we should try to do it automagically, or if we should force the code to be explicit.


Resolving patch:
diff --git a/rest_framework/serializers.py b/rest_framework/serializers.py
--- a/rest_framework/serializers.py
+++ b/rest_framework/serializers.py
@@ -561,6 +561,64 @@ def errors(self):
 # ModelSerializer & HyperlinkedModelSerializer
 # --------------------------------------------
 
+def raise_errors_on_nested_writes(method_name, serializer):
+    """
+    Give explicit errors when users attempt to pass writable nested data.
+
+    If we don't do this explicitly they'd get a less helpful error when
+    calling `.save()` on the serializer.
+
+    We don't *automatically* support these sorts of nested writes brecause
+    there are too many ambiguities to define a default behavior.
+
+    Eg. Suppose we have a `UserSerializer` with a nested profile. How should
+    we handle the case of an update, where the `profile` realtionship does
+    not exist? Any of the following might be valid:
+
+    * Raise an application error.
+    * Silently ignore the nested part of the update.
+    * Automatically create a profile instance.
+    """
+
+    # Ensure we don't have a writable nested field. For example:
+    #
+    # class UserSerializer(ModelSerializer):
+    #     ...
+    #     profile = ProfileSerializer()
+    assert not any(
+        isinstance(field, BaseSerializer) and (key in validated_attrs)
+        for key, field in serializer.fields.items()
+    ), (
+        'The `.{method_name}()` method does not support writable nested'
+        'fields by default.\nWrite an explicit `.{method_name}()` method for '
+        'serializer `{module}.{class_name}`, or set `read_only=True` on '
+        'nested serializer fields.'.format(
+            method_name=method_name,
+            module=serializer.__class__.__module__,
+            class_name=serializer.__class__.__name__
+        )
+    )
+
+    # Ensure we don't have a writable dotted-source field. For example:
+    #
+    # class UserSerializer(ModelSerializer):
+    #     ...
+    #     address = serializer.CharField('profile.address')
+    assert not any(
+        '.' in field.source and (key in validated_attrs)
+        for key, field in serializer.fields.items()
+    ), (
+        'The `.{method_name}()` method does not support writable dotted-source '
+        'fields by default.\nWrite an explicit `.{method_name}()` method for '
+        'serializer `{module}.{class_name}`, or set `read_only=True` on '
+        'dotted-source serializer fields.'.format(
+            method_name=method_name,
+            module=serializer.__class__.__module__,
+            class_name=serializer.__class__.__name__
+        )
+    )
+
+
 class ModelSerializer(Serializer):
     """
     A `ModelSerializer` is just a regular `Serializer`, except that:
@@ -624,18 +682,7 @@ def create(self, validated_data):
         If you want to support writable nested relationships you'll need
         to write an explicit `.create()` method.
         """
-        # Check that the user isn't trying to handle a writable nested field.
-        # If we don't do this explicitly they'd likely get a confusing
-        # error at the point of calling `Model.objects.create()`.
-        assert not any(
-            isinstance(field, BaseSerializer) and (key in validated_attrs)
-            for key, field in self.fields.items()
-        ), (
-            'The `.create()` method does not support nested writable fields '
-            'by default. Write an explicit `.create()` method for serializer '
-            '`%s.%s`, or set `read_only=True` on nested serializer fields.' %
-            (self.__class__.__module__, self.__class__.__name__)
-        )
+        raise_errors_on_nested_writes('create', self)
 
         ModelClass = self.Meta.model
 
@@ -675,19 +722,12 @@ def create(self, validated_data):
         return instance
 
     def update(self, instance, validated_data):
-        assert not any(
-            isinstance(field, BaseSerializer) and (key in validated_attrs)
-            for key, field in self.fields.items()
-        ), (
-            'The `.update()` method does not support nested writable fields '
-            'by default. Write an explicit `.update()` method for serializer '
-            '`%s.%s`, or set `read_only=True` on nested serializer fields.' %
-            (self.__class__.__module__, self.__class__.__name__)
-        )
+        raise_errors_on_nested_writes('update', self)
 
         for attr, value in validated_data.items():
             setattr(instance, attr, value)
         instance.save()
+
         return instance
 
     def get_validators(self):


Test patch: