Repository: encode/django-rest-framework

Issue:
Respect to_field property of ForeignKey relations
In my project i use ForeignKeys with the to_field property to link on a uuid Field. That UUID field is Unique but NOT the primary key for a few reasons. (Including that mysql uses the dumbest indextype possible for primary keys and i need to scale. you cannot change that)

I do also have multiple peers with possible network outage that have to create database entries. So using UUIDs all over is the only feasable way. Therefore i need to serialize my ForeignKeys to the uuid field specified in to_field and deserialize from it.

Despite my unusual requirements i think using to_field to unique but not primary keys is perfectly fine and should be supported.

``` python
>>> b = Statement(device_id="335680b80f204e25a0e23caa052a3e83", rule_id="d19b314a8cb04a558798d6d8026cc56e", evalOperator="EQ", evalDesiredState=False)
>>> x=StatementSerializer(b)
>>> json = JSONRenderer().render(x.data)
>>> json
'{"id":null,"created":null,"modified":null,"uuid":"8846412b-5b07-485a-9f2b-d41c0902a98c","evalOperator":"EQ","evalDesiredState":false,"device":"335680b80f204e25a0e23caa052a3e83","rule":"d19b314a8cb04a558798d6d8026cc56e"}'
```

Serialization works kind of. Notice that i have to/can use key_id syntax -> device: <uuid>

``` python
>>> mod_json = json.replace('rule','rule_id').replace('device',"device_id").replace('user',"user_id")
>>> json_data = JSONParser().parse(BytesIO(json))
>>> mjson_data = JSONParser().parse(BytesIO(mod_json))
>>> xx=StatementSerializer(data=json_data)
>>> xx.is_valid()
False
>>> xx.errors
ReturnDict([('device', [u'Incorrect type. Expected pk value, received unicode.']), ('user', [u'Incorrect type. Expected pk value, received unicode.']), ('rule', [u'Incorrect type. Expected pk value, received unicode.'])])


>>> xx=StatementSerializer(data=mjson_data)
>>> x.is_valid()
False
>>> x.errors
ReturnDict([('device', [u'This field is required.']), ('user', [u'This field is required.']), ('rule', [u'This field is required.'])])
```

Note: I also tried replacing <key> with <key>_id for deserialization but even that does not work. IMHO it should.

``` python
class PrimaryKeyRelatedField(RelatedField):
    def use_pk_only_optimization(self):
        return True
```

A hotfix could be disabling use_pk_only_optimization in PrimaryKeyRelatedField. It deserialized correctly that way but .data etc. still shows the primary key like:
ReturnDict([(u'id', None), ('created', None), ('modified', None), ('uuid', u'43c93954-e9c3-4e90-9d40-c1a87f68cb73'), ('evalOperator', 'EQ'), ('evalDesiredState', False), ('user', 1L), ('device', 2L), ('rule', 2L)])



Discussion:
Sidenote:
Seems that if i disable the use_pk_only_optimization i only get the primary key serialized - not the uuid field like above

I hacked together something that works for me. Please let me know if thats the right way designwise. I would create a pull request. Maybe with some logic so that this field would be used if to_field is set on the modelfield.

``` python
class UniqueKeyRelatedField(PrimaryKeyRelatedField):

    def __init__(self, **kwargs):
        self.to_field = kwargs.pop('to_field', None)
        super(UniqueKeyRelatedField, self).__init__(**kwargs)

    def use_pk_only_optimization(self):
        return False

    def to_internal_value(self, data):
        try:
            return self.get_queryset().get(**{self.to_field: data})
        except ObjectDoesNotExist:
            self.fail('does_not_exist', pk_value=data)
        except (TypeError, ValueError):
            self.fail('incorrect_type', data_type=type(data).__name__)

    def to_representation(self, value):
        return str(getattr(value, self.to_field))


class StatementSerializer(serializers.ModelSerializer):

    class Meta:
        model = Statement

    device = UniqueKeyRelatedField(to_field='uuid', queryset=Device.objects.all())
    user = UniqueKeyRelatedField(to_field='uuid', queryset=User.objects.all())
    # had a crazy loop because the rule model has a 'cached' version of the statement as a JSONField and tries to deserialize it
    rule = UniqueKeyRelatedField(to_field='uuid', queryset=Rule.objects.only('uuid').all())
```

`SlugRelatedField` will work for any unique fields, not necessarily just "slug" type values.
I'd assume this will solve your use case just fine...

```
device = SlugRelatedField(slug_field='uuid', queryset=Device.objects.all())
```

Happy to reopen if this needs more correspondence, and we may consider more docs/renaming at some point in the future as similar questions do come up occasionally.

Thanks your help!

I think i overlooked SlugRelatedField because the example in the doc looks like something i would use directly in a view.

However i still think this should be done automagically as this is a ModelSerializer for Django Models and i specified my intention clearly by using the to_field parameter so i am interested in that relation not in the one to the primary key. Would be more DRY also to because now i have to specify like 30 Fields only to reflect that i am using the builtin to_field for relations. Any thoughts regarding this?

Another unrelated issue i stumbled upon using SlugRelatedField:
I am using the django dev trunk so i can use the shiny new UUIDField.

``` python
  File "/Users/tomjaster/DevFolder/master/airyengine/airy/lib/rest_framework/utils/encoders.py", line 60, in default
    return super(JSONEncoder, self).default(obj)
  File "/usr/local/Cellar/python/2.7.8_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: UUID('335680b8-0f20-4e25-a0e2-3caa052a3e83') is not JSON serializable
```

My quick fix was to add this to JSONEncoder.default but i assume this needs a new class in fields.py? Will there be support for this before django 1.8 gets released? Maybe in the development branch? I don't like to rely on maintaining my own branch to keep those little patches in.

``` python
        elif hasattr(obj, 'hex'):
            return obj.hex
```

best regards

Tom

Now tracking the UUID one here: #2432

Dropping the 3.0.x milestone - valid enhancement, but no reason we need to prioritize this higher than other ongoing work right now.


Resolving patch:
diff --git a/rest_framework/serializers.py b/rest_framework/serializers.py
--- a/rest_framework/serializers.py
+++ b/rest_framework/serializers.py
@@ -794,6 +794,7 @@ class ModelSerializer(Serializer):
     if ModelJSONField is not None:
         serializer_field_mapping[ModelJSONField] = JSONField
     serializer_related_field = PrimaryKeyRelatedField
+    serializer_related_to_field = SlugRelatedField
     serializer_url_field = HyperlinkedIdentityField
     serializer_choice_field = ChoiceField
 
@@ -1129,6 +1130,11 @@ def build_relational_field(self, field_name, relation_info):
         field_class = self.serializer_related_field
         field_kwargs = get_relation_kwargs(field_name, relation_info)
 
+        to_field = field_kwargs.pop('to_field', None)
+        if to_field and to_field != 'id':
+            field_kwargs['slug_field'] = to_field
+            field_class = self.serializer_related_to_field
+
         # `view_name` is only valid for hyperlinked relationships.
         if not issubclass(field_class, HyperlinkedRelatedField):
             field_kwargs.pop('view_name', None)
diff --git a/rest_framework/utils/field_mapping.py b/rest_framework/utils/field_mapping.py
--- a/rest_framework/utils/field_mapping.py
+++ b/rest_framework/utils/field_mapping.py
@@ -222,7 +222,7 @@ def get_relation_kwargs(field_name, relation_info):
     """
     Creates a default instance of a flat relational field.
     """
-    model_field, related_model, to_many, has_through_model = relation_info
+    model_field, related_model, to_many, to_field, has_through_model = relation_info
     kwargs = {
         'queryset': related_model._default_manager,
         'view_name': get_detail_view_name(related_model)
@@ -231,6 +231,9 @@ def get_relation_kwargs(field_name, relation_info):
     if to_many:
         kwargs['many'] = True
 
+    if to_field:
+        kwargs['to_field'] = to_field
+
     if has_through_model:
         kwargs['read_only'] = True
         kwargs.pop('queryset', None)
diff --git a/rest_framework/utils/model_meta.py b/rest_framework/utils/model_meta.py
--- a/rest_framework/utils/model_meta.py
+++ b/rest_framework/utils/model_meta.py
@@ -26,6 +26,7 @@
     'model_field',
     'related_model',
     'to_many',
+    'to_field',
     'has_through_model'
 ])
 
@@ -90,6 +91,10 @@ def _get_fields(opts):
     return fields
 
 
+def _get_to_field(field):
+    return field.to_fields[0] if field.to_fields else None
+
+
 def _get_forward_relationships(opts):
     """
     Returns an `OrderedDict` of field names to `RelationInfo`.
@@ -100,6 +105,7 @@ def _get_forward_relationships(opts):
             model_field=field,
             related_model=_resolve_model(field.rel.to),
             to_many=False,
+            to_field=_get_to_field(field),
             has_through_model=False
         )
 
@@ -109,6 +115,8 @@ def _get_forward_relationships(opts):
             model_field=field,
             related_model=_resolve_model(field.rel.to),
             to_many=True,
+            # manytomany do not have to_fields
+            to_field=None,
             has_through_model=(
                 not field.rel.through._meta.auto_created
             )
@@ -133,6 +141,7 @@ def _get_reverse_relationships(opts):
             model_field=None,
             related_model=related,
             to_many=relation.field.rel.multiple,
+            to_field=_get_to_field(relation.field),
             has_through_model=False
         )
 
@@ -144,6 +153,8 @@ def _get_reverse_relationships(opts):
             model_field=None,
             related_model=related,
             to_many=True,
+            # manytomany do not have to_fields
+            to_field=None,
             has_through_model=(
                 (getattr(relation.field.rel, 'through', None) is not None) and
                 not relation.field.rel.through._meta.auto_created


Test patch: