Repository: encode/django-rest-framework

Issue:
JSONField(binary=True) represents using binary strings, which JSONRenderer does not support.
## Checklist
- [x] I have verified that that issue exists against the `master` branch of Django REST framework.
- [x] I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
- [x] This is not a usage question. (Those should be directed to the [discussion group](https://groups.google.com/forum/#!forum/django-rest-framework) instead.)
- [x] This cannot be dealt with as a third party library. (We prefer new functionality to be [in the form of third party libraries](http://www.django-rest-framework.org/topics/third-party-resources/#about-third-party-packages) where possible.)
- [x] I have reduced the issue to the simplest possible case.
- [ ] I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)
## Steps to reproduce

``` python
from rest_framework import serializers

class TestSerializer(serializers.Serializer):
    json = serializers.JSONField()

s = TestSerializer(data={'json': "{broken JSON"})
assert s.is_valid() is False
```
## Expected behavior

Invalid JSON should result in `is_valid()` returning `False`.
## Actual behavior

`is_valid()` returns `True`

Specifying `binary=True` for the field definition results in the desired behavior for serialization, but causes deserialization to fail on Python 3.



Discussion:
Unless you set `binary=True` the field will check for valid primitives, not a valid binary encoded bytestring.
As currently described, you're passing a valid primitive - a string literal.

What you actually want, as you've noted, is this...

```
class TestSerializer(serializers.Serializer):
    json = serializers.JSONField(binary=True)
```

Which then works as expected - `is_valid()` returns `False`.
I've tested this in python 3 and I can't see what you mean wrt "causes deserialization to fail" - can reconsider if that aspect gets more details.

The tricky point with binary=True is that it isn't a json renderable data because it's a binary string and the json renderer expect unicodes.

I'd point out that the documentation for that field would benefit some clarification:

> A field class that validates that the incoming data structure consists of valid JSON primitives. In its alternate binary mode, it will represent and validate JSON-encoded binary strings.

As it stands, I understand that it expects a valid JSON input for non binary mode.

> It isn't a json renderable data because it's a binary string and the json renderer expect unicodes.

Right - then that's the core issue that we should resolve here.

Either that or we should output to unicode by default. (Not strictly correct, but it'd do the trick)

Sorry, I misspoke. It doesn't break the deserialization, it breaks the `JSONRenderer`, which I see is a separate issue.

So until #4187 is resolved, what is the best way to enforce valid JSON inputs for an API?

Well, using a binary json blob as a field isn't a particularly common use case, so poss you should just reconsider if that's what you _actually_ want (ie a nested document of json primitives may be better) but if it really is what you need I'd suggest subclassing JSONField and forcing to_representation to do .decode('utf8') on the result so that you have a custom JSONField that returns Unicode, not bytestrings.


Resolving patch:
diff --git a/rest_framework/utils/encoders.py b/rest_framework/utils/encoders.py
--- a/rest_framework/utils/encoders.py
+++ b/rest_framework/utils/encoders.py
@@ -51,6 +51,9 @@ def default(self, obj):
             return six.text_type(obj)
         elif isinstance(obj, QuerySet):
             return tuple(obj)
+        elif isinstance(obj, six.binary_type):
+            # Best-effort for binary blobs. See #4187.
+            return obj.decode('utf-8')
         elif hasattr(obj, 'tolist'):
             # Numpy arrays and array scalars.
             return obj.tolist()


Test patch: