Repository: encode/django-rest-framework

Issue:
JSONP renderer fails to escape U+2028 and U+2029
When Unicode is enabled for JSON output, the JSONP renderer fails to escape U+2028 and U+2029 which are [valid in JSON but invalid in JavaScript](http://timelessrepo.com/json-isnt-a-javascript-subset).

Try adding this simple model to the example app from the README:

```
from django.db import models

class Post(models.Model):

    title = models.CharField(max_length=100)
    content = models.TextField()
```

Create an object like this:

```
from example.models import Post
Post.objects.create(title=u'Test\u2028', content=u'example')
```

Then fire up a dev server and make an HTML page with a JSONP request like this:

```
<script type="text/javascript"
        src="http://127.0.0.1:8000/posts/?format=jsonp"></script>
```

Open the page in your browser and observe the error console. In Chromium, I get:

> Uncaught SyntaxError: Unexpected token ILLEGAL



Discussion:
Which version of python is this btw?

Should this additionally be added as a Python bug.

Appears to be resolved in simplejson - https://github.com/simplejson/simplejson/blob/master/simplejson/encoder.py#L36
But not in CPython - https://hg.python.org/cpython/file/e1afae23dfcd/Lib/json/encoder.py#l26

And I don't see it listed as an issue?... http://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=2028+2029+json&submit=search&status=-1%2C1%2C2%2C3

I run Python 2.7.3. This may be a nice thing to have in the standard `json`, but it’s not a bug, because what `json.dumps()` returns is perfectly valid as JSON, it’s only problematic when you try to interpret it as JavaScript.

So unsure whether to resolve this in JSONRenderer (as an enhancement) or in JSONPRenderer (as a bugfix).

Aside: Personally I do still class this as a bug in the Python json lib, as there's no good reason why it should just always be escaped. The JSON spec clearly has a bug when it refers to being derived from "a small subset of ECMAScript" http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf Unclear why they don't at least add an ammendment _recommending_ that implementations escape those two characters.


Resolving patch:
diff --git a/rest_framework/renderers.py b/rest_framework/renderers.py
--- a/rest_framework/renderers.py
+++ b/rest_framework/renderers.py
@@ -102,6 +102,11 @@ def render(self, data, accepted_media_type=None, renderer_context=None):
         # and may (or may not) be unicode.
         # On python 3.x json.dumps() returns unicode strings.
         if isinstance(ret, six.text_type):
+            # We always fully escape \u2028 and \u2029 to ensure we output JSON
+            # that is a strict javascript subset. If bytes were returned
+            # by json.dumps() then we don't have these characters in any case.
+            # See: http://timelessrepo.com/json-isnt-a-javascript-subset
+            ret = ret.replace('\u2028', '\\u2028').replace('\u2029', '\\u2029')
             return bytes(ret.encode('utf-8'))
         return ret
 


Test patch:
diff --git a/tests/test_renderers.py b/tests/test_renderers.py
--- a/tests/test_renderers.py
+++ b/tests/test_renderers.py
@@ -384,6 +384,15 @@ def test_proper_encoding(self):
         content = renderer.render(obj, 'application/json')
         self.assertEqual(content, '{"countries":["United Kingdom","France","España"]}'.encode('utf-8'))
 
+    def test_u2028_u2029(self):
+        # The \u2028 and \u2029 characters should be escaped,
+        # even when the non-escaping unicode representation is used.
+        # Regression test for #2169
+        obj = {'should_escape': '\u2028\u2029'}
+        renderer = JSONRenderer()
+        content = renderer.render(obj, 'application/json')
+        self.assertEqual(content, '{"should_escape":"\\u2028\\u2029"}'.encode('utf-8'))
+
 
 class AsciiJSONRendererTests(TestCase):
     """