Repository: wagtail/wagtail

Issue:
Unicode name of documents raises error
Hello,

I uploaded a file which name is in unicode (Arabic).

And when I click the link to see the file it raises an error. Here's the traceback:

 Environment:

Request Method: GET
Request URL: http://0.0.0.0:8000/documents/1/docs%D8%A7%D9%84%D9%83%D8%A7%D8%AA%D8%AF%D8%B1%D8%A7%D9%8A%D9%8A%D8%A9_%D9%88%D8%A7%D9%84%D8%B3%D9%88%D9%82.pdf

Django Version: 1.7.1
Python Version: 2.7.6
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'compressor',
 'taggit',
 'modelcluster',
 'django.contrib.admin',
 'wagtail.wagtailcore',
 'wagtail.wagtailadmin',
 'wagtail.wagtaildocs',
 'wagtail.wagtailsnippets',
 'wagtail.wagtailusers',
 'wagtail.wagtailimages',
 'wagtail.wagtailembeds',
 'wagtail.wagtailsearch',
 'wagtail.wagtailredirects',
 'wagtail.wagtailforms',
 'wagtail.wagtailsites',
 'demo')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'wagtail.wagtailcore.middleware.SiteMiddleware',
 'wagtail.wagtailredirects.middleware.RedirectMiddleware')

Traceback:
File "/home/vagrant/.virtualenvs/wagtaildemo/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
1.                     response = wrapped_callback(request, _callback_args, *_callback_kwargs)
   File "/home/vagrant/.virtualenvs/wagtaildemo/local/lib/python2.7/site-packages/wagtail/wagtaildocs/views/serve.py" in serve
2.     response['Content-Disposition'] = 'attachment; filename=%s' % doc.filename
   File "/home/vagrant/.virtualenvs/wagtaildemo/local/lib/python2.7/site-packages/django/http/response.py" in **setitem**
3.         value = self._convert_to_charset(value, 'latin-1', mime_encode=True)
   File "/home/vagrant/.virtualenvs/wagtaildemo/local/lib/python2.7/site-packages/django/http/response.py" in _convert_to_charset
4.             raise BadHeaderError("Header values can't contain newlines (got %r)" % value)

Exception Type: BadHeaderError at /documents/1/docsالكاتدرايية_والسوق.pdf
Exception Value: Header values can't contain newlines (got '=?utf-8?b?YXR0YWNobWVudDsgZmlsZW5hbWU9ZG9jc9in2YTZg9in2KrYr9ix2KfZitmK2Klf?=\n =?utf-8?b?2YjYp9mE2LPZiNmCLnBkZg==?=')



Discussion:
Thanks for the report! Looks like this is a Django / Python bug, addressed here: https://github.com/django/django/commit/efb1f99f943b99624c78b91ff58502125490191e

I'll see if it's possible to apply a workaround for current Django versions that don't have this fix in place. (In the worst case, we may just have to strip out / convert non-ASCII characters from this header...)


Resolving patch:
diff --git a/wagtail/wagtaildocs/views/serve.py b/wagtail/wagtaildocs/views/serve.py
--- a/wagtail/wagtaildocs/views/serve.py
+++ b/wagtail/wagtaildocs/views/serve.py
@@ -1,6 +1,8 @@
 from django.shortcuts import get_object_or_404
 from wsgiref.util import FileWrapper
-from django.http import StreamingHttpResponse
+from django.http import StreamingHttpResponse, BadHeaderError
+
+from unidecode import unidecode
 
 from wagtail.wagtaildocs.models import Document, document_served
 
@@ -10,9 +12,13 @@ def serve(request, document_id, document_filename):
     wrapper = FileWrapper(doc.file)
     response = StreamingHttpResponse(wrapper, content_type='application/octet-stream')
 
-    # TODO: strip out weird characters like semicolons from the filename
-    # (there doesn't seem to be an official way of escaping them)
-    response['Content-Disposition'] = 'attachment; filename=%s' % doc.filename
+    try:
+        response['Content-Disposition'] = 'attachment; filename=%s' % doc.filename
+    except BadHeaderError:
+        # Unicode filenames can fail on Django <1.8, Python 2 due to
+        # https://code.djangoproject.com/ticket/20889 - try with an ASCIIfied version of the name
+        response['Content-Disposition'] = 'attachment; filename=%s' % unidecode(doc.filename)
+
     response['Content-Length'] = doc.file.size
 
     # Send document_served signal


Test patch:
diff --git a/wagtail/wagtaildocs/tests.py b/wagtail/wagtaildocs/tests.py
--- a/wagtail/wagtaildocs/tests.py
+++ b/wagtail/wagtaildocs/tests.py
@@ -1,3 +1,5 @@
+from __future__ import unicode_literals
+
 from six import b
 import unittest
 import mock
@@ -580,6 +582,20 @@ def test_with_incorrect_filename(self):
         self.assertEqual(response.status_code, 404)
 
 
+class TestServeWithUnicodeFilename(TestCase):
+    def setUp(self):
+        self.document = models.Document(title="Test document")
+
+        # Setting this filename in the content-disposition header fails on Django <1.8, Python 2
+        # due to https://code.djangoproject.com/ticket/20889
+        self.filename = 'docs\u0627\u0644\u0643\u0627\u062a\u062f\u0631\u0627\u064a\u064a\u0629_\u0648\u0627\u0644\u0633\u0648\u0642'
+        self.document.file.save(self.filename, ContentFile("A boring example document"))
+
+    def test_response_code(self):
+        response = self.client.get(reverse('wagtaildocs_serve', args=(self.document.id, self.filename)))
+        self.assertEqual(response.status_code, 200)
+
+
 class TestDocumentRichTextLinkHandler(TestCase):
     fixtures = ['test.json']