Repository: django/channels

Issue:
CookieMiddleware should decode() using `latin-1`. 
We have been periodically getting this error stack trace in our logs:

```
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 745: ordinal not in range(128)
  File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 151, in maybeDeferred
  File "/usr/local/lib/python3.7/site-packages/daphne/server.py", line 201, in create_application
  File "/usr/local/lib/python3.7/site-packages/channels/routing.py", line 54, in __call__
  File "/usr/local/lib/python3.7/site-packages/channels/security/websocket.py", line 37, in __call__
  File "/usr/local/lib/python3.7/site-packages/channels/sessions.py", line 41, in __call__
```

When investigating, I found out that the issue is at this line: https://github.com/django/channels/blob/dd304566775d6f1ce3d40a4d6e2a100f33c54889/channels/sessions.py#L41
 when there is a cookie that has a non-ascii value. 

Is there a reason that it is assumed that the cookie value will not have non-ascii values? If so, what would be the work around?

If not, is there an issue with changing this?

For example, would it be safe to do this:
`cookies = parse_cookie(value.decode("ascii", errors='ignore'))`



Discussion:
I have the same issue using the version of 2.3.0.
As far as I can tell cookies are meant to only contain US-ASCII characters... 

e.g. https://tools.ietf.org/id/draft-ietf-httpstate-cookie-23.html#sane-set-cookie-syntax

```
cookie-octet      = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E
                      ; US-ASCII characters excluding CTLs,
                      ; whitespace DQUOTE, comma, semicolon,
                      ; and backslash
```

Searching doesn't point to anything particularly definitive. 

Will close: happy to re-open if more info comes up.
Some historical discussion on Gunicorn here: https://github.com/benoitc/gunicorn/issues/1778

Eventually looks like they eased to `latin-1`. https://github.com/benoitc/gunicorn/pull/1914/files

Maybe we should do that. 
[PEP 333](https://www.python.org/dev/peps/pep-0333/#unicode-issues):

> Note also that strings passed to start_response() as a status or as response headers must follow RFC 2616 with respect to encoding. That is, they must either be ISO-8859-1 characters, or use RFC 2047 MIME encoding.

So it looks like `latin-1` is correct, even if folks SHOULD use `ascii`.

Todo: 

* adapt the test example from gunicorn https://github.com/benoitc/gunicorn/pull/1914/files
* Adjust to use `'latin-1'`

Resolving patch:
diff --git a/channels/http.py b/channels/http.py
--- a/channels/http.py
+++ b/channels/http.py
@@ -312,13 +312,13 @@ def encode_response(cls, response):
         response_headers = []
         for header, value in response.items():
             if isinstance(header, str):
-                header = header.encode("ascii")
+                header = header.encode("latin")
             if isinstance(value, str):
                 value = value.encode("latin1")
             response_headers.append((bytes(header), bytes(value)))
         for c in response.cookies.values():
             response_headers.append(
-                (b"Set-Cookie", c.output(header="").encode("ascii").strip())
+                (b"Set-Cookie", c.output(header="").encode("latin1").strip())
             )
         # Make initial response message
         yield {
diff --git a/channels/security/websocket.py b/channels/security/websocket.py
--- a/channels/security/websocket.py
+++ b/channels/security/websocket.py
@@ -28,7 +28,7 @@ async def __call__(self, scope, send, receive):
             if header_name == b"origin":
                 try:
                     # Set ResultParse
-                    parsed_origin = urlparse(header_value.decode("ascii"))
+                    parsed_origin = urlparse(header_value.decode("latin1"))
                 except UnicodeDecodeError:
                     pass
         # Check to see if the origin header is valid
diff --git a/channels/sessions.py b/channels/sessions.py
--- a/channels/sessions.py
+++ b/channels/sessions.py
@@ -38,7 +38,7 @@ async def __call__(self, scope, receive, send):
         # Go through headers to find the cookie one
         for name, value in scope.get("headers", []):
             if name == b"cookie":
-                cookies = parse_cookie(value.decode("ascii"))
+                cookies = parse_cookie(value.decode("latin1"))
                 break
         else:
             # No cookie header found - add an empty default.


Test patch:
diff --git a/tests/test_http.py b/tests/test_http.py
--- a/tests/test_http.py
+++ b/tests/test_http.py
@@ -129,7 +129,7 @@ def test_post_files(self):
                 "path": "/test/",
                 "headers": {
                     "content-type": b"multipart/form-data; boundary=BOUNDARY",
-                    "content-length": str(len(body)).encode("ascii"),
+                    "content-length": str(len(body)).encode("latin1"),
                 },
             },
             BytesIO(body),
@@ -226,7 +226,7 @@ def test_size_check_ignores_files(self):
             "path": "/test/",
             "headers": {
                 "content-type": b"multipart/form-data; boundary=BOUNDARY",
-                "content-length": str(len(body)).encode("ascii"),
+                "content-length": str(len(body)).encode("latin1"),
             },
         }
 
@@ -240,6 +240,22 @@ def test_size_check_ignores_files(self):
             # account the size of the file upload data.
             AsgiRequest(scope, BytesIO(body)).POST
 
+    def test_latin1_headers(self):
+        request = AsgiRequest(
+            {
+                "http_version": "1.1",
+                "method": "GET",
+                "path": "/test2/",
+                "headers": {
+                    "host": b"example.com",
+                    "foo": bytes("äbcd", encoding="latin1"),
+                },
+            },
+            BytesIO(b""),
+        )
+
+        self.assertEqual(request.headers["foo"], "äbcd")
+
 
 # Handler tests