Repository: tornadoweb/tornado (python)

Issue:
obs-fold is implemented incorrectly
The HTTP RFCs specify that when an HTTP/1.1 implementation receives a header line that begins with a space or horizontal tab, it has two choices:
1. Reject the request
2. Concatenate (fold) this line into the previous header line.

Tornado implements this incorrectly. In addition to folding header lines beginning with spaces and tabs, Tornado also folds header lines beginning with any of the following bytes:
- 0x0b
- 0x0c
- 0x0d
- 0x1c
- 0x1d
- 0x1e
- 0x1f
- 0x85
- 0xa0

Pull request:
httputil: Reject header lines beginning with invalid whitespace
The obs-fold feature is defined only for tabs and spaces. The str.isspace() method also accepts other whitespace characters. These characters are not valid in HTTP headers and should be treated as errors instead of triggering line folding.

Fixes #3480

Patch:
diff --git a/tornado/httputil.py b/tornado/httputil.py
index 0f443cdd..5d05cacb 100644
--- a/tornado/httputil.py
+++ b/tornado/httputil.py
@@ -248,7 +248,7 @@ class HTTPHeaders(StrMutableMapping):
         if not line:
             # Empty line, or the final CRLF of a header block.
             return
-        if line[0].isspace():
+        if line[0] in HTTP_WHITESPACE:
             # continuation of a multi-line header
             # TODO(7.0): Remove support for line folding.
             if self._last_key is None:


Test patch:
diff --git a/tornado/test/httputil_test.py b/tornado/test/httputil_test.py
index 53ae4607..221c1dcb 100644
--- a/tornado/test/httputil_test.py
+++ b/tornado/test/httputil_test.py
@@ -287,13 +287,22 @@ Foo: even
             [("Asdf", "qwer zxcv"), ("Foo", "bar baz"), ("Foo", "even more lines")],
         )
 
-    def test_malformed_continuation(self):
+    def test_continuation(self):
+        data = "Foo: bar\r\n\tasdf"
+        headers = HTTPHeaders.parse(data)
+        self.assertEqual(headers["Foo"], "bar asdf")
+
         # If the first line starts with whitespace, it's a
         # continuation line with nothing to continue, so reject it
         # (with a proper error).
         data = " Foo: bar"
         self.assertRaises(HTTPInputError, HTTPHeaders.parse, data)
 
+        # \f (formfeed) is whitespace according to str.isspace, but
+        # not according to the HTTP spec.
+        data = "Foo: bar\r\n\fasdf"
+        self.assertRaises(HTTPInputError, HTTPHeaders.parse, data)
+
     def test_unicode_newlines(self):
         # Ensure that only \r\n is recognized as a header separator, and not
         # the other newline-like unicode characters.