CVE-2015-5739 (published 2017-10-18T20:29Z)

The net/http library in net/textproto/reader.go in Go before 1.4.3 does not properly parse HTTP header keys, which allows remote attackers to conduct HTTP request smuggling attacks via a space instead of a hyphen, as demonstrated by "Content Length" instead of "Content-Length."

Severity: HIGH
CVSS v2 base score: 7.5 (AV:N/AC:L/Au:N/C:P/I:P/A:P)
CVSS v3 base score: 9.8, CRITICAL (CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H)

Weakness information:
cwe_id: CWE-444
cwe_name: Inconsistent Interpretation of HTTP Requests ('HTTP Request/Response Smuggling')
description: The product acts as an intermediary HTTP agent
         (such as a proxy or firewall) in the data flow between two
         entities such as a client and server, but it does not
         interpret malformed HTTP requests or responses in ways that
         are consistent with how the messages will be processed by
         those entities that are at the ultimate destination.
extended_description: 
	   HTTP requests or responses ("messages") can be
	   malformed or unexpected in ways that cause web servers or
	   clients to interpret the messages in different ways than
	   intermediary HTTP agents such as load balancers, reverse
	   proxies, web caching proxies, application firewalls,
	   etc. For example, an adversary may be able to add duplicate
	   or different header fields that a client or server might
	   interpret as one set of messages, whereas the intermediary
	   might interpret the same sequence of bytes as a different
	   set of messages. For example, discrepancies can arise in
	   how to handle duplicate headers like two Transfer-encoding
	   (TE) or two Content-length (CL), or the malicious HTTP
	   message will have different headers for TE and
	   CL.
	   The inconsistent parsing and interpretation of messages
	   can allow the adversary to "smuggle" a message to the
	   client/server without the intermediary being aware of it.
	   This weakness is usually the result of the usage
	   of outdated or incompatible HTTP protocol versions in the
	   HTTP agents.
         
         

Fix information:
repo_url: https://github.com/golang/go
commit_details: repo_url: https://github.com/golang/go
author: Brad Fitzpatrick
author_date: 2015-06-30 09:22:41-07:00
msg: net/textproto: don't treat spaces as hyphens in header keys

This was originally done in https://codereview.appspot.com/5690059
(Feb 2012) to deal with bad response headers coming back from webcams,
but it presents a potential security problem with HTTP request
smuggling for request headers containing "Content Length" instead of
"Content-Length".

Part of overall HTTP hardening for request smuggling. See RFC 7230.

Thanks to Régis Leroy for the report.

Change-Id: I92b17fb637c9171c5774ea1437979ae2c17ca88a
Reviewed-on: https://go-review.googlesource.com/11772
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
repository: repo_url: https://github.com/golang/go
repo_name: golang/go
description: The Go programming language
repo_language: Go
owner: golang
file_changes: filename: header.go
change_type: ModificationType.MODIFY
diff: @@ -168,6 +168,8 @@ func (h Header) WriteSubset(w io.Writer, exclude map[string]bool) error {
 // letter and any letter following a hyphen to upper case;
 // the rest are converted to lowercase.  For example, the
 // canonical key for "accept-encoding" is "Accept-Encoding".
+// If s contains a space or invalid header field bytes, it is
+// returned without modifications.
 func CanonicalHeaderKey(s string) string { return textproto.CanonicalMIMEHeaderKey(s) }
 
 // hasToken reports whether token appears with v, ASCII

programming_language: Go
filename: reader.go
change_type: ModificationType.MODIFY
diff: @@ -547,11 +547,16 @@ func (r *Reader) upcomingHeaderNewlines() (n int) {
 // the rest are converted to lowercase.  For example, the
 // canonical key for "accept-encoding" is "Accept-Encoding".
 // MIME header keys are assumed to be ASCII only.
+// If s contains a space or invalid header field bytes, it is
+// returned without modifications.
 func CanonicalMIMEHeaderKey(s string) string {
 	// Quick check for canonical encoding.
 	upper := true
 	for i := 0; i < len(s); i++ {
 		c := s[i]
+		if !validHeaderFieldByte(c) {
+			return s
+		}
 		if upper && 'a' <= c && c <= 'z' {
 			return canonicalMIMEHeaderKey([]byte(s))
 		}
@@ -565,19 +570,44 @@ func CanonicalMIMEHeaderKey(s string) string {
 
 const toLower = 'a' - 'A'
 
+// validHeaderFieldByte reports whether b is a valid byte in a header
+// field key. This is actually stricter than RFC 7230, which says:
+//   tchar = "!" / "#" / "$" / "%" / "&" / "'" / "*" / "+" / "-" / "." /
+//           "^" / "_" / "`" / "|" / "~" / DIGIT / ALPHA
+//   token = 1*tchar
+// TODO: revisit in Go 1.6+ and possibly expand this. But note that many
+// servers have historically dropped '_' to prevent ambiguities when mapping
+// to CGI environment variables.
+func validHeaderFieldByte(b byte) bool {
+	return ('A' <= b && b <= 'Z') ||
+		('a' <= b && b <= 'z') ||
+		('0' <= b && b <= '9') ||
+		b == '-'
+}
+
 // canonicalMIMEHeaderKey is like CanonicalMIMEHeaderKey but is
 // allowed to mutate the provided byte slice before returning the
 // string.
+//
+// For invalid inputs (if a contains spaces or non-token bytes), a
+// is unchanged and a string copy is returned.
 func canonicalMIMEHeaderKey(a []byte) string {
+	// See if a looks like a header key. If not, return it unchanged.
+	for _, c := range a {
+		if validHeaderFieldByte(c) {
+			continue
+		}
+		// Don't canonicalize.
+		return string(a)
+	}
+
 	upper := true
 	for i, c := range a {
 		// Canonicalize: first letter upper case
 		// and upper case after each dash.
 		// (Host, User-Agent, If-Modified-Since).
 		// MIME headers are ASCII only, so no Unicode issues.
-		if c == ' ' {
-			c = '-'
-		} else if upper && 'a' <= c && c <= 'z' {
+		if upper && 'a' <= c && c <= 'z' {
 			c -= toLower
 		} else if !upper && 'A' <= c && c <= 'Z' {
 			c += toLower

programming_language: Go
filename: reader_test.go
change_type: ModificationType.MODIFY
diff: @@ -24,11 +24,14 @@ var canonicalHeaderKeyTests = []canonicalHeaderKeyTest{
 	{"uSER-aGENT", "User-Agent"},
 	{"user-agent", "User-Agent"},
 	{"USER-AGENT", "User-Agent"},
-	{"üser-agenT", "üser-Agent"}, // non-ASCII unchanged
+
+	// Non-ASCII or anything with spaces or non-token chars is unchanged:
+	{"üser-agenT", "üser-agenT"},
+	{"a B", "a B"},
 
 	// This caused a panic due to mishandling of a space:
-	{"C Ontent-Transfer-Encoding", "C-Ontent-Transfer-Encoding"},
-	{"foo bar", "Foo-Bar"},
+	{"C Ontent-Transfer-Encoding", "C Ontent-Transfer-Encoding"},
+	{"foo bar", "foo bar"},
 }
 
 func TestCanonicalMIMEHeaderKey(t *testing.T) {
@@ -194,7 +197,7 @@ func TestReadMIMEHeaderNonCompliant(t *testing.T) {
 		"Foo":              {"bar"},
 		"Content-Language": {"en"},
 		"Sid":              {"0"},
-		"Audio-Mode":       {"None"},
+		"Audio Mode":       {"None"},
 		"Privilege":        {"127"},
 	}
 	if !reflect.DeepEqual(m, want) || err != nil {

programming_language: Go