Change:
What happened: I'm trying to establish a SPDY connection to exec into a pod. I sent this header: X-Stream-Protocol-Version: , , , The request failed with Forbidden - unable to upgrade: unable to negotiate protocol: client supports [, , , ], server accepts [ ] It appears that the server is not parsing the list out of the HTTP header and is treating it as a single value. What you expected to happen: Protocol negotiation should succeed. RFC2616 says: I can't simply send a multiple headers, because the HTTP library I'm using has no mechanism for sending separate headers as opposed to combining them. Kubernetes should parse the header whether it's given as separate headers or a list. Anything else we need to know?: I'm having to use SPDY because the web sockets implementation of exec . Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-05-16T16:14:56Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} Cloud provider or hardware configuration: Azure VMOS (e.g: ): NAME="Ubuntu" VERSION="16.04.6 LTS (Xenial Xerus)" ID=ubuntu IDLIKE=debian PRETTYNAME="Ubuntu 16.04.6 LTS" VERSIONID="16.04" HOMEURL="" SUPPORTURL="" BUGREPORTURL="" VERSIONCODENAME=xenial UBUNTUCODENAME=xenial Kernel (e.g. ): Linux AdamArisDev 4.15.0-1063-azure -Ubuntu SMP Fri Nov 8 09:30:20 UTC 2019 x8664 x8664 x8664 GNU/LinuxOthers: My cluster is running in minikube v1.1.0.
/sig api-machinery
Thinking of the following change:
Since none of the accepted protocols include a comma, we should probably just iterate over all the headers, split by , trim each resulting item, omitting empty strings when computing clientProtocols, regardless of the number of headers.
Iterate over
Thanks for the quick update. I don't know what the expected HTTP behavior is when you use multiple headers and also have lists in each header, but if the same rule applies that multiple headers should be treated the same as one header with the items separated by commas, then presumably: Foo: a, b Foo: c, d should be the same as Foo: a, b, c, d in which case suggestion of basically moving the split into the inner loop would be more correct. In any case, I suppose a bug fix of this nature would only affect newer versions of Kubernetes? (I don't know if you do bug-fix-only patches to older versions.) I guess as a workaround I could try to catch the Forbidden error, look for the "unable to negotiate protocol" text, and switch to using only a single header value for older Kubernetes versions. Or is there a better approach?
That said, presumably this could also be handled in a lower layer - in the HTTP layer where headers are parsed - which might fix similar bugs with other headers that might exist elsewhere. (I think the only general exception to the equivalency rule is the Set-Cookie header because in practice cookies can contain commas but aren't quoted.)
That's correct. Given all the currently supported protocols are available in all supported kubernetes versions (the most recent protocols were in in v1.2.0 and in in v1.4.0), I'd probably just pick the specific protocol you want. If you wanted to detect a protocol failure, the X-Accepted-Stream-Protocol-Versions header is sent on a forbidden response due to protocol negotiation failure.
That's good to know. I can just stick to v4, then, since I don't expect to need anything that old. :-) Thanks.
/reopen I agree, should this fix be more general? Some other http headers have also the comma problems.
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I would not recommend that, as not all headers value handling can safely assume commas are value delimiters.
I would keep this issue scoped to this particular header
/assign (since you are already working on this one, thanks!)
/close fixed in
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
return "" } <ins> func commaSeparatedHeaderValues(header []string) []string { var parsedClientProtocols []string for i := range header { for _, clientProtocol := range strings.Split(header[i], ",") { if proto := strings.Trim(clientProtocol, " "); len(proto) > 0 { parsedClientProtocols = append(parsedClientProtocols, proto) } } } return parsedClientProtocols } </ins> // Handshake performs a subprotocol negotiation. If the client did request a // subprotocol, Handshake will select the first common value found in // serverProtocols. If a match is found, Handshake adds a response header

// returned, along with a response header containing the list of protocols the // server can accept. func Handshake(req *http.Request, w http.ResponseWriter, serverProtocols []string) (string, error) { <del> clientProtocols := req.Header[http.CanonicalHeaderKey(HeaderProtocolVersion)] </del> <ins> clientProtocols := commaSeparatedHeaderValues(req.Header[http.CanonicalHeaderKey(HeaderProtocolVersion)]) </ins> if len(clientProtocols) == 0 { return "", fmt.Errorf("unable to upgrade: %s is required", HeaderProtocolVersion) }

expectedProtocol: "", expectError:      true, }, <ins> "no common protocol with comma separated list": { clientProtocols:  []string{"c, d"}, serverProtocols:  []string{"a", "b"}, expectedProtocol: "", expectError:      true, }, </ins> "common protocol": { clientProtocols:  []string{"b"}, serverProtocols:  []string{"a", "b"}, expectedProtocol: "b", }, <ins> "common protocol with comma separated list": { clientProtocols:  []string{"b, c"}, serverProtocols:  []string{"a", "b"}, expectedProtocol: "b", }, </ins> } for name, test := range tests {