Code change:
This encodes the conclusions from interim discussions: CR, LF, and NUL cannot appear anywhere in field names or values. SP and HTAB cannot appear at the start or end of field names or values. COLON cannot appear anywhere in a field name, except for the colon at the start of a pseudo-header field name. The strong requirements about validating fields according to ABNF has been replaced. The text is clearer about how the pieces fit together: it is HPACK that allows any octet, whereas HTTP/2 makes certain choices invalid.
This allows whitespace and control characters in field names. Is that intentional?
This was intentional, but it can be revised by expanding the set of characters we prohibit. I had imagined that this would not be different from HTTP/1.1 where "Foo\tBar: ?1" might parse, but is not valid in the same way that "Foo\Bar: ?1" is not. I thought that we had agreed that a close policing of the field name grammar was not required from HTTP/2 implementations.
NAME take another look. I've refactored your suggestion.
Yes.
NAME if you can suggest text that would address your remaining concerns, that would be good, but I think we're going to include this text in the next draft. Of course, there will be ample time to improve on this; we just want to make sure that we have an updated draft to discuss at the upcoming interim meeting.
NAME What I think is missing is a description of what this change is trying to achieve. The text identifies that there is a set of , but notes that that is a super-set of of . It then gives text description of what the set of is, but it is not clear exactly what the intent of that set is meant to be? The possibilities are: == union . But if this was the case then why not define it in terms of the HTTP ABNF? == union union . This seems a strange thing to do, but the motivation was originally given that this was describing what existing implementations do? If this is the case then it should be described as such. == intersection . I think this is what the text is actually striving for, but a) it would be good to say so; b) it is unclear to me why h2 needs a superset of option 1. above... if so it should be said why. If it is indeed option 3. then I assume that there are some fields in the set that are nether in the nor sets. What are those fields and what should an implementation do with them? We are told later in the update that if we are an intermediary then we can validate strictly against the HTTP ABNF and that if we don't we risk invalid fields (and by inference of the section title encapsulation attacks). But what do we do with those fields if we are not an intermediary? Do we pass them onto our applications and let them work it out that they are not members of ? There is nothing there that says we MAY treat such messages as malformed - so at the very least, it would be a good addition to the text to say that non-intermediaries can do stricter validation. So in short, I can see lots of text defining , but it is entirely unmotivated as to why it is > . Finally, to harp on my original complaint, I still do not understand why is defined in 4 bullet points of prose that have already confused some readers. There is a paragraph about how field names have to be lowercase and then the bullet points ignore that and define all visible characters except space (so if space is visible does that mean tab, CR, LF are also visible?) and except colon. Some simple ABNF would be so much better: Or better yet, just go with HTTP field names, but with names in lower-case: Edit: removed comment about ESC... I would looking at an ASCII table from before I was born!
No it doesn't. It gives a text description of a set of invalid H2 fields. The text does not say these are the only limitations on h2 field validity (noting that there is also an ABNF description that applies when -semantics applies), neither does it say that any field that doesn't contain any invalid characters as described in this section is valid. I do think the text could be clearer in saying that this is a non-exhaustive list of reasons why a header field may be invalid, but at the same time I'm reluctant to have this document spend too much time re-hashing reasons that exist in other specifications that are presumed to apply. This question is one for -semantics, not for HTTP/2. -semantics gives the answer to this: you MUST NOT emit (§ 2.2), but as a receiver you are cautioned that (§ 2.3): Additionally (§ 2.4): This gives you the permission you want: you are encouraged to parse the field defensively and you may attempt to recover, which implies that by default you may fail, and that -semantics does not specify exactly how you will handle these failures. HTTP/2 continues to grant your application that freedom. The specification does not bind you: you can handle this as you want.
OK True. Which is actually worse, as it leaves the set of undefined other than we know it is a subset of the compliment of the set of invalid fields define here. Also worse, as it means the document only vaguely defines validity. There is actually no definition of what is a valid h2-field. This reason could be used to say that there should be no additional invalidity check in this document. Just allow any hpack-valid field and leave the rest to be a matter of semantics. The fields excluded by this definition are only dangerous if forwarded, which is already covered by "Intermediary Encapsulation Attacks" section that suggests that full HTTP ABNF grammar should be applied (in addition to the vaguely defined exclusions of "HTTP Fields"). So I'm back to why is this document in-precisely defining valid/invalid h2-fields. It is not for intermediaries as there is a section in the document dedicated to that. It is not for locally interpreted HTTP as you say HTTP semantics should be applied. Why does this document say that a field name containing space is invalid, yet one containing a double quote is not? Neither are valid HTTP, both are valid hpack. There is no good reason I can see for either and whilst I don't know of any specific attack that either could be used for, I would not be surprised if both could be used to some evil ends. This vague definition is going to have a significant carbon footprint. Many Implementations will ultimately end up double validating field names: once to exclude the s in their h2 layer and then later in their semantic layer to include only the s. Double validation could be avoided if this document either didn't restrict field validity or precisely defined valid fields as a subset of s so that semantic layers could trust the fields received from their h2 layers as being valid HTTP. Maybe there is a reason for this double validation and the resulting CPU cycles, but I have yet to see why validity is only partially defined.
It suggests it but does not require it. The other text requires validation. Most intermediaries will not apply the full ABNF validation because a) it requires perfect understanding of the ABNF of all header fields, which they will not have, and b) even if they did, the -semantics ABNF is sufficiently costly to validate that they won't do it. Cheaper validation steps are more likely to be implemented. This document (and -semantics) is imprecisely defining them because a . The ABNF you're citing represents, fundamentally, guidance. If you want to reject things that don't conform to the ABNF you are free to do so (as -semantics says), but there is no document that obligates you to reject them. This is deliberate, because there is (IMO) absolutely no point in adding normative requirements that, if followed, would harm interoperability. In this case the document is calling out specific cases that MUST be rejected. Practically speaking the intention of this is to encode requirements from other messaging formats, such as -messaging, to ensure that HTTP/2 implementations do not encode header fields that cannot be safely translated to HTTP/1.1. This provides a lower bound on validation: so long as everybody does this, we can safely rely on HTTP/2 messages being represented in HTTP/1.1's framing. Note that this doesn't meant they'll be valid HTTP/1.1 messages, just that they are not going to parse incorrectly. With that in mind, the constraints added here are, in order: A cheap bitwise check that can be safely vectorised and applied to reject most invalid field names. This is sufficient to get close to the ABNF, but is cheaper. A check that reserves pseudo-headers and is a single-byte comparison. Three vectorizeable field-value checks (NUL, LF, CR) all ensuring valid HTTP/1.1 framing. 4 single-byte comparisons against the first byte of the field-value. This problem is solved by not doing that. The ABNF in -semantics, if enforced, is a strict superset of the requirements here except for (2). If your implementation will enforce the requirements in -semantics, you can choose to not enforce these ones at the h2 layer, knowing that the -semantics layer will cover you. The RFC does not bind your implementation, only your observable behaviour: if your implementation ends up rejecting header fields that meet these characteristics, it doesn't matter why you did it, only that you did.
I don't get that. We're still talking about field names, right?
No, all of this applies to names and values:
Well. Validating names and values are very different things. It would be best to clearly separate these topics.
I get it that there is a minimum standard being set. In section "HTTP Fields" it says that h2 implementations MUST treat as malformed any fields that violate any of the described conditions and an intermediary MUST NOT forward any such headers. Then again in section "Intermediary Encapsulation Attacks" is says the validations of the "HTTP Fields" MUST be implemented, which is kind of a duplication as they are already at MUST strength. I don't see any normative text that says that an implementation MAY or SHOULD validate against the HTTP ABNF and that if they do so then that is a superset of the conditions in section "HTTP Fields" I would very much prefer to see text in the "HTTP Fields" section that clearly says h2 implementations MAY validate against the HTTP ABNF and that any implementations that do not MUST validate fields against the following conditions. I'll have a try at this later today.... stand by....
NAME I'm reading your comments as supportive of the technical change, but a strong preference for a different way of presenting the information. I want to clarify that this change is about the mandatory validation that endpoints perform on fields. It looks like we all agree with stipulating a minimum amount of validation that applies to all implementations, especially those that would otherwise just forward fields without processing. Additionally, we all agree that additional validation is permitted, up to the point that knowledge of the semantics for the fields is applied in validating the values. Where we might disagree is in the way that these requirements are described. I've chosen to use words; NAME would prefer ABNF. As we agree on the technical substance and the question of presentation is an editorial decision in which Cory and I have discretion I'm going to merge this pull request. We would like to publish a revision ahead of the upcoming interim. If you disagree with this decision, especially the technical aspects, then I encourage you to open a new issue. One aspect that seems like it might worth discussing is whether we encourage additional validation rather than simply permitting it. That is, "SHOULD/MAY fully validate".
NAME that's a correct summation of my position. I would have preferred that some text for SHOULD/MAY fully validate to be resolved in this PR (with or without ABNF), but will open another issue.
: While most of the values that can be encoded will not alter header field parsing, carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0) might be exploited by an attacker if they are translated verbatim. Any request or response that contains a character not permitted in a header field value MUST be treated as malformed (Section 8.1.2.6). Valid characters are defined by the "field-content" ABNF rule in Section 3.2 of [RFC7230]. There are multiple issues here: it's confusing to say "HTTP/2 allows" when next they are actually forbidden with a MUST requirement; it might be better to say something like that the wire protocol in theory enables the transmission of these forbidden characters this puts a normative requirement on top of the base spec; but so does the revision of the core specs (see URL). We need to make sure that there are no conflicting requirements. It also should be checked whether the currently present requirements are actually implemented in UAs (and if they are not, what could be done about it)
It would presumably be more accurate to say that "HPACK allows", right?
HPACK also allows encoding a field line with an empty name. I'm not sure this needs to be mentioned explicitly or not.
Well, that's consistent with HTTP/1.1 :-)
I hate having to do this, but what is possible and what is permissible need to be more clearly separated.
I'm going to tag this as editorial: the possible/permissible split is clearly editorial. And we've decided to depend on the core docs, and whether or not it was clear before what was allowed or not, this is now very crisp in the core semantics. I want to wait until we get in before doing this though.
Discussed in the context of leading and trailing whitespace in -semantics. I think that we should be clearer here about a few things: CRLF and NUL are strictly prohibited leading and trailing WS are strictly prohibited Maybe loosen the requirement a little so that it is less strict about adherence to the ABNF That makes this non-editorial, I think.

Related discussion:
text: 10.3. <del> The HTTP/2 field block encoding allows the expression of names that are not valid field names in HTTP.  Requests or responses containing invalid field names MUST be treated as malformed.  An intermediary therefore cannot translate an HTTP/2 request or response containing an invalid field name into an HTTP/1.1 message. Similarly, HTTP/2 allows field values that are not valid.  While most of the values that can be encoded will not alter header field parsing, carriage return (CR, ASCII 0xd), line feed (LF, ASCII 0xa), and the zero character (NUL, ASCII 0x0) might be exploited by an attacker if they are translated verbatim.  Any request or response that contains a character not permitted in a field value MUST be treated as malformed.  Valid characters are defined by the ABNF rule in HTTP. </del> <ins> HPACK permits encoding of field names and values that might be treated as delimiters in other HTTP versions.  An intermediary that translates an HTTP/2 request or response MUST validate fields according to the rules in HttpHeaders roles before translating a message to another HTTP version.  Translating a field that includes invalid delimiters could be used to cause recipients to incorrectly interpret a message, which could be exploited by an attacker. An intermediary can reject fields that contain invalid field names or values for other reasons, in particular those that do not conform to the HTTP ABNF grammar from HTTP.  Intermediaries that do not perform any validation of fields other than the minimum required by HttpHeaders could forward messages that contain invalid field names or values. </ins> 10.4.