jiebi/CodeConvo huggingface dataset:
Code change:
header. The dictionary to use is negotiated separately and advertised in the <del> "Available-Dictionary" request header and "Content-Dictionary" response header. </del> <ins> "Available-Dictionary" request header. </ins> 6.1.

Related discussion:
text: On the header side of things, how do you all feel with respect to a bare 32-byte hash vs a 35-byte header with a 3-byte signature followed by the hash (or a 3-byte signature followed by a 1-byte header size followed by the hash to allow for changes as well as 4-byte alignment)? It's possible I'm mentally stuck in the old days of sniffing content but since the hash can literally be any value, including accidentally looking like something else, I like the explicit nature of a at the beginning of the stream. It essentially becomes (or whatever the signature is) so it doesn't add significantly to the complexity but it does add 3 bytes, potentially unnecessarily.
+1 to adding a 3 byte magic signature if that's the route we're taking.
I'm ambivalent on the added signature, so I'll defer to others. I can see how that might help a command-line tool more easily distinguish between this and a genuine brotli-/zstd- compressed file and do the right thing with it. On the other hand, it's 3 more bytes and - if the formats themselves have a magic sequence - the same tools could equally skip 32 bytes and check for their magic there.
I think I am having the same response as you all in the opposite direction, where to me it feels preferable to make it HTTP's problem so that my layer doesn't have to deal with additional complexity. :smiley: But if I overcome that bias and accept that it would be nice to avoid an additional HTTP header, here's what I think: If we used a Zstd skippable frame, that would change the stream overhead to 8 bytes (4 byte magic + 4 byte length) + 32 byte hash. But it would mean that existing binaries would be able to decode the frames and avoid ecosystem fragmentation. That's a lot more attractive to me than a new format along the lines of which (1) existing tools won't understand and (2) isn't a format I'd feel comfortable auto-detecting by default in the CLI because isn't all that unlikely a beginning of a string in the way that is (non-ASCII, invalid UTF-8). And I've thought about it more and I'm actually not concerned about colliding a skippable frame type with someone else's existing use case.
Code change:
header. The dictionary to use is negotiated separately and advertised in the <del> "Available-Dictionary" request header and "Content-Dictionary" response header. </del> <ins> "Available-Dictionary" request header. </ins> 6.1.

Related discussion:
text: It would be more of a problem if we were trying to spec a universal solution, but if we scope this to just this content-encoding, then we're free to reserve whatever code points we want and attach whatever semantics we want.
Thanks. I guess the main question I have is if there would be benefits to Zstandard itself for the files to be self-contained with the identification of the dictionary that they were encoded with or the possibility of mismatching dictionaries on compression and decompression (or being able to find the matching dictionary given just the compressed file) are issues that are HTTP-specific. I'm fine with specifying that the encoded files carry the dictionary hash (before the compressed stream data) and having different ways for Zstandard and Brotli to do that actual embedding. That said, the tooling gets more complicated on the encode and decode side to generate the frame and insert it in the correct place in the compressed file and at decompression time, that makes the client more format-aware, having to parse more of the stream to extract the dictionary and then re-send the full stream through the decoder (at least until the decoder library becomes aware of embedded dictionary hash).
NAME Is this really what you want in this case? The decoder needs to know where to find the dictionary, so wouldn't you want this to be a breaking change to the format such that a decoder that has a dictionary is fine and a decoder that doesn't knows to go get one. ... And - importantly - an older decoder will abort. (I confess that I don't know what the zstd frame extension model is and didn't check.)
I don't think we're contemplating a model where Zstd can ingest a frame and figure out on its own where to find the dictionary it needs and then load it and use it. I expect that the enclosing application will parse this header, get the hash, and then find the dictionary and provide it to Zstd. The advantage to using the skippable frame is that then you can provide the whole input to Zstd (including existing versions) and it will work rather than have to pull the header off.
One thing I realized now - by assuming that the hash length is 32 bytes, we're assuming the hash would remain SHA-256 forever. That might be how things play out, but it that we'd need to change hashes at some point.
Code change:
header. The dictionary to use is negotiated separately and advertised in the <del> "Available-Dictionary" request header and "Content-Dictionary" response header. </del> <ins> "Available-Dictionary" request header. </ins> 6.1.

Related discussion:
text: If we were to do that, having a fixed length hash as part of the format would make things more complex.
Having a fixed-length hash (or fixed hash) as part of a content coding is perfectly fine. If there is a need to update hashes, it is easy to define a new content coding.
NAME the doesn't use skippable frames and uses the same custom header for Brotli and Zstandard (with different magic numbers). I can switch to using a skippable frame instead (which effectively just becomes an 8-byte magic number since the frame length is always the same) but I'm wondering if it makes sense and is worth adding 4 bytes. It won't help in creating the files so it's just for decode time and the main benefit you get is that you can use the existing zstd cli and libraries to decode the stream without skipping the header but those also won't verify the dictionary hash, they will just skip over it. That might not be a problem but part of the decode process will be to fail the request if the hashes don't match.
In talking to the Brotli team, it looks like Brotli already and validates it during decode to select the dictionary to use. It uses a "256-bit Highwayhash checksum" so we can't use the hash to lookup a dictionary indexed by SHA-256 but we can use it to guarantee the decompression doesn't use a different dictionary (and the existing libraries and cli tools already use it). NAME when you were concerned about the , was it for both lookup and validation or just validation? I'm just wondering if we can use the existing brotli streams as they already exist or if we should still add a header to support locating the dictionary by SHA-256 hash.
There are a few places in HTTP where the interpretation of a response depends on something in the request. Everyone one of those turns out to be awful for generic processing of responses. That's my main reasoning, so I'd say both: lookup first, then validation. That is, my expectation is not that the client has a singular candidate dictionary, or that it commits to one in particular. So in the original design, when you had the client pick one, that didn't seem like a great idea to me. For validation, Highwayhash (NIH much?)
Code change:
header. The dictionary to use is negotiated separately and advertised in the <del> "Available-Dictionary" request header and "Content-Dictionary" response header. </del> <ins> "Available-Dictionary" request header. </ins> 6.1.

Related discussion:
text: doesn't appear to be pre-image resistant, so I'd be concerned if we were relying on the pre-image resistance properties of SHA-2. Can we be confident that this is not now subject to potential ployglot attacks if we relied on that hash alone? That is, could two clients think that they have the same resource, but do not?
I wouldn't be comfortable switching hashes given the wider and proven use of SHA-256 and agree on the lookup case to allow content-encoding with different and multiple dictionary negotiations. Looks like a separate header is still the cleanest so the main remaining question is if we use the same style of header for both or a skippable frame for Zstandard. I also need to update the brotli citations to point to the shared dictionary draft format instead of the original brotli format. On Sun, May 12, 2024 at 7:58 PM Martin Thomson NAME wrote: >
Sorry for the late comment. It seems to me that conveying this information in the content eases the integration with Signatures and Digest. It is not clear to me if there are still possible cases where the response does not contain all the information required for processing.
Thanks
Looks like you just need to update the PR summary to reflect this change.LGTMOverall, I like this. I want to flag for others that it doesn't use the skippable frame concept.
Code change:
map of parameters. input_string is modified to remove the parsed value. <del> Let primary_identifier be the result of Parsing a Token from Text (parse-token) from input_string. </del> <ins> Let primary_item be the result of Parsing an item from Text (parse-item) from input_string. </ins> Let parameters be an empty, ordered map.

Related discussion:
text: in , NAME asks about having Parameterised Lists whose parameterised identifiers are things other than Tokens. Because of the way SH is put together, I think this would be a fairly easy change, in terms of the spec and implementations; it would require a few more tests. The only downside I can think of immediately is that specs would need to constrain the type of the parameterised identifier themselves, and specify error handling (if it weren't the default "blow up"). Thoughts? I also wonder if we should come up with a better name than "parameterised identifier", especially since we've ditched "identifier" elsewhere. member name? member id? primary identifier?
That works, since splitting lists from parameterised lists. Are you thinking , or just adding a few types? It also changes my thinking a tiny bit on , although not much. I like "parameterised element" but that introduces a new word. "parameterised item" is a nice shade, if any item is allowed there, but "primary identifier" works too. :art: :paintbrush:
The downside here is that it would be very hard to represent in JSON -- e.g., in the . At least some programming languages support hashes/dictionaries/objects with arbitrary key types (e.g., Python, JavaScript, Ruby), but I'm not sure how widespread that is -- especially if folks represent the different between and with an object for . NAME is this a nice-to-have or something that's a blocker?
(Fwiw I do think we should call this a Parameterised Token)
At this point do we just have to accept that the SH data model is essentially a subset of the JavaScript model? Numbers are bounded by JavaScript's limitations, object keys have to be stringy, nobody likes 'token' because it isn't a native JS type... I don't mind if that's the case, but it would be good to just say it outright somewhere. It would also give us a solid position to start from when considering other proposals. Hmm, C++ and Go could be have issues mixing key types within a single map, and Perl is definitely not going to like non-string keys.
It doesn't have to be -- it's just easier for me to write tests this way :)
This was a nice-to-have for me. I’d worked around it by putting an unused string as the identifier. It looks like that workaround is no longer necessary?
Code change:
2.1.4. <del> The "type" value of the Use-As-Dictionary header is a sf-string value </del> <ins> The "type" value of the Use-As-Dictionary header is a Token value </ins> that describes the file format of the supplied dictionary. "raw" is the only defined dictionary format which represents an

Related discussion:
text: Switched from using the ABNF field types to the human names (i.e. sf-dictionary to Dictionary) for all of the Structured Field header values. Switched dictionary "type" to a Token (from String).
NAME could you PTAL at the change in referencing the header field types? Specifically, I wasn't sure if I should capitalize the types or not (I did since that's how the structured fields RFC referred to them).
Yep, this looks good. It's common to add something to you Notational Conventions section (or whatever) that says roughly "This specification uses the following terms from [ref to SF]: String, Byte Sequence, Token, ..."
Code change:
behalf of a resource owner by orchestrating an approval interaction between the resource owner and the HTTP service, or by allowing the third-party application to obtain access on its own behalf.  This <del> specification replaces and obsoletes the OAuth 1.0 protocol described in RFC 5849. </del> <ins> specification replaces and obsoletes the OAuth 2.0 Authorization Framework described in RFC 6749. </ins> 1.

Related discussion:
text: Edited intro to reflect what this document is about changed some OAuth 2.0 references to OAuth 2.1 when referring to this document changed document name to follow ID standard name (repo and filename should be updated as well?) pulled out acknowledgments as this doc will be different
Code change:
not obligated to maintain the available flow control window for streams that it is no longer sending on. <del> Flow control can be disabled for all streams or the connection using </del> <ins> Flow control can be disabled for all streams on the connection using </ins> the SETTINGS_FLOW_CONTROL_OPTIONS setting.  An implementation that <del> does not wish to perform flow control can use this in the initial SETTINGS exchange. </del> <ins> does not wish to perform stream flow control can use this in the initial SETTINGS exchange. </ins> Flow control can be disabled for an individual stream or the overall connection by sending a WINDOW_UPDATE with the END_FLOW_CONTROL flag

Related discussion:
text: Curious why the connection flow control window initial size is 64K (65536 bytes) but the stream flow control window initial size is 65535 bytes. The negative flow control example (taken from the SPDY spec) states that the initial stream flow control window is 64K (the SPDY/3 value).
I have to stop trusting Will's edits. He, rightfully, challenged the use of SETTINGS to end connection-level flow control because WINDOW_UPDATE provided another way to do the same, and two ways to do the same thing is bad. I'll fix that up.
Code change:
than needing to subvert DNS or IP routing in order to use a compromised certificate, a malicious server now only needs a client to connect to _some_ HTTPS site under its control in order to present <del> the compromised certificate.  As recommended in RFC8336, clients opting not to consult DNS ought to employ some alternative means to increase confidence that the certificate is legitimate. </del> <ins> the compromised certificate.  Clients SHOULD consult DNS for hostnames presented in secondary certificates if they would have done so for the same hostname if it were present in the primary certificate. As recommended in RFC8336, clients opting not to consult DNS ought to employ some alternative means to increase confidence that the certificate is legitimate. </ins> One such means is the Required Domain certificate extension defined in {extension}. Clients MUST require that server certificates

Related discussion:
text: The intent here is to say that your requirements for trusting a secondary certificate should be at least as strict as for trusting additional names in the primary certificate.
Add text noting the DNS check / not-check guidance in RFCs 7540 and 8336, respectively. Explicitly side-step the issue by saying that if you would do a DNS lookup for a hostname in the primary cert, you MUST do the DNS check for a hostname in a secondary cert as well; if you would omit the DNS lookup for a hostname in the primary cert, you MAY omit the DNS check for a hostname in the secondary cert as well.
Code change:
A specification can help potential contributors understand the algorithm, which can make it easier for them to suggest <del> improvements and/or identify limitations.  Further, the </del> <ins> improvements and/or identify limitations.  Furthermore, the </ins> specification can help multiple contributors align on a consensus change to the algorithm.

Related discussion:
text: Thanks for the edits. I am a big fan of losing fewer words, so I reverted changes that went in the wrong direction.
Code change:
Clients and servers MUST terminate the TCP connection if either peer does not begin with a valid connection header.  A <del> frame (GTFO) MAY be omitted if it is clear that the peer is not using HTTP/2. </del> <ins> frame (GOAWAY) MAY be omitted if it is clear that the peer is not using HTTP/2.0. </ins> 4.

Related discussion:
text: This is almost a straight revert of the various GOAWAY -> GTFO changes. It seems like this is the popular move as per the discussion on the httpbis mailing list. Arguments for the revert: GOAWAY is self-descriptive and serves it purpose well. Editorial changes that don't clearly improve the spec should be discouraged so we can reach a stable spec sooner (granted this change doesn't affect the wire at all, but documentation churn counts too). GTFO is crass and doesn't belong in internet standards. GTFO, if it really stands for "General Termination of Future Operations," has a surprising definition and may produce confusion. We should strive for absolute clarity above all else in editorial changes.
Boo! You guys are no fun.
goaway is just as bad as gtfo for the same reasons as the argument against gtfo... just called it disconnectclient or endcommunication if you're going to be a grumplestiltskin.
QUENCH may be a better, more descriptive word that has been used before in RFC's (for TCP Source Quench messages)
I think we should call it BIKESHED. , Sean McElroy EMAIL:
Here we go.. useless HN circle jerk.
:cat2:
How about STFU?
Boo!
:-1:
This commit should be reverted. It's time we stop behaving like oversensitive automatons and display the beautiful human quality that is a sense of humor every once in a while.
^ this
^ that
how about something like PLZSTOP? it's slightly humorous, but it actually reflects what is wanted (if I understood the spec correctly)
GOAWAY is clearer for non-native English speakers, easier to pronounce, and more professionally appropriate than the common connotational definition of GTFO. There are times and places for pushing the boundaries of professional speech, but the HTTP2 spec isn't one of them.
At least the pendulum didn't swing the other way NICETALKINGTOYOU
This revert is for the express purpose of stopping endless bikeshedding. We shouldn't encourage trivial changes to frame names. If the name is good enough, it should stay. I'm all for fun and games, but clarity and concision (as well as spec stability) come first.
Code change:
Clients and servers MUST terminate the TCP connection if either peer does not begin with a valid connection header.  A <del> frame (GTFO) MAY be omitted if it is clear that the peer is not using HTTP/2. </del> <ins> frame (GOAWAY) MAY be omitted if it is clear that the peer is not using HTTP/2.0. </ins> 4.

Related discussion:
text: For those that prefer GTFO, please explain how it is clearer than GOAWAY (please consider an international audience too).
so mean spirited, this change.
Could always name it SHUTDOWN (yay SCTP) , Adrian Cole EMAIL:
"International audience" here. I'm Armenian, English isn't my native language, not even my 2nd language. GTFO to me is more meaningful and more concise than GOAWAY, or any other option mentioned in the comments. GTFO to me doesn't produce any confusion and its actual definition of "General Termination of Future Operations" is actually nice, smart, and descriptive rather than "surprising" in any negative sense. It also formally de-crasses the word. Of course you can tell me that I'm just a single example, but then again you can tell that to any amount of people. Pretty much the only valid argument out of the posted ones is that GTFO is crass (see I didn't know what "crass" meant before reading this). Everything else about descriptiveness and understandability by "international audience" are in my opinion totally irrelevant and unfair.
NAME thanks for the internationalization datapoint and on-topic discussion! Can you elaborate on how GOAWAY is not clear, and how GTFO is clearer? To me, the main argument is that GTFO is not meaningfully more clear than GOAWAY.
If folks have serious contributions to make, they should be on the mailing list, not here, as per URL Now I get to try out NAME
NAME START
NAME Neither option is appropriate or clear... they're both childish choices and neither are semantically appropriate. I've seen some much better options in this thread. if this change were actually about caring about professionalism, clarity of meaning, or necessity, the innumerable amount of better choices (SHUTDOWN, ENDCONNECTION, QUENCH, etc...) should have been used over GOAWAY or GTFO... But its not.. this is about personal preference. this change made nothing better, it made nothing worse. In my opinion, for that reason, it should be reverted.
FWIW, and for lack of anyone else talking about a pro of GTFO. When at the working group, a lot of thick topics were discussed at length and everyone there seemed 100% dedicated to having the best spec there is.
Code change:
Clients and servers MUST terminate the TCP connection if either peer does not begin with a valid connection header.  A <del> frame (GTFO) MAY be omitted if it is clear that the peer is not using HTTP/2. </del> <ins> frame (GOAWAY) MAY be omitted if it is clear that the peer is not using HTTP/2.0. </ins> 4.

Related discussion:
text: GTFO, as a word, is harmless to implementation for reasons including the opcode is the same. IOTW, the binary representation is the same. There's no technical reason why it matters. I am one of the implementors of this specification. When the change was suggested towards GTFO, I felt motivated I mean the audience of this spec are implementors, some of which may be uptight about crassness others less so. If you look at github (ps this is on github) there's ample evidence that implementors are motivated by words that aren't boring. For example, there's a popular package manager called "fpm". Guess what that stands for? I'm not saying go back and re-word everything to be fresh, rather have patience with those who are literally implementing this, in open source, and are ok with the choice. Expect many more implementors to arise from github, a place relatively unburdened by crass-ness or location.
NAME I'll split the answer to 2 sections. AWAY vs. Out -- AWAY implies a certain longish physical distance, whereas Out seems to imply just "out". AWAY vs "General Termination of Future Operations" -- obviously the second is much more clear and descriptive.
GTFO would be my vote. No future operations, full connection termination, it describes the action best.
NAME if you feel strongly, mail on this thread as not everyone there are on github to see your pov URL
NAME - will do, thanks. I need to dig my W3C mailing list info; haven't used it in years. Cheers!
From the W3C mailing list: Thanks, NAME :) Cheers!
All made points aside, I couldn't find a very important one; to me actually being the most important. If you think about what "Get Out!" could also mean (the offensive TF part is actually interchangable...), it becomes even more confusing. An expression of disbelief is surely the very last thing this acronym should imply. But it did to me; even as a non-native speaker (German). "General Termination of Future Operations" is, also in my eyes, a made-up term to fit the joke.