jiebi/CodeConvo huggingface dataset:
Code change:
IANA is asked to enter the following into the "HTTP Content Coding Registry" registry (HTTP): <del> Name: zstd-d </del> <ins> Name: dcz </ins> Description: "Dictionary-Compressed Zstandard" data format.

Related discussion:
text: Delta-compressed responses will be a few bytes smaller because the header (name and value) is larger than the file header (10's of bytes, not huge by any means). In the dynamic resource case where multiple responses will re-use the same dictionary, the header can be compressed away with HPACK/QPACK, making the header case a bit smaller. I don't think it's a big change in complexity/fragility one way or the other but it does feel like there are fewer moving pieces once we get to a place where the tooling is taken care of to have the file contents themselves specify the dictionary they were compressed with. The need for tooling changes would delay adoption a little bit so it's not a free decision but I want to make sure we don't sacrifice future use cases and simplicity for an easier launch.
I'm not sure that I see the tooling process as critical, relative to the robustness. And performance will be a wash (though I tend to view the first interaction as more important than repeated interactions). For tooling, if this content is produced programmatically, then there should be no issue. Integrated tooling can do anything. If content is produced and stored in files, then seems fine. That's a one-way operation generally. I don't see the definition of new media types to be necessary as part of that. I don't see these files being used outside of HTTP, ever. Maybe you could teach the command line decompression tools to recognize the format and complain in a useful way, but that's about the extent of the work I'd undertake there. You could do as NAME suggests as well, which would be even more useful, but given the usage context, that's of pretty narrow applicability.
I'm mostly concerned about official tooling and the latency of getting them to where developers need them to be. Compression dictionaries already require jumping through some , due to latency between brotli releases and adoption by the different package managers. At the same time, if y'all feel strongly about this, this extra initial complexity won't be a deal breaker.
I think there are enough robustness benefits that it is worth some short-term pain that hopefully we will all forget about in a few years.
Code change:
IANA is asked to enter the following into the "HTTP Content Coding Registry" registry (HTTP): <del> Name: zstd-d </del> <ins> Name: dcz </ins> Description: "Dictionary-Compressed Zstandard" data format.

Related discussion:
text: On the header side of things, how do you all feel with respect to a bare 32-byte hash vs a 35-byte header with a 3-byte signature followed by the hash (or a 3-byte signature followed by a 1-byte header size followed by the hash to allow for changes as well as 4-byte alignment)? It's possible I'm mentally stuck in the old days of sniffing content but since the hash can literally be any value, including accidentally looking like something else, I like the explicit nature of a at the beginning of the stream. It essentially becomes (or whatever the signature is) so it doesn't add significantly to the complexity but it does add 3 bytes, potentially unnecessarily.
+1 to adding a 3 byte magic signature if that's the route we're taking.
I'm ambivalent on the added signature, so I'll defer to others. I can see how that might help a command-line tool more easily distinguish between this and a genuine brotli-/zstd- compressed file and do the right thing with it. On the other hand, it's 3 more bytes and - if the formats themselves have a magic sequence - the same tools could equally skip 32 bytes and check for their magic there.
I think I am having the same response as you all in the opposite direction, where to me it feels preferable to make it HTTP's problem so that my layer doesn't have to deal with additional complexity. :smiley: But if I overcome that bias and accept that it would be nice to avoid an additional HTTP header, here's what I think: If we used a Zstd skippable frame, that would change the stream overhead to 8 bytes (4 byte magic + 4 byte length) + 32 byte hash. But it would mean that existing binaries would be able to decode the frames and avoid ecosystem fragmentation. That's a lot more attractive to me than a new format along the lines of which (1) existing tools won't understand and (2) isn't a format I'd feel comfortable auto-detecting by default in the CLI because isn't all that unlikely a beginning of a string in the way that is (non-ASCII, invalid UTF-8). And I've thought about it more and I'm actually not concerned about colliding a skippable frame type with someone else's existing use case.
Code change:
IANA is asked to enter the following into the "HTTP Content Coding Registry" registry (HTTP): <del> Name: zstd-d </del> <ins> Name: dcz </ins> Description: "Dictionary-Compressed Zstandard" data format.

Related discussion:
text: It would be more of a problem if we were trying to spec a universal solution, but if we scope this to just this content-encoding, then we're free to reserve whatever code points we want and attach whatever semantics we want.
Thanks. I guess the main question I have is if there would be benefits to Zstandard itself for the files to be self-contained with the identification of the dictionary that they were encoded with or the possibility of mismatching dictionaries on compression and decompression (or being able to find the matching dictionary given just the compressed file) are issues that are HTTP-specific. I'm fine with specifying that the encoded files carry the dictionary hash (before the compressed stream data) and having different ways for Zstandard and Brotli to do that actual embedding. That said, the tooling gets more complicated on the encode and decode side to generate the frame and insert it in the correct place in the compressed file and at decompression time, that makes the client more format-aware, having to parse more of the stream to extract the dictionary and then re-send the full stream through the decoder (at least until the decoder library becomes aware of embedded dictionary hash).
NAME Is this really what you want in this case? The decoder needs to know where to find the dictionary, so wouldn't you want this to be a breaking change to the format such that a decoder that has a dictionary is fine and a decoder that doesn't knows to go get one. ... And - importantly - an older decoder will abort. (I confess that I don't know what the zstd frame extension model is and didn't check.)
I don't think we're contemplating a model where Zstd can ingest a frame and figure out on its own where to find the dictionary it needs and then load it and use it. I expect that the enclosing application will parse this header, get the hash, and then find the dictionary and provide it to Zstd. The advantage to using the skippable frame is that then you can provide the whole input to Zstd (including existing versions) and it will work rather than have to pull the header off.
One thing I realized now - by assuming that the hash length is 32 bytes, we're assuming the hash would remain SHA-256 forever. That might be how things play out, but it that we'd need to change hashes at some point.
Code change:
IANA is asked to enter the following into the "HTTP Content Coding Registry" registry (HTTP): <del> Name: zstd-d </del> <ins> Name: dcz </ins> Description: "Dictionary-Compressed Zstandard" data format.

Related discussion:
text: If we were to do that, having a fixed length hash as part of the format would make things more complex.
Having a fixed-length hash (or fixed hash) as part of a content coding is perfectly fine. If there is a need to update hashes, it is easy to define a new content coding.
NAME the doesn't use skippable frames and uses the same custom header for Brotli and Zstandard (with different magic numbers). I can switch to using a skippable frame instead (which effectively just becomes an 8-byte magic number since the frame length is always the same) but I'm wondering if it makes sense and is worth adding 4 bytes. It won't help in creating the files so it's just for decode time and the main benefit you get is that you can use the existing zstd cli and libraries to decode the stream without skipping the header but those also won't verify the dictionary hash, they will just skip over it. That might not be a problem but part of the decode process will be to fail the request if the hashes don't match.
In talking to the Brotli team, it looks like Brotli already and validates it during decode to select the dictionary to use. It uses a "256-bit Highwayhash checksum" so we can't use the hash to lookup a dictionary indexed by SHA-256 but we can use it to guarantee the decompression doesn't use a different dictionary (and the existing libraries and cli tools already use it). NAME when you were concerned about the , was it for both lookup and validation or just validation? I'm just wondering if we can use the existing brotli streams as they already exist or if we should still add a header to support locating the dictionary by SHA-256 hash.
There are a few places in HTTP where the interpretation of a response depends on something in the request. Everyone one of those turns out to be awful for generic processing of responses. That's my main reasoning, so I'd say both: lookup first, then validation. That is, my expectation is not that the client has a singular candidate dictionary, or that it commits to one in particular. So in the original design, when you had the client pick one, that didn't seem like a great idea to me. For validation, Highwayhash (NIH much?)
Code change:
IANA is asked to enter the following into the "HTTP Content Coding Registry" registry (HTTP): <del> Name: zstd-d </del> <ins> Name: dcz </ins> Description: "Dictionary-Compressed Zstandard" data format.

Related discussion:
text: doesn't appear to be pre-image resistant, so I'd be concerned if we were relying on the pre-image resistance properties of SHA-2. Can we be confident that this is not now subject to potential ployglot attacks if we relied on that hash alone? That is, could two clients think that they have the same resource, but do not?
I wouldn't be comfortable switching hashes given the wider and proven use of SHA-256 and agree on the lookup case to allow content-encoding with different and multiple dictionary negotiations. Looks like a separate header is still the cleanest so the main remaining question is if we use the same style of header for both or a skippable frame for Zstandard. I also need to update the brotli citations to point to the shared dictionary draft format instead of the original brotli format. On Sun, May 12, 2024 at 7:58 PM Martin Thomson NAME wrote: >
Sorry for the late comment. It seems to me that conveying this information in the content eases the integration with Signatures and Digest. It is not clear to me if there are still possible cases where the response does not contain all the information required for processing.
Thanks
Looks like you just need to update the PR summary to reflect this change.LGTMOverall, I like this. I want to flag for others that it doesn't use the skippable frame concept.
Code change:
An endpoint that encounters a connection error SHOULD first send a GOAWAY frame (GOAWAY) with the stream identifier of the last stream that it successfully received from its peer.  The GOAWAY frame <del> includes an error code that indicates why the connection is </del> <ins> includes an ErrorCodes that indicates why the connection is </ins> terminating.  After sending the GOAWAY frame for an error condition, the endpoint MUST close the TCP connection.

Related discussion:
text: For .
[x] 1. ----- FP: Is the working group aware that RFC 793bis is in IESG evaluation (URL) ? Was the choice of having a normative reference to 793 conscious, in order to avoid any delay that might come from publication of draft-ietf-tcpm-rfc793bis? (Just checking this was considered) [x] 2. ----- for HTTP/2 over TLS. The general TLS usage guidance in [TLSBCP] SHOULD be followed, with some additional restrictions that are specific to HTTP/2. FP: Given this requirement, I would have expected to see [TLSBCP] normatively referenced, rather than informatively. [x] 3. ----- layer. The frame and stream layers are tailored to the needs of the HTTP protocol and server push. FP: I would think the server push is part of the HTTP protocol, which makes this formulation "HTTP protocol and server push" confusing. [x] 4. ----- Type: The 8-bit type of the frame. The frame type determines the Flags: An 8-bit field reserved for boolean flags specific to the FP: I would find a reference to the IANA registry useful. [x] 5. ----- implementation of flow control can be difficult. When using flow control, the receiver MUST read from the TCP receive buffer in a timely fashion. Failure to do so could lead to a deadlock when FP: "When using flow control" might be rephrased to indicate "when flow control limits are lower than the maximum" (or something of the sort), since if I understand correctly the capability is always used, it is just the window size that changes (effectively implementing control of flow). Also "timely fashion" - I know that it is probably hard, but it would be nice to have a more precise qualification, or at least a hint of what is timely. [x] 6. ----- stream that it successfully received from its peer. The GOAWAY frame includes an error code that indicates why the connection is FP: "error code" - here as well I would have liked a reference to the IANA registry. [x] 7. ----- Exclusive: A single-bit flag. This field is only present if the PRIORITY flag is set. Stream Dependency: A 31-bit stream identifier.
Code change:
An endpoint that encounters a connection error SHOULD first send a GOAWAY frame (GOAWAY) with the stream identifier of the last stream that it successfully received from its peer.  The GOAWAY frame <del> includes an error code that indicates why the connection is </del> <ins> includes an ErrorCodes that indicates why the connection is </ins> terminating.  After sending the GOAWAY frame for an error condition, the endpoint MUST close the TCP connection.

Related discussion:
text: This field is only present if the PRIORITY flag is set. Weight: An unsigned 8-bit integer. This field is only present if the PRIORITY flag is set. FP: I would have expected to see some definition of how the fields are used. If this is defined somewhere else, a reference would be good. [x] 8. ----- SETTINGS Frame { Length (24), Type (8) = 4, Unused Flags (7), ACK Flag (1), Reserved (1), Stream Identifier (31), Setting (48) ..., } Setting { Identifier (16), Value (32), } FP: Is there any reason why the Stream Identifier line is not: Stream Identifier (31) = 0, [x] 9. ----- a server does include a value it MUST be 0. A client MUST treat receipt of a SETTINGS frame with SETTINGSENABLEPUSH set to 1 as a connection error (Section 5.4.1) of type PROTOCOLERROR. FP: This is just my curiosity: what is the reason for this stronger requirement - I would think it shouldn't be a problem for the sender if it wants to advertise that it would permit/support server push. What am I missing? [x] 10. ----- A value of 0 for SETTINGSMAXCONCURRENTSTREAMS SHOULD NOT be treated as special by endpoints. A zero value does prevent the FP: When is it ok that the 0 value is treated as special? [x] 11. ----- set. Upon receiving a SETTINGS frame with the ACK flag set, the sender of the altered settings can rely on the value having been applied. FP: nit s/value/values. Also I believe this could be misunderstood - can it be made more precise on the fact that only the values that are present in the received frame with the ACK flag set (and not those that might have been ignored because not understood) have been applied. [x] 12. ----- A receiver MUST treat the receipt of a WINDOWUPDATE frame with an flow-control window increment of 0 as a stream error (Section 5.4.2) FP: nit s/an/a [x] 13. ----- FLOWCONTROLERROR (0x3): The endpoint detected that its peer violated the flow-control protocol.
Code change:
An endpoint that encounters a connection error SHOULD first send a GOAWAY frame (GOAWAY) with the stream identifier of the last stream that it successfully received from its peer.  The GOAWAY frame <del> includes an error code that indicates why the connection is </del> <ins> includes an ErrorCodes that indicates why the connection is </ins> terminating.  After sending the GOAWAY frame for an error condition, the endpoint MUST close the TCP connection.

Related discussion:
text: STREAMCLOSED (0x5): The endpoint received a frame after a stream was half-closed. FP: would be good to add a reference to the relevant sections. [x] 14. ----- set after receiving the HEADERS frame that opens a request or after receiving a final (non-informational) status code MUST treat the FP: Where is a "non-informational status code" defined? [x] 15. ----- FP: Are the following two sentences in Section 8.1.1. in contraddiction? request or response. Malformed requests or responses that are detected MUST be treated as a stream error (Section 5.4.2) of type PROTOCOLERROR. on the remainder of the request being correct. A server or intermediary MAY use RSTSTREAM -- with a code other than REFUSEDSTREAM -- to abort a stream if a malformed request or response is received. FP: In section 5.4.2. I read: An endpoint that detects a stream error sends a RSTSTREAM frame So the first sentence above implies RSTSTREAM MUST be sent, while the second sentence states RSTSTREAM MAY be sent. [x] 16. ----- their definitions in Sections Section 5.1 of [5.1] and Section 5.5 of [5.5] of [HTTP] respectively and treat messages that contain FP: References need fixing. [x] 17. ----- from the control data of the original request, unless the the FP: nit Remove one "the" [x] 18. ----- Note that request targets for CONNECT or asterisk-form OPTIONS requests never include authority information. FP: Please add a reference to 7.1 of [HTTP] as this is the first time "asterisk-form OPTION" appear in this document. [x] 19. ----- Advertising a SETTINGSMAXCONCURRENTSTREAMS value of zero disables server push by preventing the server from creating the necessary streams. This does not prohibit a server from sending PUSHPROMISE frames; clients need to reset any promised streams that are not wanted. FP: Do these two sentences contradict each other? In the first sentence I am reading that the server can't create the necessary stream to send the PUSHPROMISE frame, in the second sentence I read that it can? [x] 20.
Code change:
An endpoint that encounters a connection error SHOULD first send a GOAWAY frame (GOAWAY) with the stream identifier of the last stream that it successfully received from its peer.  The GOAWAY frame <del> includes an error code that indicates why the connection is </del> <ins> includes an ErrorCodes that indicates why the connection is </ins> terminating.  After sending the GOAWAY frame for an error condition, the endpoint MUST close the TCP connection.

Related discussion:
text: ----- on a DATA frame is treated as being equivalent to the TCP FIN bit. A FP: Can a reference be added to the section where the TCP FIN bit is defined? [x] 21. ----- Content-Type: image/jpeg ==> - ENDSTREAM Content-Length: 123 + END_HEADERS FP: I think it would be good to add a sentence about the meaning of - and + (which I understand to be flag set or not set) in section 2.2. [x] 22. ----- treated as delimiters in other HTTP versions. An intermediary that translates an HTTP/2 request or response MUST validate fields according to the rules in Section 8.2 roles before translating a message to another HTTP version. Translating a field that includes FP: is "roles" supposed to be there? [x] 23. ----- The CONNECT method can be used to create disproportionate load on an proxy, since stream creation is relatively inexpensive when compared FP: nit s/an/a [x] 24. ----- FP: I think it would be good to keep the 3rd paragraph in Section 11 instead of asking the RFC Editor to remove it, just to keep a trace of the registries that have been defined in 7540, since those registries will now reference this document, but this document does not contain all the definitions of the different fields. [x] 25. ---- Comments: Obsolete; see Section 11.1 FP: I would suggest to be explicit and add "of this document" (unless links can be maintained in IANA registries Comments fields).
All done, closing.
Code change:
connection (===) between the user agent (UA) and the origin server (O). <del> Each major version of HTTP defines its own syntax for the inclusion of information in messages.  Nevertheless, a common abstraction is that a message includes some form of envelope/framing, a potential set of named fields up front (a header section), a potential body, and a potential following set of named fields (a trailer section). </del> <ins> Each major version of HTTP defines its own syntax for the communication of messages.  Nevertheless, a common abstraction is that each message contains some form of envelope/framing with self- descriptive control data that indicates its semantics and routing, a potential set of named fields up front (a header section), a potential body, and potential fields sent after the body begins (trailer sections). </ins> <del> A client sends an HTTP request to a server in the form of a </del> <ins> A client sends requests to a server in the form of a </ins> message with a method (methods) and request target.  The request might also contain header fields for request modifiers, client

Related discussion:
text: H1-H3 currently define a way to carry only one such block, but an extension could provide more.
Code change:
https://github.com/fiestajetsam/draft-gruessing-moq-requirements [1]. Discussion of this draft should take place on the IETF Media Over <del> QUIC (MOQ) mailing list, at https://www.ietf.org/mailman/listinfo/ moq. </del> <ins> QUIC (MoQ) mailing list, at https://www.ietf.org/mailman/listinfo/moq [2]. </ins> 1.

Related discussion:
text: This is just a starting point, but brings the document closer to alignment with our spreadsheet. I've also added some additional details which should cover themes raised in the current BoF agenda.
Code change:
HTTP/2's preference for using a single TCP connection allows correlation of a user's activity on a site.  If connections are reused for different origins, this allows tracking across those <del> origins.  While limited in practicality, this does constitute a new form of tracking. </del> <ins> origins. </ins> Because the PING and SETTINGS frames solicit immediate responses, they can be used by an endpoint to measure latency to their peer.

Related discussion:
text: Proposal for
Taking Mark's base, taking Herve's suggestions and removing some text. I think that this is good.
Folks in the Tor community point out that coalescing connections is a genuinely new vector for tracking users in HTTP/2. This should be pointed out in the Privacy Considerations subsection of Security Considerations, along with mitigations. While we're there, more specific text about mitigating HTTP/2-specific fingerprinting would be nice.
NAME I've marked this as Design, but I think we can prepare a proposal pretty easily. You want to take a crack at it, or should I do a pull?
I'm happy to take input. Since I'm basically offline this week, getting a start would help lots.
Marked as editor-ready; please consider list feedback.