jiebi/CodeConvo huggingface dataset:
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: Presuming the reference is ~50 MB, and the updated content is also ~50 MB, one would need a window size of ~50 MB to ensure access into the dictionary during the entire reconstruction process. But if the updated content is just a bit bigger, say ~51 MB, a ~50 MB window size would allow delta operation only during the first ~50 MB, the last ~1 MB would have to be compressed without dictionary. If follows that the size of the dictionary is irrelevant for the Window size. What matters is for how long one wants to access the dictionary while rebuilding the content. And as a general approximation, one wants this dictionary during the entire reconstruction process, so the window size is effectively the size of the content to rebuild. Zstandard has a "windowless" mode which states exactly that : the size of the "window" is the size of the content. For this to work, one has to set the size of the content at the beginning of the compression process . Setting the content size is either an automatic operation when data is provided entirely upfront (), or, when data is provided in small increments (), it must be stated explicitly at the beginning (). will automatically trigger the "windowless" mode when the declared content size is smaller than the set Window size (effectively reducing the memory requirement for the decoder side). I'm not sure if there is another way to trigger it. Note that the decoder will be informed of the Window Size requirement by reading the frame header, and may decide to not proceed if this value is deemed too large. For example, has a default 128 MB limit, and will refuse to decompress payload requiring more that that, unless explicitly authorized to employ more memory (). Just for reference, offers a "large delta compression" mode with the command . It has made us somewhat familiar with some of these issues. In situations where the amount of data to compress and compare is very large, we now prefer to both the dictionary content and the destination file. It has proven useful to reduce memory pressure.
Thanks. That makes things a bit more complicated for the case where we want to require a minimum window size that the server can trust that all clients advertising content-encoding will be able to decode.
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: For the dictionary case it might be cleanest to use the same 128MB limit for window sizes as the default zstd limit (knowing that the client won't allocate that much unless it actually gets a stream that uses it).
Hello. Shared dictionary is placed in the address space between backward reference window and static dictionary. In other words it does not interfere with normal compression window, but using it makes "static dictionary" references more expensive. We decided to order it that way, because shared dictionary works almost the same way as extra window. Sorry for the late response. Best regards, Eugene.
Extra note. As you can see brotli allows backward distances behind window. IIRC, regular (as opposed to large-window) brotli allows distances up to ~56MiB, i.e. larger shared dictionaries would not be useful (== referenced)
Thanks. For the dictionaries to be usable for a broad set of apps (including large wasm bundles), I'm considering spec'ing: Brotli: Regular 16MB window, allowing for delta-compression of resources up to ~50MB. ZStandard: 128MB window, allowing for delta compression of resources up to around that size, depending on where the duplicated bytes are in the updated resource. The main difference with regular content-encoding is on the ZStandard side where the window is 8MB. For platforms that are resource constrained and are not willing to support a 128MB window they would be limited to Brotli (and then decide how large of a dictionary they are willing to keep and use). Another option on the ZStandard side would be to allow for a maximum window that is a multiple of the dictionary size (i.e. 8MB or 2 * dictionary size whichever is larger with a 128MB max). That way the client can have an upper limit that it knows in advance before advertising as being available. That adds complexity to the encoding side though. We have already seen cases where people want to delta-compress resources > 50MB (wasm apps) where they had to use ZStandard instead of Brotli because of the Brotli limits so I don't think a small limit like 8MB for ZStandard would work with this use case.
That massive resources exist is not sufficient justification to build a system that will not be available to some users.
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: I would prefer a far smaller default. If only to encourage the use of smaller resources overall. Defining a separate codepoint for massive windows is possible, say . The scalable option also seems reasonable. Clients can then choose to offer dictionaries when they exceed available resources.
I'm ok using the same 8MB window ( I believe) that we use for for and note that the effectiveness of delta compression for responses > 8MB using may be limited. The scalable option concerns me that people are likely to get it wrong and it won't be obvious and the decompression will start to fail as the dictionary and resource size changes. NAME does that sound about right?
The alternative floated where 's window size is derived from the size of the dictionary is much more attractive to me. (That is, cap the window size log at , where the spec picks an somewhere in the range .) In such a scheme, where the window size is deterministically derived from the size of the dictionary--a resource the client must already have access to--the client is free to decline to advertise support for for dictionaries which would require the use of a window larger than the client is willing to support. I.e., in resource-constrained environments, Chrome could avoid having to allocate more than 8 MB windows for by not advertising support for on requests where the resolved available dictionary is larger than . Put another way, given that the client will have the tools to make these choices for themselves, I would rather give the client discretion here rather than just bake a flat restriction into the spec.
NAME would you be supportive of updating the zstd command-line tooling to add an option that would set the window to the larger of 8MB or the dictionary size (i.e. match the content-encoding behavior)? My main concern is that it is easy to document and get people to use a fixed command-line for a 8MB window but if it involves a more complicated algorithm then it is likely to break in subtle ways that they may not notice until the resources are bigger than the dictionary (and are bigger than 8MB).
NAME yeah, that would be pretty straightforward. Happy to add that.
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: With the initial formulation, since the window size is limited to the size of the dictionary, the compressor cannot use as a window size, because it's 1 byte larger than the limit. So, it will have to find the next best window size under the limit. Due to granularity limitations, this next best window size is 8 MB. So delta compression would proceed fine for the first 8 MB, but for the last 1 MB, data would have to be compressed without the benefit of the dictionary. With the newly proposed limitation to "x1.25 the dictionary size", delta compression would remain active in above scenario during the entire decompression process. And it would remain fully active if the new resource to decompress is 10 MB, 11 MB or 11.24 MB. But if the new resource is 11.25 MB or larger, then the compressor will automatically limit the window size to 11 MB (since it's the largest representable window size under the limit, which, in this particular instance, is ), achieving the wanted memory size control.
Since neither the server not the client may know what the resource size will be when compression starts (say, in a streaming HTML case), won't the 1/8th power of 2 granularity apply even for cases where the resource is the same size (or sometimes smaller) than the dictionary? For resources slightly larger than the old one, the dictionary would still be used for the majority of the resource but the last bit would fall back to regular window-based compression. I agree that having the dictionary available for the whole resource would provide better compression but I'm not sure where to draw the line. Something like a 25% fudge factor can be quite a bit when you start talking about 50 MB+ resources that might be close to where the client wants to draw the line anyway. They may have allowed the dictionary at 50 MB but not allow it's use at all if it needed 62 MB available for a window. A similar case could be made for the non-update case where you have a custom-built dictionary but the resource being compressed exceeds 8 MB. Only the first 8 MB of the resource would be able to use the dictionary.
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: Hopefully at that point the compression window would have most of the stuff that was in the dictionary but the dictionary becoming unavailable is something we should expect for some cases with ZStandard, even if we use a fudge factor. Is there a way to convey the 1/8th power of two granularity on the window size or is it something that the ZStandard libraries will take care of automatically for users (i.e. setting the window size to 11 MB when the caller sets it to 11.24 MB)? I'm wondering how much of this implementors are going to have to deal with directly.
It depends if the compression side (server) is informed upfront of the size of the payload to compress. If not, then indeed you are right, the compressor will have to set a Window Size, and the granularity rule applies. So for example, let's assume the payload to delta-compress is 50 MB, and the reference dictionary is also 50 MB. Let's then assume it implies a window size limit of 50 MB (dictionary size) and the compressor is not informed of the size of the payload. In which case, the compressor will have to set a Window Size, and it will have to settle for the largest representable window size under the limit, which happens to be 48 MB. Therefore, the dictionary won't be used during the compression of the last 2 MB of the payload. With the proposed +25% tolerance rule, the limit would become 62.5 MB, and the largest representable window size under the limit is 60 MB. Now the dictionary remains active during the entire compression of the 50 MB payload. When the data to decompress ends up being smaller than the window size, it's obviously not a problem for the decompression process: the final decompressed size is still properly reported at the end. It's just that the intermediate buffer allocation will be larger than it could have been. Note that uses for the allocation of buffers, so the address space is just reserved, but the physical memory may never be used if it's not needed (i.e. if there is no write action on the later segment part). +25% is more like a suggestion, this can certainly be discussed if some other limit feels better.
Code change:
request, the algorithm to be used is negotiated through the regular mechanism for negotiating content encoding in HTTP. <del> This document introduces two new content encoding algorithms: </del> The dictionary to use is negotiated separately and advertised in the "Available-Dictionary" request header. 3.1. <ins> This document introduces two new content encoding algorithms: br-d: Brotli RFC7932 using an external compression dictionary and a compression window of not more than 16 MB. zstd-d: Zstandard RFC8878 using an external compression dictionary and a compression window of not more than 8 MB. 3.2. </ins> The client adds the algorithms that it supports to the "Accept- Encoding" request header. e.g.: <del> 3.2. </del> <ins> 3.3. </ins> If a server supports one of the dictionary algorithms advertised by the client and chooses to compress the content of the response using

Related discussion:
text: Frankly, I'm also very open to also adding a flag (and equivalent compression parameter in the library) that would just look at the size of the dictionary (which zstd necessarily has access to) and would configure the window accordingly. In that case, users wouldn't have to do any math at all.
OK, I chaged the structure and language so that they are no longer streams of bytes compressed with a given algorithm but are now specifically and . The references for those named formats have the window restrictions (which for Zstandard includes the 1.25x dictionary requirement). Currently the byte stream itself is still a native Brotli/Zstandard stream, just with specific settings but this also makes it cleaner if we decide to . PR with the update is here: URL
lgtm other than the comment
Code change:
Recommended HTTP status code: 500 <del> 3.29. </del> <ins> 3.28. </ins> Name: tls_error

Related discussion:
text: Signed-off-by: Piotr Sikora
cc NAME
This field uses for a specific purpose, but that's already in the generic parameters. Suggest or similar. /cc NAME
Looks like has the same issue. Should we use for both? Note that the identifier is context-based, since if proxy configured SHA256-based validation, then would be , if it configured SPKI-based validation, then would be , etc. We could also use and instead of (and for ), but that doesn't seem very future-proof. What do you think?
Hmm, that's awkward. Maybe two fields? E.g., : [, ... ] : ... Is there a registry we can refer to, maybe?
I'm afraid that there isn't any registry that we can refer to, at least none that I know of. (opaque type) and (opaque value) seem a bit awkward. Wouldn't it be easier to just have and params? Also, in Envoy, we actually configure list of acceptable and values, and matching against any of them works, so we should probably emit both of them anyway. I could envision merging and into a single status type that lists all 3 params, e.g. See . What do you think?
Better than what we had :)
Code change:
operation is desired. In this context, timestamp tokens are similar to a countersignature <del> RFC9338 made by the TSA. </del> <ins> made by the TSA. </ins> 3.

Related discussion:
text: In , Orie commented that: The answer is obviously "no", but the fact that the question was raised seems to indicate that the reference to 9338 is potentially confusing. Note that if we removed it, nothing would change, which is an incentive to drop. Authors might consider providing some guidance regarding validation of CTT where the COSE claims appear (iat, nbf) to have happened after the timestamp ( time travel ). IIRC cose counter signatures apply to the protected header, payload and signature, whereas CTT only applies to the signatures. This means that the TSA does not countersign any protected information in the header? Some use cases for the 2 modes might improve the document. Security considerations seem light. OS >
Code change:
All numeric values are in network byte order.  Values are unsigned unless otherwise indicated.  Literal values are provided in decimal or hexadecimal as appropriate.  Hexadecimal literals are prefixed <del> with </del> <ins> with " </ins> <del> to distinguish them from decimal literals. </del> <ins> " to distinguish them from decimal literals. </ins> This specification describes binary formats using the convention described in QUIC.  Note that this format uses network byte order and

Related discussion:
text: A recent xml2rfc update changed the way that the element was rendered in text. Take advantage of that by adding quotes to strings that really do benefit from additional quoting, such as the "PRI ..." preface string. Most of the changes shown in a recent diff are down to losing quotes on header field names and other protocol elements. These do not need to be quoted for their meaning to be clear and unambiguous. The use of for these items improves the HTML rendering, but the loss of line noise in text renderings is a net win (at least in my opinion).
I agree with your reasoning, this seems like a good change.
Code change:
or <del> frames MUST have the END_PUSH_PROMISE or END_HEADERS flag set (respectively). </del> <ins> frames MUST have the END_HEADERS flag set. </ins> Header block fragments can only be sent as the payload of

Related discussion:
text: This simplifies the text and makes flag consistent with CONTINUATION which only bears ENDHEADERS. Technically, both have the same meaning of ending sequence of header block fragments. Therefore there is no reason to use the different flag name for the same purpose. This will also make the implementaion easier because most likely HEADERS and PUSHPROMISE handling share the same code and dancing with ENDPUSHPROMISE and END_HEADERS is not fun. Although currently they have the same number, it is not guaranteed to the last. If they are guarenteed to be the same, then it is simpler to use the same name for same value.
Code change:
Wide Web since its introduction in 1990.  It began as a trivial mechanism for low-latency requests, with a single method (GET) to request transfer of a presumed hypertext document identified by a <del> given pathname.  This original protocol is now referred to as HTTP/0.9 (see HTTP09). As the Web grew, HTTP was extended to enclose requests and responses within messages, transfer arbitrary data formats using MIME-like media types, and route requests through intermediaries, eventually being defined as HTTP/1.0 RFC1945. </del> <ins> given pathname.  As the Web grew, HTTP was extended to enclose requests and responses within messages, transfer arbitrary data formats using MIME-like media types, and route requests through intermediaries.  These protocols were eventually defined as HTTP/0.9 and HTTP/1.0 (see RFC1945). </ins> HTTP/1.1 was designed to refine the protocol's features while retaining compatibility with the existing text-based messaging

Related discussion:
text: Good point.
For .
URL be good to have tests...
goes back to URL
sent mail: URL
See URL
Proposal: leave weight in ABNF for consistency with other Accept-* fields change second requirement (on recipients) not to rely on "q" being last
Code change:
Wide Web since its introduction in 1990.  It began as a trivial mechanism for low-latency requests, with a single method (GET) to request transfer of a presumed hypertext document identified by a <del> given pathname.  This original protocol is now referred to as HTTP/0.9 (see HTTP09). As the Web grew, HTTP was extended to enclose requests and responses within messages, transfer arbitrary data formats using MIME-like media types, and route requests through intermediaries, eventually being defined as HTTP/1.0 RFC1945. </del> <ins> given pathname.  As the Web grew, HTTP was extended to enclose requests and responses within messages, transfer arbitrary data formats using MIME-like media types, and route requests through intermediaries.  These protocols were eventually defined as HTTP/0.9 and HTTP/1.0 (see RFC1945). </ins> HTTP/1.1 was designed to refine the protocol's features while retaining compatibility with the existing text-based messaging

Related discussion:
text: Im ok with replacing the 0.9 reference (although having the link here acknowledges where 0.9 came from, right). The idea was to mention 0.9 in the context of RFC 1945 as well as that is an IETF document which actually describes what 0.9 is.
URL out that RFC 1945 defines HTTP/0.9 as well (see URL). Maybe we should add this reference, or replace the existing one.
We do reference 1945 for HTTP/1.0. Revised proposal: just mention that that RFC describes 0.9 as well.