jiebi/CodeConvo huggingface dataset:
Code change:
Track Request Parameters: The parameters are defined in track-req- params. <del> 6.9.1. The Track Request Parameters identify properties of the track requested in either the ANNOUNCE or SUSBCRIBE REQUEST control messages.  The peers MUST close the connection if there are duplicates.  The Parameter Value Length field indicates the length of the Parameter Value. 6.9.1.1. The GROUP SEQUENCE parameter (key 0x00) identifies the group within the track to start delivering objects.  The publisher MUST start delivering the objects from the most recent group, when this parameter is omitted.  This parameter is applicable in SUBSCRIBE REQUEST message. 6.9.1.2. The OBJECT SEQUENCE parameter (key 0x01) identifies the object with the track to start delivering objects.  The "GROUP SEQUENCE" parameter MUST be set to identify the group under which to start delivery.  The publisher MUST start delivering from the beginning of the selected group when this parameter is omitted.  This parameter is applicable in SUBSCRIBE REQUEST message. 6.9.1.3. AUTHORIZATION INFO parameter (key 0x02) identifies track's authorization information.  This parameter is populated for cases where the authorization is required at the track level.  This parameter is applicable in SUBSCRIBE REQUEST, UNSUBSCRIBE and ANNOUNCE messages. 6.10. </del> <ins> 6.8. </ins> The receiver sends an "ANNOUNCE OK" control message to acknowledge the successful authorization and acceptance of an ANNOUNCE message.

Related discussion:
text: This PR adds support for subscribers to issue unsubscribe requests to express its interest in not receiving media for a given track. This reflects discussions from The high level call flow is
Thinking about it some more, an error code doesn't seem necessary. publisher -> subscriber should have an error code. subscriber -> publisher doesn't need an error code. Even with an error code, it would get lost in the deduplication process.
Individual reviewIt might be useful to have an error code, but I'm not sure. LGTM.We'll need to revisit using track name vs track ID for unsubscribe once we have an answer to , but for now, this LGTM.
Code change:
state (see HeaderBlock); similarly DATA frames MUST be counted toward the connection flow control window. <del> The GTFO frame does not define any flags. </del> <ins> The GOAWAY frame does not define any flags. </ins> <del> The GTFO frame applies to the connection, not a specific stream.  An endpoint MUST treat a </del> <ins> The GOAWAY frame applies to the connection, not a specific stream. An endpoint MUST treat a </ins> frame with a stream identifier other than 0x0 as a ConnectionErrorHandler of type . <del> The last stream identifier ("Last-Stream-ID") in the GTFO frame contains the highest numbered stream identifier for which the sender of the GTFO frame has received frames on and might have taken some action on.  All streams up to and including the identified stream might have been processed in some way.  The last stream identifier is set to 0 if no streams were processed. </del> <ins> The last stream identifier in the GOAWAY frame contains the highest numbered stream identifier for which the sender of the GOAWAY frame has received frames on and might have taken some action on.  All streams up to and including the identified stream might have been processed in some way.  The last stream identifier is set to 0 if no streams were processed. </ins> Note: In this case, "processed" means that some data from the stream was passed to some higher layer of software that might have taken some action as a result. <del> If a connection terminates without a GTFO frame, this value is </del> <ins> If a connection terminates without a GOAWAY frame, this value is </ins> effectively the highest stream identifier. On streams with lower or equal numbered identifiers that were not

Related discussion:
text: It is more fitting than GOAWAY, but once you have read GTFO your mind is sort of made up, since you spelled GTFO in full in your head, as soon as reading the acronym. My suggestion on this one is: Simply get rid of "General" / "G" and add "of" to the acronym! TOFO still has a nice ring to it, is quick and easy to pronounce and "Termination Of Future Operations" would be sufficiant descriptive. regards (yes, that's my real lastname :P ) On Jan 29, 2014 8:31 AM, "Eugene Ciurana" EMAIL wrote:
Code change:
"Upload-Complete" field value (upload-complete) MUST be set to false. The server SHOULD respect representation metadata received during <del> creation (upload-creation) and ignore any representation metadata received from appending. </del> <ins> creation (upload-creation).  An upload append request continues uploading the same representation as used in the upload creation (upload-creation) and thus uses the same content coding, if one was applied.  For example, if the initial upload creation included the "Content-Encoding: gzip" header field, the upload append request resumes the transfer of the gzipped data without indicating again that the gzip coding is applied. </ins> If the server does not consider the upload associated with the upload resource active, it MUST respond with a "404 (Not Found)" status

Related discussion:
text: defines a framework for problem types, which can be used as a standardized response format when errors occurs. With resumable uploads, we have a few cases where such problem types could be useful, but usage remains entirely optional. The text in this PR has been inspired by similar sections from .
Thank you very much for quick feedback, Lucas!
Code change:
Values are registered on a Specification Required RFC8126 basis after a two-week review period on the oauth-ext-review@ietf.org mailing list, on the advice of one or more Designated Experts.  However, to <del> allow for the allocation of values prior to publication, the Designated Experts may approve registration once they are satisfied that such a specification will be published. </del> <ins> allow for the allocation of values prior to publication of the final version of a specification, the Designated Experts may approve registration once they are satisfied that the specification will be completed and published.  However, if the specification is not completed and published in a timely manner, as determined by the Designated Experts, the Designated Experts may request that IANA withdraw the registration. </ins> Registration requests sent to the mailing list for review should use an appropriate subject (e.g., "Request to register OAuth Protected

Related discussion:
text: ... per additional feedback by NAME Cc: NAME
I'm checking with IANA.... will reply once they respond. Thanks for the work!
Code change:
responses available in order to fully process the response to the original request. <del> Pushing additional message exchanges in this fashion is optional, and is negotiated between individual endpoints.  The </del> <ins> A client can request that server push be disabled, though this is negotiated for each hop independently.  The </ins> setting can be set to 0 to indicate that server push is disabled.

Related discussion:
text: Only if it makes sense to do so. It's a perfectly good word that has been spoiled by an unnecessary inclusion in some popular RFC (2119, I think).
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: This changes the maximum ZStandard compression window to be the larger of 8 MB and the size of the dictionary. This allows for clients to limit the maximum window size to as low as 8 MB if they like (by not advertising support) while also allowing for clients to use larger dictionaries with ZStandard than they would be able to if the global limit was 8 MB.
NAME and NAME could you also take a look and see if the language looks ok?
Context: URL aims to clarify the window size limit for the Content Encoding token as it wasn't clear in RFC8878. Since the Compression Dictionary Transport draft hasn't been published yet, we should also add some window size considerations to it for and to maintain interoperability. I think in Chrome we ended up using . NAME
NAME could you comment on how this impacts Brotli dictionary decode? Is the dictionary independent from the compression window (so we can adopt the same defaults as or do we need to specify something relative to the size of the dictionary? NAME NAME for ZStandard, to allow for delta-encoding large resources, is reasonable or should it be ? We have use cases for delta-encoding wasm applications where the dictionary can easily be upwards of 50 MB.
For Zstandard, the states that the entirety of the dictionary is accessible as long as the input is <= Window Size. Therefore, to give an extreme case, it's possible to have a 50 MB dictionary, and it's entirely accessible even if the Window Size is only 1 KB, but only as long as the data to de/compress is <= 1 KB. As soon as we reach 1 KB + 1, reaching the dictionary is no longer valid. For delta compression, this has consequences. Presuming the reference is ~50 MB, and the updated content is also ~50 MB, one would need a window size of ~50 MB to ensure access into the dictionary during the entire reconstruction process. But if the updated content is just a bit bigger, say ~51 MB, a ~50 MB window size would allow delta operation only during the first ~50 MB, the last ~1 MB would have to be compressed without dictionary. If follows that the size of the dictionary is irrelevant for the Window size. What matters is for how long one wants to access the dictionary while rebuilding the content.
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: And as a general approximation, one wants this dictionary during the entire reconstruction process, so the window size is effectively the size of the content to rebuild. Zstandard has a "windowless" mode which states exactly that : the size of the "window" is the size of the content. For this to work, one has to set the size of the content at the beginning of the compression process . Setting the content size is either an automatic operation when data is provided entirely upfront (), or, when data is provided in small increments (), it must be stated explicitly at the beginning (). will automatically trigger the "windowless" mode when the declared content size is smaller than the set Window size (effectively reducing the memory requirement for the decoder side). I'm not sure if there is another way to trigger it. Note that the decoder will be informed of the Window Size requirement by reading the frame header, and may decide to not proceed if this value is deemed too large. For example, has a default 128 MB limit, and will refuse to decompress payload requiring more that that, unless explicitly authorized to employ more memory (). Just for reference, offers a "large delta compression" mode with the command . It has made us somewhat familiar with some of these issues. In situations where the amount of data to compress and compare is very large, we now prefer to both the dictionary content and the destination file. It has proven useful to reduce memory pressure.
Thanks. That makes things a bit more complicated for the case where we want to require a minimum window size that the server can trust that all clients advertising content-encoding will be able to decode. For the dictionary case it might be cleanest to use the same 128MB limit for window sizes as the default zstd limit (knowing that the client won't allocate that much unless it actually gets a stream that uses it).
Hello. Shared dictionary is placed in the address space between backward reference window and static dictionary. In other words it does not interfere with normal compression window, but using it makes "static dictionary" references more expensive. We decided to order it that way, because shared dictionary works almost the same way as extra window. Sorry for the late response. Best regards, Eugene.
Extra note. As you can see brotli allows backward distances behind window.
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: IIRC, regular (as opposed to large-window) brotli allows distances up to ~56MiB, i.e. larger shared dictionaries would not be useful (== referenced)
Thanks. For the dictionaries to be usable for a broad set of apps (including large wasm bundles), I'm considering spec'ing: Brotli: Regular 16MB window, allowing for delta-compression of resources up to ~50MB. ZStandard: 128MB window, allowing for delta compression of resources up to around that size, depending on where the duplicated bytes are in the updated resource. The main difference with regular content-encoding is on the ZStandard side where the window is 8MB. For platforms that are resource constrained and are not willing to support a 128MB window they would be limited to Brotli (and then decide how large of a dictionary they are willing to keep and use). Another option on the ZStandard side would be to allow for a maximum window that is a multiple of the dictionary size (i.e. 8MB or 2 * dictionary size whichever is larger with a 128MB max). That way the client can have an upper limit that it knows in advance before advertising as being available. That adds complexity to the encoding side though. We have already seen cases where people want to delta-compress resources > 50MB (wasm apps) where they had to use ZStandard instead of Brotli because of the Brotli limits so I don't think a small limit like 8MB for ZStandard would work with this use case.
That massive resources exist is not sufficient justification to build a system that will not be available to some users. I would prefer a far smaller default. If only to encourage the use of smaller resources overall. Defining a separate codepoint for massive windows is possible, say . The scalable option also seems reasonable. Clients can then choose to offer dictionaries when they exceed available resources.
I'm ok using the same 8MB window ( I believe) that we use for for and note that the effectiveness of delta compression for responses > 8MB using may be limited. The scalable option concerns me that people are likely to get it wrong and it won't be obvious and the decompression will start to fail as the dictionary and resource size changes. NAME does that sound about right?
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: The alternative floated where 's window size is derived from the size of the dictionary is much more attractive to me. (That is, cap the window size log at , where the spec picks an somewhere in the range .) In such a scheme, where the window size is deterministically derived from the size of the dictionary--a resource the client must already have access to--the client is free to decline to advertise support for for dictionaries which would require the use of a window larger than the client is willing to support. I.e., in resource-constrained environments, Chrome could avoid having to allocate more than 8 MB windows for by not advertising support for on requests where the resolved available dictionary is larger than . Put another way, given that the client will have the tools to make these choices for themselves, I would rather give the client discretion here rather than just bake a flat restriction into the spec.
NAME would you be supportive of updating the zstd command-line tooling to add an option that would set the window to the larger of 8MB or the dictionary size (i.e. match the content-encoding behavior)? My main concern is that it is easy to document and get people to use a fixed command-line for a 8MB window but if it involves a more complicated algorithm then it is likely to break in subtle ways that they may not notice until the resources are bigger than the dictionary (and are bigger than 8MB).
NAME yeah, that would be pretty straightforward. Happy to add that.
If we end up that bundles the dictionary hash with the compressed stream we can make sure the tooling for that file format handles the window size stuff for zstandard automatically (my main concern is around unexpected breakage from different server and client window requirements that only trigger when resources grow). The language will be a bit complex but will probably have to look something like
If I may chime in for some tactical consideration : In an update scenario, it's common for a new resource to be slightly larger than the old one it replaces. The expectation is that we generally tend to add stuff over time. A formulation which caps the window size to the size of the dictionary would introduce problems for all these update cases, which are expected to be the majority of update scenarios. On the other hand, we don't want this increase in size to be completely uncapped, in order to keep control of memory size.
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: And if the new resource is really much larger than the old one, the benefits of delta compression are dubious anyway. So the suggestion is to authorize a window size which is slightly larger than the external dictionary size. As to how much larger exactly, a suggestion here is by +25% (x1.25). I would expect a large majority of update scenarios to fit into this picture, with the new resource being slightly larger than the old one by a few %. For the less common cases where the new ressource is much larger than the old one, Zstandard could still employ the dictionary but only for the beginning of decompression. The compression process will have to select a window size which remains under the specified limit (x1.25 external dictionary size). As a technical insight, note that byte-accurate window size is only a thing when the window is the entire decompressed content. Otherwise, the window size is encoded in a condensed format, which can only represent increments by 1/8th of a power of 2. This is not a problem to determine this "largest window size under the specified limit", but due to limited accuracy, you can already guess what would be the problem is we were to keep the formulation "not more than the larger of 8 MB and the size of the dictionary". Imagine that the external dictionary size is bytes, and the new resource to delta-compress is , aka 1-byte larger than the dictionary. With the initial formulation, since the window size is limited to the size of the dictionary, the compressor cannot use as a window size, because it's 1 byte larger than the limit. So, it will have to find the next best window size under the limit. Due to granularity limitations, this next best window size is 8 MB. So delta compression would proceed fine for the first 8 MB, but for the last 1 MB, data would have to be compressed without the benefit of the dictionary. With the newly proposed limitation to "x1.25 the dictionary size", delta compression would remain active in above scenario during the entire decompression process. And it would remain fully active if the new resource to decompress is 10 MB, 11 MB or 11.24 MB.
Code change:
algorithm and the server would just ignore any "Available-Dictionary" requests that do not use the updated hash. <del> 6.2. </del> <ins> 8.2. </ins> The CRIME attack shows that it's a bad idea to compress data from mixed (e.g. public and private) sources - the data sources include

Related discussion:
text: But if the new resource is 11.25 MB or larger, then the compressor will automatically limit the window size to 11 MB (since it's the largest representable window size under the limit, which, in this particular instance, is ), achieving the wanted memory size control.
Since neither the server not the client may know what the resource size will be when compression starts (say, in a streaming HTML case), won't the 1/8th power of 2 granularity apply even for cases where the resource is the same size (or sometimes smaller) than the dictionary? For resources slightly larger than the old one, the dictionary would still be used for the majority of the resource but the last bit would fall back to regular window-based compression. I agree that having the dictionary available for the whole resource would provide better compression but I'm not sure where to draw the line. Something like a 25% fudge factor can be quite a bit when you start talking about 50 MB+ resources that might be close to where the client wants to draw the line anyway. They may have allowed the dictionary at 50 MB but not allow it's use at all if it needed 62 MB available for a window. A similar case could be made for the non-update case where you have a custom-built dictionary but the resource being compressed exceeds 8 MB. Only the first 8 MB of the resource would be able to use the dictionary. Hopefully at that point the compression window would have most of the stuff that was in the dictionary but the dictionary becoming unavailable is something we should expect for some cases with ZStandard, even if we use a fudge factor. Is there a way to convey the 1/8th power of two granularity on the window size or is it something that the ZStandard libraries will take care of automatically for users (i.e. setting the window size to 11 MB when the caller sets it to 11.24 MB)? I'm wondering how much of this implementors are going to have to deal with directly.
It depends if the compression side (server) is informed upfront of the size of the payload to compress. If not, then indeed you are right, the compressor will have to set a Window Size, and the granularity rule applies. So for example, let's assume the payload to delta-compress is 50 MB, and the reference dictionary is also 50 MB.