jiebi/CodeConvo huggingface dataset:
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: The reason is that I think the distinction doesn't make much sense, and maintaining it adds a lot of complexity, basically forcing browsers to recreate the digest for every connection at O(N) cost. Under this premise what servers should do is: Push all the resources that are known not to be in the cache digest Push 304 responses for resources that are in the cache digest, but are likely to be stale (short freshness lifetime, etc) Don't push resources that are in the cache digest and have a long term freshness lifetime or are immutable. Does that make sense? I'm not sure I understand your reference to the push cache vs. the HTTP cache in your comment. In light of my explanation, is there still an issue there in your view?
URL is the reference implementation. Happy to change it. Do you have any specific changes in mind?
NAME Thank you for the explanation. I now understand the intent better. I think that we need to consider two issues regarding the approach. First is the fact that a browser cache may contain more stale responses than fresh resources. Below are the numbers of cached objects found in my Firefox's cache (to be honest the date is from 2016, I haven't been using Firefox in recent weeks and therefore cannot provide up-to-date data). total ---: 2,273 1,003 As you can see, large scale websites tend to have more stale objects than fresh objects. In other words, including information of stale-cached objects increases the size of the digest roughly three times in this case. Since performance-sensitive resources (that we need to push) are likely to be stored fresh (since they are the most likely ones marked as immutable, or near-immutable), transmitting only the digest of freshly-cached responses makes sense. Second is a configuration issue on the server side. One strategy that can be employed by an H2 server (under the current draft) is to receive a digest of freshly cached resources only, compare the digest against the list of resources the browser should preload by only using the URL, and push the missing resources to the client. It is possible for a H2 server to perform the comparison without actually fetching the resource (from origin or from cache) since only the URL would be required for calculating the digest.
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: The proposal prevents such strategy from being deployed since it requires the ETag values to be always taken into consideration (should they be associated to the HTTP responses). In other words, servers would be required to load response headers of the resources to determine if it needs to be pushed, which could be a huge performance degradation on some deployments. Fortunately, servers could avoid the issue by not including ETags for resources that it may push. I think such change on the server-side configuration would be possible, but we need to make sure if we are to take the path (of removing the fresh vs. stale distinction). Let me explain using an example. Consider the following case: client has with and on server-side, the resource has been updated to When receiving a new request from the client, the server cannot determine if the client has URL in its cache. Therefore, would be pushed. The client, when observing (or equivalent tag), tries to load the resource. Since the fresh resource exists within the browser cache, that would be used. The pushed version is ignored and gets discarded (). This would be repeated every time until the cached object either becomes stale or gets removed from the cache. My understanding is that the browser behavior (explained in ) is true for Firefox and also for Chrome. Am I wrong, or missing something? Thank you for the link. I will try to use it. OTOH, do you have some working code that can actually calculate the cache-digest value taking a list of URLs as an input (something like URL)? I ask this because it would give us a better sense in how the actual size of the digest would be. One way to proceed would be to split the discussion of from Cuckoo filters into a separate issue or a PR. I do not have a strong opinion on the naming or the codepoints. What do you think? NAME
NAME Have you considered the approach using Cuckoo filter to generate GCS? I can understand the fact that you do not want to iterate through the browser cache when sending a cache digest. Per-host Cuckoo hash seems like a good solution to the issue. OTOH, as I described in my previous comment, it seems that sending the hash directly has several issues.
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: That is why I am wondering if it would be viable to generate GCS from the per-host Cuckoo filter that would be maintained within the browser. I can see three benefits in the approach, compared to sending the values of Cuckoo filter directly: the size of the digest will be smaller we can keep the distinction between fresh vs. cache. Sending digest of fresh resources only would end up in even smaller digests. Retaining the distinction lowers the bar to deploy cache-digests on the server side. note: you can store the time when the cached object becomes stale in the data associated to the Cuckoo filter entry (assuming that you would have associated data to handle resize, as we discussed in URL). That information can be used when builiding the GCS to determine if a particular object should go into a GCS of fresh resources or that of stale ones less change to the browser push handling (no need to handle pushes of 304 or replace a freshly cached object when an object with the same URL is being pushed) In case of or in the comment above, sending fresh-only digests using GCS would be about 1/3 the size of sending fresh & stale digests using Cuckoo filter. The biggest cost of calculating GCS from Cuckoo hash would be the sort operation. But I think that the cost could be negligible compared to the ECDH operation that we would be doing for every connection, considering the fact that the number of entries that we would need to sort would be small (e.g., up to 1,000 entries of uint32_t), and the fact that sort algorithms faster than O(n log n) radix sort can be deployed (e.g. radix sort). WDYT?
So have a cuckoo filter digest and then put its fingerprints in a GCS? I have not considered that. Need to give it some thought... At the same time, it's not clear to me how that would enable a "stale" vs. "fresh" digests, or handling of improperly cached resources (fresh resources that were replaced on the server).
I would appreciate it if you could consider. To me it seems it's worth giving a thought. Under the approach proposed in this PR, structure that stores the per-host digest would look like below. is required for resizing the filter (e.g., when doubling or halving ).
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: What I am suggesting that you could change the structure to the following. In addition to the hash value, each entry will contain the moment when the entry becomes stale. The moment can be calculated when the entry is added. For example, if the entry represents a HTTP response with a , can be calculated as . If the entry represents an immutable HTTP response, then should be set to a very large value (e.g.. assuming that underlying type of is ). When building a GCS digest, you would do the following: step 1. prepare an empty list that would contain hashes of fresh responses step 2. prepare an empty list that would contain hashes of stale responses step 3. foreach entry in cuckoofilter: step 3-1. check if the entry is fresh or not, by checking the value of step 3-2. if the entry is fresh, append of the entry to the list of the hashes of fresh responses step 3-3. otherwise, append of the entry to the list of the hashes of stale responses step 4. sort the list of hashes of the fresh responses, encode as GCS, and send step 5. sort the list of hashes of the stale responses, encode as GCS, and send You can skip the operations related to stale objects (i.e. step 2, 3-3, 5) if the server is unwilling to receive stale digests. Whether the approach can be implemented depends on if a client can determine the moment a response becomes stale. I anticipate that it is possible to determine that when you register the entry to Cuckoo filters (which is when you receive the response from the server).
NAME NAME I'm planning to attend the IETF 100 hackathon this weekend in Singapore. (First timer here. ) I'm happy to collaborate on a (URL) implementation of this spec if either of you are around and interested. I'm fairly familiar with the current spec, having implemented it and .
NAME Wonderful! I'll be attending the hackathon on both days (i.e. Saturday and Sunday). I do not think that I would have time to work on Cache Digests, but would love to discuss with you (or help, if you need) about your work on Cache Digests.
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: I've got an incomplete initial reference implementation at URL It doesn't yet include removal and querying (that's what I'll be adding next), but I did run it on a list of ~3250 URL (which I got out of my main profile chrome://cache/) and it seems to be creating reasonable sized digests. One more advantage, the digests seem to be highly compressible when sparse. Results so far: Digest with 1021 entries (so room for ~4K URLs): 5621 in-memory, 5233 gzipped (when filled with 3250 URLs). Digest with 2503 entries (so room for ~10K URLs): 13772 in-memory, 6879 gzipped (same 3250 URLs). Digest with 7919 entries (so room for ~31K URLs): 43560 in-memory, 9984 gzipped (same 3250 URLs). In practice, I think ~1000 entries is most probably enough, but it's good to know we can increase the digest size (to avoid having to recreate it), without significant over-the-wire penalty.
OK, I now have a complete reference implementation and it seems to be working fine. It also exposed an issue with the initial algorithm, forcing table allocation to accommodate a power of 2 number of entries. Latest results for 3250 URLs taken from my cache: One note: the 1021 entries table had 35 collisions, so it seems like it's insufficient for that number of URLs, unless we're willing to absorb extra pushes for ~1% of the resources.
NAME Interesting! It's good to know that we have numbers now. What is the value of P (the false positive ratio) that you used?
P=8 (so 1/256 false positive)
Note that the numbers here may be possible to further optimize. One example is semi-sorting of the buckets which the Cuckoo-Filters paper mentions, and which I have not yet implemented. It adds some runtime complexity, but can reduce the fingerprint size per resource by a full bit, so could have resulted in ~9% smaller digests in this case.
NAME Awesome! Me not being familiar with these data structures (despite reading Wikipedia article ), why does the 1/256 probability (~4/1000) result in 35 collisions?
Code change:
The value of the parameter is a bit-field of which the following bits are defined: <del> FRESH (0x1): When set, it indicates that the server is willing to make use of a digest of freshly-cached responses. STALE (0x2): When set, it indicates that the server is willing to make use of a digest of stale-cached responses. </del> <ins> ACCEPT (0x1): When set, it indicates that the server is willing to make use of a digest of cached responses. </ins> Rest of the bits MUST be ignored and MUST be left unset when sending.

Related discussion:
text: The 35 collisions are on top of the false positive rates, and represents resources that we failed to put into the table to begin with (due to both their buckets being full). That rate of collisions seems high compared to the results in the paper, so I need to dig further to see who's wrong...
The collisions are now fixed. It was an algorithm issue, where the entry to be pushed was always the same one at the end of the bucket. I've change that to be a random fingerprint from the bucket, which significantly improved things. The ref implementation is now collision free almost up to the point where the digest is full.
Code change:
is version dependent; some versions of HTTP use implicit ordering of messages, while others use an explicit identifier. <del> Responses (both </del> <ins> All responses, regardless of the status code (including </ins> <del> and </del> <ins> responses) can be sent at any time after a request is received, even if the request is not yet complete.  A response can complete before its corresponding request is complete.  Likewise, clients are not expected to wait any specific amount of time for a response.  Clients (including intermediaries) might abandon a request if the response is not forthcoming within a reasonable period of time. </ins> <del> ) can be sent at any time after a request is received, even if it is not yet complete.  However, clients (including intermediaries) might abandon a request if the response is not forthcoming within a reasonable period of time. </del> <ins> A client that receives a response while the associated request is still being sent SHOULD continue sending that request, unless it receives an explicit indication to the contrary (see, e.g., persistent.failures and RFC7540). </ins> 7.6.

Related discussion:
text: though I do wonder if HPACK/QPACK needs to be explained as well, since the order in which messages are processed does impact their compression, but not their interpretation.
I would probably say that as the semantics are still stateless, this document probably doesn't need to say anything about that. But that's a weak preference only.
The change looks good to me. That said, it might indeed be good to mention the impact of header compression.
I clarified that this was about semantics. I'm hesitant to talk about header compression, as that's likely to just confuse the matter.
I thought that was already clear. We might need to mention HPACK/QPACK anyway because they are only stateful with regard to the in-transit encoding between peers, and thus deliver stateless messages at the HTTP layer. Or something.
Clarify if the request needs to be the 1st request after a connection is opened.
Is there a reason why you ask? Are there problems when it's not the 1st request?
I'm not aware of any such problem, and for example in haproxy we do support upgrades anywhere.
Just a question I had .. which I feel the spec may want to call it out, either 1st request only or no such restriction.
Nothing in HTTP depends on where a request is placed upon a connection, because it's a stateless protocol. see last paragraph (this is likely to move to soon, which is more appropriate. Should we spell this out more clearly there?
Probably, it never hurts to remind it.
I see the case for proxies to reuse and upgrade an existing connection to the server. Mentioning this will encourage correct server implementations.
Code change:
is version dependent; some versions of HTTP use implicit ordering of messages, while others use an explicit identifier. <del> Responses (both </del> <ins> All responses, regardless of the status code (including </ins> <del> and </del> <ins> responses) can be sent at any time after a request is received, even if the request is not yet complete.  A response can complete before its corresponding request is complete.  Likewise, clients are not expected to wait any specific amount of time for a response.  Clients (including intermediaries) might abandon a request if the response is not forthcoming within a reasonable period of time. </ins> <del> ) can be sent at any time after a request is received, even if it is not yet complete.  However, clients (including intermediaries) might abandon a request if the response is not forthcoming within a reasonable period of time. </del> <ins> A client that receives a response while the associated request is still being sent SHOULD continue sending that request, unless it receives an explicit indication to the contrary (see, e.g., persistent.failures and RFC7540). </ins> 7.6.

Related discussion:
text: I saw that too, but favoured compactness over avoiding parenthesis. Happy to take suggestions.
Try that.
briefly touches this topic. More explicit descriptions are helpful, such as: responses can be sent at any time, successful for failed ones responses may be completed before requests are completed. This is described for but should not be limited to 100-continue. for 2), should a client that sees a failure response continues sending the remaining request content?
3) is somewhat addressed in when the server also closes the connection
For (1) and (2), we could adjust to say (3) is situational. We could say something like 'A client that receives a response to its request while that request is still being sent SHOULD continue sending the request, unless some other signal indicates otherwise.' (with appropriate references, perhaps).
Okay, I have tweaked this editorially so that the paragraph makes sense, IMO.
Code change:
dictionary usage and registers media types for content encoding Brotli and Zstandard using a negotiated dictionary. <ins> 1.1. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 RFC2119 RFC8174 when, and only when, they appear in all capitals, as shown here. This document uses the following terminology from STRUCTURED-FIELDS to specify syntax and parsing: Dictionary, String, Inner List, Token, and Byte Sequence. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119. </ins> This document uses the line folding strategies described in FOLDING. <ins> This document also uses terminology from HTTP and HTTP-CACHING. </ins> 2. 2.1.

Related discussion:
text: Switched from using the ABNF field types to the human names (i.e. sf-dictionary to Dictionary) for all of the Structured Field header values. Switched dictionary "type" to a Token (from String).
NAME could you PTAL at the change in referencing the header field types? Specifically, I wasn't sure if I should capitalize the types or not (I did since that's how the structured fields RFC referred to them).
Yep, this looks good. It's common to add something to you Notational Conventions section (or whatever) that says roughly "This specification uses the following terms from [ref to SF]: String, Byte Sequence, Token, ..."
Code change:
allow for a larger maximum header block size.  An intermediary generating client certificate header fields on connections that allow for advertising the maximum acceptable header block size (e.g. <del> HTTP/2 RFC7540 or HTTP/3 I-D.ietf-quic-http) should account for the additional size of header block of the requests it sends vs. requests it receives by advertising a value to its clients that is sufficiently smaller so as to allow for the addition of certificate data. </del> <ins> HTTP/2 RFC9113 or HTTP/3 RFC9114) should account for the additional size of the header block of the requests it sends vs. requests it receives by advertising a value to its clients that is sufficiently smaller so as to allow for the addition of certificate data. </ins> 3.3.

Related discussion:
text: update refs for HTTP Semantics, HTTP/3, and QPACK to point to the now RFCs 9110/9114/9204 for
Thanks Mike. That was much easier than using the "suggestions" option. I merged and then fixed(!?) the YAML header with . I think this one is good to go now?
Yes, good to go!
I-D.ietf-httpbis-semantics -> RFC 9110
Also HTTP/3 ietf-quic-http -> RFC9114 QPACK ietf-quic-qpack -> RFC9204