Code change:
This removes the dictionary-specific ttl and switches to using the dictionary resource freshness as the mechanism for expiring dictionaries.
NAME could you please take a look and see if this looks ok? NAME could you look at the language around the freshness of the resource relative to cache-control and stale-while-revalidate and make sure I expressed it properly (and it is something we're ok with leveraging)?
Why does the dictionary lifetime need to be independent of the cache lifetime? Having an object that is cached with a separate lifetime seems unnecessary. I would have said that clients can cache a dictionary for as long as they want (as always), but no longer than the lifetime stated in the cache-control field. Now, you might say that a server might want a resource to be usable in its own right (i.e., as an ordinary resource) for one duration, with a separate lifetime for its use as a dictionary. I see two reasons that this isn't much of a compelling argument: A server can forget about a dictionary and not provide delta compression at any time, so there is no real need for the dictionary lifetime to be shorter than the cache lifetime. Client caches won't want to retain something for longer than its validity, so I don't see much chance that a resource will be kept past its cache lifetime. Resources like this are likely exclusively used as a dictionary. There isn't much value for those resources having a separate lifetime in that case.
The main reason for a separate lifetime for the dictionaries is to allow for sites to reduce the possible variations that are in the wild and reduce the number of variations of that come in and constrain the number of previous releases that they need to compress artifacts for. Otherwise, a dictionary "miss" will result in a full version of the resource getting added to caches for every possible version of where delta-compressed artifacts aren't available. Caches can probably manage that OK on their own, but by reducing the dictionaries to a known window it both provides a lever of control on the variations and also allows for clients to "hit" on a cached version of the resource when they have an old dictionary.
This has been rattling around in the back of my head for the last month or so and I wanted to get my thoughts down for discussion. I think it might be possible to use the existing validity of max-age and stale-while-revalidate to achieve the same result but I'm a bit worried about the complexity that would bring to users. There are 2 main scenarios that express the tension: Versioned immutable resources that change the URL when the content changes. Well-known URLs that are cacheable but update the content in-place (libraries for analytics, ads, embeds, etc). For the purposes of this example, lets assume that the resources are updated twice a day and that 80% of repeat traffic re-visits within 10 days of their previous visit. Assuming a serving architecture with a server-side cache (CDN, load balancer, etc): In both cases, the response is cacheable and keyed based on the dictionary that the client sends as available (). The main concern is how to tune the requests so that the cache behaves like it does today, minimizing traffic to the origin. The "current" version of the resource will be the only one actively fetched by the application independent of when the user last visited. The current best-practice is to cache them for a long time (1 year) and mark them as immutable. Without dictionaries that means that the cache will effectively only see requests for the current version of the resource and all previous versions can age out of the cache (even if they have a long time-to-live, just by virtue of never being accessed). There will be one request to the origin (or a small number) while the cache gets populated but all future requests will always be served from the edge cache. With dictionaries, the cache key depends on when the client last visited the page since the dictionary will be the version of the resource they have in their cache. That means 80% of the requests will be covered by 21 cache keys (one without a dictionary and 20 delta-compressed variants) but that there will be 700+ variants in the wild if the 1-year resource expiration is reused for the dictionary. The cache could manage access frequency to store just the 21 "hot" variants for the last 10 days but the 20% of visits that are using older dictionaries will result in cache misses that require going back to the origin. If a developer shortens the resource cache lifetime to a shorter time, like 5 days, knowing their release schedule then the problem largely goes away. That would require tying the dictionary use to the expiration of the resource and not allow for it to be used even if it is still in cache (but expired). That would mean we are now imposing a short cache lifetime on an immutable resource in order to optimize the hit rate of the middle caches. It feels like explicitly managing the dictionary lifetime at the same place where the dictionary details are specified might be cleaner. For well-known URLs that are not immutable and change the content in-place, we want to be able to use the previous version(s) of a resource as a dictionary for the "current" version. To be able to do that we need the dictionary to be viable beyond the expiration time of the resource. Since we need to hard-clip the use of the dictionaries to prevent cache key issues we'd need to have the dictionaries use something like the time window to allow for a revalidation request of an expired resource while still using the dictionary but allowing for a hard-clip of the dictionary validity at the end of the stale-while-revalidate time. It's theoretically possible but it feels like we're tying the dictionary TTL into the cache heuristics in ways that might not be intended and might also be easy to get wrong.
"well-known" is a term of art you might want to avoid here. I get your intent. It looks like there is a fairly good suggestion for . It might take a bit of time for a cache to be able to work out the length of time from the last access that represents a good time horizon for use. But that seems manageable. The resources that churn is an interesting one. It seems like you are assuming that the cache lifetime is short, such that a replacement version is requested AFTER the cache lifetime expires. Your option does seem like a possible way of managing this, but I'd be unsure whether stale resources are good for use. You'd have to specify that behavior (reasonable, but it would need to be specified). It might be that a longer cache lifetime also works well enough. Clients do tend to re-check earlier than expiration in many cases, which would ensure that the resource is fresh in use, while making the overlap work well.
NAME NAME does using the resource max-age + stale-while-revalidate as and expiration for the client-side dictionary work ok for you (with a recommendation that immutable resources specify expirations based on their typical release cycle)?
I think that having an explicit life time for the dictionary is a little bit more flexible than relying on cache-control headers, but I think that by itself should outlive most dictionary uses I can think of, especially given that the CDN is unlikely to keep those resources in its own cache beyond that period as well.
NAME I'm not sure I understand (or we're not talking about the same part of the issue). The main concern for caches is how long clients consider a dictionary valid for which will dictate how long of a window a given resource will see request headers from. e.g. for a dictionary with a max-age of 1 year, clients will advertise that dictionary as being available for up to a year. If there are 1,000 updates to the resource used as a dictionary (not impossible for a js app that has 3 releases per day) then there will be 1,000 variants of the dictionary in the wild at any given time (with more frequent requests for more recent dictionaries but still some infrequent long-tail requests). To prevent cache misses, it would be beneficial to cap the dictionary validity to a smaller window where most return traffic falls in and have older clients download the full resource from cache rather than missing and going back to the origin. It also caps how many delta-compressed versions need to be created. The discussion point is if that "client valid" time for a dictionary should be its own separate value or if sites should adjust their max-age to be more reasonable instead if they want to limit the variants (capping the validity to the max-age but requiring a shorter max-age rather than blindly setting everything to 1-year).
Worst-case, a separate ttl could be added later if using the resource expiration ends up being a problem blocking adoption.
It seems like there are two different mental models of how caching and cache-misses could work. The difference between them may be contributing to the confusion. The first is that the CDN is keeping "resource A compressed against dictionary X" as a cache item, and if a client makes a request for "resource A compressed against dictionary Z", the CDN needs to go back to origin to retrieve that combination. The second is that the CDN is keeping "resource A" and "resource Z" as cache items, and if a client makes a request for "resource A compressed against dictionary Z", the CDN either has Z locally or not. If not, it will compress resource A in a different way the client supports. (Of course, this doesn't prevent the CDN from having some amount of second-layer caching to avoid recompressing frequent combinations.) Personally, I tend to think the second tends to have better properties -- you don't have an explosion of cache entries or origin requests for the long tail, you simply fall back to a non-dictionary compression scheme. CDNs might choose to count usage as a dictionary as a cache hit for Z, making it stay in cache as long as it continues to be used by clients, but might not choose to fetch Z again if it ages out of the cache. (The interesting quirk from a CDN perspective is that a single resource might have multiple old versions still getting "hits.") The two then probably imply different ways of thinking about lifetimes. In the first case, you could want a separate lifetime on the dictionary to feed into how long the CDN needs to keep around the f(A,Z) item. In the second, the lifetimes of the individual objects likely suffice. (For that matter, if the client is identifying the version of Z it has, then it doesn't really matter whether that object is fresh or not, only whether the server still possesses the indicated version.)
The main use case for this is for something like a web app at build time where they pre-compress the resources at build time for some number of previous releases. At serving time, the CDN doesn't have any special dictionary logic and just honors the headers to vary on and the origin checks for pre-compressed assets using the requested dictionary (in my apps that I was doing this with, I just added the hash to the end of the file name and did a quick file check). On a miss, the origin sends the full resource which ends up in a separate cache key on the CDN. This allows for origins to support dictionary-compressing the content without their CDN or load balancer having to have implemented it.
Right, that's a reasonable workaround for CDNs that don't know about this spec. But wouldn't we rather the CDNs could implement the spec and calculate the delta at the edge? Wouldn't that be the desired end state? Perhaps I'm wrong, or perhaps I've wandered into an architectural question bigger than this issue.
I'm not 100% sure it's just a workaround and origin-compressing against previous releases at build time may be preferable for some cases. The most extreme case that comes to mind is something like a huge WASM app (like Photoshop) where the files are upwards of 50MB (well, huge relative to the usual web page resources). Presumably the CDN could still do the work to do the compression (maybe as an offline job for future requests) so it doesn't require the origin to do it necessarily but it's a lot more reasonable in the extreme cases. That said, I think we can still get buy without a separate TTL and adding one later if it ends up being useful can be done as an enhancement without breaking the existing case.
I like red patches. Thanks.

Related discussion:
text: header. The Use-As-Dictionary response header is a Structured Field <del> STRUCTURED-FIELDS sf-dictionary with values for "match", "match- dest", "ttl", "id", and "type". </del> <ins> STRUCTURED-FIELDS with values for "match", "match-dest", "id", and "type". </ins> 2.1.1.