Change:
In reading this a second time through, I wonder what value there is in matching the freshness lifetime to the TTL? The implicit assumption here is that it is important that a cache never serve a response that includes an expired RRset. This simplifies the implementation of a client, but it also means that it tends to make client's stupid. That is, they might not use RRsets that are still valid from the perspective of their TTL, simply because they are no longer in cache. It might be better to have the DNS code that uses the RRset being responsible for checking the TTL. HTTP will ensure that the time that the RRset was generated is faithfully recorded and transferred with the response, so that should be enough. It potentially allows use of old responses for RRsets that are still valid. Finally, the mention of stale-while-revalidate and stale-if-error actually suggests that this requirement is more. These operate outside of the freshness lifetime.
I think it's important that DOH be usable with standard (non-DOH-aware) HTTP caches. Allowing the cache lifetime to exceed the shortest TTL might make it impossible to use DOH through such a cache: the client cannot flush the cache, and the cache entry might not expire until after its longest TTL.
+1 to NAME response.
Clients can always set Cache-Control: no-cache and avoid the cache. The point here is that the TTL is the authoritative value. Insisting on HTTP matching that both loses fidelity (multiple RRsets have different TTLs) and encourages lazy implementation of clients.
Agree with everyone. I would say that a server/parser could in that case imagine to set a "retry-after:60 " HTTP header (corresponding to the lower TTL in this example RR); when TTL < RR situation verifies (simple calculation, t0 plus lowest values in RR), a "vary: retry-after" header could control the re-freshing of object (aka, force refresh sometimes but do still do cache in the meantime). By the way, some ISP's DNS used to generally override all TTL that were lower than 60 seconds, used to be even 300 seconds. meeting halfway between some cache and good reachability/redundancy would be good enough (TTL real value is to support/improve the service continuity/validity. TTL mission is not guessing how long a service would keep being available, or running). Some services as Skype would get such cache crazy, due to 5 to 30 seconds TTL.... a choice must be done somewhere between precision and performances.. An average HTTP cache lasts between minutes and days, say everywhere from 1 to 8 hours in proxy administration praxis, 24 hours max. 24 hours in DNS terms is unacceptable. How accurately will the HTML programmer choose such blind-validity duration? In some ways, choosing HTTP validity exactly the same as the choiche of TTL for a given record, a pure bet !! Caching is one of the pillars of DOH in terms of speed and HTTP-friendliness. As one of the goals is (i believe) creating and populating a global valid cache through HTTP (as alternative or complement of existing DNS cache), doing no-cache is also a bit "against" the principle. but still a feature for privacy amateurs. I see value in HTTP cache also in case of DDoS; HTTP/DOH cache could represent a good fallback/backup of DNS data anyway, and HTTP is proved more resilent to flooding. One of the goals of DOH is to leverage use of existing infrastructures ! URL et URL
NAME In that case, setting is the only way a client can make sure that it actually gets non-expired answers. This creates a strong incentive, in both performance and complexity, to set on all DOH requests, defeating HTTP caching altogether. Yikes!
I favor keeping the current arrangement. A server that wishes to avoid the crux of the issue (i.e. that HTTP has one TTL, but the DNS response is a set of TTLs) can decompose response into multiple HTTP responses using HTTP/2 push each with their own TTLs. That seems like the right approach. I have a pretty strong inclination that the principle of least surprise says data in a successful HTTP response can always be used one time (but not reused) even if you don't implement any caching logic. I know your suggestion doesn't technically infringe on that - its more of a requirement on the processing of the response- but it seems like a much more fragile alignment. and I think NAME is on to something important with cache interop.
My .02 - I think that the advice in the current draft is good -- but it's just advice, and shouldn't have RFC2119 requirements associated with it. From the current markdown: Under what conditions can this SHOULD be violated? What is a client to do when it encounters a violation? Suggest "Specifically, the HTTP freshness lifetime should generally be set..." As above, there may be cases where this is desirable. Suggest "Be aware that if the response freshness lifetime is greater than that indicated by the DNS resource record with the smallest TTL in the response, HTTP caches can store those records for longer than they are valid for; clients can request a fresh HTTP response (e.g., using the Cache-Control: no-cache request directive) in this situation, but this can add latency." Question: should we say anything about whether a DOH client can use a stale HTTP response, and the conditions under which it should? As above. This is really vague for a MUST. implies ; is that required? Also, what is the otherwise applying to -- is this all responses that have no records in the answer section and no records in the DNS response? Many DNS API clients won't have any control over the heuristics that their (or an upstream) cache applies; this isn't an enforceable requirement.
NAME beat me to it. I think that we're making suggestions, not interoperability requirements. I would prefer to have no freshness information (and rely on heuristics) than to recommend marking things uncacheable. I think that the rules are similar to using (and inflating) TTL. Saying nothing might be easiest.
Under no circumstances does an HTTP client have to expect that a server may serve it an expired response out of cache, and recover by retrying with caching disabled. Under no circumstance does a DNS client have to expect that a recursive may serve it an expired response, and recover by bypassing the recursive cache somehow. There's a good reason for this: this architecture is complex, adds latency, and encourages the client to abandon caching rather than accept the complexity and retry latency. I do not think we can reasonably accept a specification that puts DOH clients in this position. To be clear, this is an interoperability question. A server that adopts a longer-than-min-TTL policy and a simple client (e.g. any existing client) are not interoperable.
longer-than-min-TTL is not significantly different to a DNS server extending the TTL. It's not advisable, but it still happens. The difference is that HTTP makes this visible, where DNS does not. A client that makes a request can reasonably use the response without checking either freshness lifetime or TTL.
The current doc states "The response freshness lifetime MUST NOT be greater than that indicated by the DNS resource record with the smallest TTL in the response." I am guessing that this and any other similar text should refer to RRSet not RR.
I really hate the "A DOH response that was previously stored in an HTTP cache will contain the {{RFC7234}} Age response header indicating the elapsed time between when the entry was placed in the HTTP cache and the current DOH response. DNS API clients should subtract this time from the DNS TTL in each RRset to determine the time remaining on the TTL." in DNS that is the job of the cache not the client. Or do you want to support DNS unaware HTTP caches?
yes - that's part of using http as the substrate..
I agree: in both cases, it is a protocol violation. The fact that some recursive resolvers fail to implement the DNS protocol does not mean that we need to endorse their incorrect behavior. If they want to violate RFC 1035, they can equally well violate DOH. The fact that these violations usually mostly work in practice is all the more reason to be clear about the designed behavior, because people who get it wrong won't notice for a long time. Perhaps we should state this normatively.
Hear hear. Exactly the point I was about to make.
Please take this all to the mailing list. I have (again) opened up a topic there.
I'm just reopening this because I use the github issues list as an editorial punchlist - not a comment about where to discuss :)
OK, but the rest of you: please don't comment here, but please do comment on the list. Thanks!
that is answered in the next sentence: "The response freshness lifetime MUST NOT be greater than that indicated by the DNS resource record with the smallest TTL in the response. " its basically saying you SHOULD make HTTP and DNS match, but if you can't do that (because perhaps your code module can't fully parse DNS) you can do something like max-age = 0
I will soon commit as my best interpretation of the discussion (here and on the mailling list) It accepts most of Mark's changes but preserves the normative language and adds back a couple of small things that were the result of previous explicit working group requests (e.g. noting that cache busting is not something that can be done easily to a DNS-only cache).
Seems mostly good, but see 725ad7a for my edits.

Discussion:
6.1. <del> A DNS API client may utilize a hierarchy of caches that include both HTTP and DNS specific caches.  HTTP cache entries may be bypassed with HTTP mechanisms such as the "no-cache" Cache-Control directive; however DNS caches do not have a similar mechanism. The Answer section of a DNS response can contain zero or more RRsets. (RRsets are defined in RFC7719.)  According to RFC2181, each resource record in an RRset has Time To Live (TTL) freshness information. Different RRsets in the Answer section can have different TTLs, although it is only possible for the HTTP response to have a single freshness lifetime.  The HTTP response freshness lifetime (RFC7234 Section 4.2) should be coordinated with the RRset with the smallest TTL in the Answer section of the response.  Specifically, the HTTP freshness lifetime SHOULD be set to expire at the same time any of the DNS resource records in the Answer section reach a 0 TTL.  The response freshness lifetime MUST NOT be greater than that indicated by the DNS resource record with the smallest TTL in the response. </del> <ins> A DOH exchange can pass through a hierarchy of caches that include both HTTP and DNS specific caches.  These caches may exist beteen the DNS API server and client, or on the DNS API client itself.  HTTP caches are by design generic; that is, they do not understand this protocol.  Even if a DNS API client has modified its cache implementation to be aware of DOH semantics, it does not follow that all upstream caches (for example, inline proxies, server-side gateways and Content Delivery Networks) will be. As a result, DNS API servers need to carefully consider the HTTP caching metadata they send in response to GET requests (POST requests are not cacheable unless specific response headers are sent; this is not widely implemented, and not advised for DOH). In particular, DNS API servers SHOULD assign an explicit freshness lifetime (RFC7234 Section 4.2) so that the DNS API client is more likely to use fresh DNS data.  This requirement is due to HTTP caches being able to assign their own heuristic freshness (such as that described in RFC7234 Section 4.2.2), which would take control of the cache contents out of the hands of the DNS API server. The assigned freshness lifetime of a DOH HTTP response SHOULD be the smallest TTL in the Answer section of the DNS response.  For example, if a HTTP response carries three RRsets with TTLs of 30, 600, and 300, the HTTP freshness lifetime should be 30 seconds (which could be specified as "Cache-Control: max-age=30").  The assigned freshness lifetime MUST NOT be greater than the smallest TTL in the Answer section of the DNS response.  This requirement helps assure that none of the RRsets contained in a DNS response are served stale from an HTTP cache. </ins> If the DNS response has no records in the Answer section, and the DNS response has an SOA record in the Authority section, the response freshness lifetime MUST NOT be greater than the MINIMUM field from <del> that SOA record.  (See RFC2308.)  Otherwise, the HTTP response MUST set a freshness lifetime (RFC7234 Section 4.2) of 0 by using a mechanism such as the "no-cache" Cache-Control directive (RFC7234 Section 5.2.1.4). A DNS API client that receives a response without an explicit freshness lifetime MUST NOT assign that response a heuristic freshness (RFC7234 Section 4.2.2.) greater than that indicated by the DNS Record with the smallest TTL in the response. A DOH response that was previously stored in an HTTP cache will contain the RFC7234 Age response header indicating the elapsed time between when the entry was placed in the HTTP cache and the current DOH response.  DNS API clients should subtract this time from the DNS TTL in each RRset to determine the time remaining on the TTL. HTTP revalidation (e.g., via If-None-Match request headers) of cached DNS information may be of limited value to DOH as revalidation provides only a bandwidth benefit and DNS transactions are normally latency bound.  Furthermore, the HTTP response headers that enable revalidation (such as "Last-Modified" and "Etag") are often fairly large when compared to the overall DNS response size, and have a variable nature that creates constant pressure on the HTTP/2 compression dictionary RFC7541.  Other types of DNS data, such as zone transfers, may be larger and benefit more from revalidation. DNS API servers may wish to consider whether providing these validation enabling response headers is worthwhile. The "stale-while-revalidate" and "stale-if-error" Cache-Control </del> <ins> that SOA record (see RFC2308). The stale-while-revalidate and stale-if-error Cache-Control </ins> directives (RFC5861) could be well suited to a DOH implementation when allowed by server policy.  Those mechanisms allow a client, at the server's discretion, to reuse a cache entry that is no longer <del> fresh. All HTTP servers, including DNS API servers, need to consider cache interaction when they generate responses that are not globally valid. For instance, if a DNS API server customized a response based on the client's identity then it would not want to globally allow reuse of that response.  This could be accomplished through a variety of HTTP techniques such as a Cache-Control max-age of 0, or perhaps by the Vary response header. </del> <ins> fresh.  In such a case, the client reuses all of a cached entry, or none of it. DNS API servers also need to consider caching when generating responses that are not globally valid.  For instance, if a DNS API server customizes a response based on the client's identity, it would not want to allow global reuse of that response.  This could be accomplished through a variety of HTTP techniques such as a Cache- Control max-age of 0, or by using the Vary response header (RFC7231 Section 7.1.4) to establish a secondary cache key (RFC7234 Section 4.1). DNS API clients MUST account for the Age response header's value (RFC7234) when calculating the DNS TTL of a response.  For example, if a RRset is received with a DNS TTL of 600, but the Age header indicates that the response has been cached for 250 seconds, the remaining lifetime of the RRset is 350 seconds. DNS API clients can request an uncached copy of a response by using the "no-cache" request cache control directive (RFC7234, Section 5.2.1.4) and similar controls.  Note that some caches might not honor these directives, either due to configuration or interaction with traditional DNS caches that do not have such a mechanism. HTTP conditional requests (RFC7232) may be of limited value to DOH, as revalidation provides only a bandwidth benefit and DNS transactions are normally latency bound.  Furthermore, the HTTP response headers that enable revalidation (such as "Last-Modified" and "Etag") are often fairly large when compared to the overall DNS response size, and have a variable nature that creates constant pressure on the HTTP/2 compression dictionary RFC7541.  Other types of DNS data, such as zone transfers, may be larger and benefit more from revalidation. </ins> 6.2.