Draft: draft-ietf-doh-dns-over-https-latest

Commentary: In reading this a second time through , I wonder what value there is in matching the freshness lifetime to the TTL ? The implicit assumption here is that it is important that a cache never serve a response that includes an expired RRset . This simplifies the implementation of a client , but it also means that it tends to make client 's stupid . That is , they might not use RRsets that are still valid from the perspective of their TTL , simply because they are no longer in cache . It might be better to have the DNS code that uses the RRset being responsible for checking the TTL . HTTP will ensure that the time that the RRset was generated is faithfully recorded and transferred with the response , so that should be enough . It potentially allows use of old responses for RRsets that are still valid . Finally , the mention of stale - while - revalidate and stale - if - error actually suggests that this requirement is more . These operate outside of the freshness lifetime . 
 I think it 's important that DOH be usable with standard ( non - DOH - aware ) HTTP caches . Allowing the cache lifetime to exceed the shortest TTL might make it impossible to use DOH through such a cache : the client can not flush the cache , and the cache entry might not expire until after its longest TTL . 
 +1 to response . 
 Clients can always set Cache - Control : no - cache and avoid the cache . The point here is that the TTL is the authoritative value . Insisting on HTTP matching that both loses fidelity ( multiple RRsets have different TTLs ) and encourages lazy implementation of clients . 
 Agree with everyone . I would say that a server / parser could in that case imagine to set a " retry - after:60 " HTTP header ( corresponding to the lower TTL in this example RR ) ; when TTL < RR situation verifies ( simple calculation , t0 plus lowest values in RR ) , a " vary : retry - after " header could control the re - freshing of object ( aka , force refresh sometimes but do still do cache in the meantime ) . By the way , some ISP 's DNS used to generally override all TTL that were lower than 60 seconds , used to be even 300 seconds .  halfway between some cache and good reachability / redundancy would be good enough ( TTL real value is to support / improve the service continuity / validity . TTL mission is not guessing how long a service would keep being available , or running ) . Some services as Skype would get such cache crazy , due to 5 to 30 seconds  .... a choice must be done somewhere between precision and performances .. An average HTTP cache lasts between minutes and days , say everywhere from 1 to 8 hours in proxy administration praxis , 24 hours  . 24 hours in DNS terms is unacceptable . How accurately will the HTML programmer choose such blind - validity duration ? In some ways , choosing HTTP validity exactly the same as the choiche of TTL for a given record , a pure bet ! ! Caching is one of the pillars of DOH in terms of speed and HTTP - friendliness . As one of the goals is ( i believe ) creating and populating a global valid cache through HTTP ( as alternative or complement of existing DNS cache ) , doing no - cache is also a bit " against " the principle . but still a feature for privacy amateurs . I see value in HTTP cache also in case of DDoS ; HTTP / DOH cache could represent a good fallback / backup of DNS data anyway , and HTTP is proved more resilent to flooding . One of the goals of DOH is to leverage use of existing infrastructures ! et 
 In that case , setting is the only way a client can make sure that it actually gets non - expired answers . This creates a strong incentive , in both performance and complexity , to set on all DOH requests , defeating HTTP caching altogether . Yikes ! 
 I favor keeping the current arrangement . A server that wishes to avoid the crux of the issue ( i.e. that HTTP has one TTL , but the DNS response is a set of TTLs ) can decompose response into multiple HTTP responses using HTTP/2 push each with their own TTLs . That seems like the right approach . I have a pretty strong inclination that the principle of least surprise says data in a successful HTTP response can always be used one time ( but not reused ) even if you do n't implement any caching logic . I know your suggestion does n't technically infringe on that - its more of a requirement on the processing of the response- but it seems like a much more fragile alignment . and I think is on to something important with cache interop . 
 My .02 - I think that the advice in the current draft is good -- but it 's just advice , and should n't have RFC2119 requirements associated with it . From the current markdown : Under what conditions can this SHOULD be violated ? What is a client to do when it encounters a violation ? Suggest " Specifically , the HTTP freshness lifetime should generally be set ... " As above , there may be cases where this is desirable . Suggest " Be aware that if the response freshness lifetime is greater than that indicated by the DNS resource record with the smallest TTL in the response , HTTP caches can store those records for longer than they are valid for ; clients can request a fresh HTTP response ( e.g. , using the Cache - Control : no - cache request directive ) in this situation , but this can add latency . " Question : should we say anything about whether a DOH client can use a stale HTTP response , and the conditions under which it should ? As above . This is really vague for a MUST . implies ; is that required ? Also , what is the otherwise applying to -- is this all responses that have no records in the answer section and no records in the DNS response ? Many DNS API clients wo n't have any control over the heuristics that their ( or an upstream ) cache applies ; this is n't an enforceable requirement . 
 beat me to it . I think that we 're making suggestions , not interoperability requirements . I would prefer to have no freshness information ( and rely on heuristics ) than to recommend marking things uncacheable . I think that the rules are similar to using ( and inflating ) TTL . Saying nothing might be easiest . 
 Under no circumstances does an HTTP client have to expect that a server may serve it an expired response out of cache , and recover by retrying with caching disabled . Under no circumstance does a DNS client have to expect that a recursive may serve it an expired response , and recover by bypassing the recursive cache somehow . There 's a good reason for this : this architecture is complex , adds latency , and encourages the client to abandon caching rather than accept the complexity and retry latency . I do not think we can reasonably accept a specification that puts DOH clients in this position . To be clear , this is an interoperability question . A server that adopts a longer - than - min - TTL policy and a simple client ( e.g. any existing client ) are not interoperable . 
 longer - than - min - TTL is not significantly different to a DNS server extending the TTL . It 's not advisable , but it still happens . The difference is that HTTP makes this visible , where DNS does not . A client that makes a request can reasonably use the response without checking either freshness lifetime or TTL . 
 The current doc states " The response freshness lifetime MUST NOT be greater than that indicated by the DNS resource record with the smallest TTL in the response . " I am guessing that this and any other similar text should refer to RRSet not RR . 
 I really hate the " A DOH response that was previously stored in an HTTP cache will contain the       response header indicating the elapsed time between when the entry was placed in the HTTP cache and the current DOH response . DNS API clients should subtract this time from the DNS TTL in each RRset to determine the time remaining on the TTL . " in DNS that is the job of the cache not the client . Or do you want to support DNS unaware HTTP caches ? 
 yes - that 's part of using http as the substrate .. 
 I agree : in both cases , it is a protocol violation . The fact that some recursive resolvers fail to implement the DNS protocol does not mean that we need to endorse their incorrect behavior . If they want to violate RFC 1035 , they can equally well violate DOH . The fact that these violations usually mostly work in practice is all the more reason to be clear about the designed behavior , because people who get it wrong wo n't notice for a long time . Perhaps we should state this normatively . 
 Hear hear . Exactly the point I was about to make . 
 Please take this all to the mailing list . I have ( again ) opened up a topic there . 
 I 'm just reopening this because I use the github issues list as an editorial punchlist - not a comment about where to discuss :) 
 OK , but the rest of you : please do n't comment here , but please do comment on the list . Thanks ! 
 that is answered in the next sentence : " The response freshness lifetime MUST NOT be greater than that indicated by the DNS resource record with the smallest TTL in the response . " its basically saying you SHOULD make HTTP and DNS match , but if you ca n't do that ( because perhaps your code module ca n't fully parse DNS ) you can do something like  - age = 0 
 I will soon commit as my best interpretation of the discussion ( here and on the mailling list ) It accepts most of  's changes but preserves the normative language and adds back a couple of small things that were the result of previous explicit working group requests ( e.g. noting that cache busting is not something that can be done easily to a DNS - only cache ) . 
 Seems mostly good , but see for my edits .
Paragraph (version A):
A DNS API client may utilize a hierarchy of caches that include both
HTTP and DNS specific caches.  HTTP cache entries may be bypassed
with HTTP mechanisms such as the "no-cache" Cache-Control directive;
however DNS caches do not have a similar mechanism.
Paragraph (version A):
The Answer section of a DNS response can contain zero or more RRsets.
(RRsets are defined in RFC7719.)  According to RFC2181, each resource
record in an RRset has Time To Live (TTL) freshness information.
Different RRsets in the Answer section can have different TTLs,
although it is only possible for the HTTP response to have a single
freshness lifetime.  The HTTP response freshness lifetime (RFC7234
Section 4.2) should be coordinated with the RRset with the smallest
TTL in the Answer section of the response.  Specifically, the HTTP
freshness lifetime SHOULD be set to expire at the same time any of
the DNS resource records in the Answer section reach a 0 TTL.  The
response freshness lifetime MUST NOT be greater than that indicated
by the DNS resource record with the smallest TTL in the response.
Paragraph (version A):
If the DNS response has no records in the Answer section, and the DNS
response has an SOA record in the Authority section, the response
freshness lifetime MUST NOT be greater than the MINIMUM field from
that SOA record.  (See RFC2308.)  Otherwise, the HTTP response MUST
set a freshness lifetime (RFC7234 Section 4.2) of 0 by using a
mechanism such as the "no-cache" Cache-Control directive (RFC7234
Section 5.2.1.4).
Paragraph (version A):
A DNS API client that receives a response without an explicit
freshness lifetime MUST NOT assign that response a heuristic
freshness (RFC7234 Section 4.2.2.) greater than that indicated by the
DNS Record with the smallest TTL in the response.
Paragraph (version A):
A DOH response that was previously stored in an HTTP cache will
contain the RFC7234 Age response header indicating the elapsed time
between when the entry was placed in the HTTP cache and the current
DOH response.  DNS API clients should subtract this time from the DNS
TTL in each RRset to determine the time remaining on the TTL.
Paragraph (version A):
HTTP revalidation (e.g., via If-None-Match request headers) of cached
DNS information may be of limited value to DOH as revalidation
provides only a bandwidth benefit and DNS transactions are normally
latency bound.  Furthermore, the HTTP response headers that enable
revalidation (such as "Last-Modified" and "Etag") are often fairly
large when compared to the overall DNS response size, and have a
variable nature that creates constant pressure on the HTTP/2
compression dictionary RFC7541.  Other types of DNS data, such as
zone transfers, may be larger and benefit more from revalidation.
DNS API servers may wish to consider whether providing these
validation enabling response headers is worthwhile.
Paragraph (version A):
The "stale-while-revalidate" and "stale-if-error" Cache-Control
directives (RFC5861) could be well suited to a DOH implementation
when allowed by server policy.  Those mechanisms allow a client, at
the server's discretion, to reuse a cache entry that is no longer
fresh.
Paragraph (version A):
All HTTP servers, including DNS API servers, need to consider cache
interaction when they generate responses that are not globally valid.
For instance, if a DNS API server customized a response based on the
client's identity then it would not want to globally allow reuse of
that response.  This could be accomplished through a variety of HTTP
techniques such as a Cache-Control max-age of 0, or perhaps by the
Vary response header.
Paragraph (version B):
A DOH exchange can pass through a hierarchy of caches that include
both HTTP and DNS specific caches.  These caches may exist beteen the
DNS API server and client, or on the DNS API client itself.  HTTP
caches are by design generic; that is, they do not understand this
protocol.  Even if a DNS API client has modified its cache
implementation to be aware of DOH semantics, it does not follow that
all upstream caches (for example, inline proxies, server-side
gateways and Content Delivery Networks) will be.
Paragraph (version B):
As a result, DNS API servers need to carefully consider the HTTP
caching metadata they send in response to GET requests (POST requests
are not cacheable unless specific response headers are sent; this is
not widely implemented, and not advised for DOH).
Paragraph (version B):
In particular, DNS API servers SHOULD assign an explicit freshness
lifetime (RFC7234 Section 4.2) so that the DNS API client is more
likely to use fresh DNS data.  This requirement is due to HTTP caches
being able to assign their own heuristic freshness (such as that
described in RFC7234 Section 4.2.2), which would take control of the
cache contents out of the hands of the DNS API server.
Paragraph (version B):
The assigned freshness lifetime of a DOH HTTP response SHOULD be the
smallest TTL in the Answer section of the DNS response.  For example,
if a HTTP response carries three RRsets with TTLs of 30, 600, and
300, the HTTP freshness lifetime should be 30 seconds (which could be
specified as "Cache-Control: max-age=30").  The assigned freshness
lifetime MUST NOT be greater than the smallest TTL in the Answer
section of the DNS response.  This requirement helps assure that none
of the RRsets contained in a DNS response are served stale from an
HTTP cache.
Paragraph (version B):
If the DNS response has no records in the Answer section, and the DNS
response has an SOA record in the Authority section, the response
freshness lifetime MUST NOT be greater than the MINIMUM field from
that SOA record (see RFC2308).
Paragraph (version B):
The stale-while-revalidate and stale-if-error Cache-Control
directives (RFC5861) could be well suited to a DOH implementation
when allowed by server policy.  Those mechanisms allow a client, at
the server's discretion, to reuse a cache entry that is no longer
fresh.  In such a case, the client reuses all of a cached entry, or
none of it.
Paragraph (version B):
DNS API servers also need to consider caching when generating
responses that are not globally valid.  For instance, if a DNS API
server customizes a response based on the client's identity, it would
not want to allow global reuse of that response.  This could be
accomplished through a variety of HTTP techniques such as a Cache-
Control max-age of 0, or by using the Vary response header (RFC7231
Section 7.1.4) to establish a secondary cache key (RFC7234
Section 4.1).
Paragraph (version B):
DNS API clients MUST account for the Age response header's value
(RFC7234) when calculating the DNS TTL of a response.  For example,
if a RRset is received with a DNS TTL of 600, but the Age header
indicates that the response has been cached for 250 seconds, the
remaining lifetime of the RRset is 350 seconds.
Paragraph (version B):
DNS API clients can request an uncached copy of a response by using
the "no-cache" request cache control directive (RFC7234,
Section 5.2.1.4) and similar controls.  Note that some caches might
not honor these directives, either due to configuration or
interaction with traditional DNS caches that do not have such a
mechanism.
Paragraph (version B):
HTTP conditional requests (RFC7232) may be of limited value to DOH,
as revalidation provides only a bandwidth benefit and DNS
transactions are normally latency bound.  Furthermore, the HTTP
response headers that enable revalidation (such as "Last-Modified"
and "Etag") are often fairly large when compared to the overall DNS
response size, and have a variable nature that creates constant
pressure on the HTTP/2 compression dictionary RFC7541.  Other types
of DNS data, such as zone transfers, may be larger and benefit more
from revalidation.