Change:
probability may be increased via traffic analysis or other mechanisms. <del> The Resource Record TTL determines the lifetime of the published ESNI keys.  Clients MUST NOT use ESNI keys beyond their published lifetime.  Note that the length of this structure MUST NOT exceed 2^16 - 1, as the RDLENGTH is only 16 bits RFC1035. </del> <ins> The "checksum" field provides protection against transmission errors, including those caused by intermediaries such as a DNS proxy running on a home router.  "not_before" and "not_after" fields represent the validity period of the published ESNI keys.  Clients MUST NOT use ESNI keys that was covered by an invalid checksum or beyond the published period.  Servers SHOULD set the Resource Record TTL small enough so that the record gets discarded by the cache before the ESNI keys reach the end of their validity period. Note that the length of this structure MUST NOT exceed 2^16 - 1, as the RDLENGTH is only 16 bits RFC1035. </ins> 5.

Discussion:
In ESNI, the server offers a list of ECDH keys and the client selects one of them. However, there seems to be nothing that prevents a downgrade attack. This PR adds the protection, by sending the list of the Named Groups being offered by ESNIKeys in the EncryptedSNI extension. The bonus is that the size of ESNIKeys becomes smaller when more than one keys are offered (because we now only send one label per the entire key set).
Consider the case where the authoritative server sends an esni record containing X448 and X25519 keys. Malicious middlebox rewrites the esni record to only contain the X25519 key. The client will accept the X25519 key and use that for the TLS handshake. I think that is not prevented now, and that we need to prevent it.
NAME thanks for identifying this. I agree it's a real issue and wish I'd caught it before :( Thinking about defenses, do you think it might make more sense to ditch the label and instead just carry a hash of the record? Then we could guarantee that it wasn't changed more easily? OTOH, that would preclude self-encryption of the ESNI Key....
NAME Servers might want more than one label per record to allow for migration. Why don't we have clients send the hash of the ESNKey structure for the label it chooses? The server could also include that in full in place of the truncated hash.
NAME I meant that we don't need to send the NamedGroupsList for mitigation -- we could send something else, e.g., the ESNIKey hash.
NAME The other issue is that you then need to consider negotiating the hash algorithm to be used. I did not want to deal with that and therefore went for the Named Group list. NAME Thank you for the clarification. I think the paragraph right above is my answer.
NAME The checksum you propose is already computed with SHA256 -- why do you think this is something we ought to negotiate?
NAME The checksum is only 32-bits and it's for detecting errors, not preventing attacks. Named Group selection is a cryptographic property of the TLS handshake and it needs to be protected by the digest algorithm of the negotiated cipher-suite. It is either a full-length SHA-256 or SHA-384 at the moment, but could change in the future.

NAME The intent of this is to convey to the server that the client saw the right set of options before it chose its key. We don't need to consider any properties of NamedGroup selection with TLS to do so. And picking a suitable cryptographic hash function here is good enough. I really don't think we need agility here. Also, I wasn't suggesting the client echo the 32bit hash found in the record. It should send the full SHA256 hash.
Oh, also, what if we decide to add an extension that could influence how clients choose keys? Wouldn't we want downgrade protection for that, too? (A hash would work for that, whereas echoing the NamedGroup list would not.)
My understanding is as follows: In TLS 1.3, client offers the Named Groups and server selects one. It is protected by the digest algorithm of the negotiated cipher-suite. In TLS 1.3 + ESNI, server offers the Named Groups and client selects one. Therefore, I assume that which Named Groups we offered needs to be protected by the digest algorithm of the negotiated cipher-suite, or else we are changing the security properties of TLS 1.3. I agree. So to me the choice seems to be between a) choose a specific solution (i.e. send Named Group list) or b) add digest algorithm negotiation and send the hash value. I went for a, but I won't argue hardly against b.
Well, we could just use the hash associated with the cipher suite, no?
NAME That's possible, but then the server needs to offer a list of hash algorithms that it understands in ESNIKeys, or the client needs to send hash digests for all the cipher-suites it offers.
NAME I don't think that's necessary. We can assume SHA256 or some other MTI hash function for this to work. In the interest of future proofing, let's use a hash. I care much less about agility for the algorithm in question.
Assuming that is true, I am happy to go for using hash. The primary reason is that we can remove and use the hash value as the key to identify the ESNIKeys record that the client used.

OTOH, I am still not convinced why we do not need to negotiate the hash algorithm, especially when we are talking about the possibility of transferring additional extensions in the future that will be under the same protection. But I am not a crypto person and I could well be wrong, especially when it is midnight here. Anyways thank you very much for your quick response and for quickly merging the other PR. This PR is now in sync with master. Please feel free to use this as a basis of creating a PR that implements hash-based protection, if the current text looks worth enough.
Interesting point about using the hash as the label, though it might help to keep the label in the event that we come up with another way to publish ESNI keys. Regarding negotiation, I think SHA256 is sufficient for now and the future. Negotiating yet another algorithm seems like unnecessary complexity here. I can take a stab at writing that text, though I think what you have here is nearly there.
Sorry, I'm confused: the DNS record contains a list oc cipher suites, each of which is associated with a hash. The client chooses one to use to encrypt the SNI. Why can't that be the one you use to hash the ESNIKeys record
NAME That'd work fine, I just think it's unnecessary here.
So there's the argument that always using SHA-256 should be fine, and there's the argument that that the cipher-suites list can sent in ESNIKeys to solve the issue. I disagree to both. I have already explained why I think using SHA256 always is not fine. Let me explain why I think sending cipher-suites list in ESNIKeys is not fine. NAME Traditionally, we have allowed different cipher-suites to be selected depending on the value of SNI. This becomes an even more important property in the ESNI proposal, because we split the server into fronting server and hidden server. Cipher-suite selection is a negotiation between the client and the hidden server, whereas the ESNIKeys is a key-exchange between the client and the fronting server. Therefore, we cannot / should not use ESNIKeys as a conveyer of cipher-suite negotiation.

Considering that we have disagreement on the how we can uses hashes, maybe it is a good idea to merge this as-is to fix the security hole, and get back to do the improvements later on if necessary. I thought that we wanted to submit this soon.
NAME The ciphersuite that's sent by the client in the EncryptedSNI extension only applies to that extension, not to the rest of the handshake or data, right?
NAME Ah! That's correct. We now have two cipher-suites. I am fine with using the cipher-suite then.
Cool -- I'll propose some text. :-)
Applied NAME patch that changes the protection from using a list of Named Groups to using a digest specified by the cipher-suite.
recorddigest is sent together with encryptedsni in EncryptedSNI which is supposed to be the sensitive information you want to protect from a downgrade attack. So assuming there is a downgrade attack (I'm guessing you get bad key information from the DNS record) the client falls for it, sends the ESNI in a way that allows it to be compromised, and also sends the hash for the server to check whether an attack is going on. Fronting Server Behavior text says "If the EncryptedSNI.record_digest value does not match the cryptographic hash of the associated ENSIKeys structure, it MUST abort the connection" By the time the server realizes the hash is incorrect and drops the connection the ESNI has already been sent over the wire by the client. (And the adversary has intercepted it) So what's the benefit of dropping the connection at that point? As a side-note, can you explain the specific downgrade attacks you have in mind? My understanding is that the threat here is learning the value of SNI. If you are able to tamper with the DNS record then by definition you know (or can learn) the client's query for the domain name. Why would you need to attack the key material and try to decrypt the ESNI? And on the other hand, if you don't have r/w access to the DNS records, how can you carry out a downgrade attack?
That's a good question. The issue here is the Named Group that is being selected for ESNI protection key will also be used for negotiating the handshake. So the TLS handshake gets compromised (and that is what we are calling downgrade attack here).

Please refer to the first paragraph for URL
Can we treat the ESNI key derivation and TLS key derivation as separate processes? Seems like the reason we need to protect against downgrade attacks is because we are reusing the (potentially poisoned) ESNIKeys KeyShareEntry structure in the TLS key schedule. If we were to limit its use to just ESNI key derivation would we still need to protect against downgrade attacks?
My understanding is that that is possible, and then we do not need to send the digest value of the ESNIKeys structure assuming that we do not add extensions that need downgrade protection. However, that would require the client to send and generate one more ECDH key, as well as increasing the probability of HRR due to group mismatch. To put it another way, the approach used in this PR is on par with the amount of the data transmitted (the size of the digest and the ECDH key are roughly equal), has the possibility of cutting a round-trip, requires less CPU cost.
Doing so doesn't allow a server to gracefully generate new ESNI keys and retire old ones. We should mandate that clients always choose the first ESNI key for which they have group support. If we adopt , where servers lookup keys with a (label, group) -> key mapping, should we also require that each label have the same groups?
NAME what do you think?
I am not sure what you mean by "doing so", but the model I expect is something like following assuming that we adopt . the fronting server generates a new ESNIKeys and pushes it to DNS server every 1 hour TTL will be set to 5 minutes 1 the fronting server will accept the following keys to be used in the EncryptedSNI extension: the key for the current hour the key for the previous hour, for the first 10 minutes of every hour A ESNIKeys record can contain multiple keys belonging to different groups, but not belonging to the same group. So there is no ambiguity by specifying the key using (label, group). The server can add a new Named Group or retire one when it generates a new ESNIKeys. [1] updated to reflect the fact that we can only use a constant TTL in the standard way of DNS server synchronization (i.e. DNS zone transfer) (02:47UTC May 15 2018)
That seems perfectly fine.

But it does mean that there exists only one key per group at any given point in time. Is that a restriction we are comfortable with?
I don't think so. A DNS server can send different ESNIKeys at the same hour using short TTL (e.g. 60 seconds) in response to different queries (a.k.a. round-robin DNS).
Sorry, I mean within a single record.
I am comfortable with not having that. You can only have a handful of keys in ESNIKeys anyways. I'd assume that either one would be enough or we would need a way to distribute more than just a handful, for which we can use round-robin.
SGTM. We should add this rationale as text to the DNS record section.
I think that TTL suffices here.
NAME FWIW, is now in master. We can certainly discuss about the necessity, but you might want to see the debate on .
NAME I think your point in the PR is quite valid. However it's probably worth noting that having explicit notbefore / notafter that the client has to validate, is likely to make it very difficult, if not impossible, to deploy very-short-lived keys (say, in the order of a few hours), due to the fact that a significant portion of clients have shitty clocks (URL), which affects forward secrecy. Maybe it should be noted in the spec as an appendix or something.
Hmmm, thinking about this some more. In the "split brain" case NAME described in , the not_after is used to avoid having clients send an ESNI encrypted with an old key (as fetched from an out-of-date DNS server), to a TLS server that might have already switched to a new key (thus causing the connection to break). So i'm guessing what happens is that the client would fallback to plaintext SNI if the key it fetched from DNS is expired. Though it seems to me this fallback could be dangerous for the client since they would end up exposing the SNI, but I can't come up with a practical way to exploit this (could be dependent on the specific deployment I guess). Whether this is something worth fixing is a different question, but an alternative to fix the split brain problem would be for the TLS server to somehow make sure the new key has been received by the DNS server, before discarding the old one.

Did I understand the problem you described in correctly NAME ?
NAME Thank you for your comments. I think you are correct in stating that the ESNIKeys cannot have a very short lifetime. However, I would like to argue that the fact does not prevent server operators from rotating the keys frequently. Server operators can rotate the keys at the speed they like, as well as retiring them from the servers in any way. I agree with what happens. In my view, ESNI is opportunistic encryption in sense that the unencrypted hostname is used when the ESNI resource record fails to reach the client (including the case of a middlebox dropping it intentionally). I would argue that using SNI in case of receiving an expired ESNI record is one of the failure modes in distributing the correct resource record. And therefore, I do not think that it is a security concern. That is correct. However, I fear that having such a requirement hinders us from using existing DNS deployments, because DNS authoritative servers are designed to favor availability over consistency, whereas in our case without , we need consistency over availability.
I didn't realize that was possible (that is, I kind of assumed servers would have to keep the key around until the notafter expired, but it's not actually written anywhere). In that case fine, though it might be worth clarifying. The other concern I had was risking of turning this into another OCSP (where network failure and attack basically look the same so clients end up ignoring both), but I guess that with DoH, DPRIVE and DNSSEC an attack here is much harder to pull off. So all in all I don't have a strong opinion on notbefore / not_after anymore.
I think this is fine, NAME please re-open if not.
The text in section 4, Publishing the SNI Encryption Key, specifies that if a client queries URL, the ESNI TXT name is URL That's fine, but suppose that the client queries "URL". The client needs to learn that "URL" is fronted by "URL" -- per PR , the client will do that by obtaining an ALT-SVC record. Should the client get the ESNI key from "URL", or from "URL" ?
NAME I think this is you.
covers this. Mostly.

It might pay to observe that the client does not look up and DNS records for the fronting service, it is just the case that all servers in the anonymity set point to the same IP address. I made similar comments on .
I agree with NAME here. I didn't see the client querying URL
That means all servers that share the same IP need to publish the same key. Are we confident that there is no synch issue during key rollover?
Servers can publish multiple ESNI keys with different labels in a record. That way they can retire one key and start using another as needed. Though it might be good (necessary?) to require clients to use one of those keys.
While that is possible, I am not sure if it would be the approach that would generally be used. The issue is identical to the encryption key of the session tickets. In case of session tickets, we encrypt the ticket using one key, and send the identifier of the key along with the encrypted ticket. When we rollover the keys, we first deploy a new key to the server as well as keeping the old one alive for decryption only. And we retire the old key after certain amount of period that we become sure that it is no longer used. I'd assume that we can and will use the same approach for ESNIKeys.
PS. therefore, it is possible to have just one label for the set of keys rather than having labels for each of the keys (assuming that we can remove the "choose one at random" rule). In that approach, a server will identify the key being used by looking at the tuple. The benefit will be that you will save the space of N-1 labels assuming that you are sending N key shares. Identification of the key on the server-side might seem complicated, but I actually think that it simplifies things, because a server operator is anyways expected to deploy the entire set of keys found in ESNIKeys at the same moment.
To me it seems that this is necessary to prevent downgrade attack. Hence .
How do we prevent labels from becoming identifiers?
Labels as identifiers seem problematic in both approaches -- current and .
Would you mind elaborating why label becomes an identifier? We state in the document that the probability of identifying the hidden server is 1/N where N is the number of the hosts belonging to the same set of hidden servers.

A label represents that entire set. So I do not think it is an issue.
Now that is merged, which removes the label altogether, and we've added text describing the anonymity set for hidden servers, I think this issue is covered. NAME do you agree?
The label issue is definitely covered.
This LGTM. NAME PTAL when you have time.