Original text:
0-RTT, Handshake, 1-RTT) makes it more likely the receiver will be able to process all the packets in a single pass.  A packet with a short header does not include a length, so it can only be the last <del> packet included in a UDP datagram. </del>  Senders MUST NOT coalesce QUIC packets for different connections into a single UDP datagram.  Receivers SHOULD ignore any subsequent

Reviewer comments:
This is a SHOULD per agreement in London because a) we can't decide why you might not, but b) we can't mandate any particular action on the part of a receiver.
The spec says It makes sense to continue processing if the keys are not available. However, if the keys are available, and decryption fails, this means that the peer sent an invalid packet (or that we're dealing with an on-path attacker, which we can't do anything against anyway). It seems that it would be equally valid to stop processing the datagram in that case. Since it's possible to coalesce about 70 QUIC packets into a single 1200 byte datagram, an attacker could cause a peer to attempt 70 AEAD operations with a single datagram.
Doesn't decryption failure happen also when the packet gets corrupt while being transmitted? Consider a coalesced packet sent by client that contains Initial (for ACKing ServerHello) and a Handshake packet. I'd assume that we'd prefer processing Handshake packet even if Initial was corrupt. Does that matter, considering the fact that an attacker can force AEAD operation for every 21 bytes by using a short header packet (1 type + 4 byte space + 16 byte AEAD tag)?
I assumed those to be very rare, so treating the whole datagram as corrupted might be a valid response. I don't have any numbers about that though. Maybe we can ask NAME if he can share some measurements? The assumption here is that sending a lot of small packet is more expensive than sending one large packet of the same payload size. First, there's UDP and IP overhead that we're not counting here, and second (and probably more importantly), you only need a single syscall. Looking at the numbers, I think I messed up my calculation of how many coalesced packets fit into a single datagram, since I failed to account for the AEAD tag. This brings the number QUIC packets per datagram down to about 35.
That could be true, though I am still not sure if we need to talk specifically about the attack. For example, an attacker can also send a packet that successfully decrypts, that contains many tiny CRYPTO frames (4-bytes each). I'd assume that processing of that would be more costly than AEAD-decrypting every 35 bytes. Having said that, the state of the endpoints are guaranteed to progress even if the following QUIC packets of a coalesced datagram are discarded (due to corruption, etc.). Therefore, I won't disagree to relaxing the requirement from MUST to SHOULD.
I was not suggesting to add language about this attack. My suggestion was to stop processing a datagram once unprotecting one of the QUIC packets fails, although you have the keys available for that encryption level. I don't understand this. Why is processing CRYPTO frames expensive? In my implementation, assembling stream (and crypto stream) data is not an expensive operation.
Because for every 4-byte CRYPTO frame (containing 1 octet of data), you need to decode 2 varints (length and offset fields), check if it's newly received data, and then write it to the buffer. If the data is new, there would be a call to TLS stack that causes the TLS stack to see if the entire handshake message is available, which would turn out not to be true. TLS stack will then buffer the data. After that, the transport layer will remove the newly received data from the transport-level buffer. My assumption is that doing this amount of processing is more heavyweight than running two AES operations and one GCM operation per every 35 bytes. Based on that, I do not think we need a statement that specifies how to handle these tiny CRYPTO frames, or similar types of attacks (e.g., a packet payload containing many ACK frames). The same goes for not specifying how to handle tiny QUIC packets.
From my experience, decryption failure is extraordinarily rare. Sufficiently so it's not worth worrying about or optimizing for cases when one QUIC packet decrypts and another does not.
We talked in Kista about using just such coalesced packets for PMTU probing. I.e. coalescing a short long-header packet that will not decrypt but contains Server's Connection ID with a short-header packet that has the actual payload. If the UDP datagram does get delivered, the long-header packet will be discarded and the short-header packet -- ACKed. If this UDP datagram runs into an MTU problem, the response ICMP will contain Server Connection ID, and it will be routed to the correct backend.
Discussed in Tokyo; NAME will make a PR to prohibit senders from coalescing packets of different encryption levels and to permit (but not require) receivers to enforce this constraint.
Fine, as long as a violation will not DoS the connection, since a middle box can add random data to a UDP datagram.
That is a good point. I think that the idea would be that a connection error would only be permissible if there were multiple packets of the same encryption level that could be successfully authenticated. Endpoints would otherwise be permitted to drop any extra packets. Note that you can just drop extra packets without checking if they decrypt.
What's going to prevent a man-on-a-side from recording packets with different encryption levels and then sending a coalesced packet? Both will decrypt. You do not want to kill a connection that way.
NAME I believe you meant prohibit senders from coalescing packets of the same(not different) encryption level, correct?
Yes, but NAME point made me reconsider. I think that the rule would need to be: a) senders MUST NOT coalesce packets of the same encryption level (there is no need to do so, so this is basically a no-op requirement); b) receivers SHOULD/MAY/MUST? discard packets if a packet of the same encryption level was received in the same datagram. Unfortunately we can't do any more than this because an attacker has complete control over coalescing. I realize now that this dovetails nicely with the requirement we already added here to require exactly one Retry or Version Negotiation in response to a packet.
I think b being MAY is what we agreed at the Interim. While I understand that some are concerned about this being used as an attack vector, there are people who do not care about this (including me), because there are more attractive ways to enforce the server to do more calculation. MAY is a good compromise.
I'm not (that) concerned about calculation. I'm concerned about disconnects. Either due to changes by an attacker or due to a random error in the packet.
I am a bit worried when I hear suggestions of disabling coalescing with different encryption level. Initial(CH) + 0RTT is fairly common, and useful. So is Initial(SH) + handshake, Initial(ACK) + Handshake(ACK), or Handshake(Finished) + 1RTT. Yes, I can see attacks with "Christmas tree" packets containing too many coalesced segments. But mitigation is obvious -- implementations are free to drop packets for whatever reason. In the rare cases of false positive, when the funny coalesced packet is not in fact an attack, the peer will just resend after at time out.
What if it resends to the same "Christmas tree", for example for probing to get source CID?
I don't think this issue warrants changing the protocol specification. You may want to document the potential attack in the security section, and suggest mitigations. But IMHO this is one of the many ways in which a client can be nasty with a server, and vice versa. There are many other examples, such as acking packets one by one, sending multiple 1 byte stream frames, randomly alternating between multiple streams, etc. The generic pattern is to find some operation that is expensive for a particular server implementation, and then abuse it. You can't possibly document all the ways of doing that.
My point was that it is hard to define what is a valid abuse filter without risking not being able to connect with client which sends well intended but ill conceived "Christmass tree" packets. While you can always get into trouble, I think there are some difficulties here.
NAME IIRC, one of the concerns regarding this approach was that we might see a deadlock if the receiver has dropped the keys necessary for processing the the first packet within a datagram, and decides to drop the entire datagram due to not being able to process the first packet. Then, the sender might simply retransmit the coalesced datagram that starts from the same packet type as the previous datagram, and that datagram again gets dropped by the receiver, and ... Therefore, it has been my view that it is beneficial to specify the minimum requirements that guarantees the connection to make progress. IIRC, my preference was to define a minimum number of packets within a single datagram that a receiver should process. But I am fine with what we have in the text right now, because it does not make the implementations complicated; a sensible sender would never send a datagram containing more than one packet at the same encryption level, detecting them on the receiver-side is an implementation choice.
NAME lands, we should have very few cases where "the receiver has dropped the keys necessary for processing the first packet within a datagram". I imagine we could see some corner cases, but they most likely will be transient states, not leading to deadlocks.
NAME I agree that it's likely that we would not have deadlocks. But I think it's beneficial to specify how a server should not drop coalesced packets, because there would be performance issues if there is a disagreement about what is possible between the endpoints (one of the concerns being PMTU probes using coalesced packets (URL)).
Would a note suffice? Something like "an endpoint that coalesces more than a few packets risks having the peer not process all packets." I don't think we need stronger or RFC 2119 language here, just a note for sensible implementation is adequate I think.
NAME I personally think that that might be sufficient, though it is my understanding is that we already have a rough consensus on how to proceed; see URL Maybe I am incorrect in understanding the status, but based on this understanding, what I am wondering is if there's any new information that makes the WG reconsider.
Thanks NAME for reminding me that we discussed this in Tokyo and arrived at an agreement (it's not consensus per my read, it was agreement among the participants in the "room"). I'll gently poke NAME to write the PR.
I am a bit skeptical of the idea of using coalesced packets for PMTU discovery. If I receive a handshake packet after the handshake is finished, I will most probably ignore it. I don't have the keys anymore, I cannot remove header protection or packet protection. Packet will be treated as a decryption error, and whatever is coalesced with it will most likely be ignored. I understand the goal of coalescing is to find a CID in the ICMP payload, but that seems somewhat brittle. Parsing the ICMP based on the 5 tuple would be much more robust, and does not require playing games with packet coalescing. I may be a bit biased because I would rather just completely ignore all ICMP packets, and rely only on end-to-end ACKs. It removes the need to find a portable way to obtain ICMP messages through the socket API, and it also prevents attacks by ICMP spoofing.
NAME I would point out that the draft says don't. Quoting from : if decryption fails (because the keys are not available or any other reason), the receiver MAY either discard or buffer the packet for later processing and MUST attempt to process the remaining packets. I agree that endpoints should rely on the 5 tuple whenever possible. However, if a stateless load balancer is being used, the CID becomes the only identifier for routing packets. That's why we have this mechanism (and the text quoted above).
I really wish QUIC did not use coalescing for any kind of semantics and only to avoid roundtrips.
Coalescing for other-than-PMTU-discovery is to reduce the number of small datagrams. That's useful for reducing network bandwidth and CPU cost, especially for short connections. So any kind of analysis for possible attacks should weigh "Christmas Tree" coalesced packets vs. receiving each of the coalesced packets separately. Coalescing makes sending such packets cheaper for the sender but it also makes it equally cheaper for the server to receive. My view is that packet coalescing is "neutral" in its effect on attack resilience of the protocol. (One can actually say it is slightly beneficial for network links, since they reduce the bandwidth required to send the same packets separately.)
Yeah, meant reduce UDP packet count, not roundtrip.
Discussed in London; NAME will (still) make a PR to prohibit senders from coalescing packets of different encryption levels and to permit (but not require) receivers to enforce this constraint.
Clarified in London: Sender of a packet cannot coalesce multiple packets of the same type. If you receive multiple packets of the same type in the same datagram, you're not obligated to process all of them.
LGTM

Revised text:
0-RTT, Handshake, 1-RTT) makes it more likely the receiver will be able to process all the packets in a single pass.  A packet with a short header does not include a length, so it can only be the last  <ins> packet included in a UDP datagram.  An endpoint SHOULD NOT coalesce multiple packets at the same encryption level. </ins> Senders MUST NOT coalesce QUIC packets for different connections into a single UDP datagram.  Receivers SHOULD ignore any subsequent