Code change:
This is implied in some spots, but never stated with normative language.
Under "Handshakes and New Paths", it says "Data at Initial encryption MUST be retransmitted before Handshake data and data at Handshake encryption MUST be retransmitted before any ApplicationData". I moved that down to this section, since I think it fits slightly better here.
In terms of packet size, this is similar to what the old Handshake text recommended and what PTO should have said all along, as it aligns well with TCP's TLP and RTO behavior.
The previous text read like it only described retransmission of lost data, and this now reads like it only describes retransmission in PTOs. Are both intended? Is there a restriction anywhere that retransmitted data must not be sent at a higher encryption level than it was originally sent, or similar?
All of this text is in the PTO section, so only applies to PTO. I think transport is usually the best place to talk about where data is retransmitted, but it might be worth an editorial note that data is never retransmitted at a different packet number space than the original one.
Thanks for opening other issues, NAME I'm going to merge this, since I think it's a net betterment over what we have. We can continue the other discussions on the other issues/PRs.
At the New York QUIC interim, we dealt with the problem of the server being blocked by the amplification factor by making the client arm the PTO until it knew the server had validated the path, even if it had nothing to send. From the recovery : When reviewing URL, I realized that if we allowed the server to send a PING-only packet when the PTO fired and the amplification factor had been reached, then that would elicit a padded Initial ACK frame in response, which also fixes the issue. NAME though that was a simpler solution than the current approach and this is a bit of an edge case, so simple is probably good. The original issue was
In a strict sense, this means that a client can use N bytes to trigger the server to send 3N + epsilon bytes, but this fits in nicely with Probes generally not being subject to sending limits. It is somewhat gross but more elegant that a client timer with nothing in flight.
I think this might be concerning, though I could well be wrong. Consider the case of an attacker acting as a client, using the IP address of a victim as the source address. The attacker can spoof an Initial packet that carries a Client Hello and another Initial packet that carries an ACK for the server's Initial packet that carried the Server Hello. An attacker can build the correct second packet assuming that the server uses the DCID of the client's Initial packet as it's CID (or the CID is zero length), and that the server's first packet number can be guessed. If that ACK contains a delay that indicate to the server that the RTT is very small (e.g., 1ms), then the PTO could be as small as that value. Then, if the server's connection timeout is say 1000ms, the server might send 1000 packets to the victim before dropping the connection. Am I missing something?
PTO always uses exponential backoff, so if the first PTO was 1ms, that'd be ~10 PTOs, not 1000. In terms of total bytes sent, 10 packets with a 1 byte payload are still smaller than a single full-sized packet.
Isn't this concern why we designed the Retry mechanism in the first place? If we are concerned about address spoofing attacks, the simple solution might be to generalize retry. As in, "if the server can predict that its first flight will contain more than 3 times the amount of data received with the client's Initial packets, then it SHOULD verify the source address using Retry."
My concern is that the current anti-deadlock mechanism is fairly awkward and rarely needed, which makes me concerned it will be incorrectly implemented. By sending a PING only packet, the server is essentially informing the client it has something to send, but can't send it due to the amplification limit. The deadlock happens when the client's Initial is acknowledged, but the client doesn't receive the server's Initial, so it can't decrypt any handshake packets. In this state, the client has nothing to send. It can happen even if the server's first flight fits into 3 packets. But it's expected that typically servers will bundle their ACK of the client's Initial with the server Initial, and this will be rare in practice.
NAME Fair. Due to the risk of amplification attacks, the server is probably not willing to retransmit Initial packets on timer. In that case, the current spec is a bit too lose. It should say that "If the sender is not willing to retransmit Initial packets due to protection against amplification attacks, it MUST NOT acknowledge the client's initial packet." So basically, either the server verifies the client's address with the Retry/Token mechanism, or it MUST NOT acknowledge client initial packets before continuity can be verified by obtaining an ACK from the client.
That's an interesting suggestion, but I think it require some non-trivial changes to the server's logic in addition to not sending ACKs? Something similar to what's described in . It may be worth opening up an issue for allowing the server to not arm a timer until the path has been validated, because I think that's a wider issue than just this PR?
I agree that having the server repeat a PING on timer may also work, and that the ACK of the randomly chosen Server's CID would make a reasonable address ownership test. ACK of the PING would naturally unlock any transmission that is pending address ownership. Kazuho objects that this his hackable if the chosen Server's CID is guessable. We don't have a "non-guessable" requirement in the spec. In that case, the natural fix is the challenge/response mechanism.
NAME Ah. Thank you for pointing that out. Though it seems that the amount of the data that the server can send could go like 4x the amount of data that the client has sent rather than the current 3x. To paraphrase, I think what we might be trading here is the initial amount of data the server can send vs. the capability of sending PINGs. My view is that the former is a scarce resource, and that I'd prefer having the more space in the former even if requires us to have some complexity.
NAME An Initial ping is maybe 45 bytes. So 10 pings makes the amplification ~3.4. I was under the impression that the 3 multiplier was semi-arbitrary, so I'm unconcerned about a fractional increase.
I have a couple of thoughts. First, this change is not necessarily an overall simplification. It makes things slightly simpler at the client, but it adds about as much code to the server. Additional code has to be added at the server to the PTO response to only send a PING and not a retransmission when the path is not yet validated. Second, I think it does open up an interesting attack vector that is currently absent. NAME notes that there is amplification, but I don't think byte amplification is the interesting one here, since the total load is going to be < 1 MTU. The interesting amplification here is packet amplification, which is a useful attack since UDP and packet processing costs at endpoints / clients is a relatively high cost. And you can get a fair number of packets back from the server for spoofing a couple of packets, in spite of the exponential backoff. I'll grant that the attack is a bit of a stretch, and does require that the attacker guess the server's CID and packet number. Though, as NAME pointed out to me offline, we allow the server to use a zero-length CID. I don't think this change is necessary. And on balance, I don't think this change is a good one.
I do like the idea of sending a PING frame from the server as a signal to the client that it is blocked. I think we can suggest that as an optimization that a clever server might implement under the 3x limit, or when it reaches the 3x limit.
NAME While endpoints can do that, I am not sure if it is a good idea to endorse such a behavior in the specification. The reason we have so far been fine with having the 3x limit being applied until the Handshake packets are exchanged is because we assume that the size of ServerHello would not be that large compared to the size of the ClientHello, and therefore that the chance of hitting 3x limit before path validation is minimal. It's a hack. If we are to address the corner cases that the server hits after sending 3x amount of data, then I think we should fix the hack than putting another hack (i.e. the PING that signals 3x limit) above the existing hack. I mean, if we think that defining a way that allows the server to unblock the 3x limit is important, we should be considering allowing the use of PATHCHALLENGE / PATHRESPONSE frames in Initial packets.
I believe this is simpler to implement assuming one is implementing both a client and server. I already have special case code to send a PING when there are bytes in flight but nothing to send, so this is very similar. I agree it's not necessary, but I find the simplification appealing, and when we first discussed this problem, no one suggested this solution, so I thought it was worth writing up. If we keep the existing text that the client has to keep sending, I'm not sure doing this on the server side is a valuable optimization, so I'd prefer to stick with one mechanism or the other. To me, the main negative is the server sending extra datagrams. And maybe that's enough to stick with what we have.
When a PTO fires, the endpoint is supposed to try and retransmit something if there is data to retransmit. In this case, you'd need to special-case it so that the server doesn't in fact retransmit but sends a PING instead. That's what I meant by adding more code.
It also SHOULD send a PING only packet if there are bytes in flight but nothing to send.
Not new data, any data. This was deliberate. See Section 5.3.1: "When the PTO timer expires, and there is new or previously sent unacknowledged data, it MUST be sent." The point of this text was to send data if available, new or previously sent. In the case that we are discussing the server is supposed to send a PING, but it might have data to send.
Discussed in ZRH. NAME to decide if he wants to pursue this, potentially discuss with NAME and/or NAME
I realized there is one major benefit of keeping the existing design, which is the client is guaranteed to have at least one RTT measurement in this circumstance, whereas the server may not. For this reason, I'm inclined to close this with no action unless others feel strongly otherwise.
NAME came up with the same point simultaneously, so let's close this with no action.
There are multiple mentions of spurious retransmission in the recovery draft, but no text on what detecting this may require from an implementation. Maybe there should be? "Spuriously declaring packets as lost leads to unnecessary retransmissions and may result in degraded performance due to the actions of the congestion controller upon detecting loss. Implementations that detect spurious retransmissions and increase the reordering threshold in packets or time MAY choose to start with smaller initial reordering thresholds to minimize recovery latency." From comments on
Some discussion of implementation strategies here would absolutely be helpful; it doesn't obviously fall out of Quinn's current architecture.
NAME and NAME indicated my effort at adding more text wasn't that useful, so I'm closing this.
As of URL, if the client's Initial is lost, there is no required behavior that leads to it being retransmitted. Because the client has received no ACKs, no is set, and no loss is detected, leading to the client only sending probe packets, which are not required to bear any particular data ("A probe packet MAY carry retransmitted unacknowledged data") so long as they are ACK-eliciting. The server, having no connection state, cannot do anything with these packets. Quinn probes are currently a single PING frame (plus padding, when appropriate) for simplicity, so implementing the new logic immediately broke our tests. If recovery logic now requires probe packets to bear specific data to function correctly in certain circumstances, this should be specified explicitly.
I also hit this issue too. After several hours tries and error, I came to the conclusion that implementation should mark the in-flight Initial packet lost in this case, and resend it. Draft-22 has explicit statement that implementation should not mark in-flight crypto packet lost, but draft-23 does not have it anymore. BTW, draft-23 does not allow PING in Initial or Handshake packet.
Why not retransmit the data without declaring the in-flight packet lost?
I think it is rather implementation detail to internally mark in-flight packet lost but keep it in order to catch late ACK.
There's a related problem where if the server's first flight goes missing, the only response the new logic has is to send a probe at unspecified, i.e. highest known, encryption level. The server will generally have 1-RTT keys at this point, so it sends a 1-RTT probe, which the client, not having received anything prior, cannot decrypt and drops. As above, this was previously handled by explicitly tracking whether any data was in-flight, but that got removed.
The packets should not be marked as lost, but data needs to be retransmitted before sending a PING. I sent out a PR , feel free to make further suggestions there.
I'm marking this design because it adds some normative language, though I'll note that it doesn't change what I originally intended the text to say.

Related discussion:
text: ack-eliciting packets, to avoid an expensive consecutive PTO expiration due to a single lost datagram. <del> It is possible that the sender has no new or previously-sent data to send.  As an example, consider the following sequence of events: new </del> <ins> When the PTO timer expires, and there is new or previously sent unacknowledged data, it MUST be sent.  Data that was previously sent with Initial encryption MUST be sent before Handshake data and data previously sent at Handshake encryption MUST be sent before any ApplicationData data. It is possible the sender has no new or previously-sent data to send. As an example, consider the following sequence of events: new </ins> application data is sent in a STREAM frame, deemed lost, then retransmitted in a new packet, and then the original transmission is <del> acknowledged.  In the absence of any new application data, a PTO timer expiration now would find the sender with no new or previously- sent data to send. When there is no data to send, the sender SHOULD send a PING or other ack-eliciting frame in a single packet, re-arming the PTO timer. </del> <ins> acknowledged.  When there is no data to send, the sender SHOULD send a PING or other ack-eliciting frame in a single packet, re-arming the PTO timer. </ins> Alternatively, instead of sending an ack-eliciting packet, the sender MAY mark any packets still in flight as lost.  Doing so avoids