Draft: draft-ietf-quic-retry-offload

Commentary: A different fix for . Avoids any client involvement , and covers all but extreme corner cases . 
 There are cases where an offload might send Retry on the second initial but not the first : it turns on between the two packets it is in a stochastic mode where it does n't always send Retry There is one failure mode and one degraded performance issue : 1 ) If the server sends HRR , if the client 's response triggers Retry it is likely to blackhole the connection . 2 ) Under certain conditions , dropping the client 's second Initial packet will cause either a client or server PTO . ( Specifically , there will be no penalty if the client 's 2nd flight has a datagram that ( a ) does not contain an initial , and ( b ) contains a HS ACK . ) We ought to write this down in the draft . For , it might be good practice for servers to send CONNECTION_CLOSE instead of HRR under two conditions : 1 ) There is no Retry token in the Initial , and 2 ) Other recent initials contained Retry tokens . 
 The amplification limit is a problem here . If the client 's second flight is one packet , including an Initial , which keeps getting dropped , and the server is at its amplification limit , there is no way for the connection to progress . 
 As I mentioned in OR , I think we need to handle this via an extension to the QUIC protocol itself . I recommend we define an extension that does the following : The server includes a new transport parameter that contains the value of the initial token that clients should use for any subsequent initial packets they send . If the client understands this extension / TP , it acts accordingly . If it does not , then it may hit the issues you describe in this PR and eventually time out . 
 So the problem with using a TP is that it is n't delivered until the Encrypted Extensions messages , which is far too late for most of the failure modes here . However , I 've thought more carefully about this and have realized some things . statefulness does n't completely solve it , as you could have a shared - state retry where an anti - DDoS service turns on and suddenly has all packets route through it . If an offload is going to randomly send Retry while being stateless , it really ought to be pseudorandom based on the address / port or whatever . Therefore , the two Initial packets will get the same treatment in the usual case . That reduces to the problem to " service turns on mid - handshake " . 1 is where Retry offload breaks the Initial + Handshake flight . This is n't actually a black hole . The client will retransmit handshake packets and , per RFC9001 4.9.1 , MUST NOT send Initial packets once it 's sent a handshake . So there might be a PTO , which is not ideal , but not catastrophic . In the normal case , there are multiple server handshake packets , and if the client is following spec and not doing ack delay in the handshake , it is unlikely the entire Handshake ACK will be coalesced with Initial . So let 's forget about this " problem " , and we do n't need to do any weird packet truncation as in PR . 2 is  or Multi - packet SHLO : this is the harder case as Initial ACKs are critical to progress .  is an edge case but post - quantum we may need multi - packet SHLO , and it would be good for this design to be robust to that . I do n't see a realistic fix here right now but I 'll think about it some more . 
 Crap . You 're obviously totally right . I do n't know how I forgot . I still thing the right way to handle this is to somehow give the client the along with the server 's initial payload . I might propose a slight modification of my original plan : The client includes a new transport parameter ( with no payload ) that advertises support for this new extension . If the server supports this extension , it includes a frame in all its Initial packets , and then also includes the transport parameter that also contains the value of the initial token ( so that it can be securely validated ) . Clients should use the frame value for any subsequent initial packets they send . 
 That will work , but I am very skeptical of client support , particularly in  . I just reworked things completely with . Reasoning about the Handshake is hard , but I think this works pretty well unless the handshake is also suffering losses for reasons other than the Offload dropping stuff . Basically , there 's a " transition mode " when going active where you send Retry and forward the packet to the server anyway . The handshake logic basically works out , which is surprising until you think it through . 
 I 'm very hesitant to push a solution that does n't work in all scenarios . From some tests I 've done in recent past , HRR is a lot more common in some scenarios . Also , post - quantum crypto is coming . 
 But this works ! 
 LGTM
Retry Offloads SHOULD treat Initial packets from the same connection
with a uniform policy.  Initial packets of the first and second
client flight can be difficult to distinguish without expensive
decryption of the contents, which is unsuitable under the conditions
of a DDoS attack.  If the first packet of a connection is admitted
without Retry, but the second triggers a Retry, that Retry packet
will be ignored and the loss of an Initial coalesced with other
packets can impair performance.  In some situations, the client does
not yet have handshake keys, and dropping further client Initial
packets creates a deadlock where the connection cannot progress.
The simplest means to ensure this is to require, when active, a Retry
Token for all incoming Initial packets, and send a Retry packet
otherwise.  If the Retry Offload is to be more selective, one
technique keeps state on which address/port 4-tuples have been
admitted.  Another would be to apply a secure hash to the source IP
address, port, and connection ID to deterministically compute whether
the Initial requires a Retry Token or not.  These source values
remain consistent over the handshake.
However, even with these techniques there is a potential problem when
a Retry Offload switches from inactive to active mode.  The Retry
Offload could admit the first packet while in inactive mode, and then
drop subsequent Initials in active mode.
If the Retry Offload is always on-path, it MAY keep state on incoming
connections while in inactive mode to avoid this problem.  If it
cannot or will not keep such state, it SHOULD implement "transition
mode" for an interval chosen to include the likely Initial packet
exchange of most clients (200ms is a sensible default).
In transition mode, Retry Offloads process Initial packets with Retry
tokens as in active mode.  When the Retry Offload receives an Initial
packet with no token, it issues a Retry AND forwards the packet to
the server.  If the client has already received a packet from the
server, it will ignore the Retry and the connection will progress
normally.  If not, the client will reconnect based on the Retry, the
server's response to the first initial will be discarded, and the
connection will progress normally based on the client's second
Initial.  mid-handshake explores the various possible packet
sequences in transition mode.
Note that transition mode provides no actual DDoS relief to the
server, so its duration should be as short as possible.  The Retry
Offload can choose not to implement transition mode and cause some
client connections to fail.
Servers operating behind a Retry Offload SHOULD implement a mechanism
that operates whenever a client Initial arrives with a valid Retry
token.  If there is another connection with identical client
Connection ID, IP, and Port, but with an unvalidated address, that
connection is immediately and silently terminated.  This mechanism
eliminates incorrect connection state that is an artifact of
transition mode, as explained in mid-handshake.