Draft: draft-ietf-quic-load-balancers

Commentary: This PR specifies how the retry token should be encoded using AEAD . This replaces the current method that uses AES ECB , and has potential vulnerabilities since encrypted addresses and encrypted ODCID might be encrypted independently . There are differences between the two solutions : AEAD requires key and IV , not just key . The IV will have to be provisioned with the key . AEAD requires a nonce . The proposed format defines a 64 bit " token sequence number " . We may want to discuss this further , and specify how exactly this is generated . The client address is not encrypted as part of the token , but is verified through AEAD The AEAD protected token may end up 8 bytes longer than the ECB protected token : remove IP address ( -16 ) , add a nonce ( +8 ) , add the AEAD checksum ( +16 ) . On the other hand , the ECB must be padded to the nearest 16 bytes , while AEAD does not require that . Probably not very important . I assume we are using AES128 - GCM , but I suppose the specific algorithm could be provisioned with the key . 
 Waiting for the resolution of the Token Sequence Number issue before updating the PR . I am tempted to just make that field 96 bits , and specify randomness . 
 I think we have five options : ( 1 ) A 96 - bit random number for the sequence number , and just assume there are no collisions ( you 've computed the very low probability ) -- this is wire inefficient but very easy ( 2 ) For the shared - state case , the service simply forwards packets with server - generated tokens rather than trying to validate them . Therefore this format is only generated by the retry service and it can use all the sequence numbers without fear of collision . -- this is very easy and wire - efficient , but slightly reduces the usefulness of the Retry Service . Then there are three variations of " give each server some of the sequence number space " : ( 3a ) Just put the whole ( up to 18B server ID ) in the token : easy , but makes big tokens . The retry service must be configured with at least 1 server ID it can use for its own tokens . ( 3b ) Force the Retry Service to have the same QUIC - LB decoder as the load balancer , so it can extract the server ID from the CID and use that as part of the IV : most wire - efficient , but expensive for the Retry service . ( 3c ) Assign each server a shorter ID for Retry purposes , and put in the token : medium token size , but involves the most configuration . I weakly prefer ( 2 ) . if you have a different preference , I 'm willing to go along with it ; or we could take it to the list . 
 I just did a PR in Picotls to make sure that the API can let us set a 96 bit " sequence number " . Turns out that this is also required for handling multipath . Putting the whole server ID in the token really means " putting the whole server ID in the sequence number . If you do not do that , you risk AEAD collisions , and that 's not too good . Putting the server ID in the token but not in the sequence number does not rid you of AEAD collisions , so 3a is not really an option . 3c can work if the server ID is < 32 bit . You would construct the 96 bit " sequence number " as 32 bits of " short " server ID and 64 bit of per server sequence number . I am not sure that I understand your option 2 . There is no big issue for " retry tokens " sent through " retry " packets , since the protocol also indicates the CID to be used by the client . The LB can route based on the CID . There is only an issue for " new tokens " used in conjunction with a random IDCID . 
 A fuller explanation of option 2 : This design is for the shared - state Retry token , which is for cases where the Retry Service is not guaranteed to be on the path , and therefore both the server and service have to be able to generate ( and validate ) their own tokens . The current design has both server and service generate tokens according to the same format . Therefore , if a service generates a token , the server can validate it ; if the server generates a token , the service can validate it . Obviously , this saves the server from processing invalid tokens if the service can handle it . The problem , of course , is if we increment sequence numbers there will be collisions between server- and service - generated sequence numbers . Option 2 would make the following changes : Services produce a fixed - length token , in accordance with the standard format Servers MUST NOT generate tokens of this length , but need not follow any format Services MUST NOT process or filter Initial Packets with tokens that are not of the fixed length . This entirely solves the sequence number problem , but it also means the server has to do a bit more work . Thus the Retry Service is offloading some of the  generation , but not any of the token processing . Writing it carefully out like this , I do n't like Option 2 as much anymore . 
 " Putting the server ID in the token but not in the sequence number does not rid you of AEAD collisions , so 3a is not really an option " to be clear , it would be in the sequence number . The point of putting it in the token is to allow the decoder to reconstruct the IV . I imagine that 32 bits would be plenty of server IDs if it does n't need to be sparsely populated . 
 To sum up , I think we only have the following 2 options : ( 1 ) A 96 - bit random number for the sequence number , and just assume there are no collisions ( you 've computed the very low probability ) -- this is wire inefficient but very easy ( 3c ) Assign each server a shorter ID for Retry purposes , and put in the token : medium token size , but involves the most configuration . For AEAD , compose the 96 bit sequence as " ( up to ) 32 bit server ID " + " ( up to ) 64 bit server - chosen token ID " . The advantage of ( 3c ) over ( 1 ) it potentially reduced transmission overhead . For example , the server ID might well be just 16 bits , the token ID might be just 32 bits . If we have a server ID , the LB can use it to direct the incoming Initial to the server that allocated the token , which may or may not be useful . The drawback of ( 3c ) is more configuration , and also exposing the server ID in cleartext . If server ID privacy is an issue , we need something like header protection as part of the token protection . In both cases , we need some kind of key phase bit in the token to understand which key to use . 
 I have tried to elaborate the two options for either random generation or server ID . 
 BTW I am happy to commit if you resolve the nits and we can have a longer discussion on your DISCUSS points . 
 I think this is ready now , but you may want to review . 
 I think the last commit fixes the last comments . 
 Section 5.3 specifies a Shared - State Retry Service describes a token format in which the token include the ODCID , a client IP encoded on 128 bits , and a 20 octets data - time , plus additional data . The token is encrypted using AES ECB . This seems sub - optimal : 1 ) Using AES GCM or another AEAD format seems more natural . AEAD checks will immediately detect an invalid token , while using ECB forces reliance on invalidity heuristics . 2 ) If using AEAD , there is no need to encode the IP address in the token . It can be derived from the IP header and placed in a pseudo header . The pseudo header can then be authenticated as part of AEAD decryption . 3 ) The pseudo header approach can be used to authenticate other fields , e.g. verify that the DCID matches the SCID sent in the Retry packet . 4 ) Encoding the time as 64 bits time64_t seems more natural than ASCII encoding , and also shorter . 
 The editor draft also encodes the server 's SCID in the token . That too could be moved to a pseudo - header , since the field is assume present in the DCID of the post retry Initial . 
 This was closed in error , as deals with the Stream Cipher . 
 I filed a very similar issue and then closed it when I realized the duplication . The text of that issue : I 'm not an encryption expert , but if IIUC it 's insufficiently hard to forge a shared - state retry token . By inducing a Retry , an attacker can obtain the Retry SCID Length , and then focus entirely on an ODCIDs that allow the CID + CID Length part of the token to be a multiple of 16 B. For example , if RSCIDL= 10 B , then make ODCIDL = 20 B -32 Bytes for the block . This then breaks Retry forgery into three separate problems : ( 1 ) Obtaining the mapping of IP Address to the first 16 Bytes of the token . A well - positioned observer could build a database of these in the time scale between token key rotations . ( 2 ) Generate lots of Retry tokens with an ODCID of the correct length , so there is a range of valid CID blocks . ( 3 ) Obtain a valid Retry every few seconds , using the right ODCID length , so that we have a valid timestamp . Thus , the attacker has a database of valid encrypted IP addresses valid encrypted CIDs ( really , it 's the CID length that would cause validation to fail ) valid timestamps As the spec uses AES - ECB , these blocks can be mixed and matched to create valid Retry tokens . This is not exactly trivially open to attack [ 1 ] , but it does feel like we 're conceding a lot of entropy here . I would like someone to propose an alternate design that restores some of that entropy . [ 1 ] Step ( 1 ) seems to require a fairly privileged position in the network . 
 Could I trouble you for a PR on this ? 
 Sure . I can basically propose what I am doing in picoquic . And yes , the repetitive nature of ECB enables attacks . 
 please review PR 
 Looks great ! If my proposed nits seem reasonable to you , hit commit on each of them and then I 'll merge this .
All tokens, generated by either the server or retry service, MUST use
the following format.  This format is the cleartext version.  On the
wire, these fields are encrypted using an AES-ECB cipher and the
token key.  If the token is not a multiple of 16 octets, the last
block is padded with zeroes.
A 96 bit unique token number transmitted in clear text, but
protected as part of the AEAD associated data.
An 8 bit token key identifier.
A token body, encoding the Original Destination Connection ID, the
Retry Source Connection ID, and the Timestamp, optionally followed
by server specific Opaque Data.
The token protection uses an 128 bit representation of the source IP
address from the triggering Initial packet.  The client IP address is
16 octets.  If an IPv4 address, the last 12 octets are zeroes.  If
there is a Network Address Translator (NAT) in the server
infrastructure that changes the client IP, the Retry Service MUST
either be positioned behind the NAT, or the NAT must have the token
key to rewrite the Retry token accordingly.
The 96 bit unique token number is set to a random value using a
cryptography grade random number generator.
The token key identifier and the corresponding AEAD key and AEAD IV
are provisioned by the configuration agent.
Client IP Address: The source IP address from the triggering Initial
packet.  The client IP address is 16 octets.  If an IPv4 address, the
last 12 octets are zeroes.  If there is a Network Address Translator
(NAT) in the server infrastructure that changes the client IP, the
Retry Service MUST either be positioned behind the NAT, or the NAT
must have the token key to rewrite the Retry token accordingly.
Timestamp: The timestamp is a 64-bit integer, in network order, that
expresses the number of seconds in POSIX time (see Sec. 4.16 of
TIME_T).
The configuration agent generates and distributes a "token key" and a
list of QUIC versions the service supports.  It must also inform the
service if a NAT lies between the service and the servers.
In active mode, the service SHOULD decrypt the first 16 octets of
incoming tokens.  The service SHOULD drop packets with an IP address
that does not match, and SHOULD forward packets that do, regardless
of the other fields.
Timestamp: The Timestamp is a 64-bit integer, in network order, that
expresses the expiration time of the token as a number of seconds in
POSIX time (see Sec. 4.16 of TIME_T).
Some implementations of QUIC encode in the token the Initial Packet
Number used by the client, in order to verify that the client sends
the retried Initial with a PN larger that the triggering Initial.
Such implementations will encode the Initial Packet Number as part of
the opaque data.
The token key and IV are retrieved using the Key Sequence.
The nonce, N, is formed by combining the IV with the 96 bit unique
token number.  The 96 bits of the unique token number are left-
padded with zeros to the size of the IV.  The exclusive OR of the
padded unique token number and the IV forms the AEAD nonce.
The input plaintext for the AEAD is the token body.  The output
ciphertext of the AEAD is transmitted in place of the token body.
The AEAD Checksum is computed as part of the AEAD encryption
process, and is verified during decryption.
The configuration agent generates and distributes a "token key", a
"token IV", a key sequence, and a list of QUIC versions the service
supports.  It must also inform the service if a NAT lies between the
service and the servers.
In active mode, the service SHOULD decrypt the incoming tokens.  The
service SHOULD drop packets with unknown key sequence, or an AEAD
checksum that does not match the expected value.  (By construction,
the AEAD checksum will only match is the client IP Address also
matches.)  The service SHOULD drop packets if the Timestamp of the
retry token points to a past date; however, in order to allow for
clock skew, it SHOULD NOT drop packets if the Timestamp merely points
to a a few seconds in the past.  It SHOULD forward packets if the
AEAD checksum and the Timestamp can be verified, regardless of the
other fields.
When issuing Retry or NEW_TOKEN tokens, the server MUST encode the
client IP address in the first 16 octets and encrypt that block with
the token key.  It MAY use any format or encryption for the remainder
of the token.  However, it MUST include a means of distinguishing
service-generated Retry tokens, server-generated Retry tokens (if
different), and NEW_TOKEN tokens.
When issuing Retry or NEW_TOKEN tokens, the server MUST include the
client IP address in the authenticated data as specified in token-
protection-with-aead.  It MUST include a means of distinguishing
service-generated Retry tokens, server-generated Retry tokens (if
different), and NEW_TOKEN tokens.
For Retry tokens that follow the format above, servers SHOULD use the
timestamp field to apply its expiration limits for tokens.  This need
not be precisely synchronized with the retry service.  However,
servers MAY allow retry tokens marked as being a few seconds in the
future, due to possible clock synchronization issues.
For Retry tokens that follow the format above, servers SHOULD use the
timestamp field to apply its expiration limits for tokens.  This need
not be precisely synchronized with the retry service.  However,
servers MAY allow retry tokens marked as being a few seconds in the
past, due to possible clock synchronization issues.