Draft: draft-ietf-quic-transport

Commentary: In the current spec , zero - length connection IDs can be used when a host wants to route connections on the 4 - tuple ( remote IP / port , local IP / port ) . However , doing this prevents your peer from multiplexing connections on a single port . This PR restricts zero - length connection IDs to only be used when a host can route connection on the local 2 - tuple ( IP / port ) . The current spec does not allow clients to share a local port ever , because at the time they start their connection they can not know if a given server will want to route on the 4 - tuple . Additionally , some clients might even be forced to perform this multiplexing because they are going through a proxy . Slightly restricting the use of zero - length connection IDs gives value to client connection IDs . 
 I 'm not sure that is true , but if is , it should be fixed . A P2P server scenario will definitely want to use a single port on both ends , and which one is client , might change at any given moment . In this case you have out of band knowledge about path behaviour . For client p2p I 'd hope that it is also possible to only use a single port when a CID is used . 
 in the P2P case you often have a bootstrapping protocol ( ICE , STUN , TURN , etc . ) that gives you a single - use ephemeral port on both sides , so you can then happily use zero - length connection IDs because you 'll only have one connection on that port . If you expect to share the port for multiple connections , then you can use non - zero - length CIDs , there 's nothing in the spec ( or this PR ) that prevents that . 
 On client P2P / ICE : These are yesterdays protocols . You can have a new version of these brokers . If you use IPV6 you could skipping the port step . Also a client implementation might skip the part of supporting multiple ports because you a skipping the OS and go with raw sockets , netmap , etc .. It is much simpler to just rely on the CID . Especially if you consider implementations that can also run outside of IP networks . 
 On server P2P you do not have a boot strapping protocol . Then it is integral to the service logic being implemented . I have a strong use case for this and would be very sad if ports can not be stable . 
 if your use - case uses non - zero - length connection IDs , then it is not relevant to this PR . This PR does not change how non - zero - length CIDs work . 
 My objection was to the sentence " The current spec does not allow clients to share a local port ever " which implies also with CID . Whether you PR fixes this or not is not clear to me , but I find it problematic if is correct and then not fixed . 
 Yes it is problematic , and this PR attempts to fix that . 
 Thanks . I think the current text does n't explicitly ban trial decryption : technically that endpoint is able to identify the connection using only its IP and port so it 's good to go . The important bit is that it does n't require the peer to alter its behavior . 
 I rebased this . There was a conflict that might be worth checking : rewrote the bit . Where this removed text forbidding migration with a zero - length connection ID , that text no longer appeared . 
 Thanks for the rebase 
 Rebase looks good to me too , thanks 
 What next steps should be taken here ? Is this ready for a consensus call ? 
 Yeah , the chairs have this on their list ( the issue is marked ) . Patience . 
 That came around sooner than expected :) 
 QUIC supports client connection IDs ( CIDs ) in order to allow a client to multiplex multiple QUIC connections on the same local IP and port . However , the current spec does n't allow that to work , as pointed out by on : if a server decides to use zero - length CIDs , then the client can not establish multiple connections to that server from the same local port . For this to work , the client would need to know out - of - band that the server will not use zero - length CIDs , and that 's not viable . Therefore we have two protocol features that are at odds with one another : ( 1 ) server can share port and use zero - length CIDs ( which prohibits migration ) ( 2 ) client can share port and use non - zero - length CIDs We ca n't have both ( 1 ) and ( 2 ) because the client wanting to do ( 2 ) does n't know if the server will do ( 1 ) or not . If that happens , long headers will be demultiplexed properly because they contain the client 's CID , but short headers from client to server do not and therefore can not be demultiplexed . and I have a use - case for ( 2 ) which is a QUIC proxy that only uses one port for outbound connections and relies on client CIDs for demultiplexing . I do n't see a use for ( 1 ) . For client connection IDs to be useful in QUIC , I think we should ban zero - length connection IDs when you 're multiplexing multiple connections on a local IP + port . The current spec states : We may want to change it to something like : 
 I think your suggested text is preferable . That is how I think of using connection IDs , and it still allows 0 byte CIDs for P2P cases . 
 Sure , but that use case comes with a deployment requirement . QUIC nodes using the proxy service MUST use a CID of equal or larger length than required by the proxy . This is a requirement on incoming packets , and the CID on incoming packets is chosen by the intended recipient of these packets , in our case the nodes behind the proxy . It is not a requirement of outgoing packets -- the proxy will just send the packets formatted by the client to the address and port that the clients specifies . It will work even if Azure servers are using zero length CID . Theses servers wo n't work behind a proxy , but they will reachable through the proxy . 
 Thank you for opening the issue . I agree that this is a design issue that needs to be fixed . Agreed . IIRC , the introduction of client CID was a deliberate action to allow multiple QUIC connections be coalesced on a 5 - tuple . I think the reason ( 1 ) exists is simply because we did not notice that there is a conflict . Based on that , I think we should adjust ( 1 ) to allow ( 2 ) to happen . Allowing a client use one ( or few ) UDP ports to handle many QUIC connections has benefits . It helps code reuse ( because servers are designed to use one port for many connections ) and also has performance benefits ( because you can use one system call to read / write packets belonging to many connections ) . 
 I think you may have misunderstood what I meant . If the web server uses zero - length CIDs on port 443 , then it will not be able to demultiplex incoming short packets from multiple connections that transit through the proxy . So we can not build that kind of proxy if servers like this exist . 
 Got it , OK . Server should not assume that client connections would always come from different 5 tuples , because in the case of proxies they will not . For example , two clients coming through the same proxy will appear to come from the same IP and port . This is not a problem for clients , because clients can decide whether or not to send several outgoing connections through the same port . 
 I think that the one word fix is fine . It needs to be about things that the recipient controls . I would be reluctant to say more though . This is n't that important a point . Note that clients are prevented from reusing a source port if the server chooses a zero - length connection ID , but I think that 's fine . 
 the issue is that , at the moment it creates the connection , a client has no way of knowing that a server will multiplex and use zero - length CIDs . Therefore just giving servers the possibility of multiplexing and using zero - length CIDs completely negates the value of client connection IDs . 
 Is n't that something the client can know before it creates the second connection ? 
 The problem is not the leaf client , it is the proxy . The logical implementation of the proxy is to have just one UDP port . Conceivably , the proxy could manage a pool of ports and try to guarantee that a given 5 tuple corresponds to exactly one client - server connection . But that 's significantly more complex than having just one port and relying on the CID . 
 Would it make sense to solve this by placing restrictions on QUIC configuration at the application protocol level ? E.g. forbid HTTP/3 servers from using zero - length CIDs . That avoids forcing increased header size on other application protocols which may have no need for the type of proxy described here . 
 Connection IDs are a property of the transport layer , and the proxy use - case described above uses HTTP/3 . I do n't think we should vary any of this by application . 
 I 've updated the associated PR in case anyone following here would like to go check it . 
 What do people think about allowing the server to use a zero - length connection ID only if the client chose to use a zero - length connection ID ? By selecting a zero - length connection ID , the client is indicating it will not reuse the port . So the server can rely on just the source ip and port to identify the connection . 
 That seems better than forbidding it outright , at least . 
 That does n't work . As I client we use zero - length connection IDs and migration . The server can not use our address and port as that would break connection migration and NAT rebinding . 
 If a server wishes to support migration / rebinding , it could still choose to use non - zero - length CIDs , as today . 
 Discussed in Montreal , to revise the PR as discussed and then we 'll attempt to see if there is consensus 
 Discussed in Montreal . Some concern about enabling a TCP - like usage for QUIC , but concerns about the effect that a choice of an endpoint could have on its peer . Options are : use only local information ( local host , port , cid ) to route packets . , with qualification , but leaving this open to people to shoot each other in the feet . unless you know something about what your peers ( as you identify them ) are doing . This precludes peers that attempt to use connection IDs if you intend to use source - address - based routing . I think that we might want to end up in the third bucket . Which acknowledges and endorses the use case , but properly defines the conditions under which this is acceptable . 
 I 've updated the associated PR to make this a SHOULD , and explain that it MUST NOT be violated unless an endpoint knows that these features are not in use . 
 The new text addresses my concerns 
 At the risk of tying this in with other issues , there are two separate problems here . 1 ) Migration : if I change address , I have no means of expressing that it is the same connection . On the other hand " DISABLE_MIGRATION " promises that things will fall apart if the address changes . In that event I am not sure why we would force the server to do anything . 2 ) Multiplexing . I agree that this case does not work if the server has no CID . But are servers required to support the same - port case ? As someone that will be stuck with a 4 - tuple based infrastructure for at least another year , trying to handle multiple connections on one 4 - tuple is quite laborious and I 'm not likely to support it in the near term . I 'd much rather spend the time fixing my internal routing rather than band - aiding this use case . 
 : I 'm not sure if same port should always be supported , but aside from your temporary difficulties , it is highly valuable to servers with the opposite view : band - aiding ephemeral hosts when it is much simpler to just multiplex on the raw packet based on a connection ID hash . A lot of infrastructure is artificially limited by the number of ephemeral ports available . Getting rid of this problem requires getting rid of ephemeral ports . 
 I agree that there are two separate issues . My opinion is that we should require servers to support clients multiplexing multiple connections on the same port . It 's OK if your routing infrastructure only looks at the four - tuple , but the end - server than receives the packet needs to demultiplex based on the server connection ID . 
 Note that supporting multiplexing clients does not mean always using a server CID . A server that does not support client migration may decide to use a CID only if the client is using non - zero - length CID . ( And there is little reason for a client CID other than for multiplexing . ) 
 As a server you should not use zero - length connection IDs unless you can demultiplex locally based on your own IP and port . Violating that breaks several protocol features - see . 
 I obviously disagree with this ( and we had a discussion about this in Montreal ) . Support for connection migration is optional . Hence , if the server does not support connection migration or knows that a particular client will not migrate , there is no need for a server to supply a non - zero - length CID , when such a client supplies a zero - length CID . Which protocol feature is missing when the client will not migrate , if the server is using a 4 - tuple to demux and there is a zero - length client and server CID ? 
 A client that is using a single UDP connection would not be able two establish two connections to such a server at the same time . Even worse , it would have no way of knowing if it ’s safe to dial two connections concurrently . 
 such client would not use a zero - length client CID , so the argument is moot . 
 The client would not use zero - length CIDs for receiving packets . The problem occurs when the server uses zero - length CIDs , since it wo n't be able to distinguish between the two connections . 
 +1 to what said 
 So we are saying the same thing . I do not see where you are disagreeing . I said , verbatim : Both of us are agreeing that a client who is interested in multiplexing will not chose to use a zero - length client CID . Hence the server will also not use a zero - length server CID . But when a client chooses a zero - length client CID ( meaning it will not multiplex on that 4 - tuple ) , the server should be free to choose a zero - length server CID , if it does not support connection migration . 
 How does that work in practice ? You ca n't use both zero - length and non - zero - length connection IDs on the same server without trial decryption . What happens when your first client uses zero - length - client - CIDs and you second one does n't ? 
 You can use both on the same server , and no trial decryption is required . If the 4 - tuple for an incoming packet is found in , use that connectionctx . Otherwise , assume there is a CID and look up in . That 's all . 
 thanks for clarifying . I have to admit this approach does seem convoluted . What 's the benefit of this over non - zero - length server connection IDs ? ( they have the advantage of surviving NAT rebinding ) 
 Short answer : saving bytes on the wire . Long answer : A server may need to support both regular clients ( so using a server CID ) as well as peer servers in the datacenter ( or some SDN ) . It may be trivial to identify such peers ( anyone using 10.0.0.0/8 ) , so why waste CID in each packet when talking to 10.0.0.0/8 machines ? Moreover , QUIC now supports huge Server CIDs ( up to 20 bytes ) , and if that 's what 's needed for regular clients , that 's wasting 20 bytes per packet on internal communications . 
 I think no one would stop you from doing something like what you describe in . However , most of us do not want to be forced do something as complex as that . Therefore , we are trying to clarify when the 4 - tuple can be used by the server for identifying the connections . If it is your intent is to ensure that your mixed approach is allowed as one of the approaches , then I think we can clarify that in ( I think it 's already implied ) . 
 This discussion is about not arbitrarily forbidding something that is useful . Of course different implementations will chose to do what they need to do . Once the implementation is deployed at scale , people tend to look for ways to find even 1 % improvements . I am puzzled why you think it is " complex " , though . Is it that much more complex to do than 
 Thank you for the clarification . Assuming that we can give clients the freedom coalesce multiple connections onto a single 5 - tuple , I do not have a strong argument against giving servers some wiggle room . I think it is more complex than that . The approach you are describing can be modeled as a daisy - chain of two QUIC servers . What would the server do if there is no corresponding state for a short header packet that you receive , neither in nor ? The server needs to send a stateless reset . But because the server can not tell if the packet were to be identified by the 4 - tuple or the CID ( because you no longer have state ! ) , the server needs to send two stateless reset packets , one generated using the 4 - tuple as the key , and the other generated using the CID . However , simply doing that could lead to an amplification attack ( as we know , the amount of data ( or packets ) the server can send as a reset needs to be smaller than the amount of data that the server has received ) . This might _ be a problem that we can solve . But is it worth spending the time of the WG to discuss such thing ? I 'm not sure . And I 'm not sure if this is the only problem that exists in the proposed design . 
 The main reason for this is not amplification but because it could lead to infinite reset loops . This could be by mistake , or by deliberate injection of packets to trigger this behaviour . 
 This is a good point that Stateless Reset must also be considered . In case that the server can identify a certain population of clients ( 10.0.0.0/8 ) as not needing migration , it can also identify which clients to send what kind of Stateless Reset . 
 : +1 : The requirements look good to me . I only have one editorial suggestion .
Paragraph (version B):
If the stream is in the "Recv or "Size Known" states, the transport
SHOULD signal this by sending a STOP_SENDING frame to prompt closure
of the stream in the opposite direction.  This typically indicates
that the receiving application is no longer reading data it receives
from the stream, but it is not a guarantee that incoming data will be
ignored.
Paragraph (version A):
A zero-length connection ID MAY be used when the connection ID is not
needed for routing and the address/port tuple of packets is
sufficient to identify a connection.  An endpoint whose peer has
selected a zero-length connection ID MUST continue to use a zero-
length connection ID for the lifetime of the connection and MUST NOT
send packets from any other local address.
Paragraph (version A):
Hosts try to associate a packet with an existing connection.  If the
packet has a Destination Connection ID corresponding to an existing
connection, QUIC processes that packet accordingly.  Note that more
than one connection ID can be associated with a connection; see
connection-id.
Paragraph (version A):
Valid packets sent to clients always include a Destination Connection
ID that matches a value the client selects.  Clients that choose to
receive zero-length connection IDs can use the address/port tuple to
identify a connection.  Packets that don't match an existing
connection are discarded.
Paragraph (version A):
Packets with a supported version, or no version field, are matched to
a connection using the connection ID or - for packets with zero-
length connection IDs - the address tuple.  If the packet doesn't
match an existing connection, the server continues below.
Paragraph (version A):
The client populates the Source Connection ID field with a value of
its choosing and sets the SCIL field to indicate the length.  The
first flight of 0-RTT packets use the same Destination and Source
Connection ID values as the client's first Initial.
Paragraph (version A):
On confirming a peer's ownership of its new address, an endpoint
SHOULD immediately reset the congestion controller and round-trip
time estimator for the new path.
Paragraph (version B):
An endpoint MAY send a stateless reset in response to a packet with a
long header.  Sending a stateless reset is not effective prior to the
stateless reset token being available to a peer.  In this QUIC
version, packets with a long header are only used during connection
establishment.  Because the stateless reset token is not available
until connection establishment is complete or near completion,
ignoring an unknown packet with a long header might be as effective
as sending a stateless reset.
Paragraph (version B):
To limit ACK Ranges (see ack-ranges) to those that have not yet been
received by the sender, the receiver SHOULD track which ACK frames
have been acknowledged by its peer.  The receiver SHOULD exclude
already acknowledged packets from future ACK frames whenever these
packets would unnecessarily contribute to the ACK frame size.  When
the receiver is only sending non-ACK-eliciting packets, it can bundle
a PING or other small ACK-eliciting frame with a fraction of them,
such as once per round trip, to enable dropping unnecessary ACK
ranges and any state for previously sent packets.  The receiver MUST
NOT bundle an ACK-eliciting frame, such as a PING, with all packets
that would otherwise be non-ACK-eliciting, in order to avoid an
infinite feedback loop of acknowledgements.
Paragraph (version B):
An endpoint SHOULD treat receipt of an acknowledgment for a packet it
did not send as a connection error of type PROTOCOL_VIOLATION, if it
is able to detect the condition.  This includes receiving an ACK
frame containing a packet number that the endpoint has not sent, as
well as acknowledgements for 0-RTT packets when the server has
rejected the use of 0-RTT.
Paragraph (version A):
The Destination Connection ID field follows the connection ID
lengths and is either 0 bytes in length or between 4 and 18 bytes.
negotiating-connection-ids describes the use of this field in more
detail.
Paragraph (version A):
The Source Connection ID field follows the Destination Connection
ID and is either 0 bytes in length or between 4 and 18 bytes.
negotiating-connection-ids describes the use of this field in more
detail.
Paragraph (version B):
As future versions of QUIC may support Connection IDs larger than the
version 1 limit, Version Negotiation packets could carry Connection
IDs that are longer than 20 bytes.
Paragraph (version A):
The four least-significant bits of the first byte of a Retry
packet are not protected as they are for other packets with the
long header, because Retry packets don't contain a protected
payload.  These bits instead encode the length of the Original
Destination Connection ID field.  The length uses the same
encoding as the DCIL and SCIL fields.
Paragraph (version A):
The Original Destination Connection ID contains the value of the
Destination Connection ID from the Initial packet that this Retry
is in response to.  The length of this field is given in ODCIL.
Paragraph (version A):
An 8-bit unsigned integer containing the length of the connection
ID.  Values less than 4 and greater than 18 are invalid and MUST
be treated as a connection error of type PROTOCOL_VIOLATION.