Draft: draft-schinazi-masque-h3-datagram

Commentary: introduced the following text : There are 2 problems with this , in each direction : the send - direction , this can only be ascertained in the case of no packet loss , when all packets have been acknowledged . If any packet is not acknowledged , this could be due to this packet actually having been lost or just reordered delayed . it 's possible to ascertain that all sent packets have been acknowledged , this is not possible for packets sent by the peer . Reusing a flow identifier opens up an attack vector : An attacker could delay the delivery of a packet containing the old flow identifier until it has been retired and reused . This would lead to a reinterpretation of the DATAGRAM contents in a new application context . 
 If your HTTP/3 code is closely coupled with your QUIC code , this can be solved by looking at QUIC packet numbers . If we take the CONNECT - UDP example , receiving a RESETSTREAM frame for the CONNECT - UDP stream ensures that the peer will not send subsequent DATAGRAM frames with this flow ID . If you record the packet number of the packet that carried the RESETSTREAM frame , you can then drop any DATAGRAM frames with that flow ID whose packet number is lower . That solves both problems you 've described . 
 It is not , my implementation separates the two layers . In the design of QUIC transport , we went a long way to avoid looking at the packet number at all . If this design principle applied to the transport , it should even more so apply to applications built on top of QUIC . That only applies for the server side , does n't it ? The client still would n't know if a packet it receives was sent before or after the ID was reassigned . 
 This feature is optional , so you could choose to not implement it . Or are you proposing a different mechanism that works ? I agree that we should n't violate this design principle in the mandatory parts of the spec , but doing so in optional features seems acceptable to me . This is n't really about client or server , but more about who sent the RESETSTREAM frame . The receiver of RESETSTREAM can key on the packet number that carried that frame , whereas its peer can key on the packet number of the first acknowledgement for that  . 
 How so ? How do I prevent the peer from reusing a flow identifier ? Actually , I 'm not quite sure I understand the issue that raised in . I see flow identifiers as analogues to stream IDs . For stream IDs , we never even considered reusing them , even though that would n't have required a layer violation in that case . Why are the 3 additional bytes ( I 'm assuming you 're using fewer than /2 identifiers on a typical connection , otherwise it 's 7 ) too expensive in one case , but not the other ? 
 Why would you want to prevent the peer from reusing a flow identifier ? The peer ensures the security properties of reuse . Because the payload of DATAGRAM frames needs to fit in a single QUIC packet , whereas STREAM frames allows splitting the payload across packets . When you 're tunneling , this means that the overhead impacts your MTU when using DATAGRAMs . 
 I 'd like to raise the attention towards the idea that reliably supporting reuse of flow identifiers can become quite complicated when in large - scale installations . Consider a large scale point of presence , where a large number of servers are sharing the same public IPs : In such scenario , multiple datagrams that share the same flow identifier may be served by several distinct physical servers . In order for these servers to be able to route the datagrams in a consistent way , the flow i d allocation service must be able to provide some sort of consistency guarantees : when the flow identifier expires , any server that may serve the flow needs to agree that the flow has indeed expired . In other words , there is a consensus problem lurking behind the scenes . While there are several ways to achieve such consensus , all share some propagation delay . Clients do n't have a good way to estimating such delay ( except for the observation that it 's unlikely to be significantly shorter than the retransmit timeout ) . It may be worth at least mentioning that reusing the flow identifiers rapidly is not recommended . ( edit : changed the wording that has mistakenly projected an argument as a known fact ) . 
 Hi , this is an interesting scenario but I 'm not sure I understand it : are you saying that multiple servers can send the same datagram flow ID on the same connection ? How does that work ? Given that QUIC uses incremental packet numbers as the encryption nonce , you 're going to need a synchronization point in the server - to - client datapath anyway , or am I misunderstanding something ? 
 Hi I was probably unclear . It is also possible that I am missing something . Let me try to clarify my comment . EDIT : I have reformatted / rephrased and clarified the comment . Again , I may be missing something , and will be glad to learn more in this case . I think the main point of my comment - if the flow i d is known to the applications , then it would be hard to communicate the retirement of the flow ID . In particular , in some situations it would be impossible for an endpoint to infer that a particular flow i d has been successfully " retired " by the peer endpoint , because the application can retain the state associated with that flow i d. 
 Maybe the draft is n't clear : the idea is that the application is responsible for knowing that the flow ID is no longer in use , and ensuring that the application on the peer also knows that it is no longer in use , before retiring it - the application tells the flow allocation service to retire , the flow allocation service does n't do it on its own . 
 There is a certain advantage to being able to use small flow identifiers , but there are only 32 of the smallest available for any given usage . Would it be possible for a particular usage to reuse a value ? I ask because CONNECT - UDP has a distinct start and end and could possibly take advantage of this . That would have some small , but likely meaningful , impact on efficiency . 
 I agree , this feature might make a real performance difference . The only caveat is reordering , we 'll have to be careful that if receiving a RESET_STREAM frame for the CONNECT - UDP stream makes a flow ID available and a datagram with that flow ID gets reordered to arrive after , we 'll need to check the packet number to make sure that this datagram is dropped instead of routed to its new owner . 
 I do n't like looking at packet numbers and would be opposed to that . It 's a pretty serious layering violation . A guard interval to minimize the potential for overlap might help , but I would prefer that we only allow reuse if the protocol that is carried has some sort of protection against receiving bad packets . Tunneled QUIC should work fine for that ( though Version Negotiation packets are still a little bit of a concern ) , as would tunneled STUN / TURN and WebRTC . More analysis would be required if the protocol did not provide its own integrity protection , but as this is primarily a tunneling protocol , we might be safe in advising against that .
The flow allocation service SHOULD also provide a mechanism for
applications to indicate they have completed their usage of a flow
identifier and will no longer be using that flow identifier, this
process is called "retiring" a flow identifier.  Applications MUST
NOT retire a flow identifier until after they have received
confirmation that the peer has also stopped using that flow
identifier.  The flow identifier allocation service MAY reuse
previously retired flow identifiers once they have ascertained that
there are no packets with DATAGRAM frames using that flow identifier
still in flight.  Reusing flow identifiers can improve performance by
transmitting the flow identifier using a shorter variable-length
integer encoding.