Draft: draft-ietf-quic-recovery

Commentary: Came from the discussion of 
 : I realized after we did this PR yesterday that this does not I It reduces burstiness in general , which is still a good thing , but it does n't change slow start increase . I 'll send a PR or ask  if she wants to . 
 When an ACK arrives that acknowledges N packets ( where N could be very large ) , QUIC will increase the congestion window by more than 1MSS per congestion window during Congestion Avoidance causing bursty send . This could also affect the slow start . if ( congestionwindow < ssthresh ): // Slow start . congestionwindow + = else : // Congestion avoidance . congestionwindow + = kMaxDatagramSize * / congestionwindow One could argue that QUIC ACKs every other packet , but there ACK packet could get lost or ACK for ACK packet could get lost . The draft is probably assuming packet pacing but not all implementations support that . I discussed this with  and he thinks that we should add something in pseudo code or some specific text in the draft . 
 PR more specific text on avoiding bursts into the network , FYI . 
 Please note that PR addresses the issue of app - limited or pacing - limited cases to not increase cwnd when it is under utilized . My concern is cwnd being inflated due to an ACK frame containing ACKs for large number of packets . 
 : The problem here is that we do n't talk about burstiness that can occur when pacing is not in place , and a stretch - ack is received . PRR handles this , but we do n't do PRR in the draft . 
 No we need to spend lots of time on Reno as everybody appears to be doing Cubic or BBR ? 
 Cubic has this problem . BBR can not be assumed . 
 Except Cubic opens the window based on time spent , not just ack received 
 I do n't get the scenario . It seems that is worried about receiving an ACK for a large number of packets . This is not a congestion window problem , because it happens even if the congestion windows size remains constant . The root cause is that the ACK of N packets now reduces the number of packet in flow by N. So this is strictly about pacing , not about congestion control . 
 yes , and it can and still does burst because of stretch acks 
 Not just stretch Acks -- deliberate  compression by the network happens too . Very few alternative to pacing . 
 describes 
 Discussed in Cupertino . to prepare a PR that hopefully can incorporate some ideas from RFC3465 to add safety features for non - pacing stacks . 
 But 3465 does not solve the burst problem . It limits the growth of the window , but does not do pacing of any kind . 
 I think we can change to something like this 
 I think we need to do 2MSS clamping per ACK received and not per packet acked . I am thinking that we can remove OnPacketAckedCC and instead do OnAckReceivedCC . Pseudocode change something like this : 
 I think we can keep but just move cwnd update to as you propose . 
 Yup , that 's an important missing fix , though it needs to be applied only when not pacing . Additionally , helps with burstiness in general . : I would make a variable , and apply the min here when not pacing . 
 The proposed change has the effect of reducing the effective Window if ACKs are compressed . Say you have a window of 12 packets , acks are delayed , and a single ACK acknowledges 12 packets . Now you have a window of 2 packets , your transmission rate is 6 times slower than before , and it will take you 10 RTT to recover . Is that really what you want ? 
 This is what ABC ( RFC 3465 ) does for TCP . ACK compression has side - effects , and the way to deal with it is , as you say , to pace packets out . This reduction in performance is only when the endpoint does not pace , and it is the only way to limit bursts in that case . 
 Also , ACK compression does not necessarily imply ACK pruning . I have traces in which 1 RTT worth of ACKs is compressed and delivered back to back . In that case , the proposed algorithm will do nothing : you will get 2 MSS for each ACK , back to back within less than 1 ms . 
 FYI . FreeBSD implements rfc3465 ( and enabled by default )  removed tcpabc implementation but saying it does in a different way : Considering  does n't have pacing by default , I think current proposed algorithm is not worse ( or aggressive ) than what tcp implementation does . 
 I am working on the PR for pseudocode changes . 
 I agree that when not pacing , using an ABC limit is appropriate . FWIW Windows TCP was using a limit of 4 and not 2 for a long time and now uses 8 which is pretty close to IW10 burst in slow start phase . 
 For the WG : The proposed PR attempts to include a mechanism to limit CWND increase during slow start when not pacing . I 'll note that by itself , it does not guarantee we follow the existing MUST : " Implementations MUST either use pacing or limit such bursts to the initial congestion window , which is recommended to be the minimum of 10 maxdatagramsize and max(2 maxdatagramsize , ) ) , where maxdatagramsize is the current maximum size of a datagram for the connection , not including UDP or IP overhead . " I do n't think we should add this to the pseudocode , because we say SHOULD pace packets and the pseudocode currently only covers SHOULDs , not MAYs . Adding all MAYs to the pseudocode is too complex . I do n't have a concern about adding a reference to the existing RFC as a mechanism that MAY help limit bursts during slow start . I 'd like a decision from the WG on whether to include this change and if so , some criteria on what should be included going forward . 
 I agree with the comment that burst from ACK compression or Stretch - ACKs result in bursts . I think the ways to change that is implement a maximum burst size or use pacing . The bursts do not appear a CC function to me . 
 Bringing conversation in from the list : With regards to , one of the things we found in testing was that it could ( a ) still allow an implementation to be vulnerable to too much bursting ( the original problem ) , but more importantly ( b ) locked receivers into an ack pattern of acking every other packet . My main concern is that this causes problems if receivers want to experiment with and change acking strategies going forward in way that otherwise would be fine . One way to look at the problem described by this issue is that the existing text has a minor underspecification , in which it has the notion of a burst limit but says nothing about the frequency of such bursts . Two " bursts " of 10MSS that are sent within a couple nanoseconds are really one burst that 's too big . Another way of fixing this is to remove even the specification of the 10MSS limit if we really do n't want too much text about the non - pacing cases . Essentially , what this comes down to is : pace , or even if you do n't pace , you need to not send bursts above some frequency ; and if you do n't set some pacing timers , you 'll likely underutilize the link . 
 I 'm going to put this out there : the answer is MUST pace . You might formalize this by saying the number of packets sent in any interval can not exceed by more than the initial cwnd ( i.e. , the 10MSS ) . That allows for bursting , but requires a period of quiet either preceding or following any burst . 
 Yes , it does largely reduce to " implementations MUST pace " ( even if the pacing algorithm is n't very nuanced , in which case you 'll perform poorly ) . The formalization of the rate , one option for which was specified in , I think is the heart of what should be done here . 
 There are two routes that we can take : clearly what ' delay ` refers to in the below statement , which is done in OR MUST pace to SHOULD and remove the 10 MSS burst size limit and provide general recommendation if an implementation does n't pace . Something like : 
 I 'm sympathetic to saying MUST pace , and I like the formatlization that suggests . I 've heard some pushback on saying MUST pace in the past , but it might be worth giving it a shot again . : What do you think ? 
 That said ... I will note that 's formalization allows for a burst at the end of an RTT . Consider for example when a single ACK is received that acknowledges an entire window . That would be legal by this definition . I think that 's ok . 
 Discussed in ZRH . Proposed resolution is to define and REQUIRE pacing . 
 I think requiring pacing makes good sense , with some formalization of what that means . 
 To clarify the status here , my understanding is that the proposal is to close this design issue with no action . The discussions have identified editorial improvements that will be addressed independently . 
 Let 's continue the editorial discussion on . 
 Closing this issue and continuing discussion on 
 This needs to run through the consensus call procedure , so we 'll keep it open for another week . 
 Let 's get this in -- it addresses some burstiness anyway . LGTM
Paragraph (version B):
Sending multiple packets into the network without any delay between
them creates a packet burst that might cause short-term congestion
and losses.  Implementations MUST either use pacing or limit such
bursts to minimum of 10 * kMaxDatagramSize and max(2*
kMaxDatagramSize, 14720)), the same as the recommended initial
congestion window.
Paragraph (version A):
Sending multiple packets into the network without any delay between
them creates a packet burst that might cause short-term congestion
and losses.  Implementations SHOULD either use pacing or reduce their
congestion window to limit such bursts to minimum of 10 *
kMaxDatagramSize and max(2* kMaxDatagramSize, 14720)), the same as
the recommended initial congestion window.