Draft: draft-ietf-quic-ack-frequency

Commentary: I do n't fully understand when would someone want to set ignore - order = true , i.e. , what is a valid use case for this ? Lets say for whatever reason the sender sets this to True , this would if there is actual packet loss and receiver receives OOO packets , it wo n't send an immediate ACK . Meaning the sender wo n't enter loss recovery immediately and will continue sending packets at the same rate . This is not good , as it directly increases queuing on the network node and for a tail drop queue ( most widespread deployment ) , it will also affect other flows . Meaning , other flows will experience packet loss due to non - responsiveness of this flow for certain period of time . I probably missed past conversations about ignore - order field , so my apologies if this has already been discussed . I read the draft and did n't see any explanation about where would this be useful and what are the fallbacks of setting this to true . 
 One use case that I know and care is for sending QUIC traffic from a server cluster . Some packets would be sent directly from a server node that governs the connections , but others would be sent via different nodes within the same cluster . Once we deploy a sender like this , the receiver would observe what looks like reordering ( because most packets will be essentially delivered through an additional hop , and that would cause delays ) . But we do not want the receiver to send ACK for each packet being received . Regarding the problem , I 'm not sure I agree with the impact . Typically , congestion controllers react to congestion after 1 RTT . When packet - number - based detection is turned off , that changes to 9/8 RTT . That does cause more pressure on the bottleneck queue , but it is no worse than that caused by a path with an 9/8 greater RTT . Assuming that congestion controllers are designed to be fair against flows having different RTTs , I tend to believe that this is a non - issue . 
 I agree with though I think adding a SHOULD NOT use a delay longer than an RTT ( ) would be a good addition , since if the delay is extremely large(ie multiple RTTs ) , that could be bad . 
 In general , the spirit of this draft is to provide control over acking behavior to the sender , since it is the consumer of this information and ought to be able to control how frequently it needs to see this information . If there are subtle consequences to these decisions , it is worth noting them in the draft . It seems useful to add some text on the consequences of using   , so that senders can make informed decisions . We can do that . 
 is this something that  does or is planning to do ? How would you share the connection secrets between different server nodes ? When we set ignore - reorder = true , and lets assume that   delay 1RTT , it can be 1RTT or 2 or 10RTT . this would result in loss recovery to be delayed by at least an additional RTT , i.e. it will take 16/8RTT ( 2RTT ) instead of 9/8RTT to declare packets lost . That is significant IMO . Assuming that congestion controllers are designed to be fair against flows having different RTTs , I tend to believe that this is a non - issue . CUBIC throughput is inversely proportional to RTT in New Reno mode . Nonetheless , I am more concerned about high queuing in bottleneck due to late loss recovery . Specifying the use - cases where ignore - order = true SHOULD be used will prevent new implementors from using it randomly . And as  said , some text around the consequences when it is used would allow informed decision making . 
 I do not speak for Fastly 's plan , but the OSS branch of H2O has a working prototype that does this : The calculation is correct . At the same time , I do not think that senders will be willing to use    as large as that , when setting the Ignore Order bit . The problem here is the impact on loss recovery . If    is set to a value as large as 1 RTT , then the sender needs to spend more than 2 RTTs for detecting a loss . That means that the size of the send and receive buffers have to be twice as large , and that it would take as twice as long to recover from a loss . Based on existing advice that we have in other RFCs , I think we should discourage    that is greater than 9/8 RTT when   bit is used ( please see my comment on ) . 
 I 'd suggest 5/4 , but otherwise agree with you 
 Should the Max ACK Delay be restricted to 1/8 or 1/4 RTT as there is inherent 1RTT time spent while sending data and receiving ACKs ? That would make a total of 1 + 1/8 or 1 +1/4 RTT total/ 
 I think this is best as a sender - side SHOULD , since it knows what time threshold it 's using . For example , we use 1/4 RTT , but have considered making it adaptive(ie : 1/16 RTT to 1 RTT ) based on observed reordering . Also , if your RTT is 100us , and your timers have coarser granularity , 1/8 RTT or 1/4 RTT just does n't make sense . I 'm not sure what does , because I 'm not an expert in datacenter networking , but that 's why I think this is a SHOULD , not a MUST . Of course , it 's also unenforceable , which is a good reason for a SHOULD . 
 +1 to SHOULD . The only argument I might disagree with is that it is unenforceable ( IMO a receiver can enforce by capping   to a fraction of the RTT that the receiver has observed ) . But I do not think there 's any reason to require the receiver to enforce , considering the fact that a sender can send as many packets as it likes , regardless of ACKs the receiver sends ( or lack of ) . 
 I am fine with SHOULD and with a mention of how a sender would compute    when he sets ignore - order = true . An example is fine too . 
 I believe this is the same issue as 
 There are two things that are impacted by setting a high value for ack delay : congestion control and time - threshold loss detection . On congestion control , we need some protection for the network , so we should recommend something . On loss detection , a sender that chooses and sets an ack delay higher than the time - threshold loss detection time period risks triggering unnecessary retransmissions . So , I propose adding the following for a sender : SHOULD set the ack frequency to be at least one ack per RTT ( so that congestion control responses are reasonably quick ) a word of caution that setting the ack delay to be longer than time threshold can cause the sender to unnecessary retransmit 
 In 9.3 . Window - based Congestion Controllers I thought that unless there is loss reported , a QUIC ACK Frame releases sending window . In a similar manner as the accurate byte counting style in TCP , a QUIC sender solely operates on the basis of bytes acked , not the number of ACK frames received . So while a delayed ACK could delay a round of growth when the ACK Ratio is larger , it is only delayed by the time to ACK the set of received packets . In looking at various QUIC CC over medium and longer path RTTs this effect was quite small , and the default ACK delay was not unreasonable ; for shorter path RTTs this might be different 
 See PR 
 To me , ACK Clocking is a separate point that applies to any type of CC when the cwnd limits the sender . I agree there could be a concern is around whether a less frequent ACK policy can induce a cwnd - limited sender to send bursts of packets that could induce loss or disrupt sharing of the path with other flows . The QUIC specification already permits an initial window of 10 packets , and motivates the need for pacing . I think it is important this text refers to the QUIC transport section on pacing to mitigate bursts . 
 Better worded perhaps in PR . 
 Section 9.1 - There are CC considerations with respect to how long it is reasonable for a flow to hold - off detecting congestion and responding . This needs to be discussed . When an endpoint detects persistent congestion , it MUST promptly reduce the rate of transmission when it receive or detects an indication of congestion ( e.g. , loss or ECN marking ) [ RFC2914 ] , the Ignore Order value of true ( 0x01 ) in this ID allows a sender to extend that period , postponing detection of loss . That might be reasonable , but excessive delay can be dangerous - and therefore the impact really needs to be discussed : delaying a fraction of a RTT is in my mind safe , intentionally delaying by an RTT is arguable . A delay of many RTTs is endangers other flows , and we need to at least say that in some way ( [ RFC8084 ] took an alternate view of how long might be safe ) . 
 See PR 
 I think this warrants a note about how a sender SHOULD NOT use a delay that is larger than an RTT unless the sender has other information about either the network or current network conditions . Otherwise , the receiver does not need to enforce anything here . 
 Delay of loss detection has negative impact on loss recovery as well . Considering that the time threshold defined in RFC 9002 is 9/8 RTT , I might ague that , when   is set to true , then the maximum delay being advertised SHOULD be no greater than 9/8 RTT . These values provide responsiveness comparable to  when a packet is lost - the loss would be detected within 5/4 RTT . ( stuck out , as this point is already covered by Section 8) .
There are tradeoffs inherent in a sender sending an ACK_FREQUENCY
frame to the receiver.  As such it is recommended that implementers
experiment with different strategies and find those which best suit
their applications and congestion controllers.  There are, however,
noteworthy considerations when devising strategies for sending
ACK_FREQUENCY frames.
A sender relies on receipt of acknowledgements to determine the
amount of data in flight and to detect losses, e.g. when packets
experience reordering, see QUIC-RECOVERY.  Consequently, how often a
receiver sends acknowledgments determines how long it takes for
losses to be detected at the sender.
Many congestion control algorithms have a startup mechanism during
the beginning phases of a connection.  It is typical that in this
period the congestion controller will quickly increase the amount of
data in the network until it is signalled to stop.  While the
mechanism used to achieve this increase varies, acknowledgments by
the peer are generally critical during this phase to drive the
congestion controller's machinery.  A sender can send ACK_FREQUENCY
frames while its congestion controller is in this state, ensuring
that the receiver will send acknowledgments at a rate which is
optimal for the the sender's congestion controller.
Congestion controllers that are purely window-based and strictly
adherent to packet conservation, such as the one defined in QUIC-
RECOVERY, rely on receipt of acknowledgments to move the congestion
window forward and send additional data into the network.  Such
controllers will suffer degraded performance if acknowledgments are
delayed excessively.  Similarly, if these controllers rely on the
timing of peer acknowledgments (an "ACK clock"), delaying
acknowledgments will cause undesirable bursts of data into the
network.
This section provides some guidance on a sender's choice of
acknowledgment frequency and discusses some additional
considerations.  Implementers can select an appropriate strategy to
meet the needs of their applications and congestion controllers.
A sender needs to be responsive to notifications of congestion, such
as a packet loss or an ECN CE marking.  Also, window-based congestion
controllers that strictly adhere to packet conservation, such as the
one defined in QUIC-RECOVERY, rely on receipt of acknowledgments to
send additional data into the network, and will suffer degraded
performance if acknowledgments are delayed excessively.
To enable a sender to respond to potential network congestion, a
sender SHOULD cause a receiver to send an acknowledgement at least
once per RTT if there are unacknowledged ack-eliciting packets in
flight.  A sender can accomplish this by sending an IMMEDIATE_ACK
frame once per round-trip time (RTT), or it can set the Ack-Eliciting
Threshold and Request Max Ack Delay values to be no more than a
congestion window and an estimated RTT, respectively.
Receiving an acknowledgement can allow a sender to release new
packets into the network.  If a sender is designed to rely on the
timing of peer acknowledgments ("ACK clock"), delaying
acknowledgments can cause undesirable bursts of data into the
network.  A sender MUST limit such bursts.  In keeping with
Section 7.7 of QUIC-RECOVERY, a sender can either employ pacing or
cause a receiver to send an acknowledgement for at least each initial
congestion window of received data.
Acknowledgements are fundamental to reliability in QUIC.
Consequently, delaying or reducing the frequency of acknowledgments
can cause loss detection at the sender to be delayed.
A QUIC sender detects loss using packet thresholds on receiving an
acknowledgement (Section 6.1.1 of QUIC-RECOVERY); delaying the
acknowledgement therefore delays this method of detecting losses.
Reducing acknowledgement frequency reduces the number of RTT samples
that a sender receives (Section 5 of QUIC-RECOVERY), making a
sender's RTT estimate less responsive to changes in the path's RTT.
As a result, any mechanisms that rely on an accurate RTT estimate,
such as time-threshold loss detection (Section 6.1.2 of QUIC-
RECOVERY) or Probe Timeout (Section 6.2 of QUIC-RECOVERY), will be
less responsive to changes in the path's RTT, resulting in either
delayed or unnecessary packet transmissions.
To limit these consequences of reduced acknowledgement frequency, a
sender SHOULD cause a receiver to send an acknowledgement at least
once per RTT if there are unacknowledged ack-eliciting packets in
flight.  A sender can accomplish this by sending an IMMEDIATE_ACK
frame once per round-trip time (RTT), or it can set the Ack-Eliciting
Threshold and Request Max Ack Delay values to be no more than a
congestion window and an estimated RTT, respectively.