Change:
This simplifies the definition of kPersistentThreshold from an exponent to a PTO multiplier.
I think let's keep that separate. I think might end up with different outcomes anyways.
Curious, why wasn't this listed under draft-20 C.1? URL
Oversight. Fixed now in URL . Thanks!
Both Linux and Windows TCP arm a single PTO for TLP and then arm the RTO. The recommendation in RACK draft URL is also for doing a single TLP. If we keep the value as 2 then we need to add a justification statement for deviating from TCP. It would be good to find out if Yuchung has a any data on why Linux settled on a single TLP (PTO).
The recovery draft currently says MUST send 1, and MAY send up to 2 (URL). Is that not good enough?
The issue is about the length of the persistent congestion period. If the -16 draft recommended 1 TLP before the first RTO instead of 2 TLPs before RTO(See URL), then the natural persistent congestion time period would be 2 PTO instead of 3 PTO. As an editorial issue, as Jana and I discussed yesterday, we should make kPersistentCongestionThreshold a direct constant, not an exponent - 1, because it's making this more confusing to think about.
Last time I talked to Yuchung, I remember him saying they chose 1 somewhat arbitrarily. I'm going to argue for 2 (3 * PTO) as a good number because it provides enough time for the PTO timer to go off twice now that the PTO has exponential backoff. This is in some ways more similar to 1 TLP and then 1RTO in terms of network load. Additionally, I believe making it only 1 would allow a single tail packet to be lost and persistent congestion to be declared if there was some RTT variation? That certainly seems undesirable. Maybe the rationale for why this value was chosen should be clarified further?
facepalm NAME of course. (Also, FWIW, PR makes this a simple multiplier, so let's just use that language.) So I think I may have the opposite concern to NAME The mechanisms we are talking about are subtly different, so let's just talk about behavior instead of the mechanisms. TCP RTO uses a minRTO while PTO does not. As a result, in the common case, persistent congestion for TCP will be declared at minRTO (or later). We risk declaring persistent congestion too soon with QUIC, even with 3 PTOs, since minRTO is at least 200ms for TCP, AFAIK. QUIC might be too aggressive in declaring persistent congestion. This might be actually bad for performance. Anecdotally, I remember that Yuchung had tried turning on and found that it increased spurious RTOs. While spurious PTOs are not bad in QUIC, we don't yet have a way to detect spuriously declared persistent congestion. This makes me wonder if we should use a min value for declaring persistent congestion. I think this is justifiable as based on best current practice that we know in TCP and that declaring congestion too soon might hurt throughput. (Note that a min is still unnecessary for QUIC PTO because spurious PTOs have only minor bandwidth cost.) As a strawman, I'll propose that we set the persistent congestion time period to max(PTO, 200ms). Why PTO and not 3 * PTO? Because I believe (and someone can prove me wrong) that TCP actually does use a single RTO period for declaring an RTO, despite potentially sending TLPs in that time period. From the RACK draft, my understanding is that sending TLPs in TCP does not actually change the RTO timer, which runs for the same amount of time (from the last ack received in practice or the earliest outstanding data).
I edited the title to indicate 2 or 3 instead of 1 or 2, now that has landed. Yes, TLPs in TCP do not change RTO. One of the reasons TLP was added in the way it was is that it doesn't really interact with RTO at all, which made it convenient. My understanding is that TCP uses 200ms because it's 2x the expected delayed ACK timer? If so, given we're including maxackdelay in the PTO value, shouldn't that be sufficient(assuming we keep a 2x or larger multiplier on PTO)?
The TCP minRTO is 1 sec per RFC 6298, but Linux uses 200 ms if I'm not mistaken. I could be wrong, but don't think it was based on the delayed ack timer, since the delayed ack timer value is typically 200ms. (I'll confirm this.) My high order point is that we probably do not want QUIC to declare persistent congestion (and collapse cwnd) sooner than TCP currently does in the network. Using only the PTO values might cause us to do that, and it might cause unnecessary throughput degradation. IMO we don't lose anything by declaring persistent congestion similar to TCP. PTOs still fire the way they always do, and recovery continues the way it always does. Other machinery (such as when to call a connection or path dead for instance) can continue to use PTOs, but we just declare persistent congestion based on what we know TCP behavior to be in the wild.
Well the problem with those fixed constants is that they don't work in DC for example. This has been a big pain point with TCP where we apply different minrto configs for DC and Internet. I actually liked the current QUIC definition of persistent congestion because it was dependent on actual path RTT than on some fixed value. NAME the value is not 3PTO because there is exponential doubling? Even if maxackDelay is advertised as 0 by receiver then QUIC is effectively declaring persistent congestion after Srtt + 2Srtt + 4Srtt? So 5 Srtt and that seems long enough?
Discussed in London; best to bring this up at a full IETF meeting.
Montreal: close with no action
Currently the recovery draft 19 requires the sender to either reducing the cwnd to initial cwnd OR pace the send after idle. Idle is defined as "it ceases to send data and has no bytes in flight" . This will mean that just a very small delay in the sending application thread will cause QUIC to reset its cwnd when not pacing. IMO we need to define an interval for this which could be a function of kGranularity or as PTO. FWIW in TCP the idle period for cwnd restart is defined as one retransmission timeout (reference URL Section 4.1).
However, this still has the problem that if I don't pace and I have a window of 100 packets, I can burst 100 packets into the network. This seems suboptimal from both the sender and the network's perspective. Now that I think about it, I think the concept of limiting the size of a burst is the key here, not so much idleness, and I think defining idle more accurately is both not necessary and likely to be difficult. Ideally you would send no more than max(10, BW * kTimerGranularity) packets into the network at once.
NAME pacing or limiting burst is the right idea. TCP RFC limits the burst to IW but it says do that after an idle period of an RTO. Presumably the authors and community considered that period to be small enough that network capacity / conditions hadn't changed drastically. Implementation wise how would we do this without a timer? If not pacing, when would one decide to cut cwnd back to restart cwnd (say IW)?
Your interpretation of RFC5661 looks correct and is different from what's in the QUIC recovery doc. But the implications are that it's ok to send an unlimited sized burst into the network as long as you only drained the pipe very recently. I think that recommendation is a poor one unless the window sizes are fairly small(in which case they'd be close to IW10 anyway). The sender is bursting packets into the network, which previously did not cause loss because they were ack-clocked, but one or more buffers on the network doesn't have enough space for the full CWND. It seems likely this could cause losses for other flows sharing the link as well. This could happen frequently in HTTP applications such as DASH video playback. NewCWV is already referenced for decaying CWND after idle. I could change the text to say nothing about reducing CWND to IW upon full idle and instead say you shouldn't burst more than IW into the network at once ever, and senders may use pacing, ACK-clocking, and/or timers to accomplish that? URL may also be relevant to this topic?
I guess the bigger question is what are we attempting to document in this draft. Is it whats in the TCP standard RFCs OR what most current TCP implementations do OR what the Linux kernel implementation does? By documenting NewReno instead of CUBIC we've chosen to not document the state-of-the-art and erred on the side of simplicity. In the spirit of that I think handling idle period and app limited states are optimizations for better network citizenship on cetrtain type of networks and not a hard requirement. For this particular issue I'd like to point out that the Windows TCP implementation has worked fine without handling the idle period (not capping cwnd burst and neither pacing). The idle cwnd restart knob in Windows is off by defualt and I believe Google TCP turns it off on Linux as well. Yes RFC 7661 is relevant and I think I cited it on the application limited case handling issue. I think there is a fine balance between managing burst size and taking advantage of TSO like optimizations. For example in WAN and DC bulk flows it might better to burst higher than IW with TSO for CPU efficiency than to pace and on the Internet pacing might be better. But I like your direction on this since it sidesteps what is meant by "idle". How about framing it as a suggestion and not a requirement. Something along the lines of: Bursts of more than IW into the network at once may cause losses. Implemementations should consider using pacing, ACK-clocking, and/or timers to minimize bursts. Paging NAME to this issue as well.
My goal is to document the best practices and reference existing RFCs whenever possible. And yes, my understanding is that everyone disables slow start after idle, so I'd like to come up with something I think is more likely to be deployed. When QUIC was started, Cubic wasn't an RFC, which is the other reason we documented Reno instead of Cubic, FYI. I'll write up a PR based on your suggestion and hopefully we can get some text we all think is acceptable.
RFC7661 is already cited in the section, and it was added there to defer to it so we didn't have to delve into details in this doc. I think what we're really discussing here is whether we need the MUST in the sentence "When sending data after becoming idle, a sender MUST reset its congestion window to the initial congestion window". I'm in agreement with what I'm sensing here -- that we don't need a requirement, but we can discuss the implications of various decisions here and current best practices. Higher-level note, yeah, we always wanted to document current best practices, rather than what was in TCP or even linux TCP. I think most things came from linux TCP because that was what we were familiar with. FWIW, agreed that Cubic wasn't an RFC at the time, but it was close to becoming one, and we did discuss it. We decided that it was too complex and adds a ton more complexity to this document. We wanted to have something simple but good enough that implementations could have as a starting point, and then move on to more esoteric algorithms as deployments needed. (I'll disagree that Cubic is the best known CC at the moment -- unless you're talking about DC environments, it's not.)
Thanks for the discussion, I updated based on this, though I'm a bit concerned that SHOULD is too weak in regards to bursts into the network. Then again, if that's what Windows is doing, maybe we have to accept the reality of current deployments?
Given that I think that the general guidance for QUIC is "use pacing", I think a "MUST reset to IW if not paced" is actually the right thing to do to support that pacing gets implemented with QUIC from the start.
NAME pacing may be implemented and controlled outside of the transport. I have two new concerns about text around bursting packets on network: Many CDNs and edge deployments will tweak IW to be a higher value. In Windows TCP we allow config of up to 64 MSS. So if the network admin knows what they are doing and can swallow bursts then we shouldn't discourage the practice. UDP segmentation offload is reality now and for high throughput flows using hardware offload to reduce CPU will mean larger bursts trading off against pacing. We shouldn't preclude this either.
Discussed in Montreal; people to review the PR.
Discussed in Cupertino, NAME to review and determine whether that addresses (or partially addresses) this issue.
NAME NAME This issue was supposed to be but it was waiting for feedback. It's been long enough -- can we close this issue?
This looks good, but I think we need to move a lot of the rationale back into the normative text from the pseudocode. You can do that in this PR or as we

Discussion:
can be established by consecutive PTOs, but since the PTO timer is reset when a new ack-eliciting packet is sent, an explicit duration must be used to account for those cases where PTOs do not occur or <del> are substantially delayed.  This duration is the equivalent of kPersistentCongestionThreshold consecutive PTOs, and is computed as follows: ~~~ (smoothed_rtt + 4 * rttvar + max_ack_delay) * ((2 ^ kPersistentCongestionThreshold) - 1) ~~~ </del> <ins> are substantially delayed.  This duration is computed as follows: </ins> For example, assume: smoothed_rtt = 1 rttvar = 0 max_ack_delay = 0 <del> kPersistentCongestionThreshold = 2 </del> <ins> kPersistentCongestionThreshold = 3 </ins> If an eck-eliciting packet is sent at time = 0, the following scenario would illustrate persistent congestion:

The first three packets are determined to be lost when the ACK of packet 4 is received at t=8.  The congestion period is calculated as the time between the oldest and newest lost packets: (3 - 0) = 3. <del> The duration for persistent congestion is equal to: (1 * ((2 ^ kPersistentCongestionThreshold) - 1)) = 3.  Because the threshold was </del> <ins> The duration for persistent congestion is equal to: (1 * kPersistentCongestionThreshold) = 3.  Because the threshold was </ins> reached and because none of the packets between the oldest and the newest packets are acknowledged, the network is considered to have experienced persistent congestion.