jiebi/CodeConvo huggingface dataset:
Code change:
SHOULD be able to read longer connection IDs from other QUIC versions in order to properly form a version negotiation packet. <del> The Destination Connection ID field follows the DCID Length field and is between 0 and 20 bytes in length. negotiating-connection- ids describes the use of this field in more detail. </del> <ins> The Destination Connection ID field follows the Destination Connection ID Length field and is between 0 and 20 bytes in length.  negotiating-connection-ids describes the use of this field in more detail. </ins> The byte following the Destination Connection ID contains the length in bytes of the Source Connection ID field that follows it.

Related discussion:
text: Editorial NiTs when reviewing the invariant text. These NiTs address readability issues, and do not intend to change the meaning of the text.
Is there an associated issue?
Thanks Gorry, these are all quite helpful. 1) I understand the acronyms DCID and SCID, but these aren't really expanded on first use in sect 5.1. 2) There's mention of "the high bit", but there doesn't seem to be any statement of bit order saying. Is it worth declaring network byte/bit order earlier in section 4? This also clarifies the placement of fields within a byte, which was basically assumed knowledge otherwise. 3) Personally, I don't think: "The length of the Destination Connection ID is not specified" is a great choice of words. Surely, this needs to be specified somewhere for a protocol instance to use it? This reads better as something like: The length of the Destination Connection ID is not encoded in packets with a short header and is not constrained by this specification. 4) The brackets here seem to point to connection IDs in general, not the DCID "(see Section 5.3)". Is it worth writing that as "Connection IDs are described in Section 5", rather than use brackets? The same is true for versions. 5) In section 5.3: "the wrong endpoint" is taken as the wrong transport endpoint, but some of our IP and subIP colleagues would use the term endpoint for other uses, so one insertion of "transport" would perhaps help? 6) In the appendix A, this: "QUIC forbids acknowledgments of packets" - still has me wondering what is actually intended to be understood, because the list of bullets is not guarenteed ... : /The last packet before a long period of quiescence might be assumed to contain an acknowledgment (it should be assumed that QUIC could allow acknowledgments of packets that only contain ACK frames)/ I'm not sure what to say here I think the combination of not... forbids threw me off the meaning. Changes proposed in
Thanks for changing SCID/DCID to "Source Connection ID" and "Destination Connection ID", that seems like a good change.
Code change:
However, all have an internal inconsistency or an unrecognized suffix key/ <del> value, so a recipient MUST treat the Internet Date/Time Format string as erroneous. </del> <ins> value, so a recipient MUST treat the IXDTF string as erroneous. </ins> Note that this does not mean that an application is disallowed to perform additional processing on elective suffix tags, e.g., asking

Related discussion:
text: We need a name that people will use until they can say RFC9999 timestamps (and maybe even when they can).
"Extended Timestamps" or "Extended Internet Timestamps" ?
RFC 3339 says "Internet Date/Time Format". That is what sedate says at the moment as well, but that can't stay the same. So I'm considering moving to: Internet Date/Time Format Extended The alternative .... Extended Internet Date/Time Format sounds like a date/time format for the Extended Internet (EI is actually in use as "Extensible Internet", which this shouldn't be confused with). I'd rather call it Fritz, but I haven't found a way to retronym this...
Thanks
Code change:
The "version" and "ciphersuite" values in the Welcome MUST be the same as those used by the old group. <del> Each LeafNode in the new group's tree MUST be a leaf in the old group's tree at the epoch indicated in the PreSharedKey. </del> <ins> Each LeafNode in a new subgroup MUST match some LeafNode in the original group.  In this context, a pair of LeafNodes is said to "match" if the identifiers presented by their respective credentials are considered equivalent by the application. </ins> In addition, to avoid key re-use, the "psk_nonce" included in the "PreSharedKeyID" object MUST be a randomly sampled nonce of length

Related discussion:
text: Based on , it should be made explicit that sub-group branching requires a fresh key package for each member. I think there is still a conflict of wording here that I would like feedback on because of this line (which is what triggered me thinking there was a bug)
Thanks for the PR and thanks for digging into this. I think what the sentence is meant to say is that the receiver MUST check that the members of the new group are a subset of the members of the old group at that specific epoch. Since we're using new LeafNodes in the new group and we don't have a fixed notion of identity (at least not within MLS), we have to refer to the application/the AS to ensure that the identity expressed by the credential in each LeafNode in the subgroup has a corresponding identity (i.e. one that represents the same party) expressed by a credential in a LeafNode in the original group. Maybe something like: This leaves the possibility of a collision, where the identifiers in a given leaf in the new group would be valid updates to more than one member in the old group. This should probably be forbidden in general, although outside of this case it's not really a problem, because updates are always specific to one leaf. Alternatively, we could somehow include LeafNodeRefs that specifically indicate which member represents which other member. Or just leave it up to the application to sort this all out.
Curious what you think about saying the the credential has to be equal? We can determine equality of the credential itself just by doing tls serialization.
Sure, requiring the credential to be exactly the same would be a simple way to figure out if it's the same party in both groups. However, I can imagine that some applications will want distinct credentials for each group. If we mandate that credentials are the same, this would force the application to use the same credential in every key package/LeafNode. Since subgroups can potentially be branched at any time and in any group, all key packages that a party publishes then have to have the same credential as all of the groups that party is in, so another party branching a subgroup can get hold of a KeyPackage with the same credential as the group the subgroup is branched off of. Personally, I'd prefer asking the application to check that the identities of the new group are a subset of the identities of the original group.
Code change:
The "version" and "ciphersuite" values in the Welcome MUST be the same as those used by the old group. <del> Each LeafNode in the new group's tree MUST be a leaf in the old group's tree at the epoch indicated in the PreSharedKey. </del> <ins> Each LeafNode in a new subgroup MUST match some LeafNode in the original group.  In this context, a pair of LeafNodes is said to "match" if the identifiers presented by their respective credentials are considered equivalent by the application. </ins> In addition, to avoid key re-use, the "psk_nonce" included in the "PreSharedKeyID" object MUST be a randomly sampled nonce of length

Related discussion:
text: I can see that being an issue, although I am somewhat worried about us not being opinionated in same way about how to determine equality here, mainly because of interop type things. We punt on how to validate the credentials as well in general so I guess this is in line with prior decisions
Hey NAME and NAME check the change I just posted. NAME and I put that together, it's a bit tough to get a generic definition of equivalency.
There is currently a logical flaw in the subgroup branching flow. In order to branch, you need to compose a message . A Welcome message includes which is defined as follows: The problem with this is that the field of type cannot be made for existing members of the group because the leaf nodes are now types. is now only used to add new members to the group, so we can't reuse this for sub group initialization.
I'm not sure I understand correctly. Are you suggesting when branching a group off of another group, one re-uses the leaves of the old group? If so, that's not how it's meant to work. The new group created in the process of subgroup branching should be independent from the "parent" group safe for the PSK, which all member have to provide upon creation of the group. In fact one should never use the same key material in the leaves of multiple groups except when the supply of key packages that a party provides via the DS is exhausted (see Section 17.4).
That was part of my question. When there were key packages in the leaves, you theoretically could fork the group from a given point in time and reuse the current tree, but I wasn't sure if that was a proper solution or not. Sounds like it isn't, but either way, I think there should be clarification in the document about this. Let me write up a quick PR on it.
NAME opened URL Would like your feedback there. We have conflicting wording here as well. What threw me off in the first place was the line indicating that LeafNodes should be the same in the sub-group, and if we are fetching new key packages they will not be the same.
NAME Konrad, could you safely derive new keys for the members of the subgroup instead? If so, that would be an enormous increase in efficiency and (especially in the federated case) make group creation timing consistent enough to do dynamically.
NAME Nice idea!
Code change:
The "version" and "ciphersuite" values in the Welcome MUST be the same as those used by the old group. <del> Each LeafNode in the new group's tree MUST be a leaf in the old group's tree at the epoch indicated in the PreSharedKey. </del> <ins> Each LeafNode in a new subgroup MUST match some LeafNode in the original group.  In this context, a pair of LeafNodes is said to "match" if the identifiers presented by their respective credentials are considered equivalent by the application. </ins> In addition, to avoid key re-use, the "psk_nonce" included in the "PreSharedKeyID" object MUST be a randomly sampled nonce of length

Related discussion:
text: But it's tricky, since we're dealing with public key pairs that the creator of the new group doesn't know the private keys for. If we were only using HPKE schemes that support updateable public key encryption, the creator of the new group could indeed update the key in the existing group and send the "delta" encrypted to each corresponding member (which would still represent an overhead, but we might be able to optimize this somehow). This is similar to the RTreeKEM mechanism that NAME et al. proposed at some point. To go all the way here, we'd have to do the same for the public key material in the credentials as well, which is not something we had considered before. It would be a new primitive, something like an updateable signature scheme? Probably something to consider for HPKE/MLS 2.0. We could also allow the re-use of LeadNodes for this particular case. It would be the simpler solution and I don't think it would be catastrophic, since we have to assume that LeafNode (or KeyPackage) key material is re-used in the "last resort" scenario anyway, but this would definitely be something we should discuss on the mailing list, as it would break a pretty basic assumption we've made until now.
Just catching up on this. NAME nice catch here. It seems like we have two options here: Require new KeyPackages (as in ) Invent some way to make a new group based on LeafNodes alone I probably agree that we should do (1) in the short term, but (2) also has some appeal -- it doesn't feel right to require new KeyPackages for folks you're already in touch with -- despite it probably being more spec text. So here's a sketch of (2) in the interest of completeness. Suppose we defined a message of the following form: ... which would pretty closely parallel to Commit. You would send it in MLSMessage and require a in MLSMessageAuth. Where Commit instructs the recipient to create a new epoch from the current one within the same group, Branch would instruct the recipient to create a new epoch that is the first epoch of a new group. In both cases, you would form an epoch / GroupContext as specified, then use the to ratchet into the new epoch.
Code change:
The "version" and "ciphersuite" values in the Welcome MUST be the same as those used by the old group. <del> Each LeafNode in the new group's tree MUST be a leaf in the old group's tree at the epoch indicated in the PreSharedKey. </del> <ins> Each LeafNode in a new subgroup MUST match some LeafNode in the original group.  In this context, a pair of LeafNodes is said to "match" if the identifiers presented by their respective credentials are considered equivalent by the application. </ins> In addition, to avoid key re-use, the "psk_nonce" included in the "PreSharedKeyID" object MUST be a randomly sampled nonce of length

Related discussion:
text: I'm pretty sure that would work and have the expected security properties, though obviously formal analysis would be appreciated. The main awkardness that occurs to me is that any members joining after the branch would have to provide new key packages. And obviously, it would require a fair bit more spec text. All that said, though, the exercise of working all that out has increased my impression that we should do (1) for now. If we want something like (2) later, it can be done pretty cleanly in an extension.
Fixed by
Well put!
Code change:
sender should not consider itself application limited if it would have fully utilized the congestion window without pacing delay. <del> Sending multiple packets into the network without any delay between them creates a packet burst that might cause short-term congestion and losses.  Implementations SHOULD either use pacing or reduce their congestion window to limit such bursts to minimum of 10 * kMaxDatagramSize and max(2* kMaxDatagramSize, 14720)), the same as the recommended initial congestion window. </del> A sender MAY implement alternative mechanisms to update its congestion window after periods of under-utilization, such as those proposed for TCP in RFC7661.

Related discussion:
text: Came from the discussion of
NAME : I realized after we did this PR yesterday that this does not I It reduces burstiness in general, which is still a good thing, but it doesn't change slow start increase. I'll send a PR or ask Vidhi if she wants to.
When an ACK arrives that acknowledges N packets (where N could be very large), QUIC will increase the congestion window by more than 1MSS per congestion window during Congestion Avoidance causing bursty send. This could also affect the slow start. if (congestionwindow < ssthresh): // Slow start. congestionwindow += URL else: // Congestion avoidance. congestionwindow += kMaxDatagramSize * URL / congestionwindow One could argue that QUIC ACKs every other packet, but there ACK packet could get lost or ACK for ACK packet could get lost. The draft is probably assuming packet pacing but not all implementations support that. I discussed this with Jana and he thinks that we should add something in pseudo code or some specific text in the draft.
PR added more specific text on avoiding bursts into the network, FYI.
Please note that PR addresses the issue of app-limited or pacing-limited cases to not increase cwnd when it is under utilized. My concern is cwnd being inflated due to an ACK frame containing ACKs for large number of packets.
NAME : The problem here is that we don't talk about burstiness that can occur when pacing is not in place, and a stretch-ack is received. PRR handles this, but we don't do PRR in the draft.
No we need to spend lots of time on Reno as everybody appears to be doing Cubic or BBR?
Cubic has this problem. BBR cannot be assumed.
Except Cubic opens the window based on time spent, not just ack received
I don't get the scenario. It seems that NAME is worried about receiving an ACK for a large number of packets. This is not a congestion window problem, because it happens even if the congestion windows size remains constant. The root cause is that the ACK of N packets now reduces the number of packet in flow by N. So this is strictly about pacing, not about congestion control.
Code change:
sender should not consider itself application limited if it would have fully utilized the congestion window without pacing delay. <del> Sending multiple packets into the network without any delay between them creates a packet burst that might cause short-term congestion and losses.  Implementations SHOULD either use pacing or reduce their congestion window to limit such bursts to minimum of 10 * kMaxDatagramSize and max(2* kMaxDatagramSize, 14720)), the same as the recommended initial congestion window. </del> A sender MAY implement alternative mechanisms to update its congestion window after periods of under-utilization, such as those proposed for TCP in RFC7661.

Related discussion:
text: NAME yes, and it can and still does burst because of stretch acks
Not just stretch Acks -- deliberate Ack compression by the network happens too. Very few alternative to pacing.
URL describes
Discussed in Cupertino. NAME to prepare a PR that hopefully can incorporate some ideas from RFC3465 to add safety features for non-pacing stacks.
But 3465 does not solve the burst problem. It limits the growth of the window, but does not do pacing of any kind.
I think we can change to something like this
NAME I think we need to do 2MSS clamping per ACK received and not per packet acked. I am thinking that we can remove OnPacketAckedCC and instead do OnAckReceivedCC. Pseudocode change something like this:
I think we can keep but just move cwnd update to as you propose.
Yup, that's an important missing fix, though it needs to be applied only when not pacing. Additionally, helps with burstiness in general. NAME : I would make a variable, and apply the min here when not pacing.
The proposed change has the effect of reducing the effective Window if ACKs are compressed. Say you have a window of 12 packets, acks are delayed, and a single ACK acknowledges 12 packets. Now you have a window of 2 packets, your transmission rate is 6 times slower than before, and it will take you 10 RTT to recover. Is that really what you want?
NAME This is what ABC (RFC 3465) does for TCP. ACK compression has side-effects, and the way to deal with it is, as you say, to pace packets out. This reduction in performance is only when the endpoint does not pace, and it is the only way to limit bursts in that case.
Also, ACK compression does not necessarily imply ACK pruning. I have traces in which 1 RTT worth of ACKs is compressed and delivered back to back. In that case, the proposed algorithm will do nothing: you will get 2 MSS added for each ACK, back to back within less than 1 ms.
FYI.
Code change:
sender should not consider itself application limited if it would have fully utilized the congestion window without pacing delay. <del> Sending multiple packets into the network without any delay between them creates a packet burst that might cause short-term congestion and losses.  Implementations SHOULD either use pacing or reduce their congestion window to limit such bursts to minimum of 10 * kMaxDatagramSize and max(2* kMaxDatagramSize, 14720)), the same as the recommended initial congestion window. </del> A sender MAY implement alternative mechanisms to update its congestion window after periods of under-utilization, such as those proposed for TCP in RFC7661.

Related discussion:
text: FreeBSD implements rfc3465 (and enabled by default) URL Linux removed tcpabc implementation but saying it does in a different way: URL Considering FreeBSD doesn't have pacing by default, I think current proposed algorithm is not worse (or aggressive) than what tcp implementation does.
NAME I am working on the PR for pseudocode changes.
I agree that when not pacing, using an ABC limit is appropriate. FWIW Windows TCP was using a limit of 4 and not 2 for a long time and now uses 8 which is pretty close to IW10 burst in slow start phase.
For the WG: The proposed PR attempts to include a mechanism to limit CWND increase during slow start when not pacing. I'll note that by itself, it does not guarantee we follow the existing MUST: "Implementations MUST either use pacing or limit such bursts to the initial congestion window, which is recommended to be the minimum of 10 maxdatagramsize and max(2 maxdatagramsize, 14720)), where maxdatagramsize is the current maximum size of a datagram for the connection, not including UDP or IP overhead." I don't think we should add this to the pseudocode, because we say SHOULD pace packets and the pseudocode currently only covers SHOULDs, not MAYs. Adding all MAYs to the pseudocode is too complex. I don't have a concern about adding a reference to the existing RFC as a mechanism that MAY help limit bursts during slow start. I'd like a decision from the WG on whether to include this change and if so, some criteria on what should be included going forward.
I agree with the comment that burst from ACK compression or Stretch-ACKs result in bursts. I think the ways to change that is implement a maximum burst size or use pacing. The bursts do not appear a CC function to me.
Bringing conversation in from the list: With regards to , one of the things we found in testing was that it could (a) still allow an implementation to be vulnerable to too much bursting (the original problem), but more importantly (b) locked receivers into an ack pattern of acking every other packet. My main concern is that this causes problems if receivers want to experiment with and change acking strategies going forward in way that otherwise would be fine.