jiebi/CodeConvo huggingface dataset:
There are multiple mentions of spurious retransmission in the recovery draft, but no text on what detecting this may require from an implementation. Maybe there should be? "Spuriously declaring packets as lost leads to unnecessary retransmissions and may result in degraded performance due to the actions of the congestion controller upon detecting loss. Implementations that detect spurious retransmissions and increase the reordering threshold in packets or time MAY choose to start with smaller initial reordering thresholds to minimize recovery latency." From comments on
Some discussion of implementation strategies here would absolutely be helpful; it doesn't obviously fall out of Quinn's current architecture.
NAME and NAME indicated my effort at adding more text wasn't that useful, so I'm closing this.
As of URL, if the client's Initial is lost, there is no required behavior that leads to it being retransmitted. Because the client has received no ACKs, no is set, and no loss is detected, leading to the client only sending probe packets, which are not required to bear any particular data ("A probe packet MAY carry retransmitted unacknowledged data") so long as they are ACK-eliciting. The server, having no connection state, cannot do anything with these packets. Quinn probes are currently a single PING frame (plus padding, when appropriate) for simplicity, so implementing the new logic immediately broke our tests. If recovery logic now requires probe packets to bear specific data to function correctly in certain circumstances, this should be specified explicitly.
I also hit this issue too. After several hours tries and error, I came to the conclusion that implementation should mark the in-flight Initial packet lost in this case, and resend it. Draft-22 has explicit statement that implementation should not mark in-flight crypto packet lost, but draft-23 does not have it anymore. BTW, draft-23 does not allow PING in Initial or Handshake packet.
Why not retransmit the data without declaring the in-flight packet lost?
I think it is rather implementation detail to internally mark in-flight packet lost but keep it in order to catch late ACK.
There's a related problem where if the server's first flight goes missing, the only response the new logic has is to send a probe at unspecified, i.e. highest known, encryption level.
The server will generally have 1-RTT keys at this point, so it sends a 1-RTT probe, which the client, not having received anything prior, cannot decrypt and drops. As above, this was previously handled by explicitly tracking whether any data was in-flight, but that got removed.
The packets should not be marked as lost, but data needs to be retransmitted before sending a PING. I sent out a PR , feel free to make further suggestions there.
I'm marking this design because it adds some normative language, though I'll note that it doesn't change what I originally intended the text to say.
As discussed in the virtual interim 2022-12-16, it is possible that extensions might be tempted to re-use HPKE public keys created for MLS. Even within MLS, we have two different public-key encryption cases, Welcome and Commit (though these currently use different keys). Currently, for Commit encryption, the HPKE parameter is set to the group context. The content of is actually undefined for Welcome encryption (!) This PR makes the following changes: Define general and functions that populte the HPKE parameter with an encoded label and context Apply these to Commit and Welcome encryption: Commit: label="UpdatePathNode", context= Welcome: label="Welcome", context= Add an IANA registry of public-key encryption labels (as for signing) There is also a bit of reorganization of the Ciphersuites section to split it into three subsections.
Note that any previous analysis of MLS had to take this particular reuse into account. This can't be the case for any arbitrary extensions reusing the key. So if MLS uses an extension that reuses the key, none of the existing analysis applies, which is a much bigger issue. This said, this reuse did decrease security -- it weakened FS in that a joiner had to hold on to their HPKE leaf key, which also allowed to decrypt the init secret from the welcome message, long after joining.
I think this captures what we discussed during the interim. FWIW, I'm wondering if we didn't have a domain separation issue in the past when we only had one HPKE public key in KeyPackages and used it for both the Welcome subsequently for UpdatePath as well. Either way, I think this is a good defense-in-depth mechanism.
Clarify meaning of 0 in the datagram transport parameter, based on the discussion at IETF 110. This follows one of the suggestions to have 0 mean no datagrams, and > 0 mean support for datagrams. This should be a non-breaking change with existing implementations.
This change is very surprising. Why not take the approach that was discussed in the issue, wherein the limit describes the payload size, and presence of the transport parameter indicates extension support? It's not useful to have a bunch of ill-formed possibilities where the limit describes the frame size but is smaller than needed to encode the frame. See also URL
NAME this is based on the WG discussion, minuted here: URL
So it comes down to preserving the weird semantics nobody wants because renumbering the TP is aesthetically unappealing? That seems like a shame. Unless saying "I support it, but you can't send it" is actually useful somehow?
Right now, the TP specifies a maximum frame size, including frame type, length and payload. This makes certain values invalid (0, 1?). Also, since this values practically is a kind of flow control, indicating how much data I'm willing to receive at a time, it's the payload length that's important here, not the framing. For these reasons, I'm arguing to change this to specifying a maximum payload length. Then, the question of what a value of zero means. Should a value of 0 be the same thing as not present or should it mean that only 0 length datagrams are allowed? I think it is simpler to say that a value of zero is the same as not present (i.e. disabled). (Issue copied from , by NAME on 2019-11-18)
from NAME on 2019-11-18: Wasn't the legality of zero-length frames agreed upon in URL I didn't even realize that the parameter wasn't payload size. Strongly agree that it should be.
from NAME on 2019-11-19: I agree that is simpler to define zero to mean not allowed, but it is highly confusing to be forced to require a length of one if you specifically only use 0 length datagrams for heatbeats or similar. It would be better to a have separate indicator to reject datagrams altogether.
Isn't this indicated by omitting the transport parameter entirely?
NAME If omitting the parameter indicates that data frames are not permitted, then this achieves the purpose, yes. So there is no need to have 0 means disabling. Hence it is better to have 0 mean 0 length, but still valid. This is the most natural representation, and there is a an actual meaning ful use of these, namely heartbeats. Further, by making it explicit that length 0 is the only valid, you can have a fast or lightweight implementation.
I wonder if we need this limit at all. It introduces implementation complexity and it's unclear why any implementation would like to limit the size of DATAGRAM frames (or payloads) they can receive. We already have for implementations that have a limit on their stack size. Does anyone have a use-case where they'd like to limit the size of DATAGRAM frames (or payloads) specifically?
I'm all for removing the limit. As I have it coded up in MsQuic, I always send a max value. It's not even configurable by the app. I don't think "I have a buffering limit" a good reason to have a limit. It's limited by a single packet already. That should be enough IMO.
I don't think it's particularly useful to set this smaller than a packet, given that a reasonable buffer is bigger than that, and I'm not sure what a peer could usefully do with the information that the limit is larger than a packet. :+1:
The more I think about it, limiting the size of the QUIC DATAGRAMS (frames or payloads) is weird. In part because there is no wiresignal to describe a max number of DATAGRAMS per packet, and we seem to have discounted the need for an internal API to configure how frames are composed in a packet. I don't hear anyone with use cases; removing the limit removes a whole load of edge cases and/or implementation defined behaviour. I think the simplification would be a win.
NAME One reason you might want to limit Datagrams is if you have a slottet buffering system for datagrams, for example 1K, or 256 bytes per datagram, or a paged system of multiple datagrams. In this case you cannot always fit a whole package into the buffering.
For streams you have an entirely different mechanism, and other frames are generally not large.
NAME that's an interesting thought - is this something you're planning on implementing and have a use for, or is it more of a thought experiment?
NAME Unfortunately I've had to push back on my QUIC dev, but I did work with an emulation of netmap, on top of select / poll, that uses slots to receive packets from the OS without context switching. Sometimes a packet needs to span multiple slots, but often it just drops into a 2K slot or so. I could imagine many use cases where datagrams are redistributed to other threads or processes in a similar manner without having an explicit use case. I believe there are now several kernel interfaces that uses a similar approach.
As far as I know, those kernel interfaces all use slots that are larger than the interface MTU, so I don't think they'd require reducing the payload size. Am I missing something?
Well, I can only speak for netmap, but it works by having the user specify one large memory block divided into a number of equal sized slots of a user specified size. Each slot has a control record with some flags. A record can indicate that the slot is partial and reference the next slot in the packet. So you can can choose a slot of 64K and waste a lot of memory, or choose a slot of 512 bytes and frequently have to deal with fragmenting. Or chose 2K for a use case where you know that you will not have larger payload and can afford the space overhead. EDIT: to clarify, it is the network driver that chooses to link data and update the control record when receiving, and the user process when writing. I'm not sure exactly how the other kernel interfaces behave but from memory it is rather much the same. For QUIC datagrams it is a bit different in that you likely want to avoid fragmentation at all cost due to the added complexity and the non-stream nature of the data. If you could afford fragmentation it would make less sense to have a limit on the size.
I support switching to TP specifying max payload size.
IIRC, the original reason this limit was introduced is to allow QUIC proxies to communicate the max datagram size it gets from the MTU between itself and the backend.
In my experience, a MASQUE proxy doesn't know that information at the time it opens a listening QUIC connection., and there is nothing to say that all backends would share the same value.
This is not for MASQUE, but for a direct QUIC-to-QUIC proxy.
OK, do you expect the max DATAGRAM frame size a backend expects to receive is smaller than the QUIC packet size it is willing to receive?
If we do have a max frame/perhaps size still defined, perhaps it makes sense to let this be updated by a new frame (MAXDATAGRAMSIZE).
A MAXDATAGRAMSIZE frame would be tricky because of reordering. In QUIC, all the MAX_FOOBAR frames can only increase limits, but here I suspect for this to be useful we'd want to be able to lower the limit which greatly increases complexity.
I expect this to be fairly uncommon, but whenever this does happen, it would make the entire connection unreliable for datagrams, meaning the applications would have to do their own MTU discovery on top of the one QUIC already does.
The default ought to be "whatever fits in the PMTU".
Yes, you have to do PMTUD. (What is pernicious here is that if you get too good at that, the value that you resolve to might not be good forever because your varints will use more bytes over time and so reduce available space.)
NAME which varints will use more bytes here?
I was thinking about flow IDs. If you are not reusing flow IDs, there is a good chance that those will get bigger over time. But I guess that assumes use of flow IDs. Mostly, I guess that DATAGRAM can be each sent in their own packet/datagram, so the size of other frames won't affect the space that is available.
Yeah the flow ID is part of the payload as far as this document is concerned. Our implementation will try to coalesce DATAGRAM frames with other frames, but that doesn't impact the max possible payload size because if the DATAGRAM frame doesn't fit with other frames we just send it in its own packet.
and
For example, allow multiple outstanding path_challenge messages to cater for potential packet loss.
We want to limit the application data sent to a reasonable threshold while we run the RRC.
RFC 9000, section 8 "address validation" (second paragraph, last line): 3 times the amount of data received.
Here is the text: Address Validation Address validation ensures that an endpoint cannot be used for a traffic amplification attack. In such an attack, a packet is sent to a server with spoofed source address information that identifies a victim. If a server generates more or larger packets in response to that packet, the attacker can use the server to send more data toward the victim than it would be able to send on its own. The primary defense against amplification attacks is verifying that a peer is able to receive packets at the transport address that it claims. Therefore, after receiving packets from an address that is not yet validated, an endpoint MUST limit the amount of data it sends to the unvalidated address to three times the amount of data received from that address. This limit on the size of responses is known as the anti-amplification limit.
These should with exception as described there.
Merging for lack of objections and as it makes further development easier.
The editorial points of : (plus from "mostly mandatory" line: find better term for "optional" and "mostly mandatory") I'm in progress of working them in, they'll come in a single PR.
The "For several operations" part I kept back pending and , which will affect that as well; the rest is pending review in , which I'll merge soon unless objections arrive.
Note to self: The "web interfaces" got lost somewhere: Dear authors, below are some of the comments to the draft, they are mostly editorials with few ones that may warrant further discussion. p1 - "Web interfaces" is often used interchangeably with "REST interfaces" it'd be better to use one only, maybe the latter. p3 - "with limited RAM and ROM" Adding a reference to the expected RAM and ROM would be good. URL provides an estimation of : -> : : Which in markdown would look like: ~ p22 - Endpoint Name "mostly mandatory" is ambiguous. If the ep name is not needed to register to an RD then it is optional. If, on the other hand the RD needs an ep name and assigns one then it is mandatory ( btw if the RD assigns the ep name, shouldn't it return that information to the ep, together with the location within the rd, once the registration is successful? ) Github Issue 190 (below) raises the need to define a bit better what the ep name is supposed to contain. To me, we could keep it mandatory and add some guidelines for its creation, in OMA for example they are defined as an URN URL p22 - URL "All we know from the draft is that it is ≤ 63 bytes. But we need to say that it is a UTF-8 string. Not entirely sure we want as an endpoint name either. Same for sector." p23 - Sorry for asking, but why is base URI only mandatory when filled by the Commissioning Tool? Wouldn't it be better to make it always mandatory to avoid the differentiation? p26 - Section 5.3.1 seems like could use a bit of reordering, I can explain this in another email or call with the authors.
p30 - lt, base and extra-attrs were already explained in 5.3 you might want to keep it as it is or change it by adding a reference to 5.3 and comment on the changes that the Registration Update has over the Registration. p33 - As per the document structure, there are two interfaces: registration and lookup. Shouldn't they be at the same level in the document? Registration is 5.3, shouldn't lookup be 5.5 instead of 6? p41 p42 - Just to clarify, is then the tuple [endpoint,sector] the way RD identifies endpoints? Or is it derived from the device's credentials? "A given endpoint that registers itself, needs to proof its possession of its unique (endpoint name, sector) value pair." - "An Endpoint (name, sector) pair is unique within the et of endpoints registered by the RD. An Endpoint MUST NOT be identified by its protocol, port or IP address as these may change over the lifetime of an Endpoint." p44 - typo address/Address
.
cc NAME NAME
Looks great!
For a simple application that wants to send "hello world" over a WebTransport stream, it would take a minimum of 2.5 round trips for the data to arrive on the server. This is still an improvement over WebSockets but I'm wondering if we could do better. If the client is able to create streams prior to receiving the CONNECT response, then it would remove a round trip. The server would need to buffer any data received on streams associated with an unknown WebTransport session. If the server responds with a non-200 status code, then these streams are discarded. I believe 0-RTT is disabled for WebTransport because CONNECT is not idempotent and would be subject to replay attacks. An idempotent CONNECT would remove a round trip when 0-RTT is used... although I'm not sure this is a good idea.
I think this is a reasonable optimization
The client requirement is odd in comparison to MASQUE / CONNECT-UDP which says "client can risk sending datagrams before getting a 200 response, a different response means that datagrams will be discarded". What is the motivation for the requirement in WebTransport? If you need some kind of ordering guarantees, then stream IDs provide that.
Discussed at IETF 110. There were no objections to buffering (with some limits), so I think we should go ahead and add it.
After the client initiates the WebTransport session by sending a CONNECT request, the server sends a 200 response and is free to create streams. This means that a server-initiated stream may arrive prior to the CONNECT response. Is the client able to infer that the session was successfully established, or does the client need to buffer this stream until it receives a proper response?
Since there are no ordering guarantees across streams on the receiver side, I'd say you have to buffer until the 200 is processed.
I agree with Lucas here. We also had a similar problem in QuicTransport (server has to buffer streams before checking the origin), and we handled it by buffering.
Discussed at IETF 110. There were no objections to buffering (with some limits), so I think we should go ahead and add it.
There has been a lot of suggestions and discussion regarding the key derivation latelely. Below is an attempt to summarize all the current suggstions that currently seems relatively likely to be done. This is not an issue in itself but I think it is good to have somewhere to discucess the complete changes to the key derivation. Suggested to permute the order of element in the TH2 sequence to be more resistant to any future collision attacks in hash functions. Exact order unclear but suggested to put the random and unpredictable GY first. Suggested to use PLAINTEXT instead of CIPHERTEXT in transcript hashes to simplify formal verification. Endpoints agree on that they received the same plaintext rather then the same cipertext.Suggested to use a salt derived from PRK in Extract to avoid usign the PRK keys directly in both Extract and ExpandSuggested to use unsigned integer instead of text string for label to save memory.Suggestion to add a final key derivation PRKout and use that in the exporter. Note that the exporter would also have uint labelsSuggestion to base Key-Update on Expand instead of extract. The label would need IANA registration. hashlength is a new variable that the implementation needs to keep track of. Should considered whether using Exporter for KeyUpdate has any disadvantages. Suggested that more info on how an application protocol is supposed to use KeyUpdate and Exporter together is needed.The EDHOC internal key derivations (i.e., not using the Exporter) would then be something like below. The exact labels should be discussed. Note that that only some of the derivations actually need a label to avoid collisionIncluding KeyUpdate, the IANA registered exporter labels could be:
Looks good. I had different naming in mind of the salts: PRK3e2m = Extract( SALT3e2m, GRX ) PRK3m = Extract( SALT3m, GIY ) where SALT3e2m = EDHOC-KDF( PRK2e, TH2, 0, h'', hashlength ) SALT3m = EDHOC-KDF( PRK3e2m, TH3, 0, h'', hashlength ) With this the salts are named by its use rather than from what key they are derived.
The labels was just to illustrate the fact that they don't need to be separate.
I think it is still better to just use 0-7. That makes it very easy for the reader to see that there are no collisions.
NAME Was this a comment on my proposed names of salts, or a separate comment? (I agree labels 0-7 is good.)
It was a separate comments. Your suggestion for salt naming is fine with me. Some other non-technical naming things that a hidden in my summary above I think keylength should use different terms in the derivation of K4 and OSCORE Master Secret. The current naming gives the idea that they are the same. I think the contexts in the MAC2 and MAC3 derivations should be given names. Should consider collection all the EDHOC-KDF derivations in a single place.
Might want to derive: K4 = EDHOC-KDF( PRK3m, TH4, 0, h'', keylength ) IV4 = EDHOC-KDF( PRK3m, TH4, 0, h'', ivlength ) to avoid that the applicaiton gets K4 and IV4. Another alternative is to use K4 = EDHOC-Exporter( 7, h'', keylength ) IV4 = EDHOC-Exporter( 8, h'', ivlength ) and forbid the application to use the labels for K4 and IV4
Editorial; should be: K4 = EDHOC-KDF( PRK3m, TH4, 1, h'', keylength ) IV4 = EDHOC-KDF( PRK3m, TH4, 2, h'', ivlength ) or, if we want all KDF labels distinct: K4 = EDHOC-KDF( PRK3m, TH4, 8, h'', keylength ) IV4 = EDHOC-KDF( PRK3m, TH4, 9, h'', ivlength ) Alternatively, application access to Exporter only allows uint with: K4 = EDHOC-Exporter( -1, h'', keylength ) IV4 = EDHOC-Exporter( -2, h'', ivlength )
Trying to compile the suggestions.