Code change:
This PR aims to take the design discussions we had at the last interim, and propose a potential solution. This PR should in theory contain enough information to allow folks to implement this new design. That will allow us to perform interop testing and confirm whether the design is sound.
I'm confused by this pull request (probably because I was out of the loop for a while). My understanding, at least when I first read the proposal a month ago was that we would separate multiplexing streams and multiplexing contexts into different layers, so that methods that need context would get it (CONNECT-UDP), and features that don't (WebTransport/extended CONNECT) could use stream-associated datagrams directly.
NAME you're referring to which this PR isn't trying to address. That'll happen in a subsequent PR.
In , capsules, contexts and context IDs are explicitly defined as end-to-end and there are requirements for intermediaries to not modify them, and to not even parse them. The motivation for this was to ensure that we retain the ability to deploy extensions end-to-end without having to modify intermediaries. NAME mentioned that these requirements might be too strict, as he would like to have his intermediary be able to parse the context ID, for logging purposes.
Part of the comment from the PR is also about consistency of interface for generic HTTP libraries that can be used to build either endpoints or intermediaries. If the context ID is always present, it's overly prescriptive to say that an intermediary can't parse it, and such a requirement couldn't be enforced anyways. If context IDs are negotiated on/off, whether this is possible depends on the negotiation mechanism.
I think its fine for a component that has bytes passing through it to look at those bytes. The import thing to do is set expectations on what that means for the protocol behaviour we are defining. We don't expect intermediaries to do anything with a context ID; they should not enforce the protocol semantic rules (odd, even, reuse etc). H2 and H3 somewhat solve this problem by decoupling frame parsing from frame payload interpretation. E.g. if the form of a request or response is invalid, an intermediary shouldn't pass it on. DATAGRAM is different because it isn't defined as an actual HTTP/3 frame with frame headers and frame payload. Something that might work is to formalise that quarter stream ID is part of the envelope that intermediaries have to be able to handle the syntax of, but context ID is part of content and separate.
Unless you encrypt capsules end-to-end, I don't think any text in the spec can in practice prevent intermediaries from inspecting and modifying capsules. You can send something like GREASE capsules, though.
In , we introduce the CLOSEDATAGRAMCONTEXT capsule which allows endpoints to close a context. That message currently only contains the context ID to close. NAME suggests that we may want to add more information there. Should we differentiate between "context close" and "context registration rejected"? Should we add an error code? In particular, do folks have use cases for these features?
My first concern is with infinite loops. If the context was "garbage collected" by the recipient, the sender can simply re-create it if they still want it. However, if it was closed due to incompatibility (i.e. rejected) or hitting a resource limit (e.g. max # of contexts), reopening it in response will produce a tight infinite loop. This could be maybe be avoided by some kind of heuristic, but an explicit indication of what's going on seems better.
This issue assumes that decide to go with the two-layer design described in . Given that design, some applications might not need the multiplexing provided by context IDs. We have multiple options here: Make context ID mandatory Applications that don't need it waste a byte per datagram Negotiate the presence of context IDs using an HTTP header Context IDs would still be mandatory to implement on servers because the client might send that header Have the method determine whether context IDs are present or not This would prevent extensibility on those methods Create a REGISTERNOCONTEXT message This assumes we use the Message design from We add a register message that means that this stream does not use context IDs This allows avoiding the byte overhead without sacrificing extensibility
Making the context ID mandatory is simple, although slightly wasteful. For the variations where it is optional, I prefer the one that uses a header. It could work with a variant of where we do Header + Message, and the Header determines whether or not you have contexts, and whether or not they can be dynamic.
What are the benefits of message over header to make it optional? To me it seems like header is sufficient, because the context IDs are end-to-end, and should not affect intermediaries, making the header a natural place to put it. I think having the context ID be the default behavior and the header is to opt-out is the right decision if we were to allow removing it.
This is a bit of a bike-shed, but why would it default to having context-IDs instead of not having them?
I think the second option "Negotiate the presence of context IDs using an HTTP header" is the best personally. In terms of: "Context IDs would still be mandatory to implement on servers because the client might send that header", I assume server in this context is the final application server, not a proxy? Even so, I'd argue it's applications which decide if they need this functionality, so if a server only implemented one application and it didn't use Context-IDs, then I don't see why it would need to implement it.
Ian, in your comment, are you talking about an intermediary when you say "proxy"? If so, then an intermediary need not implement anything based on the payload of than H3 DATAGRAM, it only has to pass the data along. That may be worth a clarification.
Having context ID optional has a nice future-proof property - other uses of datagrams in HTTP/3 can decide if they want the second layer or not. I wouldn't even state it as application needing multiplexing or not. Other applications might find it very useful to include a second ((third, fourth, fifth) layer of frame header. I think its fine for us to state as much in this in the spec, while also defining an explicit second layer for multiplexing.
Yes, by proxy I meant intermediary. And I agree about what intermediaries need to do, I just wanted to ensure I understood what server meant when David used it above. Yup, whenever all this gets written up, we can make it clear that intermediaries can ignore much of this.
The method defines the semantics of datagrams for a particular request, thus I think the third option makes most sense. I don't see a point in using explicit negotiation, since in the two cases I can think of the top of the head, the answer is always clear from the method (CONNECT-IP seems to always want context IDs, WebTransport does not ever want context IDs). If some method starts out without context, but wants to add context later, it can add its own negotiation.
One set of use cases for which the method approach(3) is best are those where there are multiple context IDs but they're predefined by the method and don't need negotiation. My strawman is WebRTC.
During the 2021-04 MASQUE Interim, we discussed multiple options for registering context IDs. Without going into the specific of the encoding, there are two broad classes of solutions: "Header" design Once-at-setup registration This happens during the HTTP request/response Simplest solution is to use an HTTP header such as (encoding TBD) Example use-case: CONNECT-UDP without extensions "Message" design Mid-stream registration This can happen at any point during the lifetime of the request stream This would involve some sort of "register" message (encoding TBD, see separate issue) Example use-case: CONNECT-IP on-the-fly compression (this can't be done at tunnel setup time because the bits top be compressed aren't always known at that time) It's possible to implement once-at-setup registration using the Message design, but it isn't possible to implement mid-stream registration. Therefore I think we should go with the Message design. It would also be possible to implement both but I don't think that provides much value.
I'd like to here some more from the WebTransport use case on the issue. My understanding is that the Requests are used to create WebTransport sessions and that creation of datagram flows(contexts) could be a lot more ad-hoc / server generated after the request phase, when compared to the CONNECT-UDP case. Cc NAME
Personally, I'd vote for either "message" or "message + header" (a la priorities), but not just "header". Mid-stream annotation of context/flow/etc is quite useful.
I prefer "Message" only, since the dynamic use case seems like a fairly compelling use of sub-flows(contexts?). I'm not sure there are many cases when one allocates all sub-flows up front AND needs to use a header to do that in a flexible way. If an application needs multiple sub-flows, but they're fixed in number/format(ie: data and control), I don't think there's a need for a header or a "Message", since the application can consider that part of its internal format.
During the 2021-04 MASQUE Interim, we discussed having the ability to accept or reject registration of context IDs. An important motivation for this is the fact that context ID registrations take up memory (for example it could be a compression context which contains the data that is elided) and therefore we need to prevent endpoints from arbitrarily making their peer allocate memory. While we could flow control context ID registration, a much simpler solution would be to have a way for endpoints to close/shutdown a registration.
IIUC there's an opposing view that the constraints can be negotiated apiori. Either statically (such as defining an extension which states that there can only be N contenxts of a given type) or dynamically using something like a HTTP header. Failure to meet these constraints could lead to an HTTP/3 stream or connection error rather than a flow rejection.
NAME I've filed to discuss whether to register/negotiate at request start or mid-stream.
I think that if there is the ability to register mid-stream (), we need the ability to close/de-register/reject mid-stream as well. If we don't have mid-stream changes, then no need to negotiate. However, someone trying to register something needs to be able to know whether or not the thing they registered will be accepted.
I agree that notification of the registration status is particularly important if we allow mid-stream registration.
In draft-ietf-masque-h3-datagram-00, flow IDs are bidirectional. During the 2021-04 MASQUE Interim, we discussed the possibility of making them unidirectional. Here are the differences: Bidirectional single shared namespace for both endpoints (even for client, odd for server) it's easier to refer to peer's IDs, which makes negotiation easier Unidirectional one namespace per direction every bidirectional use needs to associate both directions somehow I'm personally leaning towards keeping them bidirectional. Unlike QUIC streams, these do not have a state machine or flow control credits, so there is no cost to only using a single direction of a bidirectional flow for unidirectional use-cases.
There's another option where flows are unidirectional but they share a namespace (cleint even/server odd). I'm undecided. And I also don't know how important it is to choose. The 2 layer design means that DATAGRAMs always have a bidirectional tie via the stream ID. On balance, I think bidirectional would be good enough today but I would be receptive to extension use cases that make a strong point of unidirectional.
Bidirectional is my personal preference, but either way could work ultimately. It seems to me that any extension that wants unidirectional can just only use this context/flow/etc ID for one direction if it wants.
I think bidirectional flows are likely overcomplicated and unnecessary. For example, if either peer can "close" a flow, this requires a much more complex state machine. In my experience, half-closed states are hard to get right. Closing unidirectional flows seems much easier to reason about. Many, perhaps most, of our use cases for flows are really unidirectional. For example, DSCP marking likely only makes sense from client to server, while timestamps for congestion control tuning likely only make sense from server to client. QUIC connection IDs are different in each direction, so QUIC header compression may be easiest to understand with unidirectional flows.
Bidirectional doesn't imply half-close states. I'm thinking of a bidirectional flow that can only be closed atomically as a whole. I don't think that's true. The default extension-less CONNECT-UDP is bidirectional. So is ECN, and so would a timestamp extension.
I would prefer bidirectional without support for half-close. I feel that having unidirectional flow/context IDs adds overhead we don't need; peers now need to track those associations and it's unclear what the benefits of having them separated out are.
This issue is conflating consent (unilateral/bilateral) with scope (unidirectional/bidirectional). IIUC there are two questions here: 1) Can I send a flow-id forever once declared, or can the endpoint tell me to stop without killing the whole stream (consent)? 2) Can the receiver of a flow-id assignment use that same flow-id when sending datagrams, or could this mean something completely different? I think the answer to (1) is "yes, it can tell me to stop" (i.e. it is negotiated while allowing speculative sending). For (2), I lean toward bidirectional, but not strongly. There are plenty of codepoints, so it seems wasteful for each endpoint to define the same semantic explicitly.
NAME the intent of this issue wan't to conflate those two things. This issue is about scope. For consent, see issue .
Conceptually, I like unidirectional better. I tell the peer that when I send with context-ID=X, it has a given semantic. For bidirectional the sender is saying both that and that if the peer sends on that ID the original sender will interpret with the same semantic. That might not even make sense. Is it possible to associate datagrams with a unidirectional stream (push?). If so, does allowing for bidirectional context imply that even though the receiver cannot send frames on the stream, it can send datagrams? Every negotiation does run the risk of loss/reordering resulting in a drop or buffering event at the receiver. The appeal of bidirectional is that it removes half of this risk in the cases where the bidirectionality is used.
To second NAME I would also prefer bidirectional without support for half-close. Given you can always not send something in one direction, and there's none of the overhead of streams(ie: flow control, etc) I can't see any benefits of unidirectional over bidirectional in this case. Associating h3-datagrams with a server push seems like unnecessary complexity to me, but you're right that it should be clarified.
During the 2021-04 MASQUE Interim, we discussed a design proposal from NAME to replace the one-layer flow ID design with a two-layer design. This design replaces the flow ID with two numbers: a stream ID and a context ID. The contents of the QUIC DATAGRAM frame would now look like: While the flow ID in the draft was per-hop, the proposal has better separation: the stream ID is per-hop, it maps to an HTTP request the context ID is end-to-end, it maps to context information inside that request Intermediaries now only look at stream IDs and can be blissfully ignorant of context IDs. In the room at the 2021-04 MASQUE Interim, multiple participants spoke in support of this design and no one raised objections. This issue exists to ensure that we have consensus on this change - please speak up if you disagree.
I am fine with the 2-layer design for CONNECT-UDP, but I am not sure if the second identifier should be in H3 DATAGRAM or should be specific to CONNECT-UDP. As I indicated to the list, webtrans and httpbis might weigh in on this with broader consideration of other use cases.
Hi NAME the topic of the optionality of context IDs is discussed in issue .
I think this is a good design, because it minimizes the core functionality of the draft, but avoids every application that needs multiple dynamically created sub-flows from re-inventing a different mechanism. Nit: I would not call this a 'Frame', since it's not an HTTP(TLV) frame.
I think this is a good change. Thanks! There are some things that I think we should still discuss, but we shouldn't hold up this change on those details.

Related discussion:
text: 6. <del> "Datagram-Flow-Id" is a List Structured Field STRUCT-FIELD, whose members MUST all be Items of type Integer.  Integers MUST be non- negative.  Its ABNF is: The "Datagram-Flow-Id" header field is used to associate one or more datagram flows with an HTTP message.  As a simple example using a single flow, the definition of an HTTP method could instruct the client to use its flow ID allocation service to allocate a new flow ID, and then the client will add the "Datagram-Flow-Id" header field to its request to communicate that value to the server.  In this example, the resulting header field could look like: List members are flow ID elements, which can be named or unnamed. One element in the list is allowed to be unnamed, but all but one elements MUST carry a name.  The name of an element is encoded in the key of the first parameter of that element (parameters are defined in Section 3.1.2 of STRUCT-FIELD).  Each name MUST NOT appear more than once in the list.  The value of the first parameter of each named element (whose corresponding key conveys the element name) MUST be of type Boolean and equal to true.  The value of the first parameter of the unnamed element MUST NOT be of type Boolean.  The ordering of the list does not carry any semantics.  For example, an HTTP method that wishes to use four datagram flows for the lifetime of its request stream could look like this: In this example, 42 is the unnamed flow, 44 represents the name "foo", 46 represents "bar", and 48 represents "baz".  Note that, since the list ordering does not carry semantics, this example can be equivalently encoded as: Even if a sender attempts to communicate the meaning of a flow before it uses it in an HTTP/3 datagram, it is possible that its peer will receive an HTTP/3 datagram with a flow ID that it does not know as it has not yet received the corresponding "Datagram-Flow-Id" header field.  (For example, this could happen if the QUIC STREAM frame that contains the "Datagram-Flow-Id" header field is reordered and arrives afer the DATAGRAM frame.)  Endpoints MUST NOT treat that scenario as an error; they MUST either silently discard the datagram or buffer it until they receive the "Datagram-Flow-Id" header field. Distinct HTTP requests MAY refer to the same flow in their respective "Datagram-Flow-Id" header fields. Note that integer structured fields can only encode values up to 10^15-1, therefore the maximum possible value of an element of the "Datagram-Flow-Id" header field is lower then the theoretical maximum value of a flow ID which is 2^62-1 due to the QUIC variable length integer encoding.  If the flow allocation service of an endpoint runs out of flow ID values lower than 10^15-1, the endpoint MUST fail the flow ID allocation.  An HTTP message that carries a "Datagram-Flow- Id" header field with a flow ID value above 10^15-1 is malformed (see Section 8.1.2.6 of H2). </del> <ins> We can provide DATAGRAM support in HTTP/2 by defining the CAPSULE frame in HTTP/2. </ins> <del> 7. </del> <ins> We can provide DATAGRAM support in HTTP/1.x by defining its data stream format to a sequence of length-value capsules. </ins> <del> HTTP/3 DATAGRAM flows are specific to a given HTTP/3 connection. However, in some cases, an HTTP request may travel across multiple HTTP connections if there are HTTP intermediaries involved; see Section 2.3 of RFC7230. If an intermediary has sent the H3_DATAGRAM SETTINGS parameter with a value of 1 on its client-facing connection, it MUST inspect all HTTP requests from that connection and check for the presence of the "Datagram-Flow-Id" header field.  If the HTTP method of the request is not supported by the intermediary, it MUST remove the "Datagram- Flow-Id" header field before forwarding the request.  If the intermediary supports the method, it MUST either remove the header field or adhere to the requirements leveraged by that method on intermediaries. If an intermediary has sent the H3_DATAGRAM SETTINGS parameter with a value of 1 on its server-facing connection, it MUST inspect all HTTP responses from that connection and check for the presence of the "Datagram-Flow-Id" header field.  If the HTTP method of the request is not supported by the intermediary, it MUST remove the "Datagram- Flow-Id" header field before forwarding the response.  If the intermediary supports the method, it MUST either remove the header field or adhere to the requirements leveraged by that method on intermediaries. If an intermediary processes distinct HTTP requests that refer to the same flow ID in their respective "Datagram-Flow-Id" header fields, it MUST ensure that those requests are routed to the same backend. </del> <ins> TODO: Refactor this document into "HTTP Datagrams" with definitions for HTTP/1.x, HTTP/2, and HTTP/3. </ins> <del> 8. </del> <ins> 7. </ins> Since this feature requires sending an HTTP/3 Settings parameter, it "sticks out".  In other words, probing clients can learn whether a