Code change:
This PR aims to take the design discussions we had at the last interim, and propose a potential solution. This PR should in theory contain enough information to allow folks to implement this new design. That will allow us to perform interop testing and confirm whether the design is sound.
I'm confused by this pull request (probably because I was out of the loop for a while). My understanding, at least when I first read the proposal a month ago was that we would separate multiplexing streams and multiplexing contexts into different layers, so that methods that need context would get it (CONNECT-UDP), and features that don't (WebTransport/extended CONNECT) could use stream-associated datagrams directly.
NAME you're referring to which this PR isn't trying to address. That'll happen in a subsequent PR.
In , capsules, contexts and context IDs are explicitly defined as end-to-end and there are requirements for intermediaries to not modify them, and to not even parse them. The motivation for this was to ensure that we retain the ability to deploy extensions end-to-end without having to modify intermediaries. NAME mentioned that these requirements might be too strict, as he would like to have his intermediary be able to parse the context ID, for logging purposes.
Part of the comment from the PR is also about consistency of interface for generic HTTP libraries that can be used to build either endpoints or intermediaries. If the context ID is always present, it's overly prescriptive to say that an intermediary can't parse it, and such a requirement couldn't be enforced anyways. If context IDs are negotiated on/off, whether this is possible depends on the negotiation mechanism.
I think its fine for a component that has bytes passing through it to look at those bytes. The import thing to do is set expectations on what that means for the protocol behaviour we are defining. We don't expect intermediaries to do anything with a context ID; they should not enforce the protocol semantic rules (odd, even, reuse etc). H2 and H3 somewhat solve this problem by decoupling frame parsing from frame payload interpretation. E.g. if the form of a request or response is invalid, an intermediary shouldn't pass it on. DATAGRAM is different because it isn't defined as an actual HTTP/3 frame with frame headers and frame payload. Something that might work is to formalise that quarter stream ID is part of the envelope that intermediaries have to be able to handle the syntax of, but context ID is part of content and separate.
Unless you encrypt capsules end-to-end, I don't think any text in the spec can in practice prevent intermediaries from inspecting and modifying capsules. You can send something like GREASE capsules, though.
In , we introduce the CLOSEDATAGRAMCONTEXT capsule which allows endpoints to close a context. That message currently only contains the context ID to close. NAME suggests that we may want to add more information there. Should we differentiate between "context close" and "context registration rejected"? Should we add an error code? In particular, do folks have use cases for these features?
My first concern is with infinite loops. If the context was "garbage collected" by the recipient, the sender can simply re-create it if they still want it. However, if it was closed due to incompatibility (i.e. rejected) or hitting a resource limit (e.g. max # of contexts), reopening it in response will produce a tight infinite loop. This could be maybe be avoided by some kind of heuristic, but an explicit indication of what's going on seems better.
This issue assumes that decide to go with the two-layer design described in . Given that design, some applications might not need the multiplexing provided by context IDs. We have multiple options here: Make context ID mandatory Applications that don't need it waste a byte per datagram Negotiate the presence of context IDs using an HTTP header Context IDs would still be mandatory to implement on servers because the client might send that header Have the method determine whether context IDs are present or not This would prevent extensibility on those methods Create a REGISTERNOCONTEXT message This assumes we use the Message design from We add a register message that means that this stream does not use context IDs This allows avoiding the byte overhead without sacrificing extensibility
Making the context ID mandatory is simple, although slightly wasteful. For the variations where it is optional, I prefer the one that uses a header. It could work with a variant of where we do Header + Message, and the Header determines whether or not you have contexts, and whether or not they can be dynamic.
What are the benefits of message over header to make it optional? To me it seems like header is sufficient, because the context IDs are end-to-end, and should not affect intermediaries, making the header a natural place to put it. I think having the context ID be the default behavior and the header is to opt-out is the right decision if we were to allow removing it.
This is a bit of a bike-shed, but why would it default to having context-IDs instead of not having them?
I think the second option "Negotiate the presence of context IDs using an HTTP header" is the best personally. In terms of: "Context IDs would still be mandatory to implement on servers because the client might send that header", I assume server in this context is the final application server, not a proxy? Even so, I'd argue it's applications which decide if they need this functionality, so if a server only implemented one application and it didn't use Context-IDs, then I don't see why it would need to implement it.
Ian, in your comment, are you talking about an intermediary when you say "proxy"? If so, then an intermediary need not implement anything based on the payload of than H3 DATAGRAM, it only has to pass the data along. That may be worth a clarification.
Having context ID optional has a nice future-proof property - other uses of datagrams in HTTP/3 can decide if they want the second layer or not. I wouldn't even state it as application needing multiplexing or not. Other applications might find it very useful to include a second ((third, fourth, fifth) layer of frame header. I think its fine for us to state as much in this in the spec, while also defining an explicit second layer for multiplexing.
Yes, by proxy I meant intermediary. And I agree about what intermediaries need to do, I just wanted to ensure I understood what server meant when David used it above. Yup, whenever all this gets written up, we can make it clear that intermediaries can ignore much of this.
The method defines the semantics of datagrams for a particular request, thus I think the third option makes most sense. I don't see a point in using explicit negotiation, since in the two cases I can think of the top of the head, the answer is always clear from the method (CONNECT-IP seems to always want context IDs, WebTransport does not ever want context IDs). If some method starts out without context, but wants to add context later, it can add its own negotiation.
One set of use cases for which the method approach(3) is best are those where there are multiple context IDs but they're predefined by the method and don't need negotiation. My strawman is WebRTC.
During the 2021-04 MASQUE Interim, we discussed multiple options for registering context IDs. Without going into the specific of the encoding, there are two broad classes of solutions: "Header" design Once-at-setup registration This happens during the HTTP request/response Simplest solution is to use an HTTP header such as (encoding TBD) Example use-case: CONNECT-UDP without extensions "Message" design Mid-stream registration This can happen at any point during the lifetime of the request stream This would involve some sort of "register" message (encoding TBD, see separate issue) Example use-case: CONNECT-IP on-the-fly compression (this can't be done at tunnel setup time because the bits top be compressed aren't always known at that time) It's possible to implement once-at-setup registration using the Message design, but it isn't possible to implement mid-stream registration. Therefore I think we should go with the Message design. It would also be possible to implement both but I don't think that provides much value.
I'd like to here some more from the WebTransport use case on the issue. My understanding is that the Requests are used to create WebTransport sessions and that creation of datagram flows(contexts) could be a lot more ad-hoc / server generated after the request phase, when compared to the CONNECT-UDP case. Cc NAME
Personally, I'd vote for either "message" or "message + header" (a la priorities), but not just "header". Mid-stream annotation of context/flow/etc is quite useful.
I prefer "Message" only, since the dynamic use case seems like a fairly compelling use of sub-flows(contexts?). I'm not sure there are many cases when one allocates all sub-flows up front AND needs to use a header to do that in a flexible way. If an application needs multiple sub-flows, but they're fixed in number/format(ie: data and control), I don't think there's a need for a header or a "Message", since the application can consider that part of its internal format.
During the 2021-04 MASQUE Interim, we discussed having the ability to accept or reject registration of context IDs. An important motivation for this is the fact that context ID registrations take up memory (for example it could be a compression context which contains the data that is elided) and therefore we need to prevent endpoints from arbitrarily making their peer allocate memory. While we could flow control context ID registration, a much simpler solution would be to have a way for endpoints to close/shutdown a registration.
IIUC there's an opposing view that the constraints can be negotiated apiori. Either statically (such as defining an extension which states that there can only be N contenxts of a given type) or dynamically using something like a HTTP header. Failure to meet these constraints could lead to an HTTP/3 stream or connection error rather than a flow rejection.
NAME I've filed to discuss whether to register/negotiate at request start or mid-stream.
I think that if there is the ability to register mid-stream (), we need the ability to close/de-register/reject mid-stream as well. If we don't have mid-stream changes, then no need to negotiate. However, someone trying to register something needs to be able to know whether or not the thing they registered will be accepted.
I agree that notification of the registration status is particularly important if we allow mid-stream registration.
In draft-ietf-masque-h3-datagram-00, flow IDs are bidirectional. During the 2021-04 MASQUE Interim, we discussed the possibility of making them unidirectional. Here are the differences: Bidirectional single shared namespace for both endpoints (even for client, odd for server) it's easier to refer to peer's IDs, which makes negotiation easier Unidirectional one namespace per direction every bidirectional use needs to associate both directions somehow I'm personally leaning towards keeping them bidirectional. Unlike QUIC streams, these do not have a state machine or flow control credits, so there is no cost to only using a single direction of a bidirectional flow for unidirectional use-cases.
There's another option where flows are unidirectional but they share a namespace (cleint even/server odd). I'm undecided. And I also don't know how important it is to choose. The 2 layer design means that DATAGRAMs always have a bidirectional tie via the stream ID. On balance, I think bidirectional would be good enough today but I would be receptive to extension use cases that make a strong point of unidirectional.
Bidirectional is my personal preference, but either way could work ultimately. It seems to me that any extension that wants unidirectional can just only use this context/flow/etc ID for one direction if it wants.
I think bidirectional flows are likely overcomplicated and unnecessary. For example, if either peer can "close" a flow, this requires a much more complex state machine. In my experience, half-closed states are hard to get right. Closing unidirectional flows seems much easier to reason about. Many, perhaps most, of our use cases for flows are really unidirectional. For example, DSCP marking likely only makes sense from client to server, while timestamps for congestion control tuning likely only make sense from server to client. QUIC connection IDs are different in each direction, so QUIC header compression may be easiest to understand with unidirectional flows.
Bidirectional doesn't imply half-close states. I'm thinking of a bidirectional flow that can only be closed atomically as a whole. I don't think that's true. The default extension-less CONNECT-UDP is bidirectional. So is ECN, and so would a timestamp extension.
I would prefer bidirectional without support for half-close. I feel that having unidirectional flow/context IDs adds overhead we don't need; peers now need to track those associations and it's unclear what the benefits of having them separated out are.
This issue is conflating consent (unilateral/bilateral) with scope (unidirectional/bidirectional). IIUC there are two questions here: 1) Can I send a flow-id forever once declared, or can the endpoint tell me to stop without killing the whole stream (consent)? 2) Can the receiver of a flow-id assignment use that same flow-id when sending datagrams, or could this mean something completely different? I think the answer to (1) is "yes, it can tell me to stop" (i.e. it is negotiated while allowing speculative sending). For (2), I lean toward bidirectional, but not strongly. There are plenty of codepoints, so it seems wasteful for each endpoint to define the same semantic explicitly.
NAME the intent of this issue wan't to conflate those two things. This issue is about scope. For consent, see issue .
Conceptually, I like unidirectional better. I tell the peer that when I send with context-ID=X, it has a given semantic. For bidirectional the sender is saying both that and that if the peer sends on that ID the original sender will interpret with the same semantic. That might not even make sense. Is it possible to associate datagrams with a unidirectional stream (push?). If so, does allowing for bidirectional context imply that even though the receiver cannot send frames on the stream, it can send datagrams? Every negotiation does run the risk of loss/reordering resulting in a drop or buffering event at the receiver. The appeal of bidirectional is that it removes half of this risk in the cases where the bidirectionality is used.
To second NAME I would also prefer bidirectional without support for half-close. Given you can always not send something in one direction, and there's none of the overhead of streams(ie: flow control, etc) I can't see any benefits of unidirectional over bidirectional in this case. Associating h3-datagrams with a server push seems like unnecessary complexity to me, but you're right that it should be clarified.
During the 2021-04 MASQUE Interim, we discussed a design proposal from NAME to replace the one-layer flow ID design with a two-layer design. This design replaces the flow ID with two numbers: a stream ID and a context ID. The contents of the QUIC DATAGRAM frame would now look like: While the flow ID in the draft was per-hop, the proposal has better separation: the stream ID is per-hop, it maps to an HTTP request the context ID is end-to-end, it maps to context information inside that request Intermediaries now only look at stream IDs and can be blissfully ignorant of context IDs. In the room at the 2021-04 MASQUE Interim, multiple participants spoke in support of this design and no one raised objections. This issue exists to ensure that we have consensus on this change - please speak up if you disagree.
I am fine with the 2-layer design for CONNECT-UDP, but I am not sure if the second identifier should be in H3 DATAGRAM or should be specific to CONNECT-UDP. As I indicated to the list, webtrans and httpbis might weigh in on this with broader consideration of other use cases.
Hi NAME the topic of the optionality of context IDs is discussed in issue .
I think this is a good design, because it minimizes the core functionality of the draft, but avoids every application that needs multiple dynamically created sub-flows from re-inventing a different mechanism. Nit: I would not call this a 'Frame', since it's not an HTTP(TLV) frame.
I think this is a good change. Thanks! There are some things that I think we should still discuss, but we shouldn't hold up this change on those details.

Related discussion:
text: 2. <del> Flows are bidirectional exchanges of datagrams within a single QUIC connection.  These are conceptually similar to streams in the sense that they allow multiplexing of application data.  Flows are identified within a connection by a numeric value, referred to as the flow ID.  A flow ID is a 62-bit integer (0 to 2^62-1). </del> <ins> In order to allow multiple exchanges of datagrams to coexist on a given QUIC connection, HTTP datagrams contain two layers of multiplexing.  First, the QUIC DATAGRAM frame payload starts with an encoded stream identifier that associates the datagram with a given QUIC stream.  Second, datagrams carry a context identifier (see datagram-contexts) that allows multiplexing multiple datagram contexts related to a given HTTP request.  Conceptually, the first layer of multiplexing is per-hop, while the second is end-to-end. </ins> <del> Flows lack any of the ordering or reliability guarantees of streams. Beyond this, a sender SHOULD ensure that DATAGRAM frames within a single flow are transmitted in order relative to one another. </del> <ins> 2.1. </ins> <del> 3. </del> <ins> Within the scope of a given HTTP request, contexts provide an additional demultiplexing layer.  Contexts determine the encoding of datagrams, and can be used to implicitly convey metadata.  For example, contexts can be used for compression to elide some parts of the datagram: the context identifier then maps to a compression context that the receiver can use to reconstruct the elided data. Contexts are identified within the scope of a given request by a numeric value, referred to as the context ID.  A context ID is a 62-bit integer (0 to 2^62-1). While stream IDs are a per-hop concept, context IDs are an end-to-end concept.  In other words, if a datagram travels through one or more intermediaries on its way from client to server, the stream ID will most likely change from hop to hop, but the context ID will remain the same.  Context IDs are opaque to intermediaries. 2.2. </ins> Implementations of HTTP/3 that support the DATAGRAM extension MUST <del> provide a flow ID allocation service.  That service will allow applications co-located with HTTP/3 to request a unique flow ID that they can subsequently use for their own purposes.  The HTTP/3 implementation will then parse the flow ID of incoming DATAGRAM </del> <ins> provide a context ID allocation service.  That service will allow applications co-located with HTTP/3 to request a unique context ID that they can subsequently use for their own purposes.  The HTTP/3 implementation will then parse the context ID of incoming DATAGRAM </ins> frames and use it to deliver the frame to the appropriate application context. <del> Even-numbered flow IDs are client-initiated, while odd-numbered flow IDs are server-initiated.  This means that an HTTP/3 client implementation of the flow ID allocation service MUST only provide </del> <ins> Even-numbered context IDs are client-initiated, while odd-numbered context IDs are server-initiated.  This means that an HTTP/3 client implementation of the context ID allocation service MUST only provide </ins> even-numbered IDs, while a server implementation MUST only provide <del> odd-numbered IDs.  Note that, once allocated, any flow ID can be used by both client and server - only allocation carries separate namespaces to avoid requiring synchronization. </del> <ins> odd-numbered IDs.  Note that, once allocated, any context ID can be used by both client and server - only allocation carries separate namespaces to avoid requiring synchronization.  Additionally, note that the context ID namespace is tied to a given HTTP request: it is possible for the same numeral context ID to be used simultaneously in distinct requests. </ins> <del> 4. </del> <ins> 3. </ins> When used with HTTP/3, the Datagram Data field of QUIC DATAGRAM frames uses the following format (using the notation from the "Notational Conventions" section of QUIC): <del> A variable-length integer indicating the Flow ID of the datagram (see datagram-flows). </del> <ins> A variable-length integer that contains the value of the client- initiated bidirectional stream that this datagram is associated with, divided by four.  (The division by four stems from the fact that HTTP requests are sent on client-initiated bidirectional streams, and those have stream IDs that are divisible by four.) A variable-length integer indicating the context ID of the datagram (see datagram-contexts). The payload of the datagram, whose semantics are defined by individual applications.  Note that this field can be empty. Intermediaries parse the Quarter Stream ID field in order to associate the QUIC DATAGRAM frame with a stream.  If an intermediary receives a QUIC DATAGRAM frame whose payload is too short to allow parsing the Quarter Stream ID field, the intermediary MUST treat it as an HTTP/3 connection error of type H3_GENERAL_PROTOCOL_ERROR. Intermediaries MUST ignore any HTTP/3 Datagram fields after the Quarter Stream ID. Endpoints parse both the Quarter Stream ID field and the Context ID field in order to associate the QUIC DATAGRAM frame with a stream and context within that stream.  If an endpoint receives a QUIC DATAGRAM frame whose payload is too short to allow parsing the Quarter Stream ID field, the endpoint MUST treat it as an HTTP/3 connection error of type H3_GENERAL_PROTOCOL_ERROR.  If an endpoint receives a QUIC DATAGRAM frame whose payload is long enough to allow parsing the Quarter Stream ID field but too short to allow parsing the Context ID field, the endpoint MUST abruptly terminate the corresponding stream with a stream error of type H3_GENERAL_PROTOCOL_ERROR. If a DATAGRAM frame is received and its Quarter Stream ID maps to a stream that has already been closed, the receiver MUST silently drop that frame.  If a DATAGRAM frame is received and its Quarter Stream ID maps to a stream that has not yet been created, the receiver SHALL either drop that frame silently or buffer it temporarily while awaiting the creation of the corresponding stream. 4. CAPSULE allows reliably sending request-related information end-to- end, even in the presence of HTTP intermediaries. CAPSULE is an HTTP/3 Frame (as opposed to a QUIC frame) which SHALL only be sent in client-initiated bidirectional streams. Intermediaries MUST forward all received CAPSULE frames in their unmodified entirety on the same stream where it would forward DATA frames.  Intermediaries MUST NOT send any CAPSULE frames other than the ones it is forwarding. This specification of CAPSULE currently uses HTTP/3 frame type 0xffcab5.  If this document is approved, a lower number will be requested from IANA. The Type and Length fields follows the definition of HTTP/3 frames from H3.  The payload consists of: The type of this capsule. Data whose semantics depends on the Capsule Type. Endpoints which receive a Capsule with an unknown Capsule Type MUST silently drop that Capsule.  Intermediaries MUST forward Capsules, even if they do not know the Capsule Type or cannot parse the Capsule Data. 4.1. The REGISTER_DATAGRAM_CONTEXT capsule (type=0x00) allows an endpoint to inform its peer of the encoding and semantics of datagrams associated with a given context ID.  Its Capsule Data field consists of: The context ID to register. A string of comma-separated key-value pairs to enable extensibility.  Keys are registered with IANA, see iana-keys. The ABNF for the Extension String field is as follows (using syntax from Section 3.2.6 of RFC7230): Note that these registrations are unilateral and bidirectional: the sender of the frame unilaterally defines the semantics it will apply to the datagrams it sends and receives using this context ID.  Once a context ID is registered, it can be used in both directions. Endpoints MUST NOT send DATAGRAM frames using a Context ID until they have either sent or received a REGISTER_DATAGRAM_CONTEXT Capsule with the same Context ID.  However, due to reordering, an endpoint that receives a DATAGRAM frame with an unknown Context ID MUST NOT treat it as an error, it SHALL instead drop the DATAGRAM frame silently, or buffer it temporarily while awaiting the corresponding REGISTER_DATAGRAM_CONTEXT Capsule. Endpoints MUST NOT register the same Context ID twice on the same stream.  This also applies to Context IDs that have been closed using a CLOSE_DATAGRAM_CONTEXT capsule.  Clients MUST NOT register server- initiated Context IDs and servers MUST NOT register client-initiated Context IDs.  If an endpoint receives a REGISTER_DATAGRAM_CONTEXT capsule that violates one or more of these requirements, the endpoint MUST abruptly terminate the corresponding stream with a stream error of type H3_GENERAL_PROTOCOL_ERROR. 4.2. The CLOSE_DATAGRAM_CONTEXT capsule (type=0x01) allows an endpoint to inform its peer that it will no longer send or parse received datagrams associated with a given context ID.  Its Capsule Data field consists of: The context ID to close. A string of comma-separated key-value pairs to enable extensibility, see the definition of the same field in register- capsule for details. Note that this close is unilateral and bidirectional: the sender of the frame unilaterally informs its peer of the closure.  Endpoints can use CLOSE_DATAGRAM_CONTEXT capsules to close a context that was initially registered by either themselves, or by their peer. Endpoints MAY use the CLOSE_DATAGRAM_CONTEXT capsule to immediately reject a context that was just registered using a REGISTER_DATAGRAM_CONTEXT capsule if they find its Extension String to be unacceptable. After an endpoint has either sent or received a CLOSE_DATAGRAM_CONTEXT frame, it MUST NOT send any DATAGRAM frames with that Context ID.  However, due to reordering, an endpoint that receives a DATAGRAM frame with a closed Context ID MUST NOT treat it as an error, it SHALL instead drop the DATAGRAM frame silently. Endpoints MUST NOT close a Context ID that was not previously registered.  Endpoints MUST NOT close a Context ID that has already been closed.  If an endpoint receives a CLOSE_DATAGRAM_CONTEXT capsule that violates one or more of these requirements, the endpoint MUST abruptly terminate the corresponding stream with a stream error of type H3_GENERAL_PROTOCOL_ERROR. 4.3. The DATAGRAM capsule (type=0x02) allows an endpoint to send a datagram frame over an HTTP stream.  This is particularly useful when using a version of HTTP that does not support QUIC DATAGRAM frames. Its Capsule Data field consists of: A variable-length integer indicating the context ID of the datagram (see datagram-contexts). </ins> The payload of the datagram, whose semantics are defined by individual applications.  Note that this field can be empty. <del> Endpoints MUST treat receipt of a DATAGRAM frame whose payload is too short to parse the flow ID as an HTTP/3 connection error of type H3_GENERAL_PROTOCOL_ERROR. </del> <ins> Datagrams sent using the DATAGRAM Capsule have the exact same semantics as datagrams sent in QUIC DATAGRAM frames. </ins> 5.