jiebi/CodeConvo huggingface dataset:
Code change:
This is just a starting point, but brings the document closer to alignment with our spreadsheet. I've also added some additional details which should cover themes raised in the current BoF agenda.

Related discussion:
text: beneficial for a protocol to provide such an opportunity (e.g.  RTP/ QUIC -> RTP/UDP). <del> 7. </del> <ins> 8. </ins> This document makes no requests of IANA. <del> 8. </del> <ins> 9. </ins> As this document is intended to guide discussion and consensus, it introduces no security considerations of its own. <del> 9.  References </del> <ins> 10.  References </ins> <del> 9.1.  URIs </del> <ins> 10.1.  URIs </ins> [1] https://github.com/fiestajetsam/draft-gruessing-moq-requirements <ins> [2] https://www.ietf.org/mailman/listinfo/moq </ins>
Code change:
NAME - Please let me know if this seems about right. One thing that's worth mentioning to you and to NAME - the latency characterizations that we wrote did not make use of the new "ultra-ultra-low latency" categories. I changed the one for gaming to ull100 ("ultra-low-latency under 100 ms") based on conversations at IETF 112, but other people would be better choices to review the characterizations of the use cases they're most interested in.
The opcons drafts uses 4 bands: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) I think these bands lose some granularity that is useful. For example, a functional video call or interactive game probably needs much less than 1s latency, while low latency live may span lower than 1s in some cases?
NAME - that's very true. The opcons draft was scoped for "streaming operators", and although we've talked (in MOPS) about doing a separate draft for low-latency streaming, we haven't done that yet (in MOPS). That may be for the best, if MOQ turns out to be "not ABR, less than 500 ms", or something like that. I'm also one of the opcons drafts, so I'm thinking we should add at least one, and probably two, lower-than-ultra-low(!!!) latency bands, and explain why we're doing that. I'm assigning this to me, so expect a PR soon.
The industry is terrible at naming things. "ultra-low", "low", "non-low", I mean really? It's all terrible marketing rather than an attempt to classify protocols. I like that the opcons tries to reuse and simplify the terminology but... latency is a complicated. Some protocols have artificial delays, but for the most part, protocols are defined by their ability to deal with congestion. A protocol like RTMP will be "ultra low-latency" within a datacenter or for a user with a perfect network connection, while it will be "low-latency" for somebody on a cellular connection, while it will be "non-low-latency" for somebody going through a tunnel... all of which depends on the implementation. At the end of the day, these lines in the sand don't add to the discussion. I would rather focus on the use-cases like "having a conversation with someone" rather than arguing over millisecond budgets.

Related discussion:
text: whether a session has bi-direction flows of media from senders and receivers, and the expected lowest latency requirements using the definitions <del> specified in I-D.draft-ietf-mops-streaming-opcons. </del> <ins> specified in term. </ins> It is likely that we should add other characteristics, as we come to understand them.
Code change:
NAME - Please let me know if this seems about right. One thing that's worth mentioning to you and to NAME - the latency characterizations that we wrote did not make use of the new "ultra-ultra-low latency" categories. I changed the one for gaming to ull100 ("ultra-low-latency under 100 ms") based on conversations at IETF 112, but other people would be better choices to review the characterizations of the use cases they're most interested in.
The opcons drafts uses 4 bands: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) I think these bands lose some granularity that is useful. For example, a functional video call or interactive game probably needs much less than 1s latency, while low latency live may span lower than 1s in some cases?
NAME - that's very true. The opcons draft was scoped for "streaming operators", and although we've talked (in MOPS) about doing a separate draft for low-latency streaming, we haven't done that yet (in MOPS). That may be for the best, if MOQ turns out to be "not ABR, less than 500 ms", or something like that. I'm also one of the opcons drafts, so I'm thinking we should add at least one, and probably two, lower-than-ultra-low(!!!) latency bands, and explain why we're doing that. I'm assigning this to me, so expect a PR soon.
The industry is terrible at naming things. "ultra-low", "low", "non-low", I mean really? It's all terrible marketing rather than an attempt to classify protocols. I like that the opcons tries to reuse and simplify the terminology but... latency is a complicated. Some protocols have artificial delays, but for the most part, protocols are defined by their ability to deal with congestion. A protocol like RTMP will be "ultra low-latency" within a datacenter or for a user with a perfect network connection, while it will be "low-latency" for somebody on a cellular connection, while it will be "non-low-latency" for somebody going through a tunnel... all of which depends on the implementation. At the end of the day, these lines in the sand don't add to the discussion. I would rather focus on the use-cases like "having a conversation with someone" rather than arguing over millisecond budgets.

Related discussion:
text: 4.2. *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*: <del> Significantly less than "Ultra-Low" </del> <ins> Ull100 </ins> Where media is received, and user inputs are sent by the client. This may also include the client receiving other types of signalling,
Code change:
NAME - Please let me know if this seems about right. One thing that's worth mentioning to you and to NAME - the latency characterizations that we wrote did not make use of the new "ultra-ultra-low latency" categories. I changed the one for gaming to ull100 ("ultra-low-latency under 100 ms") based on conversations at IETF 112, but other people would be better choices to review the characterizations of the use cases they're most interested in.
The opcons drafts uses 4 bands: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) I think these bands lose some granularity that is useful. For example, a functional video call or interactive game probably needs much less than 1s latency, while low latency live may span lower than 1s in some cases?
NAME - that's very true. The opcons draft was scoped for "streaming operators", and although we've talked (in MOPS) about doing a separate draft for low-latency streaming, we haven't done that yet (in MOPS). That may be for the best, if MOQ turns out to be "not ABR, less than 500 ms", or something like that. I'm also one of the opcons drafts, so I'm thinking we should add at least one, and probably two, lower-than-ultra-low(!!!) latency bands, and explain why we're doing that. I'm assigning this to me, so expect a PR soon.
The industry is terrible at naming things. "ultra-low", "low", "non-low", I mean really? It's all terrible marketing rather than an attempt to classify protocols. I like that the opcons tries to reuse and simplify the terminology but... latency is a complicated. Some protocols have artificial delays, but for the most part, protocols are defined by their ability to deal with congestion. A protocol like RTMP will be "ultra low-latency" within a datacenter or for a user with a perfect network connection, while it will be "low-latency" for somebody on a cellular connection, while it will be "non-low-latency" for somebody going through a tunnel... all of which depends on the implementation. At the end of the day, these lines in the sand don't add to the discussion. I would rather focus on the use-cases like "having a conversation with someone" rather than arguing over millisecond budgets.

Related discussion:
text: 6.2. <del> I-D.draft-ietf-mops-streaming-opcons describes these latency requirements for streaming media: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) Support for a nominal latency in the low to ultra-low latency should be achieved, with consideration for minimum buffer a receiver playing content may need to handle congestion, packet loss, and other degradation in network quality. </del> <ins> Support for a nominal latency appropriate for the use cases that are in scope should be achieved, with consideration for the minimum buffer that a receiver playing content may need to handle congestion, packet loss, and other degradation in network quality. </ins> 6.3.
Code change:
The section "Moving Beyond RTP over QUIC" should cover the position of assessing the feasibility of the RTP family of protocols, and the size of work of any possible changes that may be required for them to be incorporated.
This We're going to need more work on RTP versus other transport protocols, but we definitely need to make this change!
I've already ranted enough about this section but I figured I'd file a ticket so it doesn't get forgotten. :)
I think what needs changing is that this is removed as a non-requirement, however I still think that we have to make it clear that a consideration "to RTP or not to RTP" must be described in the document.
I think we've now covered this, please chime up if there was anything missed.

Related discussion:
text: 7.2. <del> The creation of new media transport protocols should be avoided, and instead we should make use of RTP RFC3550 and the existing ecosystem of payload formats and methods of signalling where possible.  Work on QUIC encapsulation may reveal a need to extend these specificiations; in which case we should work with the relevant working groups and present our use-cases. 7.3. </del> Even if multicast and other network broadcasting capabilities are often used in delivering media in our use cases, QUIC doesn't yet support multicast, and would require a QUIC protocol extension to do
Code change:
This should address
Although it's true that RUSH predates DATAGRAM, streams were an intentional choice as RUSH implementations can leverage standard QUIC library reassembly of messages longer than one MTU.
Absolutely. It's disingenuous to say that RUSH is using QUIC streams because QUIC datagrams didn't exist yet. QUIC streams provide fragmentation, reliability, flow control, and cancellation (error codes). In fact, you could (but shouldn't) use QUIC streams to deliver individual RTP packets without much downside.
The above commits should fix this, please let me know if I got it wrong.

Related discussion:
text: Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames <del> RUSH introduces new frame types as its development pre-dates Datagram frames </del> All drafts take differing approaches to flow/stream identification and management; some address congestion control and others just omit the subject and leave it to QUIC to handle
Code change:
Although it's true that RUSH predates DATAGRAM, streams were an intentional choice as RUSH implementations can leverage standard QUIC library reassembly of messages longer than one MTU.
Absolutely. It's disingenuous to say that RUSH is using QUIC streams because QUIC datagrams didn't exist yet. QUIC streams provide fragmentation, reliability, flow control, and cancellation (error codes). In fact, you could (but shouldn't) use QUIC streams to deliver individual RTP packets without much downside.
The above commits should fix this, please let me know if I got it wrong.

Related discussion:
text: 3.3. <del> RUSH uses its own frame types on top of QUIC as it pre-dates the datagram specification; in addition individual media frames are given their own stream identifiers to remove HoL blocking from processing out-of-order. </del> <ins> Whilst RUSH predates the datagram specification, it uses its own frame types on top of QUIC to take advantage of QUIC implementations reassembling messages greater than MTU.  In addition individual media frames are given their own stream identifiers to remove HoL blocking from processing out-of-order. </ins> It defines its own registry for signalling codec information with room for future expansion but presently is limited to a subset of
Code change:
(related to )
NAME could you please take a look and let me know if it's an accurate description?
I'm merging this, so we make sure that WARP is included. If we got anything wrong, please let us know - we'll make sure it gets fixed.
Luke Curley has written . It's QUIC based, using CMAF for encapsulation, stream based, has no ALPN amongst other features. That said, it should be included in the "prior art" section of the document.
Agreed!
Some things worth considering, but you guys decide what is worth mentioning. Warp is the only protocol that is not frame based. Instead it attempts to map the GoP structure to QUIC streams. High quality media is basically linear within a GoP in order to maximize compression, which fits very nicely with QUIC's streams. There's no point delivering frames out of order if they cannot be decoded out of order. Warp uses prioritization instead of deadlines to achieve dynamic latency. This removes the need for feedback outside of QUIC ACKs. It avoids a whole class of problems caused by dropped/delayed feedback. Warp uses CMAF segments for full compatibility with HLS/DASH, avoiding transmuxing on edge or at origin. Warp uses the full suite of QUIC functionality. This includes fragmentation, congestion control, flow control, streams (multiplexing + cancellation), etc. Some other protocols use QUIC datagrams and disable everything else, effectively using QUIC as a complex DTLS replacement. Warp supports a bidirectional API using QUIC streams however this was not in scope for the first version of the draft. I wanted to avoid bike shedding to focus instead on the above features. The actual version in production allows the player to manually choose the rendition and pause/play. Warp uses WebTransport at the moment, but we can decide if it's better suited for native QUIC (with ALPN) or even HTTP/3. Warp would even work over HTTP/2 (and Http2Transport) albeit with additional head-of-line blocking.

Related discussion:
text: 4.5. <ins> Warp's specification attemps to map Group of Picture encoding of video on top of QUIC streams.  It depends on ISOBMFF containers to encapsulate both media as well as messaging, and defines prioritisation with separate considerations for audio and video.  It doesn't yet define bi-directionality of media flows, and can be run over protocols like WebTransport I-D.draft-ietf-webtrans-overview. 4.6. </ins> Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames All drafts take differing approaches to flow/stream identification and management; some address congestion control and others just omit the subject and leave it to QUIC to handle <del> Both QRT and RUSH specify ALPN identification; the Engelbart and SRT drafts do not. Note - need to add WARP and verify these characterizations. </del> <ins> Both QRT and RUSH specify ALPN identification; the Engelbart, Warp, and SRT drafts do not. </ins> <del> 4.6. </del> <ins> 4.7. </ins> It's worth noting that work on "RTP over QUIC" is being considered in the AVTCORE working group at this time, although no proposals have
Code change:
Based on feedback from NAME
That looks fabulous to me - merging now. There's not an issue for this, right?

Related discussion:
text: On-Demand media streaming (odstream) is unlikely to benefit from work in this space.  Without notable latency requirements, protocols such <del> as HLS and DASH largely meet the needs of this use case.  However larger deployments may benefit from the use of HTTP/3 I-D.draft-ietf- quic-http. </del> <ins> as HLS and DASH largely meet the needs of this use case. </ins> 7.
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends <del> use cases on live media contribution, syndication, and streaming as the basis for discussions that should guide the design of protocols to satisfy these use cases. </del> <ins> use cases on live media ingest, syndication, and streaming as the basis for discussions that should guide the design of protocols to satisfy these use cases. </ins> Note to Readers
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends <del> use cases on live media contribution, syndication, and streaming as the basis for requirements discussions that should guide the design of protocols to satisfy these use cases. </del> <ins> use cases on live media ingest, syndication, and streaming as the basis for requirements discussions that should guide the design of protocols to satisfy these use cases. </ins> 1.1.
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: These latency bands were appropriate for streaming media, which was the target for I-D.draft-ietf-mops-streaming-opcons, but some <del> realtime media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using </del> <ins> interactive media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using </ins> <del> ull500 (less than 500 ms) ull100 (less than 100 ms) </del> <ins> Ull-50 (less than 50 ms) Ull-200 (less than 200 ms) </ins> Obviously, these last two latency bands are the shortened form of <del> "ultra-low latency - 500 ms" and "ultra-low-latency - 100 ms".  Also obviously, bikeshedding on better names is welcomed. </del> <ins> "ultra-low latency - 50 ms" and "ultra-low-latency - 200 ms". Perhaps less obviously, bikeshedding on better names and more useful values is welcomed. </ins> 3.
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: 3.4. Secure Reliable Transport (SRT) (I-D.draft-sharabayko-srt) itself is <del> a general purpose transport protocol primarily for contribution transport use cases and this specification covers the encapsulation and delivery of SRT on top of QUIC using datagram frame types.  This </del> <ins> a general purpose transport protocol primarily for ingest transport use cases and this specification covers the encapsulation and delivery of SRT on top of QUIC using datagram frame types.  This </ins> specification sets some requirements regarding how the two interact and leaves considerations for congestion control and pacing to prevent conflict between the two protocols.  Apart from that, SRT
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: 3.6. <del> Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames </del> <ins> Some drafts attempt to use existing payloads of RTP, RTCP, and SDP, while others do not. Some use QUIC Datagram frames, while others use QUIC streams. </ins> All drafts take differing approaches to flow/stream identification <del> and management; some address congestion control and others just omit the subject and leave it to QUIC to handle Both QRT and RUSH specify ALPN identification; the Engelbart, Warp, and SRT drafts do not. </del> <ins> and management.  Some address congestion control and others just defer this to QUIC to handle. Some drafts specify ALPN identification, while others do not. </ins> 3.7.
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: 4.1. <del> *Senders/Receivers*: Many to Many *Bi-directional*: Yes *Latency*: Ultra-Low Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation.  This may be done as client/server, or peer to peer with a many to many relationship of both senders and receivers. 4.2. </del> <ins> 4.1.1. </ins> *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*: <del> Ull100 </del> <ins> Ull-50 </ins> Where media is received, and user inputs are sent by the client. This may also include the client receiving other types of signalling,
Code change:
It's likely that only NAME and I will care about this, but I think it matters :grin:

Related discussion:
text: the client such as microphone audio for in-game chat with other players. <del> 4.3. </del> <ins> 4.1.2. </ins> *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*: <del> Ultra-Low </del> <ins> Ull-50 </ins> Where media is received, and user inputs are sent by the client. Latency requirements with this usecase are marginally different than the gaming use case.  This may also include signalling and/or transmitting of files or devices connected to the user's computer. <del> 4.4. </del> <ins> 4.1.3. </ins> <del> *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: Low to Non-Low </del> <ins> *Senders/Receivers*: Many to Many *Bi-directional*: Yes *Latency*: Ull-200 </ins> <del> Where media is received from a live broadcast or stream.  This may comprise of multiple audio or video outputs with different codecs or bitrates.  This may also include other types of media essence such as subtitles or timing signalling information (e.g. markers to indicate change of behaviour in client such as advertisement breaks).  The use of "live rewind" where a window of media behind the live edge can be made available for clients to playback, either because the local player falls behind edge or because the viewer wishes to play back from a point in the past. </del> <ins> Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation.  This may be done as client/server, or peer to peer with a many to many relationship of both senders and receivers. 4.2. </ins> <del> 4.5. </del> <ins> 4.2.1. </ins> *Senders/Receivers*: One to One *Bi-directional*: No *Latency*: <del> Ultra-Low to Low </del> <ins> Ull-200 to Ultra-Low </ins> Where media is received from a source for onwards handling into a distribution platform.  The media may comprise of multiple audio and/