jiebi/CodeConvo huggingface dataset:
5. Our proposal is that "Media Over QUIC" discussions focus first on the <del> use cases for Live Media Contribution (lmcont), Syndication (lmsynd), and Streaming (lmstream).  Our reasoning is provided in usecaseanalysis. </del> <ins> use cases described in lm-media, which are Live Media Ingest (lmingest), Syndication (lmsynd), and Streaming (lmstream).  Our reasoning is provided in usecaseanalysis. </ins> 5.1. Each of the above use cases in overallusecases fit into one of three classifications of solutions. <del> The first group, covering gaming (gaming), screen sharing (remdesk), and general video conferencing (vidconf), are largely covered by WebRTC and related protocols today.  Whilst there may be benefit in these use cases having a QUIC based protocol it may be more appropriate given the size of existing deployments to extend the WebRTC protocol and specifications.  Such work could start in a QUIC specific forum, but would likely need to take place in rtcweb and the W3C. The second group of classifications, covering Live Media Contribution (lmcont), Syndication (lmsynd), and Streaming (lmstream) are likely the use cases likely to benefit most from this work.  Existing protocols used such as HLS RFC8216 and DASH DASH are reaching limits towards how low they can reduce latency in live streaming and for scenarios where low-bitrate audio streams are used add a significant amount of overheads compared to the media bitstream. On-Demand media streaming (odstream) is unlikely to benefit from work in this space.  Without notable latency requirements, protocols such as HLS and DASH largely meet the needs of this use case. </del> <ins> The first group, in interact, covering gaming (gaming), screen sharing (remdesk), and general video conferencing (vidconf), are largely covered by WebRTC and related protocols today.  Whilst there may be benefit in these use cases having a QUIC based protocol it may be more appropriate given the size of existing deployments to extend the WebRTC protocol and specifications.  Such work could start in a QUIC specific forum, but would likely need to take place in rtcweb and the W3C. The second group of classifications, in lm-media, covering Live Media Ingest (lmingest), Live Media Syndication (lmsynd), and Live Media Streaming (lmstream) are likely the use cases likely to benefit most from this work.  Existing protocols used such as HLS RFC8216 and DASH DASH are reaching limits towards how low they can reduce latency in live streaming and for scenarios where low-bitrate audio streams are used add a significant amount of overheads compared to the media bitstream. The third group, od-media, covering On-Demand Media Ingest (od- ingest) and On-Demand Media streaming (od-stream) is unlikely to benefit from work in this space.  Without notable latency requirements, protocols such as HLS and DASH largely meet the needs of this use case. </ins> 6.
4.3.1. <del> ** James, can you fill this section in?** </del> <ins> *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: On Demand Where media is ingested and processed for a system to later serve it to clients as on-demand media.  This may be media captured from live output or provided as a pre-recorded source, and may optionally be transcoded upon ingest. </ins> 4.3.2.
indirectly over the QUIC protocol, as an RTP payload"), seems to be in scope for the IETF AVTCORE working group, and was discussed at some length at the February 2022 AVTCORE working group meeting <del> AVTCORE-2022-02.  So, perhaps, that possible meaning is out of scope for "Media over QUIC". It will be SUPER HELPFUL if interested parties can come up with a term that unambiguously describes what we're trying to achieve. </del> <ins> AVTCORE-2022-02, although no drafts in this space have been adopted by the AVTCORE working group.  So, perhaps, that possible meaning is out of scope for "Media over QUIC". </ins> 2.2.
4.1. <ins> The use cases described in this section have one particular attribute in common - the target latency for these cases are on the order of one or two RTTs.  In order to meet those targets, it is not possible to rely on protocol mechanisms that require multiple RTTs to function effectively.  For example, When the target latency is on the order of one RTT, it makes sense to use FEC RFC6363 and codec-level packet loss concealment RFC6716, rather than selectively retransmitting only lost packets. These mechanisms use more bytes, but do not require multiple RTTs in order to recover from packet loss. When the target latency is on the order of one RTT, it is impossible to use congestion control schemes like BBR I-D.draft- cardwell-iccrg-bbr-congestion-control, since BBR has probing mechanisms that rely on temporarily inducing delay and amortizing the consequences of that over multiple RTTs. This may help to explain why these use cases often rely on protocols such as RTP RFC3550, which provide low-level control of packetization and transmission. </ins> 4.1.1. *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*:
4.1.3. *Senders/Receivers*: Many to Many *Bi-directional*: Yes *Latency*: <del> Ull-200 </del> <ins> Ull-50 to Ull-200 </ins> Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation.  This may be done as client/server, or peer to peer <del> with a many to many relationship of both senders and receivers. </del> <ins> with a many to many relationship of both senders and receivers.  The target for latency may be as large as Ull-200 for some media types such as audio, but other media types in this use case have much more stringent latency targets. </ins> 4.2. <ins> The use cases in this section, unlike the use cases described in interact, still have "humans in the loop", but these humans expect media to be "responsive", where the responsiveness is more on the order of 5 to 10 RTTs.  This allows the use of protocol mechanisms that require more than one or two RTTs - as noted in interact, end- to-end recovery from packet loss and congestion avoidance are two such protocol mechanisms that can be used with Live Media. To illustrate the difference, the responsiveness expected with videoconferencing is much greater than watching a video, even if the video is being produced "live" and sent to a platform for syndication and distribution. </ins> 4.2.1. *Senders/Receivers*: One to One *Bi-directional*: No *Latency*:
4.3. <ins> Finally, the "On-Demand" use cases described in this section do not have a tight linkage between ingest and streaming, allowing significant transcoding, processing, insertion of video clips in a news article, etc.  The latency constraints for the use cases in this section may be dominated by the time required for whatever actions are required before media are available for streaming. </ins> 4.3.1. *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: On Demand Where media is ingested and processed for a system to later serve it <del> to clients as on-demand media.  This may be media captured from live output or provided as a pre-recorded source, and may optionally be transcoded upon ingest. </del> <ins> to clients as on-demand media.  This media provided from a pre- recorded source, or captured from live output, but in either case, this media is not immediately passed to viewers, but is stored for "on-demand" retrieval, and may be transcoded upon ingest. </ins> 4.3.2.
HTTP/3 payload any kind of media carried indirectly over the QUIC protocol, as a WebTransport payload <del> the encapsulation of any Media Transport Protocol (mtp) in a QUIC </del> <ins> the encapsulation of any Media Transport Protocol in a QUIC </ins> payload an IETF mailing list (MOQ-ml), which was requested "... for discussion of video ingest and distribution protocols that use
2.2. <del> This document describes considerations for work on extensions to existing "Media Transport Protocols" or creation of new "Media Transport Protocols". Within this document, we use the term "Media Transport Protocol" to describe the protocol of interest.  This is easier to understand if the reader assumes that we are talking about a protocol stack that looks something like this: where "Media Format" would be something like RTP payload formats or ISOBMFF ISOBMFF, and "Media Transport Protocol" would be something like RTP or HTTP.  Not all possible proposals for "Media Over QUIC" follow this model, but for the ones that do, it seems useful to have names for "the protocol layers beteern Media and QUIC". It is worth noting explicitly that the "Media Transport Protocol" layer might include more than one protocol.  For example, a new Media Transport Protocol might be defined to run over HTTP, or even over WebTransport and HTTP. 2.3. Within this document, we extend the latency requirement categories for streaming media described in I-D.draft-ietf-mops-streaming- opcons: </del> <ins> }The "Operational Considerations for Streaming Media" document (I- D.draft-ietf-mops-streaming-opcons) described a range of latencies of interest to streaming media providers, as </ins> ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) <del> These latency bands were appropriate for streaming media, which was the target for I-D.draft-ietf-mops-streaming-opcons, but some interactive media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using </del> <ins> Because the IETF Media Over QUIC community now expresses interest in interactive media interact and live media lm-media} use cases will have requirements that are significantly less than the "streaming media"-defined "ultra-low latency". Within this document, we are using </ins> near real-time (less than 50 ms) Ull-200 (less than 200 ms) Perhaps obviously, these last two latency bands are the shortened form of "ultra-low latency - 50 ms" and "ultra-low-latency - 200 ms". Perhaps less obviously, bikeshedding on better names and more useful values is welcomed.
Abstract <del> This document describes use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", provides analysis about those use cases, recommends a subset of use cases that cover live media ingest, syndication, and streaming for further exploration, and describes requirements that should guide the design of protocols to satisfy these use cases. </del> <ins> This document describes use cases and requirements that guide the specification of a simple, low-latency media delivery solution for ingest and distribution of media, using either the QUIC protocol or WebTransport. </ins> Note to Readers
When the target latency is on the order of one RTT, it is impossible to use congestion control schemes like BBR I-D.draft- cardwell-iccrg-bbr-congestion-control, since BBR has probing <del> mechanisms that rely on temporarily inducing delay and amortizing the consequences of that over multiple RTTs. </del> <ins> mechanisms that rely on temporarily inducing delay, but these mechanisms can then amortize the consequences of induced delay over multiple RTTs. </ins> <del> This may help to explain why these use cases often rely on protocols such as RTP RFC3550, which provide low-level control of packetization and transmission. </del> <ins> This may help to explain why interactive use cases have typically relied on protocols such as RTP RFC3550, which provide low-level control of packetization and transmission, and make no provision for retransmission. </ins> <del> 4.1.1. </del> <ins> 3.1.1. </ins> Where media is received, and user inputs are sent by the client. <del> This may also include the client receiving other types of signalling, </del> <ins> This may also include the client receiving other types of signaling, </ins> such as triggers for haptic feedback.  This may also carry media from the client such as microphone audio for in-game chat with other players. <del> 4.1.2. </del> <ins> 3.1.2. </ins> Where media is received, and user inputs are sent by the client. <del> Latency requirements with this usecase are marginally different than </del> <ins> Latency requirements with this use case are marginally different than </ins> the gaming use case.  This may also include signalling and/or transmitting of files or devices connected to the user's computer. <del> 4.1.3. </del> <ins> 3.1.3. </ins> Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing"
such as audio, but other media types in this use case have much more stringent latency targets. <del> 4.2. </del> <ins> 3.2. </ins> For the video conferencing/telephony use case, there can be additional scenarios where the audience greatly outnumbers the concurrent active participants, but any member of the audience could participate.  As this has a much larger total number of participants - as many as Live Media Streaming lmstream, but with the bi- <del> directionality of confercing, this should be considered a "hybrid". </del> <ins> directionality of conferencing, this should be considered a "hybrid". </ins> <del> 4.3. </del> <ins> 3.3. </ins> The use cases in this section, unlike the use cases described in interact, still have "humans in the loop", but these humans expect
video is being produced "live" and sent to a platform for syndication and distribution. <del> 4.3.1. </del> <ins> 3.3.1. </ins> Where media is received from a source for onwards handling into a distribution platform.  The media may comprise of multiple audio and/ or video sources.  Bitrates may either be static or set dynamically <del> by signalling of connection inforation (bandwidth, latency) based on </del> <ins> by signaling of connection information (bandwidth, latency) based on </ins> data sent by the receiver. <del> 4.3.2. </del> <ins> 3.3.2. </ins> Where media is sent onwards to another platform for further distribution.  The media may be compressed down to a bitrate lower than source, but larger than final distribution output.  Streams may be redundant with failover mechanisms in place. <del> 4.3.3. </del> <ins> 3.3.3. </ins> Where media is received from a live broadcast or stream.  This may comprise of multiple audio or video outputs with different codecs or
they consider implementation and deployment of systems that rely on the MOQ protocol. <del> *Note: the initial high-level organization for this section is taken from Suhas Nandakumar's presentation, "Progressing MOQ" Prog-MOQ, at the October 2022 MOQ virtual interim meeting, which was in turn taken from the MOQ working group charter MOQ-charter.  We think this is a reasonable starting point.  We won't be surprised to see the high- level structure change a bit as things develop, but we didn't want to have this section COMPLETELY blank when we request working group adoption. TODO: Describe overall, high level requirements that we previously stated in earlier versions of this document. </del> 4.2. <ins> 4.2.1. 4.2.2. 4.2.3. 4.2.4. 4.2.5. 4.3. </ins> Many of the use cases have bi-directional flows of media, with clients both sending and receiving media concurrently, thus the protocol should have a unified approach in connection negotiation and
packaging of media, taking into consideration the various technical trade offs that each provide. <ins> 4.5.1. 4.5.2. 4.5.3. </ins> 4.6. <ins> 4.6.1. 4.6.2. 4.6.3. 4.7. 4.7.1. 4.7.2. 4.7.3. 4.7.4. 4.7.5. 4.8. </ins> End-to-end security describes the use of encryption of the media stream(s) to provide confidentiality in the presence of unauthorized intermediates or observers and prevent or restrict ability to decrypt
Our goal in this section is to understand the requirements that result from the use cases described in overallusecases. <ins> 4.1. Note: the intention for the requirements in this document is that they are useful for MOQ working group participants, to recognize constraints, and useful for readers outside the MOQ working group to understand the high-level functionality of the MOQ protocol, as they consider implementation and deployment of systems that rely on the MOQ protocol. </ins> *Note: the initial high-level organization for this section is taken from Suhas Nandakumar's presentation, "Progressing MOQ" Prog-MOQ, at the October 2022 MOQ virtual interim meeting, which was in turn taken
5. <del> This document makes no requests of IANA. </del> <ins> This section covers topics that are explicitly out of scope for the time being. 5.1. The creation of new media transport protocols should be avoided, and instead we should make use of RTP RFC3550 and the existing ecosystem of payload formats and methods of signalling where possible.  It may transpire a need to extend these specificiations; in which case we should work with the relevant working groups and present our use- cases. </ins> 6. <ins> This document makes no requests of IANA. 7. </ins> As this document is intended to create discussion and consensus and introduces no security considerations of its own. <del> 7.  References </del> <ins> 8.  References </ins> <del> 7.1.  URIs </del> <ins> 8.1.  URIs </ins> [1] https://github.com/fiestajetsam/I-D/tree/main/draft-gruessing- moq-requirements
This document describes use cases and requirements that guide the specification of a simple, low-latency media delivery solution for ingest and distribution MOQ-charter, using either the QUIC protocol <del> RFC9000 or WebTransport WebTrans-charter. </del> <ins> RFC9000 or WebTransport WebTrans-charter as transport protocols. </ins> 1.1. <del> This version of the document is intended to provide the MOQ working group with a starting point for work on the "Use Cases and Requirements document" milestone.  The update implements the work plan described in MOQ-ucr.  The authors intend to request MOQ working group adoption after IETF 115, so the working group can begin to focus on these topics in earnest. </del> <ins> When adopted, this document is intended to capture use cases that are in scope for work on the MOQ protocol MOQ-charter, and requirements that arise from these use cases. As of this writing, the authors have not planned to request publication on this document, based on our understanding of the IESG's statement on "Support Documents in IETF Working Groups" IESG- sdwg, which says (among other things): It seems reasonable for the working group to improve this document, and then consider whether the result justifies publication as a part of the RFC archival document series. </ins> 2.
4.6. <del> Packaging of media describes how encapsulation of media to carry the raw media will work.  There are at a high level two approaches to this: </del> <ins> Packaging of media describes how raw media will be encapsulated. There are at a high level two approaches to this: </ins> Within the protocol itself, where the protocol defines the <del> carrying for each media encoding the ancillary data required for decoding the media. A common encapsulation format such as ISOBMFF which defines a generic method for all media and handles ancillary decode information. </del> <ins> ancillary data required to decode each media type the protocol supports. A common encapsulation format such as ISOBMFF ISOBMFF which defines a generic method for all media and handles ancillary decode information. </ins> The working group must agree on which approach should be taken to the packaging of media, taking into consideration the various technical <del> trade offs that each provide.  If the working group decides on a common encapsulation format, the mechanisms within the protocol SHOULD allow for new encapsulation formats to be used. </del> <ins> trade offs that each approach provides. If the working group decides to describe media encapsulation as part of the MOQ protocol, this will require a new version of the MOQ protocol in order to signal the receiver that a new media encapsulation format may be present. If the working group decides to use a common encapsulation format, the mechanisms within the protocol SHOULD allow for new encapsulation formats to be used.  Without encapsulation agility, adding or changing the way media is encapsulated will also require a new version of the MOQ protocol, to signal the receiver that a new media encapsulation format may be present. MOQ protocol specifications will provide details on the supported media encapsulation(s). </ins> 4.7.
this ordering during congestion without increasing latency. The encoder determines how to behave during congestion by assigning <del> each object a numeric delivery order.  The delivery order SHOULD be followed when possible to ensure that the most important media is delivered when throughput is limited.  Note that the contents within each object are still delivered in order; this delivery order only applies to the ordering between objects. </del> <ins> each object a numeric send order.  The send order SHOULD be followed when possible to ensure that the most important media is delivered when throughput is limited.  Note that the contents within each object are still delivered in order; this send order only applies to the ordering between objects. </ins> A sender MUST send each object over a dedicated QUIC stream.  The QUIC library should support prioritization (prioritization) such that <del> streams are transmitted in delivery order. </del> <ins> streams are transmitted in send order. </ins> <del> A receiver MUST NOT assume that objects will be received in delivery </del> <ins> A receiver MUST NOT assume that objects will be received in send </ins> order for a number of reasons: <del> Newly encoded objects MAY have a smaller delivery order than </del> <ins> Newly encoded objects MAY have a smaller send order than </ins> outstanding objects. <del> Packet loss or flow control MAY delay the delivery of individual </del> <ins> Packet loss or flow control MAY delay the send of individual </ins> streams. The sender might not support QUIC stream prioritization.
The decoder will receive multiple objects in parallel and out of order. <del> Objects arrive in delivery order, but media usually needs to be processed in decode order.  The decoder SHOULD use a buffer to reassmble objects into decode order and it SHOULD skip objects after a configurable duration.  The amount of time the decoder is willing to wait for an object (buffer duration) is what ultimately determines </del> <ins> Objects arrive in send order, but media usually needs to be processed in decode order.  The decoder SHOULD use a buffer to reassmble objects into decode order and it SHOULD skip objects after a configurable duration.  The amount of time the decoder is willing to wait for an object (buffer duration) is what ultimately determines </ins> the end-to-end latency. Objects MUST synchronize frames within and between tracks using
this to priority-congestion. The producer may assign a numeric delivery order to each object <del> (delivery-order) </del> <ins> (send-order) </ins> This is a strict prioritization scheme, such that any available bandwidth is allocated to streams in ascending priority order.  The <del> sender SHOULD prioritize streams based on the delivery order.  If two streams have the same delivery order, they SHOULD receive equal bandwidth (round-robin). </del> <ins> sender SHOULD prioritize streams based on the send order.  If two streams have the same send order, they SHOULD receive equal bandwidth (round-robin). </ins> QUIC supports stream prioritization but does not standardize any mechanisms; see Section 2.3 in QUIC.  In order to support
Warp encodes the delivery information for a stream via OBJECT headers (message-object). <del> A relay SHOULD prioritize streams (prioritization) based on the delivery order.  A relay MAY change the delivery order, in which case it SHOULD update the value on the wire for future hops. </del> <ins> A relay SHOULD prioritize streams (prioritization) based on the send order.  A relay MAY change the send order, in which case it SHOULD update the value on the wire for future hops. </ins> A relay that reads from a stream and writes to stream in order will introduce head-of-line blocking.  Packet loss will cause stream data