jiebi/CodeConvo huggingface dataset:
of the past 30 seconds and replay them in order to "synchronize" the video decoder. <ins> 4.8.6. In all of the applicable use cases defined in overallusecases it may be necessary for consumers to be aware of changes to the source of media being inserted, or be instructed to consume media from a different source.  These may be done for the insertion of advertising or for operational movement of consumers, amongst other reasons. Within the media insertion scenario an existing stream being consumed may change as a result of a different source being spliced which necessitates the decoder being reset as parameters such as video frame rate, image resolution etc may have changed.  For redirection, consumers may be signalled to consume media from a different source which may also require re-initialization of decoder. In both of these scenarios, triggering may occur either through an event provided in the media such as a SCTE-35 marker, or through an external trigger.  Both should be supported. </ins> 4.9. 4.9.1.
<del> Media over QUIC Requirements and Use Cases </del> <ins> Media over RTP over QUIC Requirements and Use Cases </ins> draft-gruessing-moq-requirements-latest Abstract This document describes the uses cases, requirements, and <del> considerations that should be part of the design of a real-time media protocol over QUIC RFC9000. </del> <ins> considerations that should be form the design of the encapsulation of a real-time media transport protocol operating over QUIC RFC9000. </ins> Note to Readers
4.3. <ins> TODO: Confirm these requirements, consider looking at how RFC 8836 applies to this requirement. 4.4. </ins> TODO: confirm scope of this draft to describe lossless media transport, lossy media transport, or both lossless and lossy transport. <del> 4.4. </del> <ins> 4.5. </ins> Media should be able to flow in either direction from client to server or vice-versa, either individually or concurrently. <del> 4.5. </del> <ins> 4.6. </ins> TODO: Unsure if this should be a requirement.  If it is, we have to consider two things: WebTransport supports HTTP/2, are we going to
overview has normative language around congestion control which may be at odds with our potential requirements. <del> 4.6. </del> <ins> 4.7. The protocol SHOULD have capabilities beyond what QUIC provides to allow hosts to authenticate one another, this should be kept simple but robust in nature to prevent attacks like credential brute- forcing. </ins> <del> The protocol SHOULD have capabilities asides from TLS mutual authentication to allow hosts to authenticate one another, this should be kept simple but robust in nature to prevent attacks like credential brute-forcing. </del> <ins> TODO: More details are required here </ins> 5.
This document describes the uses cases, requirements, and considerations that should guide the design of the encapsulation of a <del> real-time media transport protocol as a payload in the the QUIC protocol RFC9000. </del> <ins> real-time media transport protocol as a payload in the QUIC protocol RFC9000. </ins> Protocol developers have been considering the implications of the QUIC protocol (RFC9000) on media transport for several years, but the
and delivery of SRT on top of QUIC using datagram frame types.  This specification sets some requirements regarding how the two interact and leaves considerations for congestion control and pacing to <del> prevent conflict between the two protocols. </del> <ins> prevent conflict between the two protocols.  Apart from that, SRT provides a native suport for stream multiplexing, thus contributing this missing functionality to QUIC datagrams. </ins> 3.5.
Interactive client-server applications.  For example, a "click here to speak to a representative" button on a website that starts an <del> interactive WebRTC call.  Such applications avoid the NAT traversal issues that complicate peer-to-peer use of QUIC, and can benefit from stream demultiplexing and (if appropriate algorithms are provided) </del> <ins> interactive WebRTC call.  Because a connection is being established to a known server, such applications avoid the NAT traversal issues that complicate peer-to-peer use of QUIC, and can benefit from stream demultiplexing and (if appropriate algorithms are provided) </ins> congestion control.  They would benefit from unreliable delivery modes to reduce latency.
4.2.2. <ins> While a media is being contributed in real time via an ingest point to a streaming platform, there may be a need to distribute the media between several servers and/or perform a load balancing of the incoming traffic.  The platform may peform transcoding at one or several points, or distribute (route) the media as is to one or many egress points.  Such a distribution and load balancing may happen with or without the access to the encrypted payload. 4.2.3. </ins> Distribution from platform to audience.  Whilst use of WebRTC or RTSP today for On-Demand media streaming is not typical with adaptive streaming like HLS and DASH being predominantly used as WebRTC is
often used in delivering media in our use cases, QUIC doesn't yet support multicast, and would require a QUIC protocol extension to do so.  In addition, the inclusion of multicast would introduce more <del> complexity in both the specification and client implimentations. </del> <ins> complexity in both the specification and client implimentations.  On the other hand, UDP multicast may be considered as the last mile delivery transport outside of QUIC transport, thus it would be beneficial for a protocol to provide such an opportunity (e.g.  RTP/ QUIC -> RTP/UDP). </ins> 7.
https://github.com/fiestajetsam/draft-gruessing-moq-requirements [1]. Discussion of this draft should take place on the IETF Media Over <del> QUIC (MOQ) mailing list, at https://www.ietf.org/mailman/listinfo/ moq. </del> <ins> QUIC (MoQ) mailing list, at https://www.ietf.org/mailman/listinfo/moq [2]. </ins> 1.
comprise of multiple audio or video outputs with different codecs or bitrates.  This may also include other types of media essence such as subtitles or timing signalling information (e.g. markers to indicate <del> change of behaviour in client such as advertisement breaks) </del> <ins> change of behaviour in client such as advertisement breaks).  The use of "live rewind" where a window of media behind the live edge can be made available for clients to playback, either because the local player falls behind edge or because the viewer wishes to play back from a point in the past. </ins> 4.6.
beneficial for a protocol to provide such an opportunity (e.g.  RTP/ QUIC -> RTP/UDP). <del> 7. </del> <ins> 8. </ins> This document makes no requests of IANA. <del> 8. </del> <ins> 9. </ins> As this document is intended to guide discussion and consensus, it introduces no security considerations of its own. <del> 9.  References </del> <ins> 10.  References </ins> <del> 9.1.  URIs </del> <ins> 10.1.  URIs </ins> [1] https://github.com/fiestajetsam/draft-gruessing-moq-requirements <ins> [2] https://www.ietf.org/mailman/listinfo/moq </ins>
for the ones that do, it seems useful to have a name for "the protocol layer immediately beneath media". <ins> Within this document, we extend the latency requirement categories for streaming media described in I-D.draft-ietf-mops-streaming- opcons: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) These latency bands were appropriate for streaming media, which was the target for I-D.draft-ietf-mops-streaming-opcons, but some realtime media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using ull500 (less than 500 ms) ull100 (less than 100 ms) Obviously, these last two latency bands are the shortened form of "ultra-low latency - 500 ms" and "ultra-low-latency - 100 ms".  Also obviously, bikeshedding on better names is welcomed. </ins> 3. Note - need to edit this section to reflect new draft scope and
6.2. <del> I-D.draft-ietf-mops-streaming-opcons describes these latency requirements for streaming media: ultra low-latency (less than 1 second) low-latency live (less than 10 seconds) non-low-latency live (10 seconds to a few minutes) on-demand (hours or more) Support for a nominal latency in the low to ultra-low latency should be achieved, with consideration for minimum buffer a receiver playing content may need to handle congestion, packet loss, and other degradation in network quality. </del> <ins> Support for a nominal latency appropriate for the use cases that are in scope should be achieved, with consideration for the minimum buffer that a receiver playing content may need to handle congestion, packet loss, and other degradation in network quality. </ins> 6.3.
7.2. <del> The creation of new media transport protocols should be avoided, and instead we should make use of RTP RFC3550 and the existing ecosystem of payload formats and methods of signalling where possible.  Work on QUIC encapsulation may reveal a need to extend these specificiations; in which case we should work with the relevant working groups and present our use-cases. 7.3. </del> Even if multicast and other network broadcasting capabilities are often used in delivering media in our use cases, QUIC doesn't yet support multicast, and would require a QUIC protocol extension to do
Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames <del> RUSH introduces new frame types as its development pre-dates Datagram frames </del> All drafts take differing approaches to flow/stream identification and management; some address congestion control and others just omit the subject and leave it to QUIC to handle
3.3. <del> RUSH uses its own frame types on top of QUIC as it pre-dates the datagram specification; in addition individual media frames are given their own stream identifiers to remove HoL blocking from processing out-of-order. </del> <ins> Whilst RUSH predates the datagram specification, it uses its own frame types on top of QUIC to take advantage of QUIC implementations reassembling messages greater than MTU.  In addition individual media frames are given their own stream identifiers to remove HoL blocking from processing out-of-order. </ins> It defines its own registry for signalling codec information with room for future expansion but presently is limited to a subset of
4.5. <ins> Warp's specification attemps to map Group of Picture encoding of video on top of QUIC streams.  It depends on ISOBMFF containers to encapsulate both media as well as messaging, and defines prioritisation with separate considerations for audio and video.  It doesn't yet define bi-directionality of media flows, and can be run over protocols like WebTransport I-D.draft-ietf-webtrans-overview. 4.6. </ins> Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames All drafts take differing approaches to flow/stream identification and management; some address congestion control and others just omit the subject and leave it to QUIC to handle <del> Both QRT and RUSH specify ALPN identification; the Engelbart and SRT drafts do not. Note - need to add WARP and verify these characterizations. </del> <ins> Both QRT and RUSH specify ALPN identification; the Engelbart, Warp, and SRT drafts do not. </ins> <del> 4.6. </del> <ins> 4.7. </ins> It's worth noting that work on "RTP over QUIC" is being considered in the AVTCORE working group at this time, although no proposals have
Abstract <del> This document describes the uses cases, requirements, and considerations that should guide the design of the encapsulation of a real-time media transport protocol as a payload in the QUIC protocol, that will be used for live media contribution, syndication, and streaming. </del> <ins> This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends use cases on live media contribution, syndication, and streaming as the basis for discussions that should guide the design of protocols to satisfy these use cases. </ins> Note to Readers
1. <del> This document describes the uses cases, requirements, and considerations that should guide the design of the encapsulation of a real-time media transport protocol as a payload in the QUIC protocol (RFC9000), that will be used for live media contribution, syndication, and streaming. Protocol developers have been considering the implications of the QUIC protocol (RFC9000) on media transport for several years, but the initial focus on QUIC in the IETF was to support web applications that used the HTTP/3 protocol I-D.draft-ietf-quic-http.  The completion of the initial versions of the QUIC specifications, and the adoption of I-D.draft-ietf-quic-datagram, have cleared the way for proposals to use QUIC as a media transport.  This document considers a number of proposals for "Media Over QUIC", and analyzes them to understand requirements and considerations. </del> <ins> This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends use cases on live media contribution, syndication, and streaming as the basis for requirements discussions that should guide the design of protocols to satisfy these use cases. </ins> 1.1.
2. <ins> 2.1. Protocol developers have been considering the implications of the QUIC protocol (RFC9000) for media transport for several years, resulting in a large number of possible meanings of the term "Media Over QUIC", or "MOQ".  As of this writing, "Media Over QUIC" has had at least these meanings: any kind of media carried directly over the QUIC protocol, as a QUIC payload any kind of media carried indirectly over the QUIC protocol, as an RTP payload any kind of media carried indirectly over the QUIC protocol, as an HTTP/3 payload any kind of media carried indirectly over the QUIC protocol, as a WebTransport payload the encapsulation of any Media Transport Protocol (mtp) in a QUIC payload an IETF mailing list (MOQ-ml) "... for discussion of video ingest and distribution protocols that use QUIC as the underlying transport" There may be IETF participants using other meanings as well. As of this writing, the second bullet ("any kind of media carried indirectly over the QUIC protocol, as an RTP payload"), seems to be in scope for the IETF AVTCORE working group, and was discussed at some length at the February 2022 AVTCORE working group meeting AVTCORE-2022-02.  So, perhaps, that possible meaning is out of scope for "Media over QUIC". It will be SUPER HELPFUL if interested parties can come up with a term that unambiguously describes what we're trying to achieve. 2.2. </ins> Within this document, we use the term "Media Transport Protocol". This is easier to understand if the reader assumes that we are starting with a protocol stack that looks like this:
On-Demand media streaming (odstream) is unlikely to benefit from work in this space.  Without notable latency requirements, protocols such <del> as HLS and DASH largely meet the needs of this use case.  However larger deployments may benefit from the use of HTTP/3 I-D.draft-ietf- quic-http. </del> <ins> as HLS and DASH largely meet the needs of this use case. </ins> 7.
This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends <del> use cases on live media contribution, syndication, and streaming as the basis for discussions that should guide the design of protocols to satisfy these use cases. </del> <ins> use cases on live media ingest, syndication, and streaming as the basis for discussions that should guide the design of protocols to satisfy these use cases. </ins> Note to Readers
This document describes the use cases that have been discussed in the IETF community under the banner of "Media Over QUIC", and recommends <del> use cases on live media contribution, syndication, and streaming as the basis for requirements discussions that should guide the design of protocols to satisfy these use cases. </del> <ins> use cases on live media ingest, syndication, and streaming as the basis for requirements discussions that should guide the design of protocols to satisfy these use cases. </ins> 1.1.
These latency bands were appropriate for streaming media, which was the target for I-D.draft-ietf-mops-streaming-opcons, but some <del> realtime media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using </del> <ins> interactive media may have requirements that are significantly less than "ultra-low latency".  Within this document, we are also using </ins> <del> ull500 (less than 500 ms) ull100 (less than 100 ms) </del> <ins> Ull-50 (less than 50 ms) Ull-200 (less than 200 ms) </ins> Obviously, these last two latency bands are the shortened form of <del> "ultra-low latency - 500 ms" and "ultra-low-latency - 100 ms".  Also obviously, bikeshedding on better names is welcomed. </del> <ins> "ultra-low latency - 50 ms" and "ultra-low-latency - 200 ms". Perhaps less obviously, bikeshedding on better names and more useful values is welcomed. </ins> 3.
3.4. Secure Reliable Transport (SRT) (I-D.draft-sharabayko-srt) itself is <del> a general purpose transport protocol primarily for contribution transport use cases and this specification covers the encapsulation and delivery of SRT on top of QUIC using datagram frame types.  This </del> <ins> a general purpose transport protocol primarily for ingest transport use cases and this specification covers the encapsulation and delivery of SRT on top of QUIC using datagram frame types.  This </ins> specification sets some requirements regarding how the two interact and leaves considerations for congestion control and pacing to prevent conflict between the two protocols.  Apart from that, SRT
3.6. <del> Both QRT and the Engelbart draft attempt to use existing payloads of RTP, RTCP, and SDP, unlike RUSH and SRT, as well as using existing Datagram frames </del> <ins> Some drafts attempt to use existing payloads of RTP, RTCP, and SDP, while others do not. Some use QUIC Datagram frames, while others use QUIC streams. </ins> All drafts take differing approaches to flow/stream identification <del> and management; some address congestion control and others just omit the subject and leave it to QUIC to handle Both QRT and RUSH specify ALPN identification; the Engelbart, Warp, and SRT drafts do not. </del> <ins> and management.  Some address congestion control and others just defer this to QUIC to handle. Some drafts specify ALPN identification, while others do not. </ins> 3.7.
4.1. <del> *Senders/Receivers*: Many to Many *Bi-directional*: Yes *Latency*: Ultra-Low Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation.  This may be done as client/server, or peer to peer with a many to many relationship of both senders and receivers. 4.2. </del> <ins> 4.1.1. </ins> *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*: <del> Ull100 </del> <ins> Ull-50 </ins> Where media is received, and user inputs are sent by the client. This may also include the client receiving other types of signalling,
the client such as microphone audio for in-game chat with other players. <del> 4.3. </del> <ins> 4.1.2. </ins> *Senders/Receivers*: One to One *Bi-directional*: Yes *Latency*: <del> Ultra-Low </del> <ins> Ull-50 </ins> Where media is received, and user inputs are sent by the client. Latency requirements with this usecase are marginally different than the gaming use case.  This may also include signalling and/or transmitting of files or devices connected to the user's computer. <del> 4.4. </del> <ins> 4.1.3. </ins> <del> *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: Low to Non-Low </del> <ins> *Senders/Receivers*: Many to Many *Bi-directional*: Yes *Latency*: Ull-200 </ins> <del> Where media is received from a live broadcast or stream.  This may comprise of multiple audio or video outputs with different codecs or bitrates.  This may also include other types of media essence such as subtitles or timing signalling information (e.g. markers to indicate change of behaviour in client such as advertisement breaks).  The use of "live rewind" where a window of media behind the live edge can be made available for clients to playback, either because the local player falls behind edge or because the viewer wishes to play back from a point in the past. </del> <ins> Where media is both sent and received; This may include audio from both microphone(s) or other inputs, or may include "screen sharing" or inclusion of other content such as slide, document, or video presentation.  This may be done as client/server, or peer to peer with a many to many relationship of both senders and receivers. 4.2. </ins> <del> 4.5. </del> <ins> 4.2.1. </ins> *Senders/Receivers*: One to One *Bi-directional*: No *Latency*: <del> Ultra-Low to Low </del> <ins> Ull-200 to Ultra-Low </ins> Where media is received from a source for onwards handling into a distribution platform.  The media may comprise of multiple audio and/
by signalling of connection inforation (bandwidth, latency) based on data sent by the receiver. <del> 4.6. </del> <ins> 4.2.2. </ins> *Senders/Receivers*: One to One *Bi-directional*: No *Latency*: <del> Ultra-Low to Low </del> <ins> Ull-200 to Ultra-Low </ins> Where media is sent onwards to another platform for further distribution.  The media may be compressed down to a bitrate lower than source, but larger than final distribution output.  Streams may be redundant with failover mechanisms in place. <del> 4.7. </del> <ins> 4.2.3. *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: Ull-200 to Ultra-Low Where media is received from a live broadcast or stream.  This may comprise of multiple audio or video outputs with different codecs or bitrates.  This may also include other types of media essence such as subtitles or timing signalling information (e.g. markers to indicate change of behaviour in client such as advertisement breaks).  The use of "live rewind" where a window of media behind the live edge can be made available for clients to playback, either because the local player falls behind edge or because the viewer wishes to play back from a point in the past. 4.3. 4.3.1. ** James, can you fill this section in?** 4.3.2. </ins> *Senders/Receivers*: One to Many *Bi-directional*: No *Latency*: On Demand