Code change:
Note - this is a rough draft, although I have done some edits while adding this material. I do have notes for to-dos, and I'm pretty sure I want to figure out how to do the diagrams in some way that will format with straight lines, for starters.
I'm happy to just merge this in now as is just to unblock other work and make progress, and make any adjustments later because there's nothing that stands out as controversial, NAME are you okay with this?
We have a plan on how to resolve our terminology migration for in this PR
Based on review comments from NAME on the mailing list .
Draft is in suhasHere/moq-drafts.
NAME - NAME and I are starting with the scenarios from URL
NAME - I think it would be good to close this issue when we have a reasonable plan for where the material from the scenarios draft goes, and then handle further changes via issues and PRs as usual. Does that make sense? NAME NAME and NAME - I wanted to let you know that NAME and I are (finally, finally!) merging text from the scenarios draft in the requirements draft, as NAME suggested. I think I have homes for almost all of the text from the requirements draft, but I do have some work to do, so that the material is in the right place, and that it flows with text from the previous version of the requirements draft. I am, of course, always interested in comments. And thank you each for the work you did on this material.
Awesome - let me know if something I can do to help
Thanks NAME .. look forward to the updates. +1 to NAME , please let me know if we can be of any help Here is my review. Firstly, I'd like to say that this document is badly needed. The requirements document doesn't get into the level of detail that this document does, and I think that this level of detail is important to make progress on the design. Terminology I think it would be useful to define some of the terminology, including terms like "track" and "media switch". Section 2.2 A few questions: Are the trackIDs "t1, t2, t3" separate MediaStreamTracks, or are they just multiple encodings of the same track? Are there scenarios in which it would be useful to be able to ingest multiple MediaStreamTracks (either audio or video)? I ask because the RUSH draft doesn't support ingestion of multiple tracks and I've been told that this is a problem by some customers. Section 2.4 A media switch, sourcing tracks that represent a subset of tracks from across all the emitters. Such subset may represent tracks representing top 5 speakers at higher qualities and lot of other tracks for rest of the emitters at lower qualities. [BA] The use of the term "media switch" suggests to me that it is forwarding tracks, rather than sourcing (e.g. transcoding) them. Can you clarify? subscribing to subset of the tracks [BA] I think there may be an implicit assumption here, which is that the receiver only indicates the track it wishes to subscribe to, rather than indicating the quality it wishes to receive. Above setup brings in following properties on the data model for the transport protocol [BA] The paragraphs that follow appear to describe different ways that the system could be implemented, rather than "properties on the data model". Do you mean to say that the design should be able to handle any of the approaches described? Media Switches to source new tracks but retain media payload from the original emitters. This implies publishing new Track IDs sourced from the SFU, with object payload unchanged from the original emitters. [BA] Is the term "source new tracks" implied by changing the Track IDs? If the payload is retained, this seems like forwarding to me, rather than "sourcing new tracks". Media Switches to propogate subset of tracks as-is from the emitters to the subscribers. This implies Track IDs to be unchanged between the emitters and the receivers. [BA] What does "as-is" mean here? Are you referring only the lack of a change in the "Track ID"? If the switch drops some frames in order to reduce quality (and bandwidth consumption) is it still "as-is"? Subscribers to explictily request multiple appropriate qualities and dynamically move between the qualtiies during the course of the session [BA] Is it a goal to be able to support such a "receiver driven" approach, as well as the other two approaches, which are sender driven? Section 3.1 For example, rather than waiting for 30 seconds before connecting, the user might quickly download the "key" frames of the past 30 seconds and replay them in order to "synchronize" the video decoder. [BA] I think this sentence points out that the term "access point" may need to be clarified, because in such a scenario, the term "access point" may have a different meaning in stored media than in media transmitted over the wire to speed up join latency. Section 3.2 It is possible to use groups as units of congestion control. When the sending strategy is understoud, the objects in the group can be assigned sequence numbers and drop priorities that capture the encoding dependencies, such that: an object can only have dependencies with other objects in the same group, an object can only have dependencies with other objects with lower sequence numbers, an object can only have dependencies with other objects with lower or equal drop priorities. [BA] This text seems to be mixing distinct concepts such as dependencies and layering. In layered coding, there is typically only a requirement not to depend on higher layers, but it's possible to depend on frames in lower layers. Mixing drop priority (a transport concept) with groups seems confusing to me. The main drawback is that if a packet with a given drop priority is actually dropped, all objects with higher sequence numbers and higher or equal drop priorities in the same group must be dropped. [BA] Leaving gaps in the sequence number space does seem problematic, but that's a choice, not a requirement. If the group duration is long, this means that the quality of experience may be lowered for a long time after a brief congestion. If the group duration is short, this can produce a jarring effect in which the quality of experience drops perdiodically at the tail of the group. [BA] What does "group duration" mean? Are you referring to time between I-frames? This paragraph seems to assume that group duration is a constant rather than something which can be set dynamically. Section 4 Some video codecs have a complex structure. [BA] Are you referring to codecs, or encodings? It would send for example: [BA] A picture might help here. I think you're referring to the scalabilty mode known as "L2T2h": URL Of course, we could encode these dependencies as properties of the object being sent, stating for example that "object 17 can only be decoded if objects 16, 11 and 7 are available." However, this approach leads to a lot of complexity in relays. We believe that a linear approach is preferable, using attributes of objects like delivery order or priorities. [BA] Why would this lead to complexity in relays in a receiver-driven scenario? The relay would only forward the layers that the receiver asked for, and wouldn't have to store state relating to the receivers. Also, the real issue here isn't dropping layers, which can occur at any point. The reason dependencies (and more) are needed (and why the "linear approach" doesn't work) comes when it is desired to add layers. It is of course possible just to start forwarding additional layers, but that doesn't improve quality unless the forwarded frames can actually be decoded. To figure this out, whoever is in charge (receiver or relay) needs to be able to determine the "upswitch points". This also implies feedback from the receivers to the encoder (e.g. LRR). Section 4.2 We propose to express dependencies using a combination of object number and object priority. [BA] How would this permit restoration of video quality after congestion clears up? I don't think you have enough information. Section 4.3 . Relay behavior In case of congestion, the relay will use the priorities to selectively drop the "least important" objects: [BA] Dropping is easy, it's adding layers back (and decoding them) that's hard.

Related discussion:
text: control of packetization and transmission, with addtional support for retransmission as an optional extension. <ins> To provide an overview of interactive use cases, we can consider a conferencing session comprised of: Multiple emitters, publishing on multiple tracks (audio, video tracks and at different qualities) A media switch, sourcing tracks that represent a subset of tracks from across all the emitters.  Such subset may represent tracks representing top 5 speakers at higher qualities and lot of other tracks for rest of the emitters at lower qualities. Multiple receivers, with varied receiving capacity (bandwidth limited), subscribing to subset of the tracks This setup relies on the following functionalities: Media Switches source new tracks but retain the media payload from the original emitters.  This implies publishing new Track IDs sourced from the SFU, with object payload unchanged from the original emitters. Media Switches propagate a subset of tracks as-is from the emitters to the subscribers.  This implies Track IDs to be unchanged between the emitters and the receivers. Subscribers explictly request one or more media tracks in appropriate qualities and dynamically move between the qualtiies during the course of the session. Another topology for the conferencing use-case is to use multiple distribution networks for delivering the media, with media switching functionality running across distribution networks and these media functions as part of the core distribution network as shown below. Such a topology needs to meet all the properties listed in the homogenous topology setup, however having multiple distribution networks, and relying on the distribution networks to carry out the media delivery, brings in further requirements towards a data model that enables tracks to be uniquely identifiable across the distribution networks and not just within a single distribution network. </ins> 3.1.1. In this use case the computation for running a video game (single or