Code change:
Note - this is a rough draft, although I have done some edits while adding this material. I do have notes for to-dos, and I'm pretty sure I want to figure out how to do the diagrams in some way that will format with straight lines, for starters.
I'm happy to just merge this in now as is just to unblock other work and make progress, and make any adjustments later because there's nothing that stands out as controversial, NAME are you okay with this?
We have a plan on how to resolve our terminology migration for in this PR
Based on review comments from NAME on the mailing list .
Draft is in suhasHere/moq-drafts.
NAME - NAME and I are starting with the scenarios from URL
NAME - I think it would be good to close this issue when we have a reasonable plan for where the material from the scenarios draft goes, and then handle further changes via issues and PRs as usual. Does that make sense? NAME NAME and NAME - I wanted to let you know that NAME and I are (finally, finally!) merging text from the scenarios draft in the requirements draft, as NAME suggested. I think I have homes for almost all of the text from the requirements draft, but I do have some work to do, so that the material is in the right place, and that it flows with text from the previous version of the requirements draft. I am, of course, always interested in comments. And thank you each for the work you did on this material.
Awesome - let me know if something I can do to help
Thanks NAME .. look forward to the updates. +1 to NAME , please let me know if we can be of any help Here is my review. Firstly, I'd like to say that this document is badly needed. The requirements document doesn't get into the level of detail that this document does, and I think that this level of detail is important to make progress on the design. Terminology I think it would be useful to define some of the terminology, including terms like "track" and "media switch". Section 2.2 A few questions: Are the trackIDs "t1, t2, t3" separate MediaStreamTracks, or are they just multiple encodings of the same track? Are there scenarios in which it would be useful to be able to ingest multiple MediaStreamTracks (either audio or video)? I ask because the RUSH draft doesn't support ingestion of multiple tracks and I've been told that this is a problem by some customers. Section 2.4 A media switch, sourcing tracks that represent a subset of tracks from across all the emitters. Such subset may represent tracks representing top 5 speakers at higher qualities and lot of other tracks for rest of the emitters at lower qualities. [BA] The use of the term "media switch" suggests to me that it is forwarding tracks, rather than sourcing (e.g. transcoding) them. Can you clarify? subscribing to subset of the tracks [BA] I think there may be an implicit assumption here, which is that the receiver only indicates the track it wishes to subscribe to, rather than indicating the quality it wishes to receive. Above setup brings in following properties on the data model for the transport protocol [BA] The paragraphs that follow appear to describe different ways that the system could be implemented, rather than "properties on the data model". Do you mean to say that the design should be able to handle any of the approaches described? Media Switches to source new tracks but retain media payload from the original emitters. This implies publishing new Track IDs sourced from the SFU, with object payload unchanged from the original emitters. [BA] Is the term "source new tracks" implied by changing the Track IDs? If the payload is retained, this seems like forwarding to me, rather than "sourcing new tracks". Media Switches to propogate subset of tracks as-is from the emitters to the subscribers. This implies Track IDs to be unchanged between the emitters and the receivers. [BA] What does "as-is" mean here? Are you referring only the lack of a change in the "Track ID"? If the switch drops some frames in order to reduce quality (and bandwidth consumption) is it still "as-is"? Subscribers to explictily request multiple appropriate qualities and dynamically move between the qualtiies during the course of the session [BA] Is it a goal to be able to support such a "receiver driven" approach, as well as the other two approaches, which are sender driven? Section 3.1 For example, rather than waiting for 30 seconds before connecting, the user might quickly download the "key" frames of the past 30 seconds and replay them in order to "synchronize" the video decoder. [BA] I think this sentence points out that the term "access point" may need to be clarified, because in such a scenario, the term "access point" may have a different meaning in stored media than in media transmitted over the wire to speed up join latency. Section 3.2 It is possible to use groups as units of congestion control. When the sending strategy is understoud, the objects in the group can be assigned sequence numbers and drop priorities that capture the encoding dependencies, such that: an object can only have dependencies with other objects in the same group, an object can only have dependencies with other objects with lower sequence numbers, an object can only have dependencies with other objects with lower or equal drop priorities. [BA] This text seems to be mixing distinct concepts such as dependencies and layering. In layered coding, there is typically only a requirement not to depend on higher layers, but it's possible to depend on frames in lower layers. Mixing drop priority (a transport concept) with groups seems confusing to me. The main drawback is that if a packet with a given drop priority is actually dropped, all objects with higher sequence numbers and higher or equal drop priorities in the same group must be dropped. [BA] Leaving gaps in the sequence number space does seem problematic, but that's a choice, not a requirement. If the group duration is long, this means that the quality of experience may be lowered for a long time after a brief congestion. If the group duration is short, this can produce a jarring effect in which the quality of experience drops perdiodically at the tail of the group. [BA] What does "group duration" mean? Are you referring to time between I-frames? This paragraph seems to assume that group duration is a constant rather than something which can be set dynamically. Section 4 Some video codecs have a complex structure. [BA] Are you referring to codecs, or encodings? It would send for example: [BA] A picture might help here. I think you're referring to the scalabilty mode known as "L2T2h": URL Of course, we could encode these dependencies as properties of the object being sent, stating for example that "object 17 can only be decoded if objects 16, 11 and 7 are available." However, this approach leads to a lot of complexity in relays. We believe that a linear approach is preferable, using attributes of objects like delivery order or priorities. [BA] Why would this lead to complexity in relays in a receiver-driven scenario? The relay would only forward the layers that the receiver asked for, and wouldn't have to store state relating to the receivers. Also, the real issue here isn't dropping layers, which can occur at any point. The reason dependencies (and more) are needed (and why the "linear approach" doesn't work) comes when it is desired to add layers. It is of course possible just to start forwarding additional layers, but that doesn't improve quality unless the forwarded frames can actually be decoded. To figure this out, whoever is in charge (receiver or relay) needs to be able to determine the "upswitch points". This also implies feedback from the receivers to the encoder (e.g. LRR). Section 4.2 We propose to express dependencies using a combination of object number and object priority. [BA] How would this permit restoration of video quality after congestion clears up? I don't think you have enough information. Section 4.3 . Relay behavior In case of congestion, the relay will use the priorities to selectively drop the "least important" objects: [BA] Dropping is easy, it's adding layers back (and decoding them) that's hard.

Related discussion:
text: Behaviours around cache durations for each MoQ entity should be defined. <ins> 4.8.3. In case of congestion, the relay will use the priorities to selectively drop the "least important" objects: if congestion is noticed, the relay will drop first the lesser priority layer.  In our example, that would mean the objects marked at priority 6.  The relay will drop all objects marked at that priority, from the first dropped object to the end of the group. if congestion persists despite dropping a first layer, the relay will start dropping the next layer, in our example the objects marked at priority 5. if congestion still persist after dropping all but the highest priority layer, the relay will have to close the group, and start relaying the next group. When dropping objects within the same priority: higher object numbers in the same group, which are later in the group, are "less important" and more likely to be dropped than objects in the same group with a lower object number.  Objects in a previous group are "less important" than objects in the current group and MAY be dropped ahead of objects in the current group. The specification above assumes that the relay can detect the onset of congestion, and has a way to drop objects.  There are several ways to achieve that result, such as sending all objects of a group in a single QUIC stream and making explicit action at the time of relaying, or mapping separate priority layers into different QUIC streams and marking these streams with different priorities.  The exact solution will have to be defined in a draft that specifies transport priorities. 4.8.4. Web conferencing systems are used on networks with well over 20% packet loss and when this happens, it is often on connections with a relatively large round trip times.  In these situtation, forward error correction or redundant transmitions are used to provide a reasonable user experience.  Often video is turned off in.  There are multiple machine learning based audio codecs in development that targeting a 2 to 3 Kbps rate. This can result in scenarios where very small audio objects are sent at a rate of several hundreds packets per second with a high network loss rate. 4.8.5. In the streaming scenarios, there is an important emphasis on resynchronization, characterized by a short distance between "access points".  This can be used for features like fast-forward or rewinding, which are common in non-real-time streaming.  For real- time streaming experiences such as watching a sport event, frequent access points allow "channel surfers" to quickly join the broadcast and enjoy the experience.  The interval between these access points will often be just a few seconds. In video encoding, each access point is mapped to a fully encoded frame that can be used as reference for the "group of blocks".  The encoding of these reference frames is typically much larger than the differential encoding of the following frames.  This creates a peak of traffic at the beginning of the group.  This peak is much easier to absorb in streaming applications that tolerate higher latencies than interactive video conferences.  In practice, many real time conferences tend to use much longer groups, resulting in higher compression ratios and smoother bandwidth consumption along with a way to request the start of a new group when needed.  Other real time conferences tend to use very short groups and just wait for the next group when needed. Of course, having longer blocks create other issues.  Realtime conferences also need to accomodate the occasional occasional late comer, or the disconnected user who want to resynchronize after a network event.  This drives a need for synchronization "between access points".  For example, rather than waiting for 30 seconds before connecting, the user might quickly download the "key" frames of the past 30 seconds and replay them in order to "synchronize" the video decoder. </ins> 4.9. 4.9.1.