Code change:
MOQ protocol specifications will provide details on the supported media encapsulation(s). <ins> 4.6.1. Some video codecs have a complex structure.  Consider an application using both temporal layering and spatial layering.  It would send for example: an object representing the 30 fps frame at 720p an object representing the spatial enhancement of that frame to 1080p an object representing the 60 fps frame at 720p an object representing the spatial enhancement of that 60 fps frame to 1080p The encoding of the 30 fps frame depends on the previous 30 fps frames, but not on any 60 fps frame.  The encoding of the 60 fps depends on the previous 30 fps frames, and possibly also on the previous 60 fps frames (there are options).  The encoding of the spatial enhancement depends on the corresponding 720p frames, and also on the previous 1080p enhancements.  Add a couple of layers, and the expression of dependencies can be very complex.  The AV1 documentation for example provides schematics of a video stream with 3 frame rate options at 15, 30 and 60 fps, and two definition options, with a complex graph of dependencies.  Other video encodings have similar provisions.  They may differ in details, but there are constants: if some object is dropped, then all objects that have a dependency on it are useless. Of course, we could encode these dependencies as properties of the object being sent, stating for example that "object 17 can only be decoded if objects 16, 11 and 7 are available."  However, this approach leads to a lot of complexity in relays.  We believe that a linear approach is preferable, using attributes of objects like delivery order or priorities. 4.6.2. The conversion from dependency graph to linear ordering is not unique.  The simple graph in our example could be ordered either "frame rate first" versus "definition first".  If the application chooses frame rate first, the policy is expressed as "in case of congestion, drop the spatial enhancement objects first, and if that is not enough drop the 60 fps frames".  If the application chooses "definition first", the policy becomes "drop the 60 fps frames and their corresponding 1080p enhancement first, and if that is not enough also drop the 1080p enhancement of the 30 fps frames". More complex graphs will allow for more complex policies, maybe for example "15 fps at 720p as a minimum, but try to ensure at least 30fps, then try to ensure 1080p, and if there is bandwidth available forward 60 fps at 1080p".  Such linearization requires choices, and the choices should be made by the application, based on the user experience requirements of the application. The relays will not understand all the variation of what the media is but the applications will need a way to indicate to the relays the information they will need to correctly order which data is sent first. 4.6.3. We propose to express dependencies using a combination of object number and object priority. Let's consider our example of an encoding providing both spatial enhancement and frame rate enhancement options, and suppose that the application has expressed a preference for frame rate.  We can express that policy as follow: the frames are ordered first by time and when the time is the same by resolution.  This determines the "object number" property. the frame priority will be set to 1 for the 720p 30 fps frame, 2 for the 720p 60 fps frames, and 3 for all the enhancement frames. If the application did instead express a preference for definition, object numbers will be assigned in the same way, but the priorities will be different: the frame priority will be set to 1 for the 720p 30 fps I frames and 2 for the 720p 30 fps P and B frames, 3 and 4 for the 1080p enhancements of the 60 fps frames, and 5 and 6 for the 60 fps frames and their enhancements. Object numbers and priorities will be set by the publisher of the track, and will not be modified by the relays. </ins> 4.7. Receivers SHOULD be able to as part of negotiation of a session MOQ-

Related discussion:
text: I think you're referring to the scalabilty mode known as "L2T2h": URL Of course, we could encode these dependencies as properties of the object being sent, stating for example that "object 17 can only be decoded if objects 16, 11 and 7 are available." However, this approach leads to a lot of complexity in relays. We believe that a linear approach is preferable, using attributes of objects like delivery order or priorities. [BA] Why would this lead to complexity in relays in a receiver-driven scenario? The relay would only forward the layers that the receiver asked for, and wouldn't have to store state relating to the receivers. Also, the real issue here isn't dropping layers, which can occur at any point. The reason dependencies (and more) are needed (and why the "linear approach" doesn't work) comes when it is desired to add layers. It is of course possible just to start forwarding additional layers, but that doesn't improve quality unless the forwarded frames can actually be decoded. To figure this out, whoever is in charge (receiver or relay) needs to be able to determine the "upswitch points". This also implies feedback from the receivers to the encoder (e.g. LRR). Section 4.2 We propose to express dependencies using a combination of object number and object priority. [BA] How would this permit restoration of video quality after congestion clears up? I don't think you have enough information. Section 4.3 . Relay behavior In case of congestion, the relay will use the priorities to selectively drop the "least important" objects: [BA] Dropping is easy, it's adding layers back (and decoding them) that's hard.