Code change:
MOQ protocol specifications will provide details on the supported media encapsulation(s). <ins> 4.6.1. Some video codecs have a complex structure.  Consider an application using both temporal layering and spatial layering.  It would send for example: an object representing the 30 fps frame at 720p an object representing the spatial enhancement of that frame to 1080p an object representing the 60 fps frame at 720p an object representing the spatial enhancement of that 60 fps frame to 1080p The encoding of the 30 fps frame depends on the previous 30 fps frames, but not on any 60 fps frame.  The encoding of the 60 fps depends on the previous 30 fps frames, and possibly also on the previous 60 fps frames (there are options).  The encoding of the spatial enhancement depends on the corresponding 720p frames, and also on the previous 1080p enhancements.  Add a couple of layers, and the expression of dependencies can be very complex.  The AV1 documentation for example provides schematics of a video stream with 3 frame rate options at 15, 30 and 60 fps, and two definition options, with a complex graph of dependencies.  Other video encodings have similar provisions.  They may differ in details, but there are constants: if some object is dropped, then all objects that have a dependency on it are useless. Of course, we could encode these dependencies as properties of the object being sent, stating for example that "object 17 can only be decoded if objects 16, 11 and 7 are available."  However, this approach leads to a lot of complexity in relays.  We believe that a linear approach is preferable, using attributes of objects like delivery order or priorities. 4.6.2. The conversion from dependency graph to linear ordering is not unique.  The simple graph in our example could be ordered either "frame rate first" versus "definition first".  If the application chooses frame rate first, the policy is expressed as "in case of congestion, drop the spatial enhancement objects first, and if that is not enough drop the 60 fps frames".  If the application chooses "definition first", the policy becomes "drop the 60 fps frames and their corresponding 1080p enhancement first, and if that is not enough also drop the 1080p enhancement of the 30 fps frames". More complex graphs will allow for more complex policies, maybe for example "15 fps at 720p as a minimum, but try to ensure at least 30fps, then try to ensure 1080p, and if there is bandwidth available forward 60 fps at 1080p".  Such linearization requires choices, and the choices should be made by the application, based on the user experience requirements of the application. The relays will not understand all the variation of what the media is but the applications will need a way to indicate to the relays the information they will need to correctly order which data is sent first. 4.6.3. We propose to express dependencies using a combination of object number and object priority. Let's consider our example of an encoding providing both spatial enhancement and frame rate enhancement options, and suppose that the application has expressed a preference for frame rate.  We can express that policy as follow: the frames are ordered first by time and when the time is the same by resolution.  This determines the "object number" property. the frame priority will be set to 1 for the 720p 30 fps frame, 2 for the 720p 60 fps frames, and 3 for all the enhancement frames. If the application did instead express a preference for definition, object numbers will be assigned in the same way, but the priorities will be different: the frame priority will be set to 1 for the 720p 30 fps I frames and 2 for the 720p 30 fps P and B frames, 3 and 4 for the 1080p enhancements of the 60 fps frames, and 5 and 6 for the 60 fps frames and their enhancements. Object numbers and priorities will be set by the publisher of the track, and will not be modified by the relays. </ins> 4.7. Receivers SHOULD be able to as part of negotiation of a session MOQ-

Related discussion:
text: Are there scenarios in which it would be useful to be able to ingest multiple MediaStreamTracks (either audio or video)? I ask because the RUSH draft doesn't support ingestion of multiple tracks and I've been told that this is a problem by some customers. Section 2.4 A media switch, sourcing tracks that represent a subset of tracks from across all the emitters. Such subset may represent tracks representing top 5 speakers at higher qualities and lot of other tracks for rest of the emitters at lower qualities. [BA] The use of the term "media switch" suggests to me that it is forwarding tracks, rather than sourcing (e.g. transcoding) them. Can you clarify? subscribing to subset of the tracks [BA] I think there may be an implicit assumption here, which is that the receiver only indicates the track it wishes to subscribe to, rather than indicating the quality it wishes to receive. Above setup brings in following properties on the data model for the transport protocol [BA] The paragraphs that follow appear to describe different ways that the system could be implemented, rather than "properties on the data model". Do you mean to say that the design should be able to handle any of the approaches described? Media Switches to source new tracks but retain media payload from the original emitters. This implies publishing new Track IDs sourced from the SFU, with object payload unchanged from the original emitters. [BA] Is the term "source new tracks" implied by changing the Track IDs? If the payload is retained, this seems like forwarding to me, rather than "sourcing new tracks". Media Switches to propogate subset of tracks as-is from the emitters to the subscribers. This implies Track IDs to be unchanged between the emitters and the receivers. [BA] What does "as-is" mean here? Are you referring only the lack of a change in the "Track ID"? If the switch drops some frames in order to reduce quality (and bandwidth consumption) is it still "as-is"? Subscribers to explictily request multiple appropriate qualities and dynamically move between the qualtiies during the course of the session [BA] Is it a goal to be able to support such a "receiver driven" approach, as well as the other two approaches, which are sender driven?