Change:
Specifically consider simulcast Remove upscaling Clean up some contradictory text
I was trying to understand what kind of emoji :GT< was until I realized it was a flubbed LGTM :-)
I am going to rework this PR to deal with what we discussed regarding upscaling today.
Note to self: target , with this
I like the new text - I'd like to do a final read before we merge but so far looks good.
New rev uploaded with editorial fixes from Eric's review.
If a sender is transmitting video at 180p, 360p, and 720p, and it receives an a=imageattr that limits received resolution to 480p, what should happen? a) 180p and 360p are transmitted intact, 720p is not transmitted b) 180p and 360p are transmitted intact; 720p is scaled to 480p c) Everything is scaled down; transmission is 480p, 240p, 120p I would argue that a) makes the most sense here, but this is somewhat at odds with the guidance we are giving for the non-simulcast case, which is more like b).
This might be a case where the browser needs to do b and the JS need to figure out what happened and turn off the 480 so you end up with an effective results of a. Once we move into simulcast land, the JS just need to deal with taking care of more stuff.
I'm good with A.
To meet bandwidth limits, the encoder may dynamically apply downscaling. This may conflict with upscaling performed to meet a=imageattr requirements. This conflict needs to be resolved somehow. I suggest that we remove the language around upscaling and policy and instead state that JSEP implementations SHOULD have appropriate fallback to ensure that they can decode 'too-low' resolutions in software.
Resolution on editor's call: Just remove the ability to upscale (WebRTC API doesn't permit this anyway) If you are in a situation where you would need to upscale to meet the imageattr reqs, just don't send. Implementations SHOULD have software fallback if they have some sort of goofy minimum size decoder. Someone needs to email the mailing list about this change.
BLOCKER: Section 3.6.2, second paragraph, second sentence says a sender MUST scale to match imageattr. The same section, paragraph seven, says it SHOULD scale to match imageattr. There needs to be only one normative statement on this topic, so as to avoid confusion among implementors.
Agreed. It's a bit nuanced because the text says "MUST limit" and "SHOULD scale", but this could be clarified.

Discussion:
that it does not absolutely constrain the video formats that the sender can use, but gives an indication of the preferred values. <del> This specification prescribes more specific behavior.  When a sender of a given MediaStreamTrack, which is producing video of a certain resolution, receives an "a=imageattr recv" attribute, it MUST check to see if the original resolution meets the size criteria specified in the attribute, and adapt the resolution accordingly by scaling (if appropriate).  Note that when considering a MediaStreamTrack that is producing rotated video, the unrotated resolution MUST be used.  This is required regardless of whether the receiver supports performing receive-side rotation (e.g., through CVO TS26.114), as it significantly simplifies the matching logic. For the purposes of resolution negotiation, only size limits are considered.  Any other values, e.g. picture or sample aspect ratio, MUST be ignored. When communicating with a non-JSEP endpoint, multiple relevant "a=imageattr recv" attributes may be present in a received m= section.  If this occurs, attributes other than the one with the highest "q=" value MUST be ignored.  If multiple attributes have the same "q=" value, those that appear after the first such attribute in the m= section MUST be ignored. If an "a=imageattr recv" attribute references a different video payload type than what has been selected for sending the MediaStreamTrack, it MUST be ignored. If the original resolution matches the size limits in the attribute, the track MUST be transmitted untouched. If the original resolution exceeds the size limits in the attribute, the sender SHOULD apply downscaling to the output of the MediaStreamTrack in order to satisfy the limits.  Downscaling MUST NOT change the track aspect ratio. If the original resolution is less than the size limits in the attribute, upscaling is needed, but this may not be appropriate in all cases.  To address this concern, the application can set an upscaling policy for each sent track.  For this case, if upscaling is permitted by policy, the sender SHOULD apply upscaling in order to provide the desired resolution.  Otherwise, the sender MUST NOT apply upscaling.  The sender SHOULD NOT upscale in other cases, even if the policy permits it.  Upscaling MUST NOT change the track aspect ratio. If there is no appropriate and permitted scaling mechanism that allows the received size limits to be satisfied, the sender MUST NOT transmit the track. If the attribute includes a "sar=" (sample aspect ratio) value set to something other than "1.0", indicating the receiver wants to receive non-square pixels, this cannot be satisfied and the sender MUST NOT transmit the track. In the special case of receiving a maximum resolution of [0, 0], as described above, the sender MUST NOT transmit the track. </del> <ins> This specification prescribes more specific behavior.  When a MediaStreamTrack, which is producing video of a certain resolution (the "track resolution"), is attached to a RtpSender, which is encoding the track video at the same or lower resolution(s) (the "encoder resolutions"), and a remote description is applied that references the sender and contains valid "a=imageattr recv" attributes, it MUST follow the rules below to ensure the sender does not transmit a resolution that would exceed the size criteria specified in the attributes.  These rules MUST be followed as long as the attributes remain present in the remote description, including cases in which the track changes its resolution, or is replaced with a different track. Depending on how the RtpSender is configured, it may be producing a single encoding at a certain resolution, or, if simulcast sec.simulcast has been negotiated, multiple encodings, each at their own specific resolution.  In addition, depending on the configuration, each encoding may have the flexibility to reduce resolution when needed, or may be locked to a specific output resolution. For each encoding being produced by the RtpSender, the following rules are applied to determine what should be transmitted: First, the most suitable "a=imageattr recv" attribute is selected. This is performed by taking the attribute with the highest "q=" value from the set of attributes that reference the media format that has been selected for the specified encoding.  If multiple attributes have the same "q=" value, the one that appears first in the m= section is used.  Note that while JSEP endpoints will include at most one "a=imageattr recv" attribute per media format, JSEP endpoints may receive session descriptions from non-JSEP endpoints with m= sections that contain multiple such attributes. If there is an applicable "a=imageattr recv" attribute for the encoding, the limits from the attribute are then compared to the encoder resolution.  Only the specific limits mentioned below are considered; any other values, such as picture aspect ratio, MUST be ignored.  Note that when considering a MediaStreamTrack that is producing rotated video, the unrotated resolution MUST be used for the checks.  This is required regardless of whether the receiver supports performing receive-side rotation (e.g., through CVO TS26.114), as it significantly simplifies the matching logic. If the attribute includes a "sar=" (sample aspect ratio) value set to something other than "1.0", indicating the receiver wants to receive non-square pixels, this cannot be satisfied and the sender MUST NOT transmit the encoding. If the encoder resolution exceeds the maximum size permitted by the attribute, and the encoder is allowed to adjust its resolution, the encoder SHOULD apply downscaling in order to satisfy the limits, although the downscaling MUST NOT change the picture aspect ratio of the encoding.  For example, if the encoder resolution is 1280x720, and the attribute specified a maximum of 640x480, the expected output resolution would be 640x360.  If downscaling cannot be applied, the encoding MUST NOT be transmitted, and an error SHOULD be surfaced to the application. If the encoder resolution is less than the minimum size permitted by the attribute, the encoding MUST NOT be transmitted, and an error SHOULD be surfaced to the application; the encoder MUST NOT apply upscaling.  JSEP implementations SHOULD avoid this situation by allowing receipt of arbitrarily small resolutions, perhaps via fallback to a software decoder. In the special case of a maximum resolution of [0, 0], as described above, the sender MUST NOT transmit the encoding. </ins> 3.7.