Draft: draft-ietf-rtcweb-jsep

Commentary: Specifically consider simulcast Remove upscaling Clean up some contradictory text 
 I was trying to understand what kind of emoji : GT < was until I realized it was a flubbed LGTM :-) 
 I am going to rework this PR to deal with what we discussed regarding upscaling today . 
 Note to self : target , with this 
 I like the new text - I 'd like to do a final read before we merge but so far looks good . 
 New rev uploaded with editorial fixes from  's review . 
 If a sender is transmitting video at 180p , 360p , and 720p , and it receives an a = imageattr that limits received resolution to 480p , what should happen ? a ) 180p and 360p are transmitted intact , 720p is not transmitted b ) 180p and 360p are transmitted intact ; 720p is scaled to 480p c ) Everything is scaled down ; transmission is 480p , 240p , 120p I would argue that a ) makes the most sense here , but this is somewhat at odds with the guidance we are giving for the non - simulcast case , which is more like b ) . 
 This might be a case where the browser needs to do b and the JS need to figure out what happened and turn off the 480 so you end up with an effective results of a. Once we move into simulcast land , the JS just need to deal with taking care of more stuff . 
 I 'm good with A. 
 To meet bandwidth limits , the encoder may dynamically apply downscaling . This may conflict with upscaling performed to meet a = imageattr requirements . This conflict needs to be resolved somehow . I suggest that we remove the language around upscaling and policy and instead state that JSEP implementations SHOULD have appropriate fallback to ensure that they can decode ' too - low ' resolutions in software . 
 Resolution on editor 's call : Just remove the ability to upscale ( WebRTC API does n't permit this anyway ) If you are in a situation where you would need to upscale to meet the imageattr reqs , just do n't send . Implementations SHOULD have software fallback if they have some sort of goofy minimum size decoder . Someone needs to email the mailing list about this change . 
 BLOCKER : Section 3.6.2 , second paragraph , second sentence says a sender MUST scale to match imageattr . The same section , paragraph seven , says it SHOULD scale to match imageattr . There needs to be only one normative statement on this topic , so as to avoid confusion among implementors . 
 Agreed . It 's a bit nuanced because the text says " MUST limit " and " SHOULD scale " , but this could be clarified .
Paragraph (version A):
In the special case of receiving a maximum resolution of [0, 0], as
described above, the sender MUST NOT transmit the track.
Paragraph (version B):
This specification prescribes more specific behavior.  When a
MediaStreamTrack, which is producing video of a certain resolution
(the "track resolution"), is attached to a RtpSender, which is
encoding the track video at the same or lower resolution(s) (the
"encoder resolutions"), and a remote description is applied that
references the sender and contains valid "a=imageattr recv"
attributes, it MUST follow the rules below to ensure the sender does
not transmit a resolution that would exceed the size criteria
specified in the attributes.  These rules MUST be followed as long as
the attributes remain present in the remote description, including
cases in which the track changes its resolution, or is replaced with
a different track.
Paragraph (version B):
Depending on how the RtpSender is configured, it may be producing a
single encoding at a certain resolution, or, if simulcast
sec.simulcast has been negotiated, multiple encodings, each at their
own specific resolution.  In addition, depending on the
configuration, each encoding may have the flexibility to reduce
resolution when needed, or may be locked to a specific output
resolution.
Paragraph (version B):
First, the most suitable "a=imageattr recv" attribute is selected.
This is performed by taking the attribute with the highest "q="
value from the set of attributes that reference the media format
that has been selected for the specified encoding.  If multiple
attributes have the same "q=" value, the one that appears first in
the m= section is used.  Note that while JSEP endpoints will
include at most one "a=imageattr recv" attribute per media format,
JSEP endpoints may receive session descriptions from non-JSEP
endpoints with m= sections that contain multiple such attributes.
Paragraph (version B):
If there is an applicable "a=imageattr recv" attribute for the
encoding, the limits from the attribute are then compared to the
encoder resolution.  Only the specific limits mentioned below are
considered; any other values, such as picture aspect ratio, MUST
be ignored.  Note that when considering a MediaStreamTrack that is
producing rotated video, the unrotated resolution MUST be used for
the checks.  This is required regardless of whether the receiver
supports performing receive-side rotation (e.g., through CVO
TS26.114), as it significantly simplifies the matching logic.
Paragraph (version B):
If the attribute includes a "sar=" (sample aspect ratio) value set
to something other than "1.0", indicating the receiver wants to
receive non-square pixels, this cannot be satisfied and the sender
MUST NOT transmit the encoding.
Paragraph (version B):
If the encoder resolution exceeds the maximum size permitted by
the attribute, and the encoder is allowed to adjust its
resolution, the encoder SHOULD apply downscaling in order to
satisfy the limits, although the downscaling MUST NOT change the
picture aspect ratio of the encoding.  For example, if the encoder
resolution is 1280x720, and the attribute specified a maximum of
640x480, the expected output resolution would be 640x360.  If
downscaling cannot be applied, the encoding MUST NOT be
transmitted, and an error SHOULD be surfaced to the application.
Paragraph (version B):
If the encoder resolution is less than the minimum size permitted
by the attribute, the encoding MUST NOT be transmitted, and an
error SHOULD be surfaced to the application; the encoder MUST NOT
apply upscaling.  JSEP implementations SHOULD avoid this situation
by allowing receipt of arbitrarily small resolutions, perhaps via
fallback to a software decoder.
Paragraph (version B):
In the special case of a maximum resolution of [0, 0], as
described above, the sender MUST NOT transmit the encoding.