Draft: draft-ietf-rtcweb-jsep

Commentary: … er . 
 lgtm 
 Just one observation : If a JSEP implementation offers codec A to an SFU , and the SFU responds with codecs A and B in the answer ( which it 's allowed to do ) , with the " most recent answer " wording , that means the JSEP implementation is technically allowed to offer codecs A and B in a subsequent offer . Is that the intention ? I 'm fine with it , I 'm just making sure . This would then be the only way to seamlessly switch to a new codec . For example , after a hardware encoder becomes available . 
 oh , hmm , I see that is legal in 3264 . That was n't the intention , but let me think about it . 
 I think the case points out sounds fine . 
 LGTM but if we end up changing this a bunch I would like to look at it again 
 The rationale for keeping to the negotiated set was that it becomes ambiguous how to prioritize the codecs in a reoffer . e.g. if you offer ABC and the answer is CB ( while supporting DCB ) , if we keep to the negotiated set then clearly both sides by default should re - offer with CB . Otherwise , it 's not clear where A or D should go . In this case , with an A offer and an AB answer , the callee 's reoffer would have to be A , and the caller 's reoffer could be either A or AB ( if B somehow was now available ) . That is non - ambiguous , and seems fine . Generally , this is a corner case , and as such taking the simplest path seems best . 
 Section 5.2.2 , " Subsequent Offers " , says : However , consider a situation where  offers codecs and , and  answers with only codec , thereby rejecting . If  were to now create a subsequent offer , his remote description still contains both and , so he would technically be allowed to offer both codecs , even though was previously rejected . Is this intentional ? It seems to go against the spirit of the rule . Perhaps the rule is meant to be " only include codecs present in the current local and remote descriptions " . For context , see : 
 Yes , your understanding is correct . The set to be offered should only include the subset of codecs that were negotiated . 
 Please , let 's specify the context of those already negotiated codecs . Is that per m = line ? is that per " RTP session " ( whatever that means nowadays ) ? is that per full SDP ? If  just offers an m = audio line in the initial SDP offer and  answers , are n't they allowed to re - offer later with an m = video line ? 
 Also , are all those rules take into account multi - stream / participants when in a SFU scenario ?  may join a SFU conference by offering VP8 and H264 for her sending m = video section , and the SFU may reply with just VP8 . But later when a new non - VP8 capable peer join the conference , the SFU may want to generate a re - offer for  in behalf of  with a new m = video line ( a = sendonly ) having just H264 codec ( since  ) . Why is this illegal ? 
 Related issue open in Chromium project : 
 I assumed it meant per m= line , though this should definitely be made clear . If my assumption is correct , there would be no issue in your later comment about an SFU adding a new m= line . 
 Thanks it makes sense . Hope it 's clarified in the spec . 
  and I discussed today and I think the simplest change is to reference " the codecs present in the most recent answer " , since that is the set of negotiated codecs . This should also be extended to the other bullets in 5.2.2 regarding negotiated properties . 
 have you consider this scenario ? :  may join a SFU conference by offering VP8 and H264 in her m = video ( a = sendrecv ) , and the SFU may reply with just VP8 to ensure VP8 is used . Later , when a new non - VP8 capable peer joins the conference , the SFU may want to generate a re - offer for  by changing the codec to just H264 . With your change this is not legal . And I do n't agree because it is an artificial limitation . If we play the SDP game we should follow its rules . Does some RFC state that my suggested use case is illegal ? 
 SFUs can do whatever they want . This spec is for JSEP implementations . 
 please , re - read my concern . It 's about a valid SDP received by WebRTC browser . 
 Nothing prohibits the SFU from re - offering with    , and the JSEP endpoint will handle this properly when it is received ( generating an appropriate answer ) . The text being discussed here only affects the offers that JSEP generates . 
 Re - openened due to discution on ietf list 
 I think re - offer should contain the full set of codecs or this will cause interesting problems with 3pcc or SIP interop . The order of codecs should ideally be the same as currently negotiated to avoid codec flips . Essentially , what this means all supported codecs not currently negotiated should be to the end of the offer after the currently negotiated ones . 
 Note that there is a generic problem about what we mean by " codec " . For example , assuming we are only allowed to re - offer previously negotiated codecs ( e.g. H.264 ) , are we still allowed to re - offer a different H.264 " codec configuration " ? 
 Related : can an endpoint re - offer by introducing new PT values that were not present in previous O / A ? 
 So one way to create a re - offer with all the new PT value that got removed in prev O / A is to create a new PC , add all the steams / tracks to it . Then call create offer to create a " fresh " offer and use that for the offer . However , might be a bit complicated with what happens with incoming media , some of it would be incoming on old PC then move to new PC . 
 This is even more complicated since you are not supposed to re - use the old PT in the new offer for different codecs . Since this is a completely new offer with no information about previously used PT , this restriction is hard to enforce . 
 Honestly , I do n't think such an approach is acceptable . 
 Hmm - I see your point . 
 I do n't want to renegotiate ICE+DTLS . I just want to add my audio / video on an already connected transport , and make it work :) 
 From , it appears the decision was to keep the spec as - is ( only negotiate a subset ) , and possibly add an API in the future for adding codecs . Also , non - JSEP endpoints are still allowed to add codecs , which is covered by . So , I think this issue can be closed . 
 This will definitely break things . 
 I just listened to the recording , and the conclusion appeared to be " no one in the room objects to this , though some people on the mailing list might , so we 'll continue discussion there " . Did that discussion never happen ? If so , it needs to . I 'll reopen . 
 I do not think discussion ever happened on the list . My issue is that there is no API to generate a new offer which reuses currently negotiated payload types but lists all available codecs . Without this functionality third party call control is impossible to implement and it is impossible to simulate SIP INVITE without the body which will affect interop . 
 A possible addition to the API to allow reoffer codecs that were excluded previously : And the JSEP part : 
   since his PR seems like the way forward . 
 I do worry that handling this in an ad hoc fashion is going to bite us , e.g. what if in the future a SFU wants a new offer with all supported RTP header extensions ? I 'm going to go dig up the initial issue that led to the approach that is currently in the document and see if it helps frame the situation . 
 TL ; DR : After researching this issue , I think we should resolve it by always putting any supported - but - not - previously - negotiated codecs into the offer , but after the previously - negotiated codecs . [ I will note that this is what advocated for in August . ] Full analysis OK , I trawled through the archives , and to the best of my understanding there are three issues : to handle the case where EP1 ( offerer ) supports codecs [ A , B , C ] ( in that preference order ) and EP2 ( answerer ) supports [ D , C , B ] ( in that order ) . See for more info . with the codec ordering API discussed in and related - how to prevent problems when both sides do n't agree on ordering . SDP traffic ( resending stuff that wo n't be used ) Turning our attention to 1 ) , we can consider left with the situation where EP1 and EP2 negotiate [ B , C ] , via default 3264 rules , and EP2 now wants to reoffer . Under the current text , EP2 would have to reoffer [ B , C ] , which prevents any change to the codec in use by either side , but does n't allow for use of D. However , we definitely do not want to reoffer [ D , C , B ] , since that will lead to [ C , B ] being negotiated , i.e. , it will cause a codec switch . That leaves [ D , B , C ] or [ B , C , D ] as the only reasonable options to consider , i.e. " put new ones first " or " put new ones last " . " New ones first " is complicated if we consider the case where EP2 suddenly supports [ D , C , B , A ] . Then the offer would become [ D , A , B , C ] , and result in [ A , B , C ] being negotiated , which is probably not what EP2 wants . " New ones last " results in [ B , C , D , A ] being offered in the situation above , and [ B , C , A ] being negotiated . No codec switch occurs , unless explicit priority tweaking is performed .  , " new ones last " seems like a reasonable policy that should address 1 ) . 2 ) I think can be handled by the solution I am about to propose in . 3 ) I believe to be of minor importance . In the common case ( e.g. when communicating between like endpoints ) , there will be no changes , and even in interop cases , the difference should be insignificant . Following up on my earlier comment , what about other things that might need to be renegotiated , such as a = rtcp - fb fields , and a = extmap extensions ? It seems likely that if we are going to allow new codecs , we need to allow new a = rtcp - fb ( which can be associated with a PT ) . And if we 're going to do that , new a = extmap seems largely harmless . However , I think we should draw the line at any changes to TRANSPORT attributes , e.g. a = rtcp - rsize , etc . So the changes to S 5.2.2 would basically just be to incorporate the policy mentioned above into the first 3 bullets on page 45 ( and any related changes needed in the subsequent - answer section ) . The End . 
 I can live with this . I would note that in bullet 3 above you could also do " B D C " but I think we can agree that that would be silly . 
 sounds good to me . Lots of SIP stuff does exactly that so I do n't foresee any problem wiht his . 
 I can live with this as well . I just want to remind about two arguments that I got when I proposed this solution : a. Allocating a new codec can consume resources . For instance some video codecs have hardware resources required for their operation which need to be reserved before codec is offered . If codec is offered but never used , this can lead to waste of resources . b. In case of SIP , complete list of codecs is not produced for every offer . It is only produced for the offer which is generated in response to INVITE with no SDP body . Proposed behavior does not allow to exactly match currently implemented SIP end point behavior . As I have stated , I am fine with the proposed solution . I just want to make sure these two arguments do not cause any issues for anybody else . 
 wdyt about the most recent comment from ? As I see it we can either do what I propose here always , or only upon a createOffer flag ( as suggested ) . Always seems simpler , if we are n't worried about the issues Roman raises . 
 So what we sorted on editor call is we are going to make this happen by default for now . If a strong use case for having a knob to control it comes up , we can consider that . One point is you can use codec prefs to eliminate codecs so they do n't show up in future offers . 
 can you update your PR to reflect this consensus ? 
 Hi , sorry as I could n't track all the conclusions in this issue . However I 'd like to confirm whether the current  behavior is correct ( according to the conclusion above ) or not : Chrome is offered with VP8 PT 123 . Chrome answers with VP8 PT 123 ( including a sending video track ) and no other video codecs . Chrome removes the video track from its PeerConnection . Chrome re - adds the same video track into its PeerConnection . Chrome calls . The new offer contains VP8 PT 123 , but also VP9 PT 101 , H264 etc etc etc . I did n't expect it since the initial SDP O / A did not contain them . Of course , there is a single BUNDLED transport . If this behavior correct and should  fix it ? 
 I think that behavior is correct by the new definition which is roughly reoffers will offer everything the browser can do at that point of time , but will put the stuff currently in use first . 
 Fine with me :) 
 Note that removal / readdition from a MediaStream should have no impact on offers / answers . IOW , the case you describe above is essentially : offer with all codecs , answer with just VP8 , reoffer contains all codecs . 
 See also as a trivial followup on this issue . 
 Which has been merged , bringing this issue , finally , to a close . 
 What about if the video track I add to the local stream is not the same as the one I previously removed from it ? For example , it is a video track retrieved via with different video resolution / constraints settings . 
 It has no impact . Only tracks given to addTrack or setTrack are used . 
 I did not understand you , sorry . I know that , according to the new API , changes in the local do not affect what the sends ( although current  does not behave like that and reacts on track addition / removal into the ) . Thanks . 
 Just to be 200 % sure , I would like to confirm that the following behaviour in Firefox is wrong . The scenario is the same as but now with Firefox : Firefox is offered with VP8 PT 123 . Firefox answers with VP8 PT 123 ( including a sending video track ) and no other video codecs . Firefox removes the video track from its PeerConnection . Firefox adds another local video track into its PeerConnection . Firefox calls . The new offer contains VP8 PT 120 instead of 123 ( wrong ) . Bug / issue detailed here : 
 What exact text do you believe prohibits this ? 
 Per RFC 3264 this is allowed . What is not allowed is to use PT 123 for anything except VP8 . Per : 
 Well , probably nothing prohibits a re - offer to include new PT values pointing to same codec+configurations as previously negotiated . Said that , a re - offer can not offer a previously used PT for other codec+configuration . So IMHO this could be simplified by mandating reusing same PT values as before . 
 I 'm not seeing a lot of value in requiring  impls to behave this way . 
 Just noting that if it was a different m line , then it could use any PT it wanted for VP8 . And in the above example , it is fine for it to use 120 in the 2nd offer as long as 120 had not previously been used for something other than VP8 . To answer EKR question , the only thing of value here is following use case . In offer / answer 1 , we have PT 120 = H264 and PT 123 = VP8 . If in offer / answer 2 ( on same m - lines , same mid , same blah blah blah ) we swapped to PT120 = VP8 , if we got a late media packet that was sent before the switch , but arrived after the switch , it might get delivered to the VP8 codec when actually it was encoded with H.264 . I think that is all the is meant to stop . And I have allwasy viewed that trying to sop this is silly because what is going to happen is that late packet is going to get discarded . If sending a 264 packet to your VP8 codecs causes anything to happen other than it getting discarded , well seem like that is a problem . 
 The behavior mentioned by does seem complicated , since the offerer needs to decode VP8 on both PT 120 and 123 at least until the O / A completes , but I agree it appears to be legal , and AFAICT it does not create any interoperability problems . 
 Uh no no . That 's not possible . As long as such a m line belongs to the same BUNDLE group , all the m lines in that group are part of the same RTP session [ 1 ] and , hence , their PT values must be unique ( in the sense that they must point to the same codec+configuration ) . I already asked about this exact subject several months ago . The given rationale was : " Existing RTP intermediaries [ 2 ] may rely on a single 1:1 mapping between PT and codec+configuration when carried over the same IP : port tuple , this is , the same RTP session " Those are the related threads : [ 1 ] Future generations may experience difficulties in understanding the concept of " RTP session " . [ 2 ] Although no one has ever seen them in a WebRTC environment , they may exist . 
 The sad thing of all of this is that , at the end , for things to work the offerer and re - offerer must always be the same peer . 
 I think we definitely need to be able to have the reoffer move the media to a different IP since we do that all the time on non webrtc stuff . I might be missing what the issue is here but I want that to work . 
 Move the m= section into a separate transport just because it includes PT values already present in the other m= sections but pointing to different  ? 
 I do n't see how any of this is a an issue . The rule is you ca n't reuse the PT . There 's no rule that you ca n't reuse the codec with a new PT . 
 That 's right , but it 's also true that it causes errors in implementations ( such as when Firefox renegotiates with  by offering two different PT for  and  assumes receipt of just the first PT ) .
Mailing list commentary: Hi , The following scenario causes setLocalDescription ( ) to fail after renegotiation because the re - offerer ( Firefox ) introduces new codec PT in the same m= line ): 1 ) Chrome calls to Firefox with an audio track . SDP offer ( simplified ): m = audio UDP / TLS / RTP / SAVPF 111 103 104 9 0 8 106 105 13 126 a = setup : actpass a = mid : audio a = sendrecv a = rtcp - mux a = rtpmap:111 opus//2 a = rtcp - fb:111 transport - cc a = fmtp:111 minptime=10;useinbandfec=1 a = rtpmap:103 ISAC/ a = rtpmap:104 ISAC/ a = rtpmap:9 G722/8000 a = rtpmap:0 PCMU/8000 a = rtpmap:8 PCMA/8000 a = rtpmap:106 CN/ a = rtpmap:105 CN/ a = rtpmap:13 CN/8000 a = rtpmap:126 telephone - event/8000 2 ) Firefox SDP answer : m = audio UDP / TLS / RTP / SAVPF 111 a = recvonly a = fmtp:111 maxplaybackrate=;stereo=1 a = mid : audio a = rtcp - mux a = rtpmap:111 opus//2 a = setup : active  has chosen PT 111 ( opus ) . Fine . 3 ) Firefox re - offers with its own audio track . SDP re - offer : m = audio UDP / TLS / RTP / SAVPF 109 9 0 8 111 a = sendrecv a = fmtp:111 maxplaybackrate=;stereo=1 a = mid : audio a = rtcp - mux a = rtpmap:109 opus//2 a = rtpmap:9 G722/8000/1 a = rtpmap:0 PCMU/8000 a = rtpmap:8 PCMA/8000 a = rtpmap:111 opus//2 a = setup : actpass Here Firefox has introduced a new PT value : 109 ( also opus ) . Note that PT 111 ( opus ) is still present . 4 ) Chrome gets the SDP re - offer , calls setRemoteDescription ( ) ( success ) and calls createAnswer ( ) , which produces the following SDP answer : m = audio UDP / TLS / RTP / SAVPF 109 9 0 8 a = setup : passive a = mid : audio a = sendrecv a = rtcp - mux a = rtpmap:109 opus//2 a = fmtp:109 minptime=10;useinbandfec=1 a = rtpmap:9 G722/8000 a = rtpmap:0 PCMU/8000 a = rtpmap:8 PCMA/8000 Note that has chosen PT 109 ( opus ) instead of the previously used PT 111 ( opus ) . 5 ) Then calls to setLocalDescription ( ) with such an answer , which produces this error : So my main question is : can an endpoint perform a PeerConnection renegotiation and introduce new PT values within an existing m= line ? AFAIR this is NOT valid . If valid , then this is a bug . If invalid , then this is a Firefox bug . Thanks a lot .
Paragraph (version A):
The m= line and corresponding "a=rtpmap" and "a=fmtp" lines MUST
only include codecs present in the remote description.
Paragraph (version A):
The RTP header extensions MUST only include those that are present
in the remote description.
Paragraph (version A):
The RTCP feedback extensions MUST only include those that are
present in the remote description.
Paragraph (version A):
The "a=rtcp-mux" line MUST only be added if present in the remote
description.
Paragraph (version A):
The "a=rtcp-rsize" line MUST only be added if present in the
remote description.
Paragraph (version B):
The m= line and corresponding "a=rtpmap" and "a=fmtp" lines MUST
only include codecs present in the most recent answer.
Paragraph (version B):
The RTP header extensions MUST only include those that are present
in the most recent answer.
Paragraph (version B):
The RTCP feedback extensions MUST only include those that are
present in the most recent answer.
Paragraph (version B):
The "a=rtcp-mux" line MUST only be added if present in the most
recent answer.
Paragraph (version B):
The "a=rtcp-rsize" line MUST only be added if present in the most
recent answer.