Change:
…er.
lgtm
Just one observation: If a JSEP implementation offers codec A to an SFU, and the SFU responds with codecs A and B in the answer (which it's allowed to do), with the "most recent answer" wording, that means the JSEP implementation is technically allowed to offer codecs A and B in a subsequent offer. Is that the intention? I'm fine with it, I'm just making sure. This would then be the only way to seamlessly switch to a new codec. For example, after a hardware encoder becomes available.
oh, hmm, I see that is legal in 3264. That wasn't the intention, but let me think about it.
I think the case NAME points out sounds fine.
LGTM but if we end up changing this a bunch I would like to look at it again
The rationale for keeping to the negotiated set was that it becomes ambiguous how to prioritize the codecs in a reoffer. e.g. if you offer ABC and the answer is CB (while supporting DCB), if we keep to the negotiated set then clearly both sides by default should re-offer with CB. Otherwise, it's not clear where A or D should go. In this case, with an A offer and an AB answer, the callee's reoffer would have to be A, and the caller's reoffer could be either A or AB (if B somehow was now available). That is non-ambiguous, and seems fine. Generally, this is a corner case, and as such taking the simplest path seems best.
Section 5.2.2, "Subsequent Offers", says: However, consider a situation where Alice offers codecs and , and Bob answers with only codec , thereby rejecting . If Bob were to now create a subsequent offer, his remote description still contains both and , so he would technically be allowed to offer both codecs, even though was previously rejected. Is this intentional? It seems to go against the spirit of the rule. Perhaps the rule is meant to be "only include codecs present in the current local and remote descriptions". For context, see: URL
Yes, your understanding is correct. The set to be offered should only include the subset of codecs that were negotiated.
Please, let's specify the context of those already negotiated codecs. Is that per m=line? is that per "RTP session" (whatever that means nowadays)? is that per full SDP? If Alice just offers an m=audio line in the initial SDP offer and Bob answers, aren't they allowed to re-offer later with an added m=video line?
Also, are all those rules take into account multi-stream/participants when in a SFU scenario? Alice may join a SFU conference by offering VP8 and H264 for her sending m=video section, and the SFU may reply with just VP8. But later when a new non-VP8 capable peer join the conference, the SFU may want to generate a re-offer for Alice in behalf of Bob with a new m=video line (a=sendonly) having just H264 codec (since Bob). Why is this illegal?
Related issue open in Chromium project: URL
I assumed it meant per m= line, though this should definitely be made clear. If my assumption is correct, there would be no issue in your later comment about an SFU adding a new m= line.
Thanks NAME it makes sense. Hope it's clarified in the spec.
Eric and I discussed today and I think the simplest change is to reference "the codecs present in the most recent answer", since that is the set of negotiated codecs. This should also be extended to the other bullets in 5.2.2 regarding negotiated properties.
NAME NAME have you consider this scenario?: Alice may join a SFU conference by offering VP8 and H264 in her m=video (a=sendrecv), and the SFU may reply with just VP8 to ensure VP8 is used. Later, when a new non-VP8 capable peer joins the conference, the SFU may want to generate a re-offer for Alice by changing the codec to just H264. With your change this is not legal. And I don't agree because it is an artificial limitation. If we play the SDP game we should follow its rules. Does some RFC state that my suggested use case is illegal?
SFUs can do whatever they want. This spec is for JSEP implementations.
NAME please, re-read my concern. It's about a valid SDP received by WebRTC browser.
Nothing prohibits the SFU from re-offering with H.264 + VP8, and the JSEP endpoint will handle this properly when it is received (generating an appropriate answer). The text being discussed here only affects the offers that JSEP generates.
Re-openened due to discution on ietf list
I think re-offer should contain the full set of codecs or this will cause interesting problems with 3pcc or SIP interop. The order of codecs should ideally be the same as currently negotiated to avoid codec flips. Essentially, what this means all supported codecs not currently negotiated should be added to the end of the offer after the currently negotiated ones.
Note that there is a generic problem about what we mean by "codec". For example, assuming we are only allowed to re-offer previously negotiated codecs (e.g. H.264), are we still allowed to re-offer a different H.264 "codec configuration"?
Related: can an endpoint re-offer by introducing new PT values that were not present in previous O/A? URL
So one way to create a re-offer with all the new PT value that got removed in prev O/A is to create a new PC, add all the steams/tracks to it. Then call create offer to create a "fresh" offer and use that for the offer. However, might be a bit complicated with what happens with incoming media, some of it would be incoming on old PC then move to new PC.
This is even more complicated since you are not supposed to re-use the old PT in the new offer for different codecs. Since this is a completely new offer with no information about previously used PT, this restriction is hard to enforce.
Honestly, I don't think such an approach is acceptable.
Hmm - I see your point.
I don't want to renegotiate ICE+DTLS. I just want to add my audio/video on an already connected transport, and make it work :)
From , it appears the decision was to keep the spec as-is (only negotiate a subset), and possibly add an API in the future for adding codecs. Also, non-JSEP endpoints are still allowed to add codecs, which is covered by . So, I think this issue can be closed.
This will definitely break things.
I just listened to the recording, and the conclusion appeared to be "no one in the room objects to this, though some people on the mailing list might, so we'll continue discussion there". Did that discussion never happen? If so, it needs to. I'll reopen.
I do not think discussion ever happened on the list. My issue is that there is no API to generate a new offer which reuses currently negotiated payload types but lists all available codecs. Without this functionality third party call control is impossible to implement and it is impossible to simulate SIP INVITE without the body which will affect interop.
A possible addition to the API to allow reoffer codecs that were excluded previously: URL And the JSEP part: URL
Assigning Peter since his PR seems like the way forward.
I do worry that handling this in an ad hoc fashion is going to bite us, e.g. what if in the future a SFU wants a new offer with all supported RTP header extensions? I'm going to go dig up the initial issue that led to the approach that is currently in the document and see if it helps frame the situation.
TL; DR: After researching this issue, I think we should resolve it by always putting any supported-but-not-previously-negotiated codecs into the offer, but after the previously-negotiated codecs. [I will note that this is what NAME advocated for in August.] Full analysis OK, I trawled through the archives, and to the best of my understanding there are three issues: How to handle the case where EP1 (offerer) supports codecs [A, B, C] (in that preference order) and EP2 (answerer) supports [D, C, B] (in that order). See URL for more info. Interactions with the codec ordering API discussed in and related - how to prevent problems when both sides don't agree on ordering. Unnecessary SDP traffic (resending stuff that won't be used) Turning our attention to 1), we can consider left with the situation where EP1 and EP2 negotiate [B, C], via default 3264 rules, and EP2 now wants to reoffer. Under the current text, EP2 would have to reoffer [B, C], which prevents any change to the codec in use by either side, but doesn't allow for use of D. However, we definitely do not want to reoffer [D, C, B], since that will lead to [C, B] being negotiated, i.e., it will cause a codec switch. That leaves [D, B, C] or [B, C, D] as the only reasonable options to consider, i.e. "put new ones first" or "put new ones last". "New ones first" is complicated if we consider the case where EP2 suddenly supports [D, C, B, A]. Then the offer would become [D, A, B, C], and result in [A, B, C] being negotiated, which is probably not what EP2 wants. "New ones last" results in [B, C, D, A] being offered in the situation above, and [B, C, A] being negotiated. No codec switch occurs, unless explicit priority tweaking is performed. Ergo, "new ones last" seems like a reasonable policy that should address 1). 2) I think can be handled by the solution I am about to propose in . 3) I believe to be of minor importance. In the common case (e.g. when communicating between like endpoints), there will be no changes, and even in interop cases, the difference should be insignificant. Following up on my earlier comment, what about other things that might need to be renegotiated, such as a=rtcp-fb fields, and a=extmap extensions? It seems likely that if we are going to allow new codecs, we need to allow new a=rtcp-fb (which can be associated with a PT). And if we're going to do that, new a=extmap seems largely harmless. However, I think we should draw the line at any changes to TRANSPORT attributes, e.g. a=rtcp-rsize, etc. So the changes to S 5.2.2 would basically just be to incorporate the policy mentioned above into the first 3 bullets on page 45 (and any related changes needed in the subsequent-answer section). The End.
I can live with this. I would note that in bullet 3 above you could also do "B D C" but I think we can agree that that would be silly.
sounds good to me. Lots of SIP stuff does exactly that so I don't foresee any problem wiht his.
I can live with this as well. I just want to remind about two arguments that I got when I proposed this solution: a. Allocating a new codec can consume resources. For instance some video codecs have hardware resources required for their operation which need to be reserved before codec is offered. If codec is offered but never used, this can lead to waste of resources. b. In case of SIP, complete list of codecs is not produced for every offer. It is only produced for the offer which is generated in response to INVITE with no SDP body. Proposed behavior does not allow to exactly match currently implemented SIP end point behavior. As I have stated, I am fine with the proposed solution. I just want to make sure these two arguments do not cause any issues for anybody else.
NAME wdyt about the most recent comment from NAME ? As I see it we can either do what I propose here always, or only upon a createOffer flag (as NAME suggested). Always seems simpler, if we aren't worried about the issues Roman raises.
So what we sorted on editor call is we are going to make this happen by default for now. If a strong use case for having a knob to control it comes up, we can consider that. One point is you can use codec prefs to eliminate codecs so they don't show up in future offers.
NAME can you update your PR to reflect this consensus?
Hi, sorry as I couldn't track all the conclusions in this issue. However I'd like to confirm whether the current Chrome behavior is correct (according to the conclusion above) or not: Chrome is offered with VP8 PT 123. Chrome answers with VP8 PT 123 (including a sending video track) and no other video codecs. Chrome removes the video track from its PeerConnection. Chrome re-adds the same video track into its PeerConnection. Chrome calls . The new offer contains VP8 PT 123, but also VP9 PT 101, H264 etc etc etc. I didn't expect it since the initial SDP O/A did not contain them. Of course, there is a single BUNDLED transport. If this behavior correct and should Chrome fix it?
I think that behavior is correct by the new definition which is roughly reoffers will offer everything the browser can do at that point of time, but will put the stuff currently in use first.
Fine with me :)
Note that removal/readdition from a MediaStream should have no impact on offers/answers. IOW, the case you describe above is essentially: offer with all codecs, answer with just VP8, reoffer contains all codecs.
See also as a trivial followup on this issue.
Which has been merged, bringing this issue, finally, to a close.
What about if the video track I add to the local stream is not the same as the one I previously removed from it? For example, it is a video track retrieved via with different video resolution/constraints settings.
It has no impact. Only tracks given to addTrack or setTrack are used.
I did not understand you, sorry. I know that, according to the new API, changes in the local do not affect what the sends (although current Chrome does not behave like that and reacts on track addition/removal into the ). Thanks.
Just to be 200% sure, I would like to confirm that the following behaviour in Firefox is wrong. The scenario is the same as but now with Firefox: Firefox is offered with VP8 PT 123. Firefox answers with VP8 PT 123 (including a sending video track) and no other video codecs. Firefox removes the video track from its PeerConnection. Firefox adds another local video track into its PeerConnection. Firefox calls . The new offer contains VP8 PT 120 instead of 123 (wrong). Bug/issue detailed here: URL
What exact text do you believe prohibits this?
Per RFC 3264 this is allowed. What is not allowed is to use PT 123 for anything except VP8. Per :
Well, probably nothing prohibits a re-offer to include new PT values pointing to same codec+configurations as previously negotiated. Said that, a re-offer cannot offer a previously used PT for other codec+configuration. So IMHO this could be simplified by mandating reusing same PT values as before.
I'm not seeing a lot of value in requiring JSEP impls to behave this way.
Just noting that if it was a different m line, then it could use any PT it wanted for VP8. And in the above example, it is fine for it to use 120 in the 2nd offer as long as 120 had not previously been used for something other than VP8. To answer EKR question, the only thing of value here is following use case. In offer/answer 1, we have PT 120 => H264 and PT 123 => VP8. If in offer/answer 2 (on same m-lines, same mid, same blah blah blah) we swapped to PT120 => VP8, if we got a late media packet that was sent before the switch, but arrived after the switch, it might get delivered to the VP8 codec when actually it was encoded with H.264. I think that is all the is meant to stop. And I have allwasy viewed that trying to sop this is silly because what is going to happen is that late packet is going to get discarded. If sending a 264 packet to your VP8 codecs causes anything to happen other than it getting discarded, well seem like that is a problem.
The behavior mentioned by NAME does seem complicated, since the offerer needs to decode VP8 on both PT 120 and 123 at least until the O/A completes, but I agree it appears to be legal, and AFAICT it does not create any interoperability problems.
Uh no no. That's not possible. As long as such a m line belongs to the same BUNDLE group, all the m lines in that group are part of the same RTP session [1] and, hence, their PT values must be unique (in the sense that they must point to the same codec+configuration). I already asked about this exact subject several months ago. The given rationale was: "Existing RTP intermediaries [2] may rely on a single 1:1 mapping between PT and codec+configuration when carried over the same IP:port tuple, this is, the same RTP session" Those are the related threads: URL URL URL [1] Future generations may experience difficulties in understanding the concept of "RTP session". [2] Although no one has ever seen them in a WebRTC environment, they may exist.
The sad thing of all of this is that, at the end, for things to work the offerer and re-offerer must always be the same peer.
I think we definitely need to be able to have the reoffer move the media to a different IP since we do that all the time on non webrtc stuff. I might be missing what the issue is here but I want that to work.
Move the m= section into a separate transport just because it includes PT values already present in the other m= sections but pointing to different codec+configurations?
I don't see how any of this is a an issue. The rule is you can't reuse the PT. There's no rule that you can't reuse the codec with a new PT.
That's right, but it's also true that it causes errors in implementations (such as when Firefox renegotiates with Chrome by offering two different PT for Opus and Chrome assumes receipt of just the first PT). Hi, The following scenario causes setLocalDescription() to fail after renegotiation because the re-offerer (Firefox) introduces new codec PT in the same m= line): 1) Chrome calls to Firefox with an audio track. SDP offer (simplified): m=audio 58234 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 a=setup:actpass a=mid:audio a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=rtcp-fb:111 transport-cc a=fmtp:111 minptime=10;useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 a=rtpmap:13 CN/8000 a=rtpmap:126 telephone-event/8000 2) Firefox SDP answer: m=audio 65515 UDP/TLS/RTP/SAVPF 111 a=recvonly a=fmtp:111 maxplaybackrate=48000;stereo=1 a=mid:audio a=rtcp-mux a=rtpmap:111 opus/48000/2 a=setup:active Firefox has chosen PT 111 (opus). Fine. 3) Firefox re-offers with its own audio track. SDP re-offer: m=audio 65515 UDP/TLS/RTP/SAVPF 109 9 0 8 111 a=sendrecv a=fmtp:111 maxplaybackrate=48000;stereo=1 a=mid:audio a=rtcp-mux a=rtpmap:109 opus/48000/2 a=rtpmap:9 G722/8000/1 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:111 opus/48000/2 a=setup:actpass Here Firefox has introduced a new PT value: 109 (also opus). Note that PT 111 (opus) is still present. 4) Chrome gets the SDP re-offer, calls setRemoteDescription() (success) and calls createAnswer(), which produces the following SDP answer: m=audio 58234 UDP/TLS/RTP/SAVPF 109 9 0 8 a=setup:passive a=mid:audio a=sendrecv a=rtcp-mux a=rtpmap:109 opus/48000/2 a=fmtp:109 minptime=10;useinbandfec=1 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 Note that Chrome has chosen PT 109 (opus) instead of the previously used PT 111 (opus). 5) Then Chrome calls to setLocalDescription() with such an answer, which produces this error: So my main question is: can an endpoint perform a PeerConnection renegotiation and introduce new PT values within an existing m= line? AFAIR this is NOT valid. If valid, then this is a Chrome bug. If invalid, then this is a Firefox bug. Thanks a lot. Iñaki Baz Castillo

Discussion:
of the corresponding m= section in the current remote description: The m= line and corresponding "a=rtpmap" and "a=fmtp" lines MUST <del> only include codecs present in the remote description. </del> <ins> only include codecs present in the most recent answer. </ins> The RTP header extensions MUST only include those that are present <del> in the remote description. </del> <ins> in the most recent answer. </ins> The RTCP feedback extensions MUST only include those that are <del> present in the remote description. </del> <ins> present in the most recent answer. </ins> <del> The "a=rtcp-mux" line MUST only be added if present in the remote description. </del> <ins> The "a=rtcp-mux" line MUST only be added if present in the most recent answer. </ins> <del> The "a=rtcp-rsize" line MUST only be added if present in the remote description. </del> <ins> The "a=rtcp-rsize" line MUST only be added if present in the most recent answer. </ins> The "a=group:BUNDLE" attribute MUST include the mid identifiers specified in the bundle group in the most recent answer, minus any m=