Repository: pion/webrtc
Pull request #2130 (2022-02-22 09:52:39)

Support for send-side simulcast
@Sean-Der @davidzhao Not a real PR, but setting up one to get feedback on the direction. There are more bits to do here to make this simulcast capable, but wanted to make sure that the general approach is okay before I go further.

This makes sure that the API is not affected.

I was then planning to take encodings supplied in `RTPTransceiverInit` and apply it on the sender. Something like `setParamaters` on `RTPSender`. `setParameters` is much more strict. For v1 of this, I was thinking of not being that strict and apply the encodings. Then use that in `sdp.go` to generate the `simulcast:send` SDP. That will have both SSRC and RID, but I think that is fine.

And then I will have to figure out the `Read` and friends for the simulcast aware sender.

codecov[bot]: # [Codecov](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) Report
> Merging [#2130](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) (b3e1164) into [master](https://codecov.io/gh/pion/webrtc/commit/b45852835031afc94b53da157431bbf64465f11d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) (b458528) will **decrease** coverage by `7.11%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pion/webrtc/pull/2130/graphs/tree.svg?width=650&height=150&src=pr&token=XOOuQo07I2&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion)](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion)

```diff
@@            Coverage Diff             @@
##           master    #2130      +/-   ##
==========================================
- Coverage   77.14%   70.02%   -7.12%     
==========================================
  Files          87       64      -23     
  Lines        8820     3256    -5564     
==========================================
- Hits         6804     2280    -4524     
+ Misses       1605      858     -747     
+ Partials      411      118     -293     
```

| Flag | Coverage Δ | |
|---|---|---|
| go | `?` | |
| wasm | `70.02% <ø> (ø)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) | Coverage Δ | |
|---|---|---|
| [constants.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-Y29uc3RhbnRzLmdv) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [icecandidatepair.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-aWNlY2FuZGlkYXRlcGFpci5nbw==) | `0.00% <0.00%> (-81.82%)` | :arrow_down: |
| [rtpcodec.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-cnRwY29kZWMuZ28=) | `13.33% <0.00%> (-80.00%)` | :arrow_down: |
| [track\_local.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-dHJhY2tfbG9jYWwuZ28=) | `0.00% <0.00%> (-80.00%)` | :arrow_down: |
| [stats.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-c3RhdHMuZ28=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [internal/mux/muxfunc.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-aW50ZXJuYWwvbXV4L211eGZ1bmMuZ28=) | `9.09% <0.00%> (-63.64%)` | :arrow_down: |
| [icemux.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-aWNlbXV4Lmdv) | `0.00% <0.00%> (-54.55%)` | :arrow_down: |
| [atomicbool.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-YXRvbWljYm9vbC5nbw==) | `57.14% <0.00%> (-42.86%)` | :arrow_down: |
| [sdpsemantics.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-c2Rwc2VtYW50aWNzLmdv) | `0.00% <0.00%> (-34.49%)` | :arrow_down: |
| [internal/mux/endpoint.go](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion#diff-aW50ZXJuYWwvbXV4L2VuZHBvaW50Lmdv) | `24.13% <0.00%> (-34.49%)` | :arrow_down: |
| ... and [37 more](https://codecov.io/gh/pion/webrtc/pull/2130/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion). Last update [b458528...b3e1164](https://codecov.io/gh/pion/webrtc/pull/2130?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pion).

boks1971: Thank you @davidzhao . 

There will be multiple `TrackLocal`s inside `RTPSender` (something similar to multiple `TrackRemote`s in `RTPReceiver`), one per encoding with its own RID/SSRC etc. I did change the code in `RTPSender::Send()` (sorry about the C++ style of showing a struct method there :-) ) to call `Bind` on all the encodings. Yeah, still need to figure out the Read per TrackLocal and those bits.

Also, `ReplaceTrack` will not be supported. Currently, it passes in a `TrackLocal`. There is no concept of encodings there. So, I have added a check if the existing track has multiple encodings, just fail `ReplaceTrack` with a new error `ErrRTPSenderNewTrackHasIncorrectEnvelope`.

There are other bits like `srtp_writer_future` which is provided the `RTPSender` itself and it reaches into `RTPSender` structure to grab fields. I am not sure how that should all work in the context on multiple encodings. Need to figure out.
boks1971: One `TrackLocal` per encoding is not according to spec, but I am thinking keeping one `TrackLocal` with multiple encodings is more disruptive.
davidzhao: > One `TrackLocal` per encoding is not according to spec, but I am thinking keeping one `TrackLocal` with multiple encodings is more disruptive.

got it! if it's one TrackLocal per layer, then writing to it should be the same.

would you add multiple simulcast tracks by calling `PeerConnection.AddTrack` for each simulcast layer? then it would append to the list of encodings in RTPSender?
boks1971: @davidzhao I went with adding a method `AddEncoding` on `RTPSender` to minimise blast radius on existing stuff. Please look at this commit - https://github.com/pion/webrtc/pull/2130/commits/aaa528da913d4863f20861048b9834f445bd238e.

Signalling works now. Need to have code to send in proper data to test out further.
Sean-Der: @kevmo314 Does this fit with how you are using Simulcast/RID today?

@boks1971 If you can replace my hacky Simulcast sender in https://github.com/pion/webrtc/blob/master/peerconnection_media_test.go#L1213 that would be amazing! Would be the perfect way to show the API in action :) 
boks1971: > @boks1971 If you can replace my hacky Simulcast sender in https://github.com/pion/webrtc/blob/master/peerconnection_media_test.go#L1213 that would be amazing! Would be the perfect way to show the API in action :)

Will take a look at it @Sean-Der .
boks1971: @Sean-Der Can you please review this commit when you get a chance and let me know if it looks good to test out the new API? - https://github.com/pion/webrtc/pull/2130/commits/9c9518c12ab974479fdbb0c2110ae596840f27e5
kevmo314: > @kevmo314 Does this fit with how you are using Simulcast/RID today?
> 
> @boks1971 If you can replace my hacky Simulcast sender in [https://github.com/pion/webrtc/blob/master/peerconnection_media_test.go#L1213](https://github.com/pion/webrtc/blob/master/peerconnection_media_test.go?rgh-link-date=2022-02-23T16%3A50%3A18Z#L1213) that would be amazing! Would be the perfect way to show the API in action :)

Yeah this looks good to me, no concerns here 👍 
davidzhao: One request, is it possible to surface the `interceptor.RTCPReader` in `TrackLocalContext` ? this will make it easier for the TrackLocal to grab its own RTCPReader during Bind
boks1971: It is simple enough to do @davidzhao . Did it in this commit - https://github.com/pion/webrtc/pull/2130/commits/34e4bec0ffaab0b46b15301a9404bb862288f423. Please give it a try when you get a chance. Not sure if there are potentially bad things that can happen like interceptor leaks and such.
davidzhao: > It is simple enough to do @davidzhao . Did it in this commit - [34e4bec](https://github.com/pion/webrtc/commit/34e4bec0ffaab0b46b15301a9404bb862288f423). Please give it a try when you get a chance. Not sure if there are potentially bad things that can happen like interceptor leaks and such.

looks good! would need a getter too in order to export it. maybe `RTCPReader` ?
boks1971: @davidzhao @Sean-Der I have squashed the commits and this is ready to go. David tried an impl in LiveKit server SDK with this and it is working. I may be offline for the next three days. I can merge when I get back or please feel free to merge if it looks good.

Patch:
diff --git a/errors.go b/errors.go
index a28389f5..e152b27c 100644
--- a/errors.go
+++ b/errors.go
@@ -142,6 +142,9 @@ var (
 	// ErrRTPSenderNewTrackHasIncorrectKind indicates that the new track is of a different kind than the previous/original
 	ErrRTPSenderNewTrackHasIncorrectKind = errors.New("new track must be of the same kind as previous")
 
+	// ErrRTPSenderNewTrackHasIncorrectEnvelope indicates that the new track has a different envelope than the previous/original
+	ErrRTPSenderNewTrackHasIncorrectEnvelope = errors.New("new track must have the same envelope as previous")
+
 	// ErrUnbindFailed indicates that a TrackLocal was not able to be unbind
 	ErrUnbindFailed = errors.New("failed to unbind TrackLocal from PeerConnection")
 
@@ -202,10 +205,16 @@ var (
 	errRTPReceiverWithSSRCTrackStreamNotFound = errors.New("unable to find stream for Track with SSRC")
 	errRTPReceiverForRIDTrackStreamNotFound   = errors.New("no trackStreams found for RID")
 
-	errRTPSenderTrackNil          = errors.New("Track must not be nil")
-	errRTPSenderDTLSTransportNil  = errors.New("DTLSTransport must not be nil")
-	errRTPSenderSendAlreadyCalled = errors.New("Send has already been called")
-	errRTPSenderTrackRemoved      = errors.New("Sender Track has been removed or replaced to nil")
+	errRTPSenderTrackNil             = errors.New("Track must not be nil")
+	errRTPSenderDTLSTransportNil     = errors.New("DTLSTransport must not be nil")
+	errRTPSenderSendAlreadyCalled    = errors.New("Send has already been called")
+	errRTPSenderStopped              = errors.New("Sender has already been stopped")
+	errRTPSenderTrackRemoved         = errors.New("Sender Track has been removed or replaced to nil")
+	errRTPSenderRidNil               = errors.New("Sender cannot add encoding as rid is empty")
+	errRTPSenderNoBaseEncoding       = errors.New("Sender cannot add encoding as there is no base track")
+	errRTPSenderBaseEncodingMismatch = errors.New("Sender cannot add encoding as provided track does not match base track")
+	errRTPSenderRIDCollision         = errors.New("Sender cannot encoding due to RID collision")
+	errRTPSenderNoTrackForRID        = errors.New("Sender does not have track for RID")
 
 	errRTPTransceiverCannotChangeMid        = errors.New("errRTPSenderTrackNil")
 	errRTPTransceiverSetSendingInvalidState = errors.New("invalid state change in RTPTransceiver.setSending")
diff --git a/rtpsender.go b/rtpsender.go
index 7319cf84..0de09625 100644
--- a/rtpsender.go
+++ b/rtpsender.go
@@ -4,6 +4,7 @@
 package webrtc
 
 import (
+	"fmt"
 	"io"
 	"sync"
 	"time"
@@ -12,23 +13,30 @@ import (
 	"github.com/pion/randutil"
 	"github.com/pion/rtcp"
 	"github.com/pion/rtp"
+	"github.com/pion/webrtc/v3/internal/util"
 )
 
-// RTPSender allows an application to control how a given Track is encoded and transmitted to a remote peer
-type RTPSender struct {
+type trackEncoding struct {
 	track TrackLocal
 
-	srtpStream      *srtpWriterFuture
+	srtpStream *srtpWriterFuture
+
 	rtcpInterceptor interceptor.RTCPReader
 	streamInfo      interceptor.StreamInfo
 
 	context TrackLocalContext
 
+	ssrc SSRC
+}
+
+// RTPSender allows an application to control how a given Track is encoded and transmitted to a remote peer
+type RTPSender struct {
+	trackEncodings []*trackEncoding
+
 	transport *DTLSTransport
 
 	payloadType PayloadType
 	kind        RTPCodecType
-	ssrc        SSRC
 
 	// nolint:godox
 	// TODO(sgotti) remove this when in future we'll avoid replacing
@@ -60,23 +68,15 @@ func (api *API) NewRTPSender(track TrackLocal, transport *DTLSTransport) (*RTPSe
 	}
 
 	r := &RTPSender{
-		track:      track,
 		transport:  transport,
 		api:        api,
 		sendCalled: make(chan struct{}),
 		stopCalled: make(chan struct{}),
-		ssrc:       SSRC(randutil.NewMathRandomGenerator().Uint32()),
 		id:         id,
-		srtpStream: &srtpWriterFuture{},
 		kind:       track.Kind(),
 	}
 
-	r.srtpStream.rtpSender = r
-
-	r.rtcpInterceptor = r.api.interceptor.BindRTCPReader(interceptor.RTPReaderFunc(func(in []byte, a interceptor.Attributes) (n int, attributes interceptor.Attributes, err error) {
-		n, err = r.srtpStream.Read(in)
-		return n, a, err
-	}))
+	r.addEncoding(track)
 
 	return r, nil
 }
@@ -108,24 +108,26 @@ func (r *RTPSender) Transport() *DTLSTransport {
 }
 
 func (r *RTPSender) getParameters() RTPSendParameters {
-	var rid string
-	if r.track != nil {
-		rid = r.track.RID()
+	var encodings []RTPEncodingParameters
+	for _, trackEncoding := range r.trackEncodings {
+		var rid string
+		if trackEncoding.track != nil {
+			rid = trackEncoding.track.RID()
+		}
+		encodings = append(encodings, RTPEncodingParameters{
+			RTPCodingParameters: RTPCodingParameters{
+				RID:         rid,
+				SSRC:        trackEncoding.ssrc,
+				PayloadType: r.payloadType,
+			},
+		})
 	}
 	sendParameters := RTPSendParameters{
 		RTPParameters: r.api.mediaEngine.getRTPParametersByKind(
 			r.kind,
 			[]RTPTransceiverDirection{RTPTransceiverDirectionSendonly},
 		),
-		Encodings: []RTPEncodingParameters{
-			{
-				RTPCodingParameters: RTPCodingParameters{
-					RID:         rid,
-					SSRC:        r.ssrc,
-					PayloadType: r.payloadType,
-				},
-			},
-		},
+		Encodings: encodings,
 	}
 	if r.rtpTransceiver != nil {
 		sendParameters.Codecs = r.rtpTransceiver.getCodecs()
@@ -143,11 +145,81 @@ func (r *RTPSender) GetParameters() RTPSendParameters {
 	return r.getParameters()
 }
 
+// AddEncoding adds an encoding to RTPSender. Used by simulcast senders.
+func (r *RTPSender) AddEncoding(track TrackLocal) error {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+
+	if track == nil {
+		return errRTPSenderTrackNil
+	}
+
+	if track.RID() == "" {
+		return errRTPSenderRidNil
+	}
+
+	if r.hasStopped() {
+		return errRTPSenderStopped
+	}
+
+	if r.hasSent() {
+		return errRTPSenderSendAlreadyCalled
+	}
+
+	var refTrack TrackLocal
+	if len(r.trackEncodings) != 0 {
+		refTrack = r.trackEncodings[0].track
+	}
+	if refTrack == nil || refTrack.RID() == "" {
+		return errRTPSenderNoBaseEncoding
+	}
+
+	if refTrack.ID() != track.ID() || refTrack.StreamID() != track.StreamID() || refTrack.Kind() != track.Kind() {
+		return errRTPSenderBaseEncodingMismatch
+	}
+
+	for _, encoding := range r.trackEncodings {
+		if encoding.track == nil {
+			continue
+		}
+
+		if encoding.track.RID() == track.RID() {
+			return errRTPSenderRIDCollision
+		}
+	}
+
+	r.addEncoding(track)
+	return nil
+}
+
+func (r *RTPSender) addEncoding(track TrackLocal) {
+	ssrc := SSRC(randutil.NewMathRandomGenerator().Uint32())
+	trackEncoding := &trackEncoding{
+		track:      track,
+		srtpStream: &srtpWriterFuture{ssrc: ssrc},
+		ssrc:       ssrc,
+	}
+	trackEncoding.srtpStream.rtpSender = r
+	trackEncoding.rtcpInterceptor = r.api.interceptor.BindRTCPReader(
+		interceptor.RTPReaderFunc(func(in []byte, a interceptor.Attributes) (n int, attributes interceptor.Attributes, err error) {
+			n, err = trackEncoding.srtpStream.Read(in)
+			return n, a, err
+		}),
+	)
+
+	r.trackEncodings = append(r.trackEncodings, trackEncoding)
+}
+
 // Track returns the RTCRtpTransceiver track, or nil
 func (r *RTPSender) Track() TrackLocal {
 	r.mu.RLock()
 	defer r.mu.RUnlock()
-	return r.track
+
+	if len(r.trackEncodings) == 0 {
+		return nil
+	}
+
+	return r.trackEncodings[0].track
 }
 
 // ReplaceTrack replaces the track currently being used as the sender's source with a new TrackLocal.
@@ -161,26 +233,38 @@ func (r *RTPSender) ReplaceTrack(track TrackLocal) error {
 		return ErrRTPSenderNewTrackHasIncorrectKind
 	}
 
-	if r.hasSent() && r.track != nil {
-		if err := r.track.Unbind(r.context); err != nil {
+	// cannot replace simulcast envelope
+	if track != nil && len(r.trackEncodings) > 1 {
+		return ErrRTPSenderNewTrackHasIncorrectEnvelope
+	}
+
+	var replacedTrack TrackLocal
+	var context *TrackLocalContext
+	if len(r.trackEncodings) != 0 {
+		replacedTrack = r.trackEncodings[0].track
+		context = &r.trackEncodings[0].context
+	}
+	if r.hasSent() && replacedTrack != nil {
+		if err := replacedTrack.Unbind(*context); err != nil {
 			return err
 		}
 	}
 
 	if !r.hasSent() || track == nil {
-		r.track = track
+		r.trackEncodings[0].track = track
 		return nil
 	}
 
 	codec, err := track.Bind(TrackLocalContext{
-		id:          r.context.id,
-		params:      r.api.mediaEngine.getRTPParametersByKind(track.Kind(), []RTPTransceiverDirection{RTPTransceiverDirectionSendonly}),
-		ssrc:        r.context.ssrc,
-		writeStream: r.context.writeStream,
+		id:              context.id,
+		params:          r.api.mediaEngine.getRTPParametersByKind(track.Kind(), []RTPTransceiverDirection{RTPTransceiverDirectionSendonly}),
+		ssrc:            context.ssrc,
+		writeStream:     context.writeStream,
+		rtcpInterceptor: context.rtcpInterceptor,
 	})
 	if err != nil {
 		// Re-bind the original track
-		if _, reBindErr := r.track.Bind(r.context); reBindErr != nil {
+		if _, reBindErr := replacedTrack.Bind(*context); reBindErr != nil {
 			return reBindErr
 		}
 
@@ -189,10 +273,10 @@ func (r *RTPSender) ReplaceTrack(track TrackLocal) error {
 
 	// Codec has changed
 	if r.payloadType != codec.PayloadType {
-		r.context.params.Codecs = []RTPCodecParameters{codec}
+		context.params.Codecs = []RTPCodecParameters{codec}
 	}
 
-	r.track = track
+	r.trackEncodings[0].track = track
 	return nil
 }
 
@@ -204,29 +288,42 @@ func (r *RTPSender) Send(parameters RTPSendParameters) error {
 	switch {
 	case r.hasSent():
 		return errRTPSenderSendAlreadyCalled
-	case r.track == nil:
+	case r.trackEncodings[0].track == nil:
 		return errRTPSenderTrackRemoved
 	}
 
-	writeStream := &interceptorToTrackLocalWriter{}
-	r.context = TrackLocalContext{
-		id:          r.id,
-		params:      r.api.mediaEngine.getRTPParametersByKind(r.track.Kind(), []RTPTransceiverDirection{RTPTransceiverDirectionSendonly}),
-		ssrc:        parameters.Encodings[0].SSRC,
-		writeStream: writeStream,
-	}
+	for idx, trackEncoding := range r.trackEncodings {
+		writeStream := &interceptorToTrackLocalWriter{}
+		trackEncoding.context = TrackLocalContext{
+			id:              r.id,
+			params:          r.api.mediaEngine.getRTPParametersByKind(trackEncoding.track.Kind(), []RTPTransceiverDirection{RTPTransceiverDirectionSendonly}),
+			ssrc:            parameters.Encodings[idx].SSRC,
+			writeStream:     writeStream,
+			rtcpInterceptor: trackEncoding.rtcpInterceptor,
+		}
 
-	codec, err := r.track.Bind(r.context)
-	if err != nil {
-		return err
+		codec, err := trackEncoding.track.Bind(trackEncoding.context)
+		if err != nil {
+			return err
+		}
+		trackEncoding.context.params.Codecs = []RTPCodecParameters{codec}
+
+		trackEncoding.streamInfo = *createStreamInfo(
+			r.id,
+			parameters.Encodings[idx].SSRC,
+			codec.PayloadType,
+			codec.RTPCodecCapability,
+			parameters.HeaderExtensions,
+		)
+		srtpStream := trackEncoding.srtpStream
+		rtpInterceptor := r.api.interceptor.BindLocalStream(
+			&trackEncoding.streamInfo,
+			interceptor.RTPWriterFunc(func(header *rtp.Header, payload []byte, attributes interceptor.Attributes) (int, error) {
+				return srtpStream.WriteRTP(header, payload)
+			}),
+		)
+		writeStream.interceptor.Store(rtpInterceptor)
 	}
-	r.context.params.Codecs = []RTPCodecParameters{codec}
-
-	r.streamInfo = *createStreamInfo(r.id, parameters.Encodings[0].SSRC, codec.PayloadType, codec.RTPCodecCapability, parameters.HeaderExtensions)
-	rtpInterceptor := r.api.interceptor.BindLocalStream(&r.streamInfo, interceptor.RTPWriterFunc(func(header *rtp.Header, payload []byte, attributes interceptor.Attributes) (int, error) {
-		return r.srtpStream.WriteRTP(header, payload)
-	}))
-	writeStream.interceptor.Store(rtpInterceptor)
 
 	close(r.sendCalled)
 	return nil
@@ -252,16 +349,20 @@ func (r *RTPSender) Stop() error {
 		return err
 	}
 
-	r.api.interceptor.UnbindLocalStream(&r.streamInfo)
+	errs := []error{}
+	for _, trackEncoding := range r.trackEncodings {
+		r.api.interceptor.UnbindLocalStream(&trackEncoding.streamInfo)
+		errs = append(errs, trackEncoding.srtpStream.Close())
+	}
 
-	return r.srtpStream.Close()
+	return util.FlattenErrs(errs)
 }
 
-// Read reads incoming RTCP for this RTPReceiver
+// Read reads incoming RTCP for this RTPSender
 func (r *RTPSender) Read(b []byte) (n int, a interceptor.Attributes, err error) {
 	select {
 	case <-r.sendCalled:
-		return r.rtcpInterceptor.Read(b, a)
+		return r.trackEncodings[0].rtcpInterceptor.Read(b, a)
 	case <-r.stopCalled:
 		return 0, nil, io.ErrClosedPipe
 	}
@@ -283,10 +384,50 @@ func (r *RTPSender) ReadRTCP() ([]rtcp.Packet, interceptor.Attributes, error) {
 	return pkts, attributes, nil
 }
 
+// ReadSimulcast reads incoming RTCP for this RTPSender for given rid
+func (r *RTPSender) ReadSimulcast(b []byte, rid string) (n int, a interceptor.Attributes, err error) {
+	select {
+	case <-r.sendCalled:
+		for _, t := range r.trackEncodings {
+			if t.track != nil && t.track.RID() == rid {
+				return t.rtcpInterceptor.Read(b, a)
+			}
+		}
+		return 0, nil, fmt.Errorf("%w: %s", errRTPSenderNoTrackForRID, rid)
+	case <-r.stopCalled:
+		return 0, nil, io.ErrClosedPipe
+	}
+}
+
+// ReadSimulcastRTCP is a convenience method that wraps ReadSimulcast and unmarshal for you
+func (r *RTPSender) ReadSimulcastRTCP(rid string) ([]rtcp.Packet, interceptor.Attributes, error) {
+	b := make([]byte, r.api.settingEngine.getReceiveMTU())
+	i, attributes, err := r.ReadSimulcast(b, rid)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	pkts, err := rtcp.Unmarshal(b[:i])
+	return pkts, attributes, err
+}
+
 // SetReadDeadline sets the deadline for the Read operation.
 // Setting to zero means no deadline.
 func (r *RTPSender) SetReadDeadline(t time.Time) error {
-	return r.srtpStream.SetReadDeadline(t)
+	return r.trackEncodings[0].srtpStream.SetReadDeadline(t)
+}
+
+// SetReadDeadlineSimulcast sets the max amount of time the RTCP stream for a given rid will block before returning. 0 is forever.
+func (r *RTPSender) SetReadDeadlineSimulcast(deadline time.Time, rid string) error {
+	r.mu.RLock()
+	defer r.mu.RUnlock()
+
+	for _, t := range r.trackEncodings {
+		if t.track != nil && t.track.RID() == rid {
+			return t.srtpStream.SetReadDeadline(deadline)
+		}
+	}
+	return fmt.Errorf("%w: %s", errRTPSenderNoTrackForRID, rid)
 }
 
 // hasSent tells if data has been ever sent for this instance
diff --git a/sdp.go b/sdp.go
index c181bfe6..843bcb7f 100644
--- a/sdp.go
+++ b/sdp.go
@@ -340,7 +340,60 @@ func populateLocalCandidates(sessionDescription *SessionDescription, i *ICEGathe
 	}
 }
 
-func addTransceiverSDP(d *sdp.SessionDescription, isPlanB, shouldAddCandidates bool, dtlsFingerprints []DTLSFingerprint, mediaEngine *MediaEngine, midValue string, iceParams ICEParameters, candidates []ICECandidate, dtlsRole sdp.ConnectionRole, iceGatheringState ICEGatheringState, mediaSection mediaSection) (bool, error) {
+func addSenderSDP(
+	mediaSection mediaSection,
+	isPlanB bool,
+	media *sdp.MediaDescription,
+) {
+	for _, mt := range mediaSection.transceivers {
+		sender := mt.Sender()
+		if sender == nil {
+			continue
+		}
+
+		track := sender.Track()
+		if track == nil {
+			continue
+		}
+
+		sendParameters := sender.GetParameters()
+		for _, encoding := range sendParameters.Encodings {
+			media = media.WithMediaSource(uint32(encoding.SSRC), track.StreamID() /* cname */, track.StreamID() /* streamLabel */, track.ID())
+			if !isPlanB {
+				media = media.WithPropertyAttribute("msid:" + track.StreamID() + " " + track.ID())
+			}
+		}
+
+		if len(sendParameters.Encodings) > 1 {
+			sendRids := make([]string, 0, len(sendParameters.Encodings))
+
+			for _, encoding := range sendParameters.Encodings {
+				media.WithValueAttribute(sdpAttributeRid, encoding.RID+" send")
+				sendRids = append(sendRids, encoding.RID)
+			}
+			// Simulcast
+			media.WithValueAttribute("simulcast", "send "+strings.Join(sendRids, ";"))
+		}
+
+		if !isPlanB {
+			break
+		}
+	}
+}
+
+func addTransceiverSDP(
+	d *sdp.SessionDescription,
+	isPlanB bool,
+	shouldAddCandidates bool,
+	dtlsFingerprints []DTLSFingerprint,
+	mediaEngine *MediaEngine,
+	midValue string,
+	iceParams ICEParameters,
+	candidates []ICECandidate,
+	dtlsRole sdp.ConnectionRole,
+	iceGatheringState ICEGatheringState,
+	mediaSection mediaSection,
+) (bool, error) {
 	transceivers := mediaSection.transceivers
 	if len(transceivers) < 1 {
 		return false, errSDPZeroTransceivers
@@ -410,16 +463,7 @@ func addTransceiverSDP(d *sdp.SessionDescription, isPlanB, shouldAddCandidates b
 		media.WithValueAttribute("simulcast", "recv "+strings.Join(recvRids, ";"))
 	}
 
-	for _, mt := range transceivers {
-		if sender := mt.Sender(); sender != nil && sender.Track() != nil {
-			track := sender.Track()
-			media = media.WithMediaSource(uint32(sender.ssrc), track.StreamID() /* cname */, track.StreamID() /* streamLabel */, track.ID())
-			if !isPlanB {
-				media = media.WithPropertyAttribute("msid:" + track.StreamID() + " " + track.ID())
-				break
-			}
-		}
-	}
+	addSenderSDP(mediaSection, isPlanB, media)
 
 	media = media.WithPropertyAttribute(t.Direction().String())
 
diff --git a/srtp_writer_future.go b/srtp_writer_future.go
index 94299674..834d4613 100644
--- a/srtp_writer_future.go
+++ b/srtp_writer_future.go
@@ -16,6 +16,7 @@ import (
 // srtpWriterFuture blocks Read/Write calls until
 // the SRTP Session is available
 type srtpWriterFuture struct {
+	ssrc           SSRC
 	rtpSender      *RTPSender
 	rtcpReadStream atomic.Value // *srtp.ReadStreamSRTCP
 	rtpWriteStream atomic.Value // *srtp.WriteStreamSRTP
@@ -52,7 +53,7 @@ func (s *srtpWriterFuture) init(returnWhenNoSRTP bool) error {
 		return err
 	}
 
-	rtcpReadStream, err := srtcpSession.OpenReadStream(uint32(s.rtpSender.ssrc))
+	rtcpReadStream, err := srtcpSession.OpenReadStream(uint32(s.ssrc))
 	if err != nil {
 		return err
 	}
diff --git a/track_local.go b/track_local.go
index 0002e609..4b1c0ca6 100644
--- a/track_local.go
+++ b/track_local.go
@@ -1,6 +1,9 @@
 package webrtc
 
-import "github.com/pion/rtp"
+import (
+	"github.com/pion/interceptor"
+	"github.com/pion/rtp"
+)
 
 // TrackLocalWriter is the Writer for outbound RTP Packets
 type TrackLocalWriter interface {
@@ -14,10 +17,11 @@ type TrackLocalWriter interface {
 // TrackLocalContext is the Context passed when a TrackLocal has been Binded/Unbinded from a PeerConnection, and used
 // in Interceptors.
 type TrackLocalContext struct {
-	id          string
-	params      RTPParameters
-	ssrc        SSRC
-	writeStream TrackLocalWriter
+	id              string
+	params          RTPParameters
+	ssrc            SSRC
+	writeStream     TrackLocalWriter
+	rtcpInterceptor interceptor.RTCPReader
 }
 
 // CodecParameters returns the negotiated RTPCodecParameters. These are the codecs supported by both
@@ -49,6 +53,11 @@ func (t *TrackLocalContext) ID() string {
 	return t.id
 }
 
+// RTCPReader returns the RTCP interceptor for this TrackLocal. Used to read RTCP of this TrackLocal.
+func (t *TrackLocalContext) RTCPReader() interceptor.RTCPReader {
+	return t.rtcpInterceptor
+}
+
 // TrackLocal is an interface that controls how the user can send media
 // The user can provide their own TrackLocal implementations, or use
 // the implementations in pkg/media


Test patch:
diff --git a/peerconnection_media_test.go b/peerconnection_media_test.go
index 8c5c7aa1..6390abfa 100644
--- a/peerconnection_media_test.go
+++ b/peerconnection_media_test.go
@@ -162,7 +162,7 @@ func TestPeerConnection_Media_Sample(t *testing.T) {
 	go func() {
 		for {
 			time.Sleep(time.Millisecond * 100)
-			if routineErr := pcOffer.WriteRTCP([]rtcp.Packet{&rtcp.PictureLossIndication{SenderSSRC: uint32(sender.ssrc), MediaSSRC: uint32(sender.ssrc)}}); routineErr != nil {
+			if routineErr := pcOffer.WriteRTCP([]rtcp.Packet{&rtcp.PictureLossIndication{SenderSSRC: uint32(sender.trackEncodings[0].ssrc), MediaSSRC: uint32(sender.trackEncodings[0].ssrc)}}); routineErr != nil {
 				awaitRTCPSenderSend <- routineErr
 			}
 
@@ -643,12 +643,12 @@ func TestAddTransceiverAddTrack_Reuse(t *testing.T) {
 	track1, sender1 := addTrack()
 	assert.Equal(t, 1, len(pc.GetTransceivers()))
 	assert.Equal(t, sender1, tr.Sender())
-	assert.Equal(t, track1, tr.Sender().track)
+	assert.Equal(t, track1, tr.Sender().Track())
 	require.NoError(t, pc.RemoveTrack(sender1))
 
 	track2, _ := addTrack()
 	assert.Equal(t, 1, len(pc.GetTransceivers()))
-	assert.Equal(t, track2, tr.Sender().track)
+	assert.Equal(t, track2, tr.Sender().Track())
 
 	addTrack()
 	assert.Equal(t, 2, len(pc.GetTransceivers()))
@@ -1256,23 +1256,47 @@ func TestPeerConnection_Simulcast(t *testing.T) {
 		pcOffer, pcAnswer, err := NewAPI(WithMediaEngine(m)).newPair(Configuration{})
 		assert.NoError(t, err)
 
-		vp8Writer, err := NewTrackLocalStaticRTP(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion2")
+		vp8WriterA, err := NewTrackLocalStaticRTP(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion2", WithRTPStreamID("a"))
 		assert.NoError(t, err)
 
-		_, err = pcOffer.AddTrack(vp8Writer)
+		sender, err := pcOffer.AddTrack(vp8WriterA)
+		assert.NoError(t, err)
+		assert.NotNil(t, sender)
+
+		vp8WriterB, err := NewTrackLocalStaticRTP(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion2", WithRTPStreamID("b"))
+		assert.NoError(t, err)
+		err = sender.AddEncoding(vp8WriterB)
+		assert.NoError(t, err)
+
+		vp8WriterC, err := NewTrackLocalStaticRTP(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion2", WithRTPStreamID("c"))
+		assert.NoError(t, err)
+		err = sender.AddEncoding(vp8WriterC)
 		assert.NoError(t, err)
 
 		ridMap = map[string]int{}
 		pcAnswer.OnTrack(onTrackHandler)
 
-		assert.NoError(t, signalPairWithModification(pcOffer, pcAnswer, func(sessionDescription string) string {
-			sessionDescription = strings.Split(sessionDescription, "a=end-of-candidates\r\n")[0]
-			sessionDescription = filterSsrc(sessionDescription)
-			for _, rid := range rids {
-				sessionDescription += "a=" + sdpAttributeRid + ":" + rid + " send\r\n"
+		parameters := sender.GetParameters()
+		assert.Equal(t, "a", parameters.Encodings[0].RID)
+		assert.Equal(t, "b", parameters.Encodings[1].RID)
+		assert.Equal(t, "c", parameters.Encodings[2].RID)
+
+		var midID, ridID, rsidID uint8
+		for _, extension := range parameters.HeaderExtensions {
+			switch extension.URI {
+			case sdp.SDESMidURI:
+				midID = uint8(extension.ID)
+			case sdp.SDESRTPStreamIDURI:
+				ridID = uint8(extension.ID)
+			case sdesRepairRTPStreamIDURI:
+				rsidID = uint8(extension.ID)
 			}
-			return sessionDescription + "a=simulcast:send " + strings.Join(rids, ";") + "\r\n"
-		}))
+		}
+		assert.NotZero(t, midID)
+		assert.NotZero(t, ridID)
+		assert.NotZero(t, rsidID)
+
+		assert.NoError(t, signalPair(pcOffer, pcAnswer))
 
 		for sequenceNumber := uint16(0); !ridsFullfilled(); sequenceNumber++ {
 			time.Sleep(20 * time.Millisecond)
@@ -1284,17 +1308,26 @@ func TestPeerConnection_Simulcast(t *testing.T) {
 					SequenceNumber: sequenceNumber,
 					PayloadType:    96,
 				}
-				assert.NoError(t, header.SetExtension(1, []byte("0")))
+				assert.NoError(t, header.SetExtension(midID, []byte("0")))
 
 				// Send RSID for first 10 packets
 				if sequenceNumber >= 10 {
-					assert.NoError(t, header.SetExtension(2, []byte(rid)))
+					assert.NoError(t, header.SetExtension(ridID, []byte(rid)))
 				} else {
-					assert.NoError(t, header.SetExtension(3, []byte(rid)))
+					assert.NoError(t, header.SetExtension(rsidID, []byte(rid)))
 					header.SSRC += 10
 				}
 
-				_, err := vp8Writer.bindings[0].writeStream.WriteRTP(header, []byte{0x00})
+				var writer *TrackLocalStaticRTP
+				switch rid {
+				case "a":
+					writer = vp8WriterA
+				case "b":
+					writer = vp8WriterB
+				case "c":
+					writer = vp8WriterC
+				}
+				_, err = writer.bindings[0].writeStream.WriteRTP(header, []byte{0x00})
 				assert.NoError(t, err)
 			}
 		}
diff --git a/peerconnection_renegotiation_test.go b/peerconnection_renegotiation_test.go
index ad3c6a67..e081c1c9 100644
--- a/peerconnection_renegotiation_test.go
+++ b/peerconnection_renegotiation_test.go
@@ -371,7 +371,7 @@ func TestPeerConnection_Transceiver_Mid(t *testing.T) {
 	// Must have 3 media descriptions (2 video channels)
 	assert.Equal(t, len(offer.parsed.MediaDescriptions), 2)
 
-	assert.True(t, sdpMidHasSsrc(offer, "0", sender1.ssrc), "Expected mid %q with ssrc %d, offer.SDP: %s", "0", sender1.ssrc, offer.SDP)
+	assert.True(t, sdpMidHasSsrc(offer, "0", sender1.trackEncodings[0].ssrc), "Expected mid %q with ssrc %d, offer.SDP: %s", "0", sender1.trackEncodings[0].ssrc, offer.SDP)
 
 	// Remove first track, must keep same number of media
 	// descriptions and same track ssrc for mid 1 as previous
@@ -382,7 +382,7 @@ func TestPeerConnection_Transceiver_Mid(t *testing.T) {
 
 	assert.Equal(t, len(offer.parsed.MediaDescriptions), 2)
 
-	assert.True(t, sdpMidHasSsrc(offer, "1", sender2.ssrc), "Expected mid %q with ssrc %d, offer.SDP: %s", "1", sender2.ssrc, offer.SDP)
+	assert.True(t, sdpMidHasSsrc(offer, "1", sender2.trackEncodings[0].ssrc), "Expected mid %q with ssrc %d, offer.SDP: %s", "1", sender2.trackEncodings[0].ssrc, offer.SDP)
 
 	_, err = pcAnswer.CreateAnswer(nil)
 	assert.Equal(t, err, &rtcerr.InvalidStateError{Err: ErrIncorrectSignalingState})
@@ -402,8 +402,8 @@ func TestPeerConnection_Transceiver_Mid(t *testing.T) {
 	// We reuse the existing non-sending transceiver
 	assert.Equal(t, len(offer.parsed.MediaDescriptions), 2)
 
-	assert.True(t, sdpMidHasSsrc(offer, "0", sender3.ssrc), "Expected mid %q with ssrc %d, offer.sdp: %s", "0", sender3.ssrc, offer.SDP)
-	assert.True(t, sdpMidHasSsrc(offer, "1", sender2.ssrc), "Expected mid %q with ssrc %d, offer.sdp: %s", "1", sender2.ssrc, offer.SDP)
+	assert.True(t, sdpMidHasSsrc(offer, "0", sender3.trackEncodings[0].ssrc), "Expected mid %q with ssrc %d, offer.sdp: %s", "0", sender3.trackEncodings[0].ssrc, offer.SDP)
+	assert.True(t, sdpMidHasSsrc(offer, "1", sender2.trackEncodings[0].ssrc), "Expected mid %q with ssrc %d, offer.sdp: %s", "1", sender2.trackEncodings[0].ssrc, offer.SDP)
 
 	closePairNow(t, pcOffer, pcAnswer)
 }
diff --git a/rtpsender_test.go b/rtpsender_test.go
index ebdbc760..28831bb6 100644
--- a/rtpsender_test.go
+++ b/rtpsender_test.go
@@ -117,7 +117,7 @@ func Test_RTPSender_GetParameters(t *testing.T) {
 	parameters := rtpTransceiver.Sender().GetParameters()
 	assert.NotEqual(t, 0, len(parameters.Codecs))
 	assert.Equal(t, 1, len(parameters.Encodings))
-	assert.Equal(t, rtpTransceiver.Sender().ssrc, parameters.Encodings[0].SSRC)
+	assert.Equal(t, rtpTransceiver.Sender().trackEncodings[0].ssrc, parameters.Encodings[0].SSRC)
 	assert.Equal(t, "", parameters.Encodings[0].RID)
 
 	closePairNow(t, offerer, answerer)
@@ -340,3 +340,64 @@ func Test_RTPSender_Send_Track_Removed(t *testing.T) {
 
 	assert.NoError(t, peerConnection.Close())
 }
+
+func Test_RTPSender_Add_Encoding(t *testing.T) {
+	track, err := NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion")
+	assert.NoError(t, err)
+
+	peerConnection, err := NewPeerConnection(Configuration{})
+	assert.NoError(t, err)
+
+	rtpSender, err := peerConnection.AddTrack(track)
+	assert.NoError(t, err)
+
+	assert.Equal(t, errRTPSenderTrackNil, rtpSender.AddEncoding(nil))
+
+	track1, err := NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion")
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderRidNil, rtpSender.AddEncoding(track1))
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion", WithRTPStreamID("h"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderNoBaseEncoding, rtpSender.AddEncoding(track1))
+
+	track, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion", WithRTPStreamID("q"))
+	assert.NoError(t, err)
+
+	rtpSender, err = peerConnection.AddTrack(track)
+	assert.NoError(t, err)
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video1", "pion", WithRTPStreamID("h"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderBaseEncodingMismatch, rtpSender.AddEncoding(track1))
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion1", WithRTPStreamID("h"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderBaseEncodingMismatch, rtpSender.AddEncoding(track1))
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeOpus}, "video", "pion", WithRTPStreamID("h"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderBaseEncodingMismatch, rtpSender.AddEncoding(track1))
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion", WithRTPStreamID("q"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderRIDCollision, rtpSender.AddEncoding(track1))
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion", WithRTPStreamID("h"))
+	assert.NoError(t, err)
+	assert.NoError(t, rtpSender.AddEncoding(track1))
+
+	err = rtpSender.Send(rtpSender.GetParameters())
+	assert.NoError(t, err)
+
+	track1, err = NewTrackLocalStaticSample(RTPCodecCapability{MimeType: MimeTypeVP8}, "video", "pion", WithRTPStreamID("f"))
+	assert.NoError(t, err)
+	assert.Equal(t, errRTPSenderSendAlreadyCalled, rtpSender.AddEncoding(track1))
+
+	err = rtpSender.Stop()
+	assert.NoError(t, err)
+
+	assert.Equal(t, errRTPSenderStopped, rtpSender.AddEncoding(track1))
+
+	assert.NoError(t, peerConnection.Close())
+}