Multiplexing FEC protection of multiple streams with different delay requirements

In one embodiment, a device in a network identifies delay requirements of each of a plurality of media streams. The device selects a joint forward error correction (FEC) encoding strategy for the plurality of media streams based on the identified delay requirements of the streams and on a burst loss length of a communication channel. The device applies the selected joint FEC encoding strategy to the plurality of media streams, to form a multiplexed packet stream. The device sends the multiplexed packet stream to one or more nodes in the network via the communication channel.

TECHNICAL FIELD

The present disclosure relates generally to computer networks, and, more particularly, to multiplexing forward error correction (FEC) protection of multiple streams with different delay requirements.

BACKGROUND

It is common in conferencing systems today to have multiple streams with varying delay requirements traversing the same paths. For example, consider the use case of a video conferencing presentation. In such a case, there may be at least three different media streams: the actual presentation stream (e.g., presentation slides), a video stream (e.g., a webcam feed of the presenter), and an associated audio stream (e.g., the captured voice of the presenter).

Different types of media may have different delay requirements. For example, audio and video streams may have much tighter delay requirements than that of a slide presentation stream. Notably, a conferencing participant may not even notice a slight delay in the presentation stream. However, the video and audio streams may have much tighter delay requirements than the corresponding slide presentation stream. Furthermore, even with the video and audio for the same visual session, the audio data may have a tighter delay constraint than the associated video stream. For example, displaying a video frame slightly late may be imperceptible to the user, while a gap in the audio of a speaker can be highly distracting.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

According to one or more embodiments of the disclosure, a device in a network identifies delay requirements of each of a plurality of media streams. The device selects a joint forward error correction (FEC) encoding strategy for the plurality of media streams based on the identified delay requirements of the streams and on a burst loss length of a communication channel. The device applies the selected joint FEC encoding strategy to the plurality of media streams, to form a multiplexed packet stream. The device sends the multiplexed packet stream to one or more nodes in the network via the communication channel.

DESCRIPTION

FIG. 1Aillustrates an example computer system100, according to various embodiments of the present disclosure. As shown, a client device102may be in communication with a media source device106via one or more computer networks104. Media source device106provides media stream packets108through network(s)104to client device102. As will be appreciated, network(s)104may include, but are not limited to, local area networks (LANs), wide area networks (WANs), the Internet, cellular networks, infrared networks, satellite networks, or any other form of data network configured to convey data between computing devices.

Network(s)104may include any number of wired or wireless links between client device102and media source device106. Example wired links may include, but are not limited to, fiber optic links, Ethernet-based links (e.g., Category 5/5e cabling, Category 6 cabling, etc.), digital subscriber line (DSL) links, coaxial links, T carrier links, E carrier links, combinations thereof, or the like. Example wireless links may include, but are not limited to, near field-based links, WiFi links, satellite links, cellular links, infrared links, combinations thereof, or the like.

Client device102may be of any form of electronic device operable to communicate via network(s)104. For example, client device102may be a desktop computer, a laptop computer, a tablet device, a smartphone, a wearable electronic device (e.g., a smart watch, a head up display, etc.), a smart television, a set-top device for a television, etc.

In general, client device102may be operable to receive media stream packets108and render the received content data on an electronic display. For example, client device102may execute a media streaming application that, when executed by client device102, is configured to request streamed media, such as streaming video, audio, or both. In various embodiments, the media streaming application may be a stand-alone application or, alternatively, may be another form of application that is operable to render and display streaming media (e.g., a mobile application, etc.).

As shown inFIG. 1A, client device102may send a media streaming request to media source device106through network(s)104. In response to receiving the request, media source device106may send media streaming packets108to client device102through network(s)104. The client device may repeat the above process any number of times with the same or different media source devices, depending on the contents of streaming media.

As would be appreciated, whileFIG. 1Adepicts media source device106sending media stream packets108to client device102, some implementations may also provide for client device102to send media stream packets108in the opposite direction, as well. For example, in the case of an online conference, client device102may send locally-captured audio and/or video to media source device106. Said differently, while client device102may act as a client with respect to media source device106, client device102may also act as its own media source device, in some embodiments.

FIG. 1Billustrates another potential configuration for communication network100, in a further example. In this example, rather than media source106sending media stream packets108directly to client device102via network104, media source106may instead send media stream packets108to a relay node106a. For example, relay node106amay be part of a cloud-based service, such as a conferencing service or the like. Thus, in some cases, relay node106amay itself be a media source from the perspective of client device102and may combine streams from any number of originators (e.g., audio, video, presentation, etc.).

FIG. 2is a schematic block diagram of an example node/device200that may be used with one or more embodiments described herein, e.g., as any of the nodes shown inFIG. 1above. In particular, device200may be client device102or media source device106. The device may comprise one or more network interfaces210(e.g., wired, wireless, etc.), at least one processor220, and a memory240interconnected by a system bus250, as well as a power supply260(e.g., battery, plug-in, etc.).

The network interface(s)210contain the mechanical, electrical, and signaling circuitry for communicating data to network104. The network interfaces may be configured to transmit and/or receive data using a variety of different communication protocols. Note, further, that the nodes/devices may have two different types of network connections210, e.g., wireless and wired/physical connections, and that the view herein is merely for illustration.

The memory240comprises a plurality of storage locations that are addressable by the processor220and the network interfaces210for storing software programs and data structures associated with the embodiments described herein. Note that certain devices may have limited memory or no memory (e.g., no memory for storage other than for programs/processes operating on the device and associated caches). The processor220may comprise hardware elements or hardware logic adapted to execute the software programs and manipulate the data structures245. An operating system242, portions of which is typically resident in memory240and executed by the processor, functionally organizes the device by, inter alia, invoking operations in support of software processes and/or services executing on the device. These software processes and/or services may include media streaming process248, as described herein.

As noted above, streaming media for a client media session may include multiple streams of different media. For example, a media stream may include both a video stream and a corresponding audio stream. Another example may be a media stream from a cloud-based conferencing system that relays media from different conferencing sessions. Also as noted above, the different media streams being combined in the session may have different delay requirements/deadlines before the user experience is impacted. For example, presentation streams may be more tolerant of delays than audio streams, etc.

In general, forward error correction (FEC) is a mechanism that attempts to protect network communications from errors and losses/erasures. In particular, FEC allows a sender to encode a communication in such a way that if portions/packets of the communication are lost during transmission, the receiver is still able to reconstruct the original message. In the context of media streaming, FEC is often used, due to the timing constraints involved. For example, the delay requirements of media streams are often such that it would be impossible for the receiver to request retransmission of a lost packet, while still presenting the lost data to the user in time.

Typically, media streams (e.g., audio, video, presentation, etc.) are protected by FEC independently and sent as separate streams through the network. In contrast, according to various embodiments, the sender may combine the media streams into a unified media stream and apply the same FEC in a straight-forward manner across the aggregated media streams, thereby improving FEC efficiency (e.g., reducing FEC redundancy, etc.). However, simply applying the same FEC across all of the streams would effectively ignore the different delay requirements of the streams.

Multiplexing FEC Protection of Multiple Streams with Different Delay Requirements

The techniques herein allow a system to jointly apply FEC protection to a plurality of media streams with different delay requirements to hit the theoretical optimal bounds for recovery from burst loss, at least in some cases. Regardless of whether the optimal bounds are achieved, the techniques herein demonstrate better performance in terms of efficiency and redundancy over encoding each stream individually. In some aspects, depending on the delay requirements of the different media streams, and potentially other factors such as the bit rate ratios between the streams, the system may dynamically switch between FEC encoding strategies.

Specifically, according to one or more embodiments of the disclosure as described in detail below, a device in a network identifies delay requirements of each of a plurality of media streams. The device selects a joint forward error correction (FEC) encoding strategy for the plurality of media streams based on the identified delay requirements of the streams and on a burst loss length of a communication channel. The device applies the selected joint FEC encoding strategy to the plurality of media streams, to form a multiplexed packet stream. The device sends the multiplexed packet stream to one or more nodes in the network via the communication channel.

Illustratively, the techniques described herein may be performed by hardware, software, and/or firmware, such as in accordance with the media streaming process248, which may contain computer executable instructions executed by the processor220to perform functions relating to the techniques described herein. For example, the techniques herein may be treated as extensions to conventional protocols, such as the various communication or media streaming protocols, and as such, may be processed by similar components understood in the art that execute those protocols, accordingly.

Operationally,FIG. 3illustrates an example forward error correction (FEC) mechanism300for a media stream, in accordance with the embodiments herein. Continuing the example ofFIG. 1, media source106may execute an encoder302and client device102may execute a corresponding decoder306, both of which may be sub-processes of a media streaming application (e.g., media streaming process248). In further embodiments, media source106may itself be a relay node, such as relay node106ashown inFIG. 1B.

In various embodiments, encoder302may apply FEC encoding to any number of source media streams, such as media streams308a-308bshown. For purposes of simplicity, the techniques herein are described with respect to multiplexing two media streams. However, as would be appreciated, the techniques herein can be applied similarly to any number of multiplexed media streams. For example, in one embodiment, three or more media streams may be grouped into two categories based on their individual delay requirements. In turn, the techniques herein can be applied to the two groups by setting the delay requirements of each group to be the minimum delay required by any of the steams in a given category.

As a result of the multiplexing and FEC encoding by encoder302, media source106may send packets108to client device102via network106that include the source and parity information needed by client device102to recover the original media streams308a-308b. In particular, media source106may send packets108via a so-called “erasure” channel306in network104that derives its name from the fact that packets108may be dropped/erased during transmission to client device102. More generally, however, channel306may be any form of communication channel in the network and the techniques herein may also be applied to channels in which no actual erasures occur (e.g., channels that experience delays, etc.).

Client device102may receive packets108and process packets108using decoder306, to produce recovered media streams310a-310b. Ideally, recovered media streams310a-310bwill be identical to media streams308a-308b. However, packet losses along channel304may result in the loss of some information during transit. Accordingly, decoder306may operate to correct for erasures/losses along erasures channel304by identifying packets that did not reach decoder306within the expected time and recovering lost data using the parity packets that decoder306actually receives. In turn, decoder306may provide the recovered media streams310a-310bto the corresponding processes/devices for presentation of the media to the user of client device102(e.g., via a display, speaker, etc.).

In further case, streams308a-308bmay be sent to client device102via a relay node, such as the case illustrated inFIG. 1A. When streams308a-308bare originally sent to the relay node, streams308a-308bmay have either the same or different latency requirements. If, for example, streams308a-308bhave different latency constraints, then the techniques herein can be directly applied at the sender (e.g., the original media source106), in one embodiment. In either case, assume that stream308ais affected by packet loss while enroute to the relay node. In turn, the relay node may be able to reconstruct the lost packets and retransmit to client device102. However, reconstruction of the packets by the relay node will also result in additional delay. Thus, in this example, stream308amay have tighter delay constraints than before and, in various embodiments, the techniques herein can also be applied at the relay node, or reapplied and adapted at the relay node, if the techniques herein are initially applied at the sender, and across streams308a-308bwhile accounting for the different delay constraints. Note that the above network configurations are illustrative only and the techniques herein may be applied by any device in a network that applies encoding to media streams.

To fully recover the media data lost during a bust loss of length B, decoder306must wait enough time to receive B parity packets. However, as noted previously, each media stream may its own delay requirements before such a delay is noticeable to the user of client device102. For decoder306to recover a single burst B by a deadline T, there is a lower bound on the FEC overhead. In particular, this corresponds to an upper limit on the FEC “rate,” which is defined as follows:

R=BytesDataBytesData+BytesFEC
where R is the FEC encoding rate, BytesDatais the amount of media data bytes transmitted by packets108, and BytesFECis the amount of FEC bytes added by encoder302to packets108. For example, a 100% FEC overhead corresponds to an FEC encoding rate of 1/(1+1)=0.5, while a 10% FEC overhead corresponds to an FEC encoding rate of 1(1+0.1)=0.91. In other words, the encoding rate is a measure of efficiency and a high rate corresponds to high efficiency and low overhead.

In order to satisfy the delay requirements of media streams308a-308b, certain constraints may exist for the encoding methodology used by encoder302. Notably, let S1[i] represent media stream308aand S2[i] represent media stream308band let S[i]=(S1[i], S2[i]). The sizes of S1[i] and S2[i] are αk and (1−α)k, respectively, where α is the fraction of bytes in the low-deadline stream, S1[i]. Further, assume that the maximum tolerable delay for S1[i] and S2[i] are T1and T2, respectively. Thus, in the presence of a burst of length B or less in erasure channel304, decoder306must:1. decode S1[i] at time i+T1; and2. decode S2[i] at time i+T2.

Now, assume that encoder302generates a stream of parities p[i] of size n−k bytes. In such a case, the FEC encoding rate is k/n and the overhead is (n−k)/k. This gives the following bounds for the FEC rate:

Example plots illustrating the above bounds on the FEC encoding rate are show inFIGS. 4A-4B, 5A-5B and 6A-6Bas rates R1, R2and R3, respectively. In particular, plots400-410shown inFIGS. 4A-4Bplot the above bounds when B=10, T1=20, and T2=25. In other words, plots400-410inFIGS. 4A-4Billustrate the case in which one media stream has a relatively low delay requirement, e.g., T2≦T1+B. Plots500-510inFIGS. 5A-5Billustrate the case when the second stream has a relatively large delay, T2≧T1+2B, and T2=45 is used instead.FIGS. 6A-6Billustrate another case where T2=35, medium delay case, i.e., T1+B≦T2≦T1+2B. The minimum of the three bounds is plotted as min(R1, R2, R3). This is also the maximum possible FEC rate that can be achieved. The achievable FEC rate using the proposed FEC is also plotted as RMUX.

Also shown in plots400-410,500-510and600-610are curves RMSthat denote the FEC encoding rate achieved by applying separate error correcting codes independently to the two streams with different delays, but otherwise use optimal codes for their respective delays. In particular, such codes may use the encoding proposed in the paper, “Burst Erasure Correction Codes with Low Decoding Delay,” by E. Martinian and C. W. Sundburg, published in the IEEE Transactions on Information Theory, vol. 50, no. 10, pp. 2494-2502 (2004). Such an encoding is optimized for a single stream having a delay requirement.

Referring to plots410,510and610, particularly, the regions A-G above RMSand within the bound of min(R1, R2, R3) represent the potential gains that can be realized by a joint FEC code over simply encoding the streams separately. In particular, these regions demonstrate an increase in the achievable bit rate and a corresponding reduction in FEC overhead, while still providing the same performance as that of separate encoding (e.g., in terms of error correction capability and meeting the deadlines for each stream).

A key observation is that regions A-G shown in plots410,510and610ofFIGS. 4B, 5B and 6Bare achievable using different FEC encoding strategies, to jointly encode the media streams based on their delay requirements. In particular, encoder302may select an FEC encoding strategy that results in an encoding rate that falls within one of the optimal regions A-G, thereby demonstrating improved performance over encoding the streams separately.

FIG. 7illustrates an example decision tree700that encoder302may use to select an FEC encoding strategy based on the delay requirements of the multiplexed media streams, according to some embodiments. In particular encoder302may start the decision process by determining whether the delay requirement T2≦T1+B (block705). If so, encoder302may proceed to block710of decision tree700. However, if T2≧T1+2B, encoder302may proceed to block715. Otherwise, if T1+B≦T2≦T1+2B, encoder302may proceed to block720. As would be appreciated, these are the three different possible scenarios illustrated inFIGS. 4A-4B, 5A-5B, and 6A-6B.

Next, encoder302may proceed to one of blocks725-755, depending on the value of a, which is the fraction of bytes in the lower deadline media stream, i.e., stream S1[i] with delay requirement T1. In particular, if processing by encoder302is currently at block710(i.e., T2≦T1+B), encoder302may then determine whether α≦(T2−B)/T2. If so, then the optimal achievable FEC rate is given by R1which is the upper limit of region A shown in plot410ofFIG. 4B, and proceed to block725. However, if α≧(T2−B)/T2, encoder302may instead determine that the optimal achievable FEC rate is given by R3, the upper limit of region B shown in plot410ofFIG. 4B, and proceed to block725.

If processing by encoder302is currently at block715(i.e., T2≧T1+2B), encoder302may determine whether α≦T1/T2. If so, then encoder302may determine that the optimal achievable FEC rate is R2, the upper limit of region C shown in plot510ofFIG. 5B, and proceed to block735. Otherwise, encoder302may proceed to block740to achieve the optimal FEC rate of R1, the upper limit of region D in plot510ofFIG. 5B.

Similarly, if processing by encoder302is currently at block720(i.e., T1+B≦T2≦T1+2B), encoder302may determine where αε[0,(T2−B)/T2], αε((T2−2B)/T2,T1/T1+B), or αε[T1/T1+B,1], then encoder302may proceed to block745,750, or755and achieve an FEC rate given by the upper limits of region E, F or G shown in plot610ofFIG. 6B, respectively.

Depending on the results of decision tree700, encoder302may select an FEC encoding strategy that achieves an FEC rate for the corresponding one of regions A-G, according to various embodiments. Notably, encoder302may dynamically switch between encoding strategies, based on the delay requirements of the media streams, the burst loss length B, and/or the fraction of bytes in the low deadline stream (a).

By way of illustration of decision tree700, consider the case of a video stream and a presentation stream that are to be multiplexed with respective delay requirements of 120 ms and 230 ms. In other words, the presentation stream is more tolerable of delays than that of the video stream. Also, assume that the bit rates of the two streams are equal and that the channel mostly introduces burst losses of length below or equal to B=50 ms. In such a case, then [T2=230 ms]>[(T1=120 ms)+(2B=100 ms)=220 ms] and α=0.5<[(T1=120 ms)/(T2=230 ms)≈0.52], meaning that encoder302should proceed to block735in plot700inFIG. 7and achieve a rate of R2, the upper limit of region C in plot510inFIG. 5B. With these same parameters, optimally encoding the two streams independently would require an FEC rate of 0.76 and an FEC overhead of 32%. However, encoding the streams jointly to form a multiplexed stream using the encoding strategy for region A detailed below would require only an FEC rate of R2=(T2=230 ms)/(T2+B=280 ms)=0.82 and an FEC overhead of 22%, which also means a reduction in overhead from that of individual encoding by (32−22)/22=45%.

Encoding Strategy for Region A

If encoder302determines that an encoding strategy associated with region A should be used, encoder302may encode media streams308a-308bas follows, according to various embodiments. First, let S1[i], and S2[i] be the sequence of packets for the low and high delay streams308a-308b, respectively. Note that they are in general of different sizes. Then, encoder302may first split every packet of the second sequence into two sub-packets, i.e., S2[i]=(S2,1[ ],S2,2[i]) where S2,1[ ] and S2,2[i] are of sizes αBk/(T2−T1) and ((1−α)T2−B)k/(Tα−B) bytes, respectively. To obtain intermediate packets sequences P1[i], P2,1[i] and P2,2[i], encoder302may then apply the following codes:
S1[i]→MS(B,T2−B)→P1[i]
S2,1[i]→MS(T2,T2)→P2,1[i]
S2,2[i]→MS(B,T2)→P2,2[i].
where MS represents the optimal encoding for individual streams proposed by Martinian and C. W. Sundburg, mentioned earlier. In turn, encoder302may combine the intermediate packet sequences P1[i], P2,1[i] and P2,2[i] to form the final FEC packet=(P1[i]+P2,1[i], P2,2[i]), where + denotes a bit-wise XOR operation. Doing so achieves the desired rate T2/(T2+B)=R2, the upper limit of region A, and allows decoder306to recover from a burst loss B within the respective deadlines of the two streams.
Encoding Strategy for Region B

To achieve an FEC rate according to region B above, encoder302may instead apply the following encoding strategy, according to various embodiments. First, encoder302may split each packet in the low delay sequence, i.e., S1[i]=(S1,1[i],S1,2[i]), where S1,1[i] and S1,2[i] are of sizes (1−α)(T2−B)k/B bytes and (B−(1−α)T2)k/B bytes, respectively. Then, encoder302may form intermediate packet sequences P1,1[i], P1,2[i] and P2[i], as follows:
S1,1[i]→MS(B,T2−B)→P1,1[i],
S1,2[i]→MS(B,T1)→P1,2[i],
S2[i]→MS(T2,T2)→P2[i].
Encoder302may then form the final FEC packet as (P1,1[i]+P1,2[i], P2[i]). Doing so achieves the desired rate T1/(T1+B−(1−α)(T2−T1))=R3, the upper limit of region B, and allows decoder306to recover from a burst loss B within the respective deadlines of the two streams.
Encoding Strategy for Region C

To achieve an FEC rate according to region C above, encoder302may instead apply the following encoding strategy, according to various embodiments. First, encoder302splits each packet in the second sequence into three sub-packets, S2[i]=(S2,1[i], S2,2[i], S2,3[i]) of sizes α(T2−T1−B)k/T1bytes, αBk/T1 bytes and (T1−αT2)k/T1 bytes, respectively. Then, encoder302may generate intermediate packets P1[i], P2,1[i], P2,2[i] and P2,3[i] as follows:
S1[i]→MS(B,T1)→P1[i].
S2,1[i]→MS(B,T2−T1−B)→P2,1[i],
S2,2[i]→MS(T1+B,T1+B)→P2,2[i],
S2,3[i]→MS(B,T2)→P2,3[i],

In turn, encoder302may form the final FEC packet as (P1[i]+P2,1[i−T1−B]+P2,2[i], P2,3[i]). Doing so achieves the desired rate T2/(T2+B)=R2, the upper limit of region C, and allows decoder306to recover from a burst loss B within the respective deadlines of the two streams.

Encoding Strategy for Region D

To achieve an FEC rate according to region D above, encoder302may instead apply the following encoding strategy, according to various embodiments. First, encoder302splits each packet in the first sequence into two sub-packets, S1[i]=(S1,1[i], S1,2[i]) of sizes (1−α)T1k/(T2−T1) bytes and (αT2−T1)k/(T2−T1) bytes, respectively. Similarly, encoder302splits each packet in the second sequence into two sub-packets, S2[i]=(S2,1[i], S2,2[i]) of sizes (1−α)(T2−T1−B)k/(T2−T1) bytes and (1−α)Bk/(T2−T1) bytes, respectively. Then, encoder302may generate intermediate packets P1,1[i], P1,2[i], P2,1[i] and P2,2[i] as follows:
S1,1[i]→MS(B,T1)→P1,1[i].
S1,2[i]→MS(B,T1)→P1,2[i].
S2,1[i]→MS(B,T2−T1−B)→P2,1[i],
S2,2[i]→MS(T1+B,T1+B)→P2,2[i],
In turn, encoder302may form the final FEC packet as (P1,1[i]+P2,1[i−T1−B]+P2,2[i], P1,2[i]). Doing so achieves the desired rate T1/(T1+αB)=R1, the upper limit of region D, and allows decoder306to recover from a burst loss B within the respective deadlines of the two streams.
Encoding Strategy for Region E

To achieve an FEC rate according to region E above, encoder302may instead apply the following encoding strategy, according to various embodiments. First, encoder302updates the values of T1to T1*=T2−2B<T1. Since T2=T1*+2B and α≦T1*/T2=(T2−2B)/T2, encoder302can use the FEC of Region C above with parameters B, T1* and T2to achieve a rate of R2=T2/(T2+B), the upper limit of region E.

Encoding Strategy for Region F

To achieve an FEC rate according to region F above, encoder302may instead apply the following encoding strategy, according to various embodiments. Depending on the following two cases, encoder302may proceed differentlyCase 1: (T2−2B)/T2<α≦(T2−2B)/(T1+B)Encoder302may first update the value of T1to T1*=T2−2B<T1. Since α>T1*/T2=(T2−2B)/T2, encoder302may use the FEC of region D with parameters B, T1* and T2to achieve the FEC rate R1=T1*/(T1*+αB)=(T2−2B)/(T2−2B+αB) which falls within region F.Case 2: (T2−2B)/(T1+B)<α<T1/(T1+B)Encoder302may first update the value of T2to T2*=T1+B<T2. Since α<(T2*−B)/T2*=T1/(T1+B), encoder302may use the FEC of region A with parameters B, T1and T2* to achieve the FEC rate R2=T2*/(T2*+B)=(T1+B)/(T1+2B) which falls within region F.
Encoding Strategy for Region G

To achieve an FEC rate according to region G above, encoder302may instead apply the following encoding strategy, according to various embodiments. First, encoder302updates the values of T2to T2*=T1+B<T2. Since T2*=T1+B and α≧(T2*−B)/T2*=(T1)/(T1+B), encoder302can use the FEC of Region B above with parameters B, T1and T2* to achieve a rate of R3=T1/(T1+B−(1−α)(T2*−T1))=T1/(T1+αB)=R1, the upper limit of region G.

FIG. 8illustrates an example simplified procedure for encoding multiplexed media stream, in accordance with one or more embodiments described herein. The procedure800may start at step805, and continues to step810, where, as described in greater detail above, a device in a network (e.g., device200executing media streaming process248) may identify the delay requirements of each of a plurality of media streams. In some embodiments, the device may do so based on the type of media in each stream. For example, the device may determine the delay requirements of a particular stream based on whether the stream includes audio, video, or presentation data.

At step815, as detailed above, the device may select a joint FEC encoding strategy for the plurality of media streams. In some embodiments, the device may do so based on the identified delay requirements of the streams and on a burst loss length of an erasure/communication channel. For example, if the plurality of media streams includes two different streams, the device may determine whether any of the conditions depicted inFIG. 7are met.

At step820, the device may apply the selected joint FEC encoding strategy to the plurality of media streams, to form a multiplexed packet stream, as described in greater detail above. In particular, rather than encode the streams separately, the device may apply an appropriate encoding strategy across all of the streams, to achieve an encoding rate that is greater than that of encoding the streams separately and with a reduced FEC overhead. For example, the device may use a selected one of the encoding strategies described above, depending on the corresponding region of operation.

At step825, as detailed above, the device may send the stream of packets to one or more nodes in the network. In turn, the receiving nodes may decode and de-multiplex the received packets, attempt to recover any lost information, and provide the recovered media streams to their appropriate user interfaces (e.g., a display, a speaker, etc.). Procedure800then ends at step830.

The techniques described herein, therefore, provide for the dynamic application of different FEC encoding strategies, to jointly encode a plurality of media streams while taking into account their different delay requirements. In particular, the techniques herein enable a media streaming system to achieve the same loss recovery performance and deadlines as that of a disjoint/individual approach, while achieving a lower FEC overhead.

While there have been shown and described illustrative embodiments that provide for the multiplexing of FEC protection of multiple media streams with different delay requirements, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, the embodiments have been shown and described herein with relation to certain network configurations. However, the embodiments in their broader sense are not as limited, and may, in fact, be used with other types of networks. In addition, while certain protocols are shown, other suitable protocols may be used, accordingly.