Dynamic video delivery for in-home applications

Methods, systems, and computer readable media may be operable to provide dynamic transrating of video content within a DLNA network in the home. A gateway or other DLNA Digital Media Source providing a video stream in response to a user request from a Digital Media Player may determine that the video stream delivery may be close to failure based upon the behavior of the video server's transmit buffer. The gateway may direct an internal transrating resource to provide a lower bit rate content stream then advertise that new stream to the player.

TECHNICAL FIELD

This disclosure relates to dynamically transcoding video streams based on local conditions for in-home DLNA clients.

BACKGROUND

In response to consumer requests to be able to access video content seamlessly from a variety of different devices, the FCC mandated that operators provide a digital interface with an open format to allow consumers to access video content over their home networks. The Digital Living Network Alliance (DLNA) publishes a set of specifications that can be used to comply with the FCC's mandate. The DLNA specifications provide a means for consumer devices to identify interesting content, and request it over a home network. DLNA specifications define two basic types of devices, Digital Media Players (DMPs) and Digital Media Servers (DMSs). A DMP can discover and request content from a DMS. An example DMP is a laptop or mobile device. An example DMS is a set top box or media gateway.

In operation, a DMS publishes periodically a DLNA content directory though the DLNA Content Directory Service (CDS). Within a content directory is information about the programs available and their respective Universal Resource Indicators (URIs) that a DMP may use to retrieve content. From the directory, a DMP may request a specific piece of content by accessing the URI given in the directory. DLNA specifications do not provide for dynamic interactions between a DMS and a DMP. The content within a DLNA CDS is presented as is; a DMP cannot request a new item that is not currently on the CDS and also cannot request an alternate codec or bitrate. A DMP can only choose from the options provided by the CDS.

Home networks, where DLNA is typically used, are typically a combination of different technologies. Ethernet wired links may be combined with MoCA or G.hn wired links as well as with wireless technologies, such as Wi-Fi. The resulting network does not typically provide any guaranteed throughput or quality of service. Indeed, home networks often provide inconsistent bit rates over time in a connection between any two devices. While this is not a problem for many services, video services may be seriously affected if the bit rate available between two devices, such as a DMS and a DMP, drops below a given level. The end user may experience a frozen picture or macroblocking when the home network cannot accommodate the bit rate of the content streaming across it. Within the current DLNA architecture, no mechanisms are provided to detect or remedy video streaming issues due to home networking problems.

Therefore, a need exists for improved methods and systems for dynamic video delivery in home networks that can adjust to changing conditions within the home network.

SUMMARY OF INVENTION

Methods and apparatus to provide dynamic distribution of video to in-home clients are presented including: advertising a content to a Digital Media Player (DMP); receiving a request for the content from a DMP, the content having original content attributes associated with the content as provided to the home; providing requested content to the DMP through a transmit buffer associated with a Digital Media Server (DMS); monitoring the DMS's associated transmit buffer behavior; comparing the transmit buffer behavior to a threshold; and if the transmit buffer behavior meets the threshold, modifying the content to have new content attributes; and advertising the modified content to the DMP. The threshold mentioned may include a certain upper transmit buffer depth being present for a certain length of time. Modifying the content may include transrating the content to a bit rate lower than the bit rate of the original content attributes. The operation of transrating the content may also include transcoding the content. The transrated content may be advertised with a new Uniform Resource Indicator (URI) or with a Program Map Table (PMT) version update on the streamed content. In accordance with the new bit rate of the requested content stream a new threshold may be provided and the bit rate of the content stream may be adjusted further if the new threshold is met.

An apparatus is also disclosed that includes an interface configured to communicate with a DMP, wherein the communication includes at least Digital Living Network Alliance (DLNA) control messages and a content stream, an interface configured to receive content streams, one or more modules configured to transrate content streams, one or more modules configured to provide content streams in response to DMP requests, and one or more modules configured to control content distribution activity, The control module may compare the one or more attributes relating to transmit buffer behavior to a threshold, direct a transrating module to create a first transrated content stream if the one or more attributes relating to transmit buffer behavior meets the threshold; and advertise the first transrated content stream to the DMP. The interface used to communicate with a DMP may be Ethernet, IEEE 802.11, MoCA, G.hn, or Powerline.

Alternately, one or more non-transitory computer readable media may have instructions operable to cause one or more processors to advertise content in a Content Directory Service (CDS) to a DMP, receive a request for a content from a DMP, provide the requested content to the DMP from a DMS, monitor transmit buffer behavior in the DMS, compare transmit buffer behavior to a threshold; and if the transmit buffer behavior meets the threshold, transrate the requested content to a first transrated bit rate; then advertise the transrated content to the DLNA sink device.

The methods and apparatus described provide advantages to the in-home distribution of multimedia content over DLNA. A DMS without this invention can only offer content with the attributes provided by the broadband network. If a DMP requests content from a DMS but later has difficulties retrieving that content over the home network, the DLNA session may have intermittent problems or fail outright. With this invention, the bit rate may be smoothly adjusted to enable the session to continue with some decrease in video quality. Detecting when home network conditions have improved also allows the quality of the streaming session to improve without the user having to take action.

DETAILED DESCRIPTION

It is desirable to improve upon methods and systems for dynamically transcoding video streams based on local conditions for the delivery of multimedia content to in-home DMPs using DLNA.

FIG. 1is a block diagram illustrating an example network100operable to dynamically transcode video streams based on local conditions for in-home DMPs. In embodiments, one or more customer premise equipment (CPE) devices (e.g., gateway110, DMP101, and other devices that are not shown) may operate together providing video and/or data services to a subscriber. A gateway110may include a connection to a content network120. In embodiments, the connection to a content network could involve a broadband network, such as Fiber to the Premise (FTTP), Digital Subscriber Line (DSL) or Hybrid Fiber Coax (HFC), or a satellite connection, or other content network connections as are well known in the art. In embodiments, the content provided by content network120may be in a native IP fragmented format or it may be provided as an MPEG transport stream.

Within gateway110, a DMS/controller116may provide DLNA content discovery and distribution services to the home network. DMS116may provide a CDS that specifies URIs for a DMP to access certain video content. The behavior of a DMS116is generally well known and is documented in the CVP-2 family of standards from the Digital Living Network Alliance. Within DMS116is a transmit buffer118. The process of transmitting content over home network105is not instantaneous. Content requested from DMS116by DMP101will reside in transmit buffer118until it can be sent to the requester.

Within gateway110, a transcoder/transrater114may be provided. It is well known to those skilled in the art that a transcoder114may provide transcoding services that transform a given piece of video content from one content format to another (e.g. from one encoded format to a second encoded format). The goal of a transcoding operation may be to provide content in a format acceptable to a specific type of device from a format that is generally provided by the content network120. For example, a piece of content may be distributed in MPEG4 format, but may be transcoded to MPEG2 for a device that does not support MPEG4. A transcoding operation may also be performed to reduce the bit rate of a piece of content. For example, a piece of content in MPEG4 is typically smaller than a piece of content in MPEG2. A transcoder may also perform transrating operations, wherein a piece of content is kept in the same format, but its bit rate is reduced, typically at the cost of reduced picture resolution or quality.

Within gateway110, a video distribution monitor112may support distribution of DLNA content. In embodiments, monitor112may communicate with DMS116to monitor the behavior of DLNA content distribution sessions. In embodiments, monitor112may also communicate with transcoder114to direct its operations.

While this example provides that functions112,114, and166are integrated within a gateway110, one skilled in the art would understand that other divisions of functionality are possible. For example, transcoder114may be implemented in a separate device in the home alongside gateway110. Alternatively, transcoder114might exist in content network120, sometimes described as transcoding or transrating functionality being provided from the cloud.

A gateway110may communicate with a DMP101over a home network105. In embodiments, home network103may include one or more networking technologies. The networking technologies may include any combinations of wired and wireless protocols, as mentioned earlier.

DMP101may retrieve a content directory from DMS116. Based on the content directory, a user of DMP101may select a piece of content for display. DMP101may begin retrieving sections of content through home network105from DMS116. In embodiments, monitor112may monitor the behavior of transmit buffer118on DMS116to ensure that DMP101is retrieving content at a rate consistent with acceptable video performance.

FIG. 2is a flow chart200illustrating an example process operable to provide dynamic video streams based on local conditions for an in-home DMP101.

In201, a content directory is published to a DMP101by DMS116. In embodiments, the publishing process may conform to DLNA CVP-2.

In202, DMP101requests a specific piece of content. In embodiments, the player may request content from the DMS116using the URI address provided in the CDS from that server. A DMP typically requests content based upon an end user's selection, but may also request content based on other sources of direction, such as a DVR scheduler.

In203, the content transmission process begins. In embodiments, the content served by DLNA is typically stored as fragmented MPEG2 or MPEG4 segments. Each segment may contain 2 to 10 seconds of content with an overall file size ranging from 1 Mb for 500 kb/s content in a 2 second fragment to 150 Mb for 15 Mb/s content in a 10 second fragment. When DMP101retrieves the content from a DMS116, DMP101performs a series of HTTP GET commands, pulling each fragment across in a file transfer exchange.

In204, the behavior of DMS116's transmit buffer118is monitored as DMP101periodically requests content. In embodiments, each HTTP GET command fills transmit buffer118, and buffer118drains at a rate set by the bandwidth available on the home network between the DMS116and DMP101. The link between DMS116and DMP101over the home network may vary in available bandwidth due to many factors such as variations in wireless transmission characteristics or the presence of other service flows on the network competing for bandwidth.FIG. 4, to be discussed later, contains an example of buffer behavior.

In embodiments, monitor112might periodically compare the amount of data in buffer118to a threshold set with reference to the bit rate of the content to ensure that data is not building up in buffer118. For example, if the content represented 2 second fragments, monitor112could check the buffer depth every 2 seconds. If DMP101could not retrieve fragments over the home network105fast enough to keep up with the stream's bit rate, then the amount of data in buffer118will increase. In other embodiments, monitor112's evaluation might compare a rate of change of the amount of data in buffer118to one or more thresholds, to ensure that buffer118is being emptied at least fast enough to keep up with the known data rate of the requested stream. In other embodiments, the two previous methods could be combined, so that the buffer fill level was compared against a threshold that decreased with time at a rate proportional to the desired bit rate of the video stream.

In205, as long as the transmit buffer behavior is within normal limits, the process will continue monitoring transmit buffer118without taking other action.

In206, if the transmit buffer behavior falls outside of the desired conditions expected by monitor112, an alert may be generated. In embodiments, monitor112may generate an alert if the fill level in the buffer increases past a threshold level, where the threshold level may be set by the type of video stream or may be related to the type of DMP. In other embodiments, the buffer fill level that may generate an alert may be relative to the rate of decrease of the data within buffer118. A slowing rate of decrease may indicate that the link between DMP and DMS is becoming congested. If the rate of decrease becomes slower than the playout rate of the video, the DMP's decode engine will fail for lack of data eventually. In embodiments, the behavior thresholds may be set to proactively detect issues before they would affect video playout. If the jitter buffer in a DMP holds 2-3 fragments, a monitor may assume that it may take action to move the DMP to a new stream before video problems are seen by the end user.

In207, after detecting that the stream may be running at too high of a bit rate for the current network conditions, a determination is made as to whether it is possible to reduce the bit rate of the stream. In embodiments, the decision may be affected by many factors. If there is no available transcoding or transrating resource114, then the content cannot be adjusted to reflect local network conditions. The transcoding or transrating resource114may already be engaged with other content, or the current encoding of the stream may not be compatible with the transcoding or transrating resources available. In embodiments, there may be a lower bound for the bit rate, which is well known to relate to overall video quality. Some content providers may specify that their content may not be reduced in bit rate below a certain bit rate to preserve their artistic vision, for example. Monitor112may have instructions or rules controlling whether or not action can be taken on any particular content. In any case where the bit rate of the stream cannot be decreased, the process returns to a monitoring state in204. If the bit rate of the stream can be decreased, then the process continues to208.

In208, if the bit rate of the stream currently exhibiting transmit buffer behavior issues can be decreased, a resource114is allocated to produce a lower bit rate stream. In embodiments, the lower bit rate stream may be produced by a transcoding operation in which the new codec has a lower bit rate. In other embodiments, the lower bit rate stream may be produced by a transrating operation. If the monitor112has information about what codecs are acceptable to DMP101, then transcoding and transrating operations may be considered. If monitor112does not have information about DMP101's capabilities, then monitor112may only direct the content to be transrated.

In209, the new lower bit rate content stream is advertised to DMP101by DMS116through a new entry in the CDS. In embodiments, the old content entry may be removed from the CDS to encourage DMP101to migrate to the new stream.

Alternatively, in another embodiment, the Program Map Table (“PMT”) version of the transport stream may be updated indicating the new change in the program components. This change would alert DMP101to update its decoder and play the new stream without having to recognize and request a new CDS item.

In210, DMP101begins using the new stream. In embodiments, transmit buffer118may be purged of older fragments that were still in the buffer, allowing the buffer to be reset. In other embodiments, the new fragments may be added to the older higher bit rate fragments already in buffer118. In either case, the process returns to204, monitoring the transmit buffer behavior for this stream.

FIG. 3is a flowchart illustrating an example process300operable to extend process200described inFIG. 2. Process300may begin at204continuing from process200as discussed above. In204, the current state of the transmit buffer is monitored as previously discussed.

In301, an underflow of transmit buffer118is detected. In embodiments, the underflow might be detected when the amount of video in buffer118goes below a certain level for a certain period of time. For example, if DMP101is consuming 2 second fragments from buffer118in substantially less than 1 second, a higher bit rate stream could be supported to DMP101over the home network. Since a video stream's bit rate generally reflects the quality of the video displayed, detecting when a stream's bit rate is lower than what the network could optimally provide offers the ability to provide the end user with a better video experience if a higher bit rate alternative stream can be produced.

In302, the process may continue with a determination whether it is possible to increase the bit rate of the stream. In embodiments, the possibilities for increasing the bit rate of a stream may be influenced by whether the current stream is the product of a transcoding or transrating operation. If the stream rate cannot be increased then the process returns to204. If the stream rate can be increased then the process continues to303.

In303, a higher bit rate stream is produced. In embodiments, the higher bit rate stream may be produced by changing the transcoding or transrating configuration choices. In other embodiments, DMS116may return to the original source stream to provide a higher bit rate stream.

In304, the higher bit rate stream is advertised to DMP101. In embodiments, the new stream may be presented through a change to the CDS by DMS116.

In305, DMP101may migrate to the higher bit rate stream. After DMP101has migrated to the new stream, the process may return to204to monitor transmit buffer behavior. In embodiments, if DMP101has migrated to a higher bit rate stream, then the monitor thresholds may also be adjusted to account for the new bit rate.

FIG. 4is a graph illustrating an example400of transmit buffer118behavior within DMS116. Example400illustrates potential transmit buffer utilization or buffer depth from time T=1 to time T=8 with line401. Lines410and420represent buffer thresholds. It is important to note that video delivery systems typically request multiple fragments, and usually request the fragments before they are actually required to allow time for their delivery.

This example begins at T=1, with DMP101requesting 2 fragments from DMS116. At T=1, the arrival of 2 video fragments into buffer118is shown. Line401jumps up at T=1 once the fragments are loaded into buffer118. Buffer118begins sending video toward DMP101and so the line slopes down as the buffer empties toward T=2. At T=2, DMP101requests another fragment to ensure timely delivery. Again, line401jumps up as the new fragment is added to buffer118and slowly drains toward T=3. At T=3, another fragment is added, but as home networking conditions between DMS116and DMP101begin to worsen the data remains in buffer118longer as shown by the slower rate of decrease in the buffer depth shown in line410between T=3 and T=4.

After the new fragments were added at T=4 and T=5, the amount of data in transmit buffer118has increased past line410due to continuing poor home networking conditions. The problems the server has supplying data to the DMP are reflected in the slower rate of decrease of data in the buffer. In embodiments, line410may have been set considering the bit rate of the video currently streaming or it may have been set by use of a default value.

At T=6, line415indicates the time of an example shift to a new lower bit rate video stream. As mentioned earlier, in embodiments, when moving to a new stream, transmit buffer118might be cleared or the fragments that were already loaded for delivery might be left in queue. In this example, line401at T=6 shows the transmit buffer cleared, then loaded with fragments from a new stream. A new threshold shown by line420is also shown. In embodiments, as mentioned earlier, a new threshold may be set to account for the lower bit rate feeding the transmit buffer. In other embodiments, the threshold level may remain unchanged.

The stream used after T=6 has a lower bit rate, as shown by the smaller peaks after T=6. The playout rate shown by the sloping line from T=6 to T=7 still has a shallower slope than the original line segments, indicating that the causes of the network delivery issues are still present.

FIG. 5is a message flow diagram500illustrating an example process operable to dynamically transcode video streams based on local conditions for in-home DLNA clients.

Message501from DMS116is sent to DMP101advertising the content available from DMS116. In embodiments, the message may advertise the CDS available from DMS116.

Message502is sent from DMP101to DMS116to request the content. In embodiments, the DMS116may host the URI used for the DMP101's request. In other embodiments, different servers might provide the CDS and host the URIs. In embodiments, message502may be an HTTP GET message that requests a single fragment file.

In response to message502, DMS116provides the requested content to DMP101through a series of transactions collectively labelled503. For at least each fragment, the DMP101sends a request to the DMS116, and the DMS116replies with the requested content.

DMS116alerts monitor112that a streaming session is active with message504. In embodiments, message504may include the requesting device and the specific piece of content chosen, among other information.

Monitor112sends a query message505to check the behavior of the transmit buffer118in DMS116. In embodiments, monitor112may send a query message like message505at periodic intervals. The intervals may be set by a variety of factors including but not limited to the bit rate of the stream, the number of simultaneous sessions monitor112is tracking, and knowledge of the network connections used by DMP101.

DMS116sends a message506back to the monitor task in response to message505. Message506may comprise information about the current state of transmit buffer118in DMS116. In embodiments, DMS116may send further responses periodically to monitor112, or it may wait and respond to further queries from monitor112.

In this example, DMS116receives another query message507from monitor112checking on transmit buffer118's behavior. DMS116replies in message508with an update on buffer118's status. In embodiments, the message508may contain an actual byte count of information in buffer118for monitor112to evaluate or it may indicate that buffer118's behavior has met a certain threshold. In other embodiments, the message may indicate a rate of change for buffer118. In other embodiments, DMS116may process the buffer status information to indicate to monitor112that a threshold has been crossed.

After receiving message508from DMS116, monitor112determines that a lower bit rate video stream is needed. Monitor112sends message509to transcoder114to request a reduced bit rate stream for DMP101to be delivered to DMS116. In embodiments, monitor112may instruct transcoder114to produce a stream with a specific bit rate, or it may select a bit rate from a menu of transcoder options.

With message510, transcoder114sends the new lower bit rate stream to DMS116. While this transaction is shown as a single message, embodiments of this transaction may incorporate multiple transactions. These transactions are simplified here since video file transfers are well known in the art.

Once DMS116has access to the lower bit rate stream from transcoder114, DMS116may advertise it to DMP101in message511. DMP101may request the new lower bit rate stream with message512. The server may again notify the monitor112of the new stream playout to DMP101with message513, and begin delivering the new stream to DMP101with message514.

FIG. 6is a block diagram of a hardware configuration600operable to dynamically transcode video streams based on local conditions for in-home DLNA clients. The hardware configuration600may include a processor610, a memory620, a transcoder unit630, and an input/output device640. Each of the components610,620,630, and640may, for example, be interconnected using a system bus650. Processor610may be capable of processing instructions for execution within hardware configuration600. In one implementation, processor610may be a single-threaded processor. In another implementation, processor610may be a multi-threaded processor. Processor610may be capable of processing instructions stored in memory620.

Memory620may store information within hardware configuration600. In one implementation, memory620may be a computer-readable medium. In one implementation, memory620may be a volatile memory unit. In another implementation, memory620may be a non-volatile memory unit. In yet another implementation, memory620could be a combination of both volatile and non-volatile memories.

In some implementations, transcoder unit630may be capable of providing video and audio transcoding for hardware configuration600. In one implementation, the transcoder unit may be capable of providing transrating capabilities for hardware configuration600as well as transcoding. In various different implementations, transcoder unit630may, for example, comprise a single chip transcoder implementation. In other implementations, transcoder unit630may be a multi-chip module. In still other implementations, transcoder unit630may be integrated within processor610.

Input/output device640provides input/output operations for hardware configuration600. In one implementation, input/output device640may include one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-232 port), one or more universal serial bus (USB) interfaces (e.g., a USB 2.0 port), one or more wireless interface devices (e.g., an 802.11 interface), and/or one or more interfaces for outputting video and/or data services to a CPE device (e.g., DMP101ofFIG. 1, cable set top box, tablet, wireless extender, or other client device). In another implementation, input/output device640may include driver devices configured to send communications to, and receive communications from one or more networks (e.g., home network105ofFIG. 1, content network120ofFIG. 1, etc.).

Those skilled in the art will appreciate that the invention described herein improves upon methods and systems for providing reliable video to a DMP. The subject matter of this disclosure, and components thereof, may be realized by instructions that upon execution cause one or more processing devices to carry out the processes and functions described above. Such instructions may, for example, comprise interpreted instructions, such as script instructions, e.g., JavaScript or ECMAScript instructions, or executable code, or other instructions stored in a computer readable medium.

Particular embodiments of the subject matter described in this specification have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve desirable results, unless expressly noted otherwise. As one example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.