Patent Description:
Over the top video streaming is becoming ever more popular in comparison to television. While popular, over the top (OTT) video streaming does have several downsides in comparison to broadcast and cable television. Specifically adaptive bitrate streaming, e.g., adaptive bitrate streaming, requires a connection handshaking and playlist collection. Additionally, streaming video applications must download and buffer the video data and its related manifests before the video can be played. Thus, there is typically some delay between when a user chooses a streaming video to watch and when the video is actually played. This delay can be frustrating to users especially during times of high network congestion or slow network condition.

In addition, it would be desirable to implement a single solution to reduce video start-time across multiple devices, such as PS4, Roku, Apple-TV, Android Mobile, and the like.

However, an impediment to such a solution arises from the fact that all HTTP requests need to go through connection, setup, and handshaking operations. HTTP generally is stateless therefore does not always remember or leverage information from previous requests. The major drawback is that servers are not optimized for many adaptive bitrate streaming protocols because they do not take advantage of mostly predicable request patterns. It is also difficult or infeasible for all playback clients to implement all networking optimizations when interacting with the servers.

It is within this context that embodiments of the present disclosure arise.

Previously proposed arrangements are disclosed in <CIT>, <CIT> Al, and <CIT>. In this regard, <CIT> discloses a client device that obtains a manifest file and an edge server which pre-fetches media segments.

The aspects of the present disclosure can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:.

Although the following detailed description contains many specific details for the purposes of illustration, anyone of ordinary skill in the art will appreciate that many variations and alterations to the following details are within the scope of the disclosure. Accordingly, examples of embodiments of the disclosure described below are set forth without any loss of generality to, and without imposing limitations upon, the claimed disclosure.

While numerous specific details are set forth in order to provide a thorough understanding of embodiments of the disclosure, it will be understood by those skilled in the art that other embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components and circuits have not been described in detail so as not to obscure the present disclosure. Some portions of the description herein are presented in terms of algorithms and symbolic representations of operations on data bits or binary digital signals within a computer memory. These algorithmic descriptions and representations may be the techniques used by those skilled in the data processing arts to convey the substance of their work to others skilled in the art.

An algorithm, as used herein, is a self-consistent sequence of actions or operations leading to a desired result. These include physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

Unless specifically stated or otherwise as apparent from the following discussion, it is to be appreciated that throughout the description, discussions utilizing terms such as "processing", "computing", "converting", "reconciling", "determining" or "identifying," refer to the actions and processes of a computer platform which is an electronic computing device that includes a processor which manipulates and transforms data represented as physical (e.g., electronic) quantities within the processor's registers and accessible platform memories into other data similarly represented as physical quantities within the computer platform memories, processor registers, or display screen.

A computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks (e.g., compact disc read only memory (CD-ROMs), digital video discs (DVDs), Blu-Ray Discs™, etc.), and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, flash memories, or any other type of non-transitory media suitable for storing electronic instructions.

Aspects of the present disclosure address the drawbacks associated with existing adaptive bitrate streaming systems through use of a reductive edger to address parity performance issues in a cross-platform client environment. Video is consumed on variety of client devices that have relatively equal capability when it comes to handling adaptive bitrate streaming.

The problem, referred to herein as the "edging" problem is how to make all these cross-platform clients equally capable at accessing contents hosted by a content delivery network (CDN). It is not likely nor reasonable to assume these devices can all implement all advanced edging techniques. Reductive Edging architecture according to aspects of the present disclosure provides edge intelligence in a component external to client devices or client applications so the client can capably use advanced edging technology without having to radically reconfigure a client device or application.

A related issues is one of "playback robustness", i.e., the ability of a client device or application to handle adaptive bitrate stream efficiently. This is non-trivial, especially in dealing with fringe syntax and features, latency, errors, adaptive bit rate, etc. This is even more complex than the edging issue.

Reductive Edging techniques, as described herein, relieve client devices and applications of edging and robustness responsibilities allows them to perform well at simpler tasks. A Reductive Edger that takes over these responsibilities is the salient feature of aspects of the present disclosure.

<FIG> shows the general architecture of an adaptive bit rate (ABR) multimedia streaming system <NUM>. Examples of common protocols for ABR streaming multimedia on personal computers, set-top television devices, phones, and tablets include HTTP Live streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH). The multimedia streaming system <NUM> comprises a media framework <NUM>, a content distribution network and server <NUM>, Key server <NUM>, and client devices <NUM> connected over a network <NUM>.

The content distribution network (CDN) <NUM> comprises multiple servers and data servers that are configured to provide efficient multi-media content to geographically dispersed client devices <NUM> over the network <NUM>. The Key server <NUM> provides encryption keys to the client devices <NUM>. The encryption keys are used to decrypt the media content from the CDN <NUM>, which may be, encrypted for digital rights management (DRM) purposes. The Media Framework <NUM> provides the network <NUM> location of the video content delivered by the CDN <NUM> through a Uniform Resource Locator (URL) or the like.

<FIG> shows the operation of a prior art client device in the HLS system <NUM>. Before media can be played on the client device <NUM>, the client device <NUM> sends a request to the media framework <NUM> for the location of a media stream on the CDN <NUM>. The media framework <NUM> sends the location of the CDN <NUM> back to the client device <NUM>. This location may be, for example and without the limitation, in the form of a URL, internet protocol address (IP address) or similar.

The client device <NUM> then contacts the CDN <NUM> with a request for the Master playlist. The Master Playlist provides a list containing the locations of the media presentation at different bitrates, in different formats or with alternate renditions of the content. The CDN <NUM> then responds by sending the Master Playlist back to the client device <NUM>.

The client device may then choose a particular rendition of a media presentation. This may be done via the client's ABR mechanism to computationally and dynamically determine the optimal bit rate selection and corresponding rendition. In response to a choice of rendition the client device <NUM> sends a request to the CDN <NUM> for a media playlist, The media playlist provides a list containing the location of the keys and video segments of the media presentation at a particular format, bit-rate, etc. The CDN <NUM> sends the media playlist and it is received at the client device <NUM>.

In preparation for media playback of an encrypted stream, the client device <NUM> may contact a key server <NUM> by sending a request for encryption key specific to the media presentation video segments. The key server <NUM> then sends the requested key, which is received by the client device <NUM>. This procedure is optional as not all streams are encrypted, and with other forms of security there may not be keys.

The client device begins streaming the media presentation by sending a request for a media segment to the CDN <NUM>. The CDN <NUM> in kind begins sending media segments, which are received at the client device <NUM>. The media segments are encoded audio, closed captioning data, meta and control data and/or video segments of a media presentation that when played together in sequence creates the full-length media presentation. The media segments may be encoded, in which case the client device may use the key received from the key server to decrypt the media segment and begin playback. For more information on the HLS system architecture see: <NPL>).

<FIG> shows a HLS system enhanced with Reductive Edging according to aspects of the present disclosure. The user experience for HLS based streaming may be enhanced with faster device startup time and reduced buffering times using a Reductive Edging device or embedded devices <NUM>. <NUM>% faster buffering times and <NUM>% faster start up times have been achieved using embodiments of the present disclosure.

Reductive Edging may be performed by client devices <NUM> having embedded Reductive Edging devices <NUM> or by client devices <NUM> connected to a local Reductive Edging device <NUM> or by a client device in communication with the Reductive Edging device <NUM> over a network <NUM>. Embedded Reductive Edging devices may be created within specialized circuitry of the client device or as a separate specialized module with specialized instructions, processor and memory coupled to the processor of the client device through for example a communication bus. In some implementations, Reductive Edging may be implemented as a software service on a device or its operating system (OS).

The network may be for example and without limitation, the internet, a Local Area Network (LAN) or some other type of Wide Area Network (WAN). The prefetching device may be in communication over a network <NUM> with the media framework <NUM>, the key server <NUM>, the CDN <NUM> or any combination thereof during operation. In some implementation, the prefetching device may communicate through the client device. For example, the media-prefetching device may use a network interface of the client device to communicate with servers over the network <NUM>.

<FIG> depicts the method of operation of the Reductive Edging device in communication with a client device according to aspects of the present disclosure. The Media Prefetching device begins operation before the client device begins communication with the servers hosting HLS data as indicated by the double line <NUM>. In some implementations, the client device may not be powered while the Reductive Edging performs the operation above the double line <NUM>.

The Reductive Edging device <NUM> may begin operation by sending a request to the domain name system (DNS) for the location of the Media Framework Server <NUM>. The DNS sends the location, typically in the form of an IP address, of the Media Framework Server back the to the Reductive Edging device <NUM>, where it is received. In some implementations the Reductive Edging device <NUM> may contact a trusted source and download a secure socket layer (SSL) certificate for encrypted communication. This trusted source may be part of the media framework <NUM>. The SSL certificate is used during communication with SSL enabled servers to encrypt communication. This encryption ensures that communications made by the server to the client and vice versa, intercepted by third parties will be unintelligible.

The Reductive Edging device <NUM> may then send a request to the Media Framework <NUM> for the location of the CDN. The Media Framework <NUM> may return the location of the CDN to the Reductive Edging device <NUM>. The CDN location is then stored and used for subsequent communications. The Reductive Edging device <NUM> sends a communication through the network to the CDN <NUM> requesting the Master Playlists. In response, the CDN <NUM> sends the Master Playlists through the network where it is received by the Reductive Edging device <NUM>. The Reductive Edging device <NUM> then stores the Master Playlist.

As discussed above the Master Playlist contains a list of locations of a media presentation at different bitrates, in different formats or with alternate versions of the content. The number of different media presentations on the CDN <NUM> is finite. Therefore to accelerate the streaming process, in some implementations the Media Pre-fetcher may request a Master Playlist comprising every Media Presentation on the CDN <NUM>. The Master playlists are then stored in memory or mass storage. In some implementations, the Reductive Edging device <NUM> may predictively request Master Playlists to reduce download time and memory space required for the Master Playlists. For Example and without limitation the Reductive Edging device may track the user's viewing habits in a table so that the device has a record of what piece of content the user last viewed. The device may download the Master list for that content before the user request to view the content or even starts the client device. Similarly, for games the system may track games the user frequently plays and download a master list for if playtime or number of sessions for the game is above a certain threshold. Note that while this implementation is described in the context of games it may also be applied to other media types such as video or music.

Once the Master Playlist is received at the Reductive Edging Device <NUM>, a request may be sent to the CDN <NUM> for Media Playlists by the Reductive Edging device <NUM>. Similar to the Master Playlist, there are a finite number of Media presentations on the CDN <NUM>. Therefore, the Reductive Edging device may request a media playlist for every media presentation on the CDN <NUM>. Predictive methods may also be used to reduce the number of content playlists downloaded. The Predictions methods for prospective fetching of content lists are substantially similar to those described above with respect to the master playlist. With the caveat that the tracking of user activity for Content Playlist must be more granular because the Content Playlist represents a particular media presentation whereas the Master Playlist represents a set of media presentations having, different bitrates, different formats or alternate versions of the content.

After the CDN <NUM> receives the request for Media Playlists, it sends the Media Playlists through the network to the Reductive Edging device <NUM>. The Media Playlists are received by the Reductive Edging device <NUM> and stored either in memory or in mass storage.

The Reductive Edging device <NUM> may then request keys from the Key Server <NUM> for each media playlist received from the CDN <NUM>. These keys are stored in memory or in mass storage after they are received by the Reductive Edging device <NUM> from the Key server <NUM>. Once the user initiates the streaming system on the client device <NUM> communication between the client device <NUM> and the Reductive Edging device <NUM> may begin. The client device may be configured to communicate with the Reductive Edging device as though it were part of the media framework and in the content delivery network. Thus, the Reductive Edging device <NUM> may receive a request for the CDN location from the client device <NUM>. The Reductive Edging device <NUM> forwards the request to the Media Framework <NUM>. The Media framework <NUM> replies with the CDN location, which is received by the client device <NUM>. Concurrently the Reductive Edging device <NUM> sends a content playlist request to the CDN <NUM>. The CDN <NUM> sends the content playlist and the content playlist is received by the Reductive Edging device <NUM>.

After receipt of the CDN location, the client device <NUM> sends a request for a master playlist from the Reductive Edging device <NUM>. The client device <NUM> may be configured to request Playlists from the Reductive Edging device <NUM> before contacting the CDN <NUM> or instead of the CDN <NUM>. After receiving the request for a master playlist the Reductive Edging device <NUM> sends the requested master playlist to the client device <NUM>. The Reductive Edging device already has every master playlist in memory as the device has received all of the master playlists from the CDN earlier. In implementations, that predictively request master playlists the memory contains master playlists likely to be requested as such there is the possibility that the prediction was incorrect and in such a case the request will be forwarded to the CDN <NUM>.

After receiving the master playlist request, the Reductive Edging device <NUM> also sends a request to the CDN <NUM> for media segments for prospective caching. In some implementations the Reductive Edging device <NUM> may request media segments for every media playlist in the master playlist that was request by the client device <NUM>. In other non-claimed implementations, the Reductive Edging device <NUM> may track the most recently requested media playlist and may request media segments from the same media playlist or a congruent media playlist. By way of example and not by way of limitation the Reductive Edging device may keep a table of the five most recently requested media playlists, media segments downloaded etc. There may be some meta-information about each media playlist for example, the table entry may have information about the order in which media playlists are typically requested and information about the location next media playlist likely to be requested i.e. "Previous Watched: Breaking Bad Episode <NUM> - Next episode: Breaking Bad Episode <NUM> - Location of Next Episode: Location. " The Reductive Edging device may request media segments from media congruent with the meta-information, for example and without limitation the next episode in the series. In other implementations, the table may track user preferences such as Language, and connection speed and request media segments from the Master List that match those preferences.

The CDN <NUM> begins sending media segments to the Reductive Edging device <NUM> after the request for media segments is received. The Reductive Edging device <NUM> receives and stores the media segments from the CDN <NUM>. The media segments may be stored in memory, for example in a buffer or in Mass Storage.

The Client device <NUM> may request a media playlist from the Reductive Edging device <NUM>. In response the Reductive Edging device <NUM> may send the media playlist to the Client device <NUM>. The Reductive Edging device <NUM> already has all of the media playlists for the requested master playlist in memory and therefore only needs to retrieve the requested media playlist from memory.

Next, the client device <NUM> may request a media segment from the Reductive Edging device <NUM>. When the request for a media segment is received at the Reductive Edging device <NUM>, it may query it's memory or mass storage for the media segment. If the media segment is found, the Reductive Edging device sends the segment to the client device <NUM>. If the media segment is not found a request for the media segment is sent to the CDN <NUM>. Upon receiving the request for a media segment, the CDN sends the requested media segment to the Reductive Edging device <NUM>. The Reductive Edging device <NUM> stores the media segment in memory or mass storage after receiving the media segment. The newly stored media segment may then be sent to the client device <NUM>. As discussed above the media segment may be encrypted as part of the DRM protection, therefore as part of the storage process the Reductive Edging device <NUM> decrypts the media segments with the correct key received from the key server. In other implementations, the media segments may be decrypted before being sent to the client device. Once media segments are being sent from the CDN <NUM> and received at the Reductive Edging device <NUM> the Reductive Edging device <NUM>, may store media segments ahead of the client device <NUM> and send them to the client <NUM> device as necessary.

<FIG> shows a standalone Reductive Edging device according to aspects of the present disclosure. The standalone Reductive Edging device or Edger <NUM> may be coupled to a local client device <NUM> through a network interface <NUM> over a LAN or WAN. In other alternative implementation the standalone Reductive Edging device may be in communication through the network interface <NUM> with a non-local device <NUM> e.g., servers or another client, through a large network <NUM> such as the internet. In some implementations the client device is connected to the stand alone Reductive Edging device through a communication bus (not shown) such as, without limitation, a peripheral interconnect (PCI) bus, PCI express bus, Universal Serial Bus (USB), Ethernet port, Fire-wire connector or similar interface.

The standalone Reductive Edging device <NUM> may include one or more processor units <NUM>, which may be configured according to well-known architectures, such as, e.g., single-core, dual-core, quad-core, multi-core, processor-coprocessor, cell processor, and the like. The standalone Reductive Edging device <NUM> may also include one or more memory units <NUM> (e.g., random access memory (RAM), dynamic random access memory (DRAM), read-only memory (ROM), and the like).

The processor unit <NUM> may execute one or more instructions <NUM>, portions of which may be stored in the memory <NUM> and the processor <NUM> may be operatively coupled to the memory through a bus or bus type connection. The instructions <NUM> may be configured to implement the method for prefetching in a HLS systems shown in <FIG>. Additionally the Memory <NUM> may contain instructions for storing Playlists in a HLS Library and a Protocol Stack defining HLS server locations. The Memory <NUM> may also contain the HLS Library <NUM> and the Protocol Stack <NUM>. The instructions <NUM> may further implement storage of media segments in a cache <NUM> during operation. The Cache <NUM> may also be located in memory <NUM>.

The standalone Reductive Edging device <NUM> may include a network interface <NUM> to facilitate communication via an electronic communications network <NUM>. The network interface <NUM> may be configured to implement wired or wireless communication over local area networks and wide area networks such as the Internet. The device <NUM> may send and receive data and/or requests for files via one or more message packets over the network <NUM>. Message packets sent over the network <NUM> may temporarily be stored in a cache <NUM> in memory <NUM>. The client device <NUM> may connect through the network interface <NUM> to the electronic communications network <NUM>. Alternatively, the client device <NUM> may be in communication with the standalone Reductive Edging device <NUM> over the electronic communication network <NUM>.

<FIG> depicts an embedded Reductive Edging system according to aspects of the present disclosure. The embedded Reductive Edging system may include a client computing device <NUM> coupled to a user input device <NUM>. The user input device <NUM> may be a controller, touch screen, microphone, keyboard, mouse, joystick or other device that allows the user to input information including sound data in to the system.

The client device of the embedded Reductive Edging system <NUM> may include one or more processor units <NUM>, which may be configured according to well-known architectures, such as, e.g., single-core, dual-core, quad-core, multi-core, processor-coprocessor, cell processor, and the like. The computing device may also include one or more memory units <NUM> (e.g., random access memory (RAM), dynamic random access memory (DRAM), read-only memory (ROM), and the like).

The processor unit <NUM> may execute one or more programs, portions of which may be stored in the memory <NUM> and the processor <NUM> may be operatively coupled to the memory, e.g., by accessing the memory via a data bus <NUM>. The programs may be configured to implement streaming media through HLS systems <NUM>. Additionally the Memory <NUM> may contain information about connections between the system and one or more streaming servers <NUM>. The Memory <NUM> may also contain a buffer of media segments <NUM>. The Media segments and connection information may also be stored as data <NUM> in the Mass Store <NUM>.

The computing device <NUM> may also include well-known support circuits, such as input/output (I/O) <NUM>, circuits, power supplies (P/S) <NUM>, a clock (CLK) <NUM>, and cache <NUM>, which may communicate with other components of the system, e.g., via the bus <NUM>. The computing device may include a network interface <NUM>. The processor unit <NUM> and network interface <NUM> may be configured to implement a local area network (LAN) or personal area network (PAN), via a suitable network protocol, e.g., Bluetooth, for a PAN. The computing device may optionally include a mass storage device <NUM> such as a disk drive, CD-ROM drive, tape drive, flash memory, or the like, and the mass storage device may store programs and/or data. The computing device may also include a user interface <NUM> to facilitate interaction between the system and a user. The user interface may include a monitor, Television screen, speakers, headphones or other devices that communicate information to the user.

The computing device <NUM> may include a network interface <NUM> to facilitate communication via an electronic communications network <NUM>. The network interface <NUM> may be configured to implement wired or wireless communication over local area networks and wide area networks such as the Internet. The device <NUM> may send and receive data and/or requests for files via one or more message packets over the network <NUM>. Message packets sent over the network <NUM> may temporarily be stored in a buffer <NUM> in memory <NUM>.

In some implementations, the embedded Reductive Edging or embedded Edger <NUM> may be an embedded hardware component of client device <NUM>, which may be coupled to the main processor via the bus and requests may be received from applications, e.g., streaming applications, running on the client device. In some implementations, the embedded Edger <NUM> may initiate and intercept network communications directed toward a CDN or other servers. In these implementations, the embedded Edger <NUM> may lack a network interface or the network interface may not be used. In other implementations, the embedded Edger, the functions of the edger may be implemented in streaming software <NUM> stored in the memory <NUM> or in programs <NUM> stored in the mass store <NUM> and executed on the processor <NUM>.

In some alternative implementation the embedded Edger <NUM> may be an external device coupled to the client device <NUM>, e.g., via a local non-network connection, such as the I/O functions <NUM>.

The processor of the embedded Edger unit <NUM> may execute one or more instructions <NUM>, portions of which may be stored in the edger memory <NUM> and the processor <NUM> may be operatively coupled to the memory <NUM> through a bus or bus type connection. The instructions <NUM> may be configured to implement the method for prefetching in a HLS systems shown in <FIG>. Additionally the Memory <NUM> may contain instructions for storing Playlists in a HLS Library and a Protocol Stack defining HLS server locations. The Memory <NUM> may also contain the HLS Library <NUM> and the Protocol Stack <NUM>. The instructions <NUM> may further implement storage of media segments as data <NUM> during operation. Alternatively the HLS Library, Protocol stack and media segments may be stored on the client device <NUM> in the buffer <NUM> or as connection information <NUM> in memory <NUM> or as data <NUM> in the Mass Store <NUM>.

While the above is a complete description of the preferred embodiment of the present disclosure, it is possible to use various alternatives and modifications.

It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, while the flow diagrams in the figures show a particular order of operations performed by certain embodiments of the disclosure, it should be understood that such order is not required (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.). Furthermore, many other embodiments will be apparent to those of skill in the art upon reading and understanding the above description. Although the present disclosure has been described with reference to specific exemplary embodiments, it will be recognized that the disclosure is not limited to the embodiments described, but can be practiced with modification and alteration.

The scope of the disclosure should therefore be determined with reference to the appended claims.

Claim 1:
A device comprising;
a processor (<NUM>);
memory (<NUM>) coupled to the processor;
non-transitory instructions embedded in the memory that when executed by the processor cause the device to perform a method, the method comprising;
prior to receiving a request from a client device (<NUM>) for information from a content distribution network, CDN, (<NUM>) requesting the information from the CDN and storing the information in memory, wherein the information from the CDN is a master playlist or a media playlist;
prior to receiving the request from the client device, requesting and receiving a media encryption key from a key server for each media playlist received from the CDN and storing the media encryption key of each media playlist received in the memory; and
requesting, after receiving from the client device a master playlist request, and receiving a media segment from the CDN and storing the media segment in memory, wherein media segments for each media playlist in the master playlist that was requested by the client device are requested,
whereby the device is configured to respond to the request from the client device for information from the CDN with the information stored in the memory prior to receiving the request, and
whereby the device is configured to decrypt the media segments corresponding to each media playlist using the corresponding received media encryption key,
wherein the master playlist provides a list containing the locations of a media presentation at different bitrates and/or in different formats, and
wherein the media playlist provides a list containing the location of the media encryption keys and media segments of a media presentation at a particular format and/or bitrate.