Patent Publication Number: US-10771824-B1

Title: System for managing video playback using a server generated manifest/playlist

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application claims priority to U.S. Provisional Patent Application No. 62/334,157, entitled “Method and Apparatus for Managing Video Playback Using a Server Generated Manifest/Playlist,” filed on May 10, 2016, which is herein incorporated by reference. 
    
    
     BACKGROUND 
     The subject matter disclosed herein relates to a method for managing video playback for users during video streaming. More specifically, a method and apparatus for using a server to generate per-user manifest file updates for controlling video playback is disclosed. 
     Video streaming allows video content to be delivered to a video player via the internet. The video content is a video signal generated by a content provider for distribution to video consumers. The video signal may be provided in an uncompressed file format, such as a serial digital interface (SDI) format, or in a compressed format, such as a Moving Picture Experts Group (MPEG) file format or Transport Stream (TS) file format. The video signal is sent to an encoder which converts the file into a live streaming signal. The live streaming signal is preferably a segmented data stream that may be transmitted using standard Hypertext Transport Protocol (HTTP) over the internet. The live streaming signal may include multiple streams, where each stream may have a different data rate and/or different resolution. 
     Two common formats for the live streaming signal include HTTP Live Streaming (HLS) implemented by Apple® and MPEG-Dynamic Adaptive bitrate Streaming over HTTP (MPEG-DASH) implemented, for example, by web browsers such as Chrome®, Microsoft Edge®, and Firefox®. In addition to the segmented data stream, the encoder generates a manifest file. The manifest file contains information for a video player to play the segmented data stream such as the data rate and resolution of each stream and a playlist providing an address from which the video content may be retrieved. Historically, the encoder has generated a single manifest file for each encoded video signal, where the manifest file is distributed along with the streaming signal. 
     The live streaming signal and the manifest file are stored in one or more Content Delivery Networks (CDN). Each CDN includes a number of edge servers which store the streaming signal and manifest file until requested by a video player. When the streaming signal is provided to multiple CDNs, the CDNs may be in different geographic locations, such as the West Coast, East Coast, or Midwest. Each video player may select a CDN based, among other things, on its geographic proximity in order to reduce transmission latencies. 
     The video player may be any suitable electronic device to receive the streaming signal such as a desktop computer, a television, a laptop computer, a tablet, or a mobile phone. A user initiates a request to view desired video content on the video player. The video player includes video management software executing on the video player which has knowledge of the address of the CDN and which may provide a list of video content stored on the CDN to the user. After the user has selected a desired video, the video player, in turn, requests that the video content be transmitted from the CDN. 
     As is further known to those skilled in the art, it is often desirable to enforce logic rules for distribution of video content which may impact a user&#39;s experience. For example, it may be desirable to allow a user to view requested video content for a predetermined amount of time, such as 120 seconds, before then being required to view advertising or other alternative content. Also, it may be desirable to inhibit a user&#39;s ability to skip such alternative content if the user has already viewed a predetermined amount of the requested video content, or conversely, allow the user to skip the alternative content if the user has not viewed the predetermined amount of requested video content. Also, it may be desirable to allow a user to skip the alternative content if the user is seeking to go back and replay a previously viewed portion of the requested video content. 
     However, these logic rules for distribution of video content can be complex to implement and enforce, particularly given the variety of possible configurations that exist for video players. As a result, there is a need to improve the implementation of such logic rules across the widest variety of video players possible. 
     BRIEF DESCRIPTION OF THE DISCLOSURE 
     The subject matter disclosed herein describes an apparatus and method for managing video playback for users during video streaming. In one aspect, a server may be used to generate manifest files and updates to video players requesting video content. As a result, instead of relying on each video player to enforce logic rules for distribution of the video content, the manifest files and updates may be used to enforce the logic rules on a wider variety of video player configurations. In addition, the manifest files and updates may be provided unique to each video player. Consequently, the manifest files and updates may further provide customized user experiences despite the same video content being requested. 
     According to one embodiment of the present disclosure a system for managing video playback is disclosed. The system includes a manifest server configured to communicate with a video player and a content delivery network. The manifest server executes a program stored in a non-transient medium to receive a request from a video player for playing a video stream. Upon receiving the request, the manifest server communicates with a content delivery network to obtain a first manifest file containing information for allowing a video player to play the video stream. The manifest server modifies the first manifest file to produce a second manifest file unique to the video player having the request, where the second manifest file contains information for allowing the video player to play the video stream according to a rule that is added to the second manifest file. 
     According to other aspects of the disclosure, the rule may direct the video player to play alternative content in addition to playing the video stream. Optionally, the rule may direct the video player to play the alternative content before playing the video stream or the rule may direct the video player to play the alternative content after playing a predetermined amount of the video stream. According to still another option, the rule may inhibit the video player from skipping the alternative content. 
     According to still other aspects of the disclosure, the second manifest file may include a session identifier for identifying a connection between the manifest server and the video player having the request. The method may include modifying the second manifest file to produce an updated second manifest file unique to the video player having the request, where the updated second manifest file contains information for allowing the video player to play the video stream according to the rule and a second request from the video player. The second request may be to seek forward or backward in the video stream, and the manifest server may compare the second request to a view history of the video player to produce the updated second manifest file. 
     According to another embodiment of the disclosure, a method for managing video playback using a manifest server configured to communicate with a video player and a content delivery network is disclosed. The method receives a request from a video player for playing a video stream and upon receiving the request, communicates with a content delivery network to obtain a first manifest file containing information for allowing a video player to play the video stream. The first manifest file is modified to produce a second manifest file unique to the video player having the request, where the second manifest file contains information for allowing the video player to play the video stream according to a rule that is added to the second manifest file. 
     These and other objects, advantages, and features of the disclosure will become apparent to those skilled in the art from the detailed description and the accompanying drawings. It should be understood, however, that the detailed description and accompanying drawings, while indicating various embodiments of the present disclosure, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the present disclosure without departing from the spirit thereof, and the disclosure includes all such modifications. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWING(S) 
       Various embodiments of the subject matter disclosed herein are illustrated in the accompanying drawings in which like reference numerals represent like parts throughout, and in which: 
         FIG. 1  is a block diagram representation of an environment incorporating the method for managing video playback using manifest files generated by a server of the present disclosure; 
         FIG. 2  is a flow diagram illustrating the managing of video playback using manifest files generated by a server according to one embodiment of the disclosure; 
         FIG. 3A  is an exemplar video signal divided into video segments; 
         FIG. 3B  is a modified video signal stitched to include alternative content segments; and 
         FIG. 3C  is a second modified video signal (updated) stitched to include alternative content segments upon a user request to seek forward or backward, each according to an embodiment of the disclosure; 
         FIG. 4  is a segment of a manifest file describing the bandwidths of the available streams for the streaming video content and the location of each stream according to one embodiment of the disclosure; and 
         FIG. 5  is a segment of a manifest file including a portion of a playlist according to one embodiment of the disclosure. 
     
    
    
     In describing the various embodiments of the disclosure which are illustrated in the drawings, specific terminology will be resorted to for the sake of clarity. However, it is not intended that the disclosure be limited to the specific terms so selected and it is understood that each specific term includes all technical equivalents which operate in a similar manner to accomplish a similar purpose. For example, the word “connected,” “attached,” or terms similar thereto are often used. They are not limited to direct connection but include connection through other elements where such connection is recognized as being equivalent by those skilled in the art. 
     DETAILED DESCRIPTION OF THE DISCLOSURE 
     The various features and advantageous details of the subject matter disclosed herein are explained more fully with reference to the non-limiting embodiments described in detail in the following description. 
     Turning initially to  FIG. 1 , one environment for managing video playback using manifest files generated by a server is illustrated. A content provider  110  generates a video signal  112  to be distributed to video consumers. The video signal may be provided in an uncompressed file format, such as a SDI format, or in a compressed format, such as an MPEG or TS file format. The video signal  112  is sent to an encoder  114  which converts the file into a live streaming signal  116 . The live streaming signal  116  is preferably a segmented data stream that may be transmitted using standard HTTP or HTTPS protocol over the internet. The live streaming signal  116  may include multiple streams, where each stream may have a different data rate and/or different resolution. The format of the live streaming signal may be, but is not limited to, HLS or MPEG-DASH. Still other protocols such as HTTP Dynamic Streaming (HDS) from Adobe® or Microsoft® Smooth Streaming and the like may be used without deviating from the scope of the disclosure. 
     In addition to the segmented data stream, the encoder generates a manifest file. The manifest file contains information for a video player  122  to play the segmented data stream such as the data rate and resolution of each stream and a playlist providing an address from which the video content may be retrieved. The encoder  114  generates a single manifest file for each encoded video signal, where the manifest file is distributed along with the streaming signal  116  and stored on a CDN  118 . Each CDN  118  includes a number of edge servers  120  which store the encoded video signal  116  and manifest file until playback of the video content is requested by a video player  122 . Although the embodiment illustrated in  FIG. 1  shows a single CDN  118 , it is contemplated that the encoded video signal  116  may be stored on multiple CDNs  118 . The manifest file may include an address of each CDN such that playback may occur from any of the CDNs  118 . 
     As further shown in  FIG. 1 , the illustrated environment includes a manifest server  124 . The manifest server  124  is used to provide a unique manifest file, also referred to herein as a per-user manifest file, to each video player  122  for each requested video content. Each video player  122  includes a native video player module  128  which provides an interface to a user and which manages video playback on the video player  122 . Some video players  122  may further include an enhanced video player module  129 , illustrated as an optional module in  FIG. 1 . The enhanced video player module  129  may be a plug-in or other software module executing on the video player  122  that either complements (i.e., adds additional capabilities) or replaces (i.e., adds additional capabilities and incorporates the video interface and playback capabilities) the native video player module  128 . As will be discussed in more detail below, when a user  125  requests video content for playback on the video player  122 , the native or enhanced video player module  129  communicates with the manifest server  124  rather than the CDN  118  to obtain the manifest files for video playback. The manifest server  124  manages the retrieval and delivery of the manifest file generated by the encoder  114  to provide the unique manifest file to each video player  122 . 
     Turning next to  FIG. 2 , the operations performed to manage video playback using manifest files generated by a server are illustrated. At block  130 , the encoder  114  receives the initial video signal  112 . It is contemplated that this video signal  112  may be a pre-recorded signal, such as an episode of a television show or a movie, or the video signal  112  may be a live stream, for example, of a sporting event, concert, or news feed. The encoder  114  converts the original video signal into a live streaming signal  116  suitable for delivery via HTTP or HTTPS. One operation in converting the video signal is to divide the video signal into segments. The segments may be, for example, 10 seconds in length. Optionally, other segment lengths, for example, from 1 second up to 10 seconds may be selected. The length of the video segment must be less than the maximum payload for an HTTP data packet. Referring briefly to  FIG. 3A , a video signal  116  from the encoder  114 , shown by way of example, may consist of video content being divided into nine segments of requested video content, illustrated as “c 1 ,” “c 2 ,” “c 3 ,” and so forth, with each segment being 10 seconds in length for a net content length of 90 seconds. After generating the video signal  116  into segments (also referred to as requested video content), the video signal and a manifest file are transmitted to the CDN  118  for storage in one of the edge servers  120 , as shown in block  132 . 
     At block  134 , a user  125  then requests playback of a desired video segment on the video player  122 . The video player  122  may be any suitable electronic device to receive the streaming signal  116  such as a desktop computer, a television, a laptop computer, a tablet, Wi-Fi enabled device connected to a video screen, or a mobile phone. At block  136 , the video player  122 , in turn, requests a manifest file from the manifest server  124  in order to retrieve the information necessary to play the requested video content. With reference also to  FIGS. 4 and 5 , segments of manifest files are illustrated that demonstrate a portion of the content that may be available in a manifest file. The manifest file is a text file and the particular content on each line of the text file is identified by the directive at the start of the line. For example,  FIG. 4  identifies four different streams in the streaming signal  116  where each stream has a different bandwidth. The location of a playlist for each of the streams is also included in the manifest file.  FIG. 5  is another manifest file which contains a portion of the playlist of an encrypted video segment. Each line may identify a particular video segment between 1 and 5 (i.e., the “4”, “−2”, etc. . . . prior to the .ts file extension), and provides the location of video segment in the CDN  118 . The manifest file may include any information corresponding to the video stream, such as metadata information for the video stream. 
     When the video player  122  requests the manifest file from the manifest server  124  a connection is established between the devices. A session identifier is also generated to identify the connection. The session identifier may be generated by the video player  122  or the manifest server  124 . For purposes of illustration, it will be assumed that the session identifier is generated by the video player  122 . The session identifier is transmitted to the manifest server  124  by the video player  122  when requesting a manifest file. The manifest server  124  then requests the manifest file from the CDN  118  at block  138 . At block  140 , the CDN  118  returns the manifest file to the manifest server  124 . 
     Because the manifest server  124  has established a connection with video player  122 , it may customize the manifest file prior to returning the manifest file to the video player  122  and provide a unique manifest file to each video player  122 . Without the manifest server  124 , the video player  122  retrieves the manifest file directly from the CDN  118  and the content of the manifest file is the same for all users. However, because the manifest server  124  is providing a unique manifest file to each player, the manifest file may include identifying information of the video player  122 , the user  125  of the video player, or a combination thereof. Further, the manifest file may be modified to include content specific for the user  125 . 
     According to one embodiment of the disclosure, the manifest server  124  is configured to generate a unique per-user manifest file (and updates) for the video player  122 . The manifest file from the manifest server  124  may reflect logic rules which may be desirable to enforce for distribution of the video content to the user  125 . For example, a video manifest file may be modified by the manifest server  124  to enforce logic rules requiring viewing alternative content, such as commercial advertisements, at the video player  122  at a very start of the streaming signal  116 , or logic rules requiring inhibiting an ability of the user  125  to skip such alternative content if the user  125  has already viewed a predetermined amount of the streaming signal  116 , or conversely, logic rules allowing the user  125  to skip the alternative content if the user  125  has not viewed the predetermined amount of the streaming signal  116 . Accordingly, the manifest server  124  may generate a customized manifest file for the video player  122  which stitches together requested video content and alternative content into a single video stream according to predetermined logic rules desired to be enforced. 
     For example, referring briefly to  FIG. 3B , with the customized manifest server generated, the video signal  116  may be reconfigured for playback as a modified video signal  116 ′ including two segments of alternative content, illustrated as “a 1 ” and “a 2 , followed by four segments of requested video content, illustrated as “c 1 ,” “c 2 , “c 3 ” and “c 4 , followed by another two segments of alternative content, illustrated as “a 2 ” and “a 3 , and followed by another four segments of requested video content, illustrated as “c 5 ,” “c 6 , “c 7 ” and “c 8 .” With segments being 10 seconds in length, the modified video signal  116 ′ may then have a gross content length of 130 seconds. This stitched video stream may reflect, for example, enforcement of a logic rule by the manifest server  124  requiring initial viewing of a first advertisement and a midstream viewing of a second advertisement for a first time viewing of the requested video content at the video player  122 . Each time the user  125  plays a video, the video player  122  may obtain an updated manifest file from the manifest server  124 . The manifest server  124  may individually track a status of the video player  122  and the user view experiences. At block  142 , the manifest server  124  transmits the manifest file to the video player  122 . 
     Referring again to  FIG. 1 , if the video player  122  includes an enhanced video player module  129  from the provider of the manifest server  124 , the enhanced video player module  129  may be configured to store the view history of the user  125  on the video player  122  for providing to the manifest server  124 . The enhanced video player module  129  may store and provide, for example, which alternative content the user  125  has viewed, total content time the user  125  has viewed, total alternative content time the user  125  has viewed, a listing of the requested content and alternative content the user  125  has viewed by segment, and so forth. If the video player  122  does not have an enhanced video player module  129 , then the manifest server  124  may seek to record such view history directly. 
     The manifest server  124  may use the view history to enforce logic rules for each video player. For example, logic rules may require a first amount of alternative content, such as advertisement, be viewed by users at a first subscription level and may require a second amount of alternative content be viewed by users at a second subscription level. When the video player  122  establishes a connection with the manifest server  124 , the appropriate logic rules for the connection are determined based, for example, on a user name or device identification provided from the video player to the manifest server. The video player  122  is unable to distinguish between alternative content and the requested content based on the playlist provided in the manifest file. Therefore, the manifest server updates the playlist in the manifest file to enforce the logic rules associated with each connection. 
     Having received the manifest file from the manifest server  124 , at block  144 , either the enhanced video player module  129  or the native video player module  128  executes the modified video signal  116 ′ in a single video stream. The manifest file will have the address of the CDN  118  as containing the segmented video content. Therefore, the video player  122  can start retrieving the video content from the CDN. The video player  122  repeatedly requests the next segment in the playlist from the CDN  118  and the CDN  118  returns the requested segment according to the modified video signal  116 ′. 
     Referring again to  FIG. 2 , according to another aspect of the disclosure, at block  146 , the user  125  may wish to seek forward or backward when viewing the modified video signal  116 ′, such as to view a specific requested video content segment or to skip viewing alternative content stitched into the playlist. Accordingly, instead of the video player  122  seeking forward or backward to a requested video content segment in accordance with the user&#39;s request, at block  148 , the video player  122  redirects the user&#39;s request to the manifest server  124 . The manifest server  124 , in turn, may consider the logic rules to be enforced based on the requested video content and/or the view history on the video player  122 . At block  150 , the manifest server  124  may deliver an updated manifest file to the video player  122  responsive to the user&#39;s request while within the limits of the logic rules to be enforced. Then, at block  152 , either the enhanced video player module  129  or the native video player module  128  proceeds to execute according to the updated manifest file in a single video stream via the CDN  118 . 
     For example, referring briefly to  FIGS. 3A-3C , the user has requested the video signal  116  for playback on the video player  122 . The logic rules in the manifest server  124  may require the alternative content, such as commercial advertising, a 1 -a 4  be inserted within the video stream provided to the video player  122 . The video player  122 , therefore, receives a custom manifest file with a playlist for a modified video signal  116 ′ including a 1 , a 2 , a 3 , and a 4  inserted. At the beginning of the playback of the modified video stream  116 ′, the user may try to skip a portion or all of the commercial advertisements (a 1 , a 2 ) by initiating a seek forward request on the video player  122 . The enhanced video player module  129  redirects the seek forward request to the manifest server  124 . Upon the video player  122  redirecting a user&#39;s request to seek forward or backward in the modified video signal  116 ′, for example, to video content segment “c 6 ,” the manifest server  124  may consider the logic rules and/or view history on the video player  122  associated with the particular connection. If the user has satisfied the required rules for viewing alternative content, the manifest file returns a playlist starting at segment “c 6 ” and allows the user to continue viewing from segment “c 6 ”. If, however, the user has not satisfied the logic rules, the manifest server  124  provides a manifest file including alternative content, such as a 3  and a 4  prior to viewing “c 6 ”. The video player  122  then begins playback of the new alternative content requiring the user  125  to view alternative content before viewing the video content segment “c 6 .” As a result, the manifest server  124  delivers a second modified video signal  116 ″ (i.e., an updated playlist in a manifest file) to the video player  122  responsive to the user&#39;s request, but enforcing a logic rule, and thereby requiring the user  125  to first view two segments of alternative content, illustrated as “a 3 ” and “a 4 ,” before viewing the sought content segment “c 6 .” 
     Optionally, the manifest server  124  may enforce logic rules by inhibiting seek operations. For example, if the logic rules require viewing of two alternative content segments (e.g., a 1 , a 2 ) prior to viewing requested video content, the initial playlist in the manifest file may add the two alternative content segments at the beginning as shown in the modified video stream  116 ′. If a user initiates a seek forward operation, the enhanced video player module  129  again transmits the request to the manifest server  124 . The manifest server  124  may discard or ignore the seek forward operation until the view history indicated that the required alternative content has been viewed. It is, therefore, one aspect of the disclosure that a server-side assisted method may be used to enforce logic rules for video view experiences by enforcing such rules in video manifest files delivered to video players. Individual video manifest files may be updated to enforce such rules unique to each video player as desired. 
     According to another aspect of the disclosure, the manifest server  124  may record a user&#39;s viewing history from multiple devices to enforce the logic rules. For example, a user may begin watching a requested video stream  116  on a first video player  122 . The manifest server  124  generates the per-user manifest file including a playlist for the modified video stream  116 ′ according to the logic rules associated with the user. The manifest server  124  records the view history from the first video player  122  associated with the user and the requested video stream  116 . The user may stop viewing the video stream  116  on the first device and later resume watching the video stream  116  on a second video player  122 . If both video players are associated with the user, the manifest server  124  has the view history of the user stored and may begin playback on the second video player  122  according to the logic rules and the prior viewing history. Thus, a server-side assisted device hand over may allow hand over video status and/or user status information from one device to another device. When a user hands over video playback from one device to another device, such as from a smart phone to a television, the viewing history and associated user experiences may transfer to the other device. 
     In one aspect of the disclosure, a server-side assisted method may be used to enforce logic rules for video view experiences by enforcing such rules in video manifest files delivered to video players. Individual video manifest files may be updated to enforce such rules unique to each video player as desired. 
     According to another aspect of the disclosure, a video manifest file may be modified by a server to include video segments which video viewing logic rules dictate. For example, logic rules may dictate modifying the manifest file to insert alternative content, such as commercial advertisements, at a very start of the video stream. Each time a user plays a video, the video player may obtain an updated manifest file from the server. The server may individually track the status of video players and their user view experiences, such as for how much time a current video may have been played by a user, among other things. 
     According to another aspect of the disclosure, video seek operations may be performed per-user via a manifest file delivery service provided by a server which returns manifest files to users enforcing logic rules which do not allow skipping of alternative content. As a result, if a user seeks to advance forward or backward in video stream, instead of the video player seeking forward or backward, the video player may request a manifest file update from the server. 
     According to another aspect of the disclosure, a software layer may be provided for the video player to provide, among other things, video content length, current play position, and location of alternative content, when a corresponding manifest file does not contain one or more of such segments. This may hide complexity in total length computations and accurate seek position determinations in the event that a manifest file contains only partial segments. In one aspect, content length may include strictly requested video content and not alternative content. 
     According to another aspect of the disclosure, a server side assisted device hand over may be provided to hand over video status and/or user status information from one device to another device. A per-user device manipulator may keep track of user status and/or video status, and when a user hands over video streaming from one device to another device, such as from a smart phone to a media player connected to a television, all user experiences may also transfer to the other device. 
     Portions of the disclosed embodiment are described herein as being implemented on various physical devices, including, but not limited to the video player  122 , the manifest server  124 , the encoder  114 , or the edge server  120  within a CDN  118 . It would be understood by one skilled in the art that these devices may include processing devices, such as a single microprocessor, multiple microprocessors, co-processors, application specific integrated circuits (ASICs), or multiple computing devices operating separately, in tandem, or a combination thereof. Further, each of these devices includes storage which may be include transitory storage, non-transitory storage, or a combination thereof. The storage may include memory devices such as random access memory (RAM), read-only memory (ROM), solid state memory, and the like. The storage may further include devices configured to read removable storage medium such as CD-ROMs, DVDs, floppy disks, universal serial bus (USB) devices, memory cards, and the like. The processing devices may be configured to read and execute instructions stored in non-transitory storage to perform various operations in the methods described herein. 
     It should be understood that the disclosure is not limited in its application to the details of construction and arrangements of the components set forth herein. The disclosure is capable of other embodiments and of being practiced or carried out in various ways. Variations and modifications of the foregoing are within the scope of the present disclosure. It also being understood that the technology disclosed and defined herein extends to all alternative combinations of two or more of the individual features mentioned or evident from the text and/or drawings. All of these different combinations constitute various alternative aspects of the present disclosure. The embodiments described herein explain the best modes known for practicing the disclosure and will enable others skilled in the art to utilize the disclosure.