Source: http://www.patentsencyclopedia.com/app/20100169458
Timestamp: 2018-05-26 11:00:46
Document Index: 609014341

Matched Legal Cases: ['Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 12', 'Application No. 12', 'Application No. 12']

Real-Time or Near Real-Time Streaming - Patent application
Patent application title: Real-Time or Near Real-Time Streaming
Patent application number: 20100169458
Real-Time or Near Real-Time Streaming - Patent application - Methods and apparatuses for real-time or near real-time streaming of content using transfer protocols such as an <?php require_once('/home/patents/php/mtc.config.php'); require_once('/home/patents/php/mtc.class.php'); $MTC = new MTC(); $MTC->init(); ?>
Methods and apparatuses for real-time or near real-time streaming of content using transfer protocols such as an HTTP compliant protocol. In one embodiment, a method includes dividing a stream of data representing the contiguous time based content of a program (e.g. a live video broadcast), into a plurality of distinct media files, and generating a playlist file having a plurality of tags and Universal Resource Indicators (URIs) indicating an order of presentation of the plurality of distinct media files. The plurality of media files and the playlist file can be made available for transmission to a client device which can retrieve the media files using the playlist file.
1. A method comprising:dividing a stream of data into multiple media files each containing a portion of the stream of data on the server device, the multiple media files to be stored as individual files in a memory on the server device in a non-streaming transfer protocol compliant format; andgenerating a playlist file having a plurality of tags and a plurality of Universal Resource Indicators (URIs), the plurality of URIs indicating an ordering of the multiple media files to recreate the stream of data.
2. The method of claim 1 wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol and wherein the method further comprises:transmitting the playlist file to a client device using the non-streaming transfer protocol; andtransferring one or more of the multiple media files to the client device using the non-streaming transfer protocol in response to requests from the client device utilizing one or more of the plurality of URIs.
3. The method of claim 1 further comprising:generating an updated playlist file corresponding to changes to the multiple media files, the updated playlist file comprising a plurality of updated URIs indicating an ordering of the updated multiple media files to recreate the stream of data.
4. The method of claim 3 wherein the changes to the multiple media files comprises at least one of: (a) additions of one or more media files and the updated playlist file includes URIs to the multiple media files and the added one or more media files; (b) modifications to the one or more media files and the updated playlist file comprises URIs to the multiple media files, and one or more of the tags in the plurality of tags is updated to reflect the modifications to the one or more media files: or (c) removal of one or more selected media files from the multiple media files and the addition of further media files to result in one or more remaining media files and the updated playlist file comprises URIs to the remaining media files.
5. The method of claim 4 wherein the updated playlist is generated at the expiration of a selected period of time based on an attribute of one of the tags in the playlist file.
6. The method of claim 1 further comprising:determining an approximate duration of a next media file to be added to the playlist file:causing the playlist file to include a tag indicating the approximate duration.
7. The method of claim 1 further comprising:determining an encryption method to be applied to the multiple media files;causing the playlist file to include a tag to indicate the encryption method to be applied to the multiple media files and wherein the tag comprises a URI to a security key.
8. The method of claim 1 further comprising:determining a time and/or data associated with a beginning of a next media file; andcausing the playlist to include a tag indicating the time and/or date associated with the beginning of the next media file.
9. The method of claim 1 further comprising:determining whether a client device to receive the media files is authorized to store the media files after playback;causing the playlist to include a tag that indicates whether the client device is authorized to store the media files after playback.
10. The method of claim 1 further comprising:determining whether additional URIs corresponding to additional media files will be added to the playlist file;causing the playlist to include a tag that indicates whether additional URIs corresponding to additional media files will be added to the playlist file wherein the tag comprises an EXT-X-ENDLIST tag.
11. The method of claim 1 further comprising:determining whether a next URI in the playlist file indicates another playlist file;causing the playlist to include a tag to indicate that the next URI in the playlist indicates another playlist wherein the tag indicates a bandwidth associated with the another playlist.
12. An article comprising a computer-readable medium having stored thereon executable instructions that, when executed, cause one or more processors to:divide a stream of data into multiple media files each containing a portion of the stream of data on the server device, the multiple media files to be stored as individual files in a memory on the server device in a non-streaming transfer protocol compliant format; andgenerate a playlist file having a plurality of tags and a plurality of Universal Resource Indicators (URIs), the plurality of URIs indicating an ordering of the multiple media files to recreate the stream of data.
13. The article of claim 12 wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol and wherein further instructions cause the one or more processors to:transmit the playlist file to a client device using the non-streaming transfer protocol;transfer one or more of the multiple media files to the client device using the non-streaming transfer protocol in response to requests from the client device utilizing one or more of the plurality of URIs.
14. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:generate an updated playlist corresponding to chances to the multiple media files, the updated playlist comprising a plurality of updated URIs indicating an ordering of the updated multiple media files to recreate the stream of data.
15. The article of claim 14 wherein the changes to the multiple media files comprises at least one of: (a) additions of one or more media files and the updated playlist file includes URIs to the multiple media files and the added one or more media files; (b) modifications to the one or more media files and the updated playlist file comprises URIs to the multiple media files, and one or more of the tags in the plurality of tags is updated to reflect the modifications to the one or more media files; or (c) removal of one or more selected media files from the multiple media files and the addition of further media files to result in one or more remaining media files and the updated playlist file comprises URIs to the remaining media files.
16. The article of claim 14 wherein the updated playlist is generated at the expiration of a selected period of time based on an attribute of one of the tags in the playlist file.
17. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine an approximate duration of a next media file to be added to the playlist file;cause the playlist file to include a tag indicating the approximate duration.
18. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine an encryption method to be applied to the multiple media files;cause the playlist file to include a tag to indicate the encryption method to be applied to the multiple media files, and wherein the tag comprises a URI to a security key.
19. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine a time and/or data associated with a beginning of a next media file; andcause the playlist to include a tag indicating the time and/or date associated with the beginning of the next media file.
20. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine whether a client device to receive the media files is authorized to store the media files after playback;cause the playlist to include a tag that indicates whether the client device is authorized to store the media files after playback.
21. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine whether additional URIs corresponding to additional media files will be added to the playlist file;cause the playlist to include a tag that indicates whether additional URIs corresponding to additional media files will be added to the playlist file wherein the tag comprises an EXT-X-ENDLIST tag.
22. The article of claim 12 further comprising instructions that, when executed, cause the one or more processors to:determine whether a next URI in the playlist file indicates another playlist file;cause the playlist to include a tag to indicate that the next URI in the playlist indicates another playlist wherein the tag indicates a bandwidth associated with the another playlist.
23. A method comprising:requesting, with a client device, a playlist file over a network using a non-streaming transfer protocol;receiving, in response to the request and with the client device, the playlist file from a server device, the playlist file having Universal Resource Indicators (URIs) indicating a plurality of media files and a plurality of tags having parameters related to playback of the plurality of media files;requesting one or more of the media files in an order indicated by the playlist file;receiving the one or more requested media files over the network using the non-streaming transfer protocol.
24. The method of claim 23 further comprising:generating an audio and/or video output representing the stream of content by playing the media files with the client device in the order indicated by the playlist file, and wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol.
25. The method of claim 23 further comprising:sending repeatedly a subsequent request for the playlist file to the server device;receiving an updated playlist file in response to the subsequent request;using the updated playlist file received in response to the subsequent request.
26. The method of claim 25 further comprising:comparing the playlist file, received in response to the request, and the updated playlist file;determining whether the updated playlist file received in response to the subsequent request includes a URI of a last loaded media file;halting playback if the URI of the last loaded media file is not included in the updated playlist file;utilizing a tag in the updated playlist file received in response to the subsequent request to access a next media file.
27. The method of claim 26 wherein the tag comprises an EXT-X-MEDIA-SEQUENCE tag.
28. The method of claim 26 further comprising:sending a second subsequent request for the playlist file to the server device after a first period of time if the playlists differ; andsending the second subsequent request for the playlist file to the server device after a second period of time if the playlists do not differ, wherein the first period of time is based on a first multiple of attribute of one of the tags in the playlist file and the second period of time is based on a second multiple of the attribute.
29. The method of claim 23 further comprising:determining, with the client device, whether the playlist file includes a tag indicating whether additional URIs corresponding to media files may be added to the playlist file;selecting, with the client device, any one of the URIs from the playback file if the tag indicates that additional URIs may not be added to the playlist file or selecting any one of the URIs from the playlist file except for the second-to-last URI and the last URI if the tag indicates that additional URIs may be added to the playlist file;accessing the media files in an order indicated by the URIs in the playlist file starting with the selected URI and wherein the tag indicates the end of a program presented through the media files.
30. The method of claim 23 further comprising:determining, with the client device, whether a tag in the playlist file indicates whether the client device is authorized to store media files after playback;storing the media files on the client device after playback if the tag indicates that the client is authorized to store the media files after playback;preventing access to the media files after playback if the tag indicates that the client is not authorized to store the media files after playback.
31. An article comprising a computer-readable medium having stored thereon instructions that, when executed, cause one or more processors to:request, with a client device, a playlist file from a server device over a network using a non-streaming transfer protocol;receive, in response to the request and with the client device, the playlist file from the server device, the playlist file having Universal Resource Indicators (URIs) indicating a plurality of media files and a plurality of tags having parameters related to playback of the plurality of media files;request one or more of the media files in an order indicated by the playlist file;receive the one or more requested media files over the network using the non-streaming transfer protocol.
32. The article of claim 31 further comprising instructions that, when executed, cause the one or more processors to:generate an audio and/or video output representing the stream of content by playing the media files with the client device in the order indicated by the playlist file and wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol.
33. The article of claim 31 further comprising instructions that, when executed, cause the one or more processors to:send repeatedly a subsequent request for the playlist file to the server device;receive an updated playlist file in response to the subsequent request;use the updated playlist file received in response to the subsequent request.
34. The article of claim 33 further comprising instructions that, when executed, cause the one or more processors to:compare the playlist file, received in response to the request, and the updated playlist file;determine whether the updated playlist file received in response to the subsequent request includes a URI of a last loaded media file;halt playback if the URI of the last loaded media file is not included in the updated playlist file;utilize a tag in the updated playlist file received in response to the subsequent request to access a next media file.
35. The article of claim 33 further comprising instructions that when executed, cause the one or more processors to:send a second subsequent request for the playlist file to the server device after a first period of time if the playlists differ; andsend the second subsequent request for the playlist file to the server device after a second period of time if the playlists do not differ, wherein the first period of time is based on a first multiple of attribute of one of the tags in the playlist file and the second period of time is based on a second multiple of the attribute.
36. The article of claim 33 further comprising instructions that, when executed, cause the one or more processors to:determine, with the client device, whether the playlist file includes a tag indicating whether additional URIs corresponding to media files may be added to the playlist file;select, with the client device, any one of the URIs from the playback file if the tag indicates that additional URIs may not be added to the playlist file or selecting any one of the URIs from the playlist file except for the second-to-last URI and the last URI if the tag indicates that additional URIs may be added to the playlist file;access the media files in an order indicated by the URIs in the playlist file starting with the selected URI and wherein the tag indicates the end of a program presented through the media files.
37. A server device comprising:a network interface;a memory, the memory to store a playlist file and multiple media file;a processing system coupled with the network interface and the memory, the processing system configured to divide a stream of data to generate the multiple media files where each media file stores a portion of the stream of data on the server device, the processing system configured to store the media files in the memory, the processing system configured to generate the playlist file having a plurality of tags and a plurality of Universal Resource Indicators (URIs) indicating an ordering of the multiple media files to recreate the stream of data.
38. The server device of claim 37 wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol and wherein the processing system is configured to cause to be transmitted the playlist file to a client device using a non-streaming transfer protocol, and to cause to be transferred one or more of the multiple media files to the client device using the non-streaming transfer protocol in response to requests from the client device utilizing one or more of the plurality of URIs.
39. The server device of claim 37 wherein the processing system further generates an updated playlist file corresponding to changes to the multiple media files, the updated playlist file comprising a plurality of updated URIs indicating an ordering of the updated multiple media files to recreate the stream of data and causes to be transmitted the updated playlist file to the client device.
40. A client device comprising:a network interface;a memory coupled with the processor, the memory to store a playlist file and media files received via the network interface,a processor coupled with the network interface, the processor to request the playlist file via the network interface using a non-streaming transfer protocol, to receive the playlist file from a server device, the playlist file having Universal Resource Indicators (URIs) indicating a plurality of media files and a plurality of tags having parameters related to playback of the plurality of media files, to request one or more of the media files in an order indicated by the playlist file, to receive the one or more requested media files over the network using the non-streaming transfer protocol, and to generate an audio and/or video output representing the stream of content by playing the media files in the order indicated by the playlist file.
41. The client device of claim 40 wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol.
42. The client device of claim 40, the processor further to send a subsequent request for the playlist file to the server device, to receive the playlist file from server in response to the subsequent request, to compare the playlist file received in response to the request and the playlist file received in response to the subsequent request, to use the playlist file received in response to the subsequent request if the playlist files differ.
43. A machine readable storage medium storing executable instructions which when executed by a data processing system cause the data processing system to perform a method comprising:transmitting a playlist file to a client device using a non-streaming transfer protocol, the playlist file having a plurality of tags and a plurality of Universal Resource Indicators (URIs), the plurality of URIs indicating an ordering of multiple media files which have been divided out of a stream of data to recreate the stream of data by sequential presentation of the multiple media files; andtransferring one or more of the multiple media files to the client device using the non-streaming transfer protocol in response to requests from the client device utilizing one or more of the plurality of URIs.
44. The medium of claim 43 wherein the non-streaming transfer protocol comprises a hypertext transfer protocol (HTTP)-compliant protocol and wherein the method further comprises:transmitting an updated playlist file corresponding to changes to the multiple media files, the updated playlist file comprising a plurality of updated URIs indicating an ordering of the updated multiple media files to recreate the stream of data.
[0002](1) Application No. 61/142,110 filed on Dec. 31, 2008 (Docket No. P7437Z);
[0003](2) Application No. 61/160,693 filed on Mar. 16, 2009 (Docket No. P7437Z2);
[0004](3) Application No. 61/161,036 filed on Mar. 17, 2009 (Docket No. P7437Z3); and
[0005](4) Application No. 61/167,524 filed on Apr. 7, 2009 (Docket No. P7437Z4). All of these U.S. provisional applications are incorporated herein by reference to the extent that they are consistent with this disclosure.
[0007](1) Application No. 12/______ (Docket No. P7437US2), filed Jun. 5, 2009, entitled "VARIANT STREAMS FOR REAL-TIME OR NEAR REAL-TIME STREAMING;"
[0008](2) Application No. 12/______ (Docket No. P7437US3), filed Jun. 5, 2009, entitled "UPDATABLE REAL-TIME OR NEAR REAL-TIME STREAMING;" and
[0009](3) Application No. 12/______ (Docket No. P7437US4), filed Jun. 5, 2009, entitled "PLAYLISTS FOR REAL-TIME OR NEAR REAL-TIME STREAMING."
[0013]In one embodiment, a server device stores at least a portion of content to be streamed. The content is typically a time based stream of images or audio (e.g. sounds or music) or both; an example of a time based stream is a movie in which the order and presentation of images is based on time, and hence it can be considered a time based stream. The server includes a segmenter agent to decompose the content to be streamed into segments to be transmitted via packets according to a network protocol and an indexer agent to generate one or more playlist files that can facilitate a client in presenting the segmented user data. A client device is coupled with the server device (or another server which stores the segments and playlists and transmits them but does not generate them) via a network. The client device has an assembler agent to receive the one or more playlist files and facilitate retrieval of the segmented media files into the content according to the one or more playlist files. The client device can also have an output generator agent to output the content via one or more output components of the client device.
[0018]In one embodiment, a client device can switch from a first playlist in the variant playlist to a second playlist in that variant playlist while receiving and presenting content. For example, a client device can be receiving a program, using the first playlist, and a first bit rate and can determine through measurements of the throughput rate of the network that it can receive content of the same program at a higher, second bit rate, that content being specified by the second playlist. In this case, the client device can request the second playlist, receive the second playlist and begin retrieving the media files specified in the second playlist while continuing to present the content specified by the first playlist. The client device can store the media files and the resulting decompressed content in buffers for both playlists, and the client device can perform an automatic operation to determine when and how to switch or transition between the two versions of the content. For example, a client device can use pattern matching of the audio content in the two versions of the content to find a matching point in the two versions and then cause a switch after identifying a transition in the new content from the second playlist.
[0019]The invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements.
[0036]The present description includes material protected by copyrights, such as illustrations of graphical user interface images. The owners of the copyrights, including the assignee of the present invention, hereby reserve their rights, including copyright in these materials. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office file or records, but otherwise reserves all copyrights whatsoever. Copyright Apple Inc. 2009.
[0046]Client devices 150 and 180 may receive the playlist files and media files from server 120 over network 110. Client devices may be any type of electronic device that is capable of receiving data transmitted over a network and generate output utilizing the data received via the network, for example, wireless mobile devices, PDAs, entertainment devices, consumer electronic devices, etc. The output may be any media type of combination of media types, including, for example, audio, video or any combination thereof.
[0054]Lines that start with the comment character can be either comments or tags. Tags can begin with #EXT, while comment lines can begin with #. Comment lines are normally ignored by the server and client. In one embodiment, playlist files are encoded in UTF-8 format. UTF-8 (8-bit Unicode Transformation Format) is a variable-length character encoding format. In alternate embodiments, other character encoding formats can be used.
[0055]In the examples that follow, an Extended M3U format is utilized that includes two tags: EXTM3U and EXTINF. An Extended M3U file may be distinguished from a basic M3U file by a first line that includes "#EXTM3U".
[0056]EXTINF is a record marker that describes the media file identified by the URI that follows the tag. In one embodiment, each media file URI is preceded by an EXTINF tag, for example: [0057]#EXTINF: <duration>,<title>where "duration" specifies the duration of the media file and "title" is the title of the target media file.
These tags will each be described in greater detail below. While specific formats and attributes are described with respect to each new tag, alterative embodiments can also be supported with different attributes, names, formats, etc.
[0059]The EXT-X-TARGETDURATION tag can indicate the approximate duration of the next media file that will be added to the presentation. It can be included in the playback file and the format can be: [0060]#EXT-X-TARGETDURATION:<seconds>where "seconds" indicates the duration of the media file. In one embodiment, the actual duration may differ slightly from the target duration indicated by the tag. In one embodiment, every URI indicating a segment will be associated with an approximate duration of the segment; for example, the URI for a segment may be prefixed with a tag indicating the approximate duration of that segment.
[0061]Each media file URI in a playlist file can have a unique sequence number. The sequence number, if present, of a URI is equal to the sequence number of the URI that preceded it, plus one in one embodiment. The EXT-X-MEDIA-SEQUENCE tag can indicate the sequence number of the first URI that appears in a playlist file and the format can be: [0062]#EXT-X-MEDIA-SEQUENCE:<number>where "number" is the sequence number of the URI. If the playlist file does not include a #EXT-X-MEDIA-SEQUENCE tag, the sequence number of the first URI in the playlist can be considered 1. In one embodiment, the sequence numbering can be non-sequential; for example, non-sequential sequence numbering such as 1, 5, 7, 17, etc. can make it difficult to predict the next number in a sequence and this can help to protect the content from pirating. Another option to help protect the content is to reveal only parts of a playlist at any given time.
[0063]Some media files may be encrypted. The EXT-X-KEY tag provides information that can be used to decrypt media files that follow it and the format can be: [0064]#EXT-X-KEY:METHOD=<method>[, URI="<URI>"]The METHOD parameter specifies the encryption method and the URI parameter, if present, specifies how to obtain the key.
[0066]An EXT-X-KEY tag with a URI parameter identifies the key file. A key file may contain the cipher key that is to be used to decrypt subsequent media files listed in the playlist file. For example, the AS-128 encryption method uses 16-octet keys. The format of he key file can be a packed array of 16 octets in binary format.
[0074]The EXT-X-STREAM-INF tag can be used to indicate that the next URI in the playlist file identifies another playlist file. The tag format can be, in one embodiment: [0075]EXT-X-STREAM-INF:[attribute=value][,attribute-value]*<URI>whe- re the following attributes may be used. The attribute BANDWIDTH=<n> is an approximate upper bound of the stream bit rate expressed as a number of bits per second. The attribute PROGRAM-ID=<i> is a number that uniquely identifies a particular presentation within the scope of the playlist file. A playlist file may include multiple EXT-X-STREAM-INF URIs with the same PROGRAM-ID to describe variant streams of the same presentation. Variant streams and variant playlists are described further in this disclosure (e.g. see FIGS. 9A-9D).
[0077]Each media file URI in a playlist file identifies a media file that is a segment of the original presentation (i.e., original media content). In one embodiment, each media file is formatted as a MPEG-2 transport stream, a MPEG-2 program stream, or a MPEG-2 audio elementary stream. The format can be specified by specifying a CODEC, and the playlist can specify a format by specifying a CODEC. In one embodiment all media files in a presentation have the same format; however, multiple formats may be supported in other embodiments. A transport stream file should, in one embodiment, contain a single MPEG-2 program, and there should be a Program Association Table and a Program Map Table at the start of each file. A file that contains video SHOULD have at least one key frame and enough information to completely initialize a video decoder. Clients SHOULD be prepared to handle multiple tracks of a particular type (e.g. audio or video) by choosing a reasonable subset. Clients should, in one embodiment, ignore private streams inside Transport Streams that they do not recognize. The encoding parameters for samples within a stream inside a media file and between corresponding streams across multiple media files SHOULD remain consistent. However clients SHOULD deal with encoding changes as they are encountered, for example by scaling video content to accommodate a resolution change.
[0078]FIG. 2A is a flow diagram of one embodiment of a technique for one or more server devices to support media content using non-streaming protocols. The example of FIG. 2A is provided in terms of HTTP; however, other non-streaming protocols can be utilized in a similar manner. The example of FIG. 2A is provided in terms of a single server performing certain tasks. However, any number of servers may be utilized. For example, the server that provides media files to client devices may be a different device than a server that segments the content into multiple media files.
[0079]The server device receives content to be provided in operation 200. The content may represent live audio and/or video (e.g., a sporting event, live news, a Web camera feed). The content may also represent pre-recorded content (e.g., a concert that has been recorded, a training seminar, etc.). The content may be received by the server according to any format and protocol known in the art, whether streamed or not. In one embodiment, the content is received by the server in the form of a MPEG-2 stream; however, other formats can also be supported.
[0080]The server may then store temporarily at least portions of the content in operation 210. The content or at least portions of the content may be stored temporarily, for example, on a storage device (e.g., hard disk in a Storage Area Network, etc.) or in memory. Alternatively, the content may be received as via a storage medium (e.g., compact disc, flash drive) from which the content may be transferred to a storage device or memory. In one embodiment, the server has an encoder that converts, if necessary, the content to one or more streams (e.g., MPEG-2). This conversion can occur without storing permanently the received content, and in some embodiments, the storage operation 210 may be omitted or it may be a longer term storage (e.g. an archival storage) in other embodiments.
[0087]If media files are to be encrypted the playlist file(s) can define a URI that allows authorized client devices to obtain a key file containing an encryption key to decrypt the media files. An encryption key can be transmitted using a secure connection (e.g., HTTPS). As another example, the playlist file(s) may be transmitted using HTTPS. As a further example, media files may be arranged in an unpredictable order so that the client cannot recreate the stream without the playlist file(s).
[0094]FIG. 2B is a flow diagram of one embodiment of a technique for one or more server devices to provide dynamically updated playlists to one or more client devices. The playlists can be updated using either of the cumulative mode or the rolling mode described herein. The example of FIG. 2B is provided in terms of HTTP, however, other non-streaming protocols (e.g. HTTPS, etc.) can be utilized in a similar manner. The example of FIG. 2B is provided in terms of a server performing certain tasks. However, any number of servers may be utilized. For example, the server that provides media files to client devices may be a different device than the server that segments the content into multiple media files.
[0098]The playlist file(s) may be updated by a server for various reasons. The server may receive additional data to be provided to the client devices in operation 285. The additional data can be received after the playlist file(s) are stored in operation 255. The additional data may be, for example, additional portions of a live presentation, or additional information for an existing presentation. Additional data may include advertisements or statistics (e.g. scores or data relating to a sporting event). The additional data could be overlaid (through translucency) on the presentation or be presented in a sidebar user interface. The additional data can be segmented in the same manner as the originally received data. If the additional data constitutes advertisements, or other content to be inserted into the program represented by the playlist, the additional data can be stored (at least temporarily) in operation 215, segmented in operation 225 and stored in operation 235; prior to storage of the segmented additional data, the segments of the additional data can be encrypted. Then in operation 245 an updated playlist, containing the program and the additional data, would be generated. The playlist is updated based on the additional data and stored again in operation 255. Changes to the playlist file(s) should be made atomically from the perspective of the client device. The updated playlist replaces, in one embodiment, the previous playlist. As discussed below in greater detail, client devices can request the playlist multiple times. These requests enable the client devices to utilize the most recent playlist. In one embodiment, the additional data may be metadata; in this case, the playlist does not need to be updated, but the segments can be updated to include metadata. For example, the metadata may contain timestamps which can be matched with timestamps in the segments, and the metadata can be added to segments having matching timestamps.
[0100]Subsequent requests for the playlist from client devices result in the server providing the updated playlist in operation 275. In one embodiment, playlists are updated on a regular basis, for example, a period of time related to the target duration. Periodic updates of the playlist file allow the server to provide access to servers to a dynamically changing presentation.
[0110]The client device can receive the playlist file in operation 310. The playlist file can be stored in a memory of the client device in operation 320. The memory can be, for example, a hard disk, a flash memory, a random-access memory. In one embodiment, each time a playlist file is loaded or reloaded from the playlist URI, the client checks to determine that the playlist file begins with a #EXTM3U tag and does not continue if the tag is absent. As discussed above, the playlist file includes one or more tags as well as one or more URIs to media files.
[0119]In one embodiment, the initial minimum reload delay is the duration of the last media file in the playlist file or three times the target duration, whichever is less. The media file duration is specified by the EXTINF tag. If the client reloads a playlist file and finds that it has not chanced then the client can wait for a period of time before retrying. The minimum delay in one embodiment is three times the target duration or a multiple of the initial minimum reload delay, whichever is less. In one embodiment, this multiple is 0.5 for a first attempt, 1.5 for a second attempt and 3.0 for subsequent attempts; however, other multiples may be used.
[0126]The playlist file(s) can be retrieved by the client device in operation 375. The playlist file(s) can be stored in the client device memory in operation 380. The client device may select the bit rate to be used in operation 385 based upon current network connection speeds. Media files are requested from the server utilizing URIs included in the playlist file corresponding to the selected bit rate in operation 390. The retrieved media files can be stored in the client device memory. Output is provided by the client device utilizing the media files in operation 394 and the client device determines whether to change the bit rate.
[0127]In one embodiment, a client device selects the lowest available bit rate initially. While playing the media, the client device can monitor available bandwidth (e.g. current network connection bit rates) to determine whether the available bandwidth can support use of a higher bit rate for playback. If so, the client device can select a higher bit rate and access the media files indicated by the higher bit rate media playlist file. The reverse can also be supported. If the playback consumes too much bandwidth, the client device can select a lower bit rate and access the media files indicated by the lower bit rate media playlist file.
[0128]If the client device chances the bit rate in operation 394, for example, in response to a change in available bandwidth or in response to user input, the client device may select a different bit rate in operation 385. In one embodiment, to select a different bit rate the client device may utilize a different list of URIs included in the playlist file that corresponds to the new selected bit rate. In one embodiment, the client device may change bit rates during access of media files within a playlist.
[0131]Server stream agent 400 includes memory 414, which represents a memory device or access to a memory resource for storing data or instructions. Memory 414 may include memory local to server stream agent 400, as well as, or alternatively, including memory of the host system on which sever stream agent 400 resides. Server stream agent 400 also includes one or more interfaces 416, which represent access interfaces to/from (an input/output interface) server stream agent 400 with regard to entities (electronic or human) external to server stream agent 400.
[0132]Server stream agent 400 also can include server stream engine 420, which represents one or more functions that enable server stream agent 400 to provide the real-time or near real-time, streaming as described herein. The example of FIG. 4 provides several components that may be included in server stream engine 420; however, different or additional components may also be included. Example components that may be involved in providing the streaming environment include segmenter 430, indexer 440, security 450 and file server 460. Each of these components may further include other components to provide other functions. As used herein, a component refers to routine, a subsystem, etc., whether implemented in hardware, software, firmware or some combination thereof.
[0138]Client stream agent 500 also can include client stream engine 520, which represents one or more functions that enable client stream agent 500 to provide the real-time, or near real-time, streaming as described herein. The example of FIG. 5 provides several components that may be included in client stream engine 520; however different or additional components may also be included. Example components that may be involved in providing the streaming environment include assembler 530, output generator 540 and security 550. Each of these components may further include other components to provide other functions. As used herein, a component refers to routine, a subsystem, etc., whether implemented in hardware, software, firmware or some combination thereof.
[0142]Begin tag 610 can indicate the beginning of a playlist file. In one embodiment, begin tag 610 is a #EXTM3U tag. Duration tag 620 can indicate he duration of the playback list. That is, the duration of the playback of the media files indicated by playback list 600. In one embodiment, duration tag 620 is an EXTX-TARGETDURATION tag; however, other tags can also be used.
[0143]Date/Time tag 625 can provide information related to the date and time of the content provided by the media files indicated by playback list 600. In one embodiment. Date/Time tag 625 is an EXT-X-PROGRAM-DATE-TIME tag; however, other tags can also be used. Sequence tag 630 can indicate the sequence of playlist file 600 in a sequence of playlists. In one embodiment, sequence tag 630 is an EXT-X-MEDIA-SEQUENCE tag; however, other tags can also be used.
TABLE-US-00002 Simple Playlist file #EXTM3U #EXT-X-TARGETDURATION:10 #EXTINF:5220, http://media.example.com/entire.ts #EXT-X-ENDLIST Sliding Window Playlist, using HTTPS #EXTM3U #EXT-X-TARGETDURATION:8 #EXT-X-MEDIA-SEQUENCE:2680 #EXTINF:8, https://priv.example.com/fileSequence2680.ts #EXTINF:8, https://priv.example.com/fileSequence2681.ts #EXTINF:8, https://priv.example.com/fileSequence2682.ts Playlist file with encrypted media files #EXTM3U #EXT-X-MEDIA-SEQUENCE:7794 #EXT-X-TARGETDURATION:15 #EXT-X-KEY:METHOD=AES-128, URI=" https://priv.example.com/key.php?r=52" #EXTINF:15, http://media.example.com/fileSequence7794.ts #EXTINF:15, http://media.example.com/fileSequence7795.ts #EXTINF:15, http://media.example.com/fileSequence7796.ts #EXT-X-KEY:METHOD=AES-128,URI=" http://priv.example.com/key.php?r=53" #EXTINF:15, http://media.example.com/fileSequence7797.ts Variant Playlist file #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000 http://example.com/low.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2560000 http://example.com/mid.m3u8 #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=7680000 http://example.com/hi.m3u8 #EXT-X-STREAM-INF:PROGRAM- ID=1,BANDWIDTH=65000,CODECS="mp4a.40.5" http://example.com/audio-only.m3u8
[0149]If control input is detected in operation 730, the client device can determine if the input indicates a stop in operation 740. If the input is a stop, the process concludes and playback stops. If the input indicates a rewind or forward request in operation 750, the client device can generate output based on previously played media files still stored in memory in operation 760. If these files are no longer in a cache, then processing reverts to operation 710 to retrieve the media files and repeats the process. In an alternate embodiment, playback can support a pause feature that halts playback without concluding playback as with a stop input.
[0150]Methods for transitioning from one stream to another stream are further described with reference to FIGS. 9A-9D. One client device can perform each of these methods or the operations of each of these methods can be distributed across multiple client devices as described herein; for example, in the distributed case, one client device can retrieve the variant playlist and the two media playlists and provide those to another client device which retrieves media files specified by the two media playlists and switches between the two streams provided by the retrieved media files. It will also be understood that, in alternative embodiments, the order of the operations shown may be modified or there can be more or fewer operations than shown in these figures. The methods can use a variant playlist to select different streams. A variant playlist can be retrieved and processed in operation 901 to determine available streams for a program (e.g. a sporting event). Operation 901 can be done by a client device. A first stream can be selected from the variant playlist in operation 903, and a client device can then retrieve a media playlist for the first stream. The client device can process the media playlist for the first stream in operation 905 and also measure or otherwise determine a bit rate of the network connection for the first stream in operation 907. It will be appreciated that the sequence of operations may be performed in an order which is different than what is shown in FIG. 9A; for example, operation 907 may he performed during operation 903, etc. In operation 911 the client device selects an alternative media playlist from the variant playlist based on the measured bit rate from operation 907; this alternative media playlist may be at a second bit rate that is higher than thc existing bit rate of the first stream. This typically means that alternative stream will have a higher resolution than the first stream. The alternative media playlist can be selected if it is a better match than the current playlist for the first stream based on current conditions (e.g. the bit rate measured in operation 907). In operation 913, the alternative media playlist for an alternate stream is retrieved and processed. This typically means that the client device can be receiving and processing both the first stream and the alternative stream so both are available for presentation; one is presented while the other is ready to be presented. The client device then selects a transition point to switch between the versions of the streams in operation 915 and stops presenting the first stream and begins presenting the alternative stream. Examples of how this switch is accomplished are provided in conjunction with FIGS. 9B-9D. In some embodiments, the client device can stop receiving the first stream before making the switch.
[0152]The method shown in FIGS. 9C and 9D represents one embodiment for determining the transition point; this embodiment relies upon a pattern matching on audio samples from the two streams 951 and 953 to determine the transition point. It will be appreciated that alternative embodiments can use pattern matching on video samples or can use the timestamps in the two streams, etc. to determine the transition point. The method can include, in operation 941, storing content (e.g. stream 951) specified by the first media playlist in a buffer; the buffer can be used for the presentation of the content and also for the pattern matching operation. The stream 951 includes both audio samples 951A and video samples 95 lB. The video samples can use a compression technique which relies on i-frames or key frames which have all necessary content to display a single video frame. The content in stream 951 can include timestamps specifying a time (e.g. time elapsed since the beginning of the program), and these timestamps can mark the beginning of each of the samples (e.g. the beginning of each of the audio samples 951A and the beginning of each of the video samples 951B). In some cases, a comparison of the timestamps between the two streams may not be useful in determining a transition point because they may not be precise enough or because of the difference in the boundaries of the samples in the two streams; however, a comparison of the timestamps ranges can be used to verify there is an overlap 955 in time between the two streams. In operation 943, the client device stores in a buffer content specified by the second media playlist; this content is for the same program as the content obtained from the first media playlist and it can include timestamps also. In one embodiment, timestamps, if not present in a stream, can be added to a playlist for a stream; for example, in one embodiment an ID3 tag which includes one or more timestamps can be added to an entry in a playlist, such as a variant playlist or a media playlist. The entry may, for example, be in a URI for a first sample of an audio stream. FIG. 9D shows an example of content 953 obtained from the second media playlist, and this includes audio samples 953A and video samples 953B. In operation 945, the client device can perform a pattern matching on the audio samples in the two streams 951 and 953 to select from the overlap 955 the transition point 959 which can be, in one embodiment, the next self contained video frame (e.g. i-frame 961) after the matched audio segments (e.g. segments 957). Beginning with i-frame 961 (and its associated audio sample). presentation of the program uses the second stream obtained from the second media playlist. The foregoing method can be used in one embodiment for both a change from a slower to a faster bit rate and for a change from a faster to a slower bit rate, but in another embodiment the method can be used only for a change from a slower to a faster bit rate and another method (e.g. do not attempt to locate a transition point but attempt to store and present content from the slower bit rate stream as soon as possible) can be used for a change from a faster to a slower bit.
[0155]Electronic system 800 may also include read only memory (ROM) and/or other static storage device 830 coupled to bus 805 that may store static information and instructions for processor 810. Data storage device 840 may be coupled to bus 805 to store information and instructions. Data storage device 840 such as flash memory or a magnetic disk or optical disc and corresponding drive may he coupled to electronic system 800.
2009-03-05 System and method for collecting and organizing popular near real-time data in a virtual geographic grid
2008-11-20 Node and method to provide and keep real-time up-to-date data in a distributed hash table
2009-10-15 Method and system for real-time accessing of digital data stored on a remote terminal
2010-02-04 Estimating latencies for query optimization in distributed stream processing