System and method for splicing media files

A method for processing media content includes receiving a request for media content and, in response, accessing a plurality of source files associated with the requested media content, wherein the source files comprise media content in one or more source formats. The method also includes generating one or more target files based on the plurality of source files. Additionally, the method includes, generating an auxiliary file associated with the target files. The auxiliary file includes a plurality of content records, each content record associated with a portion of the media content in the target. The method further includes, while generating the target files, transmitting the requested media content from the plurality of target files to a client as a seamless media transmission by reading each of the content records in the auxiliary file and transmitting to the client content corresponding to each content record.

TECHNICAL FIELD

This invention relates, in general, to media-content delivery systems and, more particularly, to a system and method for splicing content from multiple media files together for delivery to a user.

BACKGROUND OF THE INVENTION

Recent years have seen a rapid increase in the amount and variety of media content available for delivery to users of public and private networks such as the Internet. Because this content is delivered from a wide variety of sources in numerous different formats use, hardware and/or software capable of transcoding, transrating, or otherwise modifying media content may be used to convert requested content to an appropriate format for delivery to and playback by clients. However, extensive delays in satisfying a user's request for content can significantly detract from the user's experience. As a result, there is a need for hardware and/or software that can deliver media content to a wide variety of different clients using low-latency techniques for modifying and transmitting the content to the appropriate clients.

SUMMARY OF EXAMPLE EMBODIMENTS

In accordance with the present invention, disadvantages and problems associated with transmitting media content to a user have been substantially reduced or eliminated.

In accordance with one embodiment of the present invention, a method for delivering media content includes receiving a request for media content and, in response to receiving the request, accessing a plurality of source files associated with the requested media content, wherein the source files comprise media content in one or more source formats. The method also includes generating one or more target files based on the plurality of source files. The target files include media content in a target format. Additionally, the method includes, generating an auxiliary file associated with the target files while generating the target files. The auxiliary file includes a plurality of content records. Each content record is associated with a portion of the media content in the target files and indicates a location of the associated portion of the media content in a particular one of the target files. The method further includes, while generating the target files, transmitting the requested media content from the plurality of target files to a client as a seamless media transmission by reading each of the content records in the auxiliary file and transmitting to the client content corresponding to each content record.

Technical advantages of certain embodiments include the ability to transmit content from a sequence of multiple media files to a client in a format appropriate for transmission to and playback by the client. Certain embodiments may facilitate seamless playback of the multiple files by the client. Additionally, particular embodiments may support realtime or near realtime transcoding of one or more of the media files. Furthermore, in particular embodiments, the described techniques may be used to allow advertisements or other content to be inserted into content requested by the user. Other technical advantages will be readily apparent to one skilled in the art from the following figures, descriptions, and claims. Moreover, while specific advantages have been enumerated above, various embodiments may include all, some, or none of the enumerated advantages.

DETAILED DESCRIPTION

FIG. 1illustrates a particular embodiment of a content-delivery system10for transmitting content from media files stored in original content store20to clients60. System10includes original content store20, modified content store22, transcoder30, streamer40, network50, and clients60a-c. Transcoder30retrieves source files70requested by clients60a-cand modifies content from the requested source files70for delivery to and playback by the requesting clients60. While this modification is ongoing, streamer40streams the modified content to the requesting clients60.

In certain circumstances, the requested content may be stored in multiple different source files70. Additionally, it may be desirable to provide advertisements or other content to the requesting client60in addition to the requested content. However, if the content is provided to the requesting client60as a series of separate files, additional input may be required from the user to initiate playback of each file. Additionally, any measure of the elapsed time may be reset with each file making it difficult to identify specific points in time within the requested content. As a result, particular embodiments of content-delivery system10may modify content from multiple source files70to an appropriate format and “splice” the modified content from these source files70together so that the modified content can be delivered to the requesting client60as part of a single, seamless media transmission.

However, in particular embodiments of content-delivery system10, clients60may utilize certain information (referred to generically herein as “media information”) that describes the delivered content to configure themselves for receipt and playback of the requested media content. For example, in particular embodiments, content-delivery system10may use media information provided by content-delivery system10to configure themselves for a frame rate, duration, and other parameters of the content streamed to clients60by streamer40. In particular embodiments, clients60may require any such media information to initiate playback of content. Thus, to facilitate playback of multiple different target files70, transcoder30may generate an auxiliary file80that includes media information describing multiple source files70and/or their content (either before or after any appropriate modification by transcoder30). Auxiliary file80may then be used by streamer40to transmit appropriately-modified content from multiple different source files70to the requesting client60as a single media session, as described further below.

In the example embodiment illustrated inFIG. 1, original content store20and modified content store22each represent or include any appropriate type of memory devices. Content stores20and22may each comprise, for example, volatile or non-volatile, local or remote devices suitable for storing data, such as random access memory (RAM) devices, read only memory (ROM) devices, magnetic storage devices, optical storage devices, or any other suitable data storage devices. Additionally, although content stores20and22are both shown as a single element in system10, content stores20and22may each represent a plurality of devices and may be distributed across multiple locations within system10. For example, in particular embodiments, content stores20and22may represent a network-attached storage (NAS).

Content in system10is stored in various media files, including source files70, target files72, and insert files74. Source files70, target files72, insert files74, and media stream76may comprise any suitable form of voice, non-voice audio, animation, and/or video information or data that may be played, displayed or otherwise communicated by clients60to users. As discussed further below, the content available on system10may, in particular embodiments, have a variety of different characteristics that may determine the types of clients60that can utilize the content of the various media files. For example, source files70, target files72, insert files74, and media stream76may all be associated with different codecs, bit-rates, resolutions, and/or other characteristics that may effect which clients60can utilize the relevant content of source files70, target files72, insert files74, and media stream76.

Original content store20stores source files70for delivery to clients60. In particular embodiments, original content store20may also store insert files74that transcoder30or streamer40insert into the content transmitted to the requesting client60. As noted above, source files70and insert files74may comprise any appropriate type of content files. Specific examples of source files70and insert files74include video/multimedia content files such as Moving Picture Experts Group (MPEG), MP4, 3GP, Quicktime, Windows Media Video (WMV), and Audio Video Interleave (AVI) files; audio content files such as Waveform audio (WAV), MPEG-1 Audio Layer 3 (MP3), and/or Windows Media Audio (WMA) files; and/or content files of any other appropriate type or format. In particular embodiments, source files70and insert files74in original content store20may all be stored in a single storage format supported by system10. Alternatively, original content store20may store source files70and insert files74in a variety of different formats appropriate for a variety of different clients60. Although shown inFIG. 1as part of system10, original content store20may represent components external to system10, such as memory components distributed throughout the Internet. As a result, in particular embodiments, transcoder30may be able to retrieve, receive, otherwise access and/or process media files provided by a vast number of content sources available throughout the Internet for delivery to clients60.

Modified content store22stores target files72containing content from source files70that has been generated, transcoded, or otherwise modified by transcoder30. In certain embodiments, transcoder30may store target files72that contain modified content from a particular source file70in modified content store22while transcoder30continues modifying the relevant source file70. Target files72may comprise media content of any appropriate type or format including, but not limited to, those listed above for source files70. In particular embodiments, target files72may be of types or formats (such as 3GP files) that, using conventional streaming techniques, would not be conducive to streaming until transcoder30had finished writing to the relevant target files72. Although referred to here as “files,” in certain embodiments, any particular target file72stored in modified content store22may, at a given point in time, represent a partial file that is not complete according to a type or format associated with the relevant target file72.

Modified content store22may also store auxiliary files80generated by transcoder30during the process of transcoding media files30in modified content store22. As discussed further below, auxiliary files80may be used to facilitate streaming of target files72. In particular embodiments, modified content store22represents storage local or internal to transcoder30, such as the internal memory of a web server functioning as transcoder30.

Transcoder30retrieves, receives, or otherwise accesses requested content from original content store20and modifies requested content in a manner suitable for transmission to and display by the requesting client60. Depending on the capabilities of the requesting client60and the characteristics of the requested source file70, transcoder30may transcode, transrate, adjust the resolution of, and/or otherwise modify characteristics of the source file70in any appropriate manner to make the content suitable for transmission to and playback by the requesting client60. In particular embodiments, transcoder30may be capable of transcoding content in realtime or near realtime. Additionally, as described further below, transcoder30may also generate auxiliary files80that include certain information to allow streamer40to stream transcoded content from a particular source file70to clients60before transcoder30has completed transcoding the entire source file70.

Streamer40retrieves, receives, or otherwise accesses target files72and auxiliary files80to deliver requested content to clients60. Additionally, in particular embodiments, streamer40may interact with clients60by, for example, providing information on content available in system10or processing requests from clients60for delivery of content from source files70. Streamer40may also be responsible for initiating transcoding or otherwise managing the operation of transcoder30in modifying content for clients60.

Although transcoder30and streamer40are each shown inFIG. 1as a single component, transcoder30and streamer40may each represent functionality provided by several separate physical components. Additionally, depending on the relevant configuration of system10, transcoder30and streamer40may represent a single physical component, or may alternatively represent separate components. In general, transcoder30and streamer40may each represent any appropriate combination of software and/or hardware suitable to provide the described functionality. As one specific example, in particular embodiments, transcoder30and streamer40each represent a separate web server configured to receive and respond to Hypertext Transfer Protocol (HTTP) requests from one another, clients60, or other elements of system10.

Network50represents any form of communication network supporting circuit-switched, packet-based, and/or any other suitable type of communication. Although shown inFIG. 1as a single element, communication network may represent one or more separate networks, including all or parts of various different networks that are separated and serve different groups of clients60. Network50may include routers, hubs, switches, gateways, call controllers, and/or any other suitable components. In general, network50may comprise any combination of public or private communication equipment such as elements of the public-switched telephone network (PSTN), a global computer network such as the Internet, a local area network (LAN), a wide-area network (WAN), or other appropriate communication equipment.

Additionally, althoughFIG. 1illustrates a particular configuration of elements directly connected to network50, network50may, in various embodiments, connect directly or indirectly to and/or interact with any appropriate elements of system10. For example, althoughFIG. 1shows transcoder30directly connected to original content store20and streamer40, transcoder30may, alternatively connect to either or both of original content store20and streamer40over network50. Accordingly, the components of system10may be arranged and configured in any appropriate manner to communicate over network50and/or over direct connections between the relevant components.

Clients60display, play, or otherwise communicate content retrieved from original content store20and modified content store22. In particular embodiments, various clients60may be capable of receiving and/or playing media associated with particular file formats, file types, codecs, bit-rates, and/or resolutions, and/or having other appropriate characteristics. Examples of clients60may include, but are not limited to, computers, video-enabled telephones, media players (such as, for example, audio- and/or video-capable iPods), televisions, PDAs, cell phones, portable communication devices, and other electronic devices. In general, however, clients60may include any appropriate combination of hardware, software, and/or encoded logic suitable to provide the described functionality. For example, in the illustrated embodiment, client60arepresent a computer, client60brepresents a video-enabled cellular telephone, and client60crepresents a portable media device. Clients60may couple to network50through a dedicated connection (wired or wireless) or may connect to network50only as needed to access media content. AlthoughFIG. 1illustrates, for purposes of example, a particular number and type of clients60, alternative embodiments of system10may include any appropriate number and type of clients60.

In operation, clients60request content contained in source files70stored by original content store20. Because each client60may only be capable of receiving and/or playing media content having certain formats, system10source files70may be transcoded or otherwise modified by transcoder30from a source format, such as an original format in which the relevant source file70is stored, to a target format appropriate for transmission to and display by a requesting client60. For purposes of this description and the claims that follow, the “format” of content refers to an encoding, bit-rate, resolution, frame rate, configuration, and/or any other aspect, characteristic, or property of the relevant content that is associated with transmitting the content to or displaying the content with a particular type of client.

Problems may arise in low-latency transmission, however, where a target format is not conducive to transmission or streaming until the corresponding source file70is completely transcoded. For example, certain file formats may require data blocks, headers, or other file information that is not written until file transcoding is complete, but that is needed for clients60to properly receive, play, or use the requested content. As one example, 3GP files include a structure called a “moov” box that contains information about the content stored in the 3GP file, such as the starting point of the content in the 3GP file, the type of content stored in the file, and the sequence in which the content should be played. The information contained in the “moov” box of a 3GP file is written progressively and is, thus, incomplete while a media file is being written to the 3GP format. Accordingly, for 3GP files, this “moov” information is not complete until the entire 3GP file is written and all 3GP audio and video data is available. As a result, the use of 3GP files may hinder or prevent certain conventional systems from streaming content from a 3GP file that is still being transcoded. Similar or comparable problems may arise with respect to other media file formats as well.

Thus, to facilitate streaming of content from incomplete target files72while transcoder30is still writing to these target files72, transcoder30may generate auxiliary files80as part of responding to media requests from clients60. Auxiliary files80include any suitable data, code, or information that may be used by streamer40to deliver portions of the content in a requested target file72while transcoding is ongoing. Streamer40may transmit information from auxiliary file80to clients60to allow clients60to configure themselves for receipt or playback of the requested content or may otherwise use this information to facilitate transmission and playback of the corresponding target files72.

In certain embodiments, system10may respond to a particular media request by delivering content from multiple different target files72to the requesting client60as a single media session. As a result, transcoder30may generate one or more auxiliary files80that facilitate transmission and playback of multiple different target files72. Auxiliary files80may allow streamer40to transmit content from target files72associated with the media request and/or allow the requesting client60to receive and play content from one of the target files72associated with the media request while modified content is still being written to that target file72. Thus, system10may reduce the latency associated with delivery of requested content by utilizing auxiliary files80to deliver content from target files72to clients60while one or more of the relevant target files72are still being generated. Furthermore, in particular embodiments, the use of auxiliary files80may allow clients60to seamlessly play back the various target files72associated with the media request.

An example of this process, as implemented by a particular embodiment of system10, is illustrated inFIG. 1. As shown byFIG. 1, a client60(here, client60a) may initiate the delivery process by transmitting a content request90to streamer40or another appropriate element of system10, such as a content-management server responsible for interfacing with clients60. Content request90may identify a specific source file70or may describe characteristics (using, for example, keywords) of the requested media content. Content request90may represent any appropriate message, request, or appropriately structured collection of information requesting delivery of content to the transmitting client60. For example, in particular embodiments, content request90represents a Hypertext Transfer Protocol (HTTP) request that identifies a particular source file70stored by original content store20.

As discussed above, content request90may result in content from multiple different files being sent to client60a. As one example, content request90may itself identify several target files72or may identify a playlist that references several target files72and system10may deliver these target files72to client60ain response to content request90. As another example, content request90may identify a particular movie, song, or other media work that is stored by system10as multiple separate source files70. System10may deliver content from all of these source files70to client60ain response to content request90. As yet another example, system10may break a requested source file70into multiple target files72before transmitting content form the requested source file70to client60ato allow advertisements or other content to be inserted into content from the requested source file70.

In response to client60atransmitting content request90, streamer40may determine an appropriate target format for the requested content to allow, facilitate, and/or optimize transmission to and playback by the requesting client60. Streamer40may determine the target format based on any appropriate information, considerations, or and/or circumstances. As one example, content request90may indicate the appropriate format that the requested content should have for transmission to and/or playback by the requesting client60. Thus, in particular embodiments, streamer40may receive an HTTP request specifying the codec, bit-rate, and/or other parameters for the requested content. As a result, streamer40may be able to determine, based on information contained in the HTTP request, the output format and characteristics appropriate for the requested content. As another example, streamer40may determine relevant characteristics of the requesting client60and select a target format for the requested content based on this information. For example, streamer40may identify whether the client is a computer, cellular phone, personal data assistant, or other type of client device. Based on this determination, streamer40may determine the format characteristics appropriate for client60abased on its type. As yet another example, streamer40may detect characteristics of the particular network or portion of network50that client60ais communicating over to determine an appropriate target format in which to provide the requested content to client60a. Thus, in particular embodiments, streamer40may determine, based on the fact that content request90was received over a certain type of network, an appropriate codec or bit-rate at which to transmit the requested content to client60a. In general, however, streamer40may determine, in any suitable manner and based on input from any appropriate components, the proper target format for the content requested by client60a.

If the requested source files70are stored in original content store20in the target format, streamer40may access the requested source files70in original content store20and transmit the requested source files70to client60ain the format in which they are stored. In the illustrated example, system10is configured to transmit modified content from target files72to clients as part of a media stream76. In general, however, system10may be configured to transmit modified content to clients60as a stream, as part of a progressive download, or in any other appropriate manner. Upon receiving any portion of the requested source files70, client60amay begin playback of the portion and subsequently received content from the requested source files70.

However, if one or more of the requested source files70are not in the target format, streamer40may request that transcoder30modify the requested source files70to the target format for client60a. To request this modification, streamer40may transmit any appropriate information identifying the appropriate target format for the requested source files70, specifying the client60that transmitted the relevant content request90, or communicate any other appropriate information to transcoder30indicating an appropriate target format for the requested source files70. Alternatively, in particular embodiments, streamer40may communicate such information to a load-balancing server or other element of system10responsible for managing the operation of one or more transcoders30in system10.

In the illustrated example, streamer40transmits a transcoding request92to transcoder30identifying the requested source files70and the target format for content request90. Transcoding request92may represent any appropriate packet, message, request, or other collection of information requesting that transcoder30deliver content from particular source files70to streamer40in a particular target format. In the illustrated example, transcoding request92represents a Hypertext Transfer Protocol (HTTP) request that identifies a plurality of source files70stored by original content store20and a target format suitable for client60a.

In response to receiving transcoding request92, transcoder30locates and retrieves the requested source files70(here, source files70a-c) from original content store20. Transcoder30may retrieve, receive, or otherwise access requested source files70a-cin any appropriate manner. In particular embodiments, original content store20includes one or more web servers (and associated memory elements) accessible through the Internet and transcoder30communicates with original content store20using the HTTP protocol.

After receiving one or more of the requested source files70a-c, transcoder30may begin modifying source files70a-caccording to the target format. As noted above, transcoder30may modify requested source files70a-cby transcoding, transrating, changing the resolution of, and/or otherwise modifying any aspect, characteristic, or property of the content in the requested source file70a-c. Transcoder30writes modified content from source files70a-cto one or more target files72in the target format. In the illustrated example, transcoder30writes modified content from each of source files70a-cto a corresponding target file72a-c.

As discussed above, while modifying content from source files70a-c, transcoder30generates media information describing the modified content. For example, in particular embodiments, transcoder30generates media information for a particular content request70that includes session information82and an auxiliary file80. Session information82describe parameters or characteristics for a media session in which streamer40will transmit the requested content to the requesting client60and allow the requesting client60to configure itself to receive this media session. Auxiliary files80include information that describes the content stored in target files72and enable streamer40to stream target files72to client60as part of a single media session. Auxiliary files80may include information such as file header information, framerate data, resolution data, bitrate data, a total number of video or audio frames in target files72, and/or any other appropriate information describing the resulting target files72. Auxiliary files80may also include information for specific portions of target files72, such as timestamp data, location information, and/or any other appropriate information describing specific portions of the corresponding target files72. The structure and content of an example auxiliary file80is discussed more fully below with respect toFIG. 2.

In the illustrated example, transcoder30generates session information82in the form of a valid Session Description Protocol (SDP) message describing a media session in which content from target files72will be transmitted to client60a. Transcoder30also generates a single auxiliary file80(shown as auxiliary file80a) that includes additional media information describing content in all of the corresponding target files72a-c. This media information comprises a plurality of records, each associated with a particular portion of the media content (e.g., a particular video or audio frame) in one of the target files72. These content records each indicate the location of the associated portion of media content within the relevant target file72and can be used to access the corresponding target file72and retrieve the associated media content.

Additionally, streamer40may be configured to transmit content from a plurality of target files72to client60aas a single seamless transmission, such as a media stream. As a result, transcoder30and/or streamer40may take additional steps to maintain temporal continuity between the multiple target files72. In particular embodiments, transcoder30may include, in auxiliary file80a, frame-rate information for the video content and sample-rate information for the audio content that will be written to the corresponding target files72. Streamer40may then compute an estimated time-stamp for each video-frame and each audio-frame of the corresponding target files72based on the video frame rate and an audio sample rate, respectively, of the transcoded file. Streamer40may then use this estimated time-stamp to compute a time-stamp for each of the media frames in the corresponding target files72.

After transcoder30begins generating media information for the corresponding target files72, streamer40may begin communicating this media information to the requesting client60or otherwise use this media information to facilitate transmission and playback of the corresponding target files72. In particular embodiments, streamer40may, while modification of the requested source files70is still ongoing, utilize the relevant media information. Streamer40may also begin to transmit content from target files72to the requesting client60as part of media stream76.

In the illustrated example, once transcoder30has formed session information82(representing an SDP message in this example), streamer40transmits session information82to client60avia a Real-Time Streaming Protocol (RTSP) link with client60a. Based on session information82, client60aconfigures itself for receipt and playback of content in accordance with the description provided by session information82. After transmitting session information82, in the illustrated example, streamer40reads a first record from auxiliary file80to identify a starting location for content within target file72a, retrieves a specified amount of content (e.g., a particular number of bytes) from target file72aat this starting location, and transmits the retrieved content to client60a. Streamer40may repeat this process for subsequent records of auxiliary file80and incrementally transmit all of modified content from target file72ato client60a. Because transcoder30begins generating auxiliary file80a, in this example, while modifying the first requested source file70a, streamer40can begin transmitting modified content from target file72abefore transcoder30has completed modifying any of source files70a-c.

As the requesting client60begins receiving modified content from a first target file72from streamer40, client60abegins playing the requested content in accordance with any session information82received from streamer40. As streamer40continues transmitting content from subsequent target files72, client60aplays back content from these addition target files72as well, until streamer40has finished transmitting all of the target files72associated with the requested source files70.

In addition, transcoder30or streamer40may take further steps to ensure that modified content is written to target files72and/or transmitted to clients *** in conformance with session information82transmitted to clients60. For example, while transmitting content from target files72, streamer40may take steps to maintain temporal continuity of the media transmission. Thus, streamer40may, in particular embodiments, calculate a time-stamp for each frame of every target file72a-cbased on a frame rate associated with the target format and a number of frames that were dropped by transcoder30while modifying content for target files72a-c. Streamer40may then write the calculated timestamp to the relevant frame, overwriting any existing timestamp, before transmitting the frame to client60a. By calculating timestamps for transmitted frames and modifying the frames accordingly, streamer40may be able to maintain temporal consistency between the frames of multiple different target files72and, as a result, may be able to transmit multiple target files72to client60aas part of a single seamless media stream.

In particular embodiments, transcoder30or streamer40may be further configured to transmit additional content to clients60apart form the content requested from source files70. Specifically, in particular embodiments, transcoder30and streamer40may be configured to transmit one or more insert files74to the requesting client60as part of transmitting the requested content. Insert files74may include any suitable voice, non-voice audio, animation, and/or video information or data that may be transmitted to the requesting client60as part of the same media transmission as the requested content. Examples of the media content that may be included in insert files74include, but are not limited, to advertisements to be played as part of the requested content, copyright notices to be played in advance of the requested content, and voice annotations offering explanation or analysis of the requested content.

In embodiments of system10that support the use of insert files74, streamer40, transcoder30, or other appropriate elements of system10may, in response to content request90, determine whether content from any insert files74should be added to the requested content and/or identify appropriate insert files74to add to the request content. The appropriate elements may make these determinations based on any suitable characteristics, factors, or considerations.

In particular embodiments, transcoder30may determine based on the specific source files70identified by or associated with content request90whether to include any insert files74with the requested content. For example, streamer40may determine based on a copyright status of requested source files70whether to include a copyright notice or based on a rating for requested source files70whether to include a parental advisory. In alternative embodiments, streamer40may insert one or more of a group of insert files74into every set of source files70requested by clients60. For example, streamer40may add an advertisement to the content requested by every content request90received by streamer40. Furthermore, in particular embodiments, streamer40may determine based on the client60from which streamer40received content request90whether to include insert files74and/or what insert files74to includes. For example, streamer40may determine based on whether the requesting client60is associated with a paying subscriber whether to insert commercials in the requested content.

If streamer40determines that insert files74are to be included in the requested content, streamer40identifies the relevant insert files74to transcoder30(e.g., as part of transcoding request92) and provides additional information to be used by transcoder30in inserting insert files74into the requested content, such as the order in which content from the various insert files74and target files72should be transmitted to the requesting client60. If transcoder30determines that the identified insert files74are not currently in the target format, transcoder30may modify the identified insert files74to the target format and operation may proceed as described above with transcoder30treating the identified insert files74in a similar manner to any of the requested source files70.

If, instead, transcoder30determines that any of the identified insert files74are already in the target format, transcoder30may still generate an auxiliary file80for the relevant insert file74to facilitate seamless playback of that insert file74along with any requested content. Alternatively, if transcoder30determines that a particular insert file74is already in the target format, streamer40may access the insert file74itself to read the media information streamer40uses to transmit insert file74to the requesting client60a. In such embodiments, transcoder30may not generate an auxiliary file80for the relevant insert file74.

Because streamer40can access and utilize media information while transcoder30is still modifying content from source files70, in particular embodiments of system10, streamer40may be capable of transmitting modified content from target files72to clients60, and clients60may be capable of playing the modified content, before transcoder30has finished creating the target files72corresponding to the requested source files70As a result, system10may provide low-latency delivery of content from multiple source files70in multiple different target formats. Additionally, in particular embodiments, streamer40and transcoder30may take steps to maintain temporal continuity between frames of the various target files72transmitted to requesting clients60. This may allow system10to deliver content from multiple different source files70as a single seamless media transmission. Consequently, particular embodiments of system10may provide numerous operational benefits. Specific embodiments, however, may provide none, some, or all of these benefits.

FIG. 2is a diagram illustrating in greater detail the contents and structure of an auxiliary file80that may be generated by particular embodiments of transcoder30when completing the content-delivery process described with respect toFIG. 1. As discussed above, transcoder30may generate auxiliary files80while transcoding content from requested source file70to create the corresponding target file72. Streamer40may then access auxiliary files80while transcoding is still ongoing and utilize the information in the accessed auxiliary files80to facilitate transmission of media content from the requested media files72before the relevant transcoder30has finished modifying content from the requested source files70and/or creating the corresponding target files72.

In the illustrated embodiment, auxiliary file80includes a file header200, one or more media headers (including a video header210and an audio header230), and one or more content records250. WhileFIG. 2and the description below describe a particular embodiment of auxiliary file80that includes certain information structured in a particular manner, auxiliary files80may include other appropriate information and/or be structured in a different manner than that described. In general, auxiliary files80may include any data appropriate to facilitate transmission of media content from target files72while transcoder30is still storing modified content in the relevant files. As a result, alternative embodiments of system10may generate auxiliary files80having some, none, or all of the elements described below and/or having elements in addition to those described below.

In the example auxiliary file80shown inFIG. 2, file header200holds general information relating to the media included in the corresponding target file72described by auxiliary file80. As one example, in particular embodiments, this general information may include a media types field202identifying one or more media types for the modified media being written to the corresponding target file72. For example, the illustrated auxiliary file80includes a media types field202that stores a four-byte value indicating the various media types included in the corresponding target file72. If target file72includes only audio content, transcoder30may store a value of “1” in the media types field202. If, instead, the corresponding target file72includes only video content or includes both audio and video content, then transcoder30may store a value of “2” or a value of “3,” respectively, in media types field202of the illustrated auxiliary file80.

Data in the media headers, such as video header210and audio header230, is used by streamer40to determine various properties of the different types of media included in the corresponding target files72. Each media header may include information specific to the content of that type in the corresponding target file72. For example, in the illustrated auxiliary file80, video header210holds information regarding video content in the corresponding target file72such as, for example, a video codec type, a content size, a bit rate, and a frame rate. In the illustrated auxiliary file80, this data is stored in a video codec type field212, multiple size fields (including a height field214and a width field216), a bit rate field218, and a frame rate field220, respectively. In particular embodiments, video codec type field212, height field214, width field216, bit rate field218, and frame rate field220each store a four-byte value holding the relevant data.

Audio header230may include information similar to video header210but relating to audio information in the corresponding target files72. For example, audio header230may include information such as an audio codec type, a bit rate, a sampling rate, and a number of audio channels encoded in the audio content of the corresponding target files72. In the illustrated auxiliary file80, this data is stored in audio codec type field232, bit rate field234, sampling rate field236, and channel quantity field238, respectively. In particular embodiments, audio codec type field232, bit rate field234, sampling rate field236, and channel quantity field238each store a four-byte value indicating the relevant data.

Additionally, certain target formats may utilize configuration information that is provided to streamer40to allow streamer40to transmit content from the corresponding target files72and/or to the requesting client60to allow the requesting client60to receive or play content from the relevant target files72. For example, in particular embodiments, if the relevant target file72represents a 3GP file, streamer40may utilize session information, such as a session description protocol (SDP) file to properly stream media content from the 3GP file to the requesting client60. This SDP file may contain a specific type of configuration information, such as a configuration (“config”) string, defining features of the encoded video and/or audio stream contained in the corresponding target file72. This configuration string may be used by one or both of streamer40and the requesting client60to properly configure themselves and/or the communication session over which content from the relevant target file72will be transmitted.

As a result, in certain embodiments, delays in transmitting modified content from target files72may be avoided by including configuration data in a configuration data field224of video header210and/or a configuration data field244of audio header230. In particular embodiments, the amount of information stored in these fields may vary, and as a result, the value stored in the field may be padded to bring the amount of data up to a particular size. For example, in particular embodiments, configuration information stored in configuration data field224or244is padded so that 108 bytes of data are stored in these fields. The size of the actual configuration data stored in configuration data fields224and244may then be stored in a configuration data size field226and246, respectively.

Configuration data field224of video header210and configuration data field244of audio header230may store any appropriate information to be utilized to configure streamer40, the requesting client60, and/or the transmission session used to transmit content from the corresponding target files72to the requesting client60. For example, in certain embodiments, transcoder30may generate a config string from video or audio media in the corresponding target files72during transcoding of the relevant media and store the generated config string within the appropriate configuration data field224or244of auxiliary file80. Utilizing this config string, streamer40may then initiate an RTSP session with the requesting client60to transmit content from the relevant target files72.

Content records250include information regarding the structure and contents of the corresponding target files72. Auxiliary files80may include multiple content records250, each associated with a different portion of the content in target files72. As noted above, these different content portions of the corresponding target files72may each represent a discrete portion of content (e.g., an individual frame or a predetermined number of frames) or arbitrarily-sized portions (e.g., a five-second portion of video data or a kilobyte-sized portion of audio data). In particular embodiments, each audio frame and each video frame in target files72has a corresponding content record250in auxiliary file80.

Content records250may each include any appropriate information structured in any appropriate manner to allow streamer40to access the portion of the corresponding target files72associated with that content record250and transmit the associated portion to the requesting client60. For purposes of illustration, the example content records250shown inFIG. 2each include a media type field252indicating a type of content associated with the relevant content record250(e.g., Audio-1, Video-2), a position indicator field254indicating the location of the associated content portion in the corresponding target file72, a size field256indicating the size of the associated content portion, and a dropped content field258indicating an amount of content (e.g., a number of frames) dropped during transcoding of the associated portion of media (or dropped in between the transcoding of the associated portion and the immediately preceding or succeeding portion). Additionally, to mark the end of the content records250for the corresponding media session, auxiliary file80may include an end-of-file record (shown inFIG. 2as “content record250z”), in which media type field252is set as “end-of-file” and other fields are set to zero.

Thus, in particular embodiments, when retrieving content from the corresponding target file72for transmission to the requesting client60, streamer40may access a first content record250of auxiliary file80to determine a location of the modified content in a first target file72associated with the first content record250. In particular, streamer40may access the associated content portion by reading from a location within this first file72(e.g., as measured by a number of bytes from the beginning of the corresponding target file72) based on the position indicator field254of the first content record250. Streamer40may then retrieve the associated portion of content by reading a quantity of data determined based on the size field256of the associated content record250and transmit the associated portion to the requesting client60. Streamer40may then read another content record250and repeat the process. Streamer40may continue this process on content records250associated with additional target files72after streaming all the content associated with the first target file72until streaming content from all target files72associated with the relevant content request90.

Additionally, content records250may also include dropped content field258storing information regarding an amount of content dropped during transcoding. This information may allow streamer40to transmit portions of a target file72with proper timing and synchronization, as discussed above with respect toFIG. 1. For example, in particular embodiments, each content record250is associated with a particular video frame in the corresponding target file72, and the dropped content field258for a particular video frame indicates a number of video frames dropped between transcoding of that particular video frame and the transcoding of the immediately preceding video frame in the relevant target file72. In such embodiments, streamer40may then utilize information in the dropped content field258of each content record250to increment timestamps for the associated video frames when transmitting to the requesting client60. This may allow streamer40and/or the requesting client60to maintain audio-video synchronization during streaming of the requested content.

FIG. 3illustrates a content-delivery system210according to an alternative embodiment of the present invention in which streamer40is replaced by download server140. In the illustrated embodiment, download server140transmits media content from target files72to clients60as progressive downloads. The use of progressive downloading may allow clients60to receive and play content in formats that are not conducive to streaming and formats for which packetization is not standardized. Thus, in particular embodiments, such as the one shown inFIG. 3, streamer40may be replaced or supplemented by download server140.

Download server140supports transmission of requested content to clients60as progressive downloads. In particular embodiments, download server140delivers content from multiple files to a requesting client60as part of a single, seamless transmission without any interaction required by the user in between playback of the requested files. As with streamer40of system10inFIG. 1, download server140may additionally be responsible for processing requests for content from clients60and/or for managing the operation of transcoder30. Download server140may represent any appropriate combination of hardware and/or software suitable to provide the described functionality. Although shown as a separate element from transcoder30, in particular embodiments, transcoder30and download server140may represent a single physical component.

In operation, download server140receives a request from a requesting client60and initiates the retrieval of source files70containing media content requested by the requesting client60. In particular embodiments, this may occur in a similar fashion to that described above with respect toFIG. 1. For example, in the illustrated embodiment, client60binitiates the delivery process by transmitting content request90to download server140. In response to receiving content request90, download server140determines a target format for the requested content. Additionally, download server140may, in particular embodiments, determine whether the requested content is already available in the target format (e.g., in one or more files stored in original content store20). If download server140determines that the requested content is already available in system210in the target format, download server140may begin transmitting the requested content to client60bas part of a progressive download or another suitable form of transmission.

If, instead, download server140determines that the requested content is not available in the target format or otherwise determines that modification of the requested content is appropriate, download server140transmits a transcoding request92to transcoder30or to other appropriate components of system210to initiate modification of the requested content to the target format. Transcoder30may then identify one or more source files70containing the requested content and access content in these files to generate a single target file72appropriate for transmission to and playback by client60b.

As part of this process, transcoder30may generate media information180describing the target file72to be created. Media information180describes the media content stored by the corresponding target file72. Clients60may use media information180associated with the corresponding target file72to configure themselves to receive this target file72, request the contents of the corresponding target file72, and/or otherwise facilitate receipt and playback of requested media content. As noted above, examples of media information180may include, but are not limited to, a file size frame rate, or time duration for the resulting target file72and/or frame sizes, media types, or time stamps of individual portions of the media content in target file72. Client60bmay use the transmitted media information180to prepare a media player for playback and to begin requesting blocks of content from the target file72being transcoded by transcoder30.

As discussed above with respect toFIG. 1, transcoder30may be able, when modifying certain types of source files70, to extract existing parameters directly from these source files70and generate media information180for the corresponding target file72based on this data. For example, for source files70that have the relevant parameters located at the beginning of the file (such as MP4, 3GP, and Quicktime files), transcoder may be able to extract this data from each of the requested source files70and generate media information180based on the extracted data. Thus, transcoder30may be able, at the beginning of the transcoding process, to access all the relevant parameters to generate media information180for such files.

When modifying other types of source files70, however, transcoder30may be unable to access all the relevant parameters in the relevant source files70at the start of transcoding. For example, some types of media files (such as FLV, Audio Video Interleave (AVI), and MPEG-1 files) have relevant parameters interspersed with content throughout the file. As a result, transcoder30may be unable to access the data that transcoder30would otherwise use to generate media information180for the corresponding target file72. Because the requesting client60may expect or require media information180for any resulting target file72before the requesting client60can begin downloading and/or playing content from the relevant target file72, transcoder30may estimate certain properties of the requested source files70to generate media information180for the corresponding target file72.

In the example illustrated byFIG. 3, source files70x-zare assumed to have media parameters necessary to generate media information180interspersed with media content throughout the relevant files. As a result, when transcoder30begins transcoding source files70x-z, transcoder30may be unable to access these parameters in any of source files70x-zwithout reading through the entirety of each of source files70x-z. Consequently, transcoder30may generate media information180for target file72zusing data provided by client60b(e.g., in content request90), information available at the beginning of source files70x-z, parameters estimated by transcoder30, and/or any other appropriate information available to transcoder30.

For example, in particular embodiments, part of the media information180created for target file72zis dependent on the target format and rate for the requested content. As a result, transcoder30may copy such information as the frame rate, the duration, and the types of audio and video codecs utilized by the file directly from content request90to media information180. Additionally, transcoder30may use information in content request90to determine other fields of media information180for target file72z. For example, transcoder30may calculate the total number of frames in each of source files70x-zby determining a frame rate for the relevant target format and multiplying this frame rate by the duration of the requested content. Transcoder30may then write this total number of frames to media information180.

Additionally, transcoder30may estimate parameters not available at the start of transcoding. For example, in particular embodiments, transcoder30utilizes a-priori estimation techniques to estimate fields in media information180such as time-stamps and frame sizes for audio and video frames based on parameters of relevant coding formats, such as a bit rate, frame rate, sample rate, I-frame interval, and total number of media frames; and to estimate the interleaving format of audio and video frames in the target format.

In particular embodiments, transcoder30may start this process by modeling the bits-per-frame versus encoder parameter setting characteristics. For a given video or audio encoding format and its corresponding encoding parameter settings, the output of transcoder30may be characterized to determine the minimum, maximum, and average level of bits-per-frame for given encoder parameter settings. Based on this characterization, transcoder30may model frame sizes in terms of bits-per-frame as a function of encoding bit rate, frame rate/sample rate, temporal location of the frame, resolution, and/or any other appropriate characteristic or factor. The frame size of the resulting target file72may then be estimated based upon this model. For video media, transcoder30may determine the frequency of occurrence of I-frames in a bit stream based on the encoder parameter settings and may estimate the frame size of I-frames based on this frequency. Furthermore, transcoder30may calculate the time stamp difference between two consecutive frames based on the frame rate using the following formula: (time stamp difference between two consecutive frames)=1/(frame rate). Using this formula, transcoder30can then determine the time stamp for each frame as follows: Time stamp of current frame=time stamp of immediately previous frame+time stamp difference between consecutive frames. Transcoder30may also determine an interleaving order for different types of media data in target file72z. For example, transcoder30may determine that a predetermined amount of video content will be written to target file72zfor a given amount of audio content. Transcoder30may determine this order based on constraints of the target format and/or other appropriate considerations.

Based on the time stamp for each frame, the frame size, and an estimate of the interleaving format of audio and video frames in the media, transcoder30can then determine the estimated location of each individual audio and video frame in target file72zto be generated by transcoder30. Transcoder may generate a single set of media information180containing the estimated location of every media frame contained in target file72zand other appropriate data required or expected by client60b. Transcoder30may then transmit media information180for target file72zto download server140for subsequent transmission to client60b.

After estimating media information180for use by client60b, transcoder30may begin modifying content from a first source file70(here, source file70x) from the source format to the target format. Transcoder30then writes modified content from source file70xto a target file72(here, target file72z) in modified content store22. After modifying and writing all of the content from the first source file70xto target file72z, transcoder30repeats the process for remaining source files70, modifying content from source files70yand70zand writing the modified content to target file72z.

Additionally, because media information180generated by transcoder30will be used by client60bto request content from target files72, transcoder30may use media information180to determine how content should be written to target file72. Thus, in particular embodiments, transcoder30may pad or truncate data to be written into frames in target file72zto ensure that such frames are written to conform to the parameters transmitted to client60bas part of media information180. In particular embodiments that utilize end-of-frame markers (e.g., ID_END in case of Advanced Audio Coding (AAC)), this may involve filling a frame with dummy (e.g., fill_elements in AAC) or zero-valued bits between the last bit of content data and the end-of-frame marker in the relevant frame.

After client60breceives media information180from transcoder30, client60bmay request content from target file72zbased on the received media information180. For example, in particular embodiments, client60bmay initiate a progressive download of target file72zby transmitting an HTTP “GET” request that has a range field identifying a particular portion of target file72z(such as a particular set of audio and video frames) determined by client60bbased on the received media information180. Download server140may then transmit the first requested portion of target file72zto client60b, and client60bmay initiate playback of target file72z. In particular embodiments, client60bmay be able to begin playback of content from target file72zbefore transcoder30has finished writing all of the modified media to target file72z. Client60bmay continue requesting and playing content from target file72zuntil client60bhas received and played all of the content from target file72z.

AlthoughFIG. 3illustrates an example in which none of the requested source files70x-zare stored in the target format, in certain circumstances, one or more source files70requested by a particular content request90may already be stored in the appropriate target format. As a result, particular embodiments of system210may support the downloading of both transcoded and unmodified content as part of a single progressive download. Additionally, in particular embodiments, source files70to be modified may include both files that have the data used to generate media information180segregated in a header at the beginning of the relevant file and files that have the data used to generate media information180intermixed with content throughout the relevant file. As a result, transcoder30may combine data from multiple different types of source files70to generate media information180for target files72having a target format in which all media information180is stored in a header at the beginning of the relevant target file72before writing the created media information180to the relevant target file72. Furthermore, under certain circumstances transcoder30may generate media information180using a mix of existing parameters extracted from source files70and estimated parameters generated as described above.

Furthermore, in particular embodiments, transcoder30may be configured to add insert files74to any source files70requested by a particular content request90in a similar manner to that described above with respect to the streaming example ofFIG. 1. In such embodiments, transcoder30may utilize parameters for insert files74along with those for requested source files70when generating media information180. By doing so, transcoder30may splice content from insert files74with content from requested source files70so that requested source files180and insert files74will be played by a requesting client60as a single, seamless media session.

Thus, by generating a single set of media information180that describes content from multiple source files70and/or insert files74, transcoder30may permit a requesting client60to retrieve and play appropriately-formatted content from multiple different files as a seamless progressive download. Additionally, by estimating file characteristics used to generate media information180instead of waiting until transcoding of the relevant source files70has been completed, particular embodiments of system210may permit clients60to initiate play back of requested source files70before such files have been fully transcoded. Thus, particular embodiments of system210may provide numerous operational benefits. Specific embodiments, however, may provide none, some, or all of these benefits.

FIG. 4is a flowchart illustrating example operation of the content-delivery system10shown inFIG. 1in streaming content to a requesting client60. The steps illustrated inFIG. 4may be combined, modified, or deleted where appropriate, and additional steps may also be added to the flowchart. Additionally, the steps may be performed in any suitable order without departing from the scope of the invention.

Operation, in the illustrated example, begins at step400with appropriate components of system10receiving a request identifying or referencing a group of source files70. For example, streamer40may receive a content request90for a plurality of source files70or a playlist associated with a group of source files70from the requesting client60. In particular embodiments, the requesting client60requests these source files70by transmitting an HTTP request to server50that identifies the requested source files70. The HTTP request may also specify information about the requesting client60and/or a requested target format so that the content can be modified in a manner appropriate for transmission to and display by the relevant client60.

At step402, streamer40determines a target format associated with the request. Streamer40may determine the target format for the requested media content based on the particular characteristics of client60or other appropriate considerations. The target format may represent a format that is appropriate for transmission to and playback by the requesting client60. At step404, streamer40determines whether the requested source files70are already stored or otherwise available in the target format. For example, in particular embodiments, streamer40may determine whether original content store20holds source files70that contain the requested content in the target format. If the requested content is already stored or available in the target format, operation proceeds to step412where streamer40retrieves the relevant source files70from an appropriate content store in the target format and operation moves to step416.

If the requested content is not available in the target format, at step406, streamer40transmits transcoding request92to transcoder30. Transcoding request92indicates the requested content and the target format. At step408, transcoder30retrieves source files70containing the requested content. At step410, transcoder30begins modifying content in a first of the requested target files70and generating a corresponding target file72ain the target format. Additionally, transcoder30also begins generating a first auxiliary file80aassociated with target file72aat step414. In particular embodiments, transcoder30may generate target file72aand auxiliary file80acontemporaneously.

Streamer40notifies the requesting client60that a first portion of target file72ais available in the target format at step416. In particular embodiments, streamer40may notify the requesting client60that a portion of the content in the target format is available by transmitting session information82, such as an SDP message, to the requesting client60that describes parameters for the target files72created for this request. At step418, streamer40receives a request from the requesting client60to begin transmission of a media stream containing content from the first target file72a.

Streamer40may then begin transmitting the requested media content to the requesting client60as media stream76. If streamer40attempts to transmit content from the relevant target file72while content is still being written to that target file72, then streamer40may utilize a corresponding auxiliary file80to facilitate access to and transmission of the content in the relevant target file72. In certain embodiments, streamer40may even utilize auxiliary files80when streaming from a completed target file72. Alternatively, streamer40may be configured to use auxiliary files80only if streaming target files72that are still being transcoded and may not utilize auxiliary files80to stream content from completed target files72.

In particular embodiments, streamer40accesses media information in auxiliary file80associated with a first portion of the first target file72. For example, streamer40may access a first record of auxiliary file80storing media information for a first video frame of the first target file72. Streamer40may then use this media information to determine the location of the corresponding first content portion in the first target file72, read the first content portion from the determined location in target file72, and transmit the first content portion to the requesting client60as part of media stream76. This is represented inFIG. 4by streamer40initiating transmission of modified content from the first target file72at step420.

After streamer40begins transmitting media stream76to the requesting client60, the requesting client60initiates playback of media stream76at step422. Streamer40may continue to utilize media information in auxiliary file80ato process and/or transmit content from the first target file72. Because transcoder30makes media information (including auxiliary files80) available to streamer40and/or clients60while transcoder is still modifying content from source files70to create target files72, the requesting client60may begin playback of content from the first target file72before transcoder30has finished creating the first target file72. Thus, in the illustrated example, transcoder30does not finish creating the first target file72until step424.

Once streamer40finishes transmitting content from the first target file72, streamer40may determine whether additional target files72remain to be transmitted, at step424. If so, system10repeats steps410-424an additional time for each of the remaining target files72. In doing so, streamer40transmits content from these remaining target files72to the requesting client60.

Because transcoder30also makes media information associated with these additional target files72available while transcoder30is still finishing creation of these target files72, streamer40may be able to provide content from the remaining target files72to the requesting client60in a seamless fashion as part of a single media stream, without any added breaks or pauses between content from the individual target files72. As a result, a user of the requesting client60may be able to view content from all the media files in succession without having to provide any additional input to the requesting client60.

Furthermore, in particular embodiments, transcoder30and/or streamer40may take additional steps to maintain temporal consistency between frames of the various target files72, as described above. As a result, clients60may be unable to detect that they are receiving content from multiple different target files72. Thus, in particular embodiments, common conventional media players may be used as clients60and may be capable of receiving and playing this multi-file media stream without any retrofitting or modification of hardware or software associated with the media players.

After completing the generation of all target files72for the received content request90, transcoder30may, in particular embodiments, store target files72and/or auxiliary files80a-c, as shown at step428. Target files72and auxiliary file80may be stored indefinitely to fulfill any future requests for the same source files70, cached temporarily and then discarded, or maintained in accordance with any other schedule or policies. Operation of system10with respect to delivering content requested by content request90may then end as shown inFIG. 5.