Convergence processor for media streams

Multicast groups provide a mechanism to deliver simultaneous media content to a plurality of users. In streaming media systems, such as audio and video systems, each multicast recipient simultaneously receives similar content. Accordingly, each recipient must initiate transmission at the same time. A data communications device delivers multiple media streams of similar content to each of a plurality of users by converging a set of media streams and merging the converged streams onto a single stream for simultaneous delivery to each of the plurality of users from a single media stream resource. A convergence processor identifies a set of streams carrying similar content in a slightly time-shifted manner with respect to the other streams. The convergence processor filters certain streams to augment the streams to converge at a particular point, at which time the streams align respect to media content therein, and merge onto a multicast stream for the duration.

BACKGROUND OF THE INVENTION

Modern computer networks are capable of delivering ever-increasing quantities of multimedia data to end-user recipients. In a conventional data communications network system, a media content provider delivers real-time audio and video images in a continuous sequence of packets called a media stream, or so-called streaming audio and video.

Conventional media providers employ the existing conventional network infrastructure, including TELOPS POTS lines, coaxial lines carrying cable TV signals, fiber optic lines, high-volume trunks and other interconnections supporting Internet and other related networks for transmitting media content. Consumer demand for media content services, such as streaming audio and video services, often drives network bandwidth capacity of the conventional infrastructure to its limit. Accordingly, conventional media content providers employ optimizing techniques such as compression, multiplexing, and deployment of new and additional hardware capable of increased throughput, and driven by processors at higher and higher frequencies (speeds).

One such optimizing technique is the use of multicast streams operable to deliver a single streaming audio or video content source to a plurality, or set, of recipients called a multicast group. In a conventional multicast stream, recipients elect to join a multicast group corresponding to the stream. The routers or other intermediary devices in the network propagate the membership of such a multicast group including each of the recipients to which the multicast stream is to be delivered across the network. Conventional Internet Protocol (IP) methods propagate packets in the stream to the multicast group according to a known protocol outlined by the Internet Engineering Task Force (IETF) in RFC 1075: Distance Vector Multicast Routing Protocol (DVMRP), and in RFC 3376: Internet Group Management Protocol (IGMP), and others, as is known to those skilled in the art.

Such a conventional media provider, therefore, establishes a set of recipients of the particular media content source as a multicast group, according to the known multicast protocol. The media content provider transmits a single multicast stream, therefore requiring transmission resources only for the individual multicast stream, rather than for allocating separate transmission resources for individual streams from the content provider to each of the plurality of users included in the multicast group. In this manner, a single stream emanates from the media content provider for multicast transmission to each of the multicast group recipients.

Conventional media content providers typically receive revenue from such streaming media on a fee-for-service is basis, such as via a so-called pay-per-view broadcast. Conventional multicast groups, therefore, provide a vehicle to maximize service delivery, and therefore the revenue, from each of multiple recipients, while allocating only a single transmission stream resource, therefore freeing up other transmission stream resources for other recipients.

SUMMARY

Unfortunately, there are problems associated with delivering media streams to each of multiple users, as described above, because in such a conventional multicast group, each recipient receives the multicast media stream simultaneously. Accordingly, individual recipients are bound by the delivery schedule set by the media content provider. Individual recipients are unable to commence and/or pause the media stream at will. In such a conventional multicast media stream, therefore, a latecomer recipient to a pay-per-view movie, for example, must join the movie in progress at the current point in the already transmitting media stream. Typically, therefore, the media provider must allocate a separate media stream channel to deliver a media stream to each recipient in order to provide a true demand-based service, for example.

The conventional on-demand broadcast systems, therefore, allocate separate stream resources, or stream channels, to each user receiving the media stream. The system allocates unicast point-to-point communication links to each recipient, even if the recipient is receiving similar content, which is shifted only slightly in time from another recipient. Since the multiple transmission streams are not exactly aligned, a multicast group/stream arrangement is inapplicable. Accordingly, each user consumes a dedicated stream channel for the duration of the transmission albeit being only slightly time-shifted from another user receiving the same sequence.

It would be beneficial, therefore, to provide a data communications device operable to edit, or filter, individual media streams of a set of similar streams via insertion and deletion of frames, such that the streams become aligned with, or converge with, other streams having similar content. The aligned streams, which are then carrying duplicate streams of packets, are able to merge together into a single common multicast stream to all merged recipients. Each recipient then receives the multicast stream, thereby releasing each recipient's dedicated stream channel in favor of the common multicast stream channel, such that the data communications device is able to allocate the now idle stream channel resources, formerly transmitting recipient-specific streams, to other needs, such as new incoming requests for on-demand streams.

The present invention substantially overcomes the problems of transmitting a multiplicity of media streams having similar content to each of a plurality of users by converging a set of media streams and merging the converged streams onto a single stream for simultaneous delivery to each of the plurality of users from a single media stream provider. A plurality of users, each requesting the same media content but at differing start times, begin receiving a user-specific stream from the media provider. The convergence processor identifies a set of streams which carry similar content, but at in a slightly time-shifted manner with respect to the other streams. The convergence filter enhances, or filters, certain streams in the set to augment, or effectively “speed up” the streams to converge at a particular point, at which time the streams align, or mirror each other with respect to media content therein. Following convergence, the convergence processor merges each of the streams onto the multicast stream which each of the recipients receives for the duration of the media stream transmission. Media stream resources, such as stream channels, deallocate (free up) as the receiving user terminates the unicast transmission and relinquishes the recipient specific stream channel in favor of the converged multicast stream.

For example, in a Video On Demand (VOD) media service, multiple users may request the same feature at approximately the same time. Accordingly, the media provider sets up an individual media stream to each of the requesting recipients. Therefore, each of the recipients consumes a stream channel for their respective stream. User A selects the feature “Feature 1” at 8:23. The media provider allocates a stream channel to user A, and initiates transmission of the media stream. At 8:25, recipient B also selects “Feature 1,” and media provider allocates another media stream to recipient B. Further, at 8:26, recipient C selects “Feature 1,” and media provider initiates a third media stream channel accordingly.

The convergence processor observes the three streams carrying the similar content Feature 1, and therefore identifies the three streams as a convergence set. Selection as a convergence sets implies that the streams are carrying similar content, and are substantially close in time such that convergence is feasible. The convergence processor identifies the earliest (first begun) media stream A, serving recipient A, as an unaltered convergence stream. The unaltered convergence stream need not be augmented and serves as a baseline for convergence. The media stream B to recipient B, having started two minutes after the stream to recipient A, needs to make up two minutes playing time. Similarly, the media stream to recipient C needs it to make up three minutes over recipient A's start time.

During a convergence segment, such as 15 minutes for example, the convergence processor filters, such as by performing microediting, certain frames in the streams to recipients B and C so that streams to recipients B and C will effectively replay a shorter time interval to “catch up,” and will therefore converge, with the media stream to recipient A. The microediting process, described further below, remove frames according to predetermined convergence logic in a manner that is undetectable to the casual observer. Through such edits, over the 15 minutes convergence segment, the convergence filter removes two and three minutes, respectively, from the media streams to recipients B and C. At 15 minutes playing time, for example, the streams converge, and the convergence processor merges the media streams to recipients B and C to the same stream channel delivering media content to recipient A. Following merging of the streams, that stream channels for recipients B and C free up for other users to employ. Each of recipients A, B and C are now receiving a multicast stream of Feature 1 via the stream channel initially corresponding to recipient A alone. It should be noted that alignment, as employed herein, refers to transmitting both the unaltered and the enhanced streams until both are at the same point i.e. displaying the same frames simultaneously, in the video example above. Merging refers to switching one of the aligned recipients to receive from the same source as the other recipient(s), and converging refers collectively to aligning and merging.

In such a network for delivering media content to a plurality of recipients, the converging content provider transmits and switches media streams to each of the plurality of recipients. In such a network, also called a content delivery network, the content provider organizes the media content such that the content provider selectively directs, or switches, an emanating media stream to one or multiple recipients. The media content provider, in switching a recipient from one stream to an alternate stream, terminates a current stream to a recipient while switching the alternate stream to that recipient. The media streams, therefore, effectively merge when a stream that the media provider has converged (the enhanced stream) terminates in favor of the alternate stream (the unaltered stream) such that the user undetectably begins receiving the alternate stream in lieu of the converged, enhanced stream.

Such methods for merging media streams are further disclosed in copending U.S. patent application Ser. No. 10/185,589, filed Jun. 28, 2002, entitled “Methods and Apparatus for Transmitting Media Programs”, and copending U.S. patent application Ser. No. 10/108,539, filed Mar. 28, 2002, entitled “System and Method for Distribution of Content Over a Network”, both assigned to the assignee of the present application and both incorporated herein by reference in entirety.

The converging media provider augments the converging media stream to cause the apparent time shifting, in particular arrangements, by selective frame removal, or so called “microediting” of the media stream. Such microediting serves to remove frames and sequences of frames according to a calculated mechanism, or convergence logic, such that frame removals are undetectable and loss of substantive content is minimal. The converging media provider is operable to perform such microediting according to a variety of methods, discussed further below. One such method is disclosed by Prime Image, Inc., of San Jose, Calif., which commercially markets a product called the “Digital Time Machine” (www.primeimageinc.com). Another method for manipulating media streams is disclosed by Cakewalk, of Boston, Mass., which markets “Sonar 2.2” (www.cakewalk.com).

In further detail, the converging media provider device of the invention provides a method for delivering converging media to first and second recipients by delivering a first media stream at a first delivery rate to a first end user display device, such as a video monitor, and delivering a second media stream at a second delivery rate to a second recipient display device. The convergence device has a convergence filter which applies convergence logic to the second media stream, wherein the convergence logic causes the second media stream to enhance the second media stream and transmit frames at an augmented rate and converge to a convergence threshold in the first media stream, at which point the streams are aligned in time and are delivering duplicate content. The device then continues delivering the first media stream to both the first recipient device and to the second recipient device after applying the convergence logic to the second media stream and merging the two converged (aligned) streams. In this manner, the convergence device converges media streams starting at offsets within predetermined limits by microediting, and therefore shortening, the elapsed playback duration of the later starting stream such that the media streams align in time and merge into a single multicast stream satisfies all recipients.

In a particular configuration, the point at which the streams become aligned occurs at a point called a convergence threshold. The convergence threshold, therefore, is the point at which the convergence processor is delivering the first media stream to both the first recipient and the second recipient. Accordingly, the convergence threshold is defined by a location at which the first media stream is aligned with the second media stream. At the convergence threshold, the converging media provider begins transmitting the same multicast stream to both recipients because the convergence logic processing aligns the first and second streams in time such that they are concurrent duplicates of each other.

Applying the convergence logic further comprises processing at least one of the first and second media streams, to yield a filtered, enhanced media stream, the enhanced media stream comprising an altered, or abridged version substantially similar to the corresponding media stream by selective editing of frames in the media sequence. Alternatively, implementations of the invention may pad streams with frames or other material which actually lengthens certain streams rather than shortening others in computing the enhanced media stream.

The convergence logic typically enhances (filters) the media stream by selectively removing frames which are substantially undetectable to the casual observer. Such frames may be, for example, static (still) images of an unmoving or slightly panned image, or frames differing insubstantially from the immediately preceding or following frames. The convergence filter may employ other such enhancing and/or compression schemes, as discussed above. The selective enhancing by the convergence filter produces, therefore, a different transmission rate for the second media stream relative to the first media stream for corresponding portions of the media stream. “Enhancing,” as used herein, refers to the stream modifications, such as microediting and frame insertion and deletion, made to the media stream in a manner intended to be undetectable by the user and for the purpose of converging a plurality of media streams to a convergence threshold, as described herein.

Such enhancing, therefore, provides for effective time shortening of the resultant media sequence in contrast to the corresponding unaltered, or unfiltered, media sequence. Following the convergence threshold, after the convergence portion of the media streams, both the first and second media streams include a duplicate segment, the duplicate segments being identical. Once the convergence filter aligns the multiple streams, therefore, the remainder of the streams are duplicates of each other.

The convergence filter performs the convergence processing according to the convergence logic, as indicated above. In a particular embodiment, the convergence logic is predetermined and applying the convergence logic further comprises enhancing the second media stream according to a predetermined convergence segment, the convergence segment defining a predetermined time until the convergence threshold. In alternate configurations, the convergence filter enhances the media stream in realtime to produce the enhanced version. The convergence filter, therefore, determines which frames to add or remove and at which location in the media stream. The convergence processor determines which enhanced version to direct to a particular recipient and when to merge multiple streams into a single converged stream, i.e. when the streams align. The convergence processor stores the first and second media streams in a media content repository, in which the media content repository (database) is responsive to the convergence processor for retrieving the media streams. At a successive time, the media provider receives a recipient request and, retrieves, in response to the recipient request, the media streams for delivery to the requesting recipients. The convergence processor selectively determines which of the multiple media streams to deliver to each of the requesting recipients at a corresponding media start time, depending on the timing of the request by the respective recipients.

The convergence processor begins merging, at the convergence threshold, the second media stream from a second stream channel onto a communication path from a first stream channel of the first media stream to the first recipient. The convergence processor terminates the second media stream from the second media channel to the second recipient, and reallocates the second media channel, such that the merging of the second media stream from the second stream channel to the first stream channel is substantially undetectable to the second recipient. In this manner, the convergence processor converges, or speeds up, the second, later starting recipient is to the convergence point, and, once convergence, or alignment occurs, the convergence processor merges or “switches over” the second recipient to the first stream channel to receive the same stream simultaneously. Such merging, in a particular configuration, occurs as described in the copending U.S. patent application Ser. Nos. 10/185,589 and 10/108,539, discussed above. Alternate merging mechanisms may be provided.

In a particular configuration, the convergence processor employs multicast groups as disclosed above (IETF RFC 3376: Internet Group Management Protocol, and others) for delivering the first and second media streams, including defining, at an intermediate data communications device, a set of recipient groups corresponding to at least the first and second recipients, each of the first and second recipients defined as a member of one of the set of recipient groups, and redefining the second recipient as a member of the multicast recipient group corresponding to the first recipient. The recipients join the multicast group after the convergence processor converges and aligns the respective multicast stream serving the recipient with the multicast stream such that the convergence processor is able to undetectably switch the user to the multicast stream. Therefore, a latecomer recipient joins a multicast group lagging the first recipient's multicast group, and the converging media provider converges, or switches, the user from the latecomer multicast group to the unaltered multicast group following alignment.

The features of the invention, as described above, may be employed in systems, data communications devices and methods, as well as other computer-related components such as those of Cisco Systems, Inc. of San Jose, Calif.

DETAILED DESCRIPTION

Embodiments of the invention provide for managing multiple media streams of similar content to each of a plurality of users by converging a set of media streams and merging the converged streams onto a single multicast stream for simultaneous delivery to each of the plurality of users from a single media stream provider. A plurality of users, each requesting the same media content but at differing start times, begin receiving a user specific stream from the converging media provider. A convergence processor in the converging media provider identifies a set of streams (multicast or point-to-point), each of which are carrying similar content, either identically synchronized or in a slightly time-shifted manner with respect to the other streams. A convergence filter in the converging media provider filters certain streams in the set to augment, or effectively “speed up” the streams to converge at a particular point, at which time the streams align, or mirror each other with respect to the media content therein. Following convergence, the convergence processor merges each of the streams onto a multicast stream which each of the recipients receives for the duration of the media stream transmission. Media stream resources, such as stream channels, therefore, are freed for possible reuse as the receiving user terminates the unicast transmission in favor of the converged multicast stream.

The converging media provider, therefore, attempts to group or consolidate requests for the same media content from a plurality of media recipients (users). The media provider may receive such a group of requests, for example, within several minutes of each other. The first such request triggers the stream which the convergence processor does not filter (unaltered), which the media provider begins broadcasting to the recipient.

The converging media provider may group several requests received within an attention window, 30 seconds for example, during which a recipient receives “in-progress” feedback to avoid having the user decline the broadcast due to inactivity. The media provider transmits the remaining streams, from later requesting recipients, which the media provider starts after the unaltered stream and lagging by several minutes. The lagging streams may begin converging immediately, or may transmit a start segment initially to attempt to focus and retain the attention of the recipient. The start segment is the same as the corresponding portion of the unaltered segment, albeit lagging, and avoids a user discontinuing the transmission as a result of overly aggressive convergence enhancing causing noticeable alterations in the media stream.

Both the in-progress feedback message, and to a slightly lesser extent the start segment, attempt to approximate the attention window of the average user and maintain the focus of the recipient to allow the convergence operation to operate. The attention window operates to achieve a similar effect as the converging, altered streams by switching streams to serve multiple recipients from a single stream channel. However, the effectiveness of such an attention window is limited by human factors such as the willingness of the recipient to wait for a request to be satisfied. In other words, the in-progress message and the start segment are intended to grab and maintain the attention of the recipient until the recipient decides to view the entire stream, rather than selecting an alternate channel.

The lagging streams converge with the unaltered stream, as disclosed further below, until alignment occurs with the unaltered stream. At the point of alignment, or convergence threshold, the later running streams are no longer lagging, and merge, or “cut over” to the multicast stream to receive the remainder of the unaltered version as a multicast transmission, called the duplicate segment, since enhancing need not occur after the convergence threshold.

FIG. 1is a context diagram of a communications system10which is suitable for use with the present invention employing a converging media provider device12as defined herein. Referring toFIG. 1, the system includes a media provider12, a public access network16such as the Internet, a plurality of recipients14a-14cand transmission lines18a-18c.

The media provider12connects to a public access network such as the Internet16for delivering media content to a plurality of recipients14a-14c(14ngenerally). The network16includes a variety of mediums for delivering straining media content to the recipients14n, including TELCO (telephone company) lines16a, satellite16b, servers16c, modems16e, routers16eand other suitable communications equipment. Each of the recipients14nconnects to each of the recipients14a-14cby the communications lines18a-18crespectively. Each of the recipients14nreceives the media stream via any suitable output device such as PC14a, handheld mobile device14b, or television set-top box14c.

In operation, each of the recipients14nissues a request for media services to the converging media provider12. Such media services include, by way of example only, video-on-demand, pay-per-view (PPV), voice over IP (VOIP), broadband Internet connections, and others. Such a request may be initiated directly from the user, or may be initiated by a program or automated trigger detecting a need for an alternate stream. In response to a request, the media provider12delivers the requested media content via the network16to the corresponding recipients14n. In a particular configuration, described further below, convergable media streams converge, or consolidate, similar media content from the media provider12to each of a plurality of recipients14n.

FIG. 2is a flowchart of converging media streams in a converging media provider12as defined herein. Referring toFIG. 2and also toFIG. 1, at step100, the media provider12delivers a first media stream at a first delivery rate to a first recipient device14n. The process of converging media streams typically results in the media provider12transmitting a particular media stream without modifications, or unaltered, and enhancing other media streams to align with the unaltered media stream. At step102, the media provider begins delivering a second media stream at a second delivery rate to a second recipient device14n. The converging media provider12applies convergence logic to the second media stream, the convergence logic performing enhancing microedits which cause the second media stream to converge to a convergence threshold in the first media stream. Accordingly, the media provider12begins delivering a second media stream of similar content, but at a slightly accelerated rate due to the enhancing edits. The convergence logic, described further below, specifies the enhancing process, typically removal of selected frames, such that the media provider12transmits a shorter duration of stream content in the enhanced stream then in the corresponding portion of the unaltered stream. In this manner, the enhanced stream will attain the same point in the playback sequence, such as a movie, as the unaltered stream, because the enhanced media stream requires less time to playback corresponding segments of the media content. This point is the convergence threshold, and is defined by alignment of the first media stream and the second media stream, such that both media streams transmit the same sequence of information at the same time. At step104, the media provider12delivers the first media stream to both the first recipient device14nand the second recipient device14nafter applying the convergence logic to the second media stream. Following the convergence threshold, the media provider12has, in effect, caught up the enhanced media stream with the unaltered media stream, and therefore provides a single media stream to both the first and second recipients. The media provider12transitions, or merges, the second recipient from the second media stream to the first media stream such that a second media stream channel, which had been transmitting the second media stream, becomes available for other purposes, and the first media stream becomes a multicast stream. At this point, for the duration of the media content transmission, the first recipient device and the second recipient device are both receiving the first media stream as a multicast stream.

FIG. 3is a block diagram of the convergence processing device12employed in the communications system ofFIG. 1. Referring toFIG. 3, the media provider12delivers media streams20a-20c(20ngenerally) to the plurality of recipients14a-14cby way of stream channels26a-26c(26ngenerally). Each of the media streams20a-20cis a series of packets21including the transmitted information. Each of the recipients14a-14creceives a corresponding stream20a-20cvia a dedicated respective stream channel26a-26c. However, since each of the recipients14nbegins receiving the transmission stream20nat different times, each of the streams20a-20c, although similar in content, do not yet align with respect to time. Converging the streams20a-20caligns the streams with respect to both content and time, such that the media provider12transmits a single multicast stream which each recipient14a-14creceives. Following convergence, each recipient14a-14c, in this example, receives the multicast stream20afrom the stream channel26aas shown by dotted lines20a′ and20a.″ Stream channels26band26cdeallocate for transmission to other users while stream channel26amaintains the multicast stream to each of recipients14a-14c.

As indicated above, the media provider12initiates the media stream20a-20cin response to a request from each of the recipients14a-14c. The convergence processor22delivers the media content to each of the stream channels26a-26cfor transmission to the corresponding recipients14a-14c. The convergence filter24filters media content, described further below, in response to the convergence processor22, to allow the media streams20band20cto converge with stream20a, that is, to effectively shorten the media streams20band20cvia enhancing such that they align with stream20a. In the example shown, in which the media streams20a-20care video-on-demand, enhancing of streams20band20coccurs via a process known as microediting. The microediting selectively removes frames from the media content. The microediting, therefore, allows a shorter playback time for the microedited media streams20band20crelative to the unaltered media stream20a.

The convergence processor22delivers frames for each of the media streams20a-20c, such that, in the example shown, the media streams20band20cconverge, or align, with the unaltered stream20a. In this example, the convergence processor22selects stream20aas the unaltered stream, and effectively shortens streams20band20cvia the microediting, however the convergence filter24may also insert frames23, or alternatively carry out other editing operations, to slowdown a particular stream20n, rather than speedup, in order to converge a plurality of streams20n. For example, the convergence stream need not be the earliest begun stream and an earlier stream may be, in effect, slowed to converge.

FIGS. 4aand4bdepict a flowchart of convergence processing of multiple media streams using the convergence processing device ofFIG. 3. Referring toFIGS. 4a,4b, and3, a plurality of recipient subscribers14a-14crequest a media content delivery containing similar media content. In step110, the media provider12initiates, from a plurality of stream channels26a-26c, a plurality of media streams20a-20chaving similar media content to each of a plurality of subscribers14a-14c, via transmission lines18a-18c, each media stream20a-20chaving a respective stream channel26a-26nand an independent starting time. A typical example of one such transmission is a video-on-demand service in which a set of recipients order the same feature at different starting times within a predetermined range called a convergable window (39, discussed further inFIG. 5below).

The convergable window39is a threshold maximum interval of time in which the convergence filter24is operable to filter out frames in a manner substantially undetectable to the user. For example, an enhanced media stream is adaptable to undetectably filter 3 minutes off the first 20 minutes of the unaltered media stream, and 5 minutes off the first 30 minutes. The convergable window39is 5 minutes in this example, because after 5 minutes the microediting process is unable to further filter the media stream20ain a substantially undetectable manner. Further, while the enhancing may be performed up to the duration of the media stream20atransmission, converging at the end of the unaltered stream20adoes not deallocate any stream channels because the converging recipient14nremains bound to the converging stream20nuntil the streams converge. The convergable window39, therefore, defines a time after which the benefit gained by deallocation of the stream channel26nis outweighed by the resources required to perform convergence.

At step112, the convergence processor22designates a convergence set20a-20cof media streams for convergence, the designated convergence set20a-20csharing similar media content and having starting times within the convergable window39. The convergence set is the set of media streams20a-20cwhich the convergence processor22aligns such that merging is feasible. At step114, the convergence processor22identifies one of the media streams20a-20cin the convergence set as a convergence stream20a. In a particular embodiment the convergence stream is the earliest-begun stream20ain the convergence set.

At step116the convergence filter24employs convergence logic operable to selectively alter each of the media streams to generate enhanced media streams20b,20c, including the corresponding enhancing. Note that the enhancing may take place in real-time, selectively editing frames as the media provider transmits the media stream20, or, in alternate configurations, the media provider12stores the media stream20nafter applying enhancing.

At step118, the convergence processor22determines a convergence threshold (FIG. 5,37, described further below), by defining a point of alignment of each of the media streams20nin the convergence set with the convergence stream20a. The convergence threshold37, therefore, is the point at which enhancing completes compensating for the delayed start time and aligns each of the media streams20b-20cin the convergence set with the convergence stream. Accordingly, following the convergence threshold, all of the streams20a-20cin the convergence set display media at the same position (i.e. same point in the sequence) in the media streams20a-20c. In particular embodiments, as with the enhancing described above, the convergence processor22computes the convergence threshold37in advance of playback and stores it with the media stream18a-18c. Alternatively, in real-time configurations, the convergence processor22determines alignment of the streams20a-20cduring playback.

Once the media provider12begins designating a convergence set, the media provider12transmits, or delivers, each of the media streams20a-20cto the requesting recipients14a-14c. Note that the media provider12may add recipients14nto the convergence set, within the bounds and time limits of the convergable window39. Referring now toFIG. 4b, as shown at step120, the media provider establishes an additional stream channel26b,26cfor each of the recipients (14b,14c) other than the convergence stream18a. At step122, the convergence processor22filters, according to the convergence filter24, each of the media streams20b-20cother than the convergence stream20a, in which the convergence filter24applies the convergence logic to each media stream20based on the difference in the starting times of each of the streams20n, the enhancing, or compressing, operable to align each of the media streams20nin the convergence set20a-20cwith the convergence stream20a.

At step124, the convergence processor22performs a check to determine if the convergence processor22transmissions reach the convergence threshold37of the media streams20b,20c. If the convergence processor22has not transmitted beyond the convergence threshold, then the convergence processor22continues enhancing the media streams20b,20c, and, at step126, converging continues. If convergence is complete and the convergence processor22transmits beyond the convergence threshold37, at step128the convergence processor merges the converging media streams20b,20c.

The convergence processor merges the converging media streams20b,20cby switching a source stream channel26b,26cof each of the converged media streams20b,20cin the convergence set other than the convergence stream20afrom the respective stream channel to the stream channel26acorresponding to the convergence stream20a, the converging resulting in a multicast stream20a,20a′ and20a″ from the stream channel26acorresponding to the convergence stream20ato each of the subscribers14band14chaving media streams in the convergence set20band20c.

FIG. 5shows the media streams employed for convergence processing in the convergence processing device ofFIG. 3. Referring toFIG. 5, and also toFIG. 3, the media streams20a-20care shown in more detail, including component segments30a-30c,32a-32cand34a-34cdescribed further below. Each of the segments30n,32nand34n, corresponds to a sequence of frames23in the respective media stream20a-20c. Each of the streams20a-20ccommences at a corresponding start time ST1, ST2, and ST3. For each stream20n, there are three segments: start segments30a-30c(30ngenerally), a convergence segment32a-30c(32ngenerally) and the duplicate segment34a-34c(34ngenerally). During the start segments30a-30c, the converging media provider12transmits a dedicated stream to each of the recipients14a-14c, respectively, over the dedicated stream channel26a-26c. During the start segments30a-30cthe media provider12transmits unaltered media streams20nto avoid providing an incentive to the recipients14to decline the transmission. The media streams20a-20ccontain a convergence begin marker36a-36c, such as a tag value in a packet21, to mark the convergence point for transition from the start segments30a-30cto the convergence segment32a-32n. Alternative configurations, however, may track the transition by timing, byte count, or other suitable methods to denote the convergence begin marker (point)36a-36c.

At the convergence begin marker36a-36c, the convergence processor22begins to converge the streams20a-20cby transmitting an enhanced media stream. During the convergence segment32n, the convergence processor22both filters and delivers converging media streams20naccording to convergence logic, described further below, to provide convergence of the streams20a-20c.

The convergence segment32nis the enhanced portion of the media stream20n. Accordingly, as shown inFIG. 5, the elapsed time of the convergence segment32nvaries depending on the start time ST1-ST3. A convergence complete marker38a-38cdefines the point at which the media streams20a-20calign, and therefore converge with each other. Note that, similar to the convergence start marker36a-36c, the end of the convergence segments32a-32cmay be marked with a convergence complete marker38a-38c, or may be detected by other means. Following the convergence segments32a-32c, the duplicate segment34a-34cis shown. The duplicate segments34a-34cindicate alignment of each of the respective streams20a-20c, and therefore define the point at which the convergence processor22delivers a single multicast stream20ato each of the multiple recipients.

FIG. 6is a block diagram showing the converging media provider device12ofFIG. 3in more detail. Referring toFIG. 6, the converging media provider device12includes a media repository42and a convergence database table45. The convergence database table45includes a plurality of convergence entries46a-46d. The media repository42further connects to a media source48and also to the convergence filter24having convergence logic40.

The media repository42is upgraded to store multiple media versions in both enhanced and unaltered forms. The media source48transmits, for example, unaltered (unaltered) media sequences, such as feature films for the video on demand service and audio soundtracks for soundtrack downloads (not shown) to the media repository42. Alternate configurations may transmit other types of media material for streaming transmission. The convergence logic40filters media sequences that the convergence filter24receives from the media repository42. The convergence filter24, as described above, connects to the convergence processor22, and either transmits media sequences20nfollowing enhancing operations by the convergence logic40, or returns than to the media repository42for storing in the convergence database table45. As described above, the convergence operations may occur either in realtime, in which the convergence filter24employs the convergence logic40during actual transmission (playback) for realtime microediting, or alternately, stores the media stream46nfollowing microediting.

The convergence database table45stores the media sequences in both enhanced and unaltered forms. Typically, the unaltered form is used as a baseline to which the enhanced versions converge. A plurality of entries46a-46dstore the media sequences after operations by the convergence filter24. The convergence processor22connects to the plurality of stream channels26a-26cfor transmission to the recipients14nas described above respect toFIG. 3. The media source48connects to the media repository42for receiving the media sequences20n. Alternate configurations receive both unaltered (unaltered) and enhanced media sequences20nvia the media source48, for storing directly into the convergence database table45in the media repository42, as in such cases when external sources filter media sequences prior to transmission to the media provider12.

In a particular embodiment, the convergence database table45stores media sequences20ncorresponding to several enhancing applications of a particular media sequence. Typically, the convergence logic40applies enhancing operations corresponding to enhancing parameters. The enhancing parameters include a start convergence time44band an end convergence time44c. The start convergence time44bcorresponds to the convergence begin marker (36n,FIG. 5) and the end convergence time corresponds to the convergence complete marker38nand also to the convergence threshold37ofFIG. 5. The table45also includes a convergence duration field44dindicative of the amount of time the enhancing in the particular version corresponds to in the unaltered version. For example, a particular converged version may have a convergence segment32ncorresponding to 10 minutes of media sequence time of the unaltered version, yet occupying only eight minutes in the enhanced, or converged version, therefore decreasing the playback times by two minutes over the unaltered version.

The convergence database table45maintains the plurality of entries46a-46dfor storing media sequences corresponding to several sets of enhancing parameters. The media provider12maintains, in the media repository42, multiple versions of each media sequence having different enhancing parameters for convergence of start44band convergence duration44d. The convergence database table45, therefore, maintains versions of the same media sequence with difference convergence duration44dand times for each version. Further, the convergence filter24employs the start convergence field44b, which indicates the timing of the begin convergence marker36n, to commence convergence at other than the actual start of the media sequence, described further below.

For example, the convergence database table may maintain one version of a feature covering ten minutes of media sequence time in eight minutes of real-time. Another version of the feature covers fifteen minutes of media sequence time in twelve minutes of realtime. As shown in the convergence database table45, convergence entry46b, for Feature 1, has a duration44dof 27 minutes, yet a start parameter44bof0and end parameter44cof30, indicating that the convergence starts at 0 minutes into the feature, last 27 minutes in duration, and ends at 30 minutes into the unaltered feature, therefore accumulating three minutes of playback in realtime. Similarly, entry46bhas a start time44bof 0, an end time44cof 30 and a duration44dof 25, indicating that in the first 30 minutes of the feature playback time will occur in 25 minutes of real-time duration, therefore accumulating 5 minutes of playback.

In the above-described scenario, the media stream corresponding to the convergence database table entry46bis suitable for recipient14band commences three minutes after the start of the unaltered version46a. The media stream corresponding to convergence database table entry46cis suitable for recipient14cand commences five minutes after the start of the unaltered version46a. Other combinations of media sequences and durations for covering ranges of start times will be apparent to those skilled in the art.

In response to requests from the recipients14n, the convergence processor22determines the applicable convergence entry46nwhich allows the media stream to converge with another media stream, typically the unaltered stream46a. The convergence processor22reads the frames from the media repository42for each of the corresponding media streams46b,46cin the present example, and transmits them via the corresponding stream channels26nto each of the corresponding users14n.

For example, in the table44example shown above, recipient14areceives Feature 1 according to table entry46avia a stream channel26aat 2:00. Recipient14breceives Feature 1 according to entry46bat 2:03. Recipients14creceives Feature 1 according to table entry46cat 2:05. The convergence threshold37for the media streams20band20cis therefore at 2:30, at which point each stream20band20c, read from entry46band46crespectively, will have advanced to a point equivalent with the unaltered version46a. Conversely, as shown in the convergence database table44, the media stream20bcorresponding to the entry46b, having started three minutes late, transmits for only 27 minutes to cover the first 30 minutes of Feature 1. Similarly, the media stream media stream46ccorresponding to table entry46ctransmits for only 25 minutes to cover the first 30 minutes of Feature 1. At the convergence time of 2:30, both of streams20band20c, transmitting from stream channels26band26crespectively, converge such that they align with the media stream emanating from stream channel26a. Accordingly, recipients14a,14band14call receive the single multicast stream emanating from stream channel26a, and stream channels26band26care available for reassignment by the media provider12.

As indicated above, alternate embodiments allow 40 start time field44bof the enhanced media streams46nto vary such that the convergence segments occurs in the middle rather than the beginning of the media stream. The start segments30a-30cofFIG. 5illustrate the use of a start segment preceding a convergence segments. Since the convergence segment32b,32crepresents the enhanced transmission in which the convergence filter24in effect drops frames during the enhanced sequence, the convergence processor22transmits the enhanced media stream20b,20cwhich, although presumably undetectable to the casual observer, may be perceptibly imperfect to a trained or scrutinous viewer. Accordingly, the start segment30nallows the convergence processor22to defer displaying an unaltered media stream at the onset, in the hopes of not dissuading a viewer from remaining attentive to the delivered media stream. The successive convergence segment32b,32c, which the convergence processor displays to a viewer which is already familiar with the context may be more likely to dissuade a viewer from terminating the media sequence prematurely.

As indicated above, the convergence logic40computes enhancing of the respective media sequences. The enhancing of the media sequences depends upon the start convergence44b, end convergence44cand duration44dparameters for the corresponding media stream20n. The convergence logic40is operable to selectively enhance (microedit), or in the case of a media stream containing video content, to selectively add or remove frames in an undetectable manner. The selective editing may be by any suitable means, particularly with respect to available computing resources in the case of a real-time convergence filter24operation. For example, a simple convergence logic configuration may filter, or selectively remove one of every N frames in order to achieve the desire acceleration of the media stream to the convergence, such as one out of every six frames to achieve a speed up of two minutes over a 12 minutes media stream20nduration.

Other convergence logic40mechanisms may be implemented, such as analyzing a static or dynamic component of the media content. For example a long sequence of a landscape topology having little movements may be undetectable in a shortened version, and therefore a viable candidate for enhancing.

In alternate configurations, alternate methods other than the multicast group protocol are operable to deliver a common media stream to multiple recipients following alignment of the individual streams. The multicast group is operable to relieve, after merging the streams to26a, the dedicated stream channels,26band26c, in the example shown inFIG. 3, from transmitting for the duration of the media streams20band20c, respectively. The convergence processor may also employ other merging and multi-recipient transmission methods.

While the system and method for stream convergence processing has been particularly shown and described with references to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims. Accordingly, the present invention is not intended to be limited except by the following claims.