Abstract:
A synchronization system and method coordinates the provision of interactive content provided in a broadcast signal and independently of a broadcast signal.

Description:
CROSS REFERENCE TO RELATED APPLICATION  
       [0001]    This application claims priority from Provisional Application No. 60/370,993, filed Apr. 9, 2002, which is incorporated herein by reference. 
     
    
     
       BACKGROUND OF THE INVENTION  
         [0002]    According to current research, 40 million computer users in the United States have a television near their computers and use the two media synchronously. At the same time, a number of one-screen devices (such as ‘set-top’ boxes) that can combine computer and television functionality are becoming more prevalent.  
           [0003]    A number of television companies have produced interactive television (ITV) applications utilizing a home computer for the interactivity and synchronizing the interactive content to the television broadcast. These include TBS (Friends and Dinner &amp; A Movie), Comedy Central (Win Ben Stein&#39;s Money), Game Show Network (Greed), and ABC (Who Wants To Be a Millionaire). In addition, a smaller number of households have set top boxes (i.e., cable boxes) or net top boxes (e.g. AOLTV, UlitmateTV) that are capable of providing television overlays for ITV applications. A number of television companies have either provided ITV applications specifically for this platform, for example, CBS (CSI), or provided interactive television applications that can be accessed by both a home computer and a capable set top box or net top box (e.g., Friends, Greed).  
           [0004]    Typical ITV applications produced to date have been produced after the television production was completed (in case of taped shows). The interactive content was written to complement the primary video content. Close synchronization of the interactive content with the primary video content to which it related was accomplished through coordination with personnel in the broadcast studio over the telephone. The producers of ITV applications controlled the synchronization from a remote location, and triggered each segment upon a cue from the broadcast operator.  
           [0005]    Content can be delivered through one of several different techniques. In the case of a two-screen approach in which the screen for displaying the content is a computer, an open socket can be used whereby the interactive content is pushed to the end user. It is also known to provide additional content by encoding additional data with the television signal. This process is used, for example, with closed captioning. Another known method for providing content to users, such as users of a net-top box, is to provide a trigger to the end user device, which uses the trigger to retrieve content from a server.  
           [0006]    To coordinate these different methods for providing content, the interactive content can be scheduled based on time codes in a program such that an item of content is provided, for example, at 10:30 after the start of a program.  
           [0007]    Content or triggers can be embedded in a vertical blanking interval (VBI) and therefore dubbed on a copy of a master tape prior to broadcast. Requirements for testing and preparation mandate that a dubbed copy be sent to a broadcast studio several hours or days prior to broadcast. Often, however, last minute changes are desired, and it may be desirable to introduce real-time content (i.e., content created or selected while a program is being broadcast). Currently, such real-time content is typically introduced through manual efforts at a broadcast system to be incorporated into a broadcast signal.  
         SUMMARY OF THE INVENTION  
         [0008]    The system and method described herein can provide close interaction between an interactive television system and a television broadcast system, and greater automation than prior content insertion systems. This interaction (i) more closely relates the interactive and primary video content, (ii) allows greater automation of the interactive television programming distribution, and (iii) allows real-time integration of interactive television results into the television broadcast.  
           [0009]    The system relates to integration of the broadcast studio with the ITV system to provide automated: (1) synchronization of the interactive content to the television program (both one-screen and two-screen devices), (2) insertion of interactive content or triggers into the broadcast stream at the appropriate time as it is broadcast, and (3) display on the television screen of user feedback (e.g., poll results or competition results) from the interactive programming.  
           [0010]    An embodiment of the present invention includes a system and method for synchronization of ITV programming with TV programming. Enhanced TV content (ITV content) can be either produced real-time or ahead of time and resides on an ITV server system. For synchronization of content across multiple platforms (e.g., two-screen, set-top, or wireless), the ITV server system should be in synchronization with the TV signal. Television studios use automation systems (‘playlists’) to automatically control the order and timing of various video streams. Information in an automation system (playlist) includes video stream identification, status, start time, and duration. An example of a playlist is provided in the incorporated provisional application.  
           [0011]    The system and method described in the embodiment of the present invention use the playlist in the studio, which holds relevant synchronization information (e.g., segment and commercial break start and end times) to interface with the ITV system, enabling the ITV system to synchronize the interactive content with the television programming.  
           [0012]    The embodiment of the present invention allows a more efficient production of ITV programming by automating the insertion of triggers during the television broadcast, and avoiding the need for incorporating the content or triggers into a dubbed copy of a master tape. Using conventional equipment already widely used in television studios to insert data onto broadcast tapes (such as encoder/inserter equipment sold by Norpak Corporation, located in Canada), the system allows insertion of ITV content during the broadcast, controlled from a remote location by the ITV system. Thus, the system allows for more convenient introduction of last minute changes to the content, as well as data related to unfolding events in a live program. The system can base the content/trigger imbedding timing on information retrieved from the automation system (playlist) as described above, and can thus synchronize two-screen devices (e.g., PC and TV side-by-side) and one-screen devices (e.g., set-top and net-top). Content is provided to two-screen systems via a server system, and to one-screen systems via timed real-time trigger insertion, with the content provided based on the same timing information from the automation system (the playlist).  
           [0013]    The embodiment of the present invention provides a method to automatically display, on viewers&#39;s television screens, real-time data generated by an ITV program. This provides an advantage for ITV programming by completing a feedback loop of ITV content and TV programming. Showing the data generated by the ITV program to television viewers encourages participation in the interactive content, and makes the interactive content more compelling for the viewer. This viewer-driven ITV content can be incorporated in the TV programming in an automated manner using equipment in the broadcast facility. For example, after ITV participants are polled, the poll results can be sent to character generator (CG) equipment (e.g., equipment produced by Chyron Corporation or Pinnacle Systems) in the broadcast facility to automatically insert content into an on-screen graphic template for display to all television viewers. This process also applies to trivia questions and resulting scores and leader boards.  
           [0014]    The system and method also enable the content provider to select, in advance, ITV content assets to be marked for on-air use. For example, in an interactive broadcast there might be 25 polls created, of which a subset will be integrated into the on-air broadcast. In addition to marking content for on-air use, the timing of the on-air display is coordinated such that the full process is completely automated.  
           [0015]    A system according to the present invention includes some or all of the following components: (1) a broadcast system with character generation equipment (Character Generator), playlist information, and data insertion equipment, (2) an ITV server system, (3) an ITV coordination authority, (4) an ITV client device, and (5) a television. Some of the advantages of the present invention can be utilized without all of the components above. For example, a system according to the present invention that does not include a character generator can still provide synchronization and live trigger insertion.  
           [0016]    The system described herein can (1) use information already in the television studio to automatically synchronize interactive programming with television programming, (2) permit last minute changes to the broadcast video by providing live trigger insertion controlled remotely by the ITV server system, (3) utilize character generators in the broadcast facility to display ITV programming results on viewers&#39;s television screens in real-time, and (4) provide coordination between the television broadcast and the ITV server system for multiple types of devices.  
           [0017]    While a number of advantages have been identified herein, a system may fall within the scope of the present invention and yet not have all of the advantages described here. Other features and advantages will become apparent from the following description, drawings, and claims. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0018]    [0018]FIG. 1 is a block diagram of an ITV broadcast server that communicates with an ITV server system to coordinate activities between ITV and TV programming.  
         [0019]    [0019]FIG. 2 shows a more detailed diagram of the different applications &amp; hardware that make up an ITV coordination system.  
         [0020]    [0020]FIG. 3 shows the data flow in the ITV coordination system.  
         [0021]    [0021]FIG. 4 shows a detailed diagram of a character generator application responsible for a feedback loop of ITV content to an on-air TV display.  
         [0022]    [0022]FIG. 5 shows the flow of the acts used for content to go from generation to an on-air asset.  
         [0023]    [0023]FIG. 6 shows a detailed diagram of the synchronization application responsible for the synchronization of ITV and TV content.  
         [0024]    [0024]FIG. 7 shows a detailed diagram of the trigger insertion application responsible for the coordination of interactivity between one-screen and two-screen ITV client devices. 
     
    
     DESCRIPTION  
       [0025]    The embodiment described here has a number of related functions, including: (1) synchronizing interactive content to a program, (2) enabling the insertion of interactive content or triggers at appropriate times during transmission, and (3) displaying user feedback from the interactive content on the television signal in an automated manner.  
         [0026]    Referring to FIG. 1, the ITV coordination system includes an ITV coordination authority  100 , an ITV server system  200 , and a broadcast server system  300 . The ITV coordination authority  100  is preferably implemented as a software program residing on a hardware device, such as a computer, for determining the ITV content sequence and display, and can include the technical director functionality described in the incorporated applications identified below. It can maintain the logic for content sequence and display, or it can be manually controlled to determine content sequence and display. The ITV coordination authority  100  can reside on any device connected via a communication network to the ITV server system  200  and broadcast server system  300 . In most cases, the ITV coordination authority  100  uses a communication network, such as the Internet, to receive information from the broadcast server system  300  (which typically resides in a broadcast operating center) via a broadcast aggregation server  240  and uses a communication network to transmit synchronization information to the ITV server system  200 . The ITV server system controls interactivity with ITV client devices  230 .  
         [0027]    A detailed example of a system for providing interactive content to multiple platforms and aspects thereof are described in a co-pending applications, Ser. No. 09/804,815, filed Mar. 13, 2001; Ser. No. 09/899,827, filed Jul. 6, 2001; Ser. No. 09/931,575, filed Aug. 16, 2001; Ser. No. 09/931,590, filed Aug. 16, 2001; and Ser. No. 60/293,152, filed May 23, 2001, each of which is assigned to the same assignee as the present invention, and each of which is incorporated herein by reference.  
         [0028]    The ITV server system  200  has one or more hardware devices (such as servers, computers, processors, and/or other logic) with software programs running on these devices. The ITV server system  200 , which can be distributed across multiple physical locations or reside in one location, is responsible for the serving and running of ITV programming. Examples of the type of location are a Network Operating Center or a head-end facility. The ITV server system  200  has an ITV content server  220  that holds the ITV content  210 . ITV content server  220  communicates with ITV client devices  230  to provide/display ITV content  210 .  
         [0029]    The ITV client device  230  is a local device that can interpret ITV content  210  formatted specifically for the type of ITV client device  230  by the ITV content server  220 . ITV client devices  230  can include personal computers, set-top boxes, net-top boxes, wireless computers, consoles, and hand held computers. In the case of set-top boxes and net-top boxes as ITV client devices  230 , the display of this client device is usually a television. In case of a personal computer as the ITV client device  230 , the ITV content  210  is usually displayed on a computer monitor.  
         [0030]    The ITV content server  220  receives information from the ITV coordination authority  100 , and based on this information, it presents ITV content  210  to the different ITV client devices  230 . The ITV content server  220  can service these different types of ITV client devices at the same time, allowing for synchronization to broadcast experiences across multiple client devices in real-time.  
         [0031]    The broadcast aggregation server  240  communicates with the broadcast server system  300  to exchange synchronization/timing information and character generator (CG) data. The broadcast server system  300  is responsible for the retrieval and delivery of ITV data to broadcast devices  310  and to the ITV server system  200  (in particular the broadcast aggregation server  240 ). The broadcast devices  310  are the source of the TV programming displayed by a television or other such device.  
         [0032]    Referring to FIG. 2, which illustrates in more detail different applications that make up the ITV coordination system, the broadcast server system  300  can have one or more hardware devices with several software programs interacting with broadcast devices  310  and the ITV server system  200 .  
         [0033]    A character generator (CG) application  400  communicates with a CG  410  to display ITV content  210  on-air. A character generator is a well-known device used to insert content, such as sports scores at the bottom of the television during a sports news broadcast.  
         [0034]    A synchronization application  500  retrieves information such as start times and an identification (ID) of the events from a playlist automation system  510 , and a trigger insertion application  600  communicates with an encoder  610  to provide triggers to the encoder. As will be discussed in more detail in the description of FIG. 6, in one embodiment, the ID and start time of an event passed on to the ITV coordination authority  100  by the synchronization application  500  are used by the ITV coordination authority  100  to generate a corresponding ‘segment/event’ trigger for the ITV server system  200 . In this embodiment it is not necessary for the actual time to be passed on to the ITV server system  200 , but the appropriate trigger (based on the event ID of the playlist information) will be send to the ITV server system  200  by the ITV coordination authority  100  based on the start-time of the event in the playlist automation system  510 .  
         [0035]    Referring now to FIG. 3, the character generator application  400  can interpret data from the ITV content server  220  and pass the ITV content  210  (CG data) to the CG device  410  for display as part of the TV program in a generally known manner.  
         [0036]    The timing of the delivery of ITV content is driven by information from the playlist automation system  510 , managed by the synchronization application  500  and logic provided by the ITV coordination authority  100 . The ITV coordination authority  100  has a set of rules that can associate items in the playlist automation system  510  with synchronization triggers created by the ITV coordination authority  100 . A rule can translate segment and show information present in the playlist automation system  510  into a segment trigger that can be read by the ITV server system  200 . For example, “playlist information, segment: 2, show: Grammy” can be translated into a trigger identified as “segment=2&amp;eventID=99.” The ITV coordination authority  100  is responsible for sending this trigger at the start-time of the event dictated by the synchronization application  500 .  
         [0037]    In an alternative embodiment, the synchronization application  500  not only passes on the information from the playlist automation system  500 , but also generates the appropriate data that can directly send to the ITV server system  200 . In other words, in this alternative embodiment the synchronization application  500  fulfills also the role of the ITV coordination authority  100 . The ITV content server  220  is responsive to these synchronization triggers from the ITV coordination authority  100  to push or make available ITV content  210  to the ITV client device  230 .  
         [0038]    The trigger insertion application  600  receives data from the ITV server system  200  based on the information that ITV server system  200  has received from the synchronization application  500 . The trigger insertion application  600  then sends information to the encoder  610  for insertion of content into the video signal. This information can be delivered to the encoder  610  in many ways, depending on the capabilities of the encoder  610  or other connected devices. The result is that appropriate content and/or triggers are added to the video signal provided to end users. The content and triggers may be added to the signal in one of many different ways, depending on the operator&#39;s needs, including insertion in the vertical blanking interval, insertion in in-band frequencies, and insertion in out-of-band frequencies.  
         [0039]    This system makes it more convenient to provide content on-the-fly or in real-time to systems that require triggers to pull content from another location. While it is generally known that a party at the broadcast devices can type in information to an encoder or character generator for transmission and display, the system described here allows a producer at a remote location to generate content, provide that content to the server, and through synchronization and interaction with a trigger insertion application  600 , provide a trigger encoded with the broadcast signal to allow the user to retrieve the content from a server.  
         [0040]    Synchronization application  500  thus synchronizes ITV server system  200 , which in turn synchronizes ITV client devices  230  by sending content assets corresponding to the triggers inserted by the encoder  610  and driven by the trigger insertion application  600 ; This process allows different types of ITV devices  230  (e.g., set-top, net-top, PC (including MacOS ), and wireless) to share a synchronized, enhanced TV experience.  
         [0041]    [0041]FIG. 4 illustrates components of the system that provides the functionality to automatically insert ITV content into a broadcast program. A content producer  250  allows producers to generate items of content, such as polls, and to mark one or more of those items of content as a CG content item  270 . The distributor/generator system  260  generates and distributes all content (including marked content) in an appropriate format to the ITV server system  200 . An example of an appropriate format would be an XML node that would describe the content asset as well as mark whether or not it is supposed to go up on-air. An exemplary XML schema is set out in application Ser. No. 10/118,522, filed Apr. 8, 2002, which is incorporated herein by reference. As seen from the node below, the meta-item name is emdss, which stands for EventMatrix Data Subscription Service. The value equals onair1, which is the name of the appropriate CG graphics  420 .  
                                                                                                     &lt;leaderboard alias=“Leaderboard 1 ” title=“Round 1 ” text=“Did you know that there are       thousands of people currently competing?” start-time=“00:1 7:00” segment=“3” on-the-fly=“0”       duration=“20” type=“2” location=“3” participants=“10”&gt;                &lt;media title=“Round 1 ” location=“” alt=“” src=“/images/round.gif” url=“” credit=“”/&gt;           &lt;sponsor name=“Milton”&gt;                &lt;sponsor-media title=“Brought to you by Milton” location=“” alt=“”            src=“/images/miltonlogo.gif” url=“http://www.milton.com” credit=“”/&gt;                &lt;/sponsor&gt;           &lt;field name=“Year” value=“2002”/&gt;           &lt;meta-data&gt;                &lt;meta-item name=“emdss” value=“onair1 ”/&gt;                &lt;/meta-data&gt;                 &lt;/leaderboard&gt;                      
 
         [0042]    When a poll or trivia question is provided to users, the users can respond to the poll or the trivia question, such as through a personal computer. The ITV server system can, in an automated manner, process the responses to produce processed content, such as the results of a poll, the percent of people getting the correct answer to a question, or a leader board of scores in a trivia contest. This processed content can then be provided to broadcast devices  310  through character generator  410  (or some other device for performing content insertion). Information can be included in the process content to identify the type of processed content it is, such as poll results or leader board. This information is provided to the character generator, which selects an appropriate graphic if a number of different graphics are used and displays this process data.  
         [0043]    As an example, ITV content can be provided to individuals in the form of a poll during half time of a football game asking who they think will ultimately win the game. If marked as an item of content for on-air broadcast, the results can be tabulated and provided to a character generator with a template for how to present poll results. The character generator will insert, in the on-air broadcast, a graphic with text such as: “Of 29,734 responding to poll question asking who would win the game, 32% responded the Patriots.” This system and method thus allows for an item of content that invites responses to be processed and incorporated automatically into an on-air broadcast for all viewers of the broadcast.  
         [0044]    The processed content can be subject to automated checks and revision before being displayed, such as making sure names used for a party in a leader board is not an inappropriate name.  
         [0045]    Referring to FIG. 4, once the ITV server system  200  executes the CG marked content item  270  (based on timing information from the synchronization application  500 ), this item will be sent to the CG application  400 . This application will interpret the data and pass it to the CG graphics  420  that reside on the CG  410 . In addition to the CG marked content item  270 , the CG application  400  can pass information describing the format, behavior, and presentation for the CG marked content to be displayed on CG  410 . The data from the CG application  400  targeted to a specific CG graphic  420  will generate the CG assets  430  that will be part of the TV programming transmitted for display on TV  320 .  
         [0046]    Referring now to FIG. 5, content generation step  275  generates content using the content producer  250 . The content items that are designated to go up on-air as part of the TV program are marked (CG marked content item  270 ). The CG graphics  420  are specified with the content to determine in which CG templates the content item  270  has to be displayed. Step  276  generates and distributes the files to the ITV server system  200  to run the ITV program. These files will include the CG marked content item  270 . In step  277 , the ITV server system  200  executes a CG marked content item  270 , such as an interactive poll, triggered by synchronization information from synchronization application  500 . In step  278 , the ITV clients  230  submit poll results to the ITV server system, which aggregates the poll results as described in the incorporated co-pending applications. Step  279  publishes the CG marked content item  270  (poll results in this case) to the CG application  400 . In step  280  the CG application  400  formats this data and submits it to the identified CG graphics  420 , which uses the data to populate the relevant regions within the graphics to generate the CG assets  430 . In step  281 , the broadcast system delivers the CG assets  430  to the TV device  320  as part of the broadcast signal.  
         [0047]    [0047]FIG. 6 illustrates the synchronization of ITV and TV programming using the synchronization application  500 . The synchronization application  500  retrieves timing information from the automation system  510  (playlist) and passes this TV timing data  505  to the broadcast aggregation server  240 . The broadcast aggregation server  240  passes this TV timing data  505  to the ITV coordination authority  100 . The ITV coordination authority  100  has logic to generate appropriate ITV content trigger  235  for the appropriate ITV content server  220 . This ITV content trigger  235  initiates display of ITV content  210  by the ITV content server  220 . In an example of this system, the synchronization application  500  retrieves the start-time and ID of a playlist event (e.g., start of a segment—coming out of a commercial break) from the playlist automation system  500 . The ITV coordination authority  100  receives this information (ID and start-time) and translates this into an ITV content trigger  235  for the ITV server system  200 . The ITV coordination authority  100  then sends the ITV content trigger  235  at the start-time of the segment as retrieved from the synchronization application  500 . The ITV trigger  235  could initiate the display of an individual content item  210  or a group of content items (e.g., all content items for this segment until the next commercial break). The display of each of the items within a group could then be based on relative timing kept internally by the ITV content server  220 . Alternatively, synchronization application  500  can directly communicate with the trigger insertion application  600  (FIG. 3) and the ITV content server  220  and generate the ITV content trigger  235  and ITV data  615 .  
         [0048]    Referring to FIG. 7, the synchronization of ITV content on different ITV client devices  230  might require sending ITV data  615  to ITV client device  230  (for example in case of lower-end set-top boxes). Other ITV client devices  230  can interpret pushed ITV content  210  from the ITV content server  220  and do not need the ITV data  615  to retrieve their timed ITV content  210 . The timing data  505  transformed into ITV content trigger  235  by the ITV coordination authority  100  is sufficient to synchronize ITV content with the TV programming for those devices that can accept pushed content from the server. For ITV client devices  230  that can only retrieve (pull) ITV content  210  but cannot listen for ITV content that might be available, the ITV data  615  initiates the retrieval of the ITV content  210  from the ITV content server  220 . The ITV content server  220  actually sends the ITV data  615  (based on ITV content trigger  235 ) to the trigger insertion application  600 . The trigger insertion application  600  then communicates with the encoder  610  (one of the broadcast devices  310 ) to incorporate these ITV data  615  into the video signal so the ITV client device can initiate retrieval of the available ITV content  210 . An advantage of the system and method described here is that the timing of ITV content  210  can not only be determined in real-time, but the content item itself can be generated in real-time. This ability makes the system and method of the present invention well suited for real-time enhanced ITV content production for multiple types of end user devices. The ITV coordination authority  100  can be used to produce real-time content. The ITV content server  200  can then push that content, as well as generate the appropriate trigger, to imbed ITV data  615  using the trigger insertion application  600  in a real-time manner.  
         [0049]    While a particular embodiment is described herein, it should be understood that modifications can be made without departing from the scope of the invention as defined by the claims. For example, while it has been described that the trigger insertion application receives data from the ITV server system based on information that the ITV server system receives from the synchronization application, the trigger insertion application can receive instructions directly from the synchronization application.  
         [0050]    What is claimed is: