Abstract:
A system and method for producing content for episodes of an interactive program that allows content creation during script writing and editing, film editing, after film editing, and in live production, and for content production, responsive to inputs from script writing software and non-linear editing software as well as direct user inputs, for storing content, presentation, and behavior information using an XML schema.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    The present invention relates to a system and method for creating episodes with enhanced content, including interactive television programs.  
           [0002]    Interactive television programs have existed for several years. The programs span all genres of television programming. Turner Broadcasting System (TBS), for example, has provided enhanced programming for the situation comedy series Friends, and the movie program Dinner &amp; A Movie. Several networks have provided enhanced TV productions of game shows, including Game Show Network&#39;s enhanced programming for Greed and Comedy Central&#39;s enhanced version of Win Ben Stein&#39;s Money. Reality shows have also been enhanced, including CBS&#39;s Survivor and The WB&#39;s Popstars.  
           [0003]    Current methods of creating interactive television programs create interactive content after an episode is complete and edited, and then use time codes to identify when the content will be provided.  
         SUMMARY OF THE INVENTION  
         [0004]    The embodiments of the present invention are for creating enhanced content for broadcast events, including events broadcast over television, radio, Internet, or other medium. Television is used herein as a primary example for descriptive purposes, but the description applies in most instances to the other media as well. In the case of television, for example, the embodiments of the present invention allow interactive content to be created concurrently with the production of the related primary video episode of the television program at pre-finalized stages, such as during writing, filming, and editing. Content can further be provided after the episode is finalized, and also on-the-fly during broadcast.  
           [0005]    An embodiment of the present invention includes at least some of the following components: a script writing component that is capable of managing both primary video scripts and text for interactive content; a post production editing component, which allows the insertion and management of interactive content or references to interactive content; a content tool, which manages the graphics and/or video, text, and functionality of multiple moments of interactive content, each associated with a point in the primary video stream; and a simulator for testing a completed episode. The system can be customized so that completed interactive event output files make up the required components for events on various interactive television systems.  
           [0006]    An example of an interactive television system that could run the events created with the present invention is a system in which there is a user-based hardware device with a controller (such as a personal computer), server-based interactive components, and a technical director for interacting with the server components and the user-based hardware device via the server. Examples of such as system and aspects thereof are described in a co-pending applications, Ser. No. 09/804,815, filed Mar. 13, 2001; Ser. No. 09/899,827, filed Jul. 6, 2001; Ser. No. 09/931,575, filed Aug. 16, 2001; Ser. No. 09/931,590, filed Aug. 16, 2001; and Ser. No. 60/293,152, filed May 23, 2001, each of which is assigned to the same assignee as the present invention, and each of which is incorporated herein by reference. These applications include descriptions of other aspects, including different types of content, hardware devices, and methods of delivery of content.  
           [0007]    A content creation system according to an embodiment of the present invention defines an alias that distinguishes each poll, trivia question, fun fact, video clip, or other piece of content (“content assets”) from others in the same episode. The alias could be a generic identifier (e.g., “poll number 5”), or a more descriptive identifier (e.g., “poll about favorite show”). This alias can be associated with a location of a script or in a video stream (whether edited or not) without reliance on a time code of a final video master. Once primary video editing is finalized, the alias can be further associated with the time code of the primary video. The interactive content associated with a point in the primary video can be pushed to the user hardware device of the interactive television system automatically at the related point in the primary video feed. Some interactive content assets can be reserved without association to a particular point in the video feed to be pushed on-the-fly based on a director&#39;s initiative or the direction of a live program.  
           [0008]    There are several potential advantages to producing interactive content concurrent with pre-finalized stages, such as script writing, filming, and editing. The creative talent that is writing the script can be employed to write the interactive content text as well. This approach can be cost effective, save time, and lead to a consistent voice through the primary video (television broadcast) and the interactive content. Another advantage is that film not used in the primary video can be edited and used as interactive content to provide alternative camera angles, outtakes, etc. Still another advantage is that the writers, director and producer may have access to interesting information related to the show, characters, filming, etc. that would make compelling interactive trivia questions or fun facts.  
           [0009]    Another aspect of the present invention includes a method for describing elements and attributes of interactive content that can be used to allow input from multiple content creation tools used at multiple points in a television production process for use by participants on multiple interactive television systems and using various user hardware devices and software. In one embodiment, Extensible Markup Language (XML) is used to describe the basic components of an interactive television (ITV) application: content, presentation (look and feel), and behavior (logic). The description of the content can be an object displayed as text, pictures, sounds, video, or a combination of these. The description of the presentation includes location on the screen, text styles, background colors, etc. The behavior description includes what actions happen initially and what happens in reaction to the particular user action or lack of action.  
           [0010]    Another aspect of the present invention includes a content production interface responsive to inputs from one or more of script writing software, non-linear editing software, and direct user inputs, to store content, presentation, and behavior information using an XML schema.  
           [0011]    Other features and advantages will become apparent from the following detailed description, drawings, and claims.  
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]    [0012]FIG. 1 is a schematic representation of different elements of content production.  
         [0013]    [0013]FIG. 2 provides an overview of different steps in the content production process.  
         [0014]    [0014]FIG. 3 is a block diagram of the high-level components in an ITV system.  
         [0015]    [0015]FIG. 4 is a block diagram of the components in an ITV system specifically focusing on the content production components.  
         [0016]    [0016]FIG. 5 is an exemplary interface to produce content for ITV content and the resulting XML schema in the DataEngine.  
         [0017]    [0017]FIG. 6 is a flow diagram of producing a presentation description for an Interactive TV application.  
         [0018]    [0018]FIG. 7 is an example of a frame within the presentation description.  
         [0019]    [0019]FIG. 8 is an example of panels within the presentation description. 
     
    
     DETAILED DESCRIPTION  
       [0020]    Conceptually, an interactive television (ITV) application can be broken into three main components: Content, Presentation (look and feel), and Behavior (logic).  
         [0021]    ITV programming applies to many different areas and includes applications such as video on demand (VOD), virtual channels, Enhanced TV, and T-commerce with a “walled garden” approach. At a high level, the concept of the different components can be applied to any of these applications. Consider an application from an end-user&#39;s experience:  
         [0022]    Content: can be a question, graphic, requested video, a purchase item, a piece of information, etc.  
         [0023]    Presentation: the content is presented in a certain way: e.g. the question has fontsize=18, color=#FF0000, displayed in the bottom panel, color=# . . . , the video in the upper right corner etc.  
         [0024]    Behavior: the application behaves in a certain way based on an end-user&#39;s action or lack thereof: e.g., an end-user clicks to purchase an item, to answer question and receive points or order a video.  
         [0025]    The content production component of ITV programming is ongoing and by its nature typically changes most frequently. For an enhanced TV application, for example, content can change on an episode by episode basis (the term “episode” is used to denote one instance of an ITV program—a grouping of specific content and interactive assets). An episode can contain items such as trivia question and answers, location ids, points, duration, images, hyperlinks etc. An episode can refer to one in a series of episodes, or can be a unique event.  
         [0026]    Although it depends on the ITV programming, the presentation description typically changes less frequently than the content (in case of enhanced TV, content typically changes across episodes, but the presentation description might stay very similar or the same).  
         [0027]    The presentation covers everything related to the look and feel of a show. It includes elements such as location options for interactive assets, type of interface (on screen right-side L-shape, left-side L-shape, overlay in bottom), colors, fonts, and font or windows sizes.  
         [0028]    The behavior is application specific and contains application specific logic. It includes items such as the specific scoring mechanism for a show or game-logic. In looking at this behavior component in more detail, this logic can reside on the client (in software or middleware on users&#39; hardware device), on the server side (software on the interactive television system&#39;s servers), or both. In other words the scoring model for an interactive application might compute the score locally, centrally, or both. This model depends on the platform, the type of application, and the back-end systems. Furthermore the actual logic/behavior is specific to the type of application.  
         [0029]    [0029]FIG. 1 shows an enhanced TV application interface, with one-screen and two-screen applications. In the first example, the end-user has an integrated TV and set-top experience (a TV with one-screen device  50 ), while in the second example the user has a TV  60  and a PC  70  with separate displays. In either case a content item in an ITV application is defined by multiple attributes: (1) synced Timing  90 —linking content item to certain frame in the broadcast, (2) Content type  95 —determine the type of content (e.g., trivia or poll), and (3) Content  100 —the actual content itself (e.g., text, graphic, sound clip, or video clip).  
         [0030]    As depicted in FIG. 2, ITV content can be produced at different stages of the production process, both before and after the episode is finalized as to its broadcast content, such as during (a) Script writing  200 , (b) Tape editing  210 , (c) Pre-airing  220 , and (d) Live production  230 . The Timing  90  and Content types  95  can also be decoupled and defined at different points in the process as shown in FIG. 1. The Timing  90  of interactive content, for example, can be determined by adding markers during the video editing process to indicate interactive content. A file with these markers can be exported and form the basis for Stored content item  375  (as shown in FIG. 5). The actual interactive Content  100  can be associated with the Timing  90  later on in the process. The reverse order can also be applied.  
         [0031]    The writers of the TV show can determine what the ITV Content  100  and Content type  95  could be while producing the TV show. Once a final tape is produced the Timing  90  can be associated with the interactive content assets that were already written in an earlier stage. In a live production situation, Content  100  can be pre-created and the Timing  90  can be entered live, while in another case both Timing  90  and Content  100  might be created in real-time.  
         [0032]    The content thus has an alias that distinguishes each poll, trivia question, fun fact, video clip, or other piece of content (“content assets”) from others in the same episode. The alias could be a generic identifier (e.g., “poll number 5”), or a more descriptive identifier (e.g., “poll about favorite show”). This alias can be associated with a location of a script or video stream (whether edited or not) without reliance on a time code of a final video master. Once primary video editing is finalized, the alias can be further associated with the time code of the primary video. The interactive content associated with a point in the primary video can be pushed to the user hardware device of the interactive television system automatically at the related point in the primary video feed. Some interactive content assets can be reserved without association to a particular point in the video feed to be pushed on-the-fly based on a director&#39;s initiative or the direction of a live program.  
         [0033]    [0033]FIG. 3 shows components of an ITV system. The Coordination authority  300  is a back-end system that contains one or more servers and other components that perform processing. The Content Logic Engine  310  (CLE  310 ) is responsible for interpreting information coming from the Coordination authority  300  and responsible for generating content to display on the screen. The exact role of the CLE  310  will depend upon the purpose of the application, but may include communication with a remote agent, caching content for later display, and managing local data for the client. The Rendering engine  320  is responsible for rendering the content generated by the CLE  310 . The role of the CLE  310  can be performed on both the server side and the client side.  
         [0034]    As shown in FIGS. 4 and 5, a DataEngine  330  provides a central source for storage of ITV content. The content can be produced using the Content Production Interface  340  while items can also be exchanged with other interfaces (e.g., Script writing software  360  and Post-production software  370 , also known as non-linear editing software). These other interfaces can have the ability to enter information that looks like interface  340 , or that is tailored to the underlying software. The Technical Director  350  can be used for creating and/or inserting live (on the fly) content production. The import of data to and export of data from the DataEngine  330  is preferably performed in accordance with an XML schema  335 .  
         [0035]    For example, script writing software can include an ability whereby a writer selects “create asset” (e.g., with a function key or an icon), causing a new window to open with an interface for fields similar to those in content production interface  340  to allow the writer to enter information about the content asset to be created. Later, the content asset can be edited. This interface allows easy insertion into the script and allows the writer to add during the script creation process. This ability to create the content asset with an alias allows the content asset to more easily be associated with a point in the filming and/or editing process, and allows the writer to create content while also creating a script.  
         [0036]    Referring particularly to FIG. 5, an example is shown of Content Production Interface  340  used to enter ITV content into DataEngine  330 . This example is a trivia question with three answers to select from, and includes start and duration time, and other information relating to presentation of the question. The interface has specifically identified fields  380 - 395  for entering information. Alias  380  is used to identify the piece of content, such as “poll 5” or “trivia question about lead actor&#39;s hometown.” Stored content item  375  provides an example of a format in which this content is stored and can thereafter be exchanged with different interfaces in the production process as set out in FIGS. 2 and 4. A more extended XML schema and Document Type Definition (DTD) information that describe a content production format are in the example below. The pieces of information are entered through an interface, and then are stored in XML format for later use.  
         [0037]    [0037]FIG. 6 is a flow diagram to produce the presentation description of an ITV application. The process starts with determining Textstyle definitions  400 . The Textstyle definitions  400  provide a mechanism for defining monikers for various text attribute sets. A single text style definition is composed of one or more attribute sets listed in order of decreasing priority. This system simultaneously creates content for multiple client applications (i.e., types of software, middleware and hardware configurations used by different users). Therefore, the client applications&#39; Client logic engines  310  (CLE  310 ) must determine which attribute set is most appropriate for its platform. The client application should attempt to accommodate an attribute set as close as possible to the top of the list.  
         [0038]    The next step is to determine Frame definitions  410 . The Frame definition  410  breaks the screen up into regions where content will be displayed. The Frame definition  410  does not provide any description of the content that will be displayed in these regions; this is the role of panels described in the next section. Frame definitions  410  simply define screen regions, Frames  415 , and any appropriate background attributes for those regions. Frame definitions  410  are hierarchical which allows for layering of frames. One frame is a top-level Frame, called a Master frame  500  (FIG. 7), that always encompasses the entire screen. All other frames are “children” of this Master frame  500 .  
         [0039]    The third step is to determine Panel definitions  420 . A Panel definition  420  describes the layout and formatting of content that is displayed in the regions defined by the frame definition  410 . Panels  425  also provide a transitioning mechanism for migrating content into and out of an application based on predetermined criteria. Panels  425  are not defined hierarchically as are Frames  415 . Any second, third, or higher order effects desired in the display must be achieved with Frames  415 .  
         [0040]    Each Panel  425  is mapped to a single Frame  415 , and only one panel can occupy a Frame  415  at a given time. Panels  420  are composed of text fields, images, various input fields, and buttons. When content is to be displayed on a Panel  425 , the content fields are mapped into the panel based on keyword substitutions. The keywords to be substituted are defined by the content type.  
         [0041]    Panels  425  are defined with zero or more sets of criteria for ending the display. These are called “tombstone criteria.” A Panel  425  that is displayed on screen remains on screen until a new Panel  425  takes possession of the same Frame  415 , or until one of the tombstone criteria is met. Panel tombstones can be defined with a “nextpanel” attribute that allows for another panel  425  to be transitioned onto a Frame  415  when the tombstone criterion is met.  
         [0042]    The fourth step is content mapping. The Content mapping  430  is used to associate content produced by the CLE  310  with panels used to display the content. It consists of a series of map entries defining Panels  420  to render when content should be displayed. It also contains a series of assertions intended to allow content of the same type to be rendered differently based on various parameters.  
         [0043]    [0043]FIG. 7 gives a specific example of Frames  415 . It has a master frame  500  and video frame  510 . The presentation description XML representing this figure is as follows:  
                                                                       &lt;itv:frame name=“master” bgcolor=“#FF0000” display=“persist”&gt;                &lt;itv:frame name=“video” bgimage=“tv:” top=“0” left=“33%”           bottom=“67%” right=“100%”/&gt;                &lt;/itv:frame&gt;                      
 
         [0044]    [0044]FIG. 8 provides an example of panels  420 . It shows a Poll text panel  520 , three Poll choice panels ( 530 ,  540 , and  550 ), and a Poll standby panel  560 , which replaces the poll choice panels once a poll choice has been selected. Examples of the presentation description XML representing each panel is shown below.  
         [0045]    The Poll Text Panel  520 :  
                                                                                                     &lt;itv:panels&gt;                &lt;itv:panel name=“poll_text” frame=“text”&gt;                &lt;itv:panelfield top=“15%” left=“0” right=“100%”           bottom=“85%”                justify=“left” textstyle=“general”&gt;           &lt;itv:sub value=“poll/text”/&gt;                &lt;/itv:panelfield&gt;                &lt;/itv:panel&gt;                      
 
         [0046]    The Poll Choice 1, 2 and 3 Panels  530 ,  540  and  550 :  
                                                                                                                                                                                                                                                           &lt;itv:panel name=“poll_choices” frame=“bottom”&gt;                &lt;itv:tombstone criteria=“onClick38  action=“pollChosen”           nextpanel=“poll —  standby”/&gt;           &lt;itv:panelfield top=“25%” left=“25%” right=“50%”           bottom=“50%” justify=“center” textstyle=“general”&gt;                &lt;itv:paneltext&gt;                &lt;itv:sub value=“poll/answer[1]/text”/&gt;                &lt;/itv:paneltext&gt;           &lt;itv:click-data action=“pollChosen”&gt;                &lt;poll-choice value=“1” /&gt;                &lt;/itv:click-data&gt;                &lt;/itv:panelfield&gt;           &lt;itv:panelfield top=“25%” left=“50%” right=“75%”           bottom=“50%” justify=“center” textstyle=“general”&gt;                &lt;itv:paneltext&gt;                &lt;itv:sub value=“poll/answer[2]/text”/&gt;                &lt;/itv:paneltext&gt;           &lt;itv:click-data action=“pollChosen”&gt;                &lt;poll-choice value=“2”/&gt;                &lt;/itv:click-data&gt;                &lt;/itv:panelfield&gt;           &lt;itv:panelfield top=“50%” left=“36%” right=“64%”           bottom=“75%” justify=“center” textstyle=“general”&gt;                &lt;itv:paneltext&gt;                &lt;itv:sub value=“poll/answer[3]/text”/&gt;                &lt;/itV:pafneltext&gt;           &lt;itv:click-data action=“pollChosen”&gt;                &lt;poll-choice value=“3”/&gt;                &lt;/itv:click-data&gt;                &lt;/itv:panelfield&gt;                &lt;/itv:panel&gt;                      
 
         [0047]    The Poll Standby Panel  560   
                                                                                                 &lt;itv:panel name=“poll_standby” frame=“bottom”&gt;                &lt;itv:panelfield top=“0” left=“0” right=“100%”           bottom=“100%” justify=“left” textstyle=“general”&gt;                &lt;itv:paneltext&gt;Waiting for others to answer...           &lt;/itv:paneltext&gt;                &lt;/itv:panelfield&gt;                &lt;/itv:panel&gt;            &lt;/itv:panels&gt;                  
 
         [0048]    The engines, interfaces, tools, technical directors, and other processes and functionalities can be implemented in software or a combination of hardware and software on one or more separate general purpose or specialty processors, such as personal computers, workstations, and servers, or other programmable logic, with storage, such as integrated circuit, optical, or magnetic storage.  
         [0049]    EXAMPLE