Source: https://patents.google.com/patent/US20080184143A1/en
Timestamp: 2019-08-20 23:05:43
Document Index: 774154551

Matched Legal Cases: ['§ 119', 'art 150', 'art 150', 'art 150', 'art 150', 'art 150', 'art 150', 'art 150', 'art 150', 'art 150']

US20080184143A1 - Methods for Identifying Actions in a Flowchart - Google Patents
Methods for Identifying Actions in a Flowchart Download PDF
US20080184143A1
US20080184143A1 US11/957,076 US95707607A US2008184143A1 US 20080184143 A1 US20080184143 A1 US 20080184143A1 US 95707607 A US95707607 A US 95707607A US 2008184143 A1 US2008184143 A1 US 2008184143A1
US11/957,076
2006-12-14 Priority to US87517906P priority Critical
2006-12-14 Priority to US87507106P priority
2007-12-14 Application filed by Jellyvision Lab Inc filed Critical Jellyvision Lab Inc
2007-12-14 Priority to US11/957,076 priority patent/US20080184143A1/en
2008-07-31 Publication of US20080184143A1 publication Critical patent/US20080184143A1/en
2008-10-24 Assigned to THE JELLYVISION LAB, INC. reassignment THE JELLYVISION LAB, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FRAM, STEVEN, GOTTLIEB, HARRY N., JACOVER, EVAN, LOTT, EDWARD, VAL, GABRIEL
The present patent document claims the benefit of the filing date under 35 U.S.C. § 119(e) of Provisional U.S. Patent Application Ser. No. 60/875,179, filed Dec. 14, 2006, which is hereby incorporated by reference.
This patent is related to co-pending, Provisional U.S. Patent Application Ser. No. 60/875,071, titled, “System and Method for Controlling Actions Within a Programming Environment,” filed Dec. 14, 2006; and U.S. patent application Ser. No. 10/038,527, titled, “Method for Identifying Cells in a Path in a Flowchart and for Synchronizing Graphical and Textual Views of a Flowchart,” filed on Jan. 2, 2002. The entire contents of these related patent applications are incorporated herein by reference for all purposes.
Turning now to the drawings, FIG. 1 is an illustration of a display output of an application of a preferred embodiment. Here, the application displays two display regions 100, 200. As used herein, the term “display region” refers to an area of display on one or more display devices (e.g., computer monitors). Each display region 100, 200 can be a separate window, or the display regions 100, 200 can be different areas in a single window. The first and second display regions 100, 200 can be fixed or movable and can be non-overlapping (as in FIG. 1) or can overlap each other. Additionally, the first and second display regions 100, 200 can be alternately shown (one then the other) in response to a command from the application and/or in response to a command from the user. The first display region 100 will sometimes be referred to herein as the “Structure Window,” and the second display region 200 will sometimes be referred to herein as the “Editor Pane.”
As shown in FIG. 1, the application displays a flowchart 150 representing a sample interactive multimedia experience in the first display region 100. As used herein, the term “flowchart” refers to any graphical representation of a branching structure. In this embodiment, the flowchart 150 consists of a sequence of linked cells 1-8. As described in greater detail further below, the cells represent media content and user interactivity and the links connecting the cells represent the flow of the multimedia experience through time. The first display region 100 is referred to as the “Structure Window” because it displays the graphical representation (i.e., the cells and the links connecting the cells) for the flowchart 150. In FIG. 1, the flowchart 150 is displayed in its entirety in the first display region 100. It should be noted that a flowchart is displayed in the first display region 100 even if only part of the flowchart is visible at one time in the first display region 100. For example, if the flowchart were larger than the first display region 100, only a portion of the flowchart would be visible in the first display region 100. In this situation, a scroll bar and/or other navigation devices can be provided in the first display region 100 to allow a user to select which part of the flowchart is visible.
The flowchart 150 in FIG. 1 comprises a plurality of cells 1-8. In this embodiment, the cells 1-8 are illustrated as graphical symbols, some of which are rectangular-shaped and others of which are diamond-shaped. It will be understood that the graphical symbols described herein are intended as illustrative rather than limiting and that, according to the present invention, other types of displays can be used to illustrate the cells in a flowchart. The rectangular-shaped cells are referred to herein as “simple cells,” and the diamond-shaped cells are referred to herein as “conditional cells.” A simple cell is a cell that contains a single branching link to a single cell. For example, in the flowchart 150 shown in FIG. 1, simple cell 4 branches to simple cell 5 and simple cell 5 branches to conditional cell 6. Unlike a simple cell, a conditional cell contains multiple branching links to multiple cells. The branching links are associated with conditions necessary for that branching link to be followed. For example, in the flowchart 150 shown in FIG. 1, conditional cell 6 branches to simple cells 7 and 8. The branch to simple cell 7 is followed if the condition “ready to order” is true, while the branch to simple cell 8 is followed if the condition is false.
It should be noted that a cell can contain two or more individual cells (simple cells or conditional cells). Such a cell is referred to herein as a “group cell” and can be used to simplify the display of a flowchart by grouping multiple cells together in a single cell. In addition to simple and conditional cells, a group cell can contain other group cells (i.e., groups cells can be nested) and other types of cells, such as “go to” cells, “alias” cells, and other types of cells described in the next paragraph. “Go to” cells can be used instead of a line to show the flow from that cell to another cell. In this way, “go to” cells can be used to keep a flowchart clean and readable. “Alias” cells can be used to mimic the function/operation of another cell.
The cells in a flowchart can contain any suitable content. For example, as in a traditional flowchart drawn on paper, a cell can contain text. A cell can also provide selection choices to a user and evaluate which choice was selected. For example, when conditional cell 6 in FIG. 1 is played, the user is prompted to input “yes” or “no” using a user interface device, such as a keyboard, mouse, microphone, remote control, or any other type of device. Conditional cell 6 also determines whether the user input is “yes” or “no.” If the user input is “yes,” the branch leading to cell 7 is followed, and the application runs the script “Great. Let's proceed.”
The term “media cell” shall be used herein to refer generally to a cell that can reference at least one media asset or represent a reference to the at least one media asset to be played by the application. Thus, conceptually, a media cell acts as a container for actions, or references to actions, that trigger media playback. A media cell can contain multiple actions for executing media playback that are implemented by the application (or by another application) in a coordinated manner when the cell is played. In particular, each media cell includes at least one primary action and possibly one or more secondary action whose timing is coordinated with the primary action. The primary action preferably triggers playback of media that conveys a comprehensive message at a particular moment of the experience, such as audio or video of an actor speaking. A secondary action preferably triggers playback media that illustrates, reinforces and/or supports the comprehensive message being delivered through the primary action, such as text displaying all or a subset of the script performed by the actor and/or graphics that help illustrate the message of the actor's script. Preferably, the timing of a secondary action of a media cell is based on a time X before or after the beginning or end of the primary action of that media cell. Different actions in a cell can trigger playback of media that execute simultaneously with each other. It will be understood that the timing of a secondary action can be coordinated with the timing of a primary action using other timing mechanisms. For example, the primary and secondary actions could be timed off of a common clock, the secondary action could be timed off of another secondary action that is timed off of the primary action, etc. Thus, each media cell contains one or more actions whose timing is coordinated.
In particular, when the flowchart is played from its beginning, the user first hears an audio file voicing the greeting “top of the morning” triggered by the primary action 20 in the media cell 2. The user then hears a sound effects triggered by secondary actions 22 and 24 in the media cell 2 timed off of the primary action. First, a “leaves falling” sound effect corresponding to the secondary action 22 begins to play 1.50 seconds before the end of playback of the phrase “top of the morning.” Then, a “birds chirping” sound effect the secondary action 24 begins to play 0.50 seconds before the end of playback of the phrase “top of the morning.” Thus, the “birds chirping” sound effect would be presented during the experience 1.00 second after the “leaves falling sound effect has started. Next, an audio file voicing the phrase “walking down the main path” is played at cell 3. The user then hears the greeting “Welcome” at cell 4, followed by the question “Are you ready to order?” at cell 5. Depending on the user's response, an audio file voicing the phrase “Great. Let's proceed.” is played at cell 7 or an audio file voicing the phrase “No problem. Take your time.” is played at cell 8.
The timing values for the secondary actions 22 and 24 are stored in the secondary actions and are displayed directly beneath the descriptions of the secondary actions in the media cell 2. In this example, the timing values for the secondary actions 22 and 24 of the media cell 2 are based on a time before the end of the primary action 20. Thus, the timing value for the secondary action 22 is E−1.50 seconds, which indicates that the media associated with this secondary action (“leaves falling”) would begin playing 1.50 before the end of playback of the primary action 20 (“top of the morning”). Likewise, the timing value of E−0.50 seconds for the secondary action 24 means that the media associated with this secondary action (“birds chirping”) would begin to play 0.50 seconds before the end of playback of the primary action 20 (“top of the morning”) or 1.00 seconds after the secondary action 22 begins. In addition, if the timing of a secondary action is based on a time X after the end of a primary action, the timing value can be represented, for example, as E+X; if the timing of the secondary action is based on a time X before the start of the primary action, the timing value can be represented, for example, as S−X; and if the timing of a secondary action is based on a time X after the start of a primary action, the timing value can be represented, for example, as S+X.
Additionally, the content of the cells along a single path is displayed in the second display region 200 (the Editor Pane) to allow a user to read through the content of those cells in isolation from the cells in the other paths. As shown in FIG. 1, the second display region 200 contains a textual view 250 of the cells 1-7 in the order in which the cells appear in the flowchart 150. The textual view generally shows a “description” of the primary action of a cell (e.g., the text that is contained in the primary action, the line of dialogue that will be voiced by the primary action, the description of the animation or of the SFX that will be played by the primary action, etc.). The textual view also may include “descriptions” of the secondary actions of the cell. The content displayed in the textual view 250 can also contain other cell “properties,” such as the font of displayed text, the name of the actor who will provide the media content, the cell's timing relative to other cells, the volume at which audio content of the cell should be played, the frames per second of animation, and other information. It should be noted that the “textual” view can also contain graphics. For example, the text of the cell can also be supported with a graphics button or thumbnail image next to the text that indicates the type of media in the cell (e.g., a button with a little horn indicating a SFX) and is clickable to open a properties window of the cell, to playback the media, or to perform other functions.
In the present embodiment, editing of the secondary actions 22 and 24 can be performed in-place within each media cell. To edit a secondary action, a user individually selects a cell in the flowchart. In the flowchart 150 displayed in FIG. 2, a user uses a pointing device (such as a mouse or trackball) to move a pointer 30 over the media cell 2, and then selects that cell by pressing a selector switch (such as the mouse button). Alternatively, the user can select the media cell 2 using any other type of user interface device. For example, if the cells are numbered, the user can select the media cell 2 by typing in the cell number using a keyboard or by speaking the number of the cell into a microphone. Additionally, instead of the user selecting the media cell 2, the application can automatically select the media cell 2 (e.g., based on the output of some operation being run by the application or based on the output sent to the application by another application). A user can then select one of the secondary actions 20 and 24 shown in the media cell 2 using the pointing device or another user interface device as described above in order to edit their content and/or properties, including script description, timing properties, and visual display properties. For example, as shown in FIG. 3, a user selects the secondary action 22 and changes its content from “leaves falling” to “rain drops keep falling” in-place within the media cell 2. Then, as shown in FIG. 4, the user changes the timing value for this secondary action from E−1.50 seconds to E−0.20 seconds.
Alternatively, when the user selects a secondary action, a third display region can be provided for editing its content and/or properties. For example, FIG. 5 illustrates a media cell 9, including a primary action 90 and secondary actions 92 to 96, and a display region 300 or “properties window” that displays and allows a user to edit the content and/or properties of the actions 90 to 96. In the example, the display region 300 displays different properties 94 a and 94 b of the secondary action 94 selected by the user. The display region 300 receives user input, and the application can apply the input received to the secondary action 94 in the media cell 9.
Referring again to FIGS. 3 and 4, the application automatically rearranges secondary actions within a media cell to reflect their chronological order of execution. As shown in FIG. 4, when the timing of the secondary action 22 (“leaves falling”) is changed from E−1.50 seconds to E−0.20 seconds, the order of execution of the secondary action 22 and 24 changes because the secondary action 22 would now be executed 0.30 seconds after the secondary action 24 (“birds chirping”) begins. As a result, the application automatically shifts the secondary action 22 to fall below the secondary action 24 (“birds chirping”). Therefore, if the timing of a secondary action is updated, so is its order in the list of secondary actions. This is consistent with the notion that playback in the flowchart should proceed sequentially from one action to the next.
displaying a graphical view of a flowchart representing a multimedia experience;
defining, within the flowchart, a cell including a primary action configured to trigger a playback of a primary media asset and secondary actions configured to trigger a playback of secondary media assets, wherein each of the secondary actions has a timing coordinated with the primary action;
displaying a representation of the primary action;
displaying representations of the secondary actions in chronological order according to their timing; and
visually associating each of the representations of the secondary actions with the representation of the primary action.
2. The method of claim 1 further comprising changing the timing of at least one of the secondary actions and updating the display of the representations of the secondary actions in chronological order according to their timing.
selecting a target secondary action in the graphical representation of the cell;
identifying the target secondary action by displaying a representation of the target secondary action differently from representations of other secondary actions; and
applying an input received in the cell to the target secondary action.
10. The method of claim 1, wherein the graphical view of the flowchart is displayed in a first display region, and further comprising displaying a textual view of the cell in a second display region.
selecting a first target secondary action;
displaying a first properties window, including a description of the secondary media asset triggered by the first target secondary action and a value for the timing of the first target secondary action;
selecting a second target secondary action;
displaying a second properties window, including a description of the secondary media asset triggered by the second target secondary action and a value for the timing of the second target secondary action;
applying an input received in the first properties windows to the first target secondary action; and
applying an input received in the second properties windows to the second target secondary action.
selecting a plurality of target secondary actions;
displaying descriptions of the secondary media assets triggered by the target secondary actions and values for the timing of the target secondary actions in a properties window; and
applying an input received in the properties window to at least one of the target secondary actions.
13. The method of claim 1, wherein at least one of the secondary actions is configured to trigger a presentation of text, and wherein displaying a representation of the at least one of the secondary actions includes displaying the text.
displaying a graphical view of a flowchart representing a multimedia experience, the flowchart including a cell representing a playback of coordinated media content;
defining, within the flowchart, a plurality of actions associated with the cell and configured to trigger the playback of coordinated media content, including a primary action and at least one secondary action having a timing coordinated with the primary action; and
differentiating within a graphical representation of the cell a display of the primary action and a display of the at least one secondary action.
16. The method of claim 15, wherein the at least one secondary action includes a plurality of secondary actions, and further comprising sorting the secondary actions in chronological order according to their timing.
defining, within the flowchart, a cell including a primary action configured to trigger a playback of a primary media asset and secondary actions configured to trigger a playback of secondary media assets,
associating each of the secondary actions with a timing coordinated with the primary action, wherein at least one secondary action is associated with a timing defined by a time relative to and prior to the end of the playback of the primary media asset triggered by the primary action;
displaying representations of the secondary actions; and
24. The method of claim 23, wherein at least one other secondary action is associated with a timing defined by a time relative to a start of the playback of the primary media asset.
27. The method of claim 23, wherein a representation of a secondary action includes a description of the secondary media asset triggered by the secondary action and a value for the timing of the secondary action.
US11/957,076 2006-12-14 2007-12-14 Methods for Identifying Actions in a Flowchart Abandoned US20080184143A1 (en)
US87517906P true 2006-12-14 2006-12-14
US87507106P true 2006-12-14 2006-12-14
US11/957,076 US20080184143A1 (en) 2006-12-14 2007-12-14 Methods for Identifying Actions in a Flowchart
US20080184143A1 true US20080184143A1 (en) 2008-07-31
US11/957,066 Active 2030-01-13 US8127238B2 (en) 2006-12-14 2007-12-14 System and method for controlling actions within a programming environment
US11/957,076 Abandoned US20080184143A1 (en) 2006-12-14 2007-12-14 Methods for Identifying Actions in a Flowchart
JP3307790B2 (en) 1995-03-24 2002-07-24 日立ソフトウエアエンジニアリング株式会社 Flow diagram drawing device
2007-12-14 US US11/957,066 patent/US8127238B2/en active Active
2007-12-14 US US11/957,076 patent/US20080184143A1/en not_active Abandoned
US20080163084A1 (en) 2008-07-03
US8127238B2 (en) 2012-02-28
Delmestri et al. 2005 Reputation and strength of ties in predicting commercial success and artistic merit of independents in the Italian feature film industry
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GOTTLIEB, HARRY N.;LOTT, EDWARD;JACOVER, EVAN;AND OTHERS;REEL/FRAME:021734/0658