Source: https://patents.google.com/patent/US9042707B2/en
Timestamp: 2020-01-28 17:10:20
Document Index: 629439697

Matched Legal Cases: ['art 11', 'art 13', 'art 20', 'art 11', 'art 11', 'art 13', 'art 20', 'art 11', 'art 13', 'art 13', 'art 13']

US9042707B2 - Apparatus and method for editing - Google Patents
Apparatus and method for editing Download PDF
US9042707B2
US9042707B2 US13/257,574 US200913257574A US9042707B2 US 9042707 B2 US9042707 B2 US 9042707B2 US 200913257574 A US200913257574 A US 200913257574A US 9042707 B2 US9042707 B2 US 9042707B2
US13/257,574
US20120093487A1 (en
Yasunori Nii
2009-03-19 Priority to JP2009-068566 priority Critical
2009-03-19 Priority to JP2009068566A priority patent/JP2012054619A/en
2009-07-02 Application filed by GVBB Holdings Sarl filed Critical GVBB Holdings Sarl
2009-07-02 Priority to PCT/JP2009/003066 priority patent/WO2010106586A1/en
2012-01-06 Assigned to GVBB HOLDINGS S.A.R.L. reassignment GVBB HOLDINGS S.A.R.L. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NII, YASUNORI, SUZUKI, YOSHIHARU, YAMASAKI, AKIHIRO
2012-04-19 Publication of US20120093487A1 publication Critical patent/US20120093487A1/en
2012-05-04 Assigned to THOMSON LICENSING reassignment THOMSON LICENSING ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: NII, YASUNORI, SUZUKI, YOSHIHARU, YAMASAKI, AKIHIRO
2015-05-26 Publication of US9042707B2 publication Critical patent/US9042707B2/en
239000000463 materials Substances 0 abstract claims description 285
230000004069 differentiation Effects 0 claims 4
A nonlinear editing apparatus, which is an editing apparatus according to a first embodiment of the invention, will now be described with reference to FIG. 2. As shown in FIG. 2, the nonlinear editing apparatus 1 according to the present embodiment comprises a video input part 11; a decoder 12; an input interface (shown as “INPUT I/F” in this figure) 13; an input part 13 a that is a mouse, a keyboard, a control device and/or the like; a CPU (Central Processing Unit) 14; a ROM (Read Only Memory) 15; a RAM (Random Access Memory) 16; an output interface (shown as “OUTPUT I/F” in this figure) 17; a display 17 a; an encoder 18; a hard disc 19; and a video output part 20.
The video input part 11 receives, for example, video data from a camera or the like. The decoder 12 decodes the video data from the video input part 11. The input interface 13 receives user instructions entered by operating the input part 13 a that is the mouse, keyboard, control device and/or the like. The CPU 14 receives the user instructions via the input interface 13 and controls all of the processings executed by the nonlinear editing apparatus 1. The ROM 15 stores information, such as various application programs to be executed by the CPU 14; screen data including an edition screen, various standby screens and the like; default data that are initial setting data of various functions. The RAM 16 is used as a working memory of the CPU 14. The output interface 17 outputs, under control of the CPU 14, video as processed and other information to the display 17 a. The encoder 18 encodes the video data as processed by the CPU 14, and the video output part 20 outputs the video data as encoded by the encoder 18 to another apparatus. The hard disc 19 can store video as processed by the CPU 14, video received from the video input part 11, and other information. In the present embodiment of the invention, the CPU 14 functions as a processing part.
As shown in FIG. 3, to begin with, one or more ranges are set in the material data (Step S11). Specifically, the input part 13 a is operated by a user, and in accordance with user's instructions received via the input interface 13, the CPU 14 sets, for example, ranges C and D in the material data by use of clip markers. The CPU 14 associates the clip marker information of the ranges C and D set by use of the clip markers with the material data already stored in the hard disc 19 and stores the associated clip marker information in the hard disc 19.
Next, the material data is edited to generate playback data (Step S12). Specifically, the input part 13 a is operated by the user, and in accordance with user's instructions received via the input interface 13, the CPU 14 subjects the material data stored in the hard disc 19 to a time remapping edition for generating the playback data obtained by playing back the material data according to a playback time (“second time”) the time axis of which is different from the time axis of the material data (“material time” or “first time”); adds the corresponding playback rate information and the like to the material data; and generates the playback data. The CPU 14 need not actually copy or cut the material data and may associate the editing information with the material data stored in the hard disc 19 and store the associated editing information in the hard disc 19.
As a precondition for locating, from playback time ranges, the corresponding material time ranges, it is necessary that locating, from the material time ranges, the corresponding playback time ranges as described above have already been done. This location is just what was done in Steps S11-S13 described above with reference to FIG. 3. The present example will now be explained as to a case where, as shown in FIG. 7, the input part 13 a is operated by the user and the CPU 14 sets, in response to user's instructions received via the input interface 13, for example, a clip marker in-point A and a clip marker out-point B in the material data. The CPU 14 associates clip marker information of the clip marker in-point A and out-point B and the like with the material data already stored in the hard disc 19 and then stores the associated clip marker information in the hard disc 19. In this way, the range between the clip markers A and B in the material time is specified as a material time range (first time range).
Playback time=t 3*key2_pos+3*t 2*(1−t)*ctrl2_pos+3*t*(1−t)2*ctrl1_pos+(1−t)3*key1_pos;
Material time=t 3*key2_value+3*t 2*(1−t)*ctrl2_value+3*t*(1−t)2*ctrl1_value+(1−t)3*key1_value (2)
Display screens to be displayed on the display 17 a when a user operates the nonlinear editing apparatus 1 for editions will now be described with reference to FIGS. 8A-8C. The screens to be displayed on the display 17 a transition, with the progress of the editing operation, as shown in FIGS. 8A-8C. The processings described below are executed, basically, by the CPU 14 that receives user's instructions and the like entered via the input interface 13 and reads application programs, video data and the like stored in the ROM 15.
FIG. 8A illustrates a situation in which a control screen 120 is operated to display a menu window 127. As shown in FIG. 8A, a display screen 100 displayed on the display 17 a comprises an image screen 110 and the control screen 120. In the control screen 120, a status screen 123 is displayed which shows current settings. If the menu window 127 is operated to click a clip mark setting (“Set clip mark In/Out”), the display screen is changed to a display screen as shown in FIG. 8B, in which editing operations using clip markers and time remapping can be started.
In FIG. 8B, in addition to the status screen 123 that indicates the current settings, a time line 124 that indicates the playback time of the material time, a time line cursor 124 a and a clip marker 125 are displayed in the control screen 120. FIG. 8B illustrates a situation in which the clip marker 125, with which the user sets an in-point (In) and an out-point (Out) to specify a material time range 125 a, is arranged on the time line 124 and further the time line cursor 124 a is currently operated by the user. In the example shown in FIG. 8B, it is arranged that the material time range 125 a specified as described above include, as its particular identification information, textual information of “On Clouds” and that if the time line cursor 124 a designates the material time range 125 a, “On Clouds” be displayed on the image screen 110. As a result, the user can visually recognize that this image is an image of the material time range 125 a specified by the clip marker 125.
FIG. 8C shows a working screen, which is used for the user to perform time remapping, and illustrates a situation in which a time remapping setting screen 126 appears in addition to the time line 124 and clip marker 125. The time remapping setting screen 126 comprises a time line bar 126 a that indicates the playback time after the time remapping, and a setting bar 126 b that indicates settings made to the time line bar 126 a.
The material time range 125 a of the material data set in the clip marker 125 is shown as two playback time ranges 125 b and 125 c on the time line 124 displayed in the display screen after the time remapping edition shown in FIG. 8C. Also in this screen, it is arranged that if the time line cursor 124 a designates the playback time ranges 125 b and 125 c, “On Clouds” be displayed in the image screen 110. As a result, the user can visually recognize that this image is an image within the material time range 125 a specified by use of the clip marker 125.
Thus, the nonlinear editing apparatus 1 according to the present embodiment generates, from a material data which is generated according to a material time that is a first time and in which a material time range that is a first time range is specified, a playback data, which is to be played back according to a playback time that is a second time different from the material time, and locates a playback time range, which includes the material data within the material time range, in the playback data. For example, in a case of setting the playback time such that the playback data includes a reverse playback range that is a range in which the material data is played back in a direction opposite to the playback direction of the material data, if the playback data is generated from the material data, a plurality of playback time ranges corresponding to a material time range of the material data may appear in the playback data. For example, as shown in FIG. 8C, the two playback time ranges 125 b and 125 c may appear which correspond to the material time range 125 a shown in FIG. 8B. Even in such a case, according to the nonlinear editing apparatus 1 of the present embodiment, the playback time ranges 125 b and 125 c, which include the material data within the material time range 125 a, in the playback data can be located, so that the correspondence between the material time range specified in the material data and each of the playback time ranges, in which that material time range is reflected, in the playback data can be determined.
The nonlinear editing apparatus 1 according to the present embodiment may be adapted to display, on the user interface, the playback time range (the playback time ranges 125 b and 125 c of FIG. 8C) in the playback time or, alternatively, both the material time range in the material time (the material time range 125 a of FIG. 8B) and the playback time range (the playback time ranges 125 b and 125 c of FIG. 8C) in the playback time. As a result, the user can visually recognize the playback data range in which the material data range specified in the material data is reflected.
The nonlinear editing apparatus 1 according to the present embodiment may be adapted to associate the material time range 125 a in the material time with the playback time ranges 125 b and 125 c in the playback time in response to an operation of the time remapping setting screen 126 of FIG. 8C; display, on the user interface, the material time range 125 a in the material time and the playback time ranges 125 b and 125 c in the playback time; change the manner of the displaying of the playback time ranges 125 b and 125 c, with which the material time range 125 a has been associated, when the material time range 125 a on the user interface is designated; and change the manner of the displaying of the material time range 125 a associated with the playback time ranges 125 b and 125 c when the playback time ranges 125 b and 125 c on the user interface are designated.
The above detailed descriptions were made as to preferable embodiments of the invention. However, the invention is not limited to those particular embodiments, and various modifications and variations may be contrived within the scope of the invention described in the claims. In the embodiments of the invention described above, the display screen displayed on the display 17 a was used as the user interface, but the invention is not limited to these embodiments and various other embodiments may be used instead. Moreover, the editing apparatus according to the invention may be operated by use of an editing program that allows a general-purpose computer including a CPU and a memory to function as the various means described above. Such an editing program may be distributed via a communication line or may be written into a recording medium such as CD-ROM or the like for distribution.
1. An editing method, comprising:
setting one or more time ranges in material data to be edited, the material data having been generated according to a first time axis of material time;
subjecting the material data to a time remapping for generating playback data that is to be played according to a second time axis of playback time different from the first time axis, the time remapping being such that a plurality of second time ranges corresponding to a first time range of the one or more time ranges may appear in the playback data;
associating editing information with the material data;
storing the associated editing information along with the material data; and
generating playback data that is to be played back from the material data and the stored associated editing information, wherein the generating comprises locating a second time range, which includes the material data within the first time-range, in the playback data, wherein generating the playback data that is to be played back includes:
calculating a relationship between the material time and the playback time according to a function depicted as a curve by graphing interpolation values of key frames;
determining, as reference positions, positions of the playback time that correspond to the starting and ending positions of the one or more time ranges in the material data, by determining all of the points at which the starting position and the ending position of the set one or more ranges intersect the graph of the interpolation values of the key frames; and
determining a playback direction of the material time at the respective reference positions by differentiation of the calculated relationship.
3. The editing method according to claim 2, wherein:
said first time range of said material data includes particular identification information; and
said method further comprises displaying, on said user interface, said particular identification in when said second time range on said user interface is designated.
4. The editing method according to claim 2, further comprising:
associating said first time range in said first time axis with said second time range in said second time axis;
displaying, on said user interface, said first time range in said first time axis and said second time range in said second time axis;
changing the manner of the displaying of said second time range associated with said first time range, when said first time range on said user interface is designated; and
changing the manner of the displaying of said first time range associated with said second time range, when said second time range on said user interface is designated.
5. An editing apparatus, comprising:
a processing part configured to:
set one or more time ranges in material data to be edited, the material data having been generated according to a first time axis of material time;
subject the material data to a time remapping for generating playback data that is to be played according to a second time axis of playback time different from the first time axis, the time remapping being such that a plurality of second time ranges corresponding to a first time range of the one or more time ranges may appear in the playback data;
associate editing information with the material data
store the associated editing information along with the material data; and
generate playback data that is to be played back from the material data and the stored associated editing information, wherein the playback data is generated by locating a second time range, which includes the material data within the first time-range, in the playback data, wherein generating the playback data that is to be played back includes:
calculating a relationship between the material time and the playback time according to a function depicted as a curve by graphing interpolation values of key frames:
determining, as reference positions, positions of the playback time that correspond to the starting and ending positions of the one more time ranges in the material data, by determining all of the points at which the starting position and the ending position of the set one or more ranges intersect the graph of the interpolation values of the key frames; and
7. The editing apparatus according to claim 6, wherein:
said first time range of said material data includes particular identification information:
said processing part causes said particular identification information to be displayed on said user interface when said second time range on said user interface is designated.
8. The editing apparatus according to claim 6, wherein said processing part is further configured to:
associate said first time range in said first time axis with said second time range in said second time axis;
cause said first time range in said first time axis and said second time range in said second time axis to be displayed on said user interface;
change the manner of the displaying of said second time range associated with said first time range, when said first time range on said user interface is designated; and
change the manner of the displaying of said first time range associated with said second time range, when said second time range on said user interface is designated.
9. A non-transitory machine-readable medium comprising instructions executable by a computer program, wherein said computer program causes a computer to:
associate editing information with the material data;
11. The non-transitory machine-readable medium according to claim 10, wherein:
said computer program further causes said computer to execute causing said particular identification information to be displayed on said user interface when said second time range on said user interface is designated.
12. The non-transitory machine-readable medium according to claim 10, wherein said computer program further causes said computer to:
cause said first time range in said first time axis and said second time range in said second time axis to be displayed on said user interlace;
change the manner of the displaying of said second time range associated with said first time range, when said first time range on said user interlace is designated; and
15. An editing apparatus comprising a processor configured to:
define first and second segments of a material clip;
define a material time range for the first and second segments;
define a playback time range for the first segment that is different from the material time range for the first segment;
in response to defining the playback time range for the first segment, define a playback time range for the second segment such that the sum of the playback time ranges of the first and second segments is equal to the sum of the material time ranges of the first and second segments
wherein defining the playback range includes:
calculating a relationship between material time and playback time according to a function depicted as a curve by graphing interpolation values of key frames:
determining, as reference positions, positions of the playback time that correspond to the starting and ending positions of the one or more time ranges in the material data, by determining all of the points at which the starting position and the ending position of the e set one or more ranges intersect the graph of the interpolation values of the key frames; and
determining a playback direction of the material time at the respective reference positions by differentiation of the calculated relationship;
associate editing information with the material data; and
store the associated editing information along with the material data.
increase the playback rate of the second segment when the playback rate of the first segment is decreased; and
decrease the playback rate segment then the playback rate of the first segment is increased.
21. The editing method according to claim 1, further comprising:
determining a starting position and an ending position from the reference points based on the determined reference points and their determined playback directions.
22. The editing method according to claim 21, further comprising:
determining a set of a starting position and an ending position that are within the playback range.
US13/257,574 2009-03-19 2009-07-02 Apparatus and method for editing Active 2029-10-23 US9042707B2 (en)
JP2009-068566 2009-03-19
JP2009068566A JP2012054619A (en) 2009-03-19 2009-03-19 Editing apparatus, editing method, editing program and data structure
PCT/JP2009/003066 WO2010106586A1 (en) 2009-03-19 2009-07-02 Apparatus and method for editing
PCT/JP2009/003066 A-371-Of-International WO2010106586A1 (en) 2009-03-19 2009-07-02 Apparatus and method for editing
US14/680,815 Continuation US9984725B2 (en) 2009-03-19 2015-04-07 Apparatus and method for editing
US20120093487A1 US20120093487A1 (en) 2012-04-19
US9042707B2 true US9042707B2 (en) 2015-05-26
ID=41278152
US13/257,574 Active 2029-10-23 US9042707B2 (en) 2009-03-19 2009-07-02 Apparatus and method for editing
US14/680,815 Active 2030-03-02 US9984725B2 (en) 2009-03-19 2015-04-07 Apparatus and method for editing
US15/966,675 Pending US20180247673A1 (en) 2009-03-19 2018-04-30 Apparatus and method for editing
US (3) US9042707B2 (en)
EP (1) EP2411979B1 (en)
JP (1) JP2012054619A (en)
CN (2) CN105023595A (en)
WO (1) WO2010106586A1 (en)
EP1503587A1 (en) 2002-05-07 2005-02-02 Matsushita Electric Industrial Co., Ltd. Signal processing device, signal processing method, program, and recording medium
US7103839B1 (en) 2000-10-03 2006-09-05 Adobe Systems Incorporated Tracking the validity of cache frames in digital movie editing
2009-03-19 JP JP2009068566A patent/JP2012054619A/en active Pending
2009-07-02 WO PCT/JP2009/003066 patent/WO2010106586A1/en active Application Filing
2009-07-02 EP EP09787897.9A patent/EP2411979B1/en active Active
2009-07-02 CN CN201510385445.0A patent/CN105023595A/en not_active Application Discontinuation
2009-07-02 US US13/257,574 patent/US9042707B2/en active Active
2009-07-02 CN CN200980159342.5A patent/CN102576561B/en active IP Right Grant
2015-04-07 US US14/680,815 patent/US9984725B2/en active Active
2018-04-30 US US15/966,675 patent/US20180247673A1/en active Pending
Figura, J., et al. "Tutorial: Warping Time with Avid Liquid Timewarp, Part Two", Jan. 30, 2007, retrieved from the Internet: URL:http://www.eventdv.net/Articles/News/Feature/Tutorial-Warping-Time-with-Avid-Liquid-Timewarp,-Part-Two-37977.htm, pp. 1-4, XP002556767.
Preston, B., "Time Remapping in After Effects", retrieved from the Internet: URL:http://library.creativecow.net/articles/preston-bryan/time-remapping.php, pp. 1-4, XPOO2556768.
Preston, B., "Time Remapping in After Effects", retrieved from the Internet: URL:http://library.creativecow.net/articles/preston—bryan/time—remapping.php, pp. 1-4, XPOO2556768.
EP2411979A1 (en) 2012-02-01
JP2012054619A (en) 2012-03-15
WO2010106586A1 (en) 2010-09-23
US20180247673A1 (en) 2018-08-30
CN102576561B (en) 2015-08-05
CN105023595A (en) 2015-11-04
CN102576561A (en) 2012-07-11
US9984725B2 (en) 2018-05-29
US20120093487A1 (en) 2012-04-19
US20150213836A1 (en) 2015-07-30
EP2411979B1 (en) 2016-02-10
JP4811433B2 (en) 2011-11-09 Image selection apparatus, image selection method, and program
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMASAKI, AKIHIRO;NII, YASUNORI;SUZUKI, YOSHIHARU;REEL/FRAME:027495/0789
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMASAKI, AKIHIRO;NII, YASUNORI;SUZUKI, YOSHIHARU;REEL/FRAME:028158/0326