Source: http://www.google.com/patents/US6961446?dq=7,094,863
Timestamp: 2014-03-15 06:24:33
Document Index: 328822738

Matched Legal Cases: ['art 17', 'art 17', 'art 18', 'art 18', 'art 18', 'art 26', 'art 24']

Patent US6961446 - Method and device for media editing - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsThe present media editing device generates media including messages in an easy manner in a communication terminal such as a mobile terminal. Therein, a moving image data storage part stores moving image data recorded by a user. A region extraction part extracts any region including the user from the...http://www.google.com/patents/US6961446?utm_source=gb-gplus-sharePatent US6961446 - Method and device for media editingAdvanced Patent SearchPublication numberUS6961446 B2Publication typeGrantApplication numberUS 09/950,085Publication dateNov 1, 2005Filing dateSep 12, 2001Priority dateSep 12, 2000Fee statusPaidAlso published asCN1200537C, CN1344084A, US20020031262Publication number09950085, 950085, US 6961446 B2, US 6961446B2, US-B2-6961446, US6961446 B2, US6961446B2InventorsKazuyuki Imagawa, Yuji Takata, Hideaki Matsuo, Katsuhiro Iwasa, Tetsuya YoshimuraOriginal AssigneeMatsushita Electric Industrial Co., Ltd.Export CitationBiBTeX, EndNote, RefManPatent Citations (28), Non-Patent Citations (3), Referenced by (28), Classifications (46), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetMethod and device for media editingUS 6961446 B2Abstract The present media editing device generates media including messages in an easy manner in a communication terminal such as a mobile terminal. Therein, a moving image data storage part stores moving image data recorded by a user. A region extraction part extracts any region including the user from the moving image data. A front determination part detects whether or not the user in the extracted region is facing the front. A sound detection part detects the presence or absence of a sound signal of a predetermined level or higher. A frame selection part determines starting and ending frames based on the results outputted from the front determination part and the sound detection part. An editing part performs, for example, an image conversion process by clipping out the media based on thus determined starting and ending frames. A transmission data storage part stores the resultantly edited media as transmission data.
Here, when messages and other data, e.g., �message leavings� (i.e., messages left by a caller in response to an automatic answering announcement) in the form of videomail or moving images, are once stored as media, the following steps are usually taken:
Such conventional method, however, causes the following problems if applied to general-type �message leavings� in the form of videomail or moving images showing one person, most of the time, facing to a camera.
SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a media editing method where media including moving images of, most of the time, one person facing to a camera can be easily edited as appropriate.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing the hardware structure of a media editing terminal capable of image communications realizing a media editing method of the present invention;
DESCRIPTION OF THE PREFERRED EMBODIMENTS (General Structure of Embodiments)
There have been various methods for extracting regions including the subject. For example, disclosed in Japanese Patent Laid-Open Publication No. 5-91407 (1993-91407) is a method for first defining any part where movement change is small as a background, and extracting other regions as �subject regions�. Here, the movement change is determined based on relative comparison between video signals in any two adjoining frames of the moving images. Another method for extracting subject regions is disclosed in Japanese Patent Laid-Open Publication No. 5-161131 (1993-161131). In this method, any image showing only the background is retained in advance to use for finding and computing any difference from each frame of the moving images on a pixel basis. Herein, whatever region not so different from the background is regarded as a background region, and if the difference is conspicuous, the region is extracted as the subject region. As another method, to extract any specific part of the subject such as a head or a face, images are searched for any ellipse region. Such method is described in �Human Head Tracking using Adaptive Appearance Models with a Fixed-Viewpoint Pan-Tilt-Zoom Camera� by Yachi et al., MIRU2000, Meeting on Image Recognition and Understanding, pp. 9�14. There are various other known methods for detecting a face image based on color information, focusing on a specific part of the face such as eyes, a mouth, or the like, and a method based on template matching. Under these conventional methods, the region extraction part 17 can easily extract subject regions.
Next, as for the regions extracted by the region extraction part 17, the front determination part 18 detects whether or not the user therein is facing the front. For such detection, there have been various methods. As an example, a front image is previously prepared as a template for template matching. As another example, there is a method for identifying the face orientation in images by applying SVM (Support Vector Machine) which is a statistical feature-recognition technique. The method is described in �Head Classifier: A Real-time Face Classification System� by Baba et al., the 7th Symposium on Sensing via Image Information, pp. 411�416. Under such conventional methods, the front determination part 18 can determine whether or not the person in the image is facing the front. Also, with a designated face orientation determination part provided in place of the front determination part 18, determined is whether the user in the image region is facing a predetermined direction (e.g., 45 degrees to the right). With such structure, the user's best face orientation can be designated in advance for image selection.
1. Descriptor (or Simply Referred to as �D�)
2. Description Scheme (or Simply Referred to as �DS�)
Description Definition Language is a language for defining notations for Descriptors and Description Schemes. In MPEG-7, on the basis of �XML Schema� which is a schema language standardized by W3C (World Wide Web Consortium), various data types to be needed for describing characteristics of multimedia contents are added. In this manner, Description Definition language is defined.
Description Scheme (DS) exemplarily includes �VideoSegmentDS� which is a pointer to a specific part of moving image data, and �StillRegionDS� used to describe regions in images. As for Descriptor (D), exemplified is a �MediaFormat� which describes media formats. Note that, in MPEG-7, together with Descriptors and Description Schemes standardly defined, a language for defining or extending any new Descriptor and Description Scheme (Description Definition Language) is also defined. Therefore, when meta data is used in the present media editing device, describing the meta data based on such a language will result in Description in the MPEG-7 format.
FIG. 5 is a diagram exemplarily showing meta data having index information of FIG. 4 described based on MPEG-7 standards. In the meta data of FIG. 5, �VideoSegmentDS� is used to interrelate points B and G to each corresponding image frame. Here, such interrelation to the actual image frames is established by �MediaTimePoint� of �MediaTime�, and resultantly described is the time of the corresponding VideoSegment. For example, the description of �T13:20:01:1F15� found in FIG. 5 means �the first frame at 13:20, 01 second (note that, 15 frames per second, from frames 0 to 14)�. With such description of meta data, reproduction control for selecting only a specific portion of the moving image data becomes possible.
FIG. 9 is a diagram showing exemplary meta-data Description for such a partial region. Description in FIG. 9 is, as is in the first embodiment, in MPEG-7 format. In this Description, �VideoSegmentDS� described in the first embodiment is applied to each frame, and the frames are each set by a partial region using �Still RegionDS�. As for partial region information, �ContourShape� is used to describe the partial region in rectangular (the number of peaks is 4 in the drawing) and the coordinates thereof (not shown).
In MPEG-7, not only the method for setting �StillRegionDS� on a frame basis as shown in FIG. 9, but �MovingRegionDS� being information about any moving region, or �AudioVisualRegionDS� being information about a region with sound may be used. As a comprehensive basic definition thereof, there is �SegmentDS� indicating a part of the multimedia contents. With any DS based on this definition, Description equivalent to that of FIG. 9 can be done with less space.
Described first is an assumable case in the present embodiment, specifically a case where the display image 100 of FIG. 6 is trimmed in such a manner that the moving image section 104 occupies a larger space, as much as possible, for display on a small screen (of mobile phone, for example). Here, presumably, information to be displayed on such a small screen is, at least, a �title�, �text�, and moving images. Actually, the small screen is fully occupied only by the moving images, and there is no space left for the title and text. FIG. 10 shows an exemplary display screen showing only the moving images.
FIG. 13 is a diagram showing exemplary Description of meta data about such a layout process. Description of FIG. 13 is, similar to the first and second embodiments, in MPEG-7 format. Based on a value of �MediaDuration�, that is, the length indicated by a predetermined point of the media, any sentence between �Text� tags is superimposed for display. As such, with Description of meta data, text display is enabled without the process for embedding text in moving images.
At the time of character selection, the user may have an itch to select a character relevant to moving images or details thereof. In the case that the display image is a face image, the present media editing device selects a character corresponding thereto in the layout process. In the below, the resultant mail with a character added is referred to as a �character mail�.
There have been various methods for extracting such face characteristic values. As one example, there is a method described in �Smartface��A Robust Face Recognition System under Varying Facial Pose and Expression (Publication of The Electronic Information Communications Society, Vol. J84-D-II, No. 6). In detail, in the method, a face region is first detected under the subspace method, and then the face parts (e.g., eyes, nose, mouth) are detected by using a separation filter. In the present media editing device, by applying at least one of such various known methods, the face characteristic values can be extracted in an easy manner.
The editing part 26 may notify a character string which indicates the characteristic values (or characteristic representations) inputted by the user. As an example, the user may input a character string of �the amount of hair is large, and the hair style short�. In response, the character selection part 24 then refers to such characteristic representations as shown in FIG. 16 for comparison, and selects a potential character. As such, with the help of a character string indicating the characteristic values, the potential character selected by the present media editing device can be closely analogous to the user's intended character.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5500671 *Oct 25, 1994Mar 19, 1996At&T Corp.Video conference system and method of providing parallax correction and a sense of presenceUS5864630 *Nov 20, 1996Jan 26, 1999At&T CorpMulti-modal method for locating objects in imagesUS5907604 *Mar 25, 1997May 25, 1999Sony CorporationImage icon associated with caller IDUS5999207 *May 16, 1997Dec 7, 1999Scientific-Atlanta, Inc.Method and apparatus for implementing a user interface for a videophone in a cable television networkUS6064436 *Jan 16, 1998May 16, 2000Sharp Kabushiki KaishaImage sequence encoding device and area extracting deviceUS6215890 *Sep 25, 1998Apr 10, 2001Matsushita Electric Industrial Co., Ltd.Hand gesture recognizing deviceUS6356339 *Jan 12, 1999Mar 12, 2002Fuji Photo Film Co., Ltd.Digital photo system which replaces a portion of subject matter common to two digital imagesUS6363160 *Jan 22, 1999Mar 26, 2002Intel CorporationInterface using pattern recognition and trackingUS6546052 *May 27, 1999Apr 8, 2003Canon Kabushiki KaishaImage processing apparatus and method, and computer-readable memoryUS6556775 *Feb 1, 1999Apr 29, 2003Matsushita Electric Industrial Co., Ltd.Image and sound reproduction systemUS6573908 *Apr 7, 2000Jun 3, 2003Korea Firstec Co., Ltd.Method and system for providing object information within frames of moving image dataUS6577807 *Jul 6, 1999Jun 10, 2003Hitachi Denshi Kabushiki KaishaEditing method and apparatus for moving picturesUS6594375 *Sep 22, 1999Jul 15, 2003Canon Kabushiki KaishaImage processing apparatus, image processing method, and storage mediumUS6611613 *Jun 30, 2000Aug 26, 2003Samsung Electronics Co., Ltd.Apparatus and method for detecting speaking person's eyes and faceUS6636635 *Oct 31, 1996Oct 21, 2003Canon Kabushiki KaishaObject extraction method, and image sensing apparatus using the methodUS6661448 *Jun 26, 2001Dec 9, 2003Nokia Mobile Phones Ltd.Method and system for providing and transmitting alternative video data during interruptions in video transmissionsUS6697503 *Nov 30, 2000Feb 24, 2004Matsushita Electric Industrial Co., Ltd.Device and method for face image extraction, and recording medium having recorded program for the methodUS6704029 *Apr 11, 2000Mar 9, 2004Canon Kabushiki KaishaMethod and apparatus for specifying scene information in a moving pictureUS6704433 *Dec 20, 2000Mar 9, 2004Matsushita Electric Industrial Co., Ltd.Human tracking device, human tracking method and recording medium recording program thereofUS6714216 *Sep 27, 1999Mar 30, 2004Sony CorporationVideo editing apparatus and methodUS6724417 *Nov 27, 2001Apr 20, 2004Applied Minds, Inc.Method and apparatus maintaining eye contact in video delivery systems using view morphingUS6754389 *Dec 1, 1999Jun 22, 2004Koninklijke Philips Electronics N.V.Program classification using object trackingUS6784899 *Mar 31, 2000Aug 31, 2004Ricoh Company, Ltd.Systems and methods for providing rich multimedia messages to remote users using telephones and facsimile machinesUS6804396 *Mar 28, 2001Oct 12, 2004Honda Giken Kogyo Kabushiki KaishaGesture recognition systemJPH0591407A Title not availableJPH06161131A Title not availableJPH06343146A Title not availableJPH09294239A Title not available* Cited by examinerNon-Patent CitationsReference1Atsushi Matsuo, Naruatu Baba, Tsukasa Noma, Hideaki Matsuo and Toshiaki Ejima, "Head Classifier: A Real-Time Face Classification System", 7th Symposium on Sensing Via Image Information, pp. 411-416.2Kiyotake Yachi, Toshikazu Wada and Takashi Matsuyama, "Human Head Tracking Using Adaptive Appearance Models With a Fixed-Viewpoint Pan-Tilt-Zoom Camera", MIRU2000, Meeting on Image Recognition and Understanding, pp. 9-14.3Osamu Yamaguchi and Kazuhiro Fukui, "'Smartface'-A Robust Face Recognition System Under Varying Facial Pose and Expression", Publication of The Electronic Information Communications Society, vol. J84-D-II, No. 6, Jun. 2001, pp. 1045-1052.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS7130618 *Jan 16, 2003Oct 31, 2006Nec CorporationMethod, apparatus, and system for transmitting moving image dataUS7284188 *Mar 29, 2002Oct 16, 2007Sony CorporationMethod and system for embedding MPEG-7 header data to improve digital content queriesUS7403655 *Sep 7, 2004Jul 22, 2008Canon Kabushiki KaishaMoving picture system and moving picture extraction methodUS7432983 *Nov 13, 2003Oct 7, 2008Kabushiki Kaisha ToshibaMoving-picture processing method and moving-picture processing apparatus with metadata processingUS7653246 *May 27, 2004Jan 26, 2010Fuji Xerox Co., Ltd.System and method for making a correction to a plurality of imagesUS7676092 *Jul 6, 2006Mar 9, 2010Hewlett-Packard Development Company, L.P.Data input apparatus and methodUS7747150Apr 3, 2007Jun 29, 2010Topcon CorporationImage processing device and methodUS7747151May 10, 2007Jun 29, 2010Topcon CorporationImage processing device and methodUS7756338 *Feb 14, 2007Jul 13, 2010Mitsubishi Electric Research Laboratories, Inc.Method for detecting scene boundaries in genre independent videosUS7839434 *Sep 29, 2006Nov 23, 2010Apple Inc.Video communication systems and methodsUS7860273 *Sep 28, 2007Dec 28, 2010Topcon CorporationDevice and method for position measurementUS7860276Dec 7, 2006Dec 28, 2010Topcon CorporationImage processing device and methodUS7860302 *Dec 19, 2008Dec 28, 2010Kabushiki Kaisha TopconImage processing apparatusUS7864198 *Aug 4, 2006Jan 4, 2011Vodafone Group Plc.Image processing method, image processing device and mobile communication terminalUS7925060 *Oct 26, 2006Apr 12, 2011Konica Minolta Holdings, Inc.Authentication system and registration system related to facial feature informationUS7941487 *Dec 24, 2004May 10, 2011Panasonic CorporationApparatus, system, and method of electronic mail processingUS8020097 *Mar 21, 2006Sep 13, 2011Microsoft CorporationRecorder user interfaceUS8204312Apr 6, 2007Jun 19, 2012Omron CorporationMoving image editing apparatusUS8280188Dec 14, 2009Oct 2, 2012Fuji Xerox Co., Ltd.System and method for making a correction to a plurality of imagesUS8384791 *Nov 28, 2003Feb 26, 2013Sony United Kingdom LimitedVideo camera for face detectionUS8436911Oct 22, 2009May 7, 2013Freezecrowd, Inc.Tagging cameraUS8457363 *May 19, 2005Jun 4, 2013Samsung Electronics Co., Ltd.Apparatus and method for detecting eyesUS8467509Sep 18, 2009Jun 18, 2013Canon Kabushiki KaishaVideo displaying apparatus, video displaying system and video displaying methodUS8577919 *Apr 23, 2008Nov 5, 2013Electronics And Telecommunications Research InstituteMethod and apparatus for retrieving multimedia contentsUS20070065044 *Jul 21, 2006Mar 22, 2007Samsung Electronics Co., Ltd.Method, medium, and system with a combinational photograph thumbnailUS20090062944 *May 16, 2008Mar 5, 2009Apple Inc.Modifying media filesUS20100131557 *Apr 23, 2008May 27, 2010Hee-Cheol SeoMethod and apparatus for retrieving multimedia contentsWO2005010725A2 *Jul 23, 2004Feb 3, 2005Chava LebartonStop motion capture tool* Cited by examinerClassifications U.S. Classification382/103, 348/E07.078, 382/190, 382/309, 382/107, 348/14.1, 382/118, 348/169, 715/723, 382/173, 348/14.15International ClassificationG06T5/00, H04M3/56, H04M3/53, H04N7/14, H04N5/91, G10L11/02, G06T11/60, G10L13/00, G06T7/60, H04N1/387, G06T3/00, H04M11/00Cooperative ClassificationH04W4/12, H04N21/47205, H04M2201/38, H04N21/41407, H04N21/8543, H04M3/567, H04N2007/145, H04N21/4223, H04W88/02, H04N7/141, H04N21/4331, G11B27/28, H04M3/5307, H04W4/18, H04N21/4788European ClassificationH04N21/414M, H04N21/8543, H04N21/4788, H04N21/4223, H04N21/472E, H04N21/433C, H04N7/14A, G11B27/28Legal EventsDateCodeEventDescriptionMar 7, 2013FPAYFee paymentYear of fee payment: 8Apr 1, 2009FPAYFee paymentYear of fee payment: 4Mar 21, 2006CCCertificate of correctionSep 12, 2001ASAssignmentOwner name: MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:IMAGAWA, KAZUYUKI;TAKATA, YUJI;MATSUO, HIDEAKI;AND OTHERS;REEL/FRAME:012164/0602Effective date: 20010906RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google