Source: http://www.google.com/patents/US20020031262?dq=raman+iyer&ei=oYE3T4ePBKariQKfpoi9Cg
Timestamp: 2017-07-25 06:53:12
Document Index: 102198337

Matched Legal Cases: ['art 14', 'art 17', 'art 1', 'art 2', 'art 3', 'art 4', 'art 5', 'art 6', 'art 7', 'art 8', 'art 9', 'art 10', 'art 11', 'art 12', 'art 13', 'art 1', 'art 2', 'art 2', 'art 4', 'art 4', 'art 4', 'art 2', 'art 3', 'art 5', 'art 6', 'art 2', 'art 7', 'arts 4', 'art 8', 'art 3', 'art 9', 'art 9', 'art 10', 'art 2', 'art 4', 'art 10', 'art 11', 'art 10', 'art 12', 'art 2', 'art 4', 'art 10', 'art 13', 'art 2', 'art 4', 'art 5', 'art 1', 'art 2', 'art 4', 'art 1', 'art 13', 'art 11', 'art 10', 'art 10', 'art 12', 'art 13', 'art 14', 'art 15', 'art 17', 'art 18', 'art 19', 'art 20', 'art 21', 'art 14', 'art 10', 'art 17', 'art 18', 'art 19', 'art 18', 'art 19', 'art 20', 'art 21', 'art 15', 'art 10', 'art 14', 'art 17', 'art 17', 'art 17', 'art 18', 'art 18', 'art 18', 'art 19', 'art 18', 'art 19', 'art 20', 'art 20', 'art 18', 'art 19', 'art 20', 'art 19', 'art 21', 'art 20', 'art 21', 'art 20', 'art 17', 'art 18', 'art 21', 'art 14', 'art 15', 'art 17', 'art 22', 'art 23', 'art 14', 'art 23', 'art 10', 'art 22', 'art 23', 'art 14', 'art 17', 'art 17', 'art 17', 'art 22', 'art 22', 'art 22', 'art 22', 'art 22', 'art 17', 'art 22', 'art 17', 'art 22', 'art 17', 'art 22', 'art 23', 'art 22', 'art 22', 'art 22', 'art 17', 'art 22', 'art 22', 'art 17', 'art 14', 'art 15', 'art 17', 'art 18', 'art 26', 'art 24', 'art 14', 'art 18', 'art 17', 'art 18', 'art 26', 'art 24', 'art 24', 'art 26', 'art 26', 'art 24', 'art 24', 'art 26', 'art 26', 'art 26', 'art 26', 'art 26', 'art 24', 'art 24', 'art 724', 'art 724', 'art 724', 'art 724', 'art 724']

Patent US20020031262 - Method and device for media editing - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsThe present media editing device generates media including messages in an easy manner in a communication terminal such as a mobile terminal. Therein, a moving image data storage part 14 stores moving image data recorded by a user. A region extraction part 17 extracts any region including the user from...http://www.google.com/patents/US20020031262?utm_source=gb-gplus-sharePatent US20020031262 - Method and device for media editingAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS20020031262 A1Publication typeApplicationApplication numberUS 09/950,085Publication dateMar 14, 2002Filing dateSep 12, 2001Priority dateSep 12, 2000Also published asCN1200537C, CN1344084A, US6961446Publication number09950085, 950085, US 2002/0031262 A1, US 2002/031262 A1, US 20020031262 A1, US 20020031262A1, US 2002031262 A1, US 2002031262A1, US-A1-20020031262, US-A1-2002031262, US2002/0031262A1, US2002/031262A1, US20020031262 A1, US20020031262A1, US2002031262 A1, US2002031262A1InventorsKazuyuki Imagawa, Yuji Takata, Hideaki Matsuo, Katsuhiro Iwasa, Tetsuya YoshimuraOriginal AssigneeKazuyuki Imagawa, Yuji Takata, Hideaki Matsuo, Katsuhiro Iwasa, Tetsuya YoshimuraExport CitationBiBTeX, EndNote, RefManPatent Citations (24), Referenced by (155), Classifications (38), Legal Events (6) External Links: USPTO, USPTO Assignment, EspacenetMethod and device for media editing
DESCRIPTION OF THE PREFERRED EMBODIMENTS [0050] (General Structure of Embodiments) [0051] With reference to the accompanying drawings, embodiments of the present invention are generally described below. [0052] With a method and device for media editing of the present invention, a convenient interface can be provided for a user to create message leavings in the form of videomail by using a personal or home image communications terminal such as a visualphone, a mobile terminal, a doorphone, or the like. [0053] [0053]FIG. 1 is a block diagram showing the hardware structure of a media editing terminal where image communications is carried out in such a manner as to realize the media editing method of the present invention. In FIG. 1, the present media editing terminal includes an input part 1, an image capturing part 2, an image display part 3, a sound input part 4, and a sound output part 5, all of which receive/provide information from/to the user. Further, included are an image-capturing control part 6, a sound input/output control part 7, a display control part 8, a communications part 9, a recording part 10, a recording control part 11, a signal processing part 12, and a control part 13, all of which process the information received/provided by the user. These constituents are interconnected via a system bus, an external bus, and the like. Here, the above structure is identical or similar to that of a general-type computer. [0054] The input part 1 is composed of a keyboard (e.g., a ten-key numerical pad included), a mouse, and the like, and various types of information and documents in text form necessary for the user to do media editing are inputted therefrom. [0055] The image capturing part 2 is composed of a CCD camera, and the like, and moving images structured by at least one or more images are inputted therefrom. Here, the image capturing part 2 basically works cooperatively with the sound input part 4, which will be described later, and typically captures moving images of the user doing message recording. [0056] The sound input part 4 is composed of a microphone, and the like, and sound outside the device is inputted therefrom. Here, the sound input part 4 basically works cooperatively with the image capturing part 2, and typically acquires the user's voice doing message recording. [0057] The image display part 3 is composed of a liquid crystal display, and the like, and displays, to the user, his/her recorded moving images and characters (e.g., alphanumeric), received moving images and characters, and various information operationally necessary, for example. [0058] The sound output part 5 is composed of a speaker, and the like, and outputs, to the user, his/her recorded voice, received sound, and warning sound and beep operationally necessary, for example. [0059] The image-capturing control part 6 performs ON/OFF control, exposure control, and other controls with respect to the image capturing part 2. The sound input/output control part 7 performs ON/OFF control, and other controls with respect to the sound input and output parts 4 and 5. The display control part 8 controls the image display part 3. [0060] The communications part 9 transmits/receives, to/from other information processing devices, various types of data wirelessly or over the communications path such as a public telephone line. As for the data, see the description below. The communications part 9 may be in whatever communications mode, including synchronous communications such as visualphones, or asynchronous communications such as e-mails, for example. [0061] The recording part 10 is composed of a recording medium such as a memory, a hard disk, and the like, and records data at least provided by the image capturing part 2 and the sound input part 4. The recording part 10 may include a recording medium such as a CD-ROM, a DVD, and the like, and a drive therefor. The recording control part 11 performs input/output control with respect to the recording part 10. [0062] The signal processing part 12 is composed of a digital signal processor, and the like, and in editing of the later-described embodiments, goes through any process necessary for image signals provided by the image capturing part 2, sound signals from the sound input part 4, and data recorded on the recording part 10. [0063] The control part 13 is composed of a microcomputer, a CPU, or the like, and controls the data flow for various processes. [0064] Here, the present media editing terminal may be of an integrated-type including every constituent mentioned above in one housing, or of a distributed-type performing data exchange among the constituents over a network or signal lines. For example, a camera-equipped mobile phone terminal is of the integrated-type carrying every constituent in a single housing. On the other hand, a doorphone is regarded as of the distributed-type because, at least, the image capturing part 2, the sound input part 4, and the sound output part 5 are externally located in the vicinity of the door, and the remains are placed in another housing located in the living room, for example. This is for establishing an interface with visitors. Alternatively, such a distributed-type device may have a character database (later described) located outside. [0065] Described next is the comprehensive procedure for the user to generate transmission data under the media editing method of the present invention. FIG. 2 is a block diagram showing the information flow and the procedure at the time of media editing of the present invention. [0066] First, the user inputs a command to have the input part 1 of FIG. 1 started receiving image and sound data. Then, the user inputs his/her message, for example, via the image capturing part 2 and the sound input part 4 to generate moving image data. [0067] Thus generated moving image data often includes, both at the head and tail, a portion carrying unnecessary information for the user. Accordingly, a clipping process is performed to eliminate those unnecessary portions at the head and tail of the moving image data. The details of the process are left for later description. [0068] Performed next is a layout process to display, on a single screen, any useful information (e.g., time and date when the data was generated by whom) for a data addressee together with the generated moving image data. In detail, after clipping, a trimming process is applied to cut out from the moving image data any specific region having the user (message recorder) centered. Then, the resultant region is arranged with, for example, text and cartoon-like character images, which are generated as basic data. Here, the basic data presumably denotes whatever data to be added to the moving image data, exemplified by images, text, and computer graphics. The basic data may be a previously-generated image pattern, a character pattern, or a code pattern. Moreover, the layout process is typically described in a meta data format including MIME (Multipurpose Internet Message (Mail) Extensions), HTML (Hyper Text Markup Language), XML (eXtensible Markup Language), MPEG-7, for example. After the above processes are through, eventually generated is transmission data, which is a message for a data addressee. [0069] Here, the clipping process and the layout process are both performed in the signal processing part 1, the control part 13, the recording control part 11, and the recording part 10 of FIG. 1. Typically, these processes are realized by a program executable by computer devices. The program is provided from a computer-readable recording medium, e.g., a CD-ROM, a semiconductor memory card, to the recording part 10, for example, and then downloaded over the communications lines. [0070] As already described, if those processes are applied under the conventional method, user input is required frequently. In more detail, in the clipping process, the user is expected to make an input of a clipping portion while checking the moving images and sound. Moreover, in the layout process, the user is required to do editing operation while considering what layout. Especially, for trimming in the layout process, the user has to go through the moving image data on a frame basis to choose cutting regions therefrom. This is very bothersome for the user. When attaching the basic data, the user also has to define the attachment position while referring to the moving image data for the subject's position and size. [0071] With the device and method for media editing according to each of the embodiments of the present invention, either one or both of the signal processing part 12 and the control part 13 go through a region extraction process, a front determination process, and a sound detection process, all of which will be described later. With these processes, successfully provided is a convenient interface with which any process the user finding bothersome (in particular, clipping, trimming, and editing of basic data arrangement) is automated. [0072] Generally, once the user creates a message in the form of videomail by his/her mobile terminal, he/she may have an itch to immediately send out the message. With the convenient interface provided, the user's such needs are thus met with a videomail created with a simple operation (e.g., one button operation). What is better, the resultant videomail layout is comprehensible to its addressee, having the message clipped at the beginning and end, the image trimmed to have the user centered, wallpaper and speech bubbles arranged as appropriate, for example. Herein, not all of the above processes are necessarily applied in the following embodiments, and combining any process needed for each different application will do. In the below, the embodiments of the present invention are individually described in detail. [0073] (First Embodiment) [0074] A media editing device of a first embodiment enables the aforementioned clipping process of FIG. 2 in an automatic manner. FIG. 3 is a block diagram showing the functional structure of the media editing device of the first embodiment. In FIG. 3, the present media editing device includes a moving image data storage part 14, a transmission data storage part 15, a region extraction part 17, a front determination part 18, a sound detection part 19, a frame selection part 20, and an editing part 21. These constituents carry out entirely or partially the clipping process of FIG. 2. [0075] The moving image data storage part 14 corresponds to the recording part 10 of FIG. 1, and stores moving image data recorded by the user as a message. The region extraction part 17 extracts, from the moving image data in storage, any specific region including entirely or partially the image of the subject (the user). The front determination part 18 detects whether or not the user in the extracted region is facing the front. The sound detection part 19 detects, on a frame basis in the moving image data, whether there is any sound signal of a predetermined level or higher. Based on the results outputted from the front determination part 18 and the sound detection part 19, the frame selection part 20 determines starting and ending frames. The editing part 21 performs media clipping based on thus determined starting and ending frames, and then media editing, e.g., image conversion process. The transmission data storage part 15 also corresponds to the recording part 10 of FIG. 1, and stores the resultant edited media as transmission data, which will be transmitted as required. [0076] Described next is the operation of these constituents. FIG. 4 is a diagram for illustrating the clipping process applied to certain moving image data. In FIG. 4, the clipping process is applied to the moving image data stored in the moving image data storage part 14. Here, the moving image data is composed of sound data including the user's recorded message, and image data recorded synchronously therewith. The image data and the sound data may be structured as one data, or structured separately as the image data, the sound data, and data indicating the relationship therebetween in terms of synchronization. The data exemplified in FIG. 4 is a typical message often acquired through a doorphone, and the like, and composed of eight scenes (points in time) A to H in the drawing. At each point in time, the user's (recorder's) behavior appears as follows. [0077] A. Start data recording [0078] B. Start first message [0079] C. End first message [0080] D. Pause before continuing message [0081] E. Start second message [0082] F. No sound (e.g., breathing) [0083] G. End second message [0084] H. End data recording [0085] Here, as for the graph in the drawing, the lateral axis indicates a lapse of time, while the longitudinal axis an inputted sound level. Alphabetical letters A to H each indicate a predetermined time. Each of the cartoon sketches above the graph represents a scene in the image data, which is recorded simultaneously with the sound substantially at the predetermined time point (A to H). The cartoon sketches are exemplary of the user's behavior in the course of message recording through the doorphone before leaving the place. [0086] As is known from FIG. 4, the generated moving image data often carries, at the beginning and end, information irrelevant to the user's intention. This is because people usually take pauses before and after recording their messages. Focusing attention on such characteristic of the moving image data conveying messages, the present media editing device automatically determines a clipping portion in image data and sound data under the following methods. [0087] Described first is a method for detecting a start point for clipping. To detect a start point, the region extraction part 17 first detects, on a frame basis in the image data, any region including the subject (the user) in a time sequence (i.e., from A to H in FIG. 4). [0088] There have been various methods for extracting regions including the subject. For example, disclosed in Japanese Patent Laid-Open Publication No. 5-91407 (1993-91407) is a method for first defining any part where movement change is small as a background, and extracting other regions as “subject regions”. Here, the movement change is determined based on relative comparison between video signals in any two adjoining frames of the moving images. Another method for extracting subject regions is disclosed in Japanese Patent Laid-Open Publication No. 5-161131 (1993-161131). In this method, any image showing only the background is retained in advance to use for finding and computing any difference from each frame of the moving images on a pixel basis. Herein, whatever region not so different from the background is regarded as a background region, and if the difference is conspicuous, the region is extracted as the subject region. As another method, to extract any specific part of the subject such as a head or a face, images are searched for any ellipse region. Such method is described in “Human Head Tracking using Adaptive Appearance Models with a Fixed-Viewpoint Pan-Tilt-Zoom Camera” by Yachi et al., MIRU2000, Meeting on Image Recognition and Understanding, pp. 9-14. There are various other known methods for detecting a face image based on color information, focusing on a specific part of the face such as eyes, a mouth, or the like, and a method based on template matching. Under these conventional methods, the region extraction part 17 can easily extract subject regions. [0089] Next, as for the regions extracted by the region extraction part 17, the front determination part 18 detects whether or not the user therein is facing the front. For such detection, there have been various methods. As an example, a front image is previously prepared as a template for template matching. As another example, there is a method for identifying the face orientation in images by applying SVM (Support Vector Machine) which is a statistical feature-recognition technique. The method is described in “Head Classifier: A Real-time Face Classification System” by Baba et al., the 7th Symposium on Sensing via Image Information, pp. 411-416. Under such conventional methods, the front determination part 18 can determine whether or not the person in the image is facing the front. Also, with a designated face orientation determination part provided in place of the front determination part 18, determined is whether the user in the image region is facing a predetermined direction (e.g., 45 degrees to the right). With such structure, the user's best face orientation can be designated in advance for image selection. [0090] The sound detection part 19 detects whether there is any sound signal of a predetermined level or higher. The predetermined level is exemplarily determined in consideration of the ambient noise level and the average level of the inputted sound. Alternatively, the presence or absence of human voice may be detected under any known voice recognition method. [0091] With reference to the results obtained by the front determination part 18 and the sound detection part 19, the frame selection part 20 checks, on a frame basis, the image data from the start point to the end point. Here, a frame first satisfying the conditions is regarded as a starting frame. The frame selection part 20 also checks the frames in the reverse direction, that is, from the end point to the start point of the image data, and this time, a frame first satisfying the condition is regarded as an ending frame. Here, the result from the front determination part 18 tells that the user in the images of FIG. 4 is facing the front firstly at point B, and lastly at point G. Also, the result from the sound detection part 19 tells that the inputted sound level firstly reaches a predetermined level or higher at point B, and lastly at point G. The frame selection part 20 does not perform frame selection unless otherwise all of the conditions are satisfied. Thus, in this example, the starting frame is the one at point B, and the ending frame at point G. [0092] As such, the media editing device of the present invention scans image data in both directions from a start point to an end point, and from the end point to the start point, to find time points each satisfying conditions first. In this manner, clipping can be done to a message in its entirety without cutting a time interval (before and after point D) which is a pause during message recording. Therefore, this media editing device is suitably used for transmission of videomail, which contains user's input data as a single piece. [0093] Further, since the present media editing device performs both front determination and sound detection, clipping can be done with reliability to a part recorded as a message. Specifically, even if the user is facing the camera but deep in thought, clipping never miss a time point when he/she starts speaking. Here, the present media editing device can achieve almost the same effects without sound detection. This is because the user normally faces toward the camera to start message recording, and thus front determination sufficiently serves the purpose. Also, if the user utters in spite of his/her intention before starting message recording, sound detection may not be considered effective. Therefore, the sound detection part 19 may be omissible. [0094] Next, the editing part 21 performs media (moving image data) clipping on the basis of the starting and ending frames determined by the frame selection part 20. Here, the resultant moving image data generated by the editing part 21 may include only the clipped portion and remains are all deleted, or the resultant data may be meta data including the clipped portion as an index. If the resultant data is meta data, no moving image data has been deleted, and thus any portion not clipped but important can be saved for later use. Exemplified below is a case where the meta data format is MPEG-7. [0095] There have been various standards and private standards for a description format of meta data, and among those, MPEG-7 is of the latest, for example. MPEG-7, called as Multimedia Content Description Interface (ISO/IEC 15938), is the fourth standards after MPEG-1, MPEG-2, and MPEG-4, all of which have been standardized by MPEG (Moving Picture Experts Group: ISO/IEC JTC1/SC29/WG11). These standards are the ones defining outlines for description of the details of multimedia information, and aimed to be used for applications for retrieving and editing digital libraries. As for MPEG-7, defined thereby are standard groups of Descriptor for describing the details of multimedia information mainly including video and sound information. By applying the resultant Description to contents, retrieval can be done based on the details of the multimedia information. The actual description definition language of this standards has been extended as needed with respect to XMLSchema language. Here, this extension is compatible with the grammar of XMLSchema language. [0096] Under such MPEG-7, in order to describe the characteristics of the contents, the following basic elements may be combined together. [0097] 1. Descriptor (or Simply Referred to as “D”) [0098] Descriptor is a basic tool for describing a certain single characteristic of multimedia contents. In MPEG-7, Descriptor is defined by Syntax and Semantics. [0099] 2. Description Scheme (or Simply Referred to as “DS”) [0100] Description Scheme is an outline which defines the structures or semantic relationships among a plurality of description tools. In MPEG-7, similarly, Description Scheme is defined by Syntax and Semantics. Here, as for description tools structuring Description Scheme, other Description Schemes are included together with Descriptors. [0101] 3. Description Definition Language [0102] Description Definition Language is a language for defining notations for Descriptors and Description Schemes. In MPEG-7, on the basis of “XML Schema” which is a schema language standardized by W3C (World Wide Web Consortium), various data types to be needed for describing characteristics of multimedia contents are added. In this manner, Description Definition language is defined. [0103] Description Scheme (DS) exemplarily includes “VideoSegmentDS” which is a pointer to a specific part of moving image data, and “StillRegionDS” used to describe regions in images. As for Descriptor (D), exemplified is a “MediaFormat” which describes media formats. Note that, in MPEG-7, together with Descriptors and Description Schemes standardly defined, a language for defining or extending any new Descriptor and Description Scheme (Description Definition Language) is also defined. Therefore, when meta data is used in the present media editing device, describing the meta data based on such a language will result in Description in the MPEG-7 format. [0104] [0104]FIG. 5 is a diagram exemplarily showing meta data having index information of FIG. 4 described based on MPEG-7 standards. In the meta data of FIG. 5, “VideoSegmentDS” is used to interrelate points B and G to each corresponding image frame. Here, such interrelation to the actual image frames is established by “MediaTimePoint” of “MediaTime”, and resultantly described is the time of the corresponding VideoSegment. For example, the description of “T13:20:01:1F15” found in FIG. 5 means “the first frame at 13:20, 01 second (note that, 15 frames per second, from frames 0 to 14)”. With such description of meta data, reproduction control for selecting only a specific portion of the moving image data becomes possible. [0105] Here, the above is no more than an example, and there is no limitation for description formats. In other words, any format will do for describing meta data as long as it can be interrelated to contents. Here, for convenience of illustrating by drawings, the meta data is exemplarily described in text format, but this is not restrictive. The format may be binary, for example. [0106] As such, when the resultant data is meta data including a clipping portion as an index with no moving image data deleted, editing can be done without restraint if the data needs to be corrected after automatic clipping. This is because, unlike the case where the resultant data is moving image data including only the clipped portion, there needs to re-edit only the meta data. [0107] In the above, the starting and ending frames provided by the frame selection part 20 are utilized for automatic clipping. Here, the starting frame may be defined as being an image appearing first on a terminal screen on the receiver end. In this sense, the clipping technique of the present media editing device is considered effective even better. To be more specific, assuming a case where the user first sees a still image (e.g., a preview image, thumbnail image) showing what moving images are coming or already in storage. Here, such a still image is now referred to as an initial display image. In the example of FIG. 4, the first frame image is the one at point A. However, the image at A shows the user not facing towards the camera, and it is not considered suitable for the initial display image such as a preview or a thumbnail image. Accordingly, by using the meta data as illustrated in FIG. 5, the starting frame is defined as the initial display image. As a result, the frame image at point B showing the user facing the front is suitably displayed as the initial display image. The present media editing device thus has no need to newly transmit a still image as the initial display image to the receiver end. If newly transmitting, the media editing device uses the region extraction part 17 and the front determination part 18 to scan the data from the start point to the end point. Point B is resultantly detected, and the frame image corresponding thereto is transmitted as the initial display image. In this manner, the image showing the user facing the front appropriately goes to the receiver end. [0108] To the initial display image or the moving image data in its entirety, the editing part 21 may apply an image conversion process, e.g., a resolution conversion process. If this is the case, display can be optimal in consideration of the resolution on the receiver terminal, and with lower resolution, information to be transmitted becomes less in amount. As the image conversion process, a representation conversion process may be carried out, including digitization and gray scale processing, for example. With such process, display color can be changed in accordance with that on the receiver terminal. [0109] As such, the media editing device of the present invention determines whether or not a user in an image is facing the front. Therefore, only a message part to an addressee can be automatically clipped out with reliability, and the addressee reproduces only any portion he/she needs. Further, whatever image suitable as an initial display image can be easily set. [0110] (Second Embodiment) [0111] With a media editing device according to a second embodiment, the aforementioned trimming process is automated so that the resultant layout becomes well-organized, with efficiency, even for a small screen on the receiver end. [0112] Described first is an assumable case in the present embodiment. Generally, any media to be transmitted in the form of videomail includes, not only moving image data, information about who has sent the moving images with what title, for example. FIG. 6 is a diagram showing an exemplary screen display of a terminal receiving such a videomail. As shown in FIG. 6, on a display image 100, displayed are a moving image section 104, a header section 101 exemplarily indicating who has sent the videomail to whom with what title, a text section 102, and a decoration section 103 having decorations appropriately laid out. [0113] To display such a display image of FIG. 6 on a small screen of a mobile terminal, the image is often reduced in size in its entirety. Such reduction in size, however, causes the text to be illegible, or the subject's face in the moving image data to be smaller. [0114] Here, moving image data inputted as a message is generally captured by a wide-angle lens so as not to distract the user with the positional relationship between the camera and the subject. This is the reason why the moving image section 104 of FIG. 6 contains a high proportion of background region behind the user's image. Therefore, if the image is reduced in size in its entirety for display, the user's face inconveniently looks much smaller. [0115] The media editing device of the present embodiment includes at least the following constituents so that a layout process is performed in a manner that only a partial image including the user is displayed on the display screen. Here, for the clipping process aforementioned in the first embodiment, any corresponding constituent of FIG. 3 may be added. Therefore, no further description is provided here. [0116] [0116]FIG. 7 is a block diagram showing the functional structure of the media editing device of the second embodiment. In FIG. 7, the present media editing device includes the moving image data storage part 14, the transmission data storage part 15, the region extraction part 17, a layout part 22, and a basic data storage part 23. These constituents perform partially or entirely the aforementioned layout process of FIG. 2. [0117] Here, this media editing device is almost the same in structure and operation as that of the first embodiment, and thus any constituent being the same is provided with the same reference numeral and not described again. Note that, in the present embodiment, sound data is not necessarily required. Therefore, stored in the moving image data storage part 14 may be moving image data similar to that in the first embodiment, or image data carrying no sound. [0118] In FIG. 7, the basic data storage part 23 corresponds to the recording part 10 of FIG. 1, and stored therein are such a text shown in FIG. 6, and basic data exemplified by image data for decoration. The layout part 22 reads, as appropriate, the basic data from the basic data storage part 23 by the user's operation, and performs the layout process including the trimming process. The details are left for later description. [0119] [0119]FIG. 8 shows an exemplary trimming process and the resultant display screen. In FIG. 8, shown in the upper is the moving image section 104 received from the same addresser of FIG. 6. Due to the reasons described in the above, the section contains a high proportion of background region behind the user's image. Thus, only the user region is trimmed in the following manner for laying out. [0120] First, from the moving images stored in the moving image data storage part 14, the region extraction part 17 extracts, on a frame basis, any region including the user partially (e.g., his/her face) or entirely. The operation of the region extraction part 17 can be easily realized by the above described method. Here, the resultantly extracted region is not restricted in shape. [0121] Then, based on the regions extracted by the region extraction part 17, the layout part 22 calculates a partial region from the moving image data for display. In FIG. 8, the partial region is indicated by a thick-lined box therearound in the moving image section 104. [0122] The layout part 22 then lays out the image corresponding to the partial region and the user-designated basic data (e.g., text, image) in such a way as to combine those together. In FIG. 8, the resultant display image 200 includes a moving image section 204 corresponding to the partial region, and similarly to FIG. 6, a header section 201, a text section 202, and a decoration section 203. As such, at the time of layout, the moving image data is automatically reduced in size to be an image fitting in the partial region, thereby achieving comprehensible display on a small screen. [0123] The layout part 22 generally generates meta data, which determines what layout with the moving image data and the basic data. Thus, the partial region set by the layout part 22 is preferably also in the meta data format for easy handling. [0124] [0124]FIG. 9 is a diagram showing exemplary meta-data Description for such a partial region. Description in FIG. 9 is, as is in the first embodiment, in MPEG-7 format. In this Description, “VideoSegmentDS” described in the first embodiment is applied to each frame, and the frames are each set by a partial region using “Still Region DS”. As for partial region information, “ContourShape” is used to describe the partial region in rectangular (the number of peaks is 4 in the drawing) and the coordinates thereof (not shown). [0125] When the meta data is used as such, unlike newly generating moving image data by cutting out a partial region therefrom, the amount of the moving image data is not reduced. The user on the receiver end, however, can freely change the layout according to the size of the terminal screen or his/her preference. For example, the user can relocate the partial region on the image to suit his/her preference, or make settings to display any other partial region. In such cases also, settings as the partial region set by the layout part 22 initially appearing on the screen is considered convenient. This is because the region indicating who has sent the message is displayed first. [0126] In MPEG-7, not only the method for setting “StillRegionDS” on a frame basis as shown in FIG. 9, “MovingRegionDS” being information about any moving region, or “AudioVisualRegionDS” being information about region with sound may be used. As a comprehensive basic definition thereof, there is “SegmentDS” indicating a part of the multimedia contents. With any DS based on this definition, Description equivalent to that of FIG. 9 can be done with less amount. [0127] As such, the media editing device of the present invention can define the image by a partial region for display. Therefore, even on a small display screen of a camera-equipped mobile terminal, only a region showing the subject can be displayed in a well-organized manner. Moreover, when Description of meta data is based for layout, the image can be appropriately displayed on the receiver end even with screens varying in size (e.g., camera-equipped mobile terminals, PC terminals). [0128] (Third Embodiment) [0129] With a media editing device of a third embodiment, the trimming process is performed differently from the second embodiment, and the resultant layout displays any needed text together with moving images occupying a sizable proportion of the screen. [0130] Described first is an assumable case in the present embodiment, specifically a case where the display image 100 of FIG. 6 is trimmed in such a manner that the moving image section 104 occupies a larger space as much as possible for display on a small screen (of mobile phone, for example). Here, presumably, information to be displayed on such a small screen is, at least, a “title”, a “text”, and moving images. Actually, the small screen is fully occupied only by the moving images, and there is no space left for the title and text. FIG. 10 shows an exemplary display screen showing only the moving images. [0131] Here, the present media editing device is similar in structure to that of the second embodiment. To display such text information, however, the region extraction part 17 and the layout part 22 in the present media editing device are changed in their operations In detail, onto the image region including the user (the user's image region) that has been detected by the region extraction part 17, the layout part 22 arranges the text information (e.g., title, text) so as not to overlap at all, or to overlap as little as possible if overlaps. This operation is described in detail below. [0132] First, the region extraction part 17 detects the user's image region in the moving image data, and calculates the position and size thereof. Then, the layout part 22 receives thus calculated position and size of the region, and the basic data (e.g., title, text) stored in the basic data storage part 23. The layout part 22 sets a region for arranging the basis data in the range not overlapping the user's image region at all (or overlapping as little as possible). FIG. 11 shows an exemplary display screen where a text title is arranged in a space not overlapping the user's image region. As shown in FIG. 11, the text title is arranged in a space above the user's head with no overlap. With such arrangement, the resultant layout can contain any needed text together with moving images occupying a sizable proportion. [0133] Alternatively, the layout part 22 may arbitrarily set the shape of such a region for arranging the basic data. If so, thus set region is now referred to as a speech bubble region. Typically, the speech bubble region is enclosed by a line and is in a specific color (e.g., white). Into the speech bubble region, the layout part 22 writes a main text, which is a part of the basic data. FIG. 12 shows an exemplary display screen where a main text is arranged in a region barely overlapping the user's image region. As shown in FIG. 12, the main text is arranged in a space left side of the user with little overlap. Accordingly, the resultant layout can contain any needed text together with moving images occupying a sizable proportion. [0134] The shape of the speech bubble region shown in FIG. 12 has, as quite familiar in cartoons, a sharp protrusion in the vicinity of the user's mouth. The position of the protrusion is calculated by an image recognition process. Specifically, the region extraction part 17 extracts a mouth region from the user's image region, and calculates its position. The layout part 22 arranges the protrusion onto thus calculated position (or proximal position considered appropriate), and then sets the speech bubble region in the range not overlapping the user's image region at all (or overlapping as little as possible) in consideration of the number of letters of the text. [0135] The resultant layout image is preferably displayed on the screen as the initial image (aforementioned initial display image) on the receiver end. That is, when opening incoming mails, the addressee first sees the image of FIG. 11 or 12, and checks only the title or the main text therewith. If the main text does not fit in one page, a scrolling process may be applied, for example. As such, the receiver checks a main text, for example, only in the first display image but not while the moving images are reproduced. This is surely not restrictive, and the main text or the title may be superimposed and displayed during when the moving images are reproduced so that the receiver can read the text while hearing and seeing the message in the form of the moving images. [0136] Here, the text to be displayed is not limited to the title or the text. Moreover, the image of FIG. 11 may appear first, and then the image of FIG. 12 may follow by the receiver's operation. Or these images are merged together for display at one time. As such, any structure will do as long as text arrangement is so done as not to overlap the user's image region at all (or overlapping as little as possible). [0137] As described above, in the present media editing device, the resultant layout can be well-organized, and even on a display screen showing both the moving images and text, the receiver will not confuse which is which. Further, by using speech bubble regions, the user in the image looks as if speaking the text, and accordingly communication can become active. [0138] Next, the layout part 22 preferably generates meta data which is the deciding factor for what layout in the similar manner in the first and second embodiments. This is done to perform the layout process, that is, the process for writing a text into moving images. [0139] [0139]FIG. 13 is a diagram showing exemplary Description of meta data about such a layout process. Description of FIG. 13 is, similar to the first and second embodiments, in MPEG-7 format. Based on a value of “MediaDuration”, that is, the length indicated by a predetermined point of the media, any sentence between “Text” tags is superimposed for display. As such, with Description of meta data, text display is enabled without the process for embedding text in moving images. [0140] (Fourth Embodiment) [0141] With a media editing device of a fourth embodiment, message representation can be varied in style to extend the user's range of choices for his/her fun, facilitating smooth communication. This is achieved by the region extraction part 17 extracting a face region, and selecting a character image corresponding thereto. [0142] Described first is an assumable case in the present embodiment. As already described, decorating videomails increases fun. As in the customizable photo sticker machines commonly seen in video game parlors or the like, attaching characters represented by cartoon sketches or three dimensional (3D) graphics to the user's image effectively makes the resultant videomail full of fun, and the receiver feels a closeness thereto. FIG. 14 shows an exemplary display image of a videomail on the receiver end having such a character added. As shown in FIG. 14, displayed on a display image 400 are a header section 401 indicating who has sent the videomail to whom with what title, a text section 402, a moving image section 404, and a character section 403. With such a layout of videomail on the display screen, the receiver can feel a closeness thereto to a further degree. [0143] At the time of character selection, the user may have an itch to select a character relevant to moving images or details thereof. In the case that the display image is a face image, the present media editing device selects a character corresponding thereto in the layout process. In the below, the resultant mail with a character added is referred to as a “character mail”. [0144] [0144]FIG. 15 is a block diagram showing the functional structure of the media editing device of the fourth embodiment. In FIG. 15, the present media editing device includes the moving image data storage part 14, the transmission data storage part 15, the region extraction part 17, the front determination part 18, an editing part 26, a character selection part 24, and a character database 25. These constituents carry out the layout process of FIG. 2 partially or entirely. [0145] Note that, this media editing device is the same in structure and operation as that of the first embodiment, and thus any identical constituent is under the same reference numeral, and not described again. In the present embodiment, sound data is not necessarily required. Therefore, stored in the moving image data storage part 14 may be moving image data similar to that in the first embodiment, or image data carrying no sound. For the clipping process aforementioned in the first embodiment, any corresponding constituent of FIG. 3 may be added. Therefore, no further description is provided here. Moreover, as already described, the front determination part 18 may be the designated face orientation determination part. [0146] Described next is the operation of the media editing device of the present embodiment. The region extraction part 17 and the front determination part 18 operate in the similar manner to the first embodiment, and determine whether or not the user in the moving images is facing the front. The result is forwarded to the editing part 26, from which any image determined as being the front image is provided to the character selection part 24. Based on thus received image(s), the character selection part 24 selects one or more of potential characters from the character database 25, where various many characters are stored as a database. Then, a character ID each corresponding to thus selected character(s) are inputted into the editing part 26. [0147] Here, in the present embodiment, a characteristic of the face in the front image is extracted so that one or more of various many characters stored in the character database 25 can be selected. [0148] That is, in the character database 25, character information has been previously registered. Here, the character information includes character images (e.g., two-dimensional (2D) character image data, data representing 3D characters generated by computer graphics), and a face characteristic and a character ID each corresponding to the character images. By referring to the face characteristic in the front image provided by the editing part 26, the character selection part 24 selects from the character database 25 one or more of the character images having the identical or analogous face characteristic thereto. Here, the face characteristic is exemplary of the size, the length-to-width ratio, and the partial characteristic, all of which are represented by value. Further, the partial characteristic is exemplary of the size of the eyes, the nose, or the mouth, the positional relationship thereamong, and the amount or color of the hair, all of which are also represented by value. The presence or absence of glasses may be also a possibility as the face characteristic. [0149] Described below is about the face characteristic in more detail. FIG. 16 is a diagram exemplarily showing face characteristic values specifically focusing on the hair. FIG. 16 shows six images of each different user, and each corresponding thereto, processing results and characteristic representations. Here, although the users' images are usually picture images, shown in FIG. 16 are their portraits for easy view. Moreover, the face characteristic is not limited to the characteristic values and the characteristic representations, and either one of those, or any other value or representation will do. [0150] In FIG. 16, on the presumption that the hair is black, the users' images are subjected to processing of extracting any black region therefrom, and the results are the processing results. This is not surely restrictive, and no matter what color the hair is, the processing can be similarly carried out by extracting any region in the corresponding color. Here, the characteristic values are exemplified by the normalized area and circumference. The normalized area is a value obtained by normalizing the hair area by the face area. The normalized circumference is a value obtained by normalizing the circumference of the hair area by that of the face area. As for the characteristic expressions, exemplified are the amount of hair and the hair style. The amount of hair is roughly represented in two categories based on the average amount of hair. To be specific, if the normalized area shows a smaller value than the average, the user's hair is considered large in amount, and with a larger value, the amount of hair is considered small. Similarly, the hair style is also roughly represented in two categories based on the general hair style. To be specific, if the normalized circumference shows a smaller value than the average, the user's hair style is considered short, and with a larger value, the hair style is considered long. As such, by using thus extracted face characteristic values or the characteristic representations thereof, one or more of any analogous character images can be selected from the character database 25. [0151] There have been various methods for extracting such face characteristic values. As one example, there is a method described in “Smartface”—A Robust Face Recognition System under Varying Facial Pose and Expression (Publication of The Electronic Information Communications Society, Vol. J84-D-II, No. 6). In detail, in the method, a face region is first detected under the subspace method, and then the face parts (e.g., eyes, nose, mouth) are detected by using a separation filter. In the present media editing device, by applying at least one of such various known methods, the face characteristic values can be extracted in an easy manner. [0152] In order to select any potential character registered in the character database 25 with reference to the extracted face characteristic values, used may be the aforementioned characteristic representations, or correlation values calculated with respect to the registered face characteristic values. Here, if the correlation value exceeds a threshold value set for the potential character images considered suitable, the corresponding character image is extracted as a potential. The character selection part 24 then notifies the character ID corresponding to thus extracted potential character to the editing part 26. [0153] Based on the notified character ID, the editing part 26 displays the character image selected as the potential to the user. FIG. 17 is a diagram showing an exemplary editing screen for selecting which character to use. FIG. 17 shows three potential characters, and an arrow therein is a cursor indicating which character the user is going to select. Here, using the cursor is not restrictive, and the character images may be sequentially inverted for selection, or enclosed by the thicker lines. [0154] On such an editing screen shown in FIG. 17, the user selects which character to use. The editing part 26 performs media editing for generating the meta data having the selected character ID described so as to generate transmission data. Here, the character image itself may be combined into the transmission data. The resultant transmission data is stored in the transmission data storage 15, and transmitted with a timing considered appropriate to the data addressee. [0155] [0155]FIG. 18 is a diagram showing an exemplary screen on the receiver end receiving the transmission data generated as such. As shown in FIG. 18, in the lower left of the screen, displayed is a character selected by the user (addresser), and in the lower right, a message in the form of moving images is displayed. [0156] [0156]FIG. 19 is a diagram showing another exemplary screen on the receiver end receiving the transmission data. As shown in FIG. 19, displayed in the lower part of the screen is a character selected by the user (addresser). Here, during when the message in the form of moving images is reproduced, the character may not be displayed, and in the meantime, the moving images may take over its display position. Such a layout may be generated by the editing part 26, or set on the receiver end. [0157] Here, the number of potential character to be selected may be one, and if this is the case, mail creation becomes easier without selecting any potential character. [0158] The editing part 26 may notify a character string which indicates the characteristic values (or characteristic representations) inputted by the user. As an example, the user may input a character string of “the amount of hair is large, and the hair style short”. In response, the character selection part 24 then refers to such characteristic representations as shown in FIG. 16 for comparison, and selects a potential character. As such, with the help of a character string indicating the characteristic values, the potential character selected by the present media editing device can be closely analogous to the user's intended character. [0159] Further, as already described, the present media editing device is not limited to be of an integrated-type including every constituent in one housing, but may be of a distributed-type where each constituent performs data exchange over the network or communications lines. If this is the case, the character selection part 24 and the character database 25 may be located separately from the media editing device, and be accessible over the network. FIG. 20 is a block diagram showing the structure of such a distributed-type media editing device (system). [0160] In FIG. 20, such a distributed-type media editing device includes a character mail editing terminal 501, a character selection part 724, and a character database 725, which are interconnected over a network 600. Here, the character mail editing terminal 501 has the functions, partially or entirely, of the media editing devices of the first to third embodiments, and the character selection part 724 is located separately therefrom. Since this distributed-type media editing device is similar in structure and operation to the integrated-type, the same effects are to be achieved. Further, in the distributed-type media editing device of FIG. 20, in addition to the character mail editing terminal 501, the character selection part 724 and the character database 725 may be used also by a character mail reception terminal 502, or the like, where incoming mails are received and edited. If so, when receiving a character ID in an character mail, the character mail reception terminal 502 only needs to receive the corresponding character image from the character database 725. In such a structure, terminals do not have to carry data large in amount. Moreover, in the case that the character mail reception terminal 502 operates as the media editing device when returning mails, the character selection part 724 and the character database 725 can be shared. [0161] As such, in the distributed-type media editing device, the character selection part 724 and the character database 725 can be shared by a plurality of users. Therefore, terminals have no need to include those constituents, and can use databases storing various many characters. [0162] As is known from the above, in the present media editing device, the user can easily create a character mail with any preferred character added thereto by narrowing down various many characters based on front images extracted from moving images. Further, with such a character mail, person-to-person communication can be smooth and active. [0163] While the invention has been described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is understood that numerous other modifications and variations can be devised without departing from the scope of the invention. Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5500671 *Oct 25, 1994Mar 19, 1996At&T Corp.Video conference system and method of providing parallax correction and a sense of presenceUS5864630 *Nov 20, 1996Jan 26, 1999At&T CorpMulti-modal method for locating objects in imagesUS5907604 *Mar 25, 1997May 25, 1999Sony CorporationImage icon associated with caller IDUS5999207 *May 16, 1997Dec 7, 1999Scientific-Atlanta, Inc.Method and apparatus for implementing a user interface for a videophone in a cable television networkUS6064436 *Jan 16, 1998May 16, 2000Sharp Kabushiki KaishaImage sequence encoding device and area extracting deviceUS6215890 *Sep 25, 1998Apr 10, 2001Matsushita Electric Industrial Co., Ltd.Hand gesture recognizing deviceUS6356339 *Jan 12, 1999Mar 12, 2002Fuji Photo Film Co., Ltd.Digital photo system which replaces a portion of subject matter common to two digital imagesUS6363160 *Jan 22, 1999Mar 26, 2002Intel CorporationInterface using pattern recognition and trackingUS6546052 *May 27, 1999Apr 8, 2003Canon Kabushiki KaishaImage processing apparatus and method, and computer-readable memoryUS6556775 *Feb 1, 1999Apr 29, 2003Matsushita Electric Industrial Co., Ltd.Image and sound reproduction systemUS6573908 *Apr 7, 2000Jun 3, 2003Korea Firstec Co., Ltd.Method and system for providing object information within frames of moving image dataUS6577807 *Jul 6, 1999Jun 10, 2003Hitachi Denshi Kabushiki KaishaEditing method and apparatus for moving picturesUS6594375 *Sep 22, 1999Jul 15, 2003Canon Kabushiki KaishaImage processing apparatus, image processing method, and storage mediumUS6611613 *Jun 30, 2000Aug 26, 2003Samsung Electronics Co., Ltd.Apparatus and method for detecting speaking person's eyes and faceUS6636635 *Oct 31, 1996Oct 21, 2003Canon Kabushiki KaishaObject extraction method, and image sensing apparatus using the methodUS6661448 *Jun 26, 2001Dec 9, 2003Nokia Mobile Phones Ltd.Method and system for providing and transmitting alternative video data during interruptions in video transmissionsUS6697503 *Nov 30, 2000Feb 24, 2004Matsushita Electric Industrial Co., Ltd.Device and method for face image extraction, and recording medium having recorded program for the methodUS6704029 *Apr 11, 2000Mar 9, 2004Canon Kabushiki KaishaMethod and apparatus for specifying scene information in a moving pictureUS6704433 *Dec 20, 2000Mar 9, 2004Matsushita Electric Industrial Co., Ltd.Human tracking device, human tracking method and recording medium recording program thereofUS6714216 *Sep 27, 1999Mar 30, 2004Sony CorporationVideo editing apparatus and methodUS6724417 *Nov 27, 2001Apr 20, 2004Applied Minds, Inc.Method and apparatus maintaining eye contact in video delivery systems using view morphingUS6754389 *Dec 1, 1999Jun 22, 2004Koninklijke Philips Electronics N.V.Program classification using object trackingUS6784899 *Mar 31, 2000Aug 31, 2004Ricoh Company, Ltd.Systems and methods for providing rich multimedia messages to remote users using telephones and facsimile machinesUS6804396 *Mar 28, 2001Oct 12, 2004Honda Giken Kogyo Kabushiki KaishaGesture recognition system* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7286856 *Nov 24, 2003Oct 23, 2007Nec CorporationCellular telephone set capable of self-producing display animation and method of producing display animation used in sameUS7440013Aug 12, 2004Oct 21, 2008Fujifilm CorporationImage pickup device with facial region detector and method of synthesizing image including facial regionUS7472063 *Dec 19, 2002Dec 30, 2008Intel CorporationAudio-visual feature fusion and support vector machine useful for continuous speech recognitionUS7555323Oct 16, 2006Jun 30, 2009Sharp Kabushiki KaishaCellular phoneUS7650171Oct 16, 2006Jan 19, 2010Sharp Kabushiki KaishaCellular phoneUS7650172Oct 31, 2007Jan 19, 2010Sharp Kabushiki KaishaCellular phoneUS7653420Oct 16, 2006Jan 26, 2010Sharp Kabushiki KaishaCellular phoneUS7660613Oct 31, 2007Feb 9, 2010Sharp Kabushiki KaishaCellular phoneUS7668577Aug 20, 2007Feb 23, 2010Sharp Kabushiki KaishaCellular phoneUS7747296Aug 20, 2007Jun 29, 2010Sharp Kabushiki KaishaCellular phoneUS7751849Aug 17, 2007Jul 6, 2010Sharp Kabushiki KaishaFolding cellular phone with a camera and television modeUS7751859Oct 16, 2006Jul 6, 2010Sharp Kabushiki KaishaCellular phoneUS7755566 *Dec 27, 2001Jul 13, 2010Nokia CorporationDisplaying an imageUS7787916Oct 16, 2006Aug 31, 2010Sharp Kabushiki KaishaCellular phoneUS7787918Oct 31, 2007Aug 31, 2010Sharp Kabushiki KaishaCellular phoneUS7792556Aug 20, 2007Sep 7, 2010Sharp Kabushiki KaishaCellular phoneUS7822444Oct 31, 2007Oct 26, 2010Sharp Kabushiki KaishaCellular phoneUS7864198Aug 4, 2006Jan 4, 2011Vodafone Group Plc.Image processing method, image processing device and mobile communication terminalUS7916894 *Jan 29, 2007Mar 29, 2011Adobe Systems IncorporatedSummary of a video using facesUS7937122Aug 20, 2007May 3, 2011Sharp Kabushiki KaishaCellular phoneUS7937123Aug 20, 2007May 3, 2011Sharp Kabushiki KaishaCellular phoneUS7996026Dec 13, 2007Aug 9, 2011Casio Computer Co., Ltd.Data reception apparatus having data display functionUS8049915 *Aug 6, 2007Nov 1, 2011Canon Kabushiki KaishaImage processing apparatus, image processing method and storage mediumUS8103062 *Aug 23, 2006Jan 24, 2012Sony CorporationVideo processing apparatusUS8384791Nov 28, 2003Feb 26, 2013Sony United Kingdom LimitedVideo camera for face detectionUS8457363 *May 19, 2005Jun 4, 2013Samsung Electronics Co., Ltd.Apparatus and method for detecting eyesUS8467509Sep 18, 2009Jun 18, 2013Canon Kabushiki KaishaVideo displaying apparatus, video displaying system and video displaying methodUS8532462Nov 16, 2010Sep 10, 2013Twentieth Century Fox Film CorporationNon-destructive file based mastering for multiple languages and versionsUS8558952 *May 15, 2008Oct 15, 2013Nec CorporationImage-sound segment corresponding apparatus, method and programUS8584256Apr 20, 2011Nov 12, 2013Fox Entertainment Group, Inc.Digital delivery system and user interface for enabling the digital delivery of media contentUS8712219 *Dec 28, 2004Apr 29, 2014Core Wireless Licensing S.A.R.L.Method for editing a media clip in a mobile terminal device, a terminal device utilizing the method and program means for implementing the methodUS8861889 *Mar 28, 2011Oct 14, 2014Fujifilm CorporationImage processing device, method and program for extracting still image data from moving image dataUS8872843Jul 1, 2005Oct 28, 2014Samsung Electronics Co., Ltd.Method for editing images in a mobile terminalUS8892446Dec 21, 2012Nov 18, 2014Apple Inc.Service orchestration for intelligent automated assistantUS8903716Dec 21, 2012Dec 2, 2014Apple Inc.Personalized vocabulary for digital assistantUS8930191Mar 4, 2013Jan 6, 2015Apple Inc.Paraphrasing of user requests and results by automated digital assistantUS8942986Dec 21, 2012Jan 27, 2015Apple Inc.Determining user intent based on ontologies of domainsUS8952985 *Oct 19, 2012Feb 10, 2015Fujifilm CorporationDigital comic editor, method and non-transitory computer-readable mediumUS8977255Apr 3, 2007Mar 10, 2015Apple Inc.Method and system for operating a multi-function portable electronic device using voice-activationUS8983229 *Jul 21, 2006Mar 17, 2015Samsung Electronics Co., Ltd.Method, medium, and system with a combinational photograph thumbnailUS8989492 *Jun 4, 2012Mar 24, 2015Apple Inc.Multi-resolution spatial feature extraction for automatic handwriting recognitionUS9007633 *Mar 15, 2013Apr 14, 2015Fuji Xerox Co., Ltd.Systems and methods for efficiently capturing high-quality scans of multi-page documents with hand-held devicesUS9075998Oct 7, 2013Jul 7, 2015Fox Entertainment Group, Inc.Digital delivery system and user interface for enabling the digital delivery of media contentUS9117447Dec 21, 2012Aug 25, 2015Apple Inc.Using event alert text as input to an automated assistantUS9190062Mar 4, 2014Nov 17, 2015Apple Inc.User profiling for voice input processingUS9230298 *Nov 11, 2010Jan 5, 2016Sony CorporationInformation processing device, and information processing systemUS9262612Mar 21, 2011Feb 16, 2016Apple Inc.Device access using voice authenticationUS9300784Jun 13, 2014Mar 29, 2016Apple Inc.System and method for emergency calls initiated by voice commandUS9306882Jan 12, 2015Apr 5, 2016Google Inc.Management and presentation of notification contentUS9318108Jan 10, 2011Apr 19, 2016Apple Inc.Intelligent automated assistantUS9330720Apr 2, 2008May 3, 2016Apple Inc.Methods and apparatus for altering audio output signalsUS9338493Sep 26, 2014May 10, 2016Apple Inc.Intelligent automated assistant for TV user interactionsUS9368114Mar 6, 2014Jun 14, 2016Apple Inc.Context-sensitive handling of interruptionsUS9430463Sep 30, 2014Aug 30, 2016Apple Inc.Exemplar-based natural language processingUS9431006Jul 2, 2009Aug 30, 2016Apple Inc.Methods and apparatuses for automatic speech recognitionUS9483461Mar 6, 2012Nov 1, 2016Apple Inc.Handling speech synthesis of content for multiple languagesUS9495129Mar 12, 2013Nov 15, 2016Apple Inc.Device, method, and user interface for voice-activated navigation and browsing of a documentUS9502031Sep 23, 2014Nov 22, 2016Apple Inc.Method for supporting dynamic grammars in WFST-based ASRUS9514512Dec 18, 2013Dec 6, 2016Samsung Electronics Co., Ltd.Method and apparatus for laying out image using image recognitionUS9535906Jun 17, 2015Jan 3, 2017Apple Inc.Mobile device having human language translation capability with positional feedbackUS9548050Jun 9, 2012Jan 17, 2017Apple Inc.Intelligent automated assistantUS9576574Sep 9, 2013Feb 21, 2017Apple Inc.Context-sensitive handling of interruptions by intelligent digital assistantUS9582608Jun 6, 2014Feb 28, 2017Apple Inc.Unified ranking with entropy-weighted information for phrase-based semantic auto-completionUS9620104Jun 6, 2014Apr 11, 2017Apple Inc.System and method for user-specified pronunciation of words for speech synthesis and recognitionUS9620105Sep 29, 2014Apr 11, 2017Apple Inc.Analyzing audio input for efficient speech and music recognitionUS9626955Apr 4, 2016Apr 18, 2017Apple Inc.Intelligent text-to-speech conversionUS9633004Sep 29, 2014Apr 25, 2017Apple Inc.Better resolution when referencing to conceptsUS9633660Nov 13, 2015Apr 25, 2017Apple Inc.User profiling for voice input processingUS9633674Jun 5, 2014Apr 25, 2017Apple Inc.System and method for detecting errors in interactions with a voice-based digital assistantUS9646609Aug 25, 2015May 9, 2017Apple Inc.Caching apparatus for serving phonetic pronunciationsUS9646614Dec 21, 2015May 9, 2017Apple Inc.Fast, language-independent method for user authentication by voiceUS9668024Mar 30, 2016May 30, 2017Apple Inc.Intelligent automated assistant for TV user interactionsUS9668121Aug 25, 2015May 30, 2017Apple Inc.Social remindersUS9697820Dec 7, 2015Jul 4, 2017Apple Inc.Unit-selection text-to-speech synthesis using concatenation-sensitive neural networksUS9697822Apr 28, 2014Jul 4, 2017Apple Inc.System and method for updating an adaptive speech recognition modelUS20020113757 *Dec 27, 2001Aug 22, 2002Jyrki HoiskoDisplaying an imageUS20030185191 *Mar 27, 2003Oct 2, 2003Casio Computer Co., Ltd.Data communication apparatus, data communication system, and data communication methodUS20040013416 *May 27, 2003Jan 22, 2004Kyung-Tae MokOptical disc playerUS20040036778 *Aug 22, 2002Feb 26, 2004Frederic VernierSlit camera system for generating artistic images of moving objectsUS20040072589 *Oct 9, 2003Apr 15, 2004Hiroyasu HamamuraCellular phoneUS20040102226 *Nov 24, 2003May 27, 2004Nec CorporationCellular telephone set, setting method of self-produced animation of consecutively taken pictures used in the same, and program to be used for the methodUS20040122675 *Dec 19, 2002Jun 24, 2004Nefian Ara VictorVisual feature extraction procedure useful for audiovisual continuous speech recognitionUS20050018050 *Jul 22, 2004Jan 27, 2005Casio Computer Co., Ltd.Wireless communication device, dynamic image preparation method and dynamic image preparation programUS20050021625 *Jun 23, 2004Jan 27, 2005Matsushita Elec. Ind. Co.Ltd.Communication apparatusUS20050036044 *Aug 12, 2004Feb 17, 2005Fuji Photo Film Co., Ltd.Image pickup device and image synthesizing methodUS20050129111 *Dec 16, 2003Jun 16, 2005Nokia CorporationTransform-domain video editingUS20050152668 *Dec 28, 2004Jul 14, 2005Nokia CorporationMethod for editing a media clip in a mobile terminal device, a terminal device utilizing the method and program means for implementing the methodUS20050212822 *Jun 3, 2003Sep 29, 2005Yoshinori HonmaDisplay device, display method and advertisement method using display deviceUS20050259873 *May 19, 2005Nov 24, 2005Samsung Electronics Co. Ltd.Apparatus and method for detecting eyesUS20060001758 *Jul 1, 2005Jan 5, 2006Samsung Electronics Co., Ltd.Method for editing images in a mobile terminalUS20060170791 *Nov 28, 2003Aug 3, 2006Porter Robert Mark SVideo cameraUS20070032202 *Oct 16, 2006Feb 8, 2007Hiroyasu HamamuraCellular phoneUS20070032203 *Oct 16, 2006Feb 8, 2007Hiroyasu HamamuraCellular phoneUS20070032204 *Oct 16, 2006Feb 8, 2007Hiroyasu HamamuraCellular phoneUS20070032205 *Oct 16, 2006Feb 8, 2007Hiroyasu HamamuraCellular phoneUS20070037617 *Oct 16, 2006Feb 15, 2007Hiroyasu HamamuraCellular phoneUS20070053660 *Aug 23, 2006Mar 8, 2007Mototsugu AbeVideo processing apparatusUS20070065044 *Jul 21, 2006Mar 22, 2007Samsung Electronics Co., Ltd.Method, medium, and system with a combinational photograph thumbnailUS20070298830 *Aug 20, 2007Dec 27, 2007Hiroyasu HamamuraCellular phoneUS20070298831 *Aug 20, 2007Dec 27, 2007Hiroyasu HamamuraCellular phoneUS20070298844 *Aug 20, 2007Dec 27, 2007Hiroyasu HamamuraCellular phoneUS20080004068 *Aug 20, 2007Jan 3, 2008Hiroyasu HamamuraCellular phoneUS20080020719 *Aug 20, 2007Jan 24, 2008Hiroyasu HamamuraCellular phoneUS20080020796 *Aug 17, 2007Jan 24, 2008Hiroyasu HamamuraCellular phoneUS20080064454 *Oct 31, 2007Mar 13, 2008Hiroyasu HamamuraCellular phoneUS20080070624 *Oct 31, 2007Mar 20, 2008Hiroyasu HamamuraCellular phoneUS20080070646 *Oct 31, 2007Mar 20, 2008Hiroyasu HamamuraCellular phoneUS20080070647 *Oct 31, 2007Mar 20, 2008Hiroyasu HamamuraCellular phoneUS20080077844 *Sep 26, 2007Mar 27, 2008Samsung Electronics Co., Ltd.Apparatus and method for managing multimedia content in mobile terminalUS20080102868 *Dec 13, 2007May 1, 2008Casio Computer Co., Ltd.Data Communication Apparatus, Data Communication System, and Data Communication MethodUS20080152188 *Aug 6, 2007Jun 26, 2008Canon Kabushiki KaishaImage processing apparatus, image processing method and storage mediumUS20100079675 *Sep 18, 2009Apr 1, 2010Canon Kabushiki KaishaVideo displaying apparatus, video displaying system and video displaying methodUS20100172591 *May 15, 2008Jul 8, 2010Masumi IshikawaImage-sound segment corresponding apparatus, method and programUS20100205277 *Oct 15, 2008Aug 12, 2010Junya NakayamaMobile terminal, mobile terminal system, control method of mobile terminal, program and storage mediumUS20100235746 *Sep 24, 2009Sep 16, 2010Freddy Allen AnzuresDevice, Method, and Graphical User Interface for Editing an Audio or Video Attachment in an Electronic MessageUS20100238323 *Mar 23, 2009Sep 23, 2010Sony Ericsson Mobile Communications AbVoice-controlled image editingUS20100257210 *Mar 30, 2010Oct 7, 2010Stickeryou Inc.Internet-Based Method and System for Making User-Customized StickersUS20110116764 *Nov 16, 2010May 19, 2011Twentieth Century Fox Film CorporationNon-destructive file based mastering for multiple languages and versionsUS20110242336 *Mar 28, 2011Oct 6, 2011Fujifilm CorporationImage processing device, image processing method, and programUS20120050542 *Jun 28, 2011Mar 1, 2012Tektronix, Inc.System for generating text defined test patternsUS20120287152 *Nov 11, 2010Nov 15, 2012Sony Computer Entertainment Inc.Information processing device, and information processing systemUS20120308138 *Jun 4, 2012Dec 6, 2012Apple IncMulti-resolution spatial feature extraction for automatic handwriting recognitionUS20130100161 *Oct 19, 2012Apr 25, 2013Fujifilm CorporationDigital comic editor, method and non-transitory computer-readable mediumUS20140268247 *Mar 15, 2013Sep 18, 2014Fuji Xerox Co., Ltd.Systems and Methods for Efficiently Capturing High-Quality Scans of Multi-Page Documents with Hand-Held DevicesUS20150046347 *Aug 7, 2013Feb 12, 2015Jeb C. GriebatComputer Program and Method for Jury SelectionUS20150058708 *Aug 23, 2013Feb 26, 2015Adobe Systems IncorporatedSystems and methods of character dialog generationCN103914280A *Jan 7, 2014Jul 9, 2014三星电子株式会社Method and apparatus for laying out image using image recognitionEP1349407A2 *Mar 27, 2003Oct 1, 2003Casio Computer Co., Ltd.Data communication apparatus, data communication system, and data communication methodEP1349407A3 *Mar 27, 2003Dec 6, 2006Casio Computer Co., Ltd.Data communication apparatus, data communication system, and data communication methodEP1408673A2 *Sep 23, 2003Apr 14, 2004Sharp Kabushiki KaishaFoldable cellular telephone incorporating two separate displays and a cameraEP1408673A3 *Sep 23, 2003Dec 15, 2004Sharp Kabushiki KaishaFoldable cellular telephone incorporating two separate displays and a cameraEP1424839A1Nov 19, 2003Jun 2, 2004Nec CorporationMethod for generating graphics animations from still pictures in a cellular telephone and associated cellular telephoneEP1507386A1Aug 11, 2004Feb 16, 2005Fuji Photo Film Co., Ltd.Image pickup device and image synthesizing methodEP1635569A2 *Sep 6, 2005Mar 15, 2006Nec CorporationTelevision telephone system, communication terminal device, character information transmission method used therefor, and programEP1635569A3 *Sep 6, 2005Sep 16, 2009Nec CorporationTelevision telephone system, communication terminal device, character information transmission method used therefor, and programEP1695551A1 *Oct 12, 2004Aug 30, 2006Nokia CorporationTransform-domain video editingEP1695551A4 *Oct 12, 2004Jun 13, 2007Nokia CorpTransform-domain video editingEP1713030A1 *Feb 7, 2005Oct 18, 2006Vodafone K.K.Image processing method, image processing apparatus, and mobile communication terminal apparatusEP1713030A4 *Feb 7, 2005May 2, 2007Vodafone KkImage processing method, image processing apparatus, and mobile communication terminal apparatusEP1760633A2 *Aug 30, 2006Mar 7, 2007Sony CorporationVideo processing apparatusEP1760633A3 *Aug 30, 2006Feb 9, 2011Sony CorporationVideo processing apparatusEP1773032A3 *Sep 23, 2003Sep 12, 2007Sharp CorporationFoldable cellular telephone incorporating two separate displays and a cameraEP1773034A3 *Sep 23, 2003Sep 12, 2007Sharp CorporationFoldable cellular telephone incorporating two separate displays and a cameraEP1773035A3 *Sep 23, 2003Sep 12, 2007Sharp CorporationFoldable cellular telephone incorporating two separate displays and a cameraEP1773036A3 *Sep 23, 2003Sep 12, 2007Sharp CorporationFoldable cellular telephone incorporating two separate displays and a cameraEP1906405A2Sep 21, 2007Apr 2, 2008Samsung Electronics Co., Ltd.Apparatus and method for managing multimedia content in mobile terminalEP1906405A3 *Sep 21, 2007Sep 3, 2008Samsung Electronics Co., Ltd.Apparatus and method for managing multimedia content in mobile terminalEP2753065A3 *Jan 2, 2014Jan 7, 2015Samsung Electronics Co., LtdMethod and apparatus for laying out image using image recognitionEP3086221A3 *Apr 21, 2016May 3, 2017Kyocera CorporationElectronic apparatus, image display system, control program, and method for operating electronic apparatusWO2004051981A3 *Nov 28, 2003Jul 7, 2005Clive GillardVideo cameraWO2005009044A1 *Jul 23, 2004Jan 27, 2005Casio Computer Co., Ltd.A method and device for configuring a multimedia message for presentationWO2010109274A1 *Aug 25, 2009Sep 30, 2010Sony Ericsson Mobile Communications AbVoice-controlled image editingWO2012030965A2 *Aug 31, 2011Mar 8, 2012Fox Entertainment Group, Inc.Localized media content editingWO2012030965A3 *Aug 31, 2011Jul 5, 2012Fox Entertainment Group, Inc.Localized media content editingWO2016014273A1 *Jul 13, 2015Jan 28, 2016Google Inc.Management and presentation of notification content* Cited by examinerClassifications U.S. Classification382/190, 348/E07.078, 382/309International ClassificationH04M3/53, H04M3/56, G06T5/00, H04N7/14, H04N1/387, G10L13/00, G06T7/60, H04M11/00, G06T11/60, H04N5/91, G06T3/00, G10L11/02Cooperative ClassificationH04N21/4223, H04M3/5307, H04M2201/38, H04N21/4788, H04M3/567, H04W4/12, G11B27/28, H04N21/8543, H04N21/41407, H04N21/47205, H04W4/18, H04N2007/145, H04W88/02, H04N7/141, H04N21/4331European ClassificationH04N21/8543, H04N21/4788, H04N21/433C, H04N21/414M, H04N21/4223, H04N21/472E, H04N7/14A, G11B27/28Legal EventsDateCodeEventDescriptionSep 12, 2001ASAssignmentOwner name: MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD., JAPANFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:IMAGAWA, KAZUYUKI;TAKATA, YUJI;MATSUO, HIDEAKI;AND OTHERS;REEL/FRAME:012164/0602Effective date: 20010906Mar 21, 2006CCCertificate of correctionApr 1, 2009FPAYFee paymentYear of fee payment: 4Mar 7, 2013FPAYFee paymentYear of fee payment: 8May 27, 2014ASAssignmentOwner name: PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMEFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PANASONIC CORPORATION;REEL/FRAME:033033/0163Effective date: 20140527Mar 23, 2017FPAYFee paymentYear of fee payment: 12RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services