Source: https://patents.google.com/patent/EP1443465A1/en
Timestamp: 2020-02-21 14:08:02
Document Index: 78362483

Matched Legal Cases: ['art 102', 'art 101', 'art 102', 'art 101', 'art 101', 'art 102', 'art 105', 'art 104', 'art 103', 'art 104', 'art 109', 'art 106', 'art 107', 'art 107', 'art 107', 'art 107', 'art 107', 'art 110', 'art 110', 'art 110', 'art 107', 'art 106', 'art 107', 'art 107', 'art 108', 'art 107', 'art 107']

EP1443465A1 - Face information transmission system - Google Patents
Face information transmission system Download PDF
EP1443465A1
EP1443465A1 EP04001619A EP04001619A EP1443465A1 EP 1443465 A1 EP1443465 A1 EP 1443465A1 EP 04001619 A EP04001619 A EP 04001619A EP 04001619 A EP04001619 A EP 04001619A EP 1443465 A1 EP1443465 A1 EP 1443465A1
EP04001619A
EP1443465B1 (en
Akira NTT DoCoMo Inc. IPD Matsuki
Saori NTT DoCoMo Inc. IPD Miyahara
Yukiko NTT DoCoMo Inc. IPD Tozawa
2003-01-31 Priority to JP2003024734 priority Critical
2003-01-31 Priority to JP2003024734A priority patent/JP3950802B2/en
2004-01-26 Application filed by NTT Docomo Inc filed Critical NTT Docomo Inc
2004-08-04 Publication of EP1443465A1 publication Critical patent/EP1443465A1/en
2006-08-16 Publication of EP1443465B1 publication Critical patent/EP1443465B1/en
230000001815 facial Effects 0 abstract claims description 38
The first generation part 102 generates first image information identifying the positions of characteristic points of the face, based on an image acquired and output by the image acquisition part 101. More specifically, as shown in Fig. 2A, the first generation part 102 identifies characteristic points 401 identifying the eyes and eyebrows, and characteristic points 402 identifying the mouth and nose, in a face image 403 of the subject contained in an image 40 output by the image acquisition part 101, and generates the face image 403 and characteristic points 401, 402 as first image information of a still image. When images output by the image acquisition part 101 are moving images, the image 40 shown in Fig. 2A and an image 40a after a prescribed time has elapsed are received. As shown in Fig. 2B, the image 40a includes the face image 403a having moved during the lapse of a prescribed length of time, and the characteristic points 401 a and 402a are identified in the face image 403a. Hence first image information in the case of moving images includes the face image 403 and characteristic points 401, 402, and the face image 403a and characteristic points 401 a, 402a. The first generation part 102 outputs the first image information thus generated to the image judgment part 105.
The phoneme identification part 104 is a part which identifies phonemes corresponding to utterances acquired and output by the utterance acquisition part 103. A phoneme is the smallest unit of sound which may affect the meaning [of an utterance]. For example, if an output utterance is "konnnichiwa" ("hello"), then phonemes are identified as "ko (h)", "n (ε)", "ni (l)", "chi (o)", "wa (u)". The phoneme identification part 104 outputs an utterance and identified phonemes to the phoneme judgment part 109.
The interrupt input part 106 receives an interrupt instruction input by the user of the mobile phone 10, and outputs the interrupt instruction to the second generation part 107. More specifically, when the user presses buttons to which are assigned instructions relating to such facial expressions as "laugh", "cry", or "be surprised", the corresponding instruction is input, and is output to the second generation part 107.
When the second generation part 107 generates a character image based on first image information, when for example [image generation] is based on the face image 403 and characteristic points 401, 402 shown in Fig. 2A, the positions of the characteristic points 401, 402 relative to the face image 403 are identified as a distribution. Based on this distribution, the second generation part 107 determines the positions of the characteristic points 501, 502 relative to the character face image 503 as shown in Fig. 3A, and generates a still character image 50. When the first image information corresponds to a moving image, that is, when [image generation] is based on the face image 403 and characteristic points 401, 402 shown in Fig. 2A and on the face image 403a and characteristic points 401a, 402a after the lapse of a prescribed length of time as shown in Fig. 2B, in addition to the character image 50 shown in Fig. 3A, a character image 50a is generated based on the character face image 503a and characteristic points 501a, 502a shown in Fig. 3B.
When the second generation part 107 generates a character image based on a phoneme, information stored in the character information storage part 110 is utilized. Fig. 4 shows an example of information stored in the character information storage part 110. According to the example shown in Fig. 4, a "phoneme", "characteristic point data", and a "character image" are stored in association in the character information storage part 110. The second generation part 107 extracts the "characteristic point data" and "character image" corresponding to each "phoneme" and generates a character image as a still image or as a moving image. In the example shown in Fig. 4, an image in the mouth area is displayed; but the correspondence may be with an image of the entire face. Further, images incorporating peculiarities of the user may be stored as "character images". A "character image" incorporating peculiarities of the user are not limited to those based on "phonemes", and may be applied to cases in which character images are generated based on "first image information" or on "utterances" as well.
When an interrupt instruction is output from the interrupt input part 106, the second generation part 107 changes the character image according to the interrupt instruction. For example, when a "laugh" interrupt instruction is output, the generated character image is changed so as to assume a laughing facial expression. The second generation part 107 outputs the generated or changed character image to the transmission part 108.
Next, a method of transmission of a character image using the mobile phone 10 is explained, using the flowcharts of Fig. 5 and Fig. 6. The flowcharts of Fig. 5 and Fig. 6 form a series. The user inputs to the mobile phone 10 an instruction to create and transmit a character image (step S01). In response to this input instruction, the operations of the steps S02, S03 and the steps S04, S05 explained next are performed in parallel.
When the first image information does not satisfy the prescribed conditions, and the phoneme also does not satisfy the prescribed conditions, but the utterance exceeds the prescribed threshold (step S06, and from step S08 to step S09), the second generation part 107 generates a character image according to the facial expression of the face of the subject, based on the utterance (step S 13).
When the first image information does not satisfy the prescribed conditions, and the phoneme also does not satisfy the prescribed conditions, and the utterance does not exceed the prescribed threshold (step S06, and from step S08 to step S09), the second generation part 107 generates a character image based on default information stored in advance (step S 14).
A face information transmission system, comprising:
image acquisition means to acquire an image of the face of a subject;
first generation means to generate first image information relating to said subject and including the positions of characteristic points of said face, based on the acquired image; second generation means to generate second image information according to the facial expression of the face of said subject based on the generated first image information; and, transmission means to transmit the generated second image information to a prescribed communication terminal.
The face information transmission system according to Claim 1, further comprising: utterance acquisition means to acquire utterances issued by said subject; and image judgment means to judge whether or not said first image information satisfies prescribed conditions, wherein, when said image judgment means judges that said first image information satisfies prescribed conditions, said second generation means generates second image information according to the facial expression of the face of said subject based, at least, on said first image information, and when said image judgment means judges that said first image information does not satisfy prescribed conditions, said second generation means generates second image information according to the facial expression of the face of said subject based on said utterances.
The face information transmission system according to Claim 2, further comprising: phoneme identification means to identify phonemes corresponding to an utterance acquired by said utterance acquisition means; and phoneme judgment means to judge whether an identified phoneme satisfies prescribed conditions, wherein, when said phoneme judgment means judges that said phoneme satisfies prescribed conditions, said second generation means generates second image information according to the facial expression of the face of said subject based, at least, on said phoneme, and when said phoneme judgment means judges that said phoneme does not satisfy prescribed conditions, said second generation means generates second image information according to the facial expression of the face of said subject based on said first image information.
The face information transmission system according to Claim 3, wherein, when neither said first image information nor said phoneme satisfies the respective prescribed conditions and said utterance cannot be acquired, said second generation means employs image information determined in advance as said second image information.
The face information transmission system according to Claim 1, wherein said first image information comprises information to identify the distribution of said characteristic points in the face of said subject.
The face information transmission system according to Claim 1, wherein said image acquisition means acquires images of said face along a time series, and said first generation means generates said first image information including displacements in the positions of said characteristic points along a time series based on the acquired images.
The face information transmission system according to Claim 6, wherein said first image information includes information to identify the movement of said characteristic points relative to the face of said subject.
EP04001619A 2003-01-31 2004-01-26 Face information transmission system Active EP1443465B1 (en)
JP2003024734 2003-01-31
JP2003024734A JP3950802B2 (en) 2003-01-31 2003-01-31 Face information transmission system, face information transmission method, face information transmission program, and computer-readable recording medium
EP1443465A1 true EP1443465A1 (en) 2004-08-04
EP1443465B1 EP1443465B1 (en) 2006-08-16
ID=32652930
EP04001619A Active EP1443465B1 (en) 2003-01-31 2004-01-26 Face information transmission system
US (1) US7224367B2 (en)
EP (1) EP1443465B1 (en)
JP (1) JP3950802B2 (en)
KR (1) KR100629643B1 (en)
CN (1) CN1237799C (en)
DE (1) DE602004001887T2 (en)
KR100686076B1 (en) * 2005-03-24 2007-02-26 엘지전자 주식회사 Wireless Communication Terminal with Message Transmission According to Feeling of Terminal-User and Method of Message Transmission Using Same
US20080266112A1 (en) * 2005-12-22 2008-10-30 Koninklijke Philips Electronics N.V. Valentine Pillow
KR101009151B1 (en) 2008-09-04 2011-01-18 삼성전기주식회사 Apparatus for modification in the setting and method thereof in the portable terminal
KR101541907B1 (en) 2008-10-14 2015-08-03 삼성전자 주식회사 Apparatus and method for generating face character based on voice
WO2012063123A2 (en) 2010-11-09 2012-05-18 WOOW Inc. Limited Apparatus and method for physical interaction over a distance using a telecommunication device
JP5555193B2 (en) 2011-03-01 2014-07-23 株式会社メガチップス Data processing apparatus, data processing system, and program
CN102571631B (en) * 2011-12-23 2016-08-31 上海量明科技发展有限公司 The sending method of motion images information, terminal and system in instant messaging
US9609272B2 (en) 2013-05-02 2017-03-28 Avaya Inc. Optimized video snapshot
CN106445107A (en) * 2016-08-30 2017-02-22 苏州品诺维新医疗科技有限公司 Method and device for sending information on basis of facial expressions
JP3478255B2 (en) 2000-08-01 2003-12-15 カシオ計算機株式会社 Face image creation device and face image creation control method
JP3593067B2 (en) 2001-07-04 2004-11-24 沖電気工業株式会社 Information terminal device with image communication function and image distribution system
2003-01-31 JP JP2003024734A patent/JP3950802B2/en active Active
2003-12-31 US US10/748,248 patent/US7224367B2/en active Active
2004-01-26 EP EP04001619A patent/EP1443465B1/en active Active
2004-01-26 DE DE602004001887T patent/DE602004001887T2/en active Active
2004-01-30 KR KR1020040006230A patent/KR100629643B1/en active IP Right Grant
2004-01-30 CN CNB2004100037869A patent/CN1237799C/en active IP Right Grant
KSHIRSAGAR S ET AL: "Personalized face and speech communication over the internet", PROCEEDINGS IEEE 2001 VIRTUAL REALITY. (VR). YOKOHAMA, JAPAN, MARCH 13 - 17, 2001, PROCEEDINGS IEEE VIRTUAL REALITY.(VR), LOS ALAMITOS, CA, IEEE COMP. SOC, US, 13 March 2001 (2001-03-13), pages 37 - 44, XP010535482, ISBN: 0-7695-0948-7 *
CN102156888B (en) * 2011-04-27 2013-06-12 西安电子科技大学 Image sorting method based on local colors and distribution characteristics of characteristic points
DE602004001887T2 (en) 2007-02-08
JP2004236186A (en) 2004-08-19
EP1443465B1 (en) 2006-08-16
DE602004001887D1 (en) 2006-09-28
US7224367B2 (en) 2007-05-29
KR20040070326A (en) 2004-08-07
CN1527602A (en) 2004-09-08
JP3950802B2 (en) 2007-08-01
KR100629643B1 (en) 2006-09-29
US20040207720A1 (en) 2004-10-21
CN1237799C (en) 2006-01-18
DE10063089C1 (en) 2002-07-25 User-controlled linking of information within an augmented reality system
TW571290B (en) 2004-01-11 Computer-implemented speech recognition system training
US20060184526A1 (en) 2006-08-17 Customer feedback acquisition and processing system
US20030163311A1 (en) 2003-08-28 Intelligent social agents
JP3514372B2 (en) 2004-03-31 Multimodal dialogue device
US20150073811A1 (en) 2015-03-12 Systems and Methods for Generating Markup-Language Based Expressions from Multi-Modal and Unimodal Inputs
Inventor name: TOZAWA, YUKIKOC/O INT. PROP. DEPARTMENT
Inventor name: MATSUKI, AKIRA, NTT DOCOMO INC., IPD
Inventor name: MIYAHARA, SAORI, NTT DOCOMO INC., IPD
Ref document number: 602004001887