Source: https://patents.google.com/patent/JP4862447B2/en
Timestamp: 2020-01-26 15:09:57
Document Index: 16752754

Matched Legal Cases: ['art 17', 'art 16', 'art 12', 'art 13', 'art 14', 'art 15', 'art 18', 'art 19']

JP4862447B2 - Face recognition system - Google Patents
JP4862447B2
JP4862447B2 JP2006079844A JP2006079844A JP4862447B2 JP 4862447 B2 JP4862447 B2 JP 4862447B2 JP 2006079844 A JP2006079844 A JP 2006079844A JP 2006079844 A JP2006079844 A JP 2006079844A JP 4862447 B2 JP4862447 B2 JP 4862447B2
JP2006079844A
JP2007257221A (en
誠 増田
景洋 長尾
2006-03-23 Application filed by 沖電気工業株式会社 filed Critical 沖電気工業株式会社
2006-03-23 Priority to JP2006079844A priority Critical patent/JP4862447B2/en
2007-10-04 Publication of JP2007257221A publication Critical patent/JP2007257221A/en
2012-01-25 Publication of JP4862447B2 publication Critical patent/JP4862447B2/en
The present invention relates to a face recognition system.
Conventionally, a personal authentication device for preventing an unqualified person from operating a device such as a personal authentication device or an information processing device for preventing an unqualified person from entering a building or a specific place in the building. In a personal authentication device for performing identity verification in electronic commerce using the Internet, a biometric that uses the biological characteristics of the individual to identify himself / herself without using a key or personal identification number A personal authentication device using a metrics authentication system is provided (see, for example, Patent Document 1).
In a personal authentication device using the biometrics authentication system, a system that authenticates by fingerprint, a system that authenticates by voiceprint, a system that authenticates by facial expression, etc. are used.
In a face recognition system, that is, a face recognition system, when performing face recognition, a facial feature amount is extracted from a face image, and the similarity with a pre-registered feature amount is compared and recognized. It has become. In this case, the positions of both eyes and mouth are detected, and thereby face recognition is performed by a template matching process using the corrected template as a facial feature amount. At this time, it is a precondition that the face image includes both eyes and mouth, and if either eye detection or mouth detection fails, it is determined that no face exists in the image. Warning is issued.
JP 2000-163600 A
However, in the conventional face recognition system, if the user is wearing a mask, the mouth cannot be detected, so that face recognition cannot be performed properly. For example, a user often wears a mask during a time when the common cold is prevalent, such as in winter, or when hay fever is prevalent, such as in early spring. In such a case, in the personal authentication device using the conventional face recognition system, the feature amount of the mouth changes greatly due to the influence of the mask, so that the face recognition system appropriately recognizes the face wearing the mask Cannot be verified and the user's identity fails. Therefore, the user has to remove the mask at the time of authentication, which is troublesome and feels troublesome.
The present invention solves the problems of the conventional face recognition system and includes a feature quantity selection unit that selects a face part from which a feature quantity is to be extracted. You can select the part of the face that you want to extract, and even if you wear a mask or eye patch or the hairstyle changes, emphasize the feature value of the selected part to other parts It is an object of the present invention to provide a face recognition system capable of appropriately performing face recognition by reducing the feature amount of.
For this purpose, in the face recognition system of the present invention, an image input unit that acquires a user's face image, a database that stores feature amounts of a registered person's face, and a facial part from the user's face image A face position detecting unit for detecting the position of the face, a feature amount extracting unit for extracting a feature amount of the facial part, a feature amount of the facial part extracted by the feature amount extracting unit, and a registered person stored in the database A recognition unit that recognizes the user by comparing the feature quantity of a facial part of the face and a feature quantity selection unit that selects a facial part that is operated by the user to extract the feature quantity In the system, on the feature quantity selection screen displayed on the feature quantity selection unit, the user selects a face part from which the feature quantity is extracted, and the user selects the face part from which the user extracts the feature quantity. When selected, the image input unit selects the user's face image. Tokushi, the face position detecting section, the user from the image input unit obtains the face image is detected only the position of the site of selected face.
According to the present invention, the face recognition system includes a feature amount selection unit that selects a part of a face from which a feature amount is to be extracted. Therefore, the user who performs face recognition can select the part of the face from which the feature value is to be extracted by himself / herself, even if he / she wears a mask or eye patch or the hairstyle changes. Face recognition can be performed appropriately by emphasizing the feature quantity of a part and reducing the feature quantity of another part.
FIG. 1 is a diagram showing a configuration of a face recognition system according to the first embodiment of the present invention.
In the figure, reference numeral 10 denotes a face recognition system that performs face recognition by acquiring a face image of the user 19 photographed by the camera 21, and is a kind of computer system that operates according to a program. The face recognition system 10 is a system that is used for personal authentication of the user 19 based on the facial appearance that is one of the biological characteristics of the user 19, and may be used in any application. . For example, it is used to perform identity verification when entering a specific building such as a condominium, a factory, an office, or a specific place in the building. It can also be used to verify the identity of the person who owns the air ticket at the time of check-in at the airport. In addition, request financial transactions at the counters of financial institutions such as banks, post offices, and credit unions, and operate financial transactions by operating automatic transaction machines such as ATMs (Automatic Teller Machines). It can also be used to verify your identity when Furthermore, it can also be used to perform identity verification in electronic commerce using the Internet.
The user 19 may be a specific building such as a condominium, factory, office, or a resident or employee of a specific location in the building, or a traveler who performs check-in at an airfield. It may be a customer who requests a financial transaction at a sales counter of a financial institution or operates an automatic transaction apparatus to perform a financial transaction, or performs an electronic commerce using the Internet. Any person may be used as long as the person is identified using the face recognition system 10.
The camera 21 includes an image pickup device such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor), an optical lens, a communication interface, and the like, and can be connected to the face recognition system 10 to take a picture. The digital still camera that shoots a still image or the digital video camera that shoots a moving image may be used as long as the image can be output in the form of electronic data. It may be a thing. The camera 21 may be, for example, a surveillance camera disposed at a specific building or an entrance of a specific place in the building, or may be a built-in camera built in an automatic transaction apparatus. A camera connected to a personal computer or the like, or a camera built in a notebook personal computer, PDA (Personal Digital Assistant), electronic notebook, mobile phone, PHS (Personal Handy-Phone System) phone, etc. It may be.
Then, the face recognition system 10 includes a feature quantity selection unit 11 that selects a face part from which a facial feature quantity used for face recognition is extracted, and an image input unit that obtains a face image for each frame from the camera 21. 12, a face position detection unit 13 that detects the position of the face of the user 19 from the image acquired by the image input unit 12, for example, the position of the eyes, mouth, and the like. , A feature quantity extraction unit 14 for extracting feature quantities of parts such as mouths, a database 15 in which feature quantities of registered person's face registered in advance are stored, and feature quantities of each registered person stored in the database 15 And a feature quantity extracted by the feature quantity extraction unit 14, a recognition unit 16 that recognizes a person in the image, and a recognition result output unit 17 that outputs a recognition result of the recognition unit 16. . Note that the facial parts from which feature quantities used for face recognition are extracted are eyes, mouth, ears, nose, etc., but those that can identify the person using biological features. For example, it may be any part of the face. Here, it demonstrates as the said part being eyes and a mouth.
Here, the feature quantity selection unit 11 is a means for the user 19 to select the face part from which the feature quantity is to be extracted according to the situation, for example, when the user 19 is wearing a mask. Can be selected to use the eye and mouth feature values when the eye feature value is not important, that is, when the mask is not applied.
The image input unit 12 is connected to a camera 21 as a photographing unit for photographing a face image of the user 19 so as to be communicable via a wired or wireless network. Get a face image. The face image is preferably digital image data. The network may be a communication cable such as a USB (Universal Serial Bus) cable, or a communication line network such as a telephone line network, the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network). May be.
Further, the recognition result output unit 17 is communicably connected to the speaker 22 and the monitor 23 via a wired or wireless network, and creates a message or an image for outputting the recognition result to the speaker 22 or the monitor 23. . Here, the speaker 22 may be in any form, and may be disposed in any place. For example, it may be a speaker such as an interphone disposed at the entrance of a specific building or a specific location in the building, a built-in speaker built in an automatic transaction apparatus, or a personal computer. Or a speaker built into a notebook personal computer, PDA, electronic notebook, mobile phone, PHS phone, or the like.
The monitor 23 includes display means such as a CRT, a liquid crystal display, and an LED (Light Emitting Diode) display, and may be in any form or disposed at any place. May be. For example, it may be a monitor arranged at the entrance of a specific building or a specific place in the building, may be a display means of an automatic transaction apparatus, or may be a monitor connected to a personal computer or the like. It may be a monitor provided in a notebook personal computer, PDA, electronic notebook, mobile phone, PHS phone or the like.
Further, the network may be a communication cable such as a USB cable, or a communication line network such as a telephone line network, the Internet, a LAN, or a WAN.
Further, the face recognition system 10 is incorporated in any kind of computer as long as it has arithmetic means such as CPU and MPU, storage means such as magnetic disk and semiconductor memory, input / output interface and the like. For example, it may be incorporated in a personal computer, may be incorporated in a server, or may be incorporated in a computer network in which a plurality of computers are connected by a network. Also good.
Next, the operation of the face recognition system 10 having the above configuration will be described.
FIG. 2 is a diagram illustrating an example of a feature amount selection screen according to the first embodiment of the present invention, FIG. 3 is a flowchart illustrating an operation of face position detection processing according to the first embodiment of the present invention, and FIG. It is a flowchart which shows the operation | movement of the feature-value extraction process in the 1st Embodiment of invention.
First, the user 19 uses the feature amount selection unit 11 to select where the face portion from which the feature amount is to be extracted is to be selected, that is, which portion of the face is important as the face feature amount. In this case, for example, the face recognition system 10 displays a feature amount selection screen as shown in FIG. 2 on the monitor 23 and allows the user 19 to select whether or not a mask is put on. Then, when it is selected that the mask is applied, it is assumed that the use of the eye feature amount is selected as the face feature amount. In addition, when it is selected that the mask is not applied, it is assumed that the use of the feature amount of the eyes and mouth is selected.
Similarly, the user 19 can select whether or not the eye patch is worn. Then, when it is selected to wear the eye patch, it is assumed that the use of the mouth feature amount as the face feature amount is selected. Further, when it is selected that the eyepatch is not worn, it is assumed that the use of eye and mouth feature values is selected. Further, similarly, the user 19 can select whether or not the hairstyle has changed greatly.
Subsequently, the image input unit 12 acquires the face image of the user 19 captured by the camera 21 for each frame.
Subsequently, the face position detection unit 13 detects the position of each part of the face from the face image acquired by the image input unit 12. In this case, only the position of the part selected by the feature quantity selection unit 11 is detected. For example, when using the feature amount for each of the eyes and the mouth, the respective positions are detected in order. First, it is determined whether or not the use of the eye feature is selected, and if it is selected, the position of the eye is detected. Further, the eye position is not detected when it is not selected. Subsequently, it is determined whether or not the use of the mouth feature amount is selected, and when it is selected, the position of the mouth is detected. In addition, when the position is not selected, the position of the mouth is not detected.
As a method for detecting each part of the face, for example, a method for obtaining a rough position of the face by skin color and then obtaining a position of each part by a projection histogram of the filtering result by the low frequency filter and template matching is used. Since it is known as a component search method (for example, refer to Patent Document 2), the description thereof is omitted here.
[Patent Document 2] Japanese Patent Application Laid-Open No. 2003-281539 Subsequently, the feature amount extraction unit 14 extracts a feature amount used for face recognition. At this time, the feature quantity is extracted only for the part selected by the feature quantity selection unit 11. For example, the feature amount is extracted for the selected eye or mouth. As a feature amount used for face recognition, for example, a method of filtering the eyes, nose, and mouth using several types of Gabor filters is known (see, for example, Non-Patent Document 1). Description is omitted.
[Non-patent document 1] “Face recognition by Gabor transformation using automatic feature point extraction” Yoshida, Kure, Shioyama, IEICE Technical Report PRMU2001-202
The database 15 stores facial features of one or more registered persons, and all parts of the face (for example, eyes, mouth, ears, nose, etc.) extracted from each registered person's face image. The feature amount is stored.
Subsequently, the recognition unit 16 compares the feature amount extracted from the face image of the user 19 by the feature amount extraction unit 14 with the feature amount of the registered person's face stored in the database 15, thereby performing face recognition. I do. In this case, the recognition unit 16 compares the feature amount extracted from the face image of the user 19 with the feature amount of the registered person with respect to the part (for example, eyes or mouth) selected by the feature amount selection unit 11. The recognition score is calculated. Note that, as a pattern matching method for calculating the recognition score, for example, a method using a normalized cross-correlation (for example, see Non-Patent Document 2) is known, and thus description thereof is omitted here. To do.
[Non-patent document 2] “Image processing and recognition” Aoiin, Nagao Here, the recognition score is calculated for the feature amount of each registered person. When the recognition score is maximum and exceeds a predetermined threshold (threshold) value, the recognition unit 16 determines that the corresponding registered person is the user 19. That is, the user 19 is the same person as the registered person, and determines that the face recognition is successful. In other cases, the recognition unit 16 determines that face recognition has failed.
And the recognition result output part 17 outputs the recognition result of the said recognition part 16, ie, whether the user's 19 face recognition was successful or failed from the speaker 22 and the monitor 23 with the sound or the image. Thereby, the recognition result is transmitted to the user 19.
Next, the flowchart shown in FIG. 3 will be described.
Step S1: It is determined whether or not the use of the feature amount of the first eye has been selected. If the use of the eye feature is selected, the process proceeds to step S2, and if the use of the eye feature is not selected, the process proceeds to step S3.
Step S2: The position of the eye is detected.
Step S3: It is determined whether or not the use of the mouth feature is selected. If the use of the mouth feature value is selected, the process proceeds to step S4. If the use of the mouth feature value is not selected, the process ends.
Step S4: The position of the mouth is detected, and the process is terminated.
Next, the flowchart shown in FIG. 4 will be described.
Step S11: It is determined whether to use the feature amount of the eye. If the use of the eye feature is selected, the process proceeds to step S12, and if the use of the eye feature is not selected, the process proceeds to step S13.
Step S12: The feature amount of the eye is extracted.
Step S13: It is determined whether or not the use of the mouth feature is selected. If the use of the mouth feature value is selected, the process proceeds to step S14. If the use of the mouth feature value is not selected, the process ends.
Step S14: The mouth feature value is extracted, and the process is terminated.
As described above, in this embodiment, the user 19 intentionally sets the feature amount by selecting which part of the face is important as the facial feature amount according to his / her state. Can do. Thereby, the face recognition of the user 19 can be appropriately performed even in a state where a mask is put on, a state where an eyepatch is worn, a state where the hairstyle changes greatly from the time of registration, and the like. For this reason, it is not necessary to remove the mask or eye patch during authentication, and the burden on the user 19 can be reduced.
FIG. 5 is a diagram showing a configuration of a face recognition system according to the second embodiment of the present invention.
As shown in the figure, the face recognition system 10 according to the present embodiment includes an image input unit 12, a face position detection unit 13, a feature amount extraction unit 14, a database 15, a recognition unit 16, a recognition result output unit 17, and a user. A state detection unit 18 is included. Note that the feature amount selection unit 11 is omitted. The user state detection unit 18 detects the state of the user 19 based on the result detected by the face position detection unit 13. For example, the user state detection unit 18 determines whether or not the user 19 is wearing a mask. to decide. Since the configuration of other points is the same as that of the first embodiment, description thereof is omitted.
Next, the operation of the face recognition system 10 in the present embodiment will be described.
First, the image input unit 12 acquires a face image of the user 19 captured by the camera 21 for each frame.
Subsequently, the face position detection unit 13 detects the position of each part of the face from the face image acquired by the image input unit 12.
Subsequently, the user state detection unit 18 detects the state of the user 19 by using position information of each part of the face detected by the face position detection unit 13. For example, the position of the mouth is estimated from the position of the eyes, and the vicinity of the estimated position of the mouth is matched with a mouth template registered in advance as registration data. When the cross-correlation value between the estimated mouth position and the mouth template is equal to or less than a predetermined threshold, the user state detection unit 18 determines that the mouth is not visible due to the influence of the mask, and the mouth feature amount Is selected not to be used for face recognition.
Similarly, the user state detection unit 18 matches, for example, the estimated eye position vicinity with an eye template registered in advance as registration data. When the estimated cross-correlation value between the vicinity of the eye position and the eye template is equal to or smaller than a predetermined threshold, the user state detection unit 18 determines that the eyes are not visible due to the influence of the eyepatch and the hairstyle, and the eye characteristics Choose not to use the amount for face recognition.
Subsequently, the feature amount extraction unit 14 extracts feature amounts used for face recognition. At this time, the feature quantity is extracted only for the part selected by the user state detection unit 18.
Subsequently, the recognition unit 16 compares the feature amount extracted from the face image of the user 19 by the feature amount extraction unit 14 with the feature amount of the registered person's face stored in the database 15, thereby performing face recognition. I do.
As described above, in the present embodiment, in accordance with the state of the user 19, which part of the face is regarded as important as the facial feature amount is automatically selected. Thereby, the face recognition of the user 19 can be appropriately performed even in a state where a mask is put on, a state where an eyepatch is worn, a state where the hairstyle changes greatly from the time of registration, and the like. For this reason, it is not necessary to remove the mask or eye patch during authentication, and the burden on the user 19 can be reduced.
It is a figure which shows the structure of the face recognition system in the 1st Embodiment of this invention. It is a figure which shows the example of the feature-value selection screen in the 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the face position detection process in the 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the feature-value extraction process in the 1st Embodiment of this invention. It is a figure which shows the structure of the face recognition system in the 2nd Embodiment of this invention.
DESCRIPTION OF SYMBOLS 10 Face recognition system 11 Feature quantity selection part 12 Image input part 13 Face position detection part 14 Feature quantity extraction part 15 Database 16 Recognition part 18 User state detection part 19 User
(A) an image input unit for acquiring a user's face image;
(B) a database storing feature values of facial parts of registered persons;
(C) a face position detection unit for detecting the position of a facial part from the user's face image;
(D) a feature amount extraction unit that extracts a feature amount of a facial part;
(E) a recognition unit that recognizes the user by comparing the feature quantity of the facial part extracted by the feature quantity extraction unit with the feature quantity of the facial part of the registered person stored in the database ;
(F) a face recognition system including a feature amount selection unit that selects a part of a face that is operated by the user and extracts a feature amount ;
(G) In the feature quantity selection screen displayed on the feature quantity selection unit, the user selects a face part from which the feature quantity is extracted;
(H) When the user selects a face part from which the feature amount is extracted, the image input unit acquires the face image of the user, and the face position detection unit acquires the face acquired by the image input unit. A face recognition system that detects only the position of a facial part selected by the user from an image .
JP2006079844A 2006-03-23 2006-03-23 Face recognition system Active JP4862447B2 (en)
JP2006079844A JP4862447B2 (en) 2006-03-23 2006-03-23 Face recognition system
TW096101788A TW200741562A (en) 2006-03-23 2007-01-17 Face recognition system
CN 200780009864 CN101405744A (en) 2006-03-23 2007-01-19 Face recognition system
US12/225,423 US8340366B2 (en) 2006-03-23 2007-01-19 Face recognition system
KR1020087025723A KR20090008256A (en) 2006-03-23 2007-01-19 Face recognition system
PCT/JP2007/050802 WO2007108225A1 (en) 2006-03-23 2007-01-19 Face recognition system
EP07707090A EP2026233A4 (en) 2006-03-23 2007-01-19 Face recognition system
JP2007257221A JP2007257221A (en) 2007-10-04
JP4862447B2 true JP4862447B2 (en) 2012-01-25
ID=38522261
JP2006079844A Active JP4862447B2 (en) 2006-03-23 2006-03-23 Face recognition system
US (1) US8340366B2 (en)
EP (1) EP2026233A4 (en)
JP (1) JP4862447B2 (en)
KR (1) KR20090008256A (en)
CN (1) CN101405744A (en)
TW (1) TW200741562A (en)
WO (1) WO2007108225A1 (en)
JP2010080993A (en) * 2008-09-23 2010-04-08 Brother Ind Ltd Intercom system
JP5480532B2 (en) * 2009-04-30 2014-04-23 グローリー株式会社 Image processing apparatus, image processing method, and program for causing computer to execute the method
JP4844670B2 (en) * 2009-11-13 2011-12-28 日本ビクター株式会社 Video processing apparatus and video processing method
RU2543950C2 (en) * 2011-12-28 2015-03-10 Кэнон Кабусики Кайся Image forming apparatus and control method therefor
KR102057947B1 (en) * 2013-04-01 2019-12-20 삼성전자주식회사 Display apparatus for performing an user certification and method thereof
US9892413B2 (en) 2013-09-05 2018-02-13 International Business Machines Corporation Multi factor authentication rule-based intelligent bank cards
WO2015164584A1 (en) 2014-04-23 2015-10-29 Google Inc. User interface control using gaze tracking
EP3035238A1 (en) * 2014-12-19 2016-06-22 Tata Consultancy Services Limited Video surveillance system and method for fraud detection
CN105631441A (en) * 2016-03-03 2016-06-01 暨南大学 Human face recognition method
EP3436926A4 (en) 2016-03-30 2019-11-13 Tinoq Inc Systems and methods for user detection and recognition
US20190031145A1 (en) * 2017-07-28 2019-01-31 Alclear, Llc Biometric identification system connected vehicle
TWI661398B (en) * 2018-03-31 2019-06-01 華南商業銀行股份有限公司 Transaction system based on face recognitioin for verification and method thereof
JP3970573B2 (en) * 2001-10-19 2007-09-05 アルパイン株式会社 Facial image recognition apparatus and method
JP2003281539A (en) 2002-03-25 2003-10-03 Oki Electric Ind Co Ltd Facial component searching device and facial component searching method
JP2003331264A (en) * 2002-05-13 2003-11-21 Nec Soft Ltd Facial image collation system and facial image collation method
2006-03-23 JP JP2006079844A patent/JP4862447B2/en active Active
2007-01-17 TW TW096101788A patent/TW200741562A/en unknown
2007-01-19 CN CN 200780009864 patent/CN101405744A/en not_active Application Discontinuation
2007-01-19 WO PCT/JP2007/050802 patent/WO2007108225A1/en active Application Filing
2007-01-19 KR KR1020087025723A patent/KR20090008256A/en not_active Application Discontinuation
2007-01-19 EP EP07707090A patent/EP2026233A4/en not_active Withdrawn
2007-01-19 US US12/225,423 patent/US8340366B2/en active Active
JP2007257221A (en) 2007-10-04
US8340366B2 (en) 2012-12-25
WO2007108225A1 (en) 2007-09-27
EP2026233A4 (en) 2009-12-02
EP2026233A1 (en) 2009-02-18
CN101405744A (en) 2009-04-08
TW200741562A (en) 2007-11-01
US20090110248A1 (en) 2009-04-30
KR20090008256A (en) 2009-01-21
Ref document number: 4862447