AUTHENTICATION METHOD, STORAGE MEDIUM, AND INFORMATION PROCESSING APPARATUS

An authentication method executed by a computer, the authentication method includes obtaining a captured image captured by a camera; selecting one facial image from a plurality of facial images based on a position of each of the plurality of facial images included in the captured image; referring to a memory that stores pieces of biometric information associated with the respective plurality of facial images; specifying a piece of the biometric information associated with a facial image in which a degree of similarity to the selected facial image satisfies a criterion; and performing, when biometric information detected by a sensor is received, authentication based on verification of the specified piece of the biometric information against the received biometric information.

FIELD

The present case relates to an authentication method, a storage medium, and an information processing apparatus.

BACKGROUND

There has been disclosed a biometric authentication technique of narrowing down candidates by authentication using first biometric information (e.g., facial features) and authenticating a person in question by authentication using second biometric information (e.g., palm venous features) (e.g., see Patent Document 1).

SUMMARY

According to an aspect of the embodiments, an authentication method executed by a computer, the authentication method includes obtaining a captured image captured by a camera; selecting one facial image from a plurality of facial images based on a position of each of the plurality of facial images included in the captured image; referring to a memory that stores pieces of biometric information associated with the respective plurality of facial images; specifying a piece of the biometric information associated with a facial image in which a degree of similarity to the selected facial image satisfies a criterion; and performing, when biometric information detected by a sensor is received, authentication based on verification of the specified piece of the biometric information against the received biometric information.

DESCRIPTION OF EMBODIMENTS

A plurality of faces may be imaged simultaneously depending on an installation condition of a camera and a usage condition of a user. In this case, the number of candidates to be narrowed down increases, which may increase the authentication time. When the narrowing-down rate is increased to reduce the number of candidates, the processing time for the face authentication may increase, which may cause a missing candidate (correct person may not be included in the narrowing-down list) depending on the accuracy of the face authentication.

In one aspect, it is an object of the present invention to provide an authentication method, an authentication program, and an information processing apparatus capable of shortening an authentication time.

It becomes possible to shorten an authentication time.

Prior to descriptions of embodiments, multi-biometric authentication that narrows down a search set by a first modality and identifies a user by another modality will be described.

Biometric authentication is a technique for verifying a person in question using biometric features such as fingerprints, faces, veins, and the like. In the biometric authentication, biometric information for verification obtained by a sensor is compared (verified) with registered biometric information registered in advance in a situation where confirmation is needed, and it is determined whether or not a degree of similarity is equal to or higher than an identity determination threshold value, thereby confirming the identity.

The biometric authentication is utilized in various fields such as bank automated teller machines (ATMs), entry/exit management, and the like, and particularly in recent years, it has begun to be utilized for cashless payment in supermarkets, convenience stores, and the like.

The biometric authentication includes 1:1 authentication that confirms matching with registered biometric information specified by an ID, a card, or the like, and 1:N authentication that searches multiple pieces of registered biometric information for matching registered biometric information. In stores or the like, 1:N authentication is often preferred from a viewpoint of convenience. However, the biometric information has variations depending on the acquisition condition and the like, which may increase the possibility of erroneous verification as the number of pieces of registered biometric information to be searched increases. In view of the above, operations such as executing 1:N authentication after narrowing down the search set with a simple personal identification number (PIN) code or the like to make it sufficiently small have been performed. How small it needs to be to reach a practical level depends on a method of the biometric authentication. However, PIN code input impairs the convenience even though it is simple, and thus a biometric authentication system that does not require an ID or a card has been desired.

In view of the above, a method of using multiple types of modalities to narrow down the search set with a first modality and identify the user with a second modality has been proposed. The modality indicates a type of biometric features, such as a fingerprint, vein, iris, face shape, palm shape, and the like. Therefore, the fingerprints and veins on the same finger are different modalities. Since it is inconvenient to input a plurality of modalities separately, a method of obtaining palm veins simultaneously with fingerprint input, a method of capturing a facial image at a time of palm vein input, and the like have been proposed.

According to a method of performing narrowing by face authentication and performing verification with palm veins, for example, an ID list of N people who are candidates for the face authentication is created, and 1:N authentication using palm veins is executed within the obtained set of the ID list to identify the user. Here, a plurality of faces may be imaged simultaneously depending on an installation condition of a camera for capturing a facial image and a usage condition of a user. For example, when faces of three people are obtained, the obtained ID list is for N×3 people, which increases the verification time for the palm vein authentication. If the initially set N is the performance limit of the 1:N authentication of the palm vein authentication, the risk of acceptance of another person increases. However, when attempting to narrow down the number of people to ⅓ by the face authentication, the processing time for the face authentication may increase, which may cause a missing candidate (correct person may not be included in the narrowing-down list) depending on the accuracy of the face authentication.

In view of the above, the following embodiment aims to provide an information processing apparatus, an authentication method, and an authentication program capable of shortening an authentication time.

First Embodiment

FIG.1Ais a block diagram exemplifying an overall configuration of an information processing apparatus100. As exemplified inFIG.1A, the information processing apparatus100functions as a storage unit10, a face detection unit20, a face selection unit30, a face authentication unit40, a vein acquisition unit50, a vein authentication unit60, an authentication result output unit70, and the like.

FIG.1Bis a block diagram exemplifying a hardware configuration of the information processing apparatus100. As exemplified inFIG.1B, the information processing apparatus100includes a CPU101, a RAM102, a storage device103, an interface104, a display device105, an input device106, a face imaging camera107, a venous sensor108, and the like.

The central processing unit (CPU)101is a central processing unit. The CPU101includes one or more cores. The random access memory (RAM)102is a volatile memory that temporarily stores a program to be executed by the CPU101, data to be processed by the CPU101, and the like. The storage device103is a nonvolatile storage device. For example, a read only memory (ROM), a solid state drive (SSD) such as a flash memory, a hard disk to be driven by a hard disk drive, or the like may be used as the storage device103. The storage device103stores an authentication program. The interface104is an interface device with an external device. For example, the interface104is an interface device with a local area network (LAN).

The display device105is a display device or the like such as a liquid crystal device (LCD). The input device106is an input device such as a keyboard, a mouse, or the like. The face imaging camera107is a metal oxide semiconductor (MOS) sensor, a charged coupled device (CCD) sensor, or the like. The venous sensor108includes a MOS sensor, a CCD sensor, and the like, and may also include near-infrared illuminator and the like.

With the CPU101executing the authentication program, the storage unit10, the face detection unit20, the face selection unit30, the face authentication unit40, the vein acquisition unit50, the vein authentication unit60, and the authentication result output unit70are implemented. Note that hardware such as a dedicated circuit may be used as the storage unit10, the face detection unit20, the face selection unit30, the face authentication unit40, the vein acquisition unit50, the vein authentication unit60, and the authentication result output unit70.

The storage unit10stores a plurality of types of biometric information of users registered in advance. Note that two different types of modalities are used as the plurality of types of biometric information in the present embodiment. In the present embodiment, as an example, facial features are stored as registered facial features in association with ID of each user, and venous features are further stored as registered venous features, as exemplified inFIG.2.

In the present embodiment, the display device105displays operation information related to authentication. For example, the display device105displays content instructing a user to hold the palm over the venous sensor108. When the user visually recognizes the operation information, the user inputs a palm image to the venous sensor108in accordance with the instruction. When the user inputs the palm image to the venous sensor108, the face imaging camera107obtains an image including the face of the user. The display device105is oriented to the visible range of the user. Therefore, the face position of the user viewing the information displayed on the display device105is determined within an approximate range. Facial images within the range of the obtained image are selected and face authentication is executed, thereby narrowing down candidates for a person in question. Thereafter, vein authentication is performed on the narrowed-down candidates, thereby authenticating the person in question. Hereinafter, details will be described.

The face imaging camera107is installed at a place where the face of the user may be captured when the user visually recognizes the display content of the display device105. For example, the face imaging camera107is installed above the display device105or the like.

FIGS.3A and3Bare diagrams exemplifying an installation location of the face imaging camera107.FIG.3Ais a front view.FIG.3Bis a top view. In the examples ofFIGS.3A and3B, the face imaging camera107is installed above the display device105. The venous sensor108is installed below the display device105or the like. The shooting angle of view of the face imaging camera107is set to include the visible range of the display device105for the user. In this case, as exemplified inFIG.3C, the visible range is included in the captured image obtained by the face imaging camera107.

A hood or an anti-peeping film may be used as a method of limiting the visible angle range. The anti-peeping film limits a light emission direction of a display screen by arranging fine louvers (louver boards). Here, the visible range is preferably a range that may be visually recognized by one user. Since the visible range is limited by an angle, the area of the visible range becomes larger as the distance from the face imaging camera107becomes longer, which allows multiple users to visually recognize it. However, the user comes within reach of the input device106and the venous sensor108to perform key operation, vein input, and the like. The area of the visible range at that distance is preferably set to be approximately the size of one user.

The storage unit10stores information regarding the visible range in the captured image obtained by the face imaging camera107.FIG.3Dis a diagram exemplifying the information regarding the visible range stored in the storage unit10. The visible range is a partial area within the captured image. For example, it is set to the range from Y1 (>0%) to Y2 (<100%) with respect to the vertical axis of the captured image and the range from X1 (>0%) to X2 (<100%) with respect to the horizontal axis of the captured image, or the like. For example, it is assumed that the bottommost position of the vertical axis of the captured image is 0%, the topmost position is 100%, the leftmost position of the horizontal axis is 0%, and the rightmost position is 100%. By referring to the information ofFIG.3D, it becomes possible to determine the facial image to be selected in the captured image.

The user is not necessarily positioned exactly within the visible range as illustrated inFIG.3C. The visual recognition is possible even from a position slightly shifted to the right or left as inFIG.3E, and in this case, only an image with a partially lacked face is obtained when only the visible range is imaged, which interferes with the face authentication process. In view of the above, it is also possible to avoid a lack of the facial image by imaging a range wider than the visible range and selecting the detected face largest (having largest area) in the visible range.

FIG.4is a flowchart illustrating an exemplary process of the information processing apparatus100. As exemplified inFIG.4, the vein acquisition unit50causes the display device105to display information associated with a vein input instruction (step S1). When the user visually recognizes the vein input instruction displayed on the screen of the display device105, the user holds the palm over the venous sensor108. Upon reception of a palm image from the venous sensor108, the vein acquisition unit50extracts venous features from the palm image as venous features for verification (step S2). The vein acquisition unit50sends the time at which the palm image is obtained to the face selection unit30(step S3).

The following steps S4and S5are executed in parallel with steps S1to S3. First, the face detection unit20obtains, from the face imaging camera107, captured images within a predetermined time range including the time received in step S3(step S4). This arrangement increases the accuracy in selecting the facial image of the user who holds the hand over the venous sensor108.

Next, the face detection unit20obtains the visible range stored in the storage unit10, thereby detecting a position of the facial image (step S5).

After executing steps S3and S5, the face selection unit30selects a target facial image from the captured image (step S6). For example, if there is one facial image included in the visible range, the face selection unit30selects the facial image as a target. If there is a plurality of facial images included in the visible range, the detected facial image largest (having largest area) in the visible range is selected as a target.

Next, the face authentication unit40performs face authentication using the facial image selected in step S6(step S7). First, the face authentication unit40extracts facial features from the facial image as facial features for verification. The facial features for verification used here are narrowing-down data with an emphasis on high-speed verification. The face authentication unit40collates the facial features for verification with the individual registered facial features, and obtains IDs associated with the registered facial features with a degree of similarity (narrowing-down score) to the facial features for verification equal to or higher than a threshold value. Through the process above, some of the IDs stored in the storage unit10may be narrowed down as a candidate list for the person in question.

Next, the vein authentication unit60collates the venous features for verification extracted in step S2with the registered venous features associated with the IDs in the list for the person in question obtained in step S7(step S8). When a degree of similarity (verification score) of one of the registered venous features to the venous features for verification is equal to or higher than a threshold value for determining the person in question, the authentication result output unit70outputs information associated with authentication success. When the verification score is less than the threshold value for determining the person in question, the authentication result output unit70outputs information associated with authentication failure. The information output from the authentication result output unit70is displayed on the display device105.

According to the present embodiment, a target facial image is selected from a plurality of facial images based on positions of the respective plurality of facial images included in the captured image captured by the face imaging camera107in the captured image. Accordingly, it becomes possible to specify the face to be used for the narrowing-down processing, and to shorten the narrowing-down time without lowering the accuracy in the face authentication. As a result, it becomes possible to shorten the authentication time. Furthermore, since the face verification is carried out by selecting only a person to be authenticated, it becomes possible to exclude faces other than the face of the user from the verification target, which is effective in terms of privacy protection.

In the present embodiment, the storage unit10is an example of a storage unit that stores pieces of biometric information associated with the respective plurality of facial images. The venous feature for verification is an example of biometric information detected by a sensor. The face selection unit30is an example of a selection unit that selects one facial image from a plurality of facial images based on positions of the respective plurality of facial images included in the captured image captured by a camera in the captured image, and the face authentication unit40is an example of a specifying unit that refers to the storage unit for storing pieces of the biometric information associated with the respective plurality of facial images and specifies a piece of the biometric information associated with the facial image with the degree of similarity to the selected facial image satisfying a criterion. The vein authentication unit60is an example of an authentication unit that performs, upon reception of the biometric information detected by the sensor, authentication based on verification of the specified biometric information against the received biometric information. Furthermore, the vein authentication unit60is an example of the authentication unit that executes an authentication process based on verification of registered biometric information against the biometric information detected by the sensor. Furthermore, the face selection unit30and the face authentication unit40are examples of a determination unit that determines, when a face image is included in the captured image captured by the camera, whether or not to use, as a target to be collated with the detected biometric information, a piece of the biometric information associated with the facial image with the degree of similarity to the facial image satisfying the criterion among the registered pieces of biometric information based on the position of the facial image in the captured image.

First Variation

It is difficult to arrange the display device105and the face imaging camera107coaxially (having the same central position and having the same orientation). Therefore, it is common to install the face imaging camera107at a location away from the screen center of the display device105. For example, in a case where it is installed on the upper side of the screen slightly shifted from the center in the horizontal direction as exemplified inFIG.5A, the area corresponding to the visible range is to be at a position shifted according to a distance as exemplified inFIG.5B. In view of the above, the distance from the face imaging camera107to the face may be detected, and the position of the determination area may be finely adjusted.

An area difference between the imaging range and the visible range is caused by a difference between the shooting angle of view and the visible angle. For example, as exemplified inFIG.5C, in a case where the shooting angle of view is set to 2×a and the visible angle is set to 2×β, the imaging range at a position separated by a distance d may be expressed by the following equation (1).

The width of the visible range may be expressed by the following equation (2).

Further, w1relative to w0may be expressed by the following equation (3).

From the above, the relative size of the area of the visible range within the imaging range is constant regardless of the distance. In other words, when the display device105and the face imaging camera107are coaxial, it may be considered that the visible range within the imaging range remains unchanged. Technically, since the size of the display area is superimposed as an offset, the area is larger as the distance is closer, and is smaller as the distance is farther. Even when the display device105and the face imaging camera107are not coaxial, the area size relationship is the same as in the coaxial case. When they are not coaxial, the position of the area is shifted depending on the distance. A shift amount is determined by a positional difference, an optical axis difference, and a distance between the display device105and the face imaging camera107. It may be difficult to precisely know an installation angle of something whose angle may be easily changed, such as a web camera, while it may be obtained by calculation when the difference in position and angle is known. In such a case, it may be checked by a method in which a projector screen or the like having high diffuse reflectivity is placed in front of the display screen to be observed by the face imaging camera107, and an area that looks brighter is obtained.

While a distance sensor using ultrasonic waves or light may be installed to detect the distance, it leads to an increase in device cost and restrictions on installation conditions. In view of the above, the distance may be determined informally based on the size of the face. For example, a reference size is held in advance, and the distance is shorter when the face is larger than the reference while the distance is longer when the face is smaller than the reference. The informal distance determination is sufficient due to the fact that the determination of the area does not depend on whether the face is strictly within the area and that, in visual field restriction using louvers, the area boundary is blurred as the amount of light gradually decreases near the boundary, not suddenly disappears at a certain angle.

Second Variation

The visual field restriction may be provided for, instead of the entire screen of the display device105, only a partial area for displaying operation information related to authentication. For example, angular limitation is set in such a manner that only a part of the screen that may be viewed from a wide range may be viewed from the front (or a specific direction). For example, as exemplified inFIG.6A, only a partial area105bmay be subject to the visual field restriction in a screen105aof the display device105. In the example ofFIG.6A, the area105bis viewed from the visible range. As inFIG.6A, the operation information is easy to see in the area105b.FIG.6Bis a diagram in which the area105bis viewed from outside the visible range. As inFIG.6B, it is difficult to see the operation information in the area105b.

Alternatively, in a case where only one person is detected in the imaging range, an instruction is displayed on the normal screen to execute the authentication. However, in a case where a plurality of faces is detected in the imaging range, the instruction display may be output in a range that may be viewed from a wide range first, and a face moved to a limited area in accordance with the instruction and detected may be used as a user. Moreover, as exemplified inFIG.7, partial visible ranges may be assigned in different directions. For example, in the case where only one person is detected in the imaging range, it may be displayed in the visible area in the direction that corresponds to the detected position. In the example ofFIG.7, a range A, a range B, and a range C are assigned in different directions. First, the display is output in the visible area that may be viewed from a wide range, the range A is set as the visible range, then the range B is set as the visible range, and then the range C is set as the visible range. With this arrangement, the user moves along with the switching of the visible range. The face that has moved may be used for the face authentication.

Second Embodiment

While the facial image to be used for the face authentication is selected according to the position in the image obtained by the face imaging camera107in the first embodiment, it is not limited to this. The facial image to be used for the face authentication may be selected according to the position of the operation information displayed on the display device105.

As exemplified inFIG.8, a vein acquisition unit50designates a position of operation information of a venous sensor108on a screen of a display device105, and causes the display device105to display it. In a case where the display device105has a large screen, a user is caused to move to a position where the user can view the information depending on the position of the information displayed on the screen of the display device105.

For example, in a case where a message “please hold your hand over the venous sensor” is displayed in an area a of the screen of the display device105, only a person A may see the message (person B may not see it), and thus only a facial image of the person A is to be authenticated. On the other hand, in a case where the message is displayed in an area β of the screen of the display device105, only the person B may see the message (person A may not see it), and thus only a face of the person B is to be authenticated.

A face selection unit30selects a facial image located in a positional range corresponding to a display position of the display device105in an image obtained by a face imaging camera107within a predetermined time range including time at which a venous image is extracted. After the facial image is selected, it is sufficient if a process similar to that in the first embodiment is performed.

In the present embodiment, the face selection unit30is an example of a selection unit that selects, when a plurality of facial images is included in a captured image captured by a camera, one facial image from the plurality of facial images based on a display position of operation information on a display unit that displays the operation information related to authentication. A vein authentication unit60is an example of an authentication unit that performs authentication using the facial image selected by the selection unit.

While the embodiments of the present invention have been described above in detail, the present invention is not limited to such specific embodiments, and various modifications and alterations may be made within the scope of the present invention described in the claims.