Long distance multimodal biometric system and method

A system for multimodal biometric identification has a first imaging system that detects one or more subjects in a first field of view, including a targeted subject having a first biometric characteristic and a second biometric characteristic; a second imaging system that captures a first image of the first biometric characteristic according to first photons, where the first biometric characteristic is positioned in a second field of view smaller than the first field of view, and the first image includes first data for biometric identification; a third imaging system that captures a second image of the second biometric characteristic according to second photons, where the second biometric characteristic is positioned in a third field of view which is smaller than the first and second fields of view, and the second image includes second data for biometric identification. At least one active illumination source emits the second photons.

BACKGROUND OF INVENTION

1. Field of the Invention

The present invention relates generally to systems and methods for biometric identification, and more particularly, to imaging systems and methods capable of biometric identification according to more than one modality, especially for subjects positioned at a distance from the image capture system.

2. Description of the Related Art

Due to the unique character of each individual's face or iris, various systems attempt to use either the face or the iris for biometric identification. As such, commercially available imaging systems used for biometric identification generally use a single biometric modality. In other words, these systems employ imaging systems that process images of the face or the iris, but not both. As a result, these single modal systems suffer from the limitations inherent in face-only imaging systems or iris-only imaging systems. As a further disadvantage, commercially available iris-only systems usually image one iris at a time, and not two eyes simultaneously, or near simultaneously. In addition, conventional face-only or iris-only imaging systems suffer from constraints that prevent these systems from acquiring and tracking a person among multiple persons within a specified field of view from a distance. For example, the greater the distance between the imaging system and the target, the more difficult it is to acquire images that may be used for biometric identification.

SUMMARY OF THE INVENTION

In view of the limitations of the single modal systems described previously, embodiments of the present invention provide a biometric system for capturing and combining biometric information from more than one modality. In particular, embodiments of the present invention may provide multimodal biometric systems that generate and process images from the face and the two irises of subjects. Biometrics based on a combination of data from both irises and the face, as provided by such embodiments, are more accurate and robust than using biometrics based on data from only a single iris or only the face. Furthermore, such embodiments exhibit lower fail-to-acquire (FTA) metrics than iris or face only systems and are less susceptible to spoofing.

In addition, embodiments of the present invention may provide multimodal systems that capture biometric data from subjects who are positioned at a distance from the system. For example, a multimodal biometric system may capture and process images of the face and both irises of subjects who are 50 meters away from the system. As such, the system solves the problem of capturing an image of both irises at a long distance. In particular, aspects of this system provide sufficient illumination of the iris, achieve adequate resolution with the captured iris image, and minimize the iris's exposure to any damaging illumination.

In one embodiment, a system for multimodal biometric identification includes a first imaging system that detects one or more subjects in a first field of view, where the one or more subjects includes a targeted subject having a first biometric characteristic and a second biometric characteristic. In addition, the system includes a second imaging system that captures a first image of the first biometric characteristic according to first photons reflecting from the first biometric characteristic, where the first biometric characteristic is positioned in a second field of view which is smaller than the first field of view, and the first image includes first data for biometric identification. Furthermore, the system includes a third imaging system that captures a second image of the second biometric characteristic according to second photons reflecting from the second biometric characteristic, where the second biometric characteristic is positioned in a third field of view which is smaller than the first and second fields of view, and the second image includes second data for biometric identification. At least one active illumination source emits the second photons to be reflected from the second biometric characteristic. A controller operates the first imaging system, the second imaging system, the third imaging system, and the at least one illumination source according to programmed instructions. The controller includes at least one or more independent sub-controllers and/or one or more interdependent sub-controllers. In particular embodiments, the first biometric characteristic may be a face and the second biometric characteristic may be at least one iris corresponding to an eye of the targeted subject.

In yet another embodiment, a method for multimodal biometric identification includes the steps of: identifying one or more subjects in a first field of view; selecting a targeted subject from the one or more subjects, where the targeted subject has a first biometric characteristic and a second biometric characteristic; aligning a second field of view to the first biometric characteristic, where the second field of view is smaller than the first field of view; aligning a third field of view to the second biometric characteristic, where the third field of view is smaller than the first field of view and the second field of view; actively illuminating with second photons the second biometric characteristic; capturing a first image of the first biometric characteristic according to first photons, where the first image includes first data for biometric identification; and capturing a second image of the second biometric characteristic according to the second photons, where the second image includes second data for biometric identification.

Embodiments of the present invention may employ subject tracking, face tracking and recognition, iris tracking from facial tracking and recognition, iris image capture, high speed iris image processing, optimal optics and illumination design, as well as compliance with applicable safety and technology standards.

These and other aspects of the present invention will become more apparent from the following detailed description of the preferred embodiments of the present invention when viewed in conjunction with the accompanying drawings.

DETAILED DESCRIPTION

Referring toFIG. 1, an embodiment of a biometric system100employing more than one modality is illustrated. In particular, the multimodal biometric system100includes three imaging systems. The first imaging system is a scene imaging system120for identifying one or more subjects for biometric identification from a distance. The second imaging system is a face imaging system140for capturing images of the face12of a target subject10from a distance. The third imaging system is an iris imaging system160for capturing images of each iris14of the target subject10from a distance. In some embodiments, the imaging systems120,140, and160as well as other components may be housed in a single image capture device, but the components of the biometric system100may house the components in any number of combinations and any number of devices.

The scene imaging system120may include one or more cameras that capture images based on photons with visible, near-infrared (NIR), or infrared (IR) wavelengths. The visible wavelengths detected may be in a range of approximately 400 nm to 700 nm; the NIR wavelengths detected may be in a range of approximately 700 nm to 2 μm; and the IR wavelengths detected may be in a range of approximately 2 μm to 13 μm. In some embodiments, the scene imaging system120captures images through passive imaging. Passive imaging refers to the detection of photons that are initially emitted from a source external to the biometric system100, also referred to as ambient photon generation. In certain indoor or outdoor scenarios, passive imaging by the scene imaging system120may detect photons with visible, NIR, and/or IR wavelengths. For example, the biometric system100may be used to check subjects attending a large sporting event or similar public gathering, where the ambient lighting at the venue generates a sufficient level of photons with visible wavelengths for detection by the scene imaging system120. In other embodiments, however, the scene imaging system120may detect photons that are provided by an illumination source (not shown) controlled by the biometric system100, i.e., active illumination.

The face imaging system140may include a camera that captures images of the face based on photons with visible, NIR, or IR wavelengths. The visible wavelengths detected may be in a range of approximately 400 nm to 700 nm; the NIR wavelengths detected may be in a range of approximately 700 nm to 2 μm; and the IR wavelengths detected may be in a range of approximately 2 μm to 13 μm. In some embodiments, the face imaging system140may employ passive imaging to detect photons with visible, NIR, or IR wavelengths. In other embodiments, the face imaging system140may detect photons that are provided by an illumination source controlled by the biometric system100, i.e., active illumination.

The iris imaging system160may include a camera that captures iris images based on photons with visible or NIR wavelengths. Photons with visible or NIR wavelengths may be used for iris recognition if the iris sensor is sufficiently large and an adequately high resolution is employed. The visible wavelengths detected may have a range of approximately 400 nm to 700 nm. The NIR wavelengths detected may be in a range of approximately 700 nm to 2 μm, or preferably, a range of 700 nm to 900 nm corresponding to the wavelength requirements for the ANSI specification for Iris Image Interchange Format (ANSI INCITS 379-2004). The preferable range may generally be determined according to the existing Iris Image Interchange Format standard.

The iris sensor of the iris imaging system160may have a significantly higher magnification than the face sensor of the face imaging system140. In some embodiments, commercially available sensors may be employed, where the sensors, for example, employ 752×480 pixels for each eye image, have a resolution in the range of approximately 16 to 21 pixels/mm, and have a quantum efficiency of approximately 25 to 30 percent at 850 nm illumination.

In some embodiments, the optical design of the iris imaging system160may employ a zooming telescope lens having an aperture of 100 mm for 3 m to 6 m. For other embodiments in which very long distances are involved, telescopes having an aperture of approximately 50 cm to 100 cm for 50 m may be employed. In particular, the telescope may have a Ritchey-Chrétien design, i.e. a hyperbolic Cassegrain telescope with a very flat field. In addition, the resolution may be 2 lp/mm to 4 lp/mm, thereby complying with ANSI specifications (ANSI INCITS 379-2004). Meanwhile, the opto-mechanical requirements may be met with commercially available ultra-high precision axis encoders (resolutions <0.002 arc-sec).

To illustrate the ability of embodiments to resolve features at a distance,FIG. 3illustrates the range of clear apertures required to resolve 0.25 mm with 850 nm illumination at varying object distances. In addition,FIG. 4illustrates the range of clear apertures versus minimum resolvable feature size, particularly imaging with 850 nm illumination at 50 meters with 5 μm pixel size.

One or more illumination systems, such as the illumination system180inFIG. 1, may be employed for active illumination. In particular, the illumination system180may emit photons with NIR wavelengths which are reflected from the irises14of the subject10and subsequently detected by iris imaging system160. As discussed previously, such illumination systems may also be used for active imaging by the face imaging system140. As illustrated inFIG. 1, the illumination system180may employ an NIR laser source182. Filters or coated optics may be employed in the optical train to select specific wavelengths, but still allow a visible color image. In a particular embodiment, for example, the illumination system180may have a wavelength of approximately 850 nm and a collimated beam with a spot size diameter of approximately 30 cm full width half-maximum (FWHM). In this embodiment, the laser illumination may be provided with an average power of approximately 1 W and a continuous wave with an irradiance of 2 mW/cm2 or less. Accordingly, referring toFIG. 1, an optical fiber184is coupled to the laser182. The optical fiber184is positioned to direct photons to a rotating diffuser185rotating at approximately 1000 revolutions per minute. Rotation of the diffuser185helps reduce speckle of the illumination. A notch filter186may also be placed in line to minimize any visible illumination from the illumination system180that may alert people to the presence of the biometric system100, especially when the biometric system100is intended to be used covertly, i.e. without the subjects' knowledge. As shown inFIG. 1, the illumination expands from the optical fiber184and is directed to a mirror187. The illumination is reflected to a Fresnel lens188where it is collimated. The beam may be collimated or slightly diverging.

Alternatively, rather than providing continuous wave illumination as described previously, the laser may be pulsed at 50 nsec with a 10 kHz duty cycle. Advantageously, employing a quasi-CW laser reduces laser speckle.

AsFIG. 1also shows, an optical laser rangefinder189with a wavelength, for example, of904nm may be used to detect the distance along the Z-axis from the rangefmder189to the targeted subject10. This Z-distance is utilized to improve the accuracy of estimates of the (X, Y, Z) position of the targeted subject10as well as to determine the range of focus and zoom, for example, by the iris camera lens of the iris imaging system160. The Z-distance provides a starting value for a search for an image with the highest focus measure. The Z-distance information may also be utilized to predict the anticipated movement of the subject.

As further illustrated inFIG. 1, the illumination source180may be integrated into a pan-tilt unit (PTU)195. The PTU195may be controlled to direct photons to specific biometric features which are then captured by the iris imaging system140and possibly the face imaging system160. Accordingly, in one embodiment, the illumination system180may be operated in burst mode triggered in coordination with the respective imaging system140or160. The illumination source180may also be employed for range finding to achieve auto focusing by the respective imaging system140or160, as described previously.

In further embodiments, the PTU195may be used to target and track subjects. As shown inFIG. 1, the illumination system180, the camera of face imaging system140, the camera of the iris imaging system160, and the rangefinder189may all be mounted on the single PTU195. The camera of the face imaging system140, the camera of the iris image system160, and the rangefinder189are oriented on the PTU195so that they receive photons from the illumination source180which are reflected from the targeted subject10. Therefore, the PTU195may be controlled to steer the mounted systems to direct photons from the illumination system180and to permit the co-aligned imaging systems140or160to capture the photons reflected from the respective biometric feature, i.e., face or iris.

In some embodiments, one or more beam steering systems (not shown), as are known, may additionally or alternatively be employed to direct the photons which are detected by the imaging systems120,140, and160for image capture. The beam steering systems may include galvanometric mirrors and/or imaging optics positioned on a gimbal mount. The beam steering systems may direct photons from the illumination source180to a biometric feature of the targeted subject10. Additionally or alternatively, the beam steering systems may direct photons reflected from the biometric feature to the appropriate imaging system.

Embodiments of the present invention meet the safety criteria of Class I ANSI Z136. In general, the maximum permissible exposure (MPE) for continuous wave exposure at 850 nm is approximately 2 mW/cm2. As such, the illumination source180in some embodiments may provide illumination with a wavelength of 850 nm for up to 30,000 seconds. On the other hand, the maximum permissible exposure (MPE) for repetitive pulse exposure at 850 nm is approximately 0.56 mW/cm2. Thus, the illumination source180in other embodiments may provide illumination with a wavelength of 850 nm in a 10 second pulse train with 50 nsec pulses at 10 KHz. Other considerations for laser safety include the operational environment, the use of additional optical devices, such as glasses and binoculars, by targeted subjects, as well as the presence of specular surfaces.

As illustrated inFIG. 1, the imaging systems120,140, and160each provide different fields of view. In particular,FIG. 1shows that the scene imaging system120has the larger field of view102. The field of view102, for example, may be several square meters, depending on the distance of subjects from the scene imaging system120. Any number of subjects for biometric identification may pass through and/or be positioned within the field of view102. As illustrated, the heads6of subjects5are positioned within the field of view102, while the heads3of subjects2are positioned outside the field of view102. Additionally,FIG. 1shows a smaller field of view104that corresponds with the face imaging system140. For example, the area of the field of view104may be approximately 0.15 m2. The head12of a single targeted subject10is isolated within the field of view102for further biometric analysis. Moreover,FIG. 1shows yet a smaller field of view106that corresponds with the iris imaging system160. The field of view106defines an area that isolates the irises14of the targeted subject10for biometric identification.

To obtain a full 360-degree field of view for the scene imaging system120, the scene imaging system120may employ a plurality of scene cameras. The cameras may be arranged so that the field of view102for each camera overlaps, abuts, or nearly abuts other fields of view102, whereby a series of fields of view102forms a continuous or nearly continuous a larger 360-degree field of view.

Alternatively, some embodiments may employ imaging systems which are all co-aligned using beam steering mirrors. As is known with other security monitoring systems, the use of a beam steering mirrors may be employed to enable the imaging systems to rotate through 360 degrees for observation.

Accordingly, some embodiments can identify multiple people within a 360 degree panoramic view. Employing such a system may require capturing images in rapid succession from a plurality of subjects who are moving within the panoramic view. Known techniques exist for stitching several detectors together to allow for rapid reading of the image to allow for increased frame rates. Moreover, aspects of these embodiments minimize occlusion of the subject's face and/or irises, minimize the time required to process the captured images, and overcome the constraints associated with the mechanical operation of the system.

Referring toFIG. 1, the system100may employ at least one controller190to control the operation of the imaging systems120,140, and160. The controller190may also be employed to process the image data captured by the imaging systems120,140, and160. Furthermore, the controller190may control the operation of the beam steering system195, the laser range finder189, and the illumination source180as a part of a target acquisition system. As is known, the controller190may include one or more programmable processing devices that execute software, or stored instructions. For example, the controller190may employ an external conventional computer networked with the image systems120,140, and160, as shown inFIG. 1. Alternatively, a field programmable gate array (FPGA) or digital signal processor (DSP) may be employed on board a single image capture device that houses the imaging systems120140, and160. Combinations of single and/or multiple programmable devices, including computers, FPGAs, and DSPs may be employed with various communications topologies and physical configurations to achieve scalable speed and/or performance targets.

Thus, with the face imaging system140and the iris imaging system160, the multimodal biometric system100generates images of the face and two irises for biometric identification. The controller190may operate the face imaging system140to capture an image of the subject's face12and the iris imaging system160to capture images of each iris14from the subject's right and left eyes all simultaneously, or near simultaneously.

Biometrics based on a combination of facial and iris data, as provided by the system ofFIG. 1, are more accurate and robust than using biometrics that include data from only a single iris or only the face. Furthermore, such embodiments exhibit lower fail-to-acquire (FTA) metrics than iris or face only systems and are less susceptible to spoofing. Advantageously, the iris and face present biometric features that are both independent and coupled. They are independent in that they are extracted from different biological structures. On the other hand, the iris and face biometric features are strongly coupled because there is a fixed geometric relationship between the iris and the face. Specifically, the position and orientation of an eye is reflected simultaneously in both the iris and the face. The coupling between the biometric features of the iris and the face not only facilitates the simultaneous capture of these biometric features, but allows these features to be cross-referenced or combined in a common feature space that preserves the geometric relationship between the iris and face. In addition, the use of an iris system complements the use of face system.

Referring toFIG. 2, an exemplary process200for operating the multimodal biometric system100is illustrated. With the scene imaging system120, the system100, in step202, identifies one or more objects, corresponding to the heads6of subjects5in the field of view102. In step204, the system100continuously tracks all objects, which may move within the field of view102. In step206, the system100aligns the center of the field of view104to one of the objects. As shown inFIG. 1, the object in the field of view104corresponds to the head6of the targeted subject10. In step208, the face imaging system140, which is co-aligned with the scene imaging system120, captures an image from the field of view104at a sufficiently high magnification and resolution to permit face identification. In step210, the system100identifies the eyes of the face12and takes over control of tracking. In step212, the system100adjusts the targeting system to align the face imaging system140as well as the NIR illumination source180with an area substantially centered on the eyes. In step214, the rangefmder189measures the Z distance of the object and zooms and focuses the iris imaging system160. In step216, the iris imaging system160captures images of the irises14. In step218, the iris data is segmented from the images. In step220, feedback from the iris segmentation may be used to fine tune eye centering if required. In step222, the segmented iris data is encoded and matched for enrollment or authentication. Authentication may include identifying a subject or verifying a subject's identity. As discussed previously, the iris imaging system160may have one or more iris sensors with higher magnification for capturing images of the irises14. As such, the center of the iris sensors are aligned with the eyes and the iris sensors capture the images of the irises14. As provided by step204, throughout the process200, the system100continues to track the objects it has identified in the field of view102, including the objects which have already been targeted for biometric identification and processed according to steps206through222. In this way, the system100is able to determine which objects still need to be targeted for biometric processing.

Information captured by the face imaging system140and the iris imaging system160is used to establish facial pattern recognition, iris pattern recognition, as well as biometric fusion. To achieve biometric identification, the information from the imaging systems may be used to determine a host of attributes including, but not limited to, positioning of the face or the irises, tracking of the face or irises, measurements of focus provided in the images, and interpupillary distance.

For example, the software executed by the controller190for capturing and processing images of the face12and irises14may determine characteristics such as linear (X,Y,Z) position of the head, head pose angle, and eye-gaze angle. Head pose angle indicates pitch, yaw, and roll, where pitch refers to up-and-down rotation of the head, yaw refers to side-to-side rotation of the head, and roll refers to rotation the head along a direction from ear to shoulder. Meanwhile, eye-gaze angle refers to the up-and-down or side-to-side viewing angle of the eyes.

To minimize the effect of environmental factors, such as heat from hot surfaces which can distort captured images, some embodiments may employ a Hartmann-Shack sensor to correct for these environmental factors.

Once the positioning of the biometric features is determined and images are captured by the facial/iris imaging system, the software executed by the controller190also detects and processes images of the face12and irises14in the captured data. For instance, as shown in step218ofFIG. 2, the images of the irises are segmented from the captured data, and before the segmented iris images are passed on for further biometric processing or matching, the segmented iris images may be tested according to a variety of criteria measuring the quality of an image. Such algorithms for processing iris images as well as other relevant algorithms are provided in a U.S. patent application titled MULTIMODAL OCULAR BIOMETRIC SYSTEM AND METHODS, filed on Sep. 10, 2007, which claims priority to U.S. Provisional Application No. 60/844,659 filed Sep. 15, 2006, the contents of all these applications being incorporated entirely herein by reference.

Once the iris image data is segmented and tested according to step218, the iris image data may be employed for biometric matching with databases of existing iris data or may be recorded for biometric enrollment, as shown in step222. When iris data is collected from multiple subjects in rapid succession, the enrollment may be anonymous, i.e. recorded without further identification data, such as a name.

In general, embodiments of the present invention may employ various configurations of imaging systems that capture iris images and face images. Although many of the features of embodiments of the present invention may be described with respect to the configuration shown inFIG. 1, it is understood that other configurations can implement these features in order to combine iris and face images for biometric identification.

While the embodiments described previously may employ NIR laser illumination for the facial/iris imaging system, other embodiments of the present invention may employ LEDs or flash lamps rather than laser diodes. As such, in these alternative embodiments, the system can perform facial and iris liveness testing. Facial liveness testing detects whether the biometric information comes from a living source. (U.S. patent application Ser. No. 11/258,749, filed on Oct. 26, 2005, describes a METHOD AND SYSTEM FOR DETECTING BIOMETRIC LIVENESS, and is entirely incorporated herein by reference.)

Moreover, while the embodiments described previously may direct NIR laser illumination over a long distance to the face12or the irises14, other embodiments may employ use of LED's positioned more closely to the targeted subject. For example, such LED's may be employed to illuminate a subject as the subject is guided down a specific corridor of known length and width. In other words, if the subject is guided near a known position, an illumination source may be set up near the known position so that photons for image capture do not have to be transmitted over longer distances.

Embodiments of the present invention may be fully automatic or may require some operator input, especially with regard to initial targeting of subjects. In other words, an operator selectively targets subjects for biometric analysis. Advantageously, the operator can ensure that the illumination sources are not directed at subjects who may susceptible to eye damage from photons emitted by the illumination sources. For example, embodiments of the present invention may be employed to identify and screen subjects at an event, such as a highly attended sporting event. At such events, spectators often use optical aids, such as binoculars, to view the game or match. Eye damage might result if laser illumination is conducted to the eyes of a targeted individual through such an optical aid. As a result, an operator-assisted mode can prevent the laser illumination from being directed at subjects using an optical aid.

As described above, the controller190may be a programmable processing device, such as an external conventional computer or an on-board field programmable gate array (FPGA) or digital signal processor (DSP), that executes software, or stored instructions. In general, physical processors and/or machines employed by embodiments of the present invention for any processing or evaluation may include one or more networked or non-networked general purpose computer systems, microprocessors, field programmable gate arrays (FPGA's), digital signal processors (DSP's), micro-controllers, and the like, programmed according to the teachings of the exemplary embodiments of the present invention, as is appreciated by those skilled in the computer and software arts. The physical processors and/or machines may be externally networked with the image capture device, or may be integrated to reside within the image capture device. Appropriate software can be readily prepared by programmers of ordinary skill based on the teachings of the exemplary embodiments, as is appreciated by those skilled in the software art. In addition, the devices and subsystems of the exemplary embodiments can be implemented by the preparation of application-specific integrated circuits or by interconnecting an appropriate network of conventional component circuits, as is appreciated by those skilled in the electrical art(s). Thus, the exemplary embodiments are not limited to any specific combination of hardware circuitry and/or software.

Stored on any one or on a combination of computer readable media, the exemplary embodiments of the present invention may include software for controlling the devices and subsystems of the exemplary embodiments, for driving the devices and subsystems of the exemplary embodiments, for enabling the devices and subsystems of the exemplary embodiments to interact with a human user, and the like. Such software can include, but is not limited to, device drivers, firmware, operating systems, development tools, applications software, and the like. Such computer readable media further can include the computer program product of an embodiment of the present inventions for performing all or a portion (if processing is distributed) of the processing performed in implementing the inventions. Computer code devices of the exemplary embodiments of the present inventions can include any suitable interpretable or executable code mechanism, including but not limited to scripts, interpretable programs, dynamic link libraries (DLLs), Java classes and applets, complete executable programs, and the like. Moreover, parts of the processing of the exemplary embodiment of the present inventions can be distributed for better performance, reliability, cost, and the like.

Common forms of computer-readable media may include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, any other suitable magnetic medium, a CD-ROM, CDRW, DVD, any other suitable optical medium, punch cards, paper tape, optical mark sheets, any other suitable physical medium with patterns of holes or other optically recognizable indicia, a RAM, a PROM, an EPROM, a FLASH-EPROM, any other suitable memory chip or cartridge, a carrier wave or any other suitable medium from which a computer can read.

While the present invention has been described in connection with a number of exemplary embodiments, and implementations, the present inventions are not so limited, but rather cover various modifications, and equivalent arrangements.