Facial feature evaluation based on eye location

One embodiment of the present invention includes interrogating a location along a face of a person with multiple wavelengths of electromagnetic radiation. Signals are established corresponding to detection of the wavelengths reflected from the location. A determination is made as to whether a disguising material covers at least a part of the face based on a difference in range to the location indicated by these signals. Alternatively, the signals may correspond to reflections by different portions of an eye of the person and a determination is made regarding the location of one or more eyes of the person based on the signals. In one particular nonlimiting form, a multispectral, three-dimensional signature of facial features is registered to eye location that may include the iris, nose, chin, mouth, check or the like for facial recognition/identification.

BACKGROUND

The present invention relates to facial feature evaluation, and more particularly, but not exclusively is directed to multispectral facial interrogation techniques for locating the eyes and/or related facial features.

Viable facial recognition techniques continue to be of interest in many applications; including but not limited to, security screening, transaction authorization, access control, and the like. Unfortunately, many existing systems suffer from high rates of misidentification, excessive complexity, large conspicuous device size, and/or slow processing times. Accordingly, there is an ongoing demand for further contributions in this area of technology.

SUMMARY

One embodiment of the present invention is a unique facial feature recognition technique. Other embodiments include unique systems, devices, methods, and apparatus to interrogate, recognize, and/or locate facial features. Further embodiments, forms, features, advantages, aspects, and benefits of the present invention shall become apparent from the detailed description and figures provided herewith.

DETAILED DESCRIPTION

FIG. 1depicts facial evaluation system20of one embodiment of the present invention. System20is configured to scan face F to evaluate selected features. In one form, this evaluation is performed for a facial recognition application. System20includes facial scanning equipment40operatively coupled to processing subsystem24. Also coupled to processing subsystem24are one or more operator input (I/P) devices26, one or more operator output (O/P) devices28, and computer network30. Processing subsystem24includes pre-processor32and data processor34. Also included in processing subsystem24is memory36. Memory36includes Removable Memory Device (RMD)38. Facial scanning equipment40includes color imager42, millimeter wave subsystem50, and laser subsystem70.

Operator input devices26can include a keyboard, mouse, or other pointing device; a voice recognition input arrangement; and/or a different arrangement as would occur to those skilled in the art. Operator output devices28can include a display, a printer, a speaker system, and/or a different arrangement as would occur to those skilled in the art. Computer network30can be provided in the form of a Local Area Network (LAN), a Municipal Area Network (MAN), and/or a Wide Area Network (WAN) of either a private type, a publicly accessible type, such as the internet; or a combination of these.

Pre-processor32and processor34can each be comprised of one or more components configured as a single unit, or as a number of separate units. When of multicomponent form, either may have one or more components remotely located relative to the others, or otherwise have its components distributed throughout system20. Pre-processor32and processor34can each be of a general purpose integrated circuit type, a semicustom type, a fully customized type, or such different type as would occur to those skilled in the art. In one form, pre-processor32is based on a Field Programmable Gate Array (FPGA) that is configured with operating logic to directly process input signals from equipment40and provide corresponding output-signals. In one particular FPGA embodiment of pre-processor32, the input signals are provided to the FPGA at an output frequency and the output signals are provided as multiple frames of data with an output frequency less than the input frequency. This data is typically useful to recognize scanned face F, providing a form of “signature” information specific an individual's face. By way of nonlimiting example, the ratio of input frequency to output frequency for this embodiment is about ten to one (10:1). Additionally or alternatively, processor34can be of general purpose type that is programmed with software instructions to process digital data received from pre-processor32. Processor34can be provided with only a single Central Processing Unit (CPU); or alternatively multiple CPUs arranged to operate independent of one another, and/or in a parallel, pipelined, or different processing arrangement as would occur to one skilled in the art. One or more components of pre-processor32and/or processor34may be of an electronic variety defining digital circuitry, analog circuitry, or a combination of both. As an addition or alternative to electronic circuitry, pre-processor32and/or processor34may include one or more other types of components or control elements. Either or both of pre-processor32and processor34may be programmable, a state logic machine or other operationally dedicated hardware, or a hybrid combination thereof.

In one embodiment including electronic circuitry, processor34includes one or more integrated digital processing units operatively coupled to one or more solid-state memory devices defining, at least in part, memory36. For this embodiment, memory36provides storage for programming instructions executable by the one or more processing units and/or can be arranged for reading/writing of data in accordance with one or more program routines.

It should be appreciated that pre-processor32and processor34each operate in accordance with logic arranged to perform various routines, operations, conditionals, and the like—including those described in connection hereinafter. This operating logic can be in the form of software programming instructions, firmware, a programmable gate, array, application specific circuitry, and/or other hard-wired logic/circuitry, just to name a few examples. Furthermore, such logic can be in the form of one or more signals carried with/encoded on memory36, or more specifically RMD38of memory36, and/or one or more parts of computer network30. In one example, logic signals to perform one or more operations (such as programming) are transmitted to/from pre-processor32and/or processor34via network30. Alternatively or additionally, programming can be transported or disseminated through RMD38or one or more other portable storage devices.

Memory36may include one or more types of solid-state semiconductor electronic devices and additionally or alternatively may include a magnetic or optical memory variety. For example, memory36may include solid-state electronic Random Access Memory (RAM), Sequentially Accessible Memory (SAM), Programmable Read Only Memory (PROM), Electrically Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), or flash memory; an optical disk memory (such as a CD or DVD); a magnetically encoded hard disk, floppy disk, tape, or cartridge media; or a combination of any of these types. Further, memory36may be volatile, nonvolatile, or a hybrid combination of these. RMD38may be of an optically encoded device (such as a disk) or magnetically encoded disk, tape, or cartridge type; a semiconductor-based card or “stick”; and/or such different form of portable/removable memory as would occur to those skilled in the art.

Besides the depicted devices, processing subsystem24may also include any control clocks, power supplies, interfaces, signal conditioners, filters, limiters, Analog-to-Digital Converters (ADCs), Digital-to-Analog Converters (DACs), wireless communication ports/interfaces, wire, fiber, or cable-connected communication ports/interfaces, or other types of circuits, devices, operators, elements, or components as would occur to those skilled in the art to implement the present invention.

Referring toFIG. 2, further details are depicted regarding one embodiment of laser subsystem70. Laser subsystem70includes multispectral source72, detector82, and Radio Frequency (RF) signal processing circuitry92. Source72includes lasers74aand74b(alternatively designated laser A and laser B). Detector82includes sensors84aand84b(alternatively designated sensor A and sensor B). RF signal processing circuitry92is operatively coupled to source72. Circuitry92controls certain operations of source72by providing corresponding control signals. Detector82is also operatively coupled to RF signal processing circuitry92. Detector82provides corresponding sensor signals to circuitry92for processing in relation to the control signals sent to source72. For example, circuitry92includes modulation circuitry94to provide modulating signals to source72. Circuitry92also includes RF signal information recovery circuitry96. Circuitry96processes signals received from detector82to recover interrogation information and put it in a form more suitable for subsequent processing by subsystem24. Circuitry96includes filter circuitry98ato reduce or remove one or more undesired frequencies/harmonics, and Analog-to-Digital (A/D) Converter (ADC)98bto convert recovered interrogation information to a digital format. Corresponding digital output signals are provided from ADC98bto processing subsystem24. It should be appreciated that in other embodiments subsystem24can alternatively or additionally be configured to include at least some of circuitry92, such as filter circuitry98a, ADC98b, and the like.

FIG. 3illustrates further details of one form of subsystem70. InFIG. 3, only laser74aof source72, sensor84aof detector88, and channel subcircuitry90of circuitry92are illustrated to preserve clarity—it being understood that laser74band sensor84bare configured in a like manner, with corresponding subcircuitry belonging to circuitry92(not shown). Laser74aincludes laser diode76connected to control circuitry78. Sensor84ais connected to amplifier88, which in one form could be utilized to dynamically adjust amplification in an Automatic Gain Control (AGC) arrangement. Also shown inFIG. 3, RF processing circuitry92further includes oscillator94aand modulation source95that are combined with mixer94c. Mixer94cprovides a sinusoidal modulation signal to control circuitry78of laser74a. Phase shifter94b, which is also included in subcircuitry90, provides a 90 degree phase-shifted form of the signal from oscillator94a. Oscillator94a, phase shifter94b, mixer94c, and source95are collectively included in modulation circuitry94ofFIG. 2, but are not shown specifically therein to preserve clarity.

Channel subcircuitry90also includes frequency mixer96, which is part of RF signal information recovery circuitry94. Circuitry96includes mixers96aand96bto output corresponding in-phase (I) and quadrature (Q) signals of a standard type. Also included in circuitry94is converter97. Converter97converts I and Q inputs from mixer circuitry96to provide corresponding amplitude and gain outputs. This amplitude and gain information is provided as “channel A” corresponding to laser74aand sensor84a. The amplitude and phase signals of channel A are filtered by circuitry98a, converted to a digital form by A/D converter98b, and provided to processing subsystem24, as shown inFIG. 2. It should be understood that circuitry92includes circuitry like subcircuitry90to interface with laser74band sensor84b, and provide corresponding “Channel B” amplitude and phase outputs.

In one form, circuitry78of laser A and B includes a laser current driver and a bias-T for inserting Amplitude Modulation (AM). Also included are standard collimation optics, connectors, and the like. For this form, detector82includes collection optics, a high-speed photodetector form of sensor A and B, a pre-amplifier, an automatic gain correction (AGC) amplifier, and frequency mixers96aand96bin the form of an integrated RF circuit component provided by ANALOG DEVICES as model number AD8302. Pre-processor32provides range image data by processing both the intensity and phase shift of the reflected beam (with respect to the transmitted beams of different wavelengths). The quadrature phase channel B provides a signal that can be used to accommodate target T positioning at the 2π radians range ambiguity interval. Naturally, in other embodiments, different circuitry/components can additionally or alternatively be utilized. A more detailed description of subsystem70and corresponding signal information processing follows.

Referring generally toFIGS. 2 and 3, subsystem70determines range to target T (such as an individual) in accordance with an observed phase change of modulated electromagnetic radiation of a given wavelength (or wavelength range) that is observed by comparing a modulated incident beam IB emanating from source72to the beam reflected by target T in response (as sensed with detector82). This reflected beam is symbolically designated by the reference label RB and the reflecting surface is symbolically designated by the reference label RS.

The phase comparison laser measurement technique of subsystem70utilizes amplitude modulated (AM) continuous wave (CW) laser diode transmitters (lasers74a&74b). In this AM/CW laser scheme, the laser beam amplitude is modulated between zero and a maximum intensity at a specific frequency provided by oscillator94a. Both intensity and phase shifting of the reflected beam RB (with respect to the incident beam IB from source72) are simultaneously detected with detector82. Comparing the transmitted and received signal phase provides a high resolution target range measurement. The resolution D of this measurement is determined by the phase angle between the transmitted modulated waveform (IB) and the received modulated waveform (RB). The relationship between phase angle Δφr(radians), time delay tr, speed of light c, and modulation frequency fo, is given by the following equations (1) and (2):
tr=Δφr/2πfo(1)
D=ctr/2=cΔφr/4πfo(2)
Selecting a desired distance resolution of 2.0 mm and a phase resolution of Δφr=5°, the desired AM modulation frequency is given by rearranging equation (2) to the equation (3) form that follows:
fo=cΔφr/4πD(3)

Based on equation (3), the corresponding working frequency becomes approximately fo=1.0 GHz. This modulation rate sets an ambiguity interval of the phase-loaded measurement that results for phase turns of 360° or more (2π radians). The total range corresponding to a complete phase turn (0-360°) at this frequency is 0.300 m (Drange=c/(2 fo)). This ambiguity interval provides sufficient depth to fully image face F at target T (seeFIG. 1). To address applications where such ambiguity cannot otherwise be accommodated, the AM/CW transmitter is switched to a Time-Of-Flight (TOF) measurement to measure the absolute distance to target T. This TOF information can be used to scale the laser range data, and any resulting images, to other detected/sensed information to be described hereinafter.

Stated differently, the phase-based measurement provides a degree of “fine” resolution, while the TOF measurement provides a degree of “coarse” resolution. This coarse TOF measurement effectively delimits the range that is refined with the phase-based measurement. In one particular form, TOF measurement can be made with a burst of coherent encoded energy from the laser as described in U.S. Pat. No. 5,745,437; however, in other embodiments different approaches for the “coarse” measurement can be alternatively or additionally used. It has been found that for certain applications the phase-based comparison approach provides acceptable performance in terms of cost and complexity for ranges of 30 meters or less—particularly in the area of facial recognition as will be further explained hereinafter. Naturally, in other embodiments different techniques may be alternatively or additionally employed, such as Time-Of-Flight (TOF), triangulation, and/or interferometry, to name a few.

To evaluate various facial features, determining eye location is often of interest. Facial scanning equipment40is arranged to locate one or more eyes based on the wavelength-selective retroreflection of light from the retina R of the human eye. Called “cat's eye” reflection, this phenomenon produces the red-eye effect sometimes seen in photographs. Accordingly, eye E is alternatively designated retroreflector22. By selecting different wavelengths for laser A and laser B, different ranges for retina R and a part of the eye closer to source72(such as the outer surface OS of eyeball E inFIG. 2) can be observed. A corresponding range difference relatively unique to eye E for the selected wavelengths can be used to determine where an eyeball is located relative to a timed scan of a typical human face.FIG. 2further illustrates the approach, symbolically showing that the wavelength-selective light from laser74ais reflected by retina R of eyeball E, while wavelength-selective light from laser74bis reflected by outer surface OS of eyeball E. Sensor84ais arranged to selectively detect the laser74awavelength reflected by retina R, and sensor84bis arranged to selectively detect the laser74bwavelength reflected by outer surface OS. Pupil P and Iris I of eyeball E are also illustrated isFIG. 2for reference. It should be appreciated that the radiation from laser74apasses through pupil P as it is transmitted to and returned from retina R.

For this approach, laser wavelength selection bears on the contrast between retina R and outer surface OS reflection desired for eye detection. In one embodiment, 0.9 and 1.55-micron laser wavelengths are utilized for lasers74aand74b, respectively. Transmission from the retina R is approximately 87% at 0.9 microns and 0% at 1.55 microns. The reflectivity of the eyeball outer surface OS is about 2% at 1.55 microns. Assuming all the light is returned to detector82, the amplitude ratio is about 0.87/0.02=43.5. Further, the reflectivity of human skin at 0.9 and 1.55 microns wavelength is in the range of 84% and 27% respectively. Assuming all the light is returned to detector82, the amplitude ratio is 0.84/0.27=3.1. Therefore the eyeball detection contrast is enhanced by over a factor of ten using this dual wavelength approach. Further discrimination is provided as a result of the relatively longer round-trip travel of the 0.9-micron wavelength beam (IB and RB) through the interior of eyeball E. The path difference (z1−z2) between the dual laser radar wavelengths differs by a measurable amount. A typical human eyeball E has a diameter of about 25 millimeter (mm). Accordingly, a 2 mm range resolution provides about a 25-to-1 z1−z2signal differentiation.

For the wavelength selections of 0.9 micron (laser74a) and 1.55 microns (laser74b) it should be appreciated that the two corresponding laser beams are invisible to humans. Consequently, utilization of these wavelengths is more covert than visible light. Furthermore, it has been found that these wavelengths transmit through optical glass and optical plastic at a relatively high level (about 90% and 85% respectively), which are materials commonly used to make lenses of eyeglasses. Nonetheless, in alternative embodiments, one or more of these wavelengths may be different as would occur to those skilled in the art.

More specifically describing information recovery with circuitry92, evaluation of the amplitude and phase shift of the received sinusoidal signal, represented as “S” with respect to the AM modulation sinusoidal signal, represented as “R,” provides the target range information, as expressed in the following equations (4) and (5):
R=ARsin(ωt+φR)  (4)
S=ASsin(ωt+φIBS)  (5)
ASwill have a constant amplitude and ARand φRcan be evaluated using the complex form for the sinusoidal oscillation, as given by equation (6) that follows:
ejω1t=cos(ω1t)+jsin(ω1t)  (6)
by multiplying equations (4) and (5) by equation (6), and eliminating the components at frequency ω+ω1, the quadrature down conversion of signals R and S is obtained at a frequency ωd=ω+ω1, thus obtaining equations (7) and (8) as follows:

Sd=-j⁢AS2⁢ⅇj⁡(ωd⁢t+ϕS)(7)Rd=-j⁢AS2⁢ⅇj⁡(ωd⁢t+ϕR)(8)
By multiplying equation (7) by the complex conjugate of equation (8), equation (9) is obtained as follows:

Y=AS⁢AR4⁢ⅇj⁡(ϕS-ϕR)(9)
Equation (9) represents a vector whose unit of measurement is proportional to ASand whose phase is equal to φS−φR.

FIG. 4shows a signal diagram resulting from one experimental example of the present application with an experimental equipment set-up corresponding to subsystem70. This diagram is based on 0.9 and 1.55-micron (μm) wavelength selections for lasers74aand74b, respectively. InFIG. 4, line scan99aand line scan99bshow relative detected intensity information returned at the respective 0.9 micron (μm) and 1.55 (μm) wavelengths. The ratio of intensity (magnitude) of A1/A2is illustrated in line scan99c, and the difference in range z1−z2as determined by phase change is shown in line scan99d. The position of the eye (eyeball) relative to the scan timing is also designated by reference numeral100. As a result, not only phase difference, but also magnitude difference can be used to discriminate eye location. The location of one or more eyes relative to the other facial range information can be determined from the scan timing. Beside eye location, range information, and corresponding imagery; other facial interrogation techniques are provided by system20.

Next, referring toFIGS. 1 and 5, millimeter wave subsystem50is further described. Subsystem50is included in equipment40to enhance biometric information corresponding to Target T, as will be further described hereinafter. Subsystem50is arranged with interferometer51that includes Voltage Control Oscillator (VCO)52connected to mixer54and antenna56by couplers57aand57bas shown inFIG. 5. Mixer54is operatively connected to analog signal processing circuitry58. Oscillator52outputs a signal to couplers57aand57b, and to mixer54. Coupler57bfurther includes an amplifier to drive antenna56with the oscillator output signal. It should be appreciated that in other embodiments oscillator52may be of a fixed frequency type rather than a VCO and/or be in the form of a different time varying source to provide a drive signal at the desired frequency.

Circuitry58includes components to provide an Intermediate Frequency (IF) derived from oscillator52and mixer54to provide range information relating to the distance of Target T from antenna56in a standard manner. Circuitry58provides an analog signal corresponding to this range information. Circuitry58is connected to filter60for filtering-out undesirable frequencies (i.e., harmonics) from the information signal. This filtered signal is then provided to Analog-to-Digital Converter (ADC)62for conversion to a digital format. In one form, the filtered signal input to ADC62is oversampled at four time (4×) the Nyquist criterion sampling rate to generate I and Q demodulated outputs in accordance with standard techniques. ADC62is coupled to detector63to detect the desired information. For a 4× sampling rate with ADC62to provide I and Q outputs, detector63could be defined within subsystem24. In other arrangements, different interrogation, sampling, modulation/demodulation, or the like can be used as would occur to one skilled in the art.

In the depicted embodiment, interferometer51uses a monostatic antenna arrangement that transmits and receives millimeter waves. Antenna56is configured for a narrow beam pattern (spot size) that is mechanically scanned to measure facial dimensions. For some applications, it is preferred that high millimeter-wave frequencies (200-400 GHz) be utilized to provide a relatively small subsystem size.

Subsystem50utilizes low-power millimeter waves (radar signals) to illuminate the person being measured. These interrogation signals can penetrate typical clothing material, but are reflected/scattered by skin of the human body. Reflected signals are detected and processed with system20to capture spatial coordinate data. From this data, three-dimensional (3-D) measurement and corresponding representations (images) of human facial features can be provided to complement the three-dimensional range data gathered with subsystem70. Furthermore, millimeter wave signals can readily penetrate optically opaque materials such as body hair, make-up, and disguises. Correspondingly, typical clothing, make-up, disguise materials, and hair are generally transparent to the millimeter wave interrogation signals.

As illustrated inFIG. 5, a schematic partial cross section of target T is shown, including a layer of skin64with boundary64a, and disguise layer66with boundary66a, which covers boundary64aof skin64. Boundary66ais generally coextensive with outer surface OS for theFIG. 5cross section. Subsystem50penetrates disguise layer66, reflecting from skin64at boundary64a. While subsystem50provides information representative of facial skin topology—including boundary64acovered by disguise layer66, 3-D data gathered with subsystem70typically is reflected by make-up and disguise materials, such as boundary66aof disguise layer66. By comparing data obtained with subsystem50and subsystem70, make-up or other disguise or skin covering materials can be discovered that might otherwise go undetected.

From subsystem50and/or subsystem70, 3-D facial biometrics can be comprised of dimensional information from the scanned individual's anatomy. Once an individual reaches maturity (adulthood), his or her skeletal anatomy does not normally change dramatically over time (exceptions can include accident or disease). Corresponding surface data enables calculation of critical 1D, 2D and 3D skeletal dimensions and anthropometric measurements. The length and shape of various bones can be obtained from surface evidence (e.g., joints, skin protrusions). Because the skin covering the cranium is fairly thin, volumetric measurements of the skull and critical anthropometric data (e.g., placement, shape and distance between eye sockets) can also be obtained for those applications where desired.

InFIG. 1, facial scanning equipment40further includes color imager42. Color imager42provides imagery of target T in the standard color video Red-Green-Blue (RGB) format for processing subsystem24. This color image data can be used to compliment data gathered with subsystems50and70. For example, color images of a target T can be used to provide a visual representation of target T to an operator—with or without an indication of other information determined with subsystem50and/or subsystem70.

3-D facial scanning equipment40provides a rich feature vector space, where multispectral and/or multilayer 3-D range imagery are combined to provide information regarding target T. Feature vector space can be characterized with the analog video RGB channels, the signal amplitudes A, z ranges, and/or TOF measurements from subsystem70, and millimeter wave information from subsystem50including path length and amplitude differences.FIG. 6symbolically presents a matrix of parameters that can be used to provide unique biometric characterizations of face F of target T. InFIG. 6, color/image information from imager42is represented by the RGB color components (1st column). Amplitude/range for laser A and laser B are represented by A1/z1and A2/z2, respectively. An amplitude comparison corresponding to a difference or ratio is represented by ΔA (proportional to A1/A2and/or A1-A2) and a range comparison corresponding to a difference in ratio is represented by ΔZ (proportional to Z1/Z2and/or Z1−Z2). Amplitude and range indicated by millimeter wave interrogation are represented by Ammwand Zmmw, respectively. TOF is also represented in the matrix ofFIG. 6. The feature vectors are digitized at an analog color baseband frequency. Circuitry25can correspondingly provide a “color video” data fusion output in a multispectral format. For this approach, the laser beams from both lasers of subsystem70and the millimeter waves of subsystem50are raster-scanned across the target T at scanning rates compatible with data fusion and analog video. In one form, some or all of theFIG. 6parameters are generated by pre-processor32as a set of signals that define a facial ‘signature’ in multiple frames. These signature signals provide data from which scanned face F can be recognized/identified.

Referring generally toFIGS. 1-5and specifically toFIG. 7, procedure120of another embodiment of the present application is illustrated in flowchart form. Procedure120is implemented with system20, and appropriately configured operating logic of subsystem24and equipment40. This operating logic can be in the form of programming instructions, hardwired sequential or combinational logic, and/or adaptive or fuzzy logic, to name just a few possibilities.

Procedure120begins with operation122in which equipment40is utilized to scan face F of target T. Scanning of operation122includes generating at least two different selected wavelengths A and B of subsystem70to provide a topological scan of face F and locate eyes E1and E2thereof. Also included is a scan with subsystem50to penetrate disguises, make-up, hair, and the like, while being reflected by skin of target T to provide corresponding 3-D facial data. In addition, color image information is provided with subsystem42during operation122. Collectively, this signature data can be provided and grouped into frames with pre-processor32, and input to processor34, which performs subsequent operations/conditionals.

From operation122, procedure120continues with parallel operations124aand124b. In operation124a, eye location is determined as described in connection with subsystem70. In operation124b, disguise or make-up presence is determined by comparing data obtained with subsystems50and70. Notably, while operations124aand124bare performed in parallel, in other embodiments they can be performed in sequence in any order.

From operations124aand124b, procedure120continues with operation126. In operation126, desired three-dimensional facial feature information is developed to provide a basis to uniquely identify target T for facial recognition purposes or the like. Examples of the type of information that could be developed in operation126are characterizations of the type described in connection withFIG. 6. Alternatively or additionally, this information could be based on eye-to-eye vector distance V and various vectors determined in relation to eye location or vector V including, for example, vectors to cheeks CK1and/or CK2, mouth-corners M1and/or M2, nose N1, and/or chin C1, (see FIG.1)—to name just a few possibilities.

From operation126, operation128of procedure120is performed. In operation128, at least a portion of the information developed in operation126is compared to identification data stored on a local and/or remote identification database. This database can take any of several forms. In one example, a two-dimensional image database is utilized from which 3-D constructs are created. 3-D information determined in operation126is then compared in operation128. Additionally or alternatively, 3-D information obtained in operation126can be converted to two-dimensional data for comparison to a two-dimensional database of identification information. In one particular example, the three-dimensional data from system20is converted to five two-dimensional images for comparison to images in a preexisting two-dimensional image database. In still another example, a new three-dimensional database can be developed for use in the comparison of128. In yet other examples, these approaches are combined.

Procedure120continues from operation128with conditional130. In conditional130, it is tested whether the comparison of128indicates a suitable match of target T to information present in the corresponding database. If the test of conditional130is true (affirmative), then procedure120continues with operation132in which an alert is provided to an operator. Such operator can be locally or remotely positioned with respect to equipment40and/or target T. If the test of conditional130is false (negative) then procedure120continues with conditional134. Conditional134tests whether to continue scanning target T or another target. If the test of conditional134is true (affirmative) procedure120loops back, returning to operation122to perform the 122-128 sequence again for submission to conditional130. This loop can be repeated as desired based on the outcome of conditional134. If the test of conditional134is false (negative), then procedure120halts.

Procedure120is but one example of a mode of operating system20. It should be appreciated that many combinations, rearrangements, deletions, and the like are contemplated in other embodiments. For example, in other embodiments one or more of subsystems42,50, or70may be absent. In still other embodiments, the selected wavelengths of electromagnetic radiation utilized for interrogation by subsystem70may vary as deemed appropriate. Likewise, the nature and type of interrogation performed with millimeter waves can vary with adaptations made to subsystem50as appropriate. Further, subsystem42can be altered as appropriate. In one particular example an infrared scan with corresponding color representation is provided as an addition or alternative to subsystem42. In still other embodiments, one or more subsystems or aspects of system20are applied to one or more other portions of a person's body as an addition or alternative to face F. Further, system20or subsystems thereof may be utilized in connection with the interrogation of objects other than a person. In one particular embodiment, subsystem70is utilized to determine eye location for a different facial evaluation technique that may be provided with or without subsystem50and/or42. Processing subsystem24would be adapted for any of these variations as appropriate to the particular operating goals of the alternative embodiment. In yet further embodiments, multispectral interrogation is used to detect and/or evaluate inanimate objects including one or more retroreflectors. In one particular example, retroreflectors in optical tags can be interrogated in such embodiments. Furthermore, retroreflectors can be used for device labeling, as a marker, to encode data, and the like—all of which can be interrogated/determined in accordance with the present invention.

In one example of an alternative embodiment,FIG. 8illustrates system220for evaluating an iris of the eye obtained from scanning a “sea of faces” as illustrated by the example designated by reference numeral222. In the embodiment ofFIG. 8, like reference numerals refer to like features. System220includes laser subsystem70as previously described. System220further includes optical scanning and detection subsystem324, detector processing circuitry226, and data processing subsystem228. Also represented is a scan synchronized image scene as symbolically portrayed inFIG. 8.

Subsystem224includes a multifaceted scanning mirror230coupled to drive/control232. Drive/control232controllably spins mirror230to provide a Field Of View (FOV) capable of scanning several faces from a desired separation distance. In one example, a 30° FOV is scanned, which has been found to allow up to 12 faces to be examined in one scan at 8 meters. The resulting scan is reflected on front surface mirror234, which is driven by galvometer drive/control236. The resulting beam237is directed to beam splitter238to provide a beam input237ato subsystem70and beam input237bto optics arrangement240. Optics arrangement240includes cylindrical lens242, refractive prism244, and multispectral detectors246. Cylindrical lens242compresses vertical scan pixels into a linear array that is spread with prism244across detectors246. In one form, three detectors246are provided corresponding to three different laser wavelengths used for the scan—particularly 980 nanometers, 1200 nanometers, and 1550 nanometers. Referring to the subsystem70description, it should be appreciated that 980 nanometers is approximately 0.9 microns and 1550 nanometers is approximately 1.55 microns. Signals from detectors246are input to detector processing circuitry226for recovery, conditioning, and conversion to a desired digital format. These digital signals are then output by circuitry226to processing subsystem228.

Processing subsystem228performs in accordance with operating logic as described in connection with system20. Included in this logic, is the processing of signals from subsystem70to detect eye location and to perform analysis of an iris of the eye in accordance with standard iris detection algorithms. In one particular form, a 512 byte iris code using wavelet compression is derived from iris pixels, generally independent of the degree of pupil dilation.

Referring toFIG. 10, a visible image A and infrared image B are presented for comparison. Further, in image A, multiscale, quadrature wavelet iris code is illustrated in the upper left hand corner as designated by reference numeral300. In one form, subsystem70output is processed to determine location of pupil P to process the image of iris I. A dedicated, highly integrated digital circuit can be provided to perform on-the-fly iris imaging and processing. In one form, code processing is performed by a Field Programmable Gate Array (FPGA) arranged to perform the desired processes. Notably, this arrangement can be used to process iris images provided by one or more different wavelengths. In one particular form, multiple infrared (IR) images are processed in this manner. For further background information concerning iris processing, reference is made to J. Daugman, “How Iris Recognition Works” [www.CL.cam.ac.uk/users/jgd1000/]; J. Daugman, “Biometric Product Testing” [www.cl.cam.ac.uk/users/jgd1000/NPLsummary.gif]; J. Daugman, “The Importance of Being Random: Statistical Principles of Iris Recognition” (Elsevier Science Ltd. 2002), all of which are hereby incorporated by reference.

Referring toFIG. 9, procedure320of a further embodiment of the present application is illustrated in flowchart form. Procedure320can be implemented with system220, performing various operations and conditionals in accordance with operating logic of corresponding subsystems. Procedure320begins with operation322in which scanning of a scene is performed. Such a scene is illustrated inFIG. 8as indicated by reference numeral222. From the scanned face(s) of the scene, iris/pupil location is determined utilizing subsystem70in operation324. From operation324, procedure320continues with operation326in which iris images are evaluated to provide a corresponding iris code or other iris identification information. In one form, a quadrature wavelet form of iris code is determined in operation326; however, in other forms different techniques can additionally or alternatively be utilized.

Procedure320continues with operation328in which the generated iris information is compared to an identification database. Such database may be local or remote relative to system220. In one example, database information is at least partially provided through computer network30. From operation328, procedure320continues with conditional330. Conditional330tests whether a match was identified in operation328. If the test of conditional330is true (affirmative), the corresponding match condition is indicated by providing an alert in operation322. If the test of conditional330is false (negative), then procedure320continues with conditional334. From operation332, procedure320proceeds to conditional334. Conditional334tests whether to continue execution of procedure320by scanning an additional scene or rescanning as appropriate. If the test of conditional334is true (affirmative), procedure320loops back, returning to operation332to perform operation sequence322-328again. This sequence may be repeated via the loop back from conditional334as desired. If the test of conditional334is false (negative), then procedure320halts.

As previously indicated, numerous variations, forms, and embodiments of the present application are envisioned. In another form, the iris evaluation/comparison of procedure320is performed in addition to procedure220in which a 3-D facial feature comparison is made. In still other embodiments, other recognition/identification techniques may be combined with procedures120and/or320as desired.

Monitoring humans for biometric analysis is applicable to a wide range of technologies for purposes of identification, verification, unknown threat recognition, access control, security checkpoints, and the like. Nonetheless, in other embodiments, the transmission/reception arrangement can differ. For example, in one alternative embodiment, one or more elements38are used for both transmission and reception. In another alternative embodiment, a mixture of both approaches is utilized. Typically, the signals received from array36are downshifted in frequency and converted into a processable format through the application of standard techniques. In one form, transceiver42is of a bi-static heterodyne Frequency Modulated Continuous Wave (FM/CW) type like that described in U.S. Pat. No. 5,859,609 (incorporated by reference herein). Commonly owned U.S. Pat. Nos. 6,703,964 B2; 6,507,309 B2; 5,557,283; and 5,455,590, each of which are incorporated by reference herein, provide several nonlimiting examples of transceiver arrangements. In still other embodiments, a mixture of different transceiver/sensing element configurations with overlapping or nonoverlapping frequency ranges can be utilized that may include one or more of the impulse type, monostatic homodyne type, bi-static heterodyne type, and/or such other type as would occur to those skilled in the art.

Another embodiment includes: directing multiple wavelengths of electromagnetic radiation to scan a face of a person; detecting at least one of the wavelengths reflected by a first portion of an eye of the person to establish a first signal and at least one other of the wavelengths reflected by a second portion of the eye through a pupil thereof to establish a second signal; evaluating these signals to provide a value that varies with distance separating the first portion and the second portion; and determining location of the eye as a function of the value. In one form, the second portion is behind the first portion of the eye—the second portion being a retina.

Still another embodiment of the present application includes: interrogating the face of a person with coherent electromagnetic radiation including at least one wavelength reflected by a first portion of an eye and at least one other wavelength reflected by a second portion of the eye, with the second portion being behind and interior to the first portion. This embodiment further includes determining location of the eye based on the interrogation, where such location corresponds to a difference in range relative to the first and second portions and characterizing at least part of the face relative to the location of the eye for comparison to identification information. In one form, this characterization includes evaluating three-dimensional facial information corresponding to at least a portion of the face. Alternatively or additionally, this embodiment may include locating a different eye of the person based on a range difference, determining distance separating the eyes, and/or performing the characterization as a function of this distance.

Yet another embodiment includes: detecting reflection of one or more wavelengths of electromagnetic radiation by a first boundary along a face of a person; detecting reflection of one or more other wavelengths of electromagnetic radiation by a second boundary along the face; recognizing one or more portions of the face based on one of the first and second boundaries at least partially covering another of the first and second boundaries; and comparing at least part of the face to identification information in accordance with this recognition. In one form, the first boundary corresponds to makeup or a disguise placed on the face, where the second boundary might be facial skin.

A further embodiment includes a facial scanning arrangement with a first laser to provide coherent electromagnetic radiation including a first wavelength, a source to provide electromagnetic radiation including a second wavelength, and one or more detectors to sense returned electromagnetic radiation of at least each of these wavelengths. Also included is a processing subsystem coupled to the facial scanning arrangement that is responsive to signals from the one or more detectors to locate one or more portions of the face of a person scanned with the arrangement by detecting reflection of at least the first wavelength of the electromagnetic radiation by a first boundary along the face and reflection of at least the second wavelength of the electromagnetic radiation by a second boundary along the face. One of these boundaries at least partially covers another of these boundaries, and the processing subsystem is operable to compare at least part of the face to identification information in accordance with location of the one or more portions of the face. In one form, this embodiment includes the processing subsystem evaluating three-dimensional facial information relative to the identification information.

Still a further embodiment of the present invention is a system including: a facial scanning arrangement including a first laser to provide coherent electromagnetic radiation including a first wavelength, a source to provide electromagnetic radiation including a second wavelength in a range of preferably about 0.1 mm to about 100 mm and one or more detectors to sense returned electromagnetic radiation of at least the first and the second wavelengths. This system further includes a processing subsystem operatively coupled to the facial scanning arrangement that is responsive to signals from the detectors to determine a difference in range between two locations along a face of a person scanned therewith. One of these locations at least partially covers another of these locations and corresponds to reflection of the first wavelength while the other of the locations corresponds to reflection of the second wavelength as detected with the one or more detectors. The processing subsystem is further operable to recognize that the disguising material is on the face and is a function of the difference in range. The processing subsystem is further operable to identify the person based on evaluation of at least one iris of an eye, and/or determine location of one or more eyes of the person based on a laser ranging subsystem. In a more preferred embodiment, the range of electromagnetic radiation is about 1 mm to about 50 mm. In an even more preferred embodiment, this range is about 1 mm to about 10 mm.

Yet a further embodiment of the present invention includes: means for detecting reflection of one or more wavelengths of electromagnetic radiation by a first boundary along a face of a person; means for detecting reflection of one or more other wavelengths of the electromagnetic radiation by a second boundary along the face; means for recognizing one or more portions of the face based on one of the first and second boundaries at least partially covering another of the first and second boundaries; and means for comparing at least one of these portions to identification information.

Another embodiment of the present invention includes: scanning a face of a person with laser equipment; determining eye location from this scan based on range information from the equipment that is indicative of a first eye portion being separated from and in front of a second eye portion; generating identification information for an iris of the eye located relative to the eye location; and comparing this identification information to data from an identification database.

For yet another embodiment, a system includes: a facial scanning arrangement with laser equipment to interrogate a person with electromagnetic radiation, which is operable to provide range information and image data. This system also includes a processing subsystem responsive to the facial scanning arrangement to generate a number of signals. These signals are representative of eye location determined from the range information, identification information determined from the image data for an iris of the eye located with the eye location, and a comparison of the identification information to data from an identification database. An output device may also be included that is responsive to one or more of the signals if the comparison indicates a match of the person to an individual characterized in the database.

Still another embodiment of the present invention includes: scanning a face of a person with laser equipment; determining eye location from the scan based on range information; generating three-dimensional facial feature information relative to the eye location; and comparing the three-dimensional facial feature information to data from an identification database.

Still another embodiment includes: a facial scanning arrangement with laser equipment and a processing subsystem responsive to such arrangement. The processing subsystem generates a number of signals that are representative of eye location determined from range information provided by the laser equipment, three-dimensional facial feature information determined relative to the eye location, and at least one comparison of the three-dimensional facial feature information to data from an identification database.

A further embodiment includes: means for scanning a face of a person with laser ranging equipment; means for determining an eye of the person based on range information from the scanning means; at least one of means for generating identification information from image data for an iris of the eye and means for generating three-dimensional facial feature information relative to the eye location; and means for comparing the identification information to data from an identification database.

Still a further embodiment is directed to a method that includes: directing multiple wavelengths of electromagnetic radiation to scan an object including a retro-reflector, detecting at least one of the wavelengths reflected by a first portion of the object to establish a first signal, detecting at least one other of the wavelengths reflected by a second portion of the object to establish a second signal, and determining a relative location or distance as a function of the first signal and the second signal. In other embodiments of the present application, apparatus, systems, devices, and the like can be provided that implement this method.

As used herein, “millimeter wave” or “millimeter wavelength” refers to any electromagnetic radiation that has a wavelength in the range from about 0.1 millimeter to about 100 millimeters when propagating through free space. Also, as used herein, it should be appreciated that: variable, criterion, characteristic, comparison, quantity, amount, information, value, level, term, constant, flag, data, record, threshold, limit, input, output, pixel, image, matrix, command, look-up table, profile, schedule, or memory location each generally correspond to one or more signals within processing equipment of the present invention. It is contemplated that various operations, stages, conditionals, procedures, thresholds, routines, and processes described in connection with the present invention could be altered, rearranged, substituted, deleted, duplicated, combined, or added as would occur to those skilled in the art without departing from the spirit thereof. It should be noted that implementation of the disclosed embodiments of the present invention is not limited to those depicted in the figures.

All publications and patent applications cited in this specification are herein incorporated by reference as if each individual publication or patent application were specifically and individually indicated to be incorporated by reference. Further, any theory, mechanism of operation, proof, or finding stated herein is meant to further enhance understanding of the present invention, and is not intended to limit the present invention in any way to such theory, mechanism of operation, proof, or finding. While the invention has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, it being understood that only selected embodiments have been shown and described and that all equivalents, changes, and modifications that come within the spirit of the inventions as defined herein or by the following claims are desired to be protected.