Patent Description:
An ultrasonic fingerprint sensor system or an optical fingerprint sensor system can be good means for obtaining a biometric identifying image of a target object, such as a finger. Under-display ultrasonic fingerprint sensors have been recently commercialized in the market. Although some previously-deployed fingerprint sensor systems were generally satisfactory for some applications, improved fingerprint sensor systems would be desirable.

<CIT> describes a photoacousitic fingerprint sensing module with a sensing surface. An optical pulse is irradiated towards the sensing surface. As the fingerprint portion of a finger is momentarily thermally expanded by the irradiated light pulse an ultrasonic receiving element senses the generated ultrasonic signal through a medium under the sensing surface. The sensed ultrasonic signal is imput to an image processor as fingerprint configuration information.

<CIT> describes a polyvinylidene fluoride piezoelectric material for solving problems of unsatisfactory piezoelectric property and large energy loss of piezoelectric materials used in ultrasonic fingerprint recognition modules.

<CIT> describes an ultrasonic fingerprint identification circuit in which a photosensitive unit can respond to light reflected by a fingerprint to generate an optical signal. The optical signal is superposed with a first valley signal/first ridge signal obtained by an ultrasonic signal reflected by the fingerprint to obtain a second valley signal/second ridge signal. Due to the fact that the reflection amount of the valleys of the fingerprint to the light is larger than the ridge of the fingerprint, the signal difference value of the second valleys is larger than the signal difference value of the first valleys, and the accuracy of fingerprint recognition is improved.

The invention is defined in the appended independent claims. Optional features are defined in the dependent claims.

The following description is directed to certain implementations for the purposes of describing the innovative aspects of this disclosure. However, a person having ordinary skill in the art will readily recognize that the teachings herein may be applied in a multitude of different ways. The described implementations may be implemented in any device, apparatus, or system that includes a biometric system as disclosed herein. In addition, it is contemplated that the described implementations may be included in or associated with a variety of electronic devices such as, but not limited to: mobile telephones, multimedia Internet enabled cellular telephones, mobile television receivers, wireless devices, smartphones, smart cards, wearable devices such as bracelets, armbands, wristbands, rings, headbands, patches, etc., Bluetooth® devices, personal data assistants (PDAs), wireless electronic mail receivers, hand-held or portable computers, netbooks, notebooks, smartbooks, tablets, printers, copiers, scanners, facsimile devices, global positioning system (GPS) receivers/navigators, cameras, digital media players (such as MP3 players), camcorders, game consoles, wrist watches, clocks, calculators, television monitors, flat panel displays, electronic reading devices (e.g., e-readers), mobile health devices, computer monitors, auto displays (including odometer and speedometer displays, etc.), cockpit controls and/or displays, camera view displays (such as the display of a rear view camera in a vehicle), electronic photographs, electronic billboards or signs, projectors, architectural structures, microwaves, refrigerators, stereo systems, cassette recorders or players, DVD players, CD players, VCRs, radios, portable memory chips, washers, dryers, washer/dryers, parking meters, packaging (such as in electromechanical systems (EMS) applications including microelectromechanical systems (MEMS) applications, as well as non-EMS applications), aesthetic structures (such as display of images on a piece of jewelry or clothing) and a variety of EMS devices. The teachings herein also may be used in applications such as, but not limited to, electronic switching devices, radio frequency filters, sensors, accelerometers, gyroscopes, motion-sensing devices, magnetometers, inertial components for consumer electronics, parts of consumer electronics products, steering wheels or other automobile parts, varactors, liquid crystal devices, electrophoretic devices, drive schemes, manufacturing processes and electronic test equipment. Thus, the teachings are not intended to be limited to the implementations depicted solely in the Figures, but instead have wide applicability as will be readily apparent to one having ordinary skill in the art.

An ultrasonic fingerprint sensor can, in many circumstances, obtain a suitable biometric identifying image of a finger or other target object. However, some acoustic insonification processes have significant power requirements. Some films used for screen protection can significantly attenuate ultrasonic signals. Moreover, issuing a high-voltage pulse into a piezoelectric transmitter can bring unwanted electrical noise to the system.

Some such implementations use light to generate waves of ultrasonic or acoustic energy at or near an outer surface (such as a platen surface or a cover layer) using photoacoustically-activated nanoparticles. For example, light may be provided to a leaky light waveguide and may be directed through a layer of nanoparticles. Alternatively, or additionally, the waveguide itself may include nanoparticles. Some disclosed sensor systems may include nanoparticles in a platen or a cover layer. The nanoparticles may resonate and create ultrasonic waves via thermo-elastic heating. Some of the resulting ultrasound waves may travel toward the outer surface where they interact with structures of a target object, such as the ridges and valleys of a finger in contact with the outer surface. Ultrasonic waves contacting air spaces corresponding to the valleys may be reflected and may be detected by an ultrasonic receiver array. A control system may be configured to perform an authentication process and/or an imaging process based, at least in part, on ultrasonic receiver signals received from the ultrasonic receiver array. Some implementations may include a piezoelectric ultrasonic receiver, transmitter or transceiver layer that includes micro- or nano-ferroelectric particles.

Some implementations also may include an optical receiver array. Some such implementations may include both optical and ultrasonic receivers in a single receiver layer. A control system may be configured to control the light source system to illuminate at least a portion of the target object. The control system may be configured to perform an authentication process and/or an imaging process based, at least in part, on optical receiver signals received from the optical receiver array. In some such implementations, the control system may be configured to control the light source system to induce photoacoustic emissions from the target object. In some examples, an apparatus may include a layer of second metamaterial that is configured to convert acoustic waves into light.

Particular implementations of the subject matter described in this disclosure may be implemented to realize one or more of the following potential advantages. A higher signal may be gained from a piezoelectric receiver or transmitter by including micro- or nano-particles of a ferroelectric material that has a high electromechanical coupling factor with a suitable piezoelectric polymer matrix such as polyvinylidene fluoride (PVDF) or polyvinylidene fluoride-trifluoroethylene (PVDF-TrFE) copolymers. Some disclosed ultrasonic sensor systems having photoacoustically-activated nanoparticles, as well as those configured to induce photoacoustic emissions from a target object, do not require a piezoelectric transmitter. In such sensor systems, there is no need to provide a high-voltage pulse to a piezoelectric transmitter and no associated electrical noise will be caused. Sensor systems that are configured to perform an authentication process and/or an imaging process that is based, at least in part, on optical receiver signals may be able to ameliorate the attenuation of ultrasonic signals caused by some films that are used for screen protection.

<FIG> is a block diagram that shows example components of an apparatus according to some disclosed implementations. In this example, the apparatus <NUM> includes a cover layer <NUM> and a layer of first metamaterial <NUM>. The cover layer <NUM> may, in some examples, be optically transparent. However, in some implementations the cover layer <NUM> may be optically opaque or optically translucent. For example, the cover layer <NUM> may be, or may include, an optically opaque or optically translucent platen.

According to this implementation, the layer of first metamaterial <NUM> includes nanoparticles that are configured to resonate and create an ultrasonic wave when illuminated by light. The layer of first metamaterial <NUM> may include such nanoparticles dispersed in binder material, such as a transparent binder material. In some implementations the layer of first metamaterial <NUM> may be proximate the cover layer <NUM> and may, in some examples, be adjacent to the cover layer <NUM>. However, in some examples, at least some of the nanoparticles (and/or other metamaterial) may reside in the cover layer <NUM>.

In this example, the apparatus <NUM> includes a light source system <NUM>. According to some implementations, the light source system <NUM> is configured for providing light to the layer of first metamaterial <NUM>. According to some implementations, the light source system <NUM> may include one or more light sources that are configured to emit light that includes a resonant frequency of at least some of the nanoparticles. Relatively larger particles will generally resonate at relatively lower frequencies and relatively smaller particles will generally resonate at relatively higher frequencies. It may be advantageous to have the particles sized to vibrate at one or more operating frequencies of interest. In some examples, the light source may be, or may include, one or more light-emitting diodes (LEDs) and/or one or more laser diodes. According to some implementations, the LEDs may be organic light-emitting diodes (OLEDs). In some implementations, the light source may be configured to provide light that is invisible to the human eye, such as infrared or ultraviolet light.

In some examples, the light source system <NUM> may include a waveguide proximate the layer of first metamaterial <NUM>. The waveguide may be configured for receiving light from the light source and providing the light to the layer of first metamaterial <NUM>. For example, the light source system <NUM> may include a light source and a leaky light waveguide. Light from the light source may be provided to the leaky light waveguide and may be directed towards the cover layer <NUM> (e.g., upward) through nanoparticles in the layer of first metamaterial <NUM> and/or the cover layer <NUM>. Alternatively, or additionally, the waveguide itself may include such nanoparticles.

According to some examples, the light source may be, or may include, a pulsed light source. Some such implementations may include a control system <NUM>. In some such implementations, the control system <NUM> may be configured to control the light source system to cause the nanoparticles to emit ultrasonic waves in the range of <NUM> to <NUM>. However, in other implementations the control system <NUM> may be configured cause the nanoparticles to emit ultrasonic waves in a frequency range that is above or below the range of <NUM> to <NUM>. For example, in some implementations the control system <NUM> may be configured to control the light source system to cause the nanoparticles to emit ultrasonic waves in the range of <NUM> to <NUM>, e.g., in the range of <NUM> to <NUM>. Such implementations may cause the nanoparticles of the layer of first metamaterial <NUM> to emit ultrasonic waves in a range suitable for an ultrasonic imaging process and/or authentication process.

In this example, the apparatus <NUM> includes a receiver system <NUM>. In some implementations, the receiver system <NUM> includes an ultrasonic receiver system. In some such implementations, the control system may be configured to receive ultrasonic receiver signals from the ultrasonic receiver system corresponding to the ultrasonic waves reflected from a target object in contact with, or proximate, a surface of the cover layer <NUM>. The control system may be configured to perform at least one of an authentication process or an imaging process that is based, at least in part, on the ultrasonic receiver signals.

In some such implementations, the ultrasonic receiver system includes piezoelectric material, such as a PVDF polymer or a PVDF-TrFE copolymer. In some implementations, a separate piezoelectric layer may serve as the ultrasonic transmitter. In some implementations, a single piezoelectric layer may serve as the transmitter and as a receiver. Other implementations may have an ultrasonic receiver but not an ultrasonic transmitter. In some implementations, other piezoelectric materials may be used in the piezoelectric layer, such as aluminum nitride (AlN) or lead zirconate titanate (PZT).

According to some examples, the piezoelectric material may include micro-particles or nano-particles in a piezoelectric polymer matrix. In some such implementations, the micro-particles and/or the nano-particles may include ferroelectric material. In some examples, the micro-particles and/or the nano-particles may include ZnO, PZT and/or BaTiO3. According to some implementations, the micro-particles and/or the nano-particles may have a diameter that is in the range of <NUM> to <NUM>.

In some instances, the receiver system <NUM> may include an optical receiver system. According to some such implementations, a single layer of the receiver system <NUM> may include portions of an optical receiver system and portions of an ultrasonic receiver system. Some examples are described below.

The control system <NUM> may include one or more general purpose single- or multi-chip processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, or combinations thereof. The control system <NUM> also may include (and/or be configured for communication with) one or more memory devices, such as one or more random access memory (RAM) devices, read-only memory (ROM) devices, etc. Accordingly, the apparatus <NUM> may have a memory system that includes one or more memory devices, though the memory system is not shown in <FIG>. The control system <NUM> may be capable of receiving and processing data from the receiver system <NUM>. If the apparatus <NUM> includes an ultrasonic transmitter, the control system <NUM> may be capable of controlling the ultrasonic transmitter <NUM>. In some implementations, functionality of the control system <NUM> may be partitioned between one or more controllers or processors, such as a dedicated sensor controller and an applications processor of a mobile device.

Some implementations of the apparatus <NUM> may include an interface system <NUM>. In some examples, the interface system may include a wireless interface system. In some implementations, the interface system may include a user interface system, one or more network interfaces, one or more interfaces between the control system <NUM> and a memory system and/or one or more interfaces between the control system <NUM> and one or more external device interfaces (e.g., ports or applications processors).

The interface system <NUM> may be configured to provide communication (which may include wired or wireless communication, such as electrical communication, radio communication, etc.) between components of the apparatus <NUM>. In some such examples, the interface system <NUM> may be configured to provide communication between the control system <NUM> and the receiver system <NUM> and/or the light source system <NUM>. According to some such examples, a portion of the interface system <NUM> may couple at least a portion of the control system <NUM> to the receiver system <NUM> and/or the light source system <NUM>, e.g., via electrically conducting material. If the apparatus <NUM> includes an ultrasonic transmitter <NUM> that is separate from the receiver system <NUM>, the interface system <NUM> may be configured to provide communication between at least a portion of the control system <NUM> and the ultrasonic transmitter <NUM>.

According to some examples, the interface system <NUM> may be configured to provide communication between the apparatus <NUM> and other devices and/or human beings. In some such examples, the interface system <NUM> may include one or more user interfaces. The interface system <NUM> may, in some examples, include one or more network interfaces and/or one or more external device interfaces (such as one or more universal serial bus (USB) interfaces). In some implementations, the apparatus <NUM> may include a memory system. The interface system <NUM> may, in some examples, include at least one interface between the control system <NUM> and a memory system.

The apparatus <NUM> may be used in a variety of different contexts, some examples of which are disclosed herein. For example, in some implementations a mobile device may include at least a portion of the apparatus <NUM>. The mobile device may, in some instances, be a cellular telephone, a tablet device, etc. In some implementations, a wearable device may include at least a portion of the apparatus <NUM>. The wearable device may, for example, be a bracelet, an armband, a wristband, a ring, a headband or a patch. In some implementations, the control system <NUM> may reside in more than one device. For example, a portion of the control system <NUM> may reside in a wearable device and another portion of the control system <NUM> may reside in another device, such as a mobile device (e.g., a smartphone or a tablet computer). The interface system <NUM> also may, in some such examples, reside in more than one device.

<FIG> shows another example of an apparatus configured to perform at least some methods disclosed herein. The apparatus <NUM> of <FIG> may be considered an instance of the apparatus <NUM> shown in <FIG>. In this example, the light source system <NUM> includes a light source <NUM> and a waveguide <NUM>. According to this example, the layer of first metamaterial <NUM> resides between the waveguide <NUM> and the cover layer <NUM>, on which a target object <NUM> is placed. The target object <NUM> is a finger in this example. (As used herein, the term "finger" has a meaning that is broad enough to refer to any digit, including a thumb. Accordingly, the term "fingerprint" has a meaning that is broad enough to refer to the print of any digit, including a thumbprint.

In some implementations, the light source <NUM> includes at least one LED and/or at least one laser diode. According to this example, the waveguide <NUM> is configured to direct at least some of the light provided by the light source <NUM> to the layer of first metamaterial <NUM>.

In this implementation, the layer of first metamaterial <NUM> includes nanoparticles <NUM> that are configured to resonate and create an ultrasonic wave when illuminated by light. In this example, the layer of first metamaterial <NUM> includes binder material in which the nanoparticles <NUM> are suspended. In some such examples, the binder material may include a transparent material, such as a transparent polymeric binder. According to some examples the nanoparticles <NUM> may include gold nanoparticles. In some instances, the nanoparticles <NUM> may include carbon nanoparticles. According to some such examples, the nanoparticles <NUM> may include the type of carbon black particles used in India ink. In some examples, the nanoparticles <NUM> may be as small as <NUM> nanometers, whereas in other examples the nanoparticles <NUM> may be <NUM> nanometers or more in size.

In the example shown in <FIG>, the nanoparticles <NUM> are used for ultrasonic insonification. In some examples, the light source <NUM> may be configured to emit light that includes a resonant frequency of at least some of the nanoparticles <NUM>. According to some examples, the light <NUM> includes a resonant frequency of the nanoparticles <NUM>. Therefore, illumination by the light <NUM> causes the nanoparticles <NUM> to expand.

Alternatively, or additionally, in some examples, the pulse duration and/or the time interval between the light pulses emitted by the light source <NUM> dictates the frequency of the ultrasonic waves <NUM> that are emitted by the nanoparticles as they expand while illuminated and contract between pulses. In some examples, the light pulses emitted by the light source <NUM> have a pulse duration that is in the range of <NUM>-<NUM>, e.g., <NUM>. In some such examples, the light pulses emitted by the light source <NUM> have a power density in the range of <NUM> to <NUM> mW/mL, e.g., <NUM> mW/mL.

In the photoacoustic system of apparatus <NUM>, there is a small latency while the nanoparticles heat, expand and then collapse. This latency is well-defined in the literature, e.g., in Chao Tian, Zhixing Xie, Mario L. Fabilli, Shengchun Liu, Cheng Wang, Qian Cheng, and Xueding Wang, "Dual-pulse nonlinear photoacoustic technique: a practical investigation," Optical Society of America, <NUM> (hereinafter, "Tian et al. This latency should be taken into consideration when determining appropriate criteria for the photoacoustic system, e.g., appropriate values for the range gate delay used for obtaining data from the receiver system <NUM>. As noted in Tian et al. , it is desirable for the latency to be at least as long as the "stress relaxation time" τs and no longer than the thermal relaxation time τth. The thermal relaxation time, which is the time required for a target to cool to <NUM>% of its excited state temperature, may be expressed as follows: <MAT>.

In Equation <NUM>, d represents the diameter of the target, which in some examples may be the spot size of the light source. In some disclosed implementations, d may be in the range of <NUM>-<NUM>. In Equation <NUM>, αth represents the thermal diffusivity. The thermal diffusivity will vary according to the materials used to form the nanoparticles and the binder in which the nanoparticles are dispersed. In some examples, the thermal diffusivity may be in the range of <NUM>-<NUM><NUM>/second. In one example, assuming a thermal diffusivity of <NUM><NUM>/second and a target volume of <NUM>, the thermal relaxation time that may be calculated according to Equation <NUM> is <NUM>.

The stress relaxation time may be expressed as follows: <MAT>.

In Equation <NUM>, d represents the diameter of the target and cL represents the longitudinal speed of sound in the relevant medium. In one example, assuming a longitudinal speed of sound of <NUM> meters/second (a typical value for a coating binder) and a target volume of <NUM>, the stress relaxation time that may be calculated according to Equation <NUM> is <NUM>.

Accordingly, in this example the latency while the nanoparticles heat, expand and then collapse would be in the range of <NUM> nanoseconds to <NUM> microseconds. According to the photoacoustic literature, including Tian et al. the light pulse duration should be less than either time, e.g., in the range of <NUM> to <NUM>. For implementations in which the light source system <NUM> is configured to emit a series of light pulses, the interval between light pulses should be at least <NUM> in this example in order to avoid ablation and cavitation.

Assuming an apparatus configured as shown in <FIG> or <FIG>, if t<NUM> is the time at which the light source system <NUM> a light pulse, the elapsed time between t<NUM> and the time that the light pulse reaches the nanoparticles is negligible and may be disregarded. In this example, the nanoparticles <NUM> would begin emitting ultrasonic waves approximately <NUM> after t<NUM>. The appropriate range gate delay after t<NUM> could be determined by adding this latency to the length of time that it would take for the emitted ultrasonic waves <NUM> to travel from the nanoparticles <NUM> to a target area and back to the receiver system <NUM>. For example, for fingerprint imaging, the appropriate range gate delay after t<NUM> could be determined by adding this latency to the length of time that it would take for the emitted ultrasonic waves <NUM> to travel from the nanoparticles <NUM> to the outer surface of the cover layer <NUM> and back to the receiver system <NUM>.

In the example shown in <FIG>, the ultrasonic waves <NUM> are shown impinging on ridge areas <NUM> and valley areas <NUM> of the finger. There is little impedance contrast between the ridge areas <NUM> and the cover layer <NUM>, so much of the energy of the ultrasonic waves <NUM> that impinge on the ridge areas <NUM> is transmitted into the finger. However, there is a large impedance contrast between air in the valley areas <NUM> and the cover layer <NUM>, so much of the energy of the ultrasonic waves <NUM> that impinge on the valley areas <NUM> is reflected.

Some of these reflected ultrasonic waves <NUM> are received by an ultrasonic receiver of the receiver system <NUM>. According to some implementations, the control system <NUM> may be configured to receive ultrasonic receiver signals from the ultrasonic receiver system corresponding to the ultrasonic waves <NUM> reflected from the target object and to perform an authentication process and/or an imaging process that is based, at least in part, on the ultrasonic receiver signals. Some ultrasonic waves may propagate directly to the ultrasonic receiver system, but these may be distinguished from the reflected ultrasonic waves by selecting a range-gate delay that allows the reflected waves, but not the direct waves, to be sampled.

<FIG> shows another example of an apparatus configured to perform at least some methods disclosed herein. The apparatus <NUM> of <FIG> may be considered another instance of the apparatus <NUM> shown in <FIG>. The apparatus <NUM> of <FIG> is very similar to that shown in <FIG>. However, in this example the nanoparticles <NUM> reside in the waveguide <NUM>. Accordingly, the waveguide <NUM> includes the layer of first metamaterial <NUM>. The light source <NUM> may be configured to emit light that includes a resonant frequency of at least some of the nanoparticles <NUM>. According to some examples, the control system <NUM> may be configured to control the light source <NUM> to cause the nanoparticles to emit ultrasonic waves in the range of <NUM> to <NUM>. According to some implementations, the control system <NUM> may be configured to receive ultrasonic receiver signals from the ultrasonic receiver system corresponding to the ultrasonic waves <NUM> reflected from the target object and to perform an authentication process and/or an imaging process that is based, at least in part, on the ultrasonic receiver signals.

<FIG> is a flow diagram that provides an example of a method that may be performed via some disclosed implementations. The blocks of method <NUM> (and those of at least some other disclosed methods) may, for example, be performed by the apparatus <NUM> shown in one of <FIG> or by a similar apparatus. As with other methods disclosed herein, method <NUM> may include more or fewer blocks than indicated. Moreover, the blocks of methods disclosed herein are not necessarily performed in the order indicated.

In this example, block <NUM> involves controlling a light source system to emit light that causes at least some nanoparticles of a layer of first metamaterial to emit ultrasonic waves. Here, block <NUM> involves controlling the light source system to provide pulses of the light to the layer of first metamaterial, the pulses being provided to cause the nanoparticles to emit ultrasonic waves in the range of <NUM> to <NUM>. For example, block <NUM> may involve the control system <NUM> controlling the light source system <NUM>. Such implementations may cause nanoparticles of the layer of first metamaterial <NUM> to emit ultrasonic waves in a range suitable for an ultrasonic imaging process and/or authentication process.

According to this example, block <NUM> involves receiving first ultrasonic receiver signals from an ultrasonic receiver system corresponding to ultrasonic waves generated by the nanoparticles and reflected from a target object. Block <NUM> may, for example, involve the control system <NUM> receiving the first ultrasonic receiver signals from the receiver system <NUM>. In other examples the first ultrasonic receiver signals may be received from a memory device, such as a buffer. The ultrasonic receiver signals may correspond to ultrasonic waves reflected from a target object proximate the cover layer <NUM>. The target object may, in some examples, be a person's finger, such as a user's finger. However, in other examples the target object may be an artificial finger-like object, which may be referred to herein as a "fake finger.

In this implementation, block <NUM> involves performing an imaging process and/or an authentication process based, at least in part, on the first ultrasonic receiver signals. Data received from the receiver system <NUM> may be referred to herein as "image data," although the image data will generally be received in the form of electrical signals. Accordingly, without additional processing such image data would not necessarily be perceivable by a human being as an image. In some examples, block <NUM> may involve a noise cancellation process and/or other image processing. In some examples, block <NUM> may involve constructing a two-dimensional or a three-dimensional image of at least a portion of the target object.

Some disclosed implementations may include a sensor system that is capable of obtaining image data from the epidermis, such as fingerprint image data, and image data that corresponds to sub-epidermal features. <FIG> shows examples of sub-epidermal features. As used herein, the term "sub-epidermal features" may refer to any of the tissue layers that underlie the epidermis <NUM>, including the dermis, the papillary layer, the reticular layer, the subcutis, etc., and any blood vessels, lymph vessels, sweat glands, hair follicles, hair papilla, fat lobules, etc., that may be present within such tissue layers. Accordingly, sub-epidermal features also may include features not shown in <FIG>, such as muscle tissue, bone material, etc..

If block <NUM> involves performing authentication process based, at least in part, on the first ultrasonic receiver signals, block <NUM> may involve the evaluation of epidermal and/or sub-epidermal features. Some spoofing techniques are based on forming fingerprint-like features on an object, which may be a finger-like object. However, making a finger-like object with detailed sub-epidermal features, muscle tissue features and/or bone tissue features would be challenging and expensive. Making such features accurately correspond with those of an authorized user would be even more challenging. Making such features moveable in a human-like biomimicry manner or in a manner replicating a rightful user raises the bar even higher for spoof fabrication. Because some disclosed implementations involve obtaining attribute information that is based, at least in part, on sub-epidermal features, some such implementations may provide more reliable authentication.

Some implementations may be capable of performing enrollment and authentication processes that are based, at least in part, on sub-epidermal features. Some such processes also may be based on fingerprint image data, or on fingerprint minutiae or fingerprint image features such as keypoints derived from fingerprint image data. The authentication processes may involve spoof detection and/or liveness detection.

In some examples, the user authentication process may involve comparing "attribute information" obtained from received image data, based on the signals from an ultrasonic sensor array, with stored attribute information obtained from image data that has previously been received from an authorized user during an enrollment process. According to some such examples, the attribute information may include information regarding sub-epidermal features, such as information regarding features of the dermis, features of the subcutis, blood vessel features, lymph vessel features, sweat gland features, hair follicle features, hair papilla features and/or fat lobule features, along with minutiae or keypoint information associated with an enrolled fingerprint.

Alternatively, or additionally, in some implementations the attribute information obtained from the received image data and the stored attribute information may include information regarding bone tissue features, muscle tissue features and/or epidermal or sub-epidermal tissue features. For example, according to some implementations, the user authentication process may involve obtaining fingerprint image data and sub-epidermal image data. In such examples, the authentication process may involve evaluating attribute information obtained from the fingerprint image data.

The attribute information obtained from the received image data and the stored attribute information that are compared during an authentication process may include biometric template data corresponding to the received image data and biometric template data corresponding to the stored image data. One well-known type of biometric template data is fingerprint template data, which may indicate types and locations of fingerprint minutia or keypoints. A user authentication process based on attributes of fingerprint image data may involve comparing received and stored fingerprint template data. Such a process may or may not involve directly comparing received and stored fingerprint image data.

Similarly, biometric template data corresponding to sub-epidermal features may include information regarding the attributes of blood vessels, such as information regarding the types and locations of blood vessel features, such as blood vessel size, blood vessel orientation, the locations of blood vessel branch points, etc. Alternatively, or additionally, biometric template data corresponding to sub-epidermal features may include attribute information regarding the types (e.g., the sizes, shapes, orientations, etc.) and locations of features of the dermis, features of the subcutis, lymph vessel features, sweat gland features, hair follicle features, hair papilla features, fat lobule features, muscle tissue and/or bone material.

In some implementations, method <NUM> may involve controlling the light source system to induce photoacoustic emissions from at least a portion of the target object. In some such examples, method <NUM> may involve receiving second ultrasonic receiver signals from the ultrasonic receiver system corresponding to ultrasonic waves produced via the photoacoustic emissions. Such methods may involve performing the imaging process and/or the authentication process of block <NUM> based, at least in part, on the second ultrasonic receiver signals.

According to some implementations, method <NUM> may involve controlling the light source system to illuminate at least a portion of the target object. Some such implementations may involve receiving optical receiver signals from an optical receiver system corresponding to light reflected from the target object. For example, the control system <NUM> may receive optical receiver signals from an optical receiver system component of the receiver system <NUM>. According to some such examples, the control system <NUM> may be configured for performing the imaging process or the authentication process of block <NUM> based, at least in part, on the optical receiver signals.

<FIG> shows an example of a receiver system <NUM> that includes optical receiver portions and ultrasonic receiver portions. In this example, a single layer of the receiver system <NUM> includes optical receiver portions <NUM> and ultrasonic receiver portions <NUM>. As noted above, some films used for screen protection can significantly attenuate ultrasonic signals. Sensor systems that are configured to perform an authentication process and/or an imaging process that is based, at least in part, on optical receiver signals may be able to ameliorate the attenuation of ultrasonic signals caused by screen protection films.

The receiver system <NUM> that is shown in <FIG>, or a similar receiver system <NUM>, may be included in implementations of the apparatus <NUM> that are shown in any one of <FIG> and described above. However, the receiver system <NUM> of <FIG> also included in other implementations of the apparatus <NUM>, such as that shown in <FIG> and described below.

Returning to <FIG>, it may be seen that some implementations of the apparatus <NUM> may include a layer of second metamaterial <NUM>. In some examples, the layer of second metamaterial <NUM> is configured to convert acoustic waves into light. For example, the layer of second metamaterial <NUM> may include plasmonic nanorod metamaterials.

In some examples, the control system <NUM> may be configured to receive optical receiver signals from the optical receiver system and to perform an authentication process and/or an imaging process that is based, at least in part, on the optical receiver signals. At least some of the optical receiver signals may correspond to light emitted by the second metamaterial <NUM>. However, at least some of the optical receiver signals may correspond to light reflected from the target object. According to some examples, the authentication process and/or imaging process also may be based, at least in part, on ultrasonic receiver signals.

According to some examples, the control system <NUM> may be configured to control the light source system <NUM> to induce photoacoustic emissions from a target object in contact with, or proximate, the cover layer <NUM>. In some examples, the second metamaterial <NUM> may convert ultrasonic waves resulting from these photoacoustic emissions into light.

<FIG> shows another example of an apparatus configured to perform at least some methods disclosed herein. Some implementations of the apparatus <NUM> may be considered another instance of the apparatus <NUM> shown in <FIG>. In this example, the cover layer <NUM> includes a layer of metamaterial <NUM>. The layer of metamaterial <NUM> may be considered an instance of the layer of second metamaterial <NUM> that is shown in <FIG>. However, some implementations of the apparatus <NUM> do not include a layer of first metamaterial <NUM> that includes nanoparticles suitable for ultrasonic insonification.

In this example, the light source system <NUM> does not require a waveguide. According to some examples, the light source system <NUM> may include one or more LEDs and/or laser diodes. In some examples, the light source system <NUM> may be a backlight of a display.

According to some examples, the light source system <NUM> may be configured to induce photoacoustic emissions from a target object (such as the target object <NUM>) in contact with, or proximate, the cover layer <NUM>. For example, the control system <NUM> may control the light source system <NUM> to induce photoacoustic emissions in the target object <NUM> via the light <NUM>. In some examples, the metamaterial <NUM> may convert ultrasonic waves <NUM> resulting from these photoacoustic emissions into light <NUM>, some of which may be detected by an optical receiver of the receiver system <NUM>.

In some examples, the control system <NUM> may be configured to receive optical receiver signals from the optical receiver system and to perform an authentication process and/or an imaging process that is based, at least in part, on the optical receiver signals. At least some of the optical receiver signals may correspond to light emitted by the second metamaterial <NUM>. However, at least some of the optical receiver signals may correspond to light reflected from the target object.

In some implementations, the receiver system <NUM> also may include an ultrasonic receiver system. According to some such implementations, the optical receiver system and the ultrasonic receiver system may reside in a single layer of the receiver system <NUM>, e.g., as shown in <FIG>. In some instances, at least some of the ultrasonic waves <NUM> resulting from photoacoustic emissions may be received by the ultrasonic receiver system. According to some examples, the authentication process and/or imaging process also may be based, at least in part, on ultrasonic receiver signals received from the ultrasonic receiver system. Some alternative implementations may include an ultrasonic transmitter.

An implementation such as that shown in <FIG> and described above has potential advantages. For example, an optical sensor will not generally be adversely affected by the screen protection films that can potentially degrade the performance of an ultrasonic sensor. Optical sensors may, for example, be placed under a millimeter of glass, including curved <NUM>. 5D glass, and still accurately detect fingerprints in cases where ultrasonic fingerprint sensors may provide a degraded image quality. Optical sensors may include components such as lenses, enabling a relatively large depth of focus. Accordingly, the additional thickness of a screen protector may be accommodated. Because the speed of light is much greater than the speed of sound, the additional thickness of a screen protector does not impact total time of flight significantly for an optical sensor. By contrast, the additional thickness of a screen protector may cause ultrasonic fingerprint sensors to need recalibration of range gate delay and, in some instances, transmit frequency.

Receiver systems <NUM> that include both optical receivers and ultrasonic receivers also have advantages. The frame rate and spatial resolution of the optical and ultrasonic receivers may be adjusted. For example, the optical receiver may provide relatively higher spatial resolution at a relatively lower frame rate, whereas the ultrasonic transceiver may provide a relatively lower spatial resolution at a relatively higher frame rate. One receiver may have a lower-power imaging mode and the other may have a higher-power imaging mode that is used only when needed or when advantageous. In some instances, the optical receiver and the ultrasonic transceiver may be separately tuned according to the characteristics of each component.

In some implementations, signals from the optical receiver and signals from the ultrasonic receiver may be combined to form a single image, whereas in other examples signals from the optical receiver and signals from the ultrasonic receiver may be used to form separate images. For example, if both optical receiver signals and ultrasonic receiver signals are used for an imaging process and/or for an authentication process, signal processing may involve cross-correlation of the optical receiver signals and ultrasonic receiver signals. In some instances, image data may be obtained either from optical receiver signals or ultrasonic receiver signals, depending on which is currently providing higher-quality signals. For example, if the image quality of the ultrasonic receiver signals is currently degraded (e.g., due to a dry finger), the optical receiver signals may be used for an imaging process and/or for an authentication process.

<FIG> shows an alternative example of a layer of the apparatus of <FIG>. In this example, the cover layer <NUM> includes two types of metamaterial. According to this implementation, vertical nano-rods <NUM> are configured to convert acoustic waves to light waves, whereas horizontal metamaterial layers <NUM> are configured to convert light waves to acoustic waves. The vertical nano-rods <NUM> can help to propagate information from a target object above the cover layer <NUM> to a sensor system below the cover layer <NUM>. In some implementations, the vertical nano-rods <NUM> and/or the horizontal metamaterial layers <NUM> may extend through more of the cover layer <NUM> than indicated in <FIG> and may, in some examples, extend throughout the entire cover layer <NUM>.

<FIG> representationally depicts aspects of a <NUM> x <NUM> pixel array of sensor pixels for an ultrasonic sensor system. Each pixel <NUM> may be, for example, associated with a local region of piezoelectric sensor material (PSM), a peak detection diode (D1) and a readout transistor (M3); many or all of these elements may be formed on or in a substrate to form the pixel circuit <NUM>. In practice, the local region of piezoelectric sensor material of each pixel <NUM> may transduce received ultrasonic energy into electrical charges. The peak detection diode D1 may register the maximum amount of charge detected by the local region of piezoelectric sensor material PSM. Each row of the pixel array <NUM> may then be scanned, e.g., through a row select mechanism, a gate driver, or a shift register, and the readout transistor M3 for each column may be triggered to allow the magnitude of the peak charge for each pixel <NUM> to be read by additional circuitry, e.g., a multiplexer and an A/D converter. The pixel circuit <NUM> may include one or more TFTs to allow gating, addressing, and resetting of the pixel <NUM>.

Each pixel circuit <NUM> may provide information about a small portion of the object detected by the ultrasonic sensor system. While, for convenience of illustration, the example shown in <FIG> is of a relatively coarse resolution, ultrasonic sensors having a resolution on the order of <NUM> pixels per inch or higher may be configured with an appropriately scaled structure. The detection area of the ultrasonic sensor system may be selected depending on the intended object of detection. For example, the detection area may range from about <NUM> x <NUM> for a single finger to about <NUM> inches x <NUM> inches for four fingers. Smaller and larger areas, including square, rectangular and non-rectangular geometries, may be used as appropriate for the target object.

<FIG> shows an example of an exploded view of an ultrasonic sensor system. In this example, the ultrasonic sensor system 900a includes an ultrasonic transmitter <NUM> and an ultrasonic receiver <NUM> under a platen <NUM>. According to some implementations, the ultrasonic receiver <NUM> may be an example of the ultrasonic sensor array <NUM> that is shown in <FIG> and described above. In some implementations, the ultrasonic transmitter <NUM> may be an example of the optional ultrasonic transmitter <NUM> that is shown in <FIG> and described above. The ultrasonic transmitter <NUM> may include a substantially planar piezoelectric transmitter layer <NUM> and may be capable of functioning as a plane wave generator. Ultrasonic waves may be generated by applying a voltage to the piezoelectric layer to expand or contract the layer, depending upon the signal applied, thereby generating a plane wave. In this example, the control system <NUM> may be capable of causing a voltage that may be applied to the planar piezoelectric transmitter layer <NUM> via a first transmitter electrode <NUM> and a second transmitter electrode <NUM>. In this fashion, an ultrasonic wave may be made by changing the thickness of the layer via a piezoelectric effect. This ultrasonic wave may travel towards a finger (or other object to be detected), passing through the platen <NUM>. A portion of the wave not absorbed or transmitted by the object to be detected may be reflected so as to pass back through the platen <NUM> and be received by at least a portion of the ultrasonic receiver <NUM>. The first and second transmitter electrodes <NUM> and <NUM> may be metallized electrodes, for example, metal layers that coat opposing sides of the piezoelectric transmitter layer <NUM>.

The ultrasonic receiver <NUM> may include an array of sensor pixel circuits <NUM> disposed on a substrate <NUM>, which also may be referred to as a backplane, and a piezoelectric receiver layer <NUM>. In some implementations, each sensor pixel circuit <NUM> may include one or more TFT elements, electrical interconnect traces and, in some implementations, one or more additional circuit elements such as diodes, capacitors, and the like. Each sensor pixel circuit <NUM> may be configured to convert an electric charge generated in the piezoelectric receiver layer <NUM> proximate to the pixel circuit into an electrical signal. Each sensor pixel circuit <NUM> may include a pixel input electrode <NUM> that electrically couples the piezoelectric receiver layer <NUM> to the sensor pixel circuit <NUM>.

In the illustrated implementation, a receiver bias electrode <NUM> is disposed on a side of the piezoelectric receiver layer <NUM> proximal to platen <NUM>. The receiver bias electrode <NUM> may be a metallized electrode and may be grounded or biased to control which signals may be passed to the array of sensor pixel circuits <NUM>. Ultrasonic energy that is reflected from the exposed (top) surface of the platen <NUM> may be converted into localized electrical charges by the piezoelectric receiver layer <NUM>. These localized charges may be collected by the pixel input electrodes <NUM> and passed on to the underlying sensor pixel circuits <NUM>. The charges may be amplified or buffered by the sensor pixel circuits <NUM> and provided to the control system <NUM>.

The control system <NUM> may be electrically connected (directly or indirectly) with the first transmitter electrode <NUM> and the second transmitter electrode <NUM>, as well as with the receiver bias electrode <NUM> and the sensor pixel circuits <NUM> on the substrate <NUM>. In some implementations, the control system <NUM> may operate substantially as described above. For example, the control system <NUM> may be capable of processing the amplified signals received from the sensor pixel circuits <NUM>.

The control system <NUM> may be capable of controlling the ultrasonic transmitter <NUM> and/or the ultrasonic receiver <NUM> to obtain ultrasonic image data, e.g., by obtaining fingerprint images. Whether or not the ultrasonic sensor system 900a includes an ultrasonic transmitter <NUM>, the control system <NUM> may be capable of obtaining attribute information from the ultrasonic image data. In some examples, the control system <NUM> may be capable of controlling access to one or more devices based, at least in part, on the attribute information. The ultrasonic sensor system 900a (or an associated device) may include a memory system that includes one or more memory devices. In some implementations, the control system <NUM> may include at least a portion of the memory system. The control system <NUM> may be capable of obtaining attribute information from ultrasonic image data and storing the attribute information in the memory system. In some implementations, the control system <NUM> may be capable of capturing a fingerprint image, obtaining attribute information from the fingerprint image and storing attribute information obtained from the fingerprint image (which may be referred to herein as fingerprint image information) in the memory system. According to some examples, the control system <NUM> may be capable of capturing a fingerprint image, obtaining attribute information from the fingerprint image and storing attribute information obtained from the fingerprint image even while maintaining the ultrasonic transmitter <NUM> in an "off" state.

In some implementations, the control system <NUM> may be capable of operating the ultrasonic sensor system 900a in an ultrasonic imaging mode or a force-sensing mode. In some implementations, the control system may be capable of maintaining the ultrasonic transmitter <NUM> in an "off" state when operating the ultrasonic sensor system in a force-sensing mode. The ultrasonic receiver <NUM> may be capable of functioning as a force sensor when the ultrasonic sensor system 900a is operating in the force-sensing mode. In some implementations, the control system <NUM> may be capable of controlling other devices, such as a display system, a communication system, etc. In some implementations, the control system <NUM> may be capable of operating the ultrasonic sensor system 900a in a capacitive imaging mode.

The platen <NUM> may be any appropriate material that can be acoustically coupled to the receiver, with examples including plastic, ceramic, sapphire, metal and glass. In some implementations, the platen <NUM> may be a cover plate, e.g., a cover glass or a lens glass for a display. Particularly when the ultrasonic transmitter <NUM> is in use, fingerprint detection and imaging can be performed through relatively thick platens if desired, e.g., <NUM> and above. However, for implementations in which the ultrasonic receiver <NUM> is capable of imaging fingerprints in a force detection mode or a capacitance detection mode, a thinner and relatively more compliant platen <NUM> may be desirable. According to some such implementations, the platen <NUM> may include one or more polymers, such as one or more types of parylene, and may be substantially thinner. In some such implementations, the platen <NUM> may be tens of microns thick or even less than <NUM> microns thick.

Examples of piezoelectric materials that may be used to form the piezoelectric receiver layer <NUM> include piezoelectric polymers having appropriate acoustic properties, for example, an acoustic impedance between about <NUM> MRayls and <NUM> MRayls. Specific examples of piezoelectric materials that may be employed include ferroelectric polymers such as polyvinylidene fluoride (PVDF) and polyvinylidene fluoride-trifluoroethylene (PVDF-TrFE) copolymers. Examples of PVDF copolymers include <NUM>:<NUM> (molar percent) PVDF-TrFE, <NUM>:<NUM> PVDF-TrFE, <NUM>:<NUM> PVDF-TrFE, and <NUM>:<NUM> PVDR-TrFE. Other examples of piezoelectric materials that may be employed include polyvinylidene chloride (PVDC) homopolymers and copolymers, polytetrafluoroethylene (PTFE) homopolymers and copolymers, and diisopropylammonium bromide (DIPAB).

The thickness of each of the piezoelectric transmitter layer <NUM> and the piezoelectric receiver layer <NUM> may be selected so as to be suitable for generating and receiving ultrasonic waves. In one example, a PVDF planar piezoelectric transmitter layer <NUM> is approximately <NUM> thick and a PVDF-TrFE receiver layer <NUM> is approximately <NUM> thick. Example frequencies of the ultrasonic waves may be in the range of <NUM> to <NUM>, with wavelengths on the order of a millimeter or less.

<FIG> shows an exploded view of an alternative example of an ultrasonic sensor system. In this example, the piezoelectric receiver layer <NUM> has been formed into discrete elements <NUM>. In the implementation shown in <FIG>, each of the discrete elements <NUM> corresponds with a single pixel input electrode <NUM> and a single sensor pixel circuit <NUM>. However, in alternative implementations of the ultrasonic sensor system 900b, there is not necessarily a one-to-one correspondence between each of the discrete elements <NUM>, a single pixel input electrode <NUM> and a single sensor pixel circuit <NUM>. For example, in some implementations there may be multiple pixel input electrodes <NUM> and sensor pixel circuits <NUM> for a single discrete element <NUM>.

<FIG> and <FIG> show example arrangements of ultrasonic transmitters and receivers in an ultrasonic sensor system, with other arrangements being possible. For example, in some implementations, the ultrasonic transmitter <NUM> may be above the ultrasonic receiver <NUM> and therefore closer to the object(s) to be detected. In some implementations, the ultrasonic transmitter may be included with the ultrasonic sensor array (e.g., a single-layer transmitter and receiver). In some implementations, the ultrasonic sensor system may include an acoustic delay layer. For example, an acoustic delay layer may be incorporated into the ultrasonic sensor system between the ultrasonic transmitter <NUM> and the ultrasonic receiver <NUM>. An acoustic delay layer may be employed to adjust the ultrasonic pulse timing, and at the same time electrically insulate the ultrasonic receiver <NUM> from the ultrasonic transmitter <NUM>. The acoustic delay layer may have a substantially uniform thickness, with the material used for the delay layer and/or the thickness of the delay layer selected to provide a desired delay in the time for reflected ultrasonic energy to reach the ultrasonic receiver <NUM>. In doing so, the range of time during which an energy pulse that carries information about the object by virtue of having been reflected by the object may be made to arrive at the ultrasonic receiver <NUM> during a time range when it is unlikely that energy reflected from other parts of the ultrasonic sensor system is arriving at the ultrasonic receiver <NUM>. In some implementations, the substrate <NUM> and/or the platen <NUM> may serve as an acoustic delay layer.

The hardware and data processing apparatus used to implement the various illustrative logics, logical blocks, modules and circuits described in connection with the aspects disclosed herein may be implemented or performed with a general purpose single- or multi-chip processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, or, any conventional processor, controller, microcontroller, or state machine. A processor also may be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. In some implementations, particular processes and methods may be performed by circuitry that is specific to a given function.

Implementations of the subject matter described in this specification also may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage media for execution by, or to control the operation of, data processing apparatus.

If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium, such as a non-transitory medium. Computer-readable media include both computer storage media and communication media including any medium that may be enabled to transfer a computer program from one place to another. Storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, non-transitory media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection may be properly termed a computer-readable medium.

Various modifications to the implementations described in this disclosure may be readily apparent to those having ordinary skill in the art, and the generic principles defined herein may be applied to other implementations. Thus, the disclosure is not intended to be limited to the implementations shown herein, but is to be accorded the widest scope consistent with the claims. The word "exemplary" is used exclusively herein, if at all, to mean "serving as an example, instance, or illustration.

Certain features that are described in this specification in the context of separate implementations also may be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation also may be implemented in multiple implementations separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Moreover, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products. Additionally, other implementations are within the scope of the following claims.

Claim 1:
An apparatus (<NUM>) for obtaining an identifying image of a target object, comprising:
a cover layer (<NUM>);
a layer (<NUM>) of first metamaterial within or proximate the cover layer, the first metamaterial comprising nanoparticles configured to resonate and create an ultrasonic wave when illuminated by light;
a light source system (<NUM>) configured for providing light to the layer of first metamaterial; and
a receiver system (<NUM>) comprising an ultrasonic receiver system configured to receive ultrasonic waves reflected from the target object in contact with, or proximate, a surface of the cover layer.