Integrated spectrum sensing device for real-finger judgement and sensing method

An integrated spectrum sensing device for real-finger judgement includes a fingerprint sensing array, an optical unit and a signal processing unit. The fingerprint sensing array optically coupled to the optical unit includes multiple spectrum detecting units receiving light from a finger through the optical unit to detect spectrum distributions or variations outputted from the finger to obtain multiple sets of heterogeneous spectrum data. The signal processing unit electrically coupled to the spectrum detecting units performs measurement domain analysis according to the sets of heterogeneous spectrum data to judge whether the finger is real. A sensing method is also disclosed.

BACKGROUND OF THE INVENTION

Field of the Invention

This disclosure relates to an integrated spectrum sensing device for real-finger judgement and a sensing method, and more particularly to an integrated spectrum sensing device for real-finger judgement and a sensing method using the physical phenomenon that a finger deforms after pressing so that light reflected by the finger has variations in the time domain, spatial domain and intensity, and the real finger can be judged in conjunction with spectrum detections.

Description of the Related Art

Today's mobile electronic devices (e.g., mobile phones, tablet computers, notebook computers and the like) are usually equipped with user biometrics recognition systems including different techniques relating to, for example, fingerprint, face, iris and the like, to protect security of personal data. Portable devices applied to mobile phones, smart watches and the like also have the mobile payment function, which further becomes a standard function for the user's biometrics recognition. The portable device, such as the mobile phone and the like, is further developed toward the full-display (or super-narrow border) trend, so that conventional capacitive fingerprint buttons can no longer be used, and new minimized optical imaging devices, some of which are very similar to the conventional camera module having complementary metal-oxide semiconductor (CMOS) image sensor (referred to as CIS) sensing members and an optical lens module, are thus evolved. The minimized optical imaging device is disposed under the display as an under-display device. The image of the object (more particularly the fingerprint) placed above the display can be captured through the partial light-permeable display (more particularly the organic light emitting diode (OLED) display), and this can be called as fingerprint on display (FOD).

The FOD sensing needs to correctly sense the fingerprint, and also to judge whether the finger is real to prevent someone from passing through the authentication using the fake fingerprint or finger. At present, the spoofing technology is getting more and more refined. For example, a mold may be made from a 2D image or by 3D printing, and the mold is filled with various silica gels and pigments to produce the fake finger. Alternatively, another person's fingerprint may be copied into a transparent or skin-color film attached to the finger surface, so that the fake finger attached with the transparent film cannot be easily distinguished. Special attentions needs to be paid on this fake finger recognition technology upon the FOD sensing because the display may shield partial characteristics of the finger to affect the recognition result.

According to the above-mentioned descriptions, the mechanism and method for judging the real finger need to be further improved to prevent the fake finger from passing through the fingerprint recognition.

BRIEF SUMMARY OF THE INVENTION

It is therefore an object of this disclosure to provide an integrated spectrum sensing device for real-finger judgement and a sensing method capable of judging the real finger, wherein the physical phenomenon that the finger deforms after pressing are used in conjunction with different spectrum detections in the time and/or spatial domains to judge whether the finger is real.

Another object of this disclosure is to provide an integrated spectrum sensing device for real-finger judgement and a sensing method capable of judging the real finger, wherein neighboring light sensing cells are used in conjunction with different spectrum separating cells to obtain different intensities, and the real finger is judged according to one or multiple ratios of these intensities.

To achieve the above-identified objects, this disclosure provides an integrated spectrum sensing device including: an optical unit; a fingerprint sensing array being optically coupled to the optical unit and including multiple spectrum detecting units receiving light from a finger through the optical unit to detect spectrum distributions or variations outputted from the finger to obtain multiple sets of heterogeneous spectrum data; and a signal processing unit, which is electrically coupled to the spectrum detecting units, and performs measurement domain analysis according to the sets of heterogeneous spectrum data to judge whether the finger is real.

This disclosure also provides an integrated real-finger spectrum sensing method including steps of: (a) using multiple spectrum detecting units of a fingerprint sensing array to sense spectrum distributions or variations outputted from a finger through an optical unit to obtain multiple sets of heterogeneous spectrum data, wherein the optical unit is optically coupled to the spectrum detecting units; and (b) performing measurement domain analysis according to the sets of heterogeneous spectrum data to judge whether the finger is real.

With the above-mentioned embodiments, it is possible to utilize the physical phenomenon that the finger deforms after pressing in conjunction with spectrum detection to judge whether the finger is real. On the other hand, the real finger can be effectively and correctly judged according to the spectrum verification in the time and/or spatial domains. The hardware, firmware or software can be utilized to perform the spectrum verification in the time and/or spatial domains to avoid the security problem that the fake finger passes the verification. In addition, neighboring light sensing cells are used in conjunction with different spectrum separating cells to obtain different intensities, and whether the finger is real is judged according to one or multiple ratios of these intensities. The simple optical coating treatment for the spectrum separating cells is utilized so that the manufacturing cost needs not to be significantly increased and that the anti-spoofing detection of the finger can be achieved. Also, the problem that the interested party intends to pass fingerprint verification using the fake finger can be effectively solved.

Further scope of the applicability of this disclosure will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of this disclosure, are given by way of illustration only, since various changes and modifications within the spirit and scope of this disclosure will become apparent to those skilled in the art from this detailed description.

DETAILED DESCRIPTION OF THE INVENTION

This disclosure is mainly achieved according to the nonuniform capillary distribution contained in the real finger. Thus, the finger's nonuniform color distribution in the geometric space can be normally seen. In addition, when the finger starts to touch a surface (e.g., the display surface of the mobile phone), under which a FOD sensor is disposed, the capillaries in the finger are pressed to obstruct the blood flow, and the finger's skin color is further changed, so color variations on the time axis are generated. According to one or both of the two phenomena, the property of the real finger can be judged as either true or false, and the fake finger's attack can be thus avoided. On the other hand, the embodiment is achieved mainly by the conventional CIS RGB pixels (the prior art utilizes all or almost all the RGB pixels), which can measure the full-color spectrums. In one optical fingerprint sensing array (the prior art utilizes all white pixels to receive all visible light spectrums or some infrared spectrums), some of the pixels are configured into pseudo CIS RGB pixels distributed in the array to obtain the spectrum variations on the finger surface in the spatial domain and time domain.

FIG. 1is a spectrum chart showing a finger in conditions with and without pressures. It is noted that this experiment may vary from person to person, and only shows its property without quantifying the data. Referring toFIG. 1, the horizontal axis denotes the wavelength, the vertical axis denotes the intensity, and curves CV1and CV2respectively denote the spectrum properties of the finger's certain portion in conditions with and without pressures. For example, according to the intensity level variations for the wavelengths ranging from 380 nm to 580 nm, it is obtained that the finger's spectrums with and without pressures can reach the predetermined levels. Whether the sensed finger is real can be determined according to multiple sets of training data. This is a dynamic sensing method for sensing the pressure variations of the finger's initial, middle and late pressing stages, in which different spectrum intensity information are presented. In one embodiment, however, the optimum solution can be obtained by machine learning or artificial intelligence (AI) algorithms. In one example, the algorithm of analyzing the color spectrum may include: collecting color spectrum information from different materials, fingers and/or light conditions; and then building an analysis system and algorithm to distinguish real fingers from fake fingers. In another example, an AI neural network (NN) scoring may be adopted to: get the image of the fingerprint from the sensor; and test the fingerprint image through the AI model to get the fake/real fingerprint confidence. The algorithm of analyzing the color spectrum may be combined with the AI NN scoring to decide the result of the real/fake finger.

FIG. 2is a schematic view showing the chart ofFIG. 1transformed into the CIE 1931 color space. As shown in the CIE 1931 color space ofFIG. 2, the corner RR is red, the corner GR is green, and the corner BR is blue. The point P1corresponds to the curve CV1and thus the finger with pressure. The point P2corresponds to the curve CV2and thus the finger without pressure. If “L” is used to describe the brightness value (ranging from 0 to 100), “a” is used to describe the green-to-magenta value (ranging from −500 to 500), and “b” is used to describe the blue-to-yellow value (ranging from −200 to 200), then the point P1has (L, a, b) as (39.94, 9.26 and 16.79); and the point P2has (L, a, b) as (27.73, 21.16 and 17.96). It is observed that the sensed spectrum is reddish when the finger is not pressed, and is less reddish when the finger is pressed. The main reason is that when the finger is pressed, the blood flow of the capillary will be blocked, so that the sensed spectrum is less reddish. Thus, the optimum solution serving as the useful database to assist in judging the real finger can be obtained according to a lot of training data at the points P1and P2, the machine learning, and/or the artificial intelligence algorithm.

FIG. 3Ais a schematic partial plane view showing an integrated spectrum sensing device according to a preferred embodiment of this disclosure.FIG. 3Bis a schematic enlarged view showing a second block ofFIG. 3A. Referring toFIGS. 3A and 3B, the integrated spectrum sensing device (ISSD)100includes light sensing cells11and spectrum sensing cells12(similar to the RGB pixels of CIS) arranged in a two-dimensional array. Some light sensing cells11and some spectrum sensing cells12constitute a spectrum cluster block120. Multiple spectrum cluster blocks120constitute a sensing region110, which may be referred to as a region of interest (ROI). Other light sensing cells11may also be arranged at positions between the spectrum cluster blocks120. Although the light sensing cells11and the spectrum sensing cells12are not depicted inFIG. 3A, a fingerprint sensing array10may be defined according to the light sensing cells11and spectrum sensing cells12ofFIG. 3B. In the spectrum cluster block120, heterogeneous spectrum separating cells12a,12band12c(e.g., blue (B), green (G) and red (R) spectrum separating cells) cover some of the spectrum sensing cells12, while no spectrum separating cell covers the other light sensing cells11. Herein, the heterogeneous spectrum separating cells12a,12band12cdenote the narrow spectrums (e.g., conventional RGB colors) that can correspond to the spectrum separating cells12a,12band12c. Thus, three spectrum sensing cells12and the heterogeneous spectrum separating cells12a,12band12cconstitute one spectrum detecting unit13in this example, and the spectrum sensing cells12can detect the full visible light spectrum through the heterogeneous spectrum separating cells12a,12band12cof three primary colors (RGB). The ISSD100can obtain the data of each spectrum cluster and has the hardware for calculating and storing the information of the spectrum detecting unit into the sensor's register or storage area, any internal or external module or the systems random access memory (RAM) or special memory, such as the volatile or non-volatile memory including the one-time programmable (OTP) memory, multiple-time programmable (MTP) memory, electrically-erasable programmable read-only memory (EEPROM) and the like. The information of the spectrum detecting unit13may include the full spectrum (RGB or RGBW) information (W represents the black-and-white). Properly arranging the spectrum detecting units can obtain the finger's correct local information. Of course, each spectrum detecting unit13is constituted by three spectrum sensing cells12and three heterogeneous spectrum separating cells12a,12band12cof three primary-colors in this preferred embodiment. In other embodiments, one spectrum sensing cell12can be used in conjunction with one heterogeneous spectrum separating cell12aor12bor12c, or any two spectrum sensing cells12may be used in conjunction with two heterogeneous spectrum separating cells to constitute one spectrum detecting unit13. Thus, according to another definition, one spectrum sensing cell12and one heterogeneous spectrum separating cell12amay also constitute the spectrum detecting unit. In this case, the heterogeneous spectrum separating cell is only the definition of the term, and may also be referred to as the spectrum separating cell.

FIGS. 3C and 3Dare schematic views showing two examples of spectrum detecting units13working in conjunction with optical units20. Referring toFIG. 3C, the spectrum detecting unit13of the fingerprint sensing array10is constituted by the spectrum sensing cells12and the spectrum separating cells12a,12band12c. The spectrum detecting unit13works in conjunction with one portion (e.g., the conventional lens) or the whole portion (e.g., micro-lenses) of the optical unit20to achieve the spectrum detecting effect. According to another definition, a corresponding portion of the optical unit20may also be incorporated into the fingerprint sensing array, but this disclosure is not limited thereto. The optical unit20may include structures, such as micro-lenses and light apertures, or may also be an optical collimator (an optical channel collimator, such as an optical fiber board, one of other types of structures with the same function, an angle collimator constituted by micro-lenses and light apertures, and the like). Alternatively, the optical unit20may also be a conventional refraction, diffraction or reflection lens set. It is worth noting that the arrangements ofFIGS. 3C and 3Dare only provided to schematically explain the relative relationships between cells/elements without restricting the manufacturing process relationships therebetween or implying a complete direct contact therebetween. InFIG. 3C, the optical unit20is disposed on or above the heterogeneous spectrum separating cells12a/12b/12c, which are disposed between the spectrum sensing cells12and the optical unit20. The spectrum sensing cells12receive the spectrum distributions outputted from the finger through the optical unit20and the spectrum separating cells12a/12b/12c. InFIG. 3D, the optical unit20is disposed between the spectrum sensing cells12and the heterogeneous spectrum separating cells12a,12band12c, the spectrum separating cells12a/12b/12care disposed on or above the optical unit20, and the spectrum sensing cells12receive the finger's spectrums through the spectrum separating cells12a/12b/12cand the optical unit20. According toFIGS. 3B to 3D, it is obtained that the optical unit20can be disposed in or above the spectrum detecting units13as long as the optical unit20can be optically coupled to the spectrum detecting units13to achieve the spectrum detection and optical sensing effects.

FIG. 4is a schematic operational view showing the ISSD100and a host200. Referring toFIGS. 4 and 3A to 3D, this embodiment provides the ISSD100including the optical unit20, the fingerprint sensing array10and a signal processing unit30. The fingerprint sensing array10optically coupled to the optical unit20includes spectrum detecting units13for receiving light from a finger to detect spectrum distributions or variations outputted from the finger through the optical unit20to obtain multiple sets of heterogeneous spectrum data SP. In one non-restrictive example, white light illuminates the finger. The fingerprint sensing array10may further include light sensing cells11. The spectrum detecting units13include spectrum sensing cells12, and neighboring heterogeneous spectrum separating cells12a,12band12c, which cover the spectrum sensing cells12and may be arranged in a diagonal neighboring and staggered manner, so that the spectrum sensing cells12corresponding to the heterogeneous spectrum separating cells12a,12band12csense the finger through the optical unit20and the heterogeneous spectrum separating cells12a,12band12c. In one example, the light sensing cell11and the spectrum sensing cell12have the same structure and property to decrease the manufacturing cost. In one example, the light sensing cell11and the spectrum sensing cell12may be configured to have different properties (e.g., sensitivities). For example, the spectrum sensing cell12has the high sensitivity to provide the more accurate real-finger judgement result. In addition, the ISSD100shown inFIG. 4is electrically coupled (or connected) to the host200having a display50. The host200can be the mobile phone, tablet computer, desktop computer, notebook computer and the like, wherein the associated configuration of the ISSD100and the host200has been described inFIGS. 11 to 13. The display50can display information to interact with the user, and may also provide light to illuminate the finger, which then generates reflected light or transmitted/refracted light (entering the finger and then be transmitted/refracted out). Of course, an additional light source may also be provided to illuminate the finger. The display50may also have the touch function and provide a touch event signal to the ISSD100. Thus, the signal processing unit30is electrically coupled to a central processing unit (CPU)230of the host200. The CPU230is electrically coupled to the display50of the host200, and controls operations of the display50and the signal processing unit30. In this condition, the ISSD100has the independent signal processing unit capable of directly judging the real finger and decreasing the complexity of matching with the host200.

In addition, because the heterogeneous spectrum separating cells12a,12band12care concurrently present in this example, the mixed spectrum of light may be adopted as the light source providing light to the finger, so that the fingerprint sensing array10can obtain multiple sets of heterogeneous spectrum data SP at a time. The sets of heterogeneous spectrum data SP may be stored in a storage140electrically coupled to the fingerprint sensing array10and signal processing unit30. The signal processing unit30is electrically coupled to the spectrum sensing cells12, and performs measurement domain analysis according to the sets of heterogeneous spectrum data SP to judge whether the finger is real. For example, the spectrum variations of the sets of heterogeneous spectrum data SP in one or both of a time domain and a spatial domain are analyzed to judge whether the finger is real. Herein, measurement domain analysis includes time domain analysis and/or spatial domain analysis. The ISSD100is applicable to the under-display or other independent occasion, and is not particularly restricted.

The light sensing cells11not covered by the heterogeneous spectrum separating cells sense the finger's fingerprint through the optical unit20to obtain a fingerprint image. Of course, the signals obtained by the light sensing cells11and the spectrum sensing cells12may also be integrated into the fingerprint image.

In this example, the signal processing unit30may include a sensing region selector31and a dynamic spectrum verifier (DSV)32. The sensing region selector31electrically coupled to the fingerprint sensing array10selects a sensing region110(or ROI) of the fingerprint sensing array10according to a touch event signal to enable the fingerprint sensing array10to generate the sets of heterogeneous spectrum data SP corresponding to the sensing region110. The touch event signal may come from a touch panel (not shown) above the fingerprint sensing array10. In one example, the sensing region selector31provides a spectrum cluster ROI selecting function for selecting the ROI's spectrum cluster information, and then calculating the finger spectrum distributions or variations, and storing the values into the storage140. The DSV32detects the spectrum cluster information in the time domain and calculates the spectrum variation parameters. Thus, the DSV32is in charge of analyzing the spectrum variations of the sets of heterogeneous spectrum data SP in the time domain to judge the real finger. The host200connected to the ISSD100through a wired or a wireless connection interface300can directly acquire the recognition result of the DSV32, and also acquire the heterogeneous spectrum data SP from the storage140through a sensor software development kit (SDK)210executing in the CPU230to perform the further verification the same as or different from that of the DSV32. In another example, the ISSD100may have no DSV32and directly perform the dynamic spectrum verification through the sensor SDK210.

FIG. 5is a flow chart showing one example of an integrated spectrum sensing method. Referring toFIG. 5, a finger touch event is detected, and a sensing region is selected according to the touch event signal in step S01. Next, in step S02, the fingerprint sensing array10obtains the spectrum cluster information of the first image of the sensing region. That is, the fingerprint sensing array10senses the sensing region110at a first time to obtain a first set of heterogeneous spectrum data SP. Then, in step S03, the spectrum cluster information of the first image of the sensing region (i.e., the first set of heterogeneous spectrum data SP) is stored. Next, in step S04, the fingerprint sensing array10obtains the spectrum cluster information of the second image of the sensing region. That is, the fingerprint sensing array10senses the sensing region110at a second time to obtain a second set of heterogeneous spectrum data SP. Then, in step S05, the spectrum cluster information of the second image of the sensing region (i.e., the second set of heterogeneous spectrum data SP) is stored. Then, in step S06, the dynamic spectrum verification is performed. That is, the signal processing unit30analyzes the spectrum variations of the sets of heterogeneous spectrum data SP at the first time and the second time to judge the real finger. Finally, in step S07, the verified result is stored.

Thus, this embodiment provides an integrated real-finger spectrum sensing method, which may be summarized, in conjunction withFIGS. 5 and 4, to include the following steps. Optionally, as shown in step S01, the touch event signal is received, and the sensing region110of the fingerprint sensing array10is selected according to the touch event signal to enable the fingerprint sensing array10to generate the sets of heterogeneous spectrum data SP corresponding to the sensing region110. Then, as shown in steps S02to S05, the spectrum detecting units13of the fingerprint sensing array10(constituted by the heterogeneous spectrum separating cells12a,12band12cand the spectrum sensing cells12) are utilized to sense the spectrum distributions or variations outputted from the finger (i.e., the spectrum distributions or variations of light outputted from the finger and carrying fingerprint information) to obtain multiple sets of heterogeneous spectrum data SP. Next, as shown in steps S06to S07, the spectrum variations of the sets of heterogeneous spectrum data SP in one or both of the time domain and spatial domain are analyzed to judge the real finger, and the verified result is stored. When the spectrum variations in the time domain are sensed, the first image of step S02and the second image of step S04may respectively represent global images at different times. When the spectrum variations in the spatial domain are sensed, the first image of step S02and the second image of step S04may respectively represent local images obtained at different positions in the same time period.

FIG. 6is a schematic view showing the finger touch event. InFIG. 6, the horizontal axis denotes the time, and the vertical axis denotes the level corresponding to the touch force. As shown inFIG. 6, the time point t1indicates that the touch panel senses a finger touch, the time point t4indicates that the finger leaves the touch panel, and the period from the time point t2to the time point t3indicates that the finger is stably touching the touch panel. Thus, the first time may be the starting period (t1to t2) or ending period (t3to t4) of the touch event, and the second time may be the middle period (t2to t3) of the touch event. Thus, a first pressure of the finger directly or indirectly contacting the fingerprint sensing array10at the first time is lower than a second pressure of the finger directly or indirectly contacting the fingerprint sensing array10at the second time. The human finger's touch behavior may function as the training data, and the first time and the second time can be properly selected to provide the basis for the real-finger judgement.

FIG. 7is a flow chart showing another example of the integrated spectrum sensing method. InFIGS. 7 and 6, the spectrum cluster information (heterogeneous spectrum data SP) may also be dynamically captured and stored in the period when the finger touch event is present. After the touch event disappeared, the dynamic spectrum verification can be performed according to the stored spectrum cluster information. Step S01inFIG. 7is the same as that inFIG. 5. After step S01, the spectrum cluster information of the sensing region of the next image frame is obtained (step S12), and then stored (step S13). Next, the dynamic spectrum variations are recorded (step S14), wherein the dynamic spectrum verification can be performed and the verified result can be recorded into the storage. Then, whether the finger leaves is detected (step S15). If the finger does not leave, then steps S12to14are repeated. If the finger has left, then an optional step S16is performed to verify and store the result. This step S16can be performed at the end of the host, and may also be performed by the hardware circuit additionally provided in the ISSD.

It is worth noting that in addition to analyzing the spectrum variations of the sets of heterogeneous spectrum data SP in one or both of the time domain and spatial domain, the signal processing unit30may also perform comparison according to the ratio(s) or mathematical operation result(s) of the neighboring heterogeneous spectrum data SP of the same frame to assist the real-finger judgement. That is, the intensity domain analysis pertaining to the measurement domain analysis may also be performed.

InFIGS. 4 and 1, the signal processing unit30can analyze whether the level variations of the sets of heterogeneous spectrum data SP at the wavelength ranging from 380 nm to 580 nm reach a predetermined level (e.g., the average level difference between the curves CV1and CV2) to perform the real-finger judgement. InFIGS. 4 and 2, the signal processing unit30can analyze whether the position variations of the sets of heterogeneous spectrum data SP in the CIE 1931 color space reach a predetermined offset (e.g., the average offset from points P1to P2, and the offset can be obtained according to the training data) to perform the real-finger judgement.

FIG. 8Ais a distribution chart showing the spatial-domain finger spectrum chart in the CIE 1931 color space.FIG. 8Bis a distribution chart showing multiple points corresponding toFIG. 8Aon the finger. That is, the spectrums outputted from three points P3, P4and P5on the finger F sensed by the ISSD100in the CIE 1931 color space are located at positions ofFIG. 8A. The finger's spectrum variations in the spatial domain are present because the finger's tiny blood vessels are under the nonuniform pressure. At present, no fake finger is found to have such the phenomena. That is, the spectrums correspondingly outputted from two different points on the real finger have difference under the nonuniform pressure, and the difference can be found in the CIE 1931 color space. The spectrums corresponding to two different points of the pressed fake finger has no difference. That is, the positions of the three points P3, P4and P5in the CIE 1931 color space are overlapped for the fake finger. It is worth noting that although three points are explained, examples having two points or more than four points may also be used upon actual implementation.

The real-finger judgement can be performed using the architecture ofFIG. 4in conjunction with the spatial domain variations. Referring toFIGS. 3A, 4, 8A and 8B, the fingerprint sensing array10senses different positions of the sensing region110(e.g., the points P3, P4and P5corresponding toFIG. 8B) to obtain the sets of heterogeneous spectrum data SP (e.g., three sets of heterogeneous spectrum data corresponding to the points P3, P4and P5ofFIG. 8A), and the signal processing unit30analyzes spectrum variations of the sets of heterogeneous spectrum data SP at different positions. At this time, the DSV32can provide the spatial spectrum verifying function. On the other hand, the signal processing unit30analyzes whether the position variations of the sets of heterogeneous spectrum data SP in the CIE 1931 color space reach a predetermined offset to perform the real-finger judgement.

The double verification may also be performed according to the time domain variations and spatial domain variations of the sets of heterogeneous spectrum data SP, so that the judged result becomes more accurate. In this condition, the fingerprint sensing array10senses the sensing region110to obtain multiple sets of heterogeneous spectrum data SP at the first time and the second time, and the signal processing unit30analyzes the spectrum variations of the sets of heterogeneous spectrum data SP at the first time and the second time to perform the real-finger judgement. In addition, the signal processing unit30analyzes the spectrum variations of the sets of heterogeneous spectrum data SP corresponding to one or both of the first time and the second time at different positions to perform the further real-finger judgement.

FIG. 9is one block diagram showing a hardware device for implementing the spectrum variation judgement in the spatial domain. InFIG. 9(similar toFIG. 4), the fingerprint sensing array10of the ISSD100senses different positions of the sensing region to obtain multiple sets of heterogeneous spectrum data SP, which are transmitted to the host200through a data buffer interface150and the connection interface300. The sensor SDK210executing in the CPU230of the host200works in conjunction with a spatial spectrum verifier (SSV)220and the artificial intelligent engine for fingerprint to perform the spatial spectrum verifying function and the other functions mentioned hereinabove.

FIG. 10is another block diagram showing the hardware device for implementing the spectrum variation judgement in the spatial domain. InFIG. 10, this example is made by combining the configurations ofFIGS. 4 and 9, wherein the time domain processing of dynamic spectrum verification can be performed in the DSV32of the ISSD100, and the spatial spectrum verification processing can be performed in the SSV220of the host200. However, this is only one of examples without limiting this disclosure thereto. In addition, although the CIE 1931 color space functions as a descriptive example, the concept of this disclosure may also be applied to the existing or future color space to judge whether the finger is real.

With the above-mentioned embodiments, it is possible to utilize the physical phenomenon that the finger deforms after pressing in conjunction with spectrum detection to judge whether the finger is real. On the other hand, the real finger can be effectively and correctly judged according to the spectrum verification in the time domain and/or spatial domain. The hardware, firmware or software can be utilized to perform the spectrum verification in the time domain and/or spatial domain to avoid the security problem that the fake finger passes the verification.

The above-mentioned judging method can achieve the effect of judging the real finger. That is, the to-be-detected finger is judged as either a real finger or not (the judged result is “either true or false”). As long as the real-finger conditions cannot be satisfied, the finger is judged as fake. There is no mis-judgement for some realistic fingers having properties very similar to the real finger.

FIGS. 11 to 13are schematically cross-sectional views showing partial systems of three examples of ISSDs100of the preferred embodiment of this disclosure. InFIG. 11, an ISSD100is disposed under a display50of a mobile phone, for example (such the ISSD100is also applicable toFIGS. 3A to 3D), senses the fingerprint of the finger F, and includes a fingerprint sensing array10, an optical unit20and a spectrum separating module40(constituted by heterogeneous spectrum separating cells12a,12band12c). In one example, the display50provides the illumination light to illuminate the finger F, so that the finger F generates the light (e.g., reflected light) to be received by the spectrum detecting units13. A sensing substrate15of the fingerprint sensing array10may be a semiconductor substrate forming a complementary metal oxide semiconductor (CMOS) optical image sensor, may also be another type of optical image sensor, such as an optical image sensor having thin-film transistors (TFTs) formed on a glass substrate, a sensor formed on a polymeric film, or any optical image sensor formed on any suitable substrate. The sensing substrate15has light sensing cells11and spectrum sensing cells12arranged in a two-dimensional array. In one example, the display50may provide the light to the finger F. In another example, a light emitting device (not shown) may be additionally provided to illuminate the finger.

The optical unit20is disposed above the sensing substrate15. The spectrum separating module40works in conjunction with the optical unit20to separate the spectrums of the light rays L coming from the finger F and partially representative of the fingerprint of the finger F, and transmits the separated spectrums to the spectrum sensing cells12, so that the spectrum sensing cells12obtain intensities, according to which the finger F may be judged as real or not (e.g., the intensity domain analysis pertaining to the measurement domain analysis). In this example, the finger F is judged as real according to one or multiple ratios of the intensities. In other examples, the finger F may be judged as real according to one or multiple differences or products of the mathematical combinations of the intensities, which may be used in statistics. If this technology is applied to the integrated real-finger spectrum sensing method, then the mathematical combinations (one or multiple ratios) of the intensities of the sets of heterogeneous spectrum data SP can be analyzed to judge whether the finger is real (the one or multiple ratios may be compared with the pre-detected data of the database to perform the judgement).

InFIG. 11, the spectrum separating module40includes heterogeneous spectrum separating cells12a,12band12c(or anti-spoofing spectrum separating cells) arranged in a neighboring manner (no other spectrum separating cell is present between neighboring two cells. The neighboring cells may be disposed side by side, or corner by corner), work in conjunction with the optical unit20to separate the spectrums of the light rays L including a first light ray L1, a second light ray L2and a third light ray L3, and transmit the separated spectrums to three spectrum sensing cells12, which obtain a first intensity E1, a second intensity E2and a third intensity E3of the intensities. Thus, the spectrum properties of the reflected light of the finger F can be obtained to judge the finger F as real according to the mathematical combinations (e.g., ratios (E1/E2) and (E2/E3)) of arbitrary two of the first to third intensities E1, E2and E3. At this time, the first to third light rays L1to L3sensed by the three spectrum sensing cells12represent the virtual identical portion VIP of the finger F. In one example, the pitch between the ridges of the fingerprint ranges from about 400 to 500 μm, and the pitch between the spectrum sensing cells12ranges from about 10 to 20 μm, so that three neighboring spectrum sensing cells12actually need to measure the same portion of the fingerprint, the spectrum of this portion is referred to as that of the virtual identical portion or position, and the real physical meanings can be obtained. Because the VIP represents substantially the same depth information of the same point, the intensity cannot drift. This is because that the distance from the finger F to the three spectrum sensing cells12or the optical unit20is about 1,000 μm. When the difference between the transversal dimension and longitudinal dimension is relatively large, the three spectrum sensing cells12see almost the same portion. In addition, the optical unit20further transmits the normal light ray L6coming from the finger F and partially representing the fingerprint to the corresponding light sensing cells11of the sensing substrate15to obtain the fingerprint image partially representing the fingerprint. The signals received by the spectrum sensing cell12can be used to judge the real finger, and may also be further compensated to obtain signals of several pixels of the fingerprint image.

Thus, the heterogeneous spectrum separating cells12a,12band12cadjacently cover the spectrum sensing cells12, so that the spectrum sensing cells12corresponding to the heterogeneous spectrum separating cells12a,12band12csense the spectrum distributions or variations outputted from the finger through the optical unit20and the heterogeneous spectrum separating cells12a,12band12cto obtain multiple sets of heterogeneous spectrum data SP.

In the example ofFIG. 11, the heterogeneous spectrum separating cell12a/12b/12cis the red/green/blue spectrum separating cell (the red spectrum is separated and passes through the heterogeneous spectrum separating cell12aafter the light ray impinges on the heterogeneous spectrum separating cell12a), but this disclosure is not restricted thereto as long as the heterogeneous spectrum separating cells12a,12band12callow light components with different wavelengths or spectrums to pass.

The optical unit20is also referred to as an optical engine disposed above the sensing substrate15. The optical engine may be a lens-type optical engine (e.g., having two, three or multiple pieces of lenses), or an optical collimator engine (including a micro-lens array or an optical fiber (without a micro-lens)). The optical engine (e.g., the lens-type) may be separated from the sensing substrate15by a distance. Alternatively, the optical engine (e.g., the micro-lens array or the optical fiber) may be bonded to the sensing substrate15by the bonding method integrated with the manufacturing process, or by way of adhering and assembling. In addition, the optical unit20is disposed between the spectrum separating module40and the finger F. The heterogeneous spectrum separating cells12a,12band12care respectively disposed on the three spectrum sensing cells12. For example, the heterogeneous spectrum separating cells12a,12band12care the optical films or bonding films corresponding to different wavelengths and being disposed on the three spectrum sensing cells12or the optical unit20. Actually, two or more than two spectrums may function as separated or extracted spectrums of the anti-spoofing spectrum separating cells, which may fall within the visible light band (400 to 700 nm) or near-infrared light band (700 to 1,000 nm). If the bonding film is used, then the special alignment may not be needed. When the image processing is performed, an image without a finger can be captured, or a reference pattern can be captured, and the positions of the anti-spoofing spectrum separating cells and the corresponding light sensing cells may be found to serve as the reference for the subsequent fingerprint image capturing. It is worth noting that although three heterogeneous spectrum separating cells12a,12band12care explained in the example, it is easily understood, from this disclosure, that two heterogeneous spectrum separating cells may also be used to achieve the effect of this embodiment.

The above-mentioned structure can be achieved by at least two neighboring pixels having at least two different spectrum separating configurations, wherein the first pixel receives the finger's reflected light through the first spectrum separating cell to obtain E1, the second pixel receives the finger's reflected light through the second spectrum separating cell to obtain E2, and the spectrum properties of the finger's reflected light can be obtained according to the ratio (E1/E2) and function as the basis for fake finger detection. Examples will be explained later.

It is worth noting that the ISSD100may further include the signal processing unit30, which is electrically coupled to the light sensing cells11and spectrum sensing cells12(or electrically coupled to the spectrum detecting unit13), and judges the finger F as real according to the intensities. The signal processing unit30may be an independent processor, or the CPU of an electronic device (e.g., mobile phone) working therewith. Because the light sensing cells11obtain the fingerprint image partially representing the fingerprint, the signal processing unit30processes the fingerprint image captured by the light sensing cells11only when judging the finger F as real. The heterogeneous spectrum separating cells12a,12band12care disposed between the spectrum sensing cells12and the optical unit20.

To summarize the embodiments ofFIGS. 2 to 11, it is obtained that the signal processing unit30can perform measurement domain analysis according to multiple sets of heterogeneous spectrum data SP to judge whether the finger is real or not by performing, for example, one or multiple ones of the following mechanisms: (a) analyzing multiple sets of heterogeneous spectrum data SP in the spectrum variations in a time domain; (b) analyzing multiple sets of heterogeneous spectrum data SP in the spectrum variations in a spatial domain; and (c) analyzing relationships between multiple intensities of the sets of heterogeneous spectrum data SP (the intensity domain analysis of the sets of heterogeneous spectrum data SP). Only one of the mechanisms (a), (b) and (c) may be used to speed up the real-finger judgement; or two of the mechanisms (a), (b) and (c) can be used to possess the judgement speed and accuracy; or all of the mechanisms (a), (b) and (c) can be used to increase the judgement accuracy.

The example ofFIG. 12is similar to that ofFIG. 11except for the difference that the heterogeneous spectrum separating cells12a,12band12care disposed on a transparent layer35above the sensing substrate15to facilitate the mass production and the cost reduction. It is worth noting that the anti-spoofing spectrum separating cells may be disposed on the upper surface or lower surface of the transparent layer35, and the transparent layer35may be the light-permeable glass or another light-permeable medium.

A top-side spectrum separating configuration is provided inFIG. 13. Thus, the heterogeneous spectrum separating cells12a,12band12cof the spectrum separating module40are disposed between the optical unit20and the finger F. That is, the anti-spoofing spectrum separating cells are disposed on the upper surface of the optical unit20(e.g., the upper/lower surface of the micro-lens or the upper surface of the optical fiber collimator). Thus, the manufacturing cost of the optical unit20containing the spectrum separating cells12a,12band12ccan be decreased.

FIGS. 14 and 15are schematic views showing two examples of anti-spoofing spectrum separating cells of the preferred embodiment of this disclosure. InFIG. 14, the heterogeneous spectrum separating cells12a,12band/or12cmay be a surface plasmonic spectrum separator. InFIG. 15, the heterogeneous spectrum separating cells12a,12band/or12cmay be a diffraction grating type spectrum separator.

FIGS. 16 to 20are top views showing examples of blocks of anti-spoofing spectrum separating cells. InFIG. 16, the spectrum separating module40includes: a first block40M having one or multiple ratios (central ratios) of these ratios; and one or multiple second blocks40P being disposed beside the first block40M and having one or multiple ratios (peripheral ratios) of these ratios. Thus, the finger F may be judged as real according to the one or multiple central ratios and the one or multiple peripheral ratios. The neighboring heterogeneous spectrum separating cells12a,12band12cof the spectrum separating module40allowing different wavelengths of light components to pass may be closely staggered. The first block40M includes a middle anti-spoofing spectrum separating cell40A and peripheral anti-spoofing spectrum separating cells40F (40B-40E) disposed around the middle anti-spoofing spectrum separating cell40A. In this example, four peripheral anti-spoofing spectrum separating cells40F (40B-40E) are disposed at four corners of the middle anti-spoofing spectrum separating cell40A. The light wavelengths separated by the middle anti-spoofing spectrum separating cell40A differ from the light wavelengths separated by the peripheral anti-spoofing spectrum separating cells40F (40B-40E). In this example, the peripheral anti-spoofing spectrum separating cells40F (40B-40E) include anti-spoofing spectrum separating cells40B (blue spectrum),40C (red spectrum),40D (blue spectrum) and40E (red spectrum), and the middle anti-spoofing spectrum separating cell40A is a green spectrum separating cell.

That is, the four peripheral anti-spoofing spectrum separating cells40F (40B-40E) include first peripheral anti-spoofing spectrum separating cells40B and40D and second peripheral anti-spoofing spectrum separating cells40C and40E. The first peripheral anti-spoofing spectrum separating cells40B and40D neighbor upon two diagonal corners of the middle anti-spoofing spectrum separating cell40A and generate the same first light wavelengths. The second peripheral anti-spoofing spectrum separating cells40C and40E neighbor upon the other two diagonal corner of the middle anti-spoofing spectrum separating cell40A and generate the same second light wavelengths. However, the light wavelengths separated by the first peripheral anti-spoofing spectrum separating cells40B and40D differ from the light wavelengths separated by the second peripheral anti-spoofing spectrum separating cells40C and40E.

InFIGS. 17 and 18, multiple ones of the spectrum sensing cells12correspondingly receive the light from one anti-spoofing spectrum separating cell of the spectrum separating module40. InFIG. 17, the one anti-spoofing spectrum separating cell of the spectrum separating module40covers four spectrum sensing cells12. InFIG. 18, one anti-spoofing spectrum separating cell of the spectrum separating module40covers two spectrum sensing cells12. Thus, the light sensing cells can obtain more information to increase the real-finger judging accuracy.

InFIG. 19, the diagonally neighboring heterogeneous spectrum separating cells12aand12bof the spectrum separating module40allowing different wavelengths of light components to pass are arranged in a direct neighboring manner. InFIG. 19, the peripheral anti-spoofing spectrum separating cell40F may be regarded as an upper square region, or a region constituted by squares at two diagonal positions (e.g., the dashed line frame at the lower right corner). InFIG. 20, the neighboring heterogeneous spectrum separating cells12aand12bof the spectrum separating module40allowing different wavelengths of light components to pass are arranged in a direct transversal or longitudinal neighboring manner. InFIGS. 19 and 20, at least two neighboring pixels have at least two colors of spectrum separating configurations arranged in a closely spaced manner to assist the image interpolation upon image processing and to obtain the intensity of the reflected light of fingerprint's ridge or valley of the partial uniform region.

FIG. 21is a schematic view showing design parameters of the fingerprint sensor of the preferred embodiment of this disclosure. In one single region ofFIG. 21, the neighboring heterogeneous spectrum separating cells12aand12bof the spectrum separating module40allowing different wavelengths of light components to pass cover the actual fingerprint image having the range, which is associated with the distance between the neighboring ridge and valley of the finger F, and preferably smaller than one half of the minimum cycle of the fingerprint (2*HP), where HP denotes an half cycle, and 2*HP is equivalent to the distance between the neighboring ridges or valleys. So, the half cycle HP is equivalent to the distance between the neighboring ridge and valley, and ranges from 200 to 400 microns (μm). In one example, the neighboring heterogeneous spectrum separating cells12aand12bof the spectrum separating module40allowing different wavelengths of light components to pass cover the actual fingerprint image having a range smaller than 100 microns to obtain the more reasonable and easily distinguished ratio (E1/E2) or ratio (E2/E1). The range of the covered actual fingerprint image represents the range, in which the light sensing cells11can receive the light from the virtual identical portion VIP (FIG. 11) through the neighboring anti-spoofing spectrum separating cells.

FIG. 22shows a pixel response graph of the fingerprint sensor. InFIGS. 22 and 16, response curves CA, CD and CE respectively correspond to the middle anti-spoofing spectrum separating cell40A (e.g., green spectrum separating cell), the first peripheral anti-spoofing spectrum separating cell40D (e.g., blue spectrum separating cell) and the second peripheral anti-spoofing spectrum separating cell40E (e.g., red spectrum separating cell), wherein the response curve CA has the highest peak intensity of about 36%; the response curve CD has the lowest peak intensity of about 14%; and the response curve CE has the moderate peak intensity of about 28%. At this time, the signal processing unit30can increase the sensitivity of one of the light sensing cells11(e.g., the light sensing cell corresponding to the first peripheral anti-spoofing spectrum separating cell40D) to compensate one of the intensities. Alternatively, the signal processing unit30can adjust the integration time or gain of one or multiple ones of the light sensing cells11to compensate one of the intensities.

FIGS. 23 and 24are schematic views showing two applications of fingerprint sensors. InFIG. 23, when the integrated spectrum sensing device is a TFT sensor, an ISSD100′ similar to the ISSD100and alternately integrated with display pixels (not shown) may become a liquid crystal display (LCD), an OLED display, a micro LED display or any other existing or future display having an in-cell sensor or a TFT sensor manufactured using the TFT manufacturing process. At this time, the sensing substrate15is one of two opposite light-permeable substrates51and52of the display50(the lower light-permeable substrate51in this figure, wherein the sensing substrate15may also be regarded as one portion of the light-permeable substrate51). The material layer between the two light-permeable substrates51and52may be that contained in the OLED display or LCD. AlthoughFIG. 23is explained using the partial range of the ISSD100′ as an example, this disclosure is not restricted thereto. The ISSD100′ may also extend to cover the full range of the entire display50, and become a full-screen fingerprint sensor. InFIG. 24, the ISSD100is an independent sensor, such as a TFT or CMOS sensor, disposed under the light-permeable substrate51. It is worth noting that the display50inFIGS. 11 to 13may be the LCD, OLED display, micro LED display or any other existing or future display for providing light to illuminate the finger.

One example will be illustrated to prove that this disclosure can be implemented.FIG. 25is a schematic view showing the configuration of the anti-spoofing spectrum separating cell. InFIG. 25, the neighboring red, green and blue anti-spoofing spectrum separating cells45R,45G and45B constitute one set. The signal values of all light sensing cells11are listed in Tables 1 to 3, wherein the upper left corner is defined as the first column and first row, and the lower right corner is defined as the tenth column and tenth row. Table 1 lists the values corresponding to the real finger F1, and Tables 2 and 3 list the values corresponding to the two fake fingers F2and F3.

As listed in Table 1, it is obtained that the averages (B, G, R) respectively corresponding to the blue, green and red anti-spoofing spectrum separating cells45B,45G and45R can be written as B=11.5; G=87; and R=78. Thus, it is obtained that R/G=78/87=0.9; and B/G=11.5/87=0.13, where the red component C1relative to green can be written as C1=α1*R/G=90, and the blue component C2relative to green may be written as C2=α2*B/G=13, where α1 and α2 depend on different light sources. In this example, α1=100 and α2=100.

As listed in Table 2, it is obtained that the averages (B, G, R) respectively corresponding to the blue, green and red anti-spoofing spectrum separating cells45B,45G and45R can be written as B=89; G=257; and R=141. Thus, it is obtained that R/G=141/257=0.54; and B/G=89/257=0.35, where the red component C1relative to green can be written as C1=α1*R/G=54, and the blue component C2relative to green may be written as C2=α2*B/G=35, where α1 and α2 depend on different light sources. In this example, α1=100 and α2=100.

As listed in Table 3, it is obtained that the averages (B, G, R) respectively corresponding to the blue, green and red anti-spoofing spectrum separating cells45B,45G and45R can be written as B=72; G=350; and R=162. Thus, it is obtained that R/G=162/350=0.462; and B/G=72/350=0.2, where the red component C1relative to green can be written as C1=α1*R/G=46, and the blue component C2relative to green may be written as C2=α2*B/G=20, where α1 and α2 depend on different light sources. In this example, α1=100 and α2=100.

FIG. 26is a relation chart showing C1and C2of sensing results of three fingers. InFIG. 26, the distribution range of the real finger F1falls at the lower right corner (i.e., C1is high and C2is low). Because the real finger F1has veins, the red component relative to green is high, and the blue component relative to green is low. The fake fingers F2and F3have no vein, so C1is low and C2is high. Thus, the real finger can be judged according to the ratio(s) of the intensities. Of course, the associated database can be created according to several states and test data of the fingers, wherein comparisons of C1and C2with the database can be made or even in conjunction with the artificial intelligence training to perform the real-finger judgement.

FIG. 27is a schematic block diagram showing another example of the integrated spectrum sensing device. InFIG. 27, the ISSD100is a mobile device (e.g., mobile phone, tablet computer or the like). Thus, the signal processing unit30functions as a CPU of the mobile device, and performs control and signal processing on the fingerprint sensing array10and display50. In this condition, the cost of the overall system can be decreased using the CPU of the mobile device with the powerful calculation function to perform signal processing and judgement.

With the above-mentioned integrated spectrum sensing device of the embodiments, neighboring light sensing cells are used in conjunction with different spectrum separating cells to obtain different intensities, and whether the finger is real is judged according to one or multiple ratios of these intensities. The simple optical coating treatment for the spectrum separating cells is utilized so that the manufacturing cost needs not to be significantly increased and that the anti-spoofing detection of the finger can be achieved. Also, the problem that the interested party intends to pass fingerprint verification using the fake finger can be effectively solved.

It is worth noting that all the above embodiments can be combined, replaced or modified interactively as appropriate to provide the real-finger judgement accuracy, speed and stability.

While this disclosure has been described by way of examples and in terms of preferred embodiments, it is to be understood that this disclosure is not limited thereto. To the contrary, it is intended to cover various modifications. Therefore, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications.