Image sensing method

An image sensing method, applicable to the anti-spoofing recognition of under screen optical fingerprint sensing, is provided, including: dividing the image sensor into sensing blocks, dividing the display area of the display device correspondingly according to the sensing area, and the display area including the light-emitting area; defining the luminous color of each display area and the color coordinate value of each luminous color; each sensing block sensing the light intensity of the image reflected to the sensing block from the display block emitting the light onto the reference object and the test object to be measured; calculating the anti-spoofing reference color information of the reference object and registering in the system; when sensing the fingerprint image, first obtaining the light intensity of each block, then calculating the color information of the test object; and finally, comparing the color information with the registered anti-spoofing reference color information.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to an image sensing method, and more particularly to an image sensing method applicable to fingerprint image recognition.

2. The Prior Arts

Since the development of fingerprint recognition technology, it has become the standard part of most smart phones. The advantage of fingerprint recognition is that the fingerprint is a unique feature of the human body, and the complexity of the fingerprint is sufficient for authentication. Moreover, when the reliability needs to be increased, it is only necessary to register more fingerprints and identify more fingers, say, up to ten fingerprints, and each fingerprint is unique. Furthermore, scanning fingerprints nowadays is very fast and easy to use, which is one of the main reasons that fingerprint recognition technology can dominate the market share.

Under-screen optical fingerprint recognition is currently widely used in AMOLED displays, but the under-screen optical fingerprint recognition may also be applied to LCD displays with LED backlights, and so on in the future because various types of panels are available for projecting the light to the position where the finger touches the panel, and the reflected light is received by the sensor for fingerprint recognition, which greatly enhances the application range and applicability of under-screen optical fingerprint recognition.

However, fingerprint recognition is not absolutely safe. People leave their fingerprints in many publicly visited places every day. It is really easy to obtain a copy of fingerprint. Once the fingerprint is obtained, personal devices and information security may be breached. At the same time, compared to using a password, the password protection can be reset even if the password is cracked, but the fingerprint cannot be reset.

At present, many anti-spoofing methods have been proposed, such as, using infrared light sources to detect finger capillaries or using the three-dimensional characteristics of the finger to perform peripheral calculation, or fabricating color filter layers or polarized light layer on the pixels of optical sensors for comparison. However, in many of the above methods, the addition of the infrared light source or increase the optical film layer results in an increase in the production cost. Therefore, how to achieve high-performance anti-spoofing fingerprint recognition without increasing the production cost of the fingerprint recognition module is an important issue.

Therefore, to address the above-mentioned deficiencies, the inventors disclosed the present invention.

SUMMARY OF THE INVENTION

A primary objective of the present invention is to provide an image sensing method, which confirms whether the fingerprint comes from a real finger of a holder of the registered fingerprint through the specific skin color expressed by the absorption and reflection of specific wavelengths of light by the skin. Thus, according to the image sensing method of the present invention, it is possible to provide high-efficiency anti-spoofing recognition without increasing the production cost of the anti-spoofing recognition module.

To achieve the foregoing objective, the present invention provides an image sensing method, which divides the image sensor into one or more sensing blocks, and divides the display device into correspondingly one or more display blocks according to the sensing blocks, so that each sensing block respectively senses the image light intensity of the image reflected to the sensing block generated by a different display block irradiates light onto a reference object and a test object, and defines color coordinate values of each luminous color of the display block, the image light intensity and the color coordinate values are combined to obtain the anti-spoofing reference color information of the reference object and the color information of the test object, and the color information is compared with the anti-spoofing reference color information. For anti-spoofing recognition of fingerprint recognition applied to the under screen of a display device, the display device has a display panel and an image sensor, the display panel has a display area, the image sensor correspondingly overlaps the display area. The image sensing method includes the following steps: dividing step: dividing the image sensor into a plurality of sensing blocks; corresponding step: correspondingly dividing the display area into a plurality of display blocks according to a plurality of sensing blocks, the display block comprising at least one light-emitting block, the area of the light-emitting block being smaller than the area of the sensing block; defining step: defining a luminous color of each display block and a color coordinate value of the luminous color; reference sensing step: the plurality of display blocks emitting light at the same time, and then each of the sensing blocks sensing a reference image light intensity of the light reflected to the sensing blocks from each of the display blocks emitting light unto a reference object and reflected to the sensing blocks; reference color calculation step: generating an anti-spoofing reference color information from the color coordinate value and the reference image light intensity; registration step: registering the anti-spoofing reference color information into a system, and using the anti-spoofing reference color information to generate an anti-spoofing reference interval; sensing step: the plurality of display blocks emitting light at the same time, and the sensing blocks obtaining an image light intensity of an image reflected to the sensing blocks from the display blocks emitting light unto and reflected by a test object; calculation step: generating a color information from the color coordinate value and the image light intensity; and comparison step: determining the color information and the anti-spoofing reference color information being consistent when the color information being within the anti-spoofing reference interval, otherwise determining the color information being inconsistent with the anti-spoofing reference color information.

Preferably, according to the image sensing method of the present invention, the light-emitting block comprises a first light-emitting block and a second light-emitting block, and the luminous color of the first light-emitting block is selected from one of red light, green light, and blue light, and the luminous color of the second light-emitting block is selected from another of red light, green light, and blue light.

Preferably, according to the image sensing method of the present invention, the display area further comprises a third light-emitting block, and the luminous color of the third light-emitting block is selected from one of the red light, green light, and blue light, and the luminous color of the third light-emitting block is different from luminous colors of the first light-emitting block and the second light-emitting block.

Preferably, according to the image sensing method of the present invention, the registration step is a one-time or multiple-time registration to the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the anti-spoofing reference color is an average value of the anti-spoofing reference color information registered in the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the anti-spoofing reference interval is manually set, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the anti-spoofing reference interval is the difference between the maximum and minimum values and the average value of the anti-spoofing reference color information registered by the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the display panel is one of an organic light-emitting diode (OLED) display panel and a micro light-emitting diode display panel, but the present invention is not limited hereto.

Moreover, to achieve the foregoing objective, the present invention provides yet another image sensing method, applicable to anti-spoofing recognition of fingerprint recognition of the under screen of a display device, the display device having a display panel and an image sensor, the display panel having a display area, the image sensor correspondingly overlapping the display area; the image sensing method comprising the following steps: dividing step: dividing the image sensor into one or more sensing blocks; corresponding step: diving the display area into a plurality of display blocks corresponding to the sensing blocks; definition step: defining a plurality of luminous colors and the color coordinate values of the luminous colors; reference sensing step: the display blocks sequentially emitting at least two of red light, green light, and blue light, and each of the sensing blocks sensing a reference light intensity reflected to the plurality of the sensing blocks from each of the display blocks emitting sequentially unto a reference object and reflected to the sensing blocks; reference color calculation step: generating an anti-spoofing reference color information from the color coordinate value and the reference image light intensity; registration step: registering the anti-spoofing reference color information into a system, and using the anti-spoofing reference color information to generate an anti-spoofing reference interval; sensing step: the plurality of display blocks sequentially emitting at least two of red light, green light and blue light, and the sensing blocks sensing an image light intensity of an image reflected to the sensing blocks from the display blocks emitting light unto and reflected by a test object; calculation step: generating a color information from the color coordinate value and the image light intensity; and comparison step: determining the color information and the anti-spoofing reference color information being consistent when the color information being within the anti-spoofing reference interval, otherwise determining the color information being inconsistent with the anti-spoofing reference color information.

Preferably, according to the image sensing method of the present invention, the registration step is a one-time or multiple-time registration to the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the anti-spoofing reference color is an average value of the anti-spoofing reference color information registered in the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the anti-spoofing reference interval is manually set, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the absolute value of the anti-spoofing reference interval is the average value of the maximum value minus the minimum value of the anti-spoofing reference color information registered by the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the absolute value of the anti-spoofing reference interval is one of 1 time, 2 times, and 3 times of the standard deviation of the anti-spoofing reference color information registered by the system, but the present invention is not limited hereto.

Preferably, according to the image sensing method of the present invention, the display panel is one of an organic electroluminescence display panel, an organic light-emitting diode display panel, and a micro light-emitting diode display panel, but the present invention is not limited hereto.

In summary, the image sensing method and the display device provided by the present invention mainly consist of dividing the image sensor into one or more sensing blocks, and correspondingly dividing the display device into one or more display blocks according to the sensing blocks, so that each sensing block respectively senses the image light intensity of the image reflected to the sensing block from a different display block emitting light unto and reflected by the reference object and the test object, and defines each the color coordinate value of the luminous color of the display block, combine the image light intensity and the color coordinate value to obtain the anti-spoofing reference color information of the reference object and the color information of the test object, and compare the color information with the anti-spoofing reference color information. As such, according to the image sensing method of the present invention, it is possible to successfully provide high-efficiency anti-spoofing recognition without increasing the production cost of the anti-spoofing recognition module.

To enable those skilled in the art to understand the purpose, features, and effects of the present invention, the following specific embodiments and accompanying drawings are used to explain the present invention in detail as follows.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG.1is a schematic view of a display device according to the present invention. As shown inFIG.1, the display device1according to the present invention includes: a display panel100, a display area11, a plurality of pixel units, and an image sensor200.

More specifically, referring toFIG.1andFIG.3, the display area11of the display panel100according to the present invention includes a plurality of unit pixels12, and each unit pixel12includes a red light unit pixel121, a green light unit pixel122and a blue light unit pixel123, wherein the red light unit pixel121emits red light R, the green light unit pixel122emits green light G, and the blue light unit pixel123emits blue light B, but the present invention is not limited hereto.

The display panel100may be one of a liquid crystal display panel, an organic light-emitting diode (OLED) display panel, or a micro-LED display. However, the present invention is limited hereto.

Specifically, the wavelength of the red light R according to the present invention may be between 620 nm and 750 nm, but the present invention is not limited hereto.

Specifically, the wavelength of the green light G according to the present invention may be between 495 nm and 570 nm, but the present invention is not limited hereto.

Specifically, the wavelength of the blue light B according to the present invention may be between 450 nm and 495 nm, but the present invention is not limited hereto.

Specifically, the image sensor200according to the present invention is arranged on the display panel100, the image sensor200correspondingly overlaps the display area11, and the image sensor200may have one or more sensing blocks21.

Specifically, referring toFIG.1, the display area11includes one or more display blocks111corresponding to the sensing blocks21, wherein the display block111includes at least one light-emitting block.

It should be further noted that the display block112according to the present invention may be discontinuously adjacent areas, and the display block112may include the collection of all areas of the unit pixels12in the display block11that emit the same color light, but the present invention is not limited hereto.

FIG.2is a schematic view illustrating that the light from the display block according to the present invention is reflected to the sensing block after emitted unto the test object. As shown inFIG.2, when the test object300touches the display device according to the present invention, the display block111can emit a light R to the test object300through the light-emitting block112and the light R is reflected by the test object300to the sensing block21, so that the sensing block21senses and generates an image light intensity.

Specifically, when the sensing block21senses the intensity of the image light travelled from the display block111to the test object300and then reflected to the sensing block21, the user can define the luminous color of each display block111to define the color coordinate value of the luminous color of each display block111. It should be further noted that the color coordinate values in the present specification are all expressed in CIE color coordinates, but the present invention is not limited hereto.

As such, the display device according to the present invention can combine the image light intensity sensed by each sensing block21and the color coordinate value of the luminous color of each display block111to obtain a reference image. The anti-spoofing reference color information and color information of the test object300are compared, and the color information is compared with the anti-spoofing reference color information, thereby providing a high-performance anti-spoofing recognition.

It should be further noted that the light-emitting block112may include a first light-emitting block1121and a second light-emitting block1122, and the first light-emitting block emits a light selected from the group consisting of red light R, green light G, and blue light B. The second light-emitting block emits the another light selected from the group consisting of red light R, green light G, and blue light B, but the present invention is not limited hereto.

It is worth mentioning that the light-emitting block112may further include a third light-emitting block that emits a light selected from the group consisting of red light R, green light G, and blue light B, and the color of the light of the third light-emitting block is different from the first light-emitting block and the second light-emitting block, but the present invention is not limited hereto.

Referring toFIG.3andFIG.4,FIG.3is an exemplary schematic view of a display device according to the present invention;FIG.4is another exemplary schematic view of a display device according to the present invention. As shown inFIGS.3and4, taking the image sensor200divided into a plurality of sensing blocks21as an example, the sensing blocks21include a first sensing block211and a second sensing block212. The display area11of the display panel100is correspondingly divided into a plurality of display blocks111according to the sensing blocks21. The display blocks111include a first display block1111and a second display block1112. The first sensing block211corresponds to the first display block1111, and the second sensing block212corresponds to the second display block1112. It is worth mentioning that, for the convenience of description, the following description is given on the assumption that the first display block1111includes only the red light unit pixels121among the unit pixels12, and the first sensing block211senses a first image light intensity of the first emitted light (not shown) emitted by the first display block1111, reaching and reflected by the test object300. Therefore, the first emitted light is defined as red light R, and the color coordinate value of the first emitted light is expressed by R (xr, yr), the first image light intensity is expressed by IR, but the present invention is not limited hereto. The first display block1111may include one or any combination of red unit pixels121, green unit pixels122, and blue unit pixels123, and the first emitted light may be a light having any wavelength.

Moreover, for the convenience of description, the second display block1112will only include the green unit pixels122of the unit pixels12for description, and the second sensing block212senses a second image light intensity of the second emitted light (not shown) emitted by the second display block1112, reaching and reflected by the test object300. Therefore, the second emitted light is defined as green light G, and the color coordinate value of the second emitted light is G(xg, yg). The second image light intensity is expressed by IG, but the present invention is not limited hereto. The second display block1112may include one or any combination of red unit pixels121, green unit pixels122, and blue unit pixels123, and the second emitted light may be a light having any wavelength.

Specifically, the display device1according to the present invention may further include a calculation unit, which is coupled to the image sensor200, the calculation unit uses formula (1) and formula (2) below and the first image light intensity IRand the second image light intensity IGto calculate the first light-receiving ratio a and the second light-receiving ratio b, but the present invention is not limited hereto.
R(a)=IR/(IR+IG)  (1)
G(b)=IG/(IR+IG)  (2)

Specifically, the calculation unit is based on the color coordinate value R(xr, yr) of the first emitted light and the first light receiving ratio a, and the color coordinate value G(xg, yg) of the second emitted light and the second light receiving ratio b and uses the following formula (3) and formula (4) to generate color information. The color coordinate value of the color information is represented by S(x, y), but the present invention is not limited hereto.
x=axr+bxg(3)
y=ayr+byg(4)

It should be further noted that the sensing block21can further include a third sensing block, and the display block111can correspondingly include a third display block to match the third sensing block, and the third display block may only include blue unit pixels. The third display block may emit a third emitted light to the test object300. The third emitted light is different from the first emitted light and the second emitted light. The third sensing block senses the image generated by the third emitted light to generate a third image light intensity. In terms of accuracy, when the image sensor200is divided into more sensing blocks21, the sensing blocks21can be combined to generate a color information S(x, y) closer to the original color of the test object300. However, it may cause increase in computational complexity and production cost at the same time, thereby causing a decrease in the recognition speed of the image sensor200. The user can choose an appropriate division according to application needs, but the present invention is not limited hereto.

Refer toFIG.5, which is a flowchart illustrating the steps of the image sensing method of the present invention. As shown inFIG.5, the present invention further provides an image sensing method, applicable to the aforementioned display device1. The image sensing method includes the following steps:

Dividing step S1: dividing the image sensor200into a plurality of sensing blocks21; and proceed to corresponding step S2.

Corresponding step S2: correspondingly dividing the display area11of the display100into a plurality of display blocks111according to a plurality of sensing blocks21, the display block111comprising at least one light-emitting block112, the area of the light-emitting block112being smaller than the area of the sensing block21; and proceed to defining step S3.

Defining step S3: defining a luminous color of each display block111and a color coordinate value of the luminous color of each display block111; and proceed to reference sensing step S4.

Reference sensing step S4: the plurality of display blocks111emitting light at the same time, and then each of the sensing blocks21sensing a reference image light intensity of the light reflected to the sensing blocks21from each of the display blocks111emitting light unto a reference object and reflected to the sensing blocks21; and proceed to reference color calculation step S5.

Reference color calculation step S5: generating an anti-spoofing reference color information from the color coordinate value and the reference image light intensity; and proceed to registration step S6.

Registration step S6: registering the anti-spoofing reference color information into a system, and using the anti-spoofing reference color information to generate an anti-spoofing reference interval; and proceed to sensing step S7.

Sensing step S7: the plurality of display blocks111emitting light at the same time, and the sensing blocks21obtaining an image light intensity of an image reflected to the sensing blocks21from the display blocks111emitting light unto and reflected by a test object; and proceed to calculation step S8.

Calculation step S8: generating a color information from the color coordinate value and the image light intensity; and proceed to comparison step S9.

Comparison step S9: determining the color information and the anti-spoofing reference color information being consistent when the color information being within the anti-spoofing reference interval, otherwise determining the color information being inconsistent with the anti-spoofing reference color information.

For example, refer toFIG.5, as well asFIG.1toFIG.4. First, the dividing step S1is executed to divide the image sensor200into a plurality of sensing blocks21. The sensing blocks21include the first sensing block211and the second sensing block212; then the corresponding step S2is executed to divide the display area11of the display panel100correspondingly into a plurality of display blocks111according to the sensing blocks21. The display block111includes a first display block1111and a second display block1112. The first sensing area block211corresponds to the first display block1111, and the second sensing block212corresponds to the second display block1112. Then, the definition step S3is executed. The first display block1111containing only the red unit pixel121emits the first emitted light, defined as red light and the color coordinate value of the first emitted light as R(xr, yr), and the second light emitted by the second display block1112containing only the green unit pixel122is defined as green light, and the color coordinate value of the second emitted light is G(xg, yg). The reference sensing step S4is then performed. The first display block1111and the second display block1112emit light at the same time, and the first sensing block211senses the first reference image light intensity IR′ generated by the reference object reflecting the first emitted light emitted by the first display block1111, and the second sensing block212senses the second reference image light intensity IG′ generated by the reference object reflecting the second emitted light emitted by the second display block1112. Then, the reference color calculation step S5is performed, wherein the color coordinate value R(xr, yr) of the first emitted light the color coordinate value G(xg, yg) of the second emitted light, the first reference image light intensity IR′ generated by the first sensing block211, and the second reference image light intensity IG′ generated by the second sensing block212, are used in the calculation according to the above formula (1) to formula (4) to generate the anti-spoofing reference color information, and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information. Then, the registration step S6is executed to register the color coordinate value S′(x′, y′) of the anti-spoofing reference color information to the system, and generates an anti-spoofing reference interval based on the color coordinate value S′(x′, y′) of the anti-spoofing reference color information. Then, the sensing step S7is executed, wherein the first display block1111and the second display block1112emit light at the same time, and the first sensing block211senses the first reference image light intensity IRgenerated by the test object300reflecting the first emitted light emitted by the first display block1111, and the second sensing block212senses the second reference image light intensity IGgenerated by the test object300reflecting the second emitted light emitted by the second display block1112. The calculation step S8is then performed, wherein the color coordinate value R(xr, yr) of the first emitted light the color coordinate value G(xg, yg) of the second emitted light, the first reference image light intensity IRgenerated by the first sensing block211, and the second reference image light intensity IGgenerated by the second sensing block212, are used in the calculation according to the above formula (1) to formula (4) to generate the color information, and the color coordinate value S(x, y) of the color information. Finally, in the comparison step S9, when the color coordinate value S(x, y) of the color information is within the anti-spoofing reference interval, the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information are determined as consistent, otherwise, a comparison unit determines the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information inconsistent.

It should be further noted that the anti-spoofing reference interval can be generated through big data analysis of classification, statistics, and generalization of fingerprint images of a large number of real fingers, but the present invention is not limited hereto.

It is worth mentioning that, according to the image sensing method of the present invention, the registration step S6can be one-time or multiple-time registration, in other words, the image sensing method provided by the present invention can repeatedly execute the aforementioned reference sensing step S4, the reference color calculation step S5, and the registration step S6, and execution and principles are the same as those described above, and the description will be omitted here. Moreover, the image sensing method of the present invention can also execute the registration step S6only once. It is understandable that the image sensing method can store a large amount of anti-spoofing reference color information, so the anti-spoofing reference color information can be the average value of a plurality of anti-spoofing reference color information, as shown in the following formula (5). Moreover, the absolute value of the anti-spoofing reference interval can be the average value of the maximum value minus the minimum value in the anti-spoofing reference color information registered by the system, as shown in the following formula (6). It can be understood that the anti-spoofing reference interval of the present invention may be one half of the extreme difference, but the present invention is not limited hereto.

It is worth mentioning that the anti-spoofing reference interval can be an interval arbitrarily and manually set, and the absolute value of the anti-spoofing reference interval can be, but not limited to, one of 1, 2, or 3 times the standard deviation σ of the anti-spoofing reference color information registered in the system. As shown in the following formula (8), the standard deviation σ is calculated as shown in the following formula (7). According to the law of three-sigma, all values consistent with the anti-spoofing reference color information will fall within the range of plus or minus three standard deviations σ of the anti-spoofing reference color information. Therefore, when the color coordinate value S(x, y) of the color information is within the anti-spoofing reference interval, the display device1can accurately determine the color coordinate value S(x, y) of the color information to be consistent with the color coordinate value S′(x′, y′) of the anti-spoofing reference color information, but the present invention is not limited hereto.

As such, the image sensing method provided by the present invention can successfully provide a high-efficiency anti-spoofing recognition without increasing the production cost of the anti-spoofing recognition module.

First Embodiment

Hereinafter, an embodiment of the first embodiment of the display device1of the present invention will be described with reference to the drawings.

Refer toFIG.6, which is a schematic view of a display device according to the first embodiment of the present invention. As shown inFIG.6, the display device1according to the first embodiment of the present invention is applied to a fingerprint sensing system. The display device1includes: a display panel100, a display area11, a unit pixel12, and an image sensor200, a calculation unit13, a storage unit131, and a comparison unit132.

Specifically, referring toFIG.6, the display panel100according to the first embodiment of the present invention has a display area11, the display area11includes a plurality of unit pixels12, and the unit pixels12include red unit pixels121, green unit pixels122, and blue unit pixels123, wherein the red unit pixels121emit red light R, the green unit pixels122emit green light G, and the blue unit pixels123emit blue light B. However, the present invention is not limited hereto.

Specifically, the image sensor200is disposed on the display panel100, the image sensor200correspondingly overlaps the display area11, and the image sensor200includes a plurality of sensing blocks21, however the present invention is not limited hereto.

Refer toFIG.7, which is a schematic view of the image sensor according to the first embodiment of the present invention. As shown inFIG.7, the image sensor200is divided into a plurality of sensing blocks21. The sensing blocks21include a first sensing block211, a second sensing block212, and a third sensing block213. The display area11of the display panel100is divided into a plurality of display blocks111correspondingly according to the sensing blocks21, wherein the display blocks111includes a first display block1111, a second display block1112, and a third display block1113. Moreover, the first sensing block211corresponds to the first display block1111, the second sensing block212corresponds to the second display block1112, and the third sensing block211corresponds to the third display block1113.

Specifically, referring toFIG.7, the display area11includes one or more display blocks111corresponding to the sensing blocks21, wherein the display blocks111include at least one light-emitting block112. The area of the light-emitting block112is smaller than the area of the sensing block21. In the present embodiment, the first sensing block211includes a first light-emitting block1121, the second sensing block212includes a second light-emitting block1122, and the third sensing block213includes a third light-emitting block1123, however, the present invention is not limited hereto.

Specifically, the first display block1111includes only the red unit pixels121of the unit pixels12, and the first sensing block211senses the first image light intensity generated by the first emitted light emitted from the first display block1111and reflected by the test object300, so the first emitted light is defined as red light R. The color coordinate value of the first emitted light is represented by R(xr, yr), and the first image light intensity is represented by IR, but the present invention is not limited hereto.

Specifically, the second display block1112includes only the green unit pixels122of the unit pixels12, and the second sensing block212senses the second image light intensity generated by the second emitted light emitted from the second display block1112and reflected by the test object300, so the second emitted light is defined as green light G. The color coordinate value of the second emitted light is represented by G(xg, yg), and the second image light intensity is represented by IG, but the present invention is not limited hereto.

Specifically, the third display block1113includes only the blue unit pixels123of the unit pixels12, and the third sensing block213senses the third image light intensity generated by the third emitted light emitted from the third display block1113and reflected by the test object300, so the third emitted light is defined as blue light B. The color coordinate value of the third emitted light is represented by B(xb, yb), and the third image light intensity is represented by IB, but the present invention is not limited hereto.

Specifically, the calculation unit13is coupled to the image sensor200, the calculation unit13uses the first image light intensity IR, the second image light intensity IG, and the third image light intensity IBand the following formula (9), formula (10), and formula (11) to calculate a first light-receiving ratio a, the second light-receiving ratio b, and the third light-receiving ratio c. However, the present invention does not limited hereto.
R(a)=IR/(IR+IG+IB)  (9)
G(b)=IG/(IR+IG+IB)  (10)
B(c)=IB/(IR+IG+IB)  (11)

Specifically, the calculation unit13uses the color coordinate value R(xr, yr) of the first emitted light and the first light-receiving ratio a, and the color coordinate value G(xg, yg) of the second emitted light and the second light-receiving ratio b, and the color coordinate value B(xb, yb) of the third emitted light and the third light-receiving ratio c, and the following formula (12) and formula (13), to calculate the color information S(x, y), but the invention is not limited hereto.
x=axr+bxg+cxb(12)
y=ayr+byg+cyb(13)

It should be further noted that the display device1according to the first embodiment of the present invention can use a reference object, so that the first sensing block211senses a first reference image light intensity IR′ generated by the first emitted light R(xr, yr) emitted to and reflected by the reference object; the second sensing block212senses a second reference image light intensity IG′ generated by the second emitted light G(xg, yg) emitted to and reflected by the reference object; and the third sensing blocks213sense a third reference image light intensity IB′ generated by the third emission light B(xb, yb) emitted to and reflected by the reference object. The calculation unit13is used in conjunction with the above formula (9), formula (10), and formula (11) to calculate the first light-receiving ratio a, the second light-receiving ratio b, and the third light-receiving ratio c. Finally, based on the color coordinate value R(xr, yr) of the first emitted light and the first light-receiving ratio a, the color coordinate value G(xg, yg) of the second emitted light and the second light-receiving ratio b, and the color coordinate value B(xb, yb) of the third emitted light and the third light-receiving ratio c, the above formula (12) and formula (13) are used to calculate the color coordinate value S′(x′, y′) of the anti-spoofing reference color information, but the present invention is not limited hereto.

Specifically, the storage unit131of the first embodiment of the present invention is used to store the color coordinate value S′(x′, y′) of the anti-cspoofing reference color information and the color coordinate value S(x, y) of the color information. The storage unit131may be a volatile memory, or a non-volatile memory. Specifically, the volatile memory can be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random-access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), and so on, but the present invention is not limited hereto.

Specifically, the comparison unit132is coupled to the storage unit131, and the comparison unit132generates an anti-spoofing reference interval according to the anti-spoofing reference color information. Thereby, whenever the user needs to perform unlocking, the comparison unit132uses the color coordinate value S′(x′, y′) of the anti-spoofing reference color information as a reference to perform calculations through the following formula (14), and to determine through the following formula (15); wherein when the color information S(x, y) is in the anti-spoofing reference interval, the image sensor100determines the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information are consistent; otherwise, the image sensor100determines that the color coordinate value S(x, y) of the color information is not consistent with the color coordinate value S′(x′, y′) of the anti-spoofing reference color information.
S′−S=(x′−x,y′−y)  (14)
|S′−S|≤anti-spoofing reference interval  (15)

It should be further noted that, in the present embodiment, the absolute value of the anti-spoofing reference interval may be the average value of the maximum value minus the minimum value in the anti-spoofing reference color information registered in the system. Alternatively, the absolute value of the anti-spoofing reference interval can be one of 1, 2, and 3 times of the standard deviation σ of the anti-spoofing reference color information registered in the system. It should be understood that when the color coordinate value S(x, y) and the color coordinate value S′(x′, y′) of the anti-spoofing standard color information are consistent, after the color coordinate value S′(x′, y′) of the anti-spoofing standard color information minus the color coordinate value S(x, y) of the color information, the x coordinate and the y coordinate will be less than the average value of the maximum minus the minimum value or 1 to 3 times of the standard deviation σ, but the present invention is not limited hereto.

Refer toFIG.8.FIG.8is a flowchart illustrating the steps of performing the image sensing method according to the first embodiment of the present invention. As shown inFIG.8, the present invention further provides an image sensing method, which can be applied to the display device1of the first embodiment, and the image sensing method includes the following steps:

Dividing step S1′: dividing the image sensor200into a plurality of sensing blocks21; and proceed to corresponding step S2′.

Corresponding step S2′: correspondingly dividing the display area11of the display100into a plurality of display blocks111according to a plurality of sensing blocks21, the display block111comprising at least one light-emitting block112, the area of the light-emitting block112being smaller than the area of the sensing block21; and proceed to defining step S3′.

Defining step S3′: defining a luminous color of each display block111and a color coordinate value of the luminous color of each display block111; and proceed to reference sensing step S4′.

Reference sensing step S4′: the plurality of display blocks111emitting light at the same time, and then each of the sensing blocks21sensing a reference image light intensity of the light reflected to the sensing blocks21from each of the display blocks111emitting light unto a reference object and reflected to the sensing blocks21; and proceed to reference color generation step S5′.

Reference color generation step S5′: generating an anti-spoofing reference color information from the color coordinate value and the reference image light intensity; and proceed to reference color registration step S6′.

Reference color registration step S6′: registering the anti-spoofing reference color information into a system, and proceed to reference color calculation step S7′.

Reference color calculation step S7′: calculating the average of the color coordinate value S(x, y) of a plurality of color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information stored in the storage unit131together with the anti-spoofing reference color information as the anti-spoofing reference color information; and proceed to the anti-spoofing reference interval calculation step S8′.

Anti-spoofing reference interval calculation step S8′: using the anti-spoofing reference color information to generate an anti-spoofing reference interval; and proceed to sensing step S9′.

Sensing step S9′: the plurality of display blocks111emitting light at the same time, and the sensing blocks21obtaining an image light intensity of an image reflected to the sensing blocks21from the display blocks111emitting light unto and reflected by a test object; and proceed to calculation step S10′.

Calculation step S10′: generating a color information from the color coordinate value and the image light intensity; and proceed to comparison step S11′.

Comparison step S11′: determining the color information and the anti-spoofing reference color information being consistent when the color information being within the anti-spoofing reference interval, otherwise determining the color information being inconsistent with the anti-spoofing reference color information.

For example, refer toFIG.6, as well asFIG.5. First, the dividing step S1′ is executed to divide the image sensor200into a plurality of sensing blocks21. The sensing blocks21include the first sensing block211, the second sensing block212and the third sensing block213; then the corresponding step S2′ is executed to divide the display area11of the display panel100correspondingly into a plurality of display blocks111according to the sensing blocks21. The display block111includes a first display block1111, a second display block1112, and a third display block1113. The first sensing area block211corresponds to the first display block1111, the second sensing block212corresponds to the second display block1112, and the third sensing block213corresponds to the third display block1113. Then, the definition step S3′ is executed. The first display block1111containing only the red unit pixel121emits the first emitted light, defined as red light and the color coordinate value of the first emitted light as R(xr, yr), the second light emitted by the second display block1112containing only the green unit pixel122is defined as green light, and the color coordinate value of the second emitted light is G(xg, yg), and the third light emitted by the third display block1113containing only the blue unit pixel123is defined as blue light, and the color coordinate value of the third emitted light is B(xb, yb). The reference sensing step S4′ is then performed. The first display block1111, the second display block1112and the third display block emit light at the same time, and the first sensing block211senses the first reference image light intensity IRgenerated by the reference object reflecting the first emitted light emitted by the first display block1111, the second sensing block212senses the second reference image light intensity IG′ generated by the reference object reflecting the second emitted light emitted by the second display block1112, and the third sensing block213senses the third reference image light intensity IB′ generated by the reference object reflecting the third emitted light emitted by the third display block1113. Then, the reference color generation step S5′ is performed, wherein the color coordinate value R(xr, yr) of the first emitted light, the color coordinate value G(xg, yg) of the second emitted light, the color coordinate value B(xb, yb) of the third emitted light, the first reference image light intensity IR′ generated by the first sensing block211, the second reference image light intensity IG′ generated by the second sensing block212, and the third reference image light intensity IB′ generated by the third sensing block213, are used in the calculation according to the above formula (9) to formula (13) to generate the anti-spoofing reference color information. Then, the reference color registration step S6′ is executed to store the color coordinate value S′(x′, y′) of the anti-spoofing reference color information in the storage unit131; followed by the reference color calculation step S7′ to calculate the average of the color coordinate value S(x, y) of a plurality of color information, and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information stored the storage unit131, together with the anti-spoofing reference color information using the above formula (5) as the anti-spoofing reference color information. Then, the anti-spoofing reference interval calculation step S8′ is performed wherein the anti-spoofing reference interval is calculated by least one of combination of the above formula (6) to formula (8) according to the anti-spoofing reference color information.

Then, the sensing step S9′ is executed, wherein the first display block1111, the second display block1112, and the third display block1113emit light at the same time, and the first sensing block211senses the first reference image light intensity IRgenerated by the test object300reflecting the first emitted light emitted by the first display block1111, the second sensing block212senses the second reference image light intensity IGgenerated by the test object300reflecting the second emitted light emitted by the second display block1112, and the third sensing block213senses the third reference image light intensity IBgenerated by the test object300reflecting the third emitted light emitted by the third display block1113. The calculation step S10′ is then performed, wherein the color coordinate value R(xr, yr) of the first emitted light, the color coordinate value G(xg, yg) of the second emitted light, the color coordinate value B(xb, yb) of the third emitted light, the first reference image light intensity IRgenerated by the first sensing block211, the second reference image light intensity IGgenerated by the second sensing block212, and the third reference image light intensity IBgenerated by the third sensing block213are used in the calculation according to the above formula (9) to formula (13) to generate the color information, and the color coordinate value S(x, y) of the color information. Finally, in the comparison step S11′, according to the above formula (14), when the color coordinate value S(x, y) of the color information is within the anti-spoofing reference interval, the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information are determined as consistent, otherwise, a comparison unit determines the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information inconsistent.

It should be further noted that the anti-spoofing reference interval can be generated through big data analysis of classification, statistics, and generalization of fingerprint images of a large number of real fingers, but the present invention is not limited hereto.

It is worth mentioning that, according to the image sensing method of the present invention, the registration step S6′ can be one-time or multiple-time registration, in other words, the image sensing method provided by the present invention can repeatedly execute the aforementioned reference sensing step S4′, the reference color calculation step S5′, and the registration step S6′, and execution and principles are the same as those described above, and the description will be omitted here. Moreover, the image sensing method of the present invention can also execute the registration step S6only once. It is understandable that the image sensing method can store a large amount of anti-spoofing reference color information, so the anti-spoofing reference color information can be the average value of a plurality of anti-spoofing reference color information, but the present invention is not limited hereto.

It is worth mentioning that the anti-spoofing reference interval can be an interval arbitrarily and manually set, or the anti-spoofing reference interval can be the difference between the maximum value and the minimum value and the average value in the anti-spoofing reference color information registered in the system, but the present invention is not limited hereto.

As such, based on the display device1of the first embodiment of the present invention and combined with the image sensing method provided by the present invention, it is possible to obtain, with only one run of image sensing, the reference image and the image light intensity of image of the test object under the first emitted light R(xr, yr), the second emitted light G(xg, yg), and the third emitted light B(xb, yb), and by the calculation of the calculation unit13to generate the anti-spoofing reference color information and color information. Finally, the comparison unit determines whether the color of the test object and the reference object are consistent to achieve the purpose of anti-spoofing, providing a high-performance anti-spoofing recognition.

The following provides other examples of the display device1, so that a person with ordinary knowledge in the technical field to which the present invention belongs can understand possible changes more clearly. The elements indicated by the same reference numerals as in the foregoing embodiment are substantially the same as those described above with reference toFIGS.5and6. The elements, features, and advantages that are the same as those of the display device1will not be repeated.

Second Embodiment

Hereinafter, the second embodiment of the display device1of the present invention will be described with reference to the drawings.

Refer toFIG.9, which is a schematic view of a display device according to a second embodiment of the present invention. Compared with the first embodiment, the second embodiment differs from the first embodiment in that, in the second embodiment of the present invention, the image sensor200is divided into a sensing block21, and the display area11is divided into a first display block1111, a second display block1112, and a third display block1113, which respectively correspond the red unit pixels121, the green unit pixels122, and the blue unit pixels123; in other words, the first display block1111includes all the red unit pixels121in the display area11, the second display block1112includes all the green unit pixels122in the display area11, and the third display block1113includes all the blue unit pixels123in the display area11. The display block111sequentially emits the first emitted light, the second emitted light, and the third emitted light.

Specifically, refer toFIG.9. First, the sensing block21senses the first image light intensity from the first emitted light emitted by the first display block1111and reflected by the test image200, thus defining the first emitted light as red light R, the color coordinate value of the first emitted light is represented by R(xr, yr), and the first image light intensity is represented by IR. Then, the sensing block21senses the second image light intensity from the second emitted light emitted by the second display block1112and reflected by the test image200, thus defining the second emitted light as green light G, the color coordinate value of the second emitted light is represented by G(xg, yg), and the second image light intensity is represented by IG. Finally, the sensing block21senses the third image light intensity from the third emitted light emitted by the third display block1113and reflected by the test image200, thus defining the third emitted light as blue light B, the color coordinate value of the third emitted light is represented by B(xb, yb), and the third image light intensity is represented by IB. However, the present invention is not limited hereto. It can be understood that the materials and other characteristics used in the image sensor100according to the second embodiment of the present invention are similar to those of the image sensor100according to the first embodiment of the present invention, and the details will not be repeated here.

It should be further noted that the display block111according to the second embodiment of the present invention includes at least one light-emitting block112, wherein the area of the light-emitting block112is not limited to the area of the sensing block21, and the light-emitting block112may be larger than the area of the sensing block21, but the present invention is not limited hereto.

It is worth mentioning that the number of the unit pixels12, such as, the red light unit pixels121, the green light unit pixels122, and the blue light unit pixels123, included in the current display devices on the market may not be the same, due to the consideration of efficiency and service life. For the convenience of description, the following description will take the number of green unit pixels122to be twice that of red unit pixels121and blue unit pixels123, which is only an exemplary description, but the present invention is not limited hereto.

Specifically, the calculation unit13of the second embodiment of the present invention is coupled to the image sensor200, the calculation unit13uses the first image light intensity IR, the second image light intensity IG, and the third image light intensity IBwith the following formula (16), formula (17), and formula (18) to calculate the first light-receiving ratio a, the second light-receiving ratio b, and the third light-receiving ratio c. However, the present invention is not limited hereto. It can be understood that the parameters in the following formulas can be adjusted according to the numbers of red unit pixels121, green unit pixels122, and blue unit pixels123. The following is an exemplar according to the condition that the number of green unit pixels122is twice of the red unit pixels121and the blue unit pixels123as described. Those skilled in the art can make various changes and adjustments according to the following formulas, which will not be listed here.
R(a)=IR/(IR+2*IG+IB)  (16)
2*G(b)=2*IG/(IR+2*IG+IB)  (17)
B(c)=IB/(IR+2*IG+IB)  (18)

Specifically, the calculation unit13is based on the color coordinate value R(xr, yr) of the first emitted light and the first light receiving ratio a, the color coordinate value G(xg, yg) and the second light-receiving ratio b, and the color coordinate value of the third emitted light B(xb, yb) and the third light-receiving ratio c, and uses the following formula (19) and formula (20) to calculate the color information S(x, y). However, the present invention is not limited hereto.
x=axr+bxg+cxb(19)
y=ayr+byg+xyb(20)

As such, the image sensor100according to the second embodiment of the present invention can also achieve the effects of the first embodiment while providing different structures. In terms of effect, the color information S(x, y) generated according to the second embodiment of the present invention has better resolution. In addition, the second embodiment can overcome the potential problem of the mutual influence among the first sensing block211, the second sensing and the third sensing block213to cause the risk of deviation of the color information S(x, y) in the first embodiment. However, the disadvantage of the second embodiment is that multiple image sensing runs must be performed, and the user can determine which approach to adopt depending on the application needs, but the present invention is not limited hereto.

To provide a further understanding of the structural features, technical means and expected effects of the second embodiment of the present invention, the use of the present invention is described. It is believed that a deeper and specific understanding of the present invention can be obtained from the description below:

Refer toFIG.10, which is a flowchart illustrating the steps of the image sensing method of the second embodiment of the present invention. As shown inFIG.10, the present invention further provides an image sensing method, applicable to the aforementioned display device1of the second embodiment. The image sensing method includes the following steps:

Dividing step S1″: dividing the image sensor200into one or more sensing blocks21; and proceed to corresponding step S2″.

Corresponding step S2″: correspondingly dividing the display area11of the display100into a plurality of display blocks111according to the sensing block21; and proceed to defining step S3″.

Defining step S3″: defining each luminous color and defining a color coordinate value of the luminous color of each display block111; and proceed to reference sensing step S4″.

Reference sensing step S4″: teach display block111emitting light sequentially, and then the sensing block21sequentially sensing a reference image light intensity of each light reflected to the sensing block21from the display block111emitting light unto a reference object and reflected to the sensing block21; and proceed to reference color calculation step S5″.

Reference color calculation step S5″: generating an anti-spoofing reference color information from the color coordinate value and the reference image light intensity; and proceed to registration step S6″.

Registration step S6″: registering the anti-spoofing reference color information into a system, and using the anti-spoofing reference color information to generate an anti-spoofing reference interval; and proceed to sensing step S7″.

Sensing step S7″: each display block111emitting light sequentially, and the sensing block21sequentially obtaining an image light intensity of the image reflected to the sensing block21from each display block111emitting light unto and reflected by a test object; and proceed to calculation step S8″.

Calculation step S8″: generating a color information from the color coordinate value and the image light intensity; and proceed to comparison step S9″.

Comparison step S9″: determining the color information and the anti-spoofing reference color information being consistent when the color information being within the anti-spoofing reference interval, otherwise determining the color information being inconsistent with the anti-spoofing reference color information.

For example, refer toFIG.8, as well asFIG.7. First, the dividing step S1″ is executed to divide the image sensor200into one sensing block21; then the corresponding step S2″ is executed to divide the display area11of the display panel100into a first display block1111including red unit pixels121, a second display block1112including green unit pixels122, and a third display block1113including blue unit pixels123. Then, the definition step S3″ is executed. The first display block1111containing only the red unit pixel121emits the first emitted light, defined as red light and the color coordinate value of the first emitted light as R(xr, yr), the second light emitted by the second display block1112containing only the green unit pixel122is defined as green light, and the color coordinate value of the second emitted light is G(xg, yg), and the third light emitted by the third display block1113containing only the blue unit pixel123is defined as blue light, and the color coordinate value of the second emitted light is G(xb, yb). The reference sensing step S4″ is then performed. The first display block1111, the second display block1112, and the third display block1113emit light sequentially, wherein the sensing block21senses the first reference image light intensity IRgenerated by the reference object reflecting the first emitted light emitted by the first display block1111, then the sensing block21senses the second reference image light intensity IG′ generated by the reference object reflecting the second emitted light emitted by the second display block1112, and then the sensing block21senses the third reference image light intensity IB′ generated by the reference object reflecting the third emitted light emitted by the third display block1113. Then, the reference color calculation step S5″ is performed, wherein the color coordinate value R(xr, yr) of the first emitted light, the color coordinate value G(xg, yg) of the second emitted light, the color coordinate value B(xb, yb) of the third emitted light, the first reference image light intensity IR′, the second reference image light intensity IG′, and the third reference image light intensity IB′ generated by the sensing block21, are used in the calculation according to the above formula (16) to formula (20) to generate the anti-spoofing reference color information, and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information. Then, the registration step S6″ is executed to register the color coordinate value S′(x′, y′) of the anti-spoofing reference color information to the system, and generates an anti-spoofing reference interval based on the color coordinate value S′(x′, y′) of the anti-spoofing reference color information. Then, the sensing step S7″ is executed, wherein the first display block1111, the second display block1112, and the third display block1113emit light sequentially, and the sensing block21senses the first reference image light intensity IRgenerated by the test object200reflecting the first emitted light emitted by the first display block1111, then the sensing block21senses the second reference image light intensity IGgenerated by the test object200reflecting the second emitted light emitted by the second display block1112, and then the sensing block21senses the third reference image light intensity IBgenerated by the test object200reflecting the third emitted light emitted by the third display block1113. The calculation step S8″ is then performed, wherein the color coordinate value R(xr, yr) of the first emitted light, the color coordinate value G(xg, yg) of the second emitted light, the color coordinate value B(xb, yb) of the third emitted light, the first reference image light intensity IR, the second reference image light intensity IG, and the third reference image light intensity IBgenerated by the sensing block21, are used in the calculation according to the above formula (16) to formula (20) to generate the color information, and the color coordinate value S(x, y) of the color information of the test object200. Finally, in the comparison step S9″, when the color coordinate value S(x, y) of the color information is within the anti-spoofing reference interval, the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information are determined as consistent, otherwise, a comparison unit determines the color coordinate value S(x, y) of the color information and the color coordinate value S′(x′, y′) of the anti-spoofing reference color information inconsistent.

It can be understood that those with ordinary knowledge in the technical field to which the present invention belongs can make various changes and adjustments based on the above examples, which will not be listed here.

Finally, the technical features of the present invention and its achievable technical effects are summarized as follows:

First, based on the display device1, combined with the image sensing method provided by the present invention, the present invention successfully provides a high-efficiency anti-spoofing recognition without increasing the production cost of the anti-spoofing recognition module.

Second, the image sensor100according to the present invention can obtain the first image light intensity IRand the second image light intensity IGof the object under the first emitted light and the second emitted light, and the calculation unit15calculates to generate color information S(x, y) with sufficient anti-spoofing capability, thereby providing a high-efficiency anti-spoofing recognition.

Third, the image sensor100according to the second embodiment of the present invention can further improve, through multiple image sensing runs, the resolution of the image sensor100in a fingerprint recognition environment.