Patent Description:
With the development of technology, mobile phones, digital cameras, tablet PCs, notebook computers and other portable electronic devices become more and more popular. The portable electronic devices are intended for personal use, with certain privacy. The information stored in the portable device such as phonebook, photos, personal information, etc., are privately owned. If the electronic device is lost, these data may be accessed by other people, causing unnecessary loss. Even though there are several ways using password to protect the electronic device from being used by the other people, the password may be easily to be spread or cracked, which lowers the security. Moreover, the user needs to remember the password so as to use electronic devices. If the user forgets the password, troubles would be brought to the user. Hence, personal fingerprint identification is utilized to achieve the purpose of personal identification, for enhancing the data security.

In another perspective, as the technology of fingerprint identification grows, the fingerprint identification system may be disposed under the display panel. That is, the use may press the display panel and the fingerprint identification may be performed. Specifically, the image sensing array of the fingerprint identification system may be disposed under the display pixel array of the display panel. However, for the optical fingerprint identification system, the image received by the image sensing array is easily affected by the display pixel array, such that the fingerprint is not accurately identified. Therefore, it is necessary to improve the prior art. Examples of relevant prior art includes <CIT> and <CIT>.

It is therefore a primary objective of the present application to provide a fingerprint image processing method, an optical fingerprint identification system and an electronic device enhancing fingerprint identification accuracy, to improve over disadvantages of the prior art.

To solve the problem stated in the above, the present application provides a fingerprint image processing method according to appended claim <NUM>, an optical fingerprint identification system according to appended claim <NUM>, and an electronic device according to appended claim <NUM>.

The present application is suitable for in-display fingerprint identification, which utilizes subtracting the background image to eliminate the lattice interference, and utilizes filtering at the interfering frequencies to filter out the interference components to filter out the interference components. The fingerprint image is clearer. The present application has advantages of enhancing the accuracy of fingerprint identification.

In order to make the objects, technical solutions and advantages of the present application become more apparent, the following relies on the accompanying drawings and embodiments to describe the present application in further detail. It should be understood that the specific embodiments described herein are only for explaining the present application and are not intended to limit the present application.

Please refer to <FIG> is a schematic diagram of an electronic device <NUM> according to an embodiment of the present application. The electronic device <NUM> may be an electronic device such as a smart phone or a tablet computer including a display panel. The electronic device <NUM> comprises a display panel <NUM> and an optical fingerprint identification system <NUM>. The display panel <NUM> comprises a display pixel array DP_array and a passivation layer PSV. The passivation layer PSV may be sapphire or a glass cover. The display pixel array DP_array may comprise a plurality of displaying pixel units. The plurality of displaying pixel units is arranged as an array, which may comprise a plurality of red light emitting elements, blue light emitting elements and green light emitting elements. The display pixel array DP_array of the display panel <NUM> is configured to display pictures which the electronic device <NUM> is expected to display.

The optical fingerprint identification system <NUM> is a fingerprint identification system disposed under the display panel <NUM> or the display pixel array DP_array, i.e. an In-Display fingerprint identification system. The optical fingerprint identification system <NUM> comprises an image sensing array SN_array, a computing module <NUM>, an identification module <NUM> and a light emitting unit <NUM>. The image sensing array SN_array comprises a plurality of image sensing pixel units arranged as an array.

When the optical fingerprint identification system <NUM> tends to identify a finger FG, the light emitting unit <NUM> emits an incident light toward the finger FG. The incident light would be reflected from the finger FG, and the reflected light would be received by the image sensing array SN_array. That is, the image sensing array SN_array receives a received image RCV_IM according to the reflected light. After the image sensing array SN_array receives the received image RCV_IM, the received image RCV_IM is outputted to the computing module <NUM>. The computing module <NUM> may perform a fingerprint image processing operation on the received image RCV_IM, and outputs the operational result of the fingerprint image processing operation to the identification module <NUM>, and thus the identification module <NUM> may determine a fingerprint of the finger FG according to the operational result of the fingerprint image processing operation.

Notably, an area/size of the display pixel array DP_array is not necessarily (usually not) the same as an area/size of the displaying pixel units. In addition, the display pixel array DP_array is not necessarily (usually not) aligned with the image sensing array SN_array, such that the image RCV_IM received by the image sensing array SN_array is easily affected by the display pixel array DP_array, and the identification of the fingerprint of the finger FG is not accurate.

For example, please refer to <FIG> is a schematic diagram of the received image RCV_IM received by the image sensing array SN_array, which is disposed under the display pixel array DP_array. Since it is affected by the display pixel array DP_array, there are multiple dark points and bright points within the received image RCV_IM, such that it is not easy for the identification module <NUM> to accurately identify the fingerprint of the finger FG according to the image RCV_IM.

Therefore, in order to exclude the affection of the display pixel array DP_array on the image RCV_IM, the computing module <NUM> may perform the fingerprint image processing operation on the received image RCV_IM. Please refer to <FIG>, which is a schematic diagram of a fingerprint image processing process <NUM> according to an embodiment of the present application. The fingerprint image processing process <NUM> is executed by the optical fingerprint identification system <NUM>, which comprises the following steps:.

Specifically, in Step <NUM>, when the display panel <NUM> is not pressed by the finger FG of the user, the computing module <NUM> obtains the background image BG_IM and the interfering frequencies fI,<NUM>-fI,N. In detail, when the user does not press the finger on the display panel <NUM>, the computing module <NUM> may obtain a plurality of first images IM_1 at a first time, and perform an average operation on the plurality of first images IM_1, so as to obtain a first background image BG_IM_1 corresponding to the plurality of first images IM_1. Moreover, the first background image BG_IM_1 comprises no fingerprint image, but comprises the dark points or the bright points (hereinafter, "lattice interference"), which are caused by different sizes of the displaying pixel units (within the display pixel array DP_array) and the image sensing pixel units (within the image sensing array SN_array), or caused by non-alignment between the display pixel array DP_array and the image sensing array SN_array. In addition, the computing module <NUM> may further obtain a plurality of second images IM_2 at a second time, and perform an average operation on the plurality of second images IM_2, so as to obtain a second background image BG_IM_2 corresponding to the plurality of second images IM_2. Similarly, the second background image BG_IM_2 comprises no fingerprint image but comprises the lattice interference. In addition, the background image BG_IM in Step <NUM> may be referred to the first background image BG_IM_1 or the second background image BG_IM_2.

Since the sizes/areas of the displaying pixel units and the image sensing pixel units and the arrangement of the display pixel array DP_array and the image sensing array SN_array (or an angle between the display pixel array DP_array and the image sensing array SN_array) are time-invariant, the lattice interference within the first background image BG_IM_1 and the lattice interference within the second background image BG_IM_2 should be the same. The computing module <NUM> performs a subtracting operation and a transforming operation on the first background image BG_IM_1 and the second background image BG_IM_2, to obtain a difference image spectrum Diff_IM_SPM. Specifically, the computing module <NUM> may perform the subtracting operation on the first background image BG_IM_1 and the second background image BG_IM_2 first and then perform the transforming operation on the subtraction result of the first background image BG_IM_1 and the second background image BG_IM_2. Alternatively, the computing module <NUM> may perform the transforming operation on the first background image BG_IM_1 and the second background image BG_IM_2 first and then perform subtraction on the spectrum of the first background image BG_IM_1 and the spectrum of the second background image BG_IM_2, which are all within the scope of the present application. In an embodiment, the transforming operation may be a fast Fourier transform (FFT).

Whether the subtracting operation is performed before the transforming operation is performed or the transforming operation is performed before the subtracting operation is performed, the lattice interference within the first background image BG_IM_1 and the second background image BG_IM_2 would be eliminated when the subtracting operation is executed. In other words, none of the lattice interference is included in the difference image spectrum Diff_IM_SPM. In addition, material of the display pixel array DP_array and material of the image sensing array SN_array might have different expansion coefficients. When the ambient temperature of the electronic device <NUM> varies with time, the size or relative position of the display pixel array DP_array and the image sensing array SN_array at the first time and the second time may vary, an cause interference on the image received by the image sensing array SN_array, where the ambient temperature of the electronic device <NUM> at the first time is different from the ambient temperature at the second time.

In addition, the computing module <NUM> performs an energy detection operation to derive frequency point which is easily to be interfered, called as the interfering frequency. Method of the computing module <NUM> performing the energy detection operation to obtain the interfering frequencies fI,<NUM>-fI,N is not limited. For example, the computing module <NUM> may select frequency points with spectrum energy are greater than a specific value, according to the difference image spectrum Diff_IM_SPM, as the interfering frequencies fI,<NUM>-fI,N. Alternatively, the computing module <NUM> may perform statistical analysis on energy of each frequency point of the difference image spectrum Diff_IM_SPM, and select the first α% frequency points, in terms of spectrum energy, as the interfering frequencies fI,<NUM>-fI,N. The percentage α% may be modified according to practical situation, and may be <NUM>%, <NUM>% or <NUM>%, and not limited thereto. For example, the percentage α% may be related to a statistical average p plus several standard deviations σ, wherein the statistical average p and the standard deviation σ are related to a statistical average and a standard deviation of spectrum energy on each frequency point of the difference image spectrum Diff_IM_SPM, respectively. In an embodiment, an energy distribution at a frequency point of the difference image spectrum Diff_IM_SPM may be assumed as a normal distribution. In other words, compared to other frequency points, the difference image spectrum Diff_IM_SPM has more energy at the interfering frequencies fI,<NUM>-fI,N. In addition, the computing module <NUM> is not limited to selecting multiple interfering frequencies fI,<NUM>-fI,N. The computing module <NUM> may simply select one single interfering frequency fI according to the difference image spectrum Diff_IM_SPM, which is also within the scope of the present application.

Operations of the computing module <NUM> obtaining the background image BG_IM and the interfering frequencies fI,<NUM>-fI,N may be summarized as a process <NUM>, as shown in <FIG>. The process <NUM> comprises the following steps:.

Notably, Step <NUM> and the process <NUM> are executed under a condition that the display panel <NUM> is not pressed by the finger FG of the user. That is, execution of Step <NUM> and the process <NUM> should be completed before the fingerprint identification is performed, to obtain the background image BG_IM and the interfering frequencies fI,<NUM>-fI,N in advance. In other words, Step <NUM> and the process <NUM> belong to the steps of an offline stage. When the display panel <NUM> is pressed by the finger FG of the user to perform the fingerprint identification, in Step <NUM>, the light emitting unit <NUM> emits the incident light and the image sensing array SN_array receives the received image RCV_IM.

After the image sensing array SN_array receives the received image RCV_IM, in Step <NUM>, the computing module <NUM> performs the subtracting operation on the received image RCV_IM and the background image BG_IM, to obtain the difference image. The background image BG_IM may be the first background image BG_IM_1 or the second background image BG_IM_2. The computing module <NUM> may subtract the background image BG_IM from the received image RCV_IM, or subtract the received image RCV_IM from the background image BG_IM, which are within the scope of the present application. In Step <NUM>, the computing module <NUM> performs the filtering operation on the difference image Diff_IM at the interfering frequencies fI,<NUM>-fI,N, which is to perform the filtering operation on the difference image Diff_IM especially on the interfering frequencies fI,<NUM>-fI,N, to filter out the interference components, so as to obtain the operational result OUT.

After the computing module <NUM> completes the execution of the fingerprint image processing process <NUM>, the computing module <NUM> outputs the operational result OUT to the identification module <NUM>, and the fingerprint of the finger FG is determined according to the operational result OUT. Please refer to <FIG>, which is a schematic diagram of the operational result OUT. As can be seen from <FIG> and <FIG>, the fingerprint image is more obvious in the operational result OUT <FIG> (compared to the received image RCV_IM of <FIG>), such that the identification module <NUM> determines the fingerprint of the finger FG more accurately.

As can be seen from the above, the present application obtains the background image BG_IM and the interfering frequencies fI,<NUM>-fI,N in the offline stage. When the optical fingerprint identification system <NUM> enters the fingerprint identification stage (i.e., on the flight), the computing module <NUM> may subtract the background image BG_IM from the received image RCV_IM to eliminate the lattice interference, and perform the filtering operation at the interfering frequencies fI,<NUM>-fI,N to filter out the interference components, so as to obtain the operational result OUT with clearer fingerprint image, such that the identification module <NUM> may determine the fingerprint of the finger FG accurately.

In summary, the present application is suitable for in-display fingerprint identification, which utilizes subtracting the background image to eliminate the lattice interference, and utilizes filtering at the interfering frequencies to filter out the interference components to filter out the interference components, to obtain the image with clearer fingerprint image, so as to enhance the accuracy of fingerprint identification.

Claim 1:
A fingerprint image processing method, applied in an optical fingerprint identification system of an electronic device, wherein the electronic device comprises a display panel, the display panel comprises a display pixel array, the optical fingerprint identification system comprises an image sensing array, the image sensing array is disposed under the display pixel array, characterized in that, the fingerprint image processing method comprises:
obtaining a first background image and obtaining at least an interfering frequency (fI,<NUM>-fI,N) when the display panel is not pressed by a finger of a user (<NUM>), wherein the step of obtaining the at least an interfering frequency comprises:
obtaining a second background image, wherein no fingerprint image is included in the first background image and the second background image ;
performing a subtracting operation and a transforming operation on the first background image and the second background image, to obtain a difference image spectrum (Diff_IM_SPM), wherein the transformation operation is a Fast Fourier Transform; and
performing an energy detection operation on the difference image spectrum, to derive the at least an interfering frequency from energy spectrum of the difference image spectrum;
receiving a received image when the display panel is pressed by the finger of the user (<NUM>);
performing a subtracting operation on the received image and the first background image, to obtain a difference image (<NUM>); and
performing a filtering operation on the difference image at the at least an interfering frequency to filter out the interference, to obtain an operational result (<NUM>);
wherein the optical fingerprint identification system determines a fingerprint within the received image according to the operational result.