Patent Description:
Biometrics-based authentication technology relates to user authentication using a fingerprint, an iris, a voice, a face, blood vessels, and the like which are individually unique to a user. Such biological characteristics used for the authentication differ from individual to individual, rarely change during a lifetime, and have a low risk of being stolen or copied. In addition, individuals do not need to intentionally carry such characteristics, and thus may not experience an inconvenience using the biological characteristics. In such an authentication technology, a fingerprint recognition method is very commonly used for various reasons, for example, a high level of convenience, security, and economic efficiency. The fingerprint recognition method may reinforce security of a user device and readily provide various application services such as mobile payment.

The following prior art is acknowledged:.

An aspect of the disclosure is to provide a fingerprint recognition method as set out in the appended claims.

Another aspect of the disclosure is to provide a fingerprint recognition apparatus as set out in the appended claims.

The above and other features and advantages of example embodiments of the inventive concepts will become more apparent by describing in detail example embodiments of the inventive concepts with reference to the attached drawings. The accompanying drawings are intended to depict example embodiments of the inventive concepts and should not be interpreted to limit the intended scope of the claims.

Detailed example embodiments of the inventive concepts are disclosed herein. However, specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments of the inventive concepts. Example embodiments of the inventive concepts may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.

Accordingly, while example embodiments of the inventive concepts are capable of various modifications and alternative forms, embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit example embodiments of the inventive concepts to the particular forms disclosed, but to the contrary, example embodiments of the inventive concepts are to cover all modifications, equivalents, and alternatives falling within the scope of example embodiments of the inventive concepts. Like numbers refer to like elements throughout the description of the figures.

For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments of the inventive concepts.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element or intervening elements may be present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., "between" versus "directly between", "adjacent" versus "directly adjacent", etc.).

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments of the inventive concepts. It will be further understood that the terms "comprises", "comprising,", "includes" and/or "including", when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Example embodiments of the inventive concepts are described herein with reference to schematic illustrations of idealized embodiments (and intermediate structures) of the inventive concepts. Thus, example embodiments of the inventive concepts should not be construed as limited to the particular shapes of regions illustrated herein but are to include deviations in shapes that result, for example, from manufacturing.

Although corresponding plan views and/or perspective views of some cross-sectional view(s) may not be shown, the cross-sectional view(s) of device structures illustrated herein provide support for a plurality of device structures that extend along two different directions as would be illustrated in a plan view, and/or in three different directions as would be illustrated in a perspective view. The two different directions may or may not be orthogonal to each other. The three different directions may include a third direction that may be orthogonal to the two different directions. The plurality of device structures may be integrated in a same electronic device. For example, when a device structure (e.g., a memory cell structure or a transistor structure) is illustrated in a cross-sectional view, an electronic device may include a plurality of the device structures (e.g., memory cell structures or transistor structures), as would be illustrated by a plan view of the electronic device. The plurality of device structures may be arranged in an array and/or in a two-dimensional pattern.

Example embodiments described herein may be used for recognizing a fingerprint of a user. The recognizing of the fingerprint of the user may include authenticating or identifying the user. The authenticating of the user may include, for example, determining whether the user is an enrolled user. A result of the authenticating may be output as true or false. The identifying of the user may include, for example, determining a user corresponding to the user among a plurality of enrolled users. A result of the identifying may be output as, for example, an identity (ID) of the determined enrolled user. When the user does not correspond to any one of the enrolled users, a signal indicating that the user is not identified may be output.

Example embodiments described herein may be implemented by a product in various forms example of which include, but are not limited to, a personal computer (PC), a laptop computer, a tablet computer, a smartphone, a television (TV), a smart home appliance, an intelligent vehicle, a kiosk, and a wearable device. For example, example embodiments described herein may be applied to authenticate a user in, for example, a smartphone, a mobile device, and a smart home system. In addition, example embodiments described herein may be applied to a payment service provided through user authentication. Further, example embodiments described herein may also be applied to an intelligent automobile system that automatically starts a vehicle through user authentication. Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings.

<FIG> and <FIG> illustrate examples of a fingerprint image according to at least one example embodiment.

Referring to <FIG>, a fingerprint sensor (not shown) may sense a fingerprint <NUM> of a user. An example of a fingerprint recognition apparatus including a fingerprint sensor <NUM> will be discussed in greater detail below with reference to <FIG> and <FIG>. The fingerprint sensor (e.g., the fingerprint sensor <NUM>) may sense the fingerprint <NUM> through a sensing region. Here, a size of the sensing region of the fingerprint sensor may be smaller than a size of the fingerprint <NUM>. For example, the sensing region of the fingerprint sensor may have a rectangular form smaller than the size of the fingerprint <NUM>. In such an example, the fingerprint sensor may sense a portion of the fingerprint <NUM> through the sensing region.

The fingerprint sensor may generate a fingerprint image by capturing the sensed portion of the fingerprint <NUM>. When the size of the sensing region of the fingerprint sensor is smaller than the size of the fingerprint <NUM>, the fingerprint image generated by the fingerprint sensor may correspond to a partial image including the portion of the fingerprint <NUM>. Further, according to at least some example embodiments, it is also possible for the sensing region of the fingerprint sensor to be the same size or larger than a size of fingerprint <NUM>, such that the fingerprint image generated by the fingerprint sensor may correspond to a complete image of the fingerprint <NUM>.

The fingerprint image may be used to enroll or recognize the fingerprint <NUM> of the user. For example, the fingerprint image may be enrolled in an enrollment method. The enrolled fingerprint image may be stored in, for example, memory or prearranged storage. When the size of the sensing region of the fingerprint sensor is smaller than the size of the fingerprint <NUM>, a plurality of fingerprint images corresponding to partial images of the fingerprint <NUM> of the user may be enrolled. For example, referring to <FIG>, partial images <NUM> through <NUM> may be enrolled. Each of the partial images <NUM> through <NUM> may cover a portion of the fingerprint <NUM>, and the partial images <NUM> through <NUM> in combination may entirely cover the fingerprint <NUM>. Here, the partial images <NUM> through <NUM> may overlap one another. Hereinafter, a partial image of an enrolled fingerprint will be referred to as an enrolled partial image for ease of description.

In addition, an input fingerprint image may be recognized in a recognition method. For example, the recognition method includes performing a comparison of the input fingerprint image and an enrolled fingerprint image. A result of authenticating or identifying a user may be obtained based on whether the input fingerprint image matches the enrolled fingerprint image. Here, when the size of the sensing region of the fingerprint sensor is smaller than the size of the fingerprint <NUM>, the input fingerprint image may correspond to a partial image of the fingerprint <NUM> of the user. Hereinafter, a partial image of a fingerprint of a user will be referred to as an input partial image for ease of description. Although described hereinafter, example embodiments provide a method of recognizing a fingerprint, which will be referred to as a fingerprint recognition method for simplicity, including comparing an input partial image to enrolled partial images.

Although the sensing region of the fingerprint sensor is illustrated as a rectangular form in <FIG>, various sizes and forms may be applicable to the sensing region. For example, the sensing region may be provided in a circular form as illustrated in <FIG>. Referring to <FIG>, in the enrollment method, a plurality of partial images <NUM> through <NUM> corresponding to a single fingerprint <NUM> may be enrolled. The recognition method includes performing a comparison of a fingerprint image corresponding to a portion of the fingerprint <NUM> and the enrolled partial images <NUM> through <NUM>.

According to at least some example embodiments, a fingerprint sensor used in the enrollment method may differ from a fingerprint sensor used in the recognition method. For example, a fingerprint sensor having a rectangular-shaped sensing region as illustrated in <FIG> may be used in the enrollment method, and a fingerprint sensor having a circular-shaped sensing region as illustrated in <FIG> may be used in the recognition method, or vice-versa. Alternatively, the same fingerprint sensor may be used in both the enrollment method and the recognition method.

<FIG> and <FIG> illustrate a fingerprint recognition method according to at least one example embodiment. Referring to <FIG>, a fingerprint recognition apparatus <NUM> includes a fingerprint sensor <NUM>. A size of a sensing region of the fingerprint sensor <NUM> may be smaller than a size of a fingerprint of a user. The fingerprint recognition apparatus <NUM> may obtain an input partial image <NUM> through the fingerprint sensor <NUM>. The fingerprint recognition apparatus <NUM> may obtain enrolled partial images, for example, an enrolled partial image <NUM>, an enrolled partial image <NUM>, and an enrolled partial image <NUM>, from a database <NUM>. According to at least some example embodiments, the database <NUM> may be a prearranged database. The database <NUM> may be stored in a memory (not shown) included in the fingerprint recognition apparatus <NUM>, or in an external device (not shown), for example, a server, that may be connected to the fingerprint recognition apparatus <NUM> in wire or wirelessly, or through a network. According to at least some example embodiments, the size and shape of an input partial image may be the same as the size of enrolled partial images. For example, the input partial image <NUM> may have the same size and shape as enrolled partial images <NUM>-<NUM> if the input partial images <NUM> and the enrolled partial images <NUM>-<NUM> are both captured using the fingerprint sensor <NUM>. The fingerprint recognition apparatus <NUM> may recognize the fingerprint of the user by comparing the input partial image <NUM> to the enrolled partial images <NUM> through <NUM>. Referring to <FIG>, the fingerprint recognition apparatus <NUM> may match the input partial image <NUM> to the enrolled partial image <NUM> to compare the input partial image <NUM> to the enrolled partial image <NUM>. For example, the fingerprint recognition apparatus <NUM> may scale, rotate, and/or translate the input partial image <NUM> to overlap a region of the input partial image <NUM> and a region of the enrolled partial image <NUM> which are included in both the input partial image <NUM> and the enrolled partial image <NUM>. However, each of the input partial image <NUM> and the enrolled partial image <NUM> is a partial image and thus, a size of the overlapping region may be considerably smaller than a size of each of the input partial image <NUM> and the enrolled partial image <NUM>. For example, overlapping region <NUM> is the region where partial image <NUM> and partial image <NUM> overlap. As is illustrated in <FIG>, the size of overlapping region <NUM> may be relatively small in comparison to the full size of partial image <NUM>. In such an example, matching the input partial image <NUM> and the enrolled partial image <NUM> may be ineffective.

In addition, the input partial image <NUM> may include a portion deformed by various factors. For example, a fingerprint image may be deformed by a pressure applied to a sensor. When the input partial image <NUM> is generated, a pressure may be differently applied to each portion of a sensing region of a fingerprint sensor. Thus, at least a portion of the input partial image <NUM> may be deformed. In addition, the enrolled partial images <NUM> through <NUM> may include a portion deformed by various factors. In such a case, comparing the input partial image <NUM> to the enrolled partial images <NUM> through <NUM> may reduce reliability of fingerprint recognition.

<FIG> illustrates a fingerprint recognition method including partitioning an input partial image into blocks and comparing the blocks to enrolled partial images according to at least one example embodiment. Referring to <FIG>, the fingerprint recognition apparatus <NUM> of <FIG> partitions the input partial image <NUM> into blocks, for example, a block <NUM>, a block <NUM>, a block <NUM>, and a block <NUM>. The partitioning of the input partial image <NUM> into the blocks <NUM> through <NUM> may also be referred to as block partitioning. The fingerprint recognition apparatus <NUM> compare the blocks <NUM> through <NUM> to the enrolled partial images <NUM> through <NUM> of <FIG>, rather than compare the input partial image <NUM> to the enrolled partial images <NUM> through <NUM>. As <FIG> illustrates, the fingerprint recognition apparatus <NUM> may partition an input partial image into blocks such that each of the blocks is smaller, in area, than the input partial image.

The partitioning (or block partitioning) of an image, as discussed in the present disclosure, may refer to one or both of an operation of forming several different blocks of an image that may overlap each other (as is shown in <FIG>), and an operation of forming several different non-overlapping blocks of the image that may or may not be adjacent to each other.

The fingerprint recognition apparatus <NUM> may partition the input partial image <NUM> using various methods. For example, the fingerprint recognition apparatus <NUM> may partition the input partial image <NUM> based on a desired or, alternatively, predetermined pattern. The pattern may be determined in advance based on a shape and a size of a sensing region of a fingerprint sensor, a shape and a size of enrolled partial images, and the like. As necessary, the pattern may change dynamically. In addition, the partitioning may be performed to allow blocks to overlap one another, or portions of the blocks to overlap one another.

The fingerprint recognition apparatus <NUM> may recognize a fingerprint through block pattern matching. The block pattern matching may include pattern matching of partial fingerprint images. Although described hereinafter, the fingerprint recognition apparatus <NUM> may partition a fingerprint image input through the fingerprint sensor into a plurality of blocks, perform frequency-based matching, arrange matching scores of the blocks, and determine whether to authenticate a user using a feature value of top K matching scores among the arranged matching scores. The feature value may be a value indicating a feature of the top K matching scores, and include a statistical value, for example, an average. The fingerprint recognition apparatus <NUM> may recognize the fingerprint irrespective of a direction of a finger when the fingerprint image is sensed.

The fingerprint recognition apparatus <NUM> may improve efficiency of the matching by using the blocks <NUM> through <NUM>. As is illustrated in <FIG>, overlapping region <NUM> is the region where block <NUM> and partial image <NUM> overlap, and overlapping region <NUM> is the region where block <NUM> and partial image <NUM> overlap. When the input partial image <NUM> is input, an overlapping region between the input partial image <NUM> and the enrolled partial images <NUM> through <NUM> may not be large and thus, performing the matching by partitioning the input partial image <NUM> into the blocks <NUM> through <NUM> may be effective. For example, the fingerprint recognition apparatus <NUM> may calculate a rotation angle and a translation of each block with respect to the enrolled partial images <NUM> through <NUM> based on image frequency information. Here, a proportion of a region included in both the block <NUM> and the enrolled partial image <NUM>, hereinafter referred to as an overlapping region, to the block <NUM> may be greater than a proportion of the overlapping region to the input partial image <NUM>. For example, the sizes of overlapping regions <NUM> and <NUM> may be relatively large in comparison to the full sizes of partial images <NUM> and <NUM>, respectively. Accordingly, a ratio of the size of overlapping region <NUM> to the size of block <NUM> may be larger than a ratio of the size of overlapping region <NUM> to the size of partial image <NUM> in <FIG>. Similarly, a ratio of the size of overlapping region <NUM> to the size of block <NUM> may be larger than a ratio of the size of overlapping region <NUM> to the size of partial image <NUM> in <FIG>. Thus, the matching may be performed more effectively.

In addition, using the blocks <NUM> through <NUM>, the fingerprint recognition apparatus <NUM> may operate robustly against a deformation that may be included in the input partial image <NUM> or the enrolled partial images <NUM> through <NUM>. For example, the fingerprint recognition apparatus <NUM> may use only a block that suitably matches to the enrolled partial images <NUM> through <NUM> among the blocks <NUM> through <NUM>. The fingerprint recognition apparatus <NUM> may exclude a result of a comparison performed using a deformed block, and only use a result of a comparison performed using a non-deformed block. Thus, fingerprint recognition robust against a deformation may be performed.

In an enrollment method, the fingerprint recognition apparatus <NUM> may store only enrolled partial images, and may not store additional information, for example, information about stitching the enrolled partial images and information about matching the enrolled partial images. Thus, technology having a low operation complexity and effectively using a memory may be provided when enrolling the partial images.

The fingerprint recognition apparatus <NUM> may match the blocks <NUM> through <NUM> to the enrolled partial images <NUM> through <NUM> using various methods. For example, the fingerprint recognition apparatus <NUM> may determine translation information, rotation information, scale information, and various combinations thereof between the blocks <NUM> through <NUM> and the enrolled partial images <NUM> through <NUM> based on a frequency-based matching method. The frequency-based matching method may be a method of performing matching in a frequency domain.

Translation information between a block and an enrolled partial image may include a parameter Tx indicating a translation in an x axis and a parameter Ty indicating a translation in an y axis. Rotation information between a block and an enrolled partial image may include a rotation parameter R. Scale information between a block and an enrolled partial image may include a scale parameter S. Hereinafter, Tx and Ty will be referred to as a translation, and R will be referred to as a rotation angle. The fingerprint recognition apparatus <NUM> may calculate a rotation angle, a translation, and a scale parameter by comparing the blocks <NUM> through <NUM> to the enrolled partial images <NUM> through <NUM> in the frequency domain. A method of calculating a rotation angle, a translation, and a scale parameter in a frequency domain will be described in greater detail below with reference to <FIG>.

The fingerprint recognition apparatus <NUM> may translate and rotate a block based on the translation information obtained as a result of the matching. The fingerprint recognition apparatus <NUM> may scale the block up or down based on the scale information obtained as the result of the matching. The translation information, the rotation information, and the scale information may be relative information between a block and an enrolled partial image and thus, the fingerprint recognition apparatus <NUM> may translate, rotate, scale up, or scale down the enrolled partial image in lieu of the block.

When a block and an enrolled partial image overlap due to a translation, rotation, and scaling, the fingerprint recognition apparatus <NUM> may calculate a matching score in the overlapping region. For example, the fingerprint recognition apparatus <NUM> may calculate the matching score based on normalized correlation based on an image brightness value. As described with reference to <FIG>, the fingerprint recognition apparatus <NUM> may accurately perform matching on a rotated input fingerprint image. Thus, when a fingerprint image is input at an angle, the fingerprint recognition apparatus <NUM> may accurately recognize a fingerprint of the fingerprint image.

<FIG> is a flowchart illustrating an example of a fingerprint recognition method according to at least one example embodiment. Referring to <FIG>, the fingerprint recognition method includes operation <NUM> of receiving an input partial image, operation <NUM> of partitioning the input partial image into blocks, operation <NUM> of comparing the blocks to enrolled partial images, and operation <NUM> of recognizing a fingerprint of a user.

Operation <NUM> of receiving the input partial image may further include performing preprocessing. The preprocessing may include, for example, a series of operations performed to improve a quality of a fingerprint image. The fingerprint image may include the input partial image or the enrolled partial images. For example, the preprocessing may include eliminating noise from the fingerprint image, increasing a contrast of the fingerprint image, deblurring the fingerprint image to remove a blur from the fingerprint image, and warping performed to correct a distortion included in the fingerprint image.

In addition, the preprocessing may include evaluating the quality of the fingerprint image. For example, when the quality of the fingerprint image is less than a threshold quality, the preprocessing may include discarding the obtained fingerprint image and receiving a new fingerprint image. According to at least some example embodiments, the threshold quality may be set in accordance with the preferences of a user and/or manufacturer of the fingerprint authentication device <NUM>.

Descriptions provided with reference to <FIG> may be applicable to the operations described with reference to <FIG>. Hereinafter, operations <NUM> and <NUM> will be further described in detail with reference to <FIG> and <FIG>.

<FIG> is a flowchart illustrating an example of a user authentication method according to at least one example embodiment. Referring to <FIG>, in operation <NUM>, a fingerprint recognition apparatus (e.g., the fingerprint recognition apparatus <NUM>) partitions the input partial image into N blocks. Here, N is a positive integer greater than or equal to <NUM>. In operation <NUM>, the fingerprint recognition apparatus obtains L enrolled partial images from a database <NUM>. Here, L is a positive integer greater than or equal to <NUM>. In operation <NUM>, the fingerprint recognition apparatus matches, to the L enrolled partial images, a block currently being processed among the N blocks. In an example, the fingerprint recognition apparatus matches a block to an enrolled partial image using a frequency-based matching method.

<FIG> is a diagram illustrating an example of a frequency-based matching method according to at least one example embodiment. Referring to <FIG>, in operation <NUM>, information in a time domain, hereinafter simply referred to as time-domain information, included in a block is transformed to information in a frequency domain, hereinafter simply referred to as frequency-domain information, through a fast Fourier transform (FFT). In operation <NUM>, an FFT is applied to an enrolled partial image. Here, the frequency-domain information may be based on an orthogonal coordinates system which expresses information using (x, y) coordinates.

In operation <NUM>, a coordinates system of frequency-domain information included in the block is transformed to a polar coordinates system through a log-polar transform (LPT). In an example, the LPT may be performed on magnitude values of pixels in an FFT image obtained through the FFT. The polar coordinates system may express information using a radius, an angle, or a combination of a radius and an angle. In operation <NUM>, the LPT is applied to frequency-domain information included in the enrolled partial image.

<FIG> illustrates an example of an LPT according to at least one example embodiment. Referring to <FIG>, in an orthogonal coordinates system, concentric circles are set based on a central point <NUM>. The concentric circles may be divided into areas based on a radius, an angle, or a combination of a radius and an angle. In an example, the LPT may map the areas in the orthogonal coordinates system to areas in a polar coordinates system of a radius and an angle. In such an example, the central point <NUM> in the orthogonal coordinates system may be mapped to a (<NUM>, <NUM>°) area <NUM> in the polar coordinates system. In addition, an area <NUM>, an area <NUM>, an area <NUM>, and an area <NUM> in the orthogonal coordinates system may be mapped to an area <NUM>, an area <NUM>, an area <NUM>, and an area <NUM>, respectively.

Although not illustrated, the LPT may map areas in the orthogonal coordinates system to areas in the polar coordinates system of a radius. In such a case, the area <NUM> in the orthogonal coordinates system may be mapped to a (<NUM>°) area in the polar coordinates system. The area <NUM> and the area <NUM> in the orthogonal coordinates system may be mapped to a (<NUM>°) area in the polar coordinates system. The area <NUM> in the orthogonal coordinates system may be mapped to a (<NUM>°) area in the polar coordinates system.

Referring back to <FIG>, in operation <NUM>, the FFT is applied to an LPT image of the block. In operation <NUM>, the FFT is applied to an LPT image of the enrolled partial image. In operation <NUM>, a phase correlation is performed. A peak is detected as a result of the phase correlation, and a location of the detected peak may indicate rotation information, for example, Θ between the block and the enrolled partial image.

In another example, the location of the detected peak may indicate scale information between the block and the enrolled partial image. For example, one axis of an LPT image corresponds to an angle, and the other axis corresponds to a radius. In such an example, a location of a peak detected through the phase correlation may be expressed as a coordinate of the axis corresponding to an angle and as a coordinate of the axis corresponding to a radius. The coordinate of the axis corresponding to an angle may indicate the rotation information, and the coordinate of the axis corresponding to a radius may indicate the scale information.

In general, a fingerprint image may not have a substantial scale change, and thus a radius may be fixed as a preset value, for example, <NUM>. In such a case, a location of a peak detected through the phase correlation may be expressed as the coordinate of the axis corresponding to an angle. The coordinate of the axis corresponding to an angle may indicate the rotation information.

In operation <NUM>, the block is rotated based on the rotation information. In operation <NUM>, the FFT is applied to the rotated block. In operation <NUM>, the phase correlation is performed. A location of a peak detected as a result of the phase correlation may indicate translation information, for example, (Tx, Ty), between the block and the enrolled partial image. In operation <NUM>, the rotated block is translated based on the translation information.

Referring back to <FIG>, in operation <NUM>, the fingerprint recognition apparatus calculates L block scores indicating a degree of matching of the block currently being processed to the L enrolled partial images based on a result of the matching. As described with reference to <FIG>, the fingerprint recognition apparatus may rotate and translate the block currently being processed based on a pair of translation information and rotation information with respect to a first enrolled partial image to overlap the block currently being processed and the first enrolled partial image.

The fingerprint recognition apparatus may calculate a block score in an overlapping region. The fingerprint recognition apparatus may calculate a block score using various methods. For example, the fingerprint recognition apparatus may calculate a block score based on a normalized cross correlation method based on an image brightness value. For example, the fingerprint recognition apparatus may calculate a block score based on Equation <NUM> which defines, for example, a normalized cross correlation (ncc) function.

In Equation <NUM>, "W" denotes an overlapping region between an image I<NUM> and an image I<NUM>. The image I<NUM> is a rotated block and the image I<NUM> is an enrolled partial image. The term "i" denotes an X-axis coordinate of a pixel in the overlapping region, and the term "j" denotes a Y-axis coordinate of the pixel in the overlapping region. The term "x" denotes translation information in an X-axis direction, for example, Tx, and the term "y" denotes translation information in a Y-axis direction, for example, Ty. The expression "I<NUM>(i, j)" denotes a pixel value (e.g., a pixel brightness value) on (i, j) coordinates of the image I<NUM>. The expression "I<NUM>(x + i, y + j)" denotes a pixel value (e.g., a pixel brightness value) on (x + i, y + j) coordinates of the image I<NUM>.

<FIG> illustrates an example of an operation of processing an enrolled image and an input image to calculate a block score according to at least one example embodiment. Referring to <FIG>, an enrolled image <NUM> is transformed to a first LPT image <NUM> through an FFT and an LPT. A block <NUM> of an input image is transformed to a second LPT image <NUM> through an FFT and an LPT.

Rotation information, for example, Θ, between the enrolled image <NUM> and the block <NUM> is determined through a phase correlation <NUM> between the first LPT image <NUM> and the second LPT image <NUM>. The block <NUM> is rotated based on the determined rotation information. Translation information, for example, (Tx, Ty), between the enrolled image <NUM> and the block <NUM> is determined through a phase correlation <NUM> between an FFT image of the enrolled image <NUM> and an FFT image of the rotated block <NUM>.

According to at least some example embodiments, matching of the enrolled image <NUM> and the block <NUM> is performed based on the rotation information and the translation information. A score <NUM> in an overlapping region between the enrolled image <NUM> and the rotated block <NUM> of the matching image <NUM> is calculated. The score <NUM> may be also referred to as a block score and a matching score.

Referring back to <FIG>, the fingerprint recognition apparatus may calculate block scores with respect to second through L-th enrolled partial images. The block score may also be referred to as a matching score.

In operation <NUM>, the fingerprint recognition apparatus performs a verification operation to determine whether or not a block most recently processed is a last block among the N blocks. For example, when the processed block is not the last block among the N blocks, the fingerprint recognition apparatus may perform operation <NUM> on a block yet to be processed. The fingerprint recognition apparatus may calculate N × L block scores by repetitively perform operations <NUM> and <NUM> N times.

In operation <NUM>, the fingerprint recognition apparatus selects top K block scores from among the N × L block scores. As described with reference to <FIG>, the input partial image <NUM> and the enrolled partial image <NUM> may partially overlap. In such a case, a block located in the overlapping region among the blocks of the input partial image <NUM> may obtain a significant block score, and a block located in a non-overlapping region may obtain an insignificant block score. Thus, the fingerprint recognition apparatus may exclude the insignificant block score by selecting the top K block scores from among the N × L block scores. Here, K may be determined within a range less than or equal to N × L.

The fingerprint recognition apparatus may calculate a feature value based on the top K block scores. For example, the fingerprint recognition apparatus may calculate a sum of the top K block scores using Equation <NUM>.

In Equation <NUM>, the term "↓ Score(Bi)" denotes an i-th block score arranged in a descending order. A method of calculating a feature value (Val) may be variously modified, for example, obtaining an average of the top K block scores. In addition, different weights may be applied to respective block scores to calculate the feature value based on a calculated desired or, alternatively, optimal rotation angle at which a block has a highest block score.

In operation <NUM>, the fingerprint recognition apparatus performs user authentication by comparing the feature value to a threshold value. For example, when the feature value is greater than the threshold value, the fingerprint recognition apparatus determines the authentication to be successful in operation <NUM>. Conversely, when the feature value is less than or equal to the threshold value, the fingerprint recognition apparatus determines the authentication to be a failure in operation <NUM>. The threshold value may be determined based on a method of calculating the feature value, the number of enrolled partial images, a quality of the enrolled partial images, and a quality of the input partial image.

<FIG> is a flowchart illustrating another example of a user authentication method according to at least one example embodiment.

A fingerprint recognition apparatus (e.g., the fingerprint recognition apparatus <NUM>) may calculate a rotation angle of each block with respect to enrolled partial images, and calculate desired or, alternatively, optimal rotation angles with respect to the enrolled partial images. The fingerprint recognition apparatus may identically apply a desired or, alternatively, optimal rotation angle corresponding to each enrolled partial image to all the blocks, and obtain a matching score by calculating a translation between a rotated block and an enrolled partial image.

An operation of the fingerprint recognition apparatus may be divided into two phases. In a first phase, the fingerprint recognition apparatus matches one enrolled partial image to N blocks of an input partial image (e.g., using the matching method illustrated in <FIG>) and obtains N matching scores. The fingerprint recognition apparatus may identically apply, to all the blocks, a rotation angle of a block having a highest matching score among the N matching scores obtained as a result of the matching. The block having a highest matching score among the N matching scores may be, for example, a block having a highest degree of similarity with respect to the enrolled image among the N blocks. In a second phase, the fingerprint recognition apparatus may calculate a translation of each block.

Although described in detail hereinafter, the fingerprint recognition apparatus may select enrolled partial images corresponding to top M matching scores based on the matching scores in the first phase, and calculate a translation only using the selected enrolled partial images in the second phase to effectively increase a processing speed.

Referring to <FIG>, in operation <NUM>, the fingerprint recognition apparatus obtains L enrolled partial images from a database <NUM>. In operation <NUM>, the fingerprint recognition apparatus matches N blocks to the L enrolled partial images. For example, in operation <NUM>, a matching operation may be performed for each enrolled partial image of the L enrolled partial images. For example, for a first enrolled partial image from among the L enrolled partial images, a fingerprint recognition apparatus matches the first enrolled partial image to N blocks of an input partial image (e.g., using the matching method illustrated in <FIG>) and obtains N first block scores for the first enrolled image as a result of the matching. In operation <NUM>, the same matching operation discussed above with respect to the first enrolled partial image is performed for the remaining enrolled partial images from among the L enrolled partial images. As a result, the fingerprint recognition apparatus calculates, for example, N × L block scores, based on a result of the matching. Descriptions of operations <NUM> and <NUM> provided with reference to <FIG> may be applicable to operation <NUM>.

In operation <NUM>, the fingerprint recognition apparatus determines a desired or, alternatively, optimal rotation angle (R) for each enrolled partial image. The desired or, alternatively, optimal rotation angle for each enrolled partial image may be an angle used to rotate the blocks in response to the enrolled partial images. Further, in operation <NUM>, the fingerprint recognition apparatus may select a highest block score from among first N block scores determined for the first enrolled image during the matching operation that was performed for the first enrolled image during operation <NUM>. The fingerprint recognition apparatus may extract a rotation angle from matching information in which the selected highest block score is calculated. For example, in operation <NUM>, the fingerprint recognition apparatus may extract, for the first enrolled partial image, a rotation angle R. The rotation angle R extracted for the first enrolled partial image is the rotation angle of the block that was determined to have the highest block score during the matching that was performed for the first enrolled image during operation <NUM>. The matching information may be a pair including the rotation angle R and a translation of the block that was determined to have the highest block score during the matching operation that was performed for the first enrolled image during operation <NUM>. The fingerprint recognition apparatus may determine the extracted rotation angle R to be the desired or, alternatively, optimal rotation angle to use for rotating all N blocks with respect to the first enrolled partial image. In operation <NUM>, the fingerprint recognition apparatus may use the same process described above with respect to the first enrolled partial image of the L enrolled partial images to determine a desired or, alternatively, optimal rotation angle R for each of the remaining enrolled partial images of the L enrolled partial images. Thus, in operation <NUM>, the fingerprint recognition apparatus may determine L optimal rotation angles R for L enrolled images, respectively.

In operation <NUM>, the fingerprint recognition apparatus rotates the N blocks based on the desired or, alternatively, optimal rotation angle R determined in operation <NUM> for each of the L enrolled partial images, respectively. For example, in operation <NUM>, the fingerprint recognition apparatus may rotate the N blocks by the desired or, alternatively, optimal rotation angle R determined in operation <NUM> for the first enrolled partial image of the L enrolled partial images. Also, in operation <NUM>, the fingerprint recognition apparatus may rotate the N blocks by the desired or, alternatively, optimal rotation angles R determined in operation <NUM> for the remaining enrolled partial images of the L enrolled partial images, respectively. In operation <NUM>, the fingerprint recognition apparatus compares the blocks rotated in response to the enrolled partial images to the enrolled partial images. For example, the fingerprint recognition apparatus may re-match the blocks rotated in response to each of the enrolled partial images to a corresponding enrolled partial image using a frequency-based matching method. According to at least some example embodiments, the re-matching performed by the fingerprint recognition apparatus in operation <NUM> may include operations <NUM>-<NUM> of <FIG>, and may exclude operations <NUM>-<NUM>. Here, the fingerprint recognition apparatus may determine translations of the blocks while maintaining the rotation angles of the blocks. The fingerprint recognition apparatus may calculate second block scores based on a result of the re-matching in operation <NUM>. According to at least some example embodiments, the re-matching performed by the fingerprint recognition apparatus in operation <NUM> may be performed for each of the L enrolled partial images.

According to at least some example embodiments, operations <NUM>, <NUM> and <NUM> may be performed with respect to all L enrolled partial images in the manner discussed above. However, as will be discuss in greater detail below, according to at least some example embodiments, an additional ranking operation may be performed in operation <NUM>, and operations <NUM>, <NUM> and <NUM> may be performed for less than all L enrolled partial images. For example, the fingerprint recognition apparatus may not use all the L enrolled partial images to calculate the second block scores in operation <NUM>. In operation <NUM>, the fingerprint recognition apparatus arranges the L enrolled partial images based on the first block scores (e.g., the N block scores determined for each of the L enrolled partial images in operation <NUM>). For example, the fingerprint recognition apparatus may arrange the enrolled partial images, starting from an enrolled partial image relating to a highest first block score. For example, the fingerprint recognition apparatus may rank the L enrolled partial images from highest to lowest based on first block scores determined in operation <NUM>. For example, the fingerprint recognition apparatus may rank the L enrolled partial images from highest to lowest based on block score averages of the L enrolled partial images where, for each enrolled partial image from among the L enrolled images, the block score average of the enrolled partial image is an average of the N block scores determined for the enrolled partial image in operation <NUM>. As another example, the fingerprint recognition apparatus may rank the L enrolled partial images from highest to lowest based on highest block scores of the L enrolled partial images where, for each enrolled partial image from among the L enrolled images, the highest block score of the enrolled partial image is the highest block score from among of the N block scores determined for the enrolled partial image in operation <NUM>. The fingerprint recognition apparatus may select M enrolled partial images based on the arrangement order. For example, the fingerprint recognition apparatus may select the M highest ranked enrolled partial images from among the L enrolled partial images. Here, M is a positive integer less than L. By comparing only the M enrolled partial images selected from among the L enrolled partial images to an input partial image, a processing speed of the fingerprint recognition apparatus may be increased.

In such a case, in operation <NUM>, the fingerprint recognition apparatus calculates desired or, alternatively, optimal rotation angles for the M enrolled partial images (i.e., not for all L enrolled partial images). In operation <NUM>, the fingerprint recognition apparatus rotates the blocks based on a desired or, alternatively, optimal rotation angle for each of the M enrolled partial images (i.e., not for all L enrolled partial images). In operation <NUM>, the fingerprint recognition apparatus calculates second block scores, for example, N × M block scores (i.e., not for all L enrolled partial images).

In operation <NUM>, the fingerprint recognition apparatus selects top K block scores from among the N × M block scores (or, alternatively, the N × L block scores). The fingerprint recognition apparatus calculates a feature value (Val) based on the top K block scores. In operation <NUM>, the fingerprint recognition apparatus determines user authentication by comparing the feature value to a threshold value. For example, when the feature value is greater than the threshold value, the fingerprint recognition apparatus determines the authentication to be successful in operation <NUM>. Conversely, when the feature value is less than or equal to the threshold value, the fingerprint recognition apparatus determines the authentication to be a failure in operation <NUM>.

Although the comparison using a plurality of enrolled images is described in the foregoing, the same authentication method may be applicable to a case in which a single enrolled image is present, L = <NUM>. In such a case, the value of K may be determined in a range of greater than or equal to <NUM> and less than or equal to N. In addition, although an input image and an enrolled image are described as a partial fingerprint image, the same authentication method may be applicable to a case in which the input and the enrolled images are an entire fingerprint image.

<FIG> is a diagram illustrating an example of an electronic system <NUM> according to at least one example embodiment. Referring to <FIG>, the electronic system <NUM> includes a sensor <NUM>, a processor <NUM>, and a memory <NUM>. The sensor <NUM>, the processor <NUM>, and the memory <NUM> may communicate with one another through a bus <NUM>. The processor <NUM> may be, for example, a hardware-implemented data processing device having circuitry that is physically structured to execute desired operations including, for example, operations represented as code and/or instructions included in a program. Examples of the above-referenced hardware-implemented data processing device include, but are not limited to, a microprocessor, a central processing unit (CPU), a processor core, a multi-core processor; a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA). Processors executing program code are programmed processors, and thus, are special-purpose computers. According to at least some example embodiments, the electronic system <NUM> is an example of the fingerprint recognition apparatus <NUM> discussed in <FIG>. For example, any functions described herein as being performed by the fingerprint recognition apparatus <NUM> or a fingerprint recognition apparatus may be performed by the fingerprint recognition apparatus <NUM> including the structure of the electronic system <NUM>. For example, the memory <NUM> may store program code including instructions configured to cause the processor <NUM>, when executing the program code, to perform any or all functions described in the present disclosure as being performed by the fingerprint recognition apparatus <NUM>, a fingerprint recognition apparatus, and/or the electronic system <NUM>.

The sensor <NUM> may be the fingerprint sensor <NUM> illustrated in <FIG>. The sensor <NUM> may capture a fingerprint image using a well-known method, for example, a method of converting a desired or, alternatively, optimal image to an electrical signal. The captured fingerprint image may be output to the processor <NUM>.

The processor <NUM> may include at least one device or unit described with reference to <FIG>, or perform at least one method described with reference to <FIG>. For example, the processor <NUM> may include the fingerprint recognition apparatus <NUM> of <FIG>. The memory <NUM> may store partial images captured by the sensor <NUM> and then enrolled, an input partial image captured by the sensor <NUM>, a result of matching processed by the processor <NUM>, and/or block scores calculated by the processor <NUM>. The memory <NUM> may be a volatile memory or a nonvolatile memory.

The processor <NUM> may execute a program and control the electronic system <NUM>. A program code executed by the processor <NUM> may be stored in the memory <NUM>. The electronic system <NUM> may be connected to an external device, for example, a PC and a network, through an input and output device (not shown), and may exchange data with the external device.

The electronic system <NUM> may be provided in various forms, for example, a mobile device such as a mobile phone, a smartphone, a personal digital assistant (PDA), a tablet computer, and a laptop computer, a computing device such as a PC, a tablet computer, and a netbook, and a television (TV), a smart TV, and a security device for gate control.

Although examples described herein relate to recognizing a user using a portion of a fingerprint of the user, such examples may be further applied to recognizing the user using a portion of biodata of the user. The biodata may include information about the fingerprint, blood vessels, and an iris of the user. In such an application, the processor <NUM> may receive input partial data corresponding to the portion of the biodata of the user, partition the input partial data into blocks, compare the blocks to enrolled partial data sets corresponding to partial data sets of enrolled biodata, and recognize the user based on a result of the comparing.

In an example, the sensor <NUM> may include a sensor configured to recognize a blood vessel pattern of the user. The sensor <NUM> may extract the blood vessel pattern from a dorsal hand skin of the user. The sensor <NUM> may increase or, alternatively, maximize a brightness of blood vessels against a brightness of the skin using an infrared lighting and filter, and obtain an image including the blood vessel pattern. In such an example, the processor <NUM> may recognize the user by comparing a partial image corresponding to a portion of the blood vessel pattern to a partial image corresponding to an enrolled blood vessel pattern.

In another example, the sensor <NUM> may include a sensor configured to recognize an iris pattern of the user. The sensor <NUM> may scan or capture the iris pattern between a pupil and a sclera, which is a white area of an eye, of the user. The sensor <NUM> may obtain a partial image corresponding to a portion of the iris pattern. In such an example, the processor <NUM> may recognize the user by comparing the partial image corresponding to the portion of the iris pattern to a partial image corresponding to an enrolled iris pattern.

The method as used by a competitor may require a large amount of calculation in an enrollment phase because performing geometric matching is necessary to generate such a graph. In addition, the method may use a considerable amount of a memory because all sets of information about a node (for example, an enrolled image) and an edge (for example, a rotation angle, a translation, and a degree of uncertainty) of a graph need to be stored. Further, calculating additional matching scores based on uncertainty may be necessary.

The present disclosure provides a method of partitioning an input image into a plurality of blocks and matching the blocks to enrolled images for accurate authentication because a deformation may occur in a fingerprint image due to a pressure applied when pressing a sensor. When a partial fingerprint image is received as an input, the number of overlapping regions may be small and thus, partitioning the input image into the blocks and performing the matching may be effective. The method may include calculating a rotation angle and a translation of each block with respect to the enrolled images based on image frequency information.

The method may include determining whether the input fingerprint image matches a fingerprint image of an enrolled person using an average value of top K block scores, or top K matching scores, among the matching scores calculated as described in the foregoing (<NUM>) for robustness against erroneous recognition. The average value may differently apply a weight based on a calculated optimal rotation angle which indicates a best matching score with an enrolled image.

The method may include calculating a rotation angle of a block as described in the foregoing (<NUM>), identically applying an optimal rotation angle for each enrolled image to all the blocks, calculating a translation, and obtaining a matching score. That is, the method may include operation <NUM> of identically applying, to all the blocks, a rotation angle of a block having a highest score of similarity among N matching scores obtained by matching one enrolled image to N blocks of the input image. The method may include operation <NUM> of calculating a translation of each block. The present disclosure may include a method of effectively improving a processing speed by calculating the translation in operation <NUM> only using top M enrolled images based on the matching scores obtained in operation <NUM>.

The units and/or modules described herein may be implemented using hardware components and software components. For example, the hardware components may include microphones, amplifiers, band-pass filters, audio to digital convertors, and processing devices. A processing device may be implemented using one or more hardware device configured to carry out and/or execute program code by performing arithmetical, logical, and input/output operations. The processing device(s) may include a processor, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a field programmable array, a programmable logic unit, a microprocessor or any other device capable of responding to and executing instructions in a defined manner. The processing device may run an operating system (OS) and one or more software applications that run on the OS. The processing device also may access, store, manipulate, process, and create data in response to execution of the software. For purpose of simplicity, the description of a processing device is used as singular; however, one skilled in the art will appreciated that a processing device may include multiple processing elements and multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. In addition, different processing configurations are possible, such a parallel processors.

Claim 1:
A fingerprint recognition method, comprising:
- receiving an input partial image of a fingerprint of a first user;
- partitioning the input partial image into a plurality of blocks;
- performing a comparison operation between the plurality of blocks and enrolled partial images of an enrolled fingerprint; and
- recognizing the fingerprint of the first user based on a result of the comparison operation;
wherein the performing the comparison operation comprises:
- matching the plurality of blocks to the enrolled partial images, thereby producing matching information as a result of the matching;
- comparing the plurality of blocks to the enrolled partial images, by using the matching information to calculate scores indicating a degree of the matching between each block of the plurality of blocks and the enrolled partial images,
- selecting first scores from among the calculated scores, a total number of the first scores being less than a total number of the calculated scores, and
- calculating a feature value based on the selected first scores;
wherein the recognizing comprises at least one of:
- authenticating the user if the feature value is greater than a threshold value; or
- identifying the user if the feature value is greater than the threshold value; and
wherein the matching of the plurality of blocks to the enrolled partial images comprises:
- transforming with log-polar transform, LPT, a coordinates system of frequency-domain information included in the blocks and in the enrolled partial images into LPT blocks and LPT images;
- applying a fast-Fourier transform, FFT, to the LPT blocks and LPT images; and
- performing phase correlation on the LPT blocks and LPT images to determine the matching information.