# A Digital Camera-Based Rotation-Invariant Fingerprint Verification Method

Authors: Sajid Khan; Dong-Ho Lee; Asif Khan; Ahmad Waqas; Abdul Rehman Gilal; Zahid Hussain Khand
Journal: Scientific Programming
 (2020)

---

## Abstract

Fingerprint registration and verification is an active area of research in the field of image processing. Usually, fingerprints are obtained from sensors; however, there is recent interest in using images of fingers obtained from digital cameras instead of scanners. An unaddressed issue in the processing of fingerprints extracted from digital images is the angle of the finger during image capture. To match a fingerprint with 100% accuracy, the angles of the matching features should be similar. This paper proposes a rotation and scale-invariant decision-making method for the intelligent registration and recognition of fingerprints. A digital image of a finger is taken as the input and compared with a reference image for derotation. Derotation is performed by applying binary segmentation on both images, followed by the application of speeded up robust feature (SURF) extraction and then feature matching. Potential inliers are extracted from matched features by applying the M-estimator. Matched inlier points are used to form a homography matrix, the difference in the rotation angles of the finger in both the input and reference images is calculated, and finally, derotation is performed. Input fingerprint features are extracted and compared or stored based on the decision support system required for the situation.


---

## Body

## 1. Introduction

Image processing has many applications [1–3], including biometric registration and verification. Biometric registration and verification are performed to restrict unauthorized people from entering a specific building or wing, to keep track of users of service by registering their fingerprints in a database, to take automatic attendance of employees, and to store data along with biometric features in a database for issuing passports or national identity cards.Biometric verification and registration using image processing techniques such as face detection, fingerprint detection, and retina or iris detection have been widely researched [4–8]. Face detection-based techniques [9, 10] store facial features to be compared during the registration and verification process. Face verification and registration provides accuracy at the cost of high complexity and extensive allocation of system memory. Accuracy demands that issues such as scale difference, posture, illumination, and the opening and closing of the eyes and mouth be addressed, further increasing complexity.Iris-based recognition and verification [7, 11, 12], on the other hand, is done by applying segmentation in the region of the iris to extract its radius. This is followed by feature extraction. These techniques are used along with fingerprint verification at airports and other high-risk buildings to provide accurate identification. Iris-based recognition and verification provide good accuracy, but the required equipment is costly.Fingerprint verification and registration is a widely researched area [13–23] in which work has focused on algorithms and techniques to store and compare fingerprint features obtained using a scanner. Fingerprint-based biometric verification and registration are widely used in banks [19, 20] and national databases for maintaining passport and national identity card information [24].By using different approaches, such as combining fingerprint registration with odor analysis [21] and applying a ridgelet transform [22], researchers have improved existing algorithms. Complexity can be reduced by comparing a small number of features instead of entire images. However, these methods require the use of sensors (fingerprint readers) that are not always available. This can make performing the registration and verification process difficult.Solutions that avoid the need of sensors have been proposed [24–27]. These techniques focus on processing images captured using ordinary digital cameras. Hiew et al. [24–26] applied skin color-based segmentation followed by morphological processing to extract regions of interest. Short-time Fourier transform analysis (STFT) is then applied, followed by ridge orientation calculation and core point detection. The detection and matching of features such as bifurcations and ridges are not discussed. However, detection and matching can be performed using the same fingerprint techniques used in the algorithms applied to input obtained from sensors.Sankaran et al. [27] proposed a method similar to that discussed in this paper that performs skin color segmentation followed by the region of interest (ROI) extraction. Image enhancement is applied followed by ScatNet-based feature extraction. Finally, feature-based fingerprint registration/matching is performed.When features are extracted from a digital image of a fingerprint, scanners are not needed. However, orientation alignment is an issue that is inadequately addressed in the algorithms proposed by Hiew et al. and Sankaran et al. The fingerprint in the input image must be vertically aligned when conventional methods are used. When rotation of the finger is present, a fingerprint mismatch will occur even if features of that person are present in the database.Therefore, this paper proposes a rotation and scale-invariant technique that takes an image captured using an ordinary digital camera as the input and applies skin color-based segmentation to the input image and a reference image to extract regions of interest from both images. SURF [28] algorithm is applied to the segmented binary images, and the detected features from both images are matched. Inliers are extracted from the matched features using the M-estimator [29] for accurate calculation of the rotation angle using a homography matrix [30]. The input image is derotated to obtain an image with a vertically aligned finger. The vertically aligned input image is then subjected to image enhancement techniques, such as unsharp masking and contextual blockwise Fourier domain filtering. Otsu thresholding is applied for binarization. Then, nonmaximal suppression is used to obtain ridges having a thickness of one pixel. Features such as bifurcations and ridge endings are extracted from the thin ridges to be stored in a database for registration or compared with the features of registered fingerprints for verification. Simulation results show that the proposed method provides an accuracy of 100 percent if the suggested parameter range is applied.This paper is organized as follows: Section2 discusses the proposed methodology, Section 3 contains the results and discussion, Section 4 presents the conclusions and the direction of future work.


## 2. Methodology

The objective of this work is to propose a cost-effective rotation and scale-invariant alternative to the processing of fingerprint images obtained using scanners. A flowchart of the algorithm is shown in Figure1. The major steps of the proposed algorithm are the derotation of the input image, image enhancement, binarization, thinning, minutiae extraction, and matching. The steps shown in the flowchart are explained in detail in this section.Figure 1
Flowchart of the proposed algorithm.


### 2.1. Image Acquisition

In this study, image acquisition was performed using several smartphones: Samsung Galaxy S2, iPhone 3s, Galaxy S6, and iPhone 8. Images of fingers were captured with the focus on their ridges.


### 2.2. Image Derotation

A flowchart depicting the derotation process is shown in Figure2. An image containing a finger aligned along the y-axis is selected as a reference image, as seen in Figure 3(a). Any input finger image requiring registration or verification is considered an input image and is subjected to the derotation process so that the fingerprint registration/matching process can be done smoothly. Figure 3(b) shows an example of an input image that requires rotational alignment for accurate feature matching.Figure 2
Flowchart of the derotation process.Figure 3
Stages of rotation alignment: (a) reference intensity image, (b) input intensity image, (c) segmented reference image, (d) segmented input image, (e) matched features including outliers, (f) matched inliers, and (g) derotated input image.
(a)(b)(c)(d)(e)(f)(g)


#### 2.2.1. Skin Color-Based Segmentation

Both the input and reference images are subjected to adaptive skin color-based image segmentation [31]. The images shown in Figures 3(c) and 3(d) show the results of this segmentation on the reference and input images. The input image shown in Figure 3(b) is particularly complex because the color of the background is similar to the color of the skin. Therefore, noise is detected as part of the finger. The white region in Figure 3(d) is the region of interest (ROI) and will be used in upcoming sections.


#### 2.2.2. Input Image Scale Adjustment

Both input and reference images usually are of different dimensions. Therefore, the fingers in both images might have a huge scale difference. In that case, scale variance might produce problems while detecting and comparing features. In order to provide a scale-invariant derotation, scales of both input and reference fingers should nearly be the same. If the number of pixels belongs to the finger in the segmented reference image, it is denoted byNref and if the number of pixels belongs to ROI, it is denoted by Ninput. Scaling factor is calculated using (1) as(1)Srefinput=NrefNinput.IfSrefinput<0.9, both the intensity and segmented input images are scaled down by a factor of Srefinput, whereas if Srefinput>1.1, the images are interpolated by a factor of Srefinput using simple bilinear interpolation.


#### 2.2.3. Feature Extraction and Matching

Intensity images are converted into binary images for time-efficient feature matching. Since finger shapes are all similar, features detected along their boundaries can provide better estimations of geometrical differences. For feature detection, the SURF [28] method is applied to the segmented binary images to obtain boundary features for both images. SURF is selected for its robustness and reasonable match rate for two same images with different angle of rotation [29]. After obtaining features, nearest neighbor-based feature matching by computing the pairwise distance between features is performed. Then, feature matching provides matched features that contain truly matched inliers as well as erroneously matched outliers. Figure 3(e) shows all the matched features including both inliers and outliers.


#### 2.2.4. Inlier Extraction and Derotation of Image

To extract inliers from the matched points, an M-estimator [29] algorithm is applied. Figure 3(f) shows the inliers extracted from all matched features using the M-estimator. It can be seen that extracted inliers represent true matches that can be used to estimate the angle of rotation between accurately matched inlier points.Once several potential inliers are extracted, the angle of rotation between the matched inliers must be determined. A homography matrix [30] is calculated for this purpose. For the two sets of matched points, D=xi,yi for the reference image and D′=xi′,yi′ for the input image. Then, the relation between them is as given in the following equation:(2)D′=HD,orH=D′D−1,where(3)D=xiyi1,D=xi′yi′1,The resultant matrix in (3) is of the form given in (4) as(4)H=h11h12h13h21h22h23h31h32h33=εcosθ−sinθtxεsinθcosθty001.Therefore, the angle of rotation can be calculated usingh12 or h22 using (5) as(5)θ=sin−1−h12,θ=cos−1−h22,where θ can be used to derotate the input image to obtain an image with a vertically aligned finger. Figure 3(g) shows the derotation of the input image after the calculation of θ=−176.29°. Figure 4 shows the alignment method applied to various input images showing fingers at different angles of rotation. It can be seen that the proposed algorithm predicted the angles of rotation accurately and that the fingers in the derotated images are aligned vertically irrespective of their initial alignment.Figure 4
Stages of rotation alignment method applied to four images: (a–d) input intensity image, (e–h) segmented input image, (i–l) matched features including outliers, (m–p) matched inliers, and (q–t) derotated input image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)


### 2.3. Image Enhancement

Edge-based image enhancement is applied to the grayscale image to improve the visibility of ridges on the finger. Edge-based enhancement is performed in two stages. First, unsharp masking [32] is applied to the image. This algorithm subtracts an image from a blurred version of itself and enhances those regions which provide a good response after subtraction. To calculate the blurred image, a Gaussian filter is convolved with the original grayscale image using the following equation:(6)Igray∗Gσ,where(7)Gσ=1πσe−x2/2σ2.For the second stage of image enhancement, a contextual blockwise Fourier domain filter [33] is applied to the grayscale image by dividing it into a 16 × 16 overlapping block. These blocks are then filtered in the Fourier domain using a frequency and orientation-selective filter with parameters based on the estimated local ridge orientation and frequency. The outcome of the Fourier domain-based enhancement is another image denoted as Ienhanced. Ienhanced is multiplied with a skin-segmented binary image so that only the region of interest (ROI) is processed further.Figure5 shows the effects of this method on the derotated input image. The image in Figure 5(a) shows ridges that are not visible. The results of the unsharp masking can be seen in Figure 5(b) showing improved visibility of the ridges. Figure 5(c) shows the image once it has been converted to grayscale. Figure 5(d) shows that application of Fourier-based image enhancement further sharpens unclear edges and other details. Figure 5(e) shows the extracted ROI.Figure 5
Application of preprocessing on the fingerprint image: (a) original input image, (b) unsharp masking, (c) grayscale conversion, (d) blockwise Fourier-based enhancement, and (e) extracted region of interest.
(a)(b)(c)(d)(e)


### 2.4. Image Binarization and Edge Thinning

Features such as ridge endings and bifurcations can be accurately extracted when ridges having a thickness of one pixel are detected. To detect thin ridges, all ridges are first detected by applying Otsu thresholding [34] to the enhanced region of interest. This will result in a binary image with ridge pixels represented in white and the remaining pixels in black. However, this will result in the detection of thick ridges. To limit the results to thin ridges, nonmaximal suppression [35] is applied. This method sets all the ridge pixels in a certain direction to zero if they do not have a maximum gradient value and retains ridge pixels with the maximal value. This results in conversion of the thick ridges to ridges with a thickness of only one pixel. Figure 6(a) shows Otsu thresholding being applied to the enhanced region of interest. The detected ridges are thick and thus cannot be used for feature detection. Application of nonmaximal suppression results in more desirable thin ridges, as seen in Figure 6(b).Figure 6
Applying edge thinning operations to an enhanced image: (a) binary segmentation, (b) nonmaximal suppression, and (c) minutiae extraction.
(a)(b)(c)


### 2.5. Minutiae Extraction

Minutiae are features that are used for biometric verification. Instead of storing fingerprint images in the database, minutiae are stored and compared. The most common method used for minutiae extraction is the crossing number (CN) [36].For extracting minutiae, the 3 × 3 window shown in Figure7 is moved across each pixel to calculate its crossing number using the following equation:(8)CNx,y=12∑i=18pi−pi−1,where pi denotes the pixels around the center pixel at location x,y. Using CN, the ridge pixel can then be classified as a ridge ending, bifurcation, or isolated point using Table 1. Figure 6(c) shows bifurcations and ridge endings represented by blue and red squares, respectively.Figure 7
3 × 3 window used in minutiae extraction.Table 1
Ridge pixel classifications using the crossing number.
CNDecision0Isolated point1Ridge ending2Continuing ridge3Bifurcation4CrossingFor each bifurcation and ridge ending, the Euclidean distanceDx,y between its spatial coordinate x,y and xROImin,yROImin, the normalized gradient value Gx,y, and the gradient orientation αx,y are stored in SBFin and SREin, respectively, for storage or comparison. Hence, every bifurcation and ridge ending point is stored in SBFin or SREin in the form of 3D point DBFx,y,GBFx,y,αBFx,y and DREx,y,GREx,y,αREx,y, respectively. The values donated by xROImin and yROImin are x-axis and y-axis points taken from two ROI points corresponding to the minimum x-axis and y-axis values. Figure 8 shows an example of calculation of xROImin,yROImin that is represented by a green circle. This was determined using the minimum x-axis ROI point represented by a blue cross and the minimum y-axis ROI point represented by a red cross.Figure 8
Determination of pointxROImin,yROImin: (a) minimum x-axis and y-axis ROI points and (b)xROImin,yROImin determination using two minimum axis points.
(a)(b)


### 2.6. Fingerprint Registration and Matching

Extracted minutiae are either stored or compared with minutiae already existing in the database as required by the use case. Suppose that the ridge endings extracted from the input image are represented by the setSBFin and that bifurcations are represented by SREin as follows:(9)SREin=DRE1in,GRE1in,∝RE1in,DRE2in,GRE2in,∝RE2in,…,DRENREinin,GRENREinin,∝RENREinin,(10)SBFin=DBF1in,GBF1in,∝RE1in,DBF2in,GBF2in,∝BF2in,…,DBFNBFinin,GBFNBFinin,∝BFNBFinin,where SREin and SBFin are sorted in an ascending order according to the calculated Euclidean distance such that DREkin≤DREk+1in and DBFlin≤DBFl+1in for all k∈1,2,...,NREin and l∈1,2,...,NBFin. These sets must be compared with the stored features of a number of fingerprints NDB stored in the database, as follows:(11)SREDB=SREDB1,SREDB2,…,SREDBNREDB,(12)SBFDB=SBFDB1,SBFDB2,…,SBFDBNBFDB,where(13)SREDBi=DRE1DBi,GRE1DBi,∝RE1DBi,DRE2DBi,GRE2DBi,∝RE2DBi,…,DRENREDBiDBi,GRENREDBiDBi,∝RENREDBiDBi,(14)SBFDBi=DBF1DBi,GBF1DBi,∝RE1DBi,DBF2DBi,GBF2DBi,∝BF2DBi,…,DBFNBFDBiDBi,GBFNBFDBiDBi,∝BFNBFDBiDBi.SREDBi and SBFDBi are also sorted in an ascending order according to the Euclidean distances DREmDBi and DBFnDBi where m∈1,2,....,NREDBi and n∈1,2,....,NBFDBi. Suppose that SREin and SBFin are to be compared with the stored features of the ith fingerprint having feature sets SREDBi and SBFDBi. First, NminRE, NminBF, NmaxRE, and NmaxBF are to be determined using the following equations:(15)NminRE=minNREin,NREDBi,(16)NminBF=minNBFin,NBFDBi,(17)NmaxRE=maxNREin,NREDBi,(18)NmaxBF=maxNBFin,NBFDBi,where NREin and NREDBi are the number of ridge endings extracted from the input image and the ith image features stored in the database and NBFin and NBFDBi are the number of extracted bifurcations. The input fingerprint features are compared with the ith fingerprint features if the condition in (19) is justified:(19)NminRE≥0.8NmaxRE,NminBF≥0.8Nmax.For matching, all 3D bifurcation and ridge ending points of the input fingerprint and the ith fingerprint from the database and input fingerprint are stored inMREref, MBFref, MREin, and MBFin respectively. MREref is compared with MREin, and MBFref is compared with MBFin. The steps for matching are(a)
IfMBFin has fewer bifurcations, store MBFin in MBFmin and MBFref in MBFmax and vice versa. Initialize bifurcation matching score scoreBF to zero.(b)
Compare each bifurcation 3D point inMBFmin at the pth position with all bifurcation points at locations p+t in MBFmax where t∈−5,−4,−3,−2,−1,0,1,2,3,4,5.(c)
IfGp+tmax−Gpmin<T1andαp+tmax−αpmin<T2 is true for any value of t, then increment scoreBF and move to the next bifurcation point for matching. Here, Gpmin and αpmin are the magnitude and orientation of the normalized gradient of bifurcation of MBFmin at position p, and Gp+tmax and αp+tmmax are the magnitude and orientation of MBFmax at any of the eleven locations in Step b.(d)
Repeat Steps a–c, comparing ridge endings and updatingscoreRE to track the number of ridges matched.(e)
IfscoreBF+scoreRE/NminBF+NminRE>tmatch, two fingerprints are said to be matched.


## 2.1. Image Acquisition

In this study, image acquisition was performed using several smartphones: Samsung Galaxy S2, iPhone 3s, Galaxy S6, and iPhone 8. Images of fingers were captured with the focus on their ridges.


## 2.2. Image Derotation

A flowchart depicting the derotation process is shown in Figure2. An image containing a finger aligned along the y-axis is selected as a reference image, as seen in Figure 3(a). Any input finger image requiring registration or verification is considered an input image and is subjected to the derotation process so that the fingerprint registration/matching process can be done smoothly. Figure 3(b) shows an example of an input image that requires rotational alignment for accurate feature matching.Figure 2
Flowchart of the derotation process.Figure 3
Stages of rotation alignment: (a) reference intensity image, (b) input intensity image, (c) segmented reference image, (d) segmented input image, (e) matched features including outliers, (f) matched inliers, and (g) derotated input image.
(a)(b)(c)(d)(e)(f)(g)


### 2.2.1. Skin Color-Based Segmentation

Both the input and reference images are subjected to adaptive skin color-based image segmentation [31]. The images shown in Figures 3(c) and 3(d) show the results of this segmentation on the reference and input images. The input image shown in Figure 3(b) is particularly complex because the color of the background is similar to the color of the skin. Therefore, noise is detected as part of the finger. The white region in Figure 3(d) is the region of interest (ROI) and will be used in upcoming sections.


### 2.2.2. Input Image Scale Adjustment

Both input and reference images usually are of different dimensions. Therefore, the fingers in both images might have a huge scale difference. In that case, scale variance might produce problems while detecting and comparing features. In order to provide a scale-invariant derotation, scales of both input and reference fingers should nearly be the same. If the number of pixels belongs to the finger in the segmented reference image, it is denoted byNref and if the number of pixels belongs to ROI, it is denoted by Ninput. Scaling factor is calculated using (1) as(1)Srefinput=NrefNinput.IfSrefinput<0.9, both the intensity and segmented input images are scaled down by a factor of Srefinput, whereas if Srefinput>1.1, the images are interpolated by a factor of Srefinput using simple bilinear interpolation.


### 2.2.3. Feature Extraction and Matching

Intensity images are converted into binary images for time-efficient feature matching. Since finger shapes are all similar, features detected along their boundaries can provide better estimations of geometrical differences. For feature detection, the SURF [28] method is applied to the segmented binary images to obtain boundary features for both images. SURF is selected for its robustness and reasonable match rate for two same images with different angle of rotation [29]. After obtaining features, nearest neighbor-based feature matching by computing the pairwise distance between features is performed. Then, feature matching provides matched features that contain truly matched inliers as well as erroneously matched outliers. Figure 3(e) shows all the matched features including both inliers and outliers.


### 2.2.4. Inlier Extraction and Derotation of Image

To extract inliers from the matched points, an M-estimator [29] algorithm is applied. Figure 3(f) shows the inliers extracted from all matched features using the M-estimator. It can be seen that extracted inliers represent true matches that can be used to estimate the angle of rotation between accurately matched inlier points.Once several potential inliers are extracted, the angle of rotation between the matched inliers must be determined. A homography matrix [30] is calculated for this purpose. For the two sets of matched points, D=xi,yi for the reference image and D′=xi′,yi′ for the input image. Then, the relation between them is as given in the following equation:(2)D′=HD,orH=D′D−1,where(3)D=xiyi1,D=xi′yi′1,The resultant matrix in (3) is of the form given in (4) as(4)H=h11h12h13h21h22h23h31h32h33=εcosθ−sinθtxεsinθcosθty001.Therefore, the angle of rotation can be calculated usingh12 or h22 using (5) as(5)θ=sin−1−h12,θ=cos−1−h22,where θ can be used to derotate the input image to obtain an image with a vertically aligned finger. Figure 3(g) shows the derotation of the input image after the calculation of θ=−176.29°. Figure 4 shows the alignment method applied to various input images showing fingers at different angles of rotation. It can be seen that the proposed algorithm predicted the angles of rotation accurately and that the fingers in the derotated images are aligned vertically irrespective of their initial alignment.Figure 4
Stages of rotation alignment method applied to four images: (a–d) input intensity image, (e–h) segmented input image, (i–l) matched features including outliers, (m–p) matched inliers, and (q–t) derotated input image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)


## 2.2.1. Skin Color-Based Segmentation

Both the input and reference images are subjected to adaptive skin color-based image segmentation [31]. The images shown in Figures 3(c) and 3(d) show the results of this segmentation on the reference and input images. The input image shown in Figure 3(b) is particularly complex because the color of the background is similar to the color of the skin. Therefore, noise is detected as part of the finger. The white region in Figure 3(d) is the region of interest (ROI) and will be used in upcoming sections.


## 2.2.2. Input Image Scale Adjustment

Both input and reference images usually are of different dimensions. Therefore, the fingers in both images might have a huge scale difference. In that case, scale variance might produce problems while detecting and comparing features. In order to provide a scale-invariant derotation, scales of both input and reference fingers should nearly be the same. If the number of pixels belongs to the finger in the segmented reference image, it is denoted byNref and if the number of pixels belongs to ROI, it is denoted by Ninput. Scaling factor is calculated using (1) as(1)Srefinput=NrefNinput.IfSrefinput<0.9, both the intensity and segmented input images are scaled down by a factor of Srefinput, whereas if Srefinput>1.1, the images are interpolated by a factor of Srefinput using simple bilinear interpolation.


## 2.2.3. Feature Extraction and Matching

Intensity images are converted into binary images for time-efficient feature matching. Since finger shapes are all similar, features detected along their boundaries can provide better estimations of geometrical differences. For feature detection, the SURF [28] method is applied to the segmented binary images to obtain boundary features for both images. SURF is selected for its robustness and reasonable match rate for two same images with different angle of rotation [29]. After obtaining features, nearest neighbor-based feature matching by computing the pairwise distance between features is performed. Then, feature matching provides matched features that contain truly matched inliers as well as erroneously matched outliers. Figure 3(e) shows all the matched features including both inliers and outliers.


## 2.2.4. Inlier Extraction and Derotation of Image

To extract inliers from the matched points, an M-estimator [29] algorithm is applied. Figure 3(f) shows the inliers extracted from all matched features using the M-estimator. It can be seen that extracted inliers represent true matches that can be used to estimate the angle of rotation between accurately matched inlier points.Once several potential inliers are extracted, the angle of rotation between the matched inliers must be determined. A homography matrix [30] is calculated for this purpose. For the two sets of matched points, D=xi,yi for the reference image and D′=xi′,yi′ for the input image. Then, the relation between them is as given in the following equation:(2)D′=HD,orH=D′D−1,where(3)D=xiyi1,D=xi′yi′1,The resultant matrix in (3) is of the form given in (4) as(4)H=h11h12h13h21h22h23h31h32h33=εcosθ−sinθtxεsinθcosθty001.Therefore, the angle of rotation can be calculated usingh12 or h22 using (5) as(5)θ=sin−1−h12,θ=cos−1−h22,where θ can be used to derotate the input image to obtain an image with a vertically aligned finger. Figure 3(g) shows the derotation of the input image after the calculation of θ=−176.29°. Figure 4 shows the alignment method applied to various input images showing fingers at different angles of rotation. It can be seen that the proposed algorithm predicted the angles of rotation accurately and that the fingers in the derotated images are aligned vertically irrespective of their initial alignment.Figure 4
Stages of rotation alignment method applied to four images: (a–d) input intensity image, (e–h) segmented input image, (i–l) matched features including outliers, (m–p) matched inliers, and (q–t) derotated input image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)(m)(n)(o)(p)(q)(r)(s)(t)


## 2.3. Image Enhancement

Edge-based image enhancement is applied to the grayscale image to improve the visibility of ridges on the finger. Edge-based enhancement is performed in two stages. First, unsharp masking [32] is applied to the image. This algorithm subtracts an image from a blurred version of itself and enhances those regions which provide a good response after subtraction. To calculate the blurred image, a Gaussian filter is convolved with the original grayscale image using the following equation:(6)Igray∗Gσ,where(7)Gσ=1πσe−x2/2σ2.For the second stage of image enhancement, a contextual blockwise Fourier domain filter [33] is applied to the grayscale image by dividing it into a 16 × 16 overlapping block. These blocks are then filtered in the Fourier domain using a frequency and orientation-selective filter with parameters based on the estimated local ridge orientation and frequency. The outcome of the Fourier domain-based enhancement is another image denoted as Ienhanced. Ienhanced is multiplied with a skin-segmented binary image so that only the region of interest (ROI) is processed further.Figure5 shows the effects of this method on the derotated input image. The image in Figure 5(a) shows ridges that are not visible. The results of the unsharp masking can be seen in Figure 5(b) showing improved visibility of the ridges. Figure 5(c) shows the image once it has been converted to grayscale. Figure 5(d) shows that application of Fourier-based image enhancement further sharpens unclear edges and other details. Figure 5(e) shows the extracted ROI.Figure 5
Application of preprocessing on the fingerprint image: (a) original input image, (b) unsharp masking, (c) grayscale conversion, (d) blockwise Fourier-based enhancement, and (e) extracted region of interest.
(a)(b)(c)(d)(e)


## 2.4. Image Binarization and Edge Thinning

Features such as ridge endings and bifurcations can be accurately extracted when ridges having a thickness of one pixel are detected. To detect thin ridges, all ridges are first detected by applying Otsu thresholding [34] to the enhanced region of interest. This will result in a binary image with ridge pixels represented in white and the remaining pixels in black. However, this will result in the detection of thick ridges. To limit the results to thin ridges, nonmaximal suppression [35] is applied. This method sets all the ridge pixels in a certain direction to zero if they do not have a maximum gradient value and retains ridge pixels with the maximal value. This results in conversion of the thick ridges to ridges with a thickness of only one pixel. Figure 6(a) shows Otsu thresholding being applied to the enhanced region of interest. The detected ridges are thick and thus cannot be used for feature detection. Application of nonmaximal suppression results in more desirable thin ridges, as seen in Figure 6(b).Figure 6
Applying edge thinning operations to an enhanced image: (a) binary segmentation, (b) nonmaximal suppression, and (c) minutiae extraction.
(a)(b)(c)


## 2.5. Minutiae Extraction

Minutiae are features that are used for biometric verification. Instead of storing fingerprint images in the database, minutiae are stored and compared. The most common method used for minutiae extraction is the crossing number (CN) [36].For extracting minutiae, the 3 × 3 window shown in Figure7 is moved across each pixel to calculate its crossing number using the following equation:(8)CNx,y=12∑i=18pi−pi−1,where pi denotes the pixels around the center pixel at location x,y. Using CN, the ridge pixel can then be classified as a ridge ending, bifurcation, or isolated point using Table 1. Figure 6(c) shows bifurcations and ridge endings represented by blue and red squares, respectively.Figure 7
3 × 3 window used in minutiae extraction.Table 1
Ridge pixel classifications using the crossing number.
CNDecision0Isolated point1Ridge ending2Continuing ridge3Bifurcation4CrossingFor each bifurcation and ridge ending, the Euclidean distanceDx,y between its spatial coordinate x,y and xROImin,yROImin, the normalized gradient value Gx,y, and the gradient orientation αx,y are stored in SBFin and SREin, respectively, for storage or comparison. Hence, every bifurcation and ridge ending point is stored in SBFin or SREin in the form of 3D point DBFx,y,GBFx,y,αBFx,y and DREx,y,GREx,y,αREx,y, respectively. The values donated by xROImin and yROImin are x-axis and y-axis points taken from two ROI points corresponding to the minimum x-axis and y-axis values. Figure 8 shows an example of calculation of xROImin,yROImin that is represented by a green circle. This was determined using the minimum x-axis ROI point represented by a blue cross and the minimum y-axis ROI point represented by a red cross.Figure 8
Determination of pointxROImin,yROImin: (a) minimum x-axis and y-axis ROI points and (b)xROImin,yROImin determination using two minimum axis points.
(a)(b)


## 2.6. Fingerprint Registration and Matching

Extracted minutiae are either stored or compared with minutiae already existing in the database as required by the use case. Suppose that the ridge endings extracted from the input image are represented by the setSBFin and that bifurcations are represented by SREin as follows:(9)SREin=DRE1in,GRE1in,∝RE1in,DRE2in,GRE2in,∝RE2in,…,DRENREinin,GRENREinin,∝RENREinin,(10)SBFin=DBF1in,GBF1in,∝RE1in,DBF2in,GBF2in,∝BF2in,…,DBFNBFinin,GBFNBFinin,∝BFNBFinin,where SREin and SBFin are sorted in an ascending order according to the calculated Euclidean distance such that DREkin≤DREk+1in and DBFlin≤DBFl+1in for all k∈1,2,...,NREin and l∈1,2,...,NBFin. These sets must be compared with the stored features of a number of fingerprints NDB stored in the database, as follows:(11)SREDB=SREDB1,SREDB2,…,SREDBNREDB,(12)SBFDB=SBFDB1,SBFDB2,…,SBFDBNBFDB,where(13)SREDBi=DRE1DBi,GRE1DBi,∝RE1DBi,DRE2DBi,GRE2DBi,∝RE2DBi,…,DRENREDBiDBi,GRENREDBiDBi,∝RENREDBiDBi,(14)SBFDBi=DBF1DBi,GBF1DBi,∝RE1DBi,DBF2DBi,GBF2DBi,∝BF2DBi,…,DBFNBFDBiDBi,GBFNBFDBiDBi,∝BFNBFDBiDBi.SREDBi and SBFDBi are also sorted in an ascending order according to the Euclidean distances DREmDBi and DBFnDBi where m∈1,2,....,NREDBi and n∈1,2,....,NBFDBi. Suppose that SREin and SBFin are to be compared with the stored features of the ith fingerprint having feature sets SREDBi and SBFDBi. First, NminRE, NminBF, NmaxRE, and NmaxBF are to be determined using the following equations:(15)NminRE=minNREin,NREDBi,(16)NminBF=minNBFin,NBFDBi,(17)NmaxRE=maxNREin,NREDBi,(18)NmaxBF=maxNBFin,NBFDBi,where NREin and NREDBi are the number of ridge endings extracted from the input image and the ith image features stored in the database and NBFin and NBFDBi are the number of extracted bifurcations. The input fingerprint features are compared with the ith fingerprint features if the condition in (19) is justified:(19)NminRE≥0.8NmaxRE,NminBF≥0.8Nmax.For matching, all 3D bifurcation and ridge ending points of the input fingerprint and the ith fingerprint from the database and input fingerprint are stored inMREref, MBFref, MREin, and MBFin respectively. MREref is compared with MREin, and MBFref is compared with MBFin. The steps for matching are(a)
IfMBFin has fewer bifurcations, store MBFin in MBFmin and MBFref in MBFmax and vice versa. Initialize bifurcation matching score scoreBF to zero.(b)
Compare each bifurcation 3D point inMBFmin at the pth position with all bifurcation points at locations p+t in MBFmax where t∈−5,−4,−3,−2,−1,0,1,2,3,4,5.(c)
IfGp+tmax−Gpmin<T1andαp+tmax−αpmin<T2 is true for any value of t, then increment scoreBF and move to the next bifurcation point for matching. Here, Gpmin and αpmin are the magnitude and orientation of the normalized gradient of bifurcation of MBFmin at position p, and Gp+tmax and αp+tmmax are the magnitude and orientation of MBFmax at any of the eleven locations in Step b.(d)
Repeat Steps a–c, comparing ridge endings and updatingscoreRE to track the number of ridges matched.(e)
IfscoreBF+scoreRE/NminBF+NminRE>tmatch, two fingerprints are said to be matched.


## 3. Results and Discussion

MATLAB 2018a was used in this study. A value ofσ=2.5 was used, and a Gaussian window of size 2σ+1×2σ+1,T1=.06, and T2=10° was used throughout the work. There was no dataset available for evaluation purposes, so features from the fingerprints of 40 people were stored in a database. Features obtained from digital images of the fingers of 100 people, including those of registered people, were compared with the features already stored in the database in order to evaluate the performance of the proposed algorithm. Every person provided eight images of their finger at approximately 90°, 135°, 180°, 225°, 270°, 315°, 0°, and 45°.The performance of the proposed method was evaluated by analyzing the performance of the derotation and matching methods. It was not possible to calculate the exact errors in the calculation of the rotation angles, so error calculation was done manually for ten random inputs for each set of rotation angles.According to this manual evaluation, the errors fell in the range from∼−2.5 to ∼2.5° for all cases. The error range for the angles 0°, 180°, and 270° was from ∼−1.5° to 1.5°. For a rotation angle of 90°, the error was approximately 0°.True positives, true negatives, false positives, and false negatives were calculated for different values of the matching thresholdtmatch. True positives, true negatives, false positives, and false negatives are given in (20)–(23) as(20)TP=ncorrectMatches×100nActualMatches,(21)TN=ncorrectMissMatches×100nActualMismatches,(22)FP=nmistakenMatches×100ntotal,(23)FN=nmistakenMisMatches×100nActualMismatches.Figure9 shows a comparison of the percentages of true matches (TP) and mistaken mismatches (FN). It can be seen from Figure 9 that a value of 0.7≤tmatch≤0.8 successfully verified all 40 of the registered fingerprints irrespective of their angle of rotation. However, for values of tmatch>0.8, mistaken mismatches occurred in the range from ∼2%−18%. True mismatches (TN) occurred in 100% of the cases, and mistaken matches (FP) occurred in 0% of all cases.Figure 9
Percentage of (a) TP and (b) FN for different values oftmatch.
(a)(b)


## 4. Conclusions and Future Work

This paper proposes a fingerprint detection and verification method that uses a digital image of a finger as an input instead of a sensor. The input fingerprint image can be of any resolution and depends on the quality of the digital camera. Often, the finger in the image is not vertically aligned. Therefore, the input image is derotated by applying skin color-based binary segmentation, followed by downscaling or upscaling of the input image with a scale calculated using the segmented input image and a reference image. The angle of rotation of the input finger is then calculated by applying SURF-based feature extraction on both binary images. The features detected in both images are matched, and outliers are removed from the matched points using the M-estimator. The angle between the matched inliers is calculated using a homography matrix, and the input image is derotated by the calculated angle. After derotation, enhancement techniques including unsharp masking and Fourier-based blockwise enhancement are applied to increase the visibility of the ridges in the input image. Image enhancement is followed by the application of Otsu thresholding to segment the ridge regions, and then nonmaximal suppression is applied for edge thinning. A crossing number technique is used to extract ridge endings and bifurcations. This is followed by the proposed Euclidean distance-based matching to detect a fingerprint match or mismatch based on a matching score. The proposed method is a novel method, and simulation results show that if the matching threshold is kept within a suggested range, accuracy of 100% can be achieved.Scar removal-based fingerprint detection using digital images and edge-preserving image interpolation techniques could extend the proposed method and is being considered for future work.



---

*Source: 9758049-2020-05-15.xml*