# 3D Palmprint Recognition Using Dempster-Shafer Fusion Theory

Authors: Jianyun Ni; Jing Luo; Wubin Liu
Journal: Journal of Sensors
 (2015)

---

## Abstract

This paper proposed a novel 3D palmprint recognition algorithm by combining 3D palmprint features using D-S fusion theory. Firstly, the structured light imaging is used to acquire the 3D palmprint data. Secondly, two types of unique features, including mean curvature feature and Gaussian curvature feature, are extracted. Thirdly, the belief function of the mean curvature recognition and the Gaussian curvature recognition was assigned, respectively. Fourthly, the fusion belief function from the proposed method was determined by the Dempster-shafer (D-S)  fusion theory. Finally, palmprint recognition was accomplished according to the classification criteria. A 3D palmprint database with 1000 range images from 100 individuals was established, on which extensive experiments were performed. The results show that the proposed method 3D palmprint recognition is much more robust to illumination variations and condition changes of palmprint than MCR and GCR. Meanwhile, by fusing mean curvature and Gaussian curvature feature, the experimental results are promising (the average equal error rate of 0.404%). In the future, imaging technique needs further improvement for a better recognition performance.


---

## Body

## 1. Introduction

Palmprint images can be acquired in a similar manner to face images and a number of researchers have suggested that the human palmprint is unique enough to each individual to allow practical use as a biometric. Several researchers have made palmprint recognition based on the palmprint’s appearance in 2D intensity images [1–10], whereas a smaller number of researchers have accomplished palmprint recognition using 3D palmprint information [11–13]. Although 2D palmprint recognition techniques can achieve high accuracy, the 2D palmprint recognition may be liable to strong illumination variations and condition changes of palmprint and much 3D palm structural information is lost in the 2D palmprint acquisition process. Therefore, it is of high interest to explore new palmprint recognition techniques to overcome these difficulties. Intuitively, 3D palmprint recognition is a good solution.In this paper, we will extract the local curvature features of 3D palmprint for recognition. After the 3D depth information of palm is obtained, ROI of the 3D palmprint image is extracted. Besides reducing the data size for processing, the ROI extraction process also serves to align the palmprints and normalize the area for feature extraction. The mean curvature (MC) and Gaussian curvature (GC) features of each cloud point in the ROI are then calculated. Finally, MC and GC features are fused by Dempster-Shafer (D-S) fusion theory; the input palm can be recognized. We established a 3D palmprint database with 1000 range images from 100 people. Extensive experiments are conducted to evaluate the performance of the proposed method.This paper is organized as follows. The acquisition of 3D palmprint data is given in Section2. In Section 3, we describe the ROI region determination and MC and GC feature extraction from ROI and MC and GC feature matching. The calculation of palm MC and GC features is described in detail. Section 4 presents the palmprint recognition fusion method by D-S fusion theory. In Section 5, we present the main experimental results and analyze the results. Section 6 gives the summary and conclusions.


## 2. 3D Palmprint Data Acquisition

In each acquisition session, the subject sits approximately 0.1 meters away from the sensor. Data was acquired with a CPOS MINI PLUSII range scanner. One 1280* 1024 3D scan and one 1280 * 1024 color image were obtained in a period of several seconds. Figure 1 shows the 3D palmprint data acquisition device developed by Information and Communication Technology Research (ICTR) Institute, University of Wollongong.Figure 1
3D palmprint acquisition device.As shown in Figure2, a fringe pattern generated by the grating crosses the surfaces of the object at point D and reaches the reference plane at point A; then the projected fringe is reflected to the CCD camera by the surfaces of the object through point C.Figure 2
Schematic diagram of fringe pattern profilometry system.The height of reference plane is 0. The relative height of a pointD at spatial position (x,y) to the reference plane is computed by the following [14]:
(1)h(x,y)=BD=ACtanθ01+tanθ0/tanθn(2)AC=ϕCD2πP0,
where P0 is the wavelength of the projected light on the reference plane, θ0 is the projecting angle, θn is the angle between the reference plane and the line that passes through the current point and the CCD center, and ϕCD is the phase difference between points C and D. Because the phase of point D on the 3D object is equal to the phase of point A on the reference plane, ϕCD can be calculated by the following:
(3)ϕCD=ϕCA=ϕOC-ϕOA.By using (1) and the phase shifting and unwrapping technique [15], we can retrieve the depth information of the object surface by projecting a series of phase stripes on it (15 stripes are used). Six sample patterns of the stripes on the palm are illustrated in Figure 3.Figure 3
Six sample patterns of stripes on the palm.With this processing, the relative height of each pointh(x,y) could be calculated. The range data of the palm surface can then be obtained. In the developed system, the size of the 3D image is 1280 × 1024; that is, there are totally 1310720 cloud points to represent the 3D palmprint information. Figure 4 shows an example 3D palmprint image captured by the system.Figure 4
3D palmprint image.


## 3. Feature Extraction

### 3.1. ROI Extraction

In Figure4, many cloud points, such as those in the boundary area and those in the fingers, could not be used in feature extraction and recognition. Most of the useful and stable features are located in the center area of the palm. In addition, at different times when the user puts his/her hand on the system, there will be some relative displacements of the positions of the palm, even if we impose some constraints on the users to place their hands. Therefore, before feature extraction, it is necessary to perform some preprocessing to align the palmprint and extract the central area of it. In this paper, we use the algorithm in [16] to extract the 3D ROI. Figure 5 shows the ROI of 3D palmprint.Figure 5
ROI of 3D palmprint.By the ROI extraction procedure, the 3D palmprints are aligned so that the small translation and rotation introduced in the data acquisition process are corrected. In addition, the data amount used in the following feature extraction and matching process is significantly reduced.


### 3.2. MC and GC Feature Extraction

The authors have found that Gaussian and mean curvature combine these first and second fundamental forms in two different ways to obtain scalar surface features that are invariant to rotations, translations, and changes in parameterization [17]. Considering the rotation, translation, and even some deformation in the 3D palmprint cloud points, we use the mean and Gaussian curvatures to describe the surface of 3D palmprint.Mean (H) and Gaussian (K) curvature images are computed [18] using the partial derivative estimate images:
(4)Hi,j=1+g~v2i,jg~uui,j+1+g~u2i,jg~vvi,jh1+g~v2i,j-2g~ui,jg~vi,jg~uvi,j·21+g~u2i,j+g~v2i,j3-1Ki,j=g~uui,jg~vvi,j-g~uv2i,j1+g~u2i,j+g~v2i,j2,
where
(5)g~ui,j=Du*S*g~i,jg~vi,j=Dv*S*g~i,jg~uui,j=Duu*S*g~i,jg~vvi,j=Dvv*S*g~i,jg~uvi,j=Duv*S*g~i,j,
where
(6)S=s→s→Ts→=1641615201561TDu=d→0d→1TDv=d→1d→0TDuu=d→0d→2TDvv=d→2d→0TDuv=d→1d→1T,
where
(7)d→0=171111111Td→1=128-3-2-10123Td→2=18450-3-4-305T.For more efficient computation, we first normalize the Gaussian curvatureK or mean curvature H into C¯ by the following:
(8)C¯i,j=0.5Ci,j-μ4δ+0.5μ,
where μ is the mean of the curvature. With (8), most of the curvature values will be normalized into the interval [0,1].


### 3.3. MC and GC Feature Matching

We use the method in REF [11] to calculate the matching score of MC and GC feature. Let Bd denote the binarized MC/GC image in the database and denote by Bt the input MC/GC binary image. Suppose the image size is n×m. The matching score between Bd and Bt is defined as
(9)RC=2∑i=1n∑j=1mBd(i,j)⊕Bt(i,j)∑i=1n∑j=1mBd(i,j)+∑i=1n∑j=1mBt(i,j).


## 3.1. ROI Extraction

In Figure4, many cloud points, such as those in the boundary area and those in the fingers, could not be used in feature extraction and recognition. Most of the useful and stable features are located in the center area of the palm. In addition, at different times when the user puts his/her hand on the system, there will be some relative displacements of the positions of the palm, even if we impose some constraints on the users to place their hands. Therefore, before feature extraction, it is necessary to perform some preprocessing to align the palmprint and extract the central area of it. In this paper, we use the algorithm in [16] to extract the 3D ROI. Figure 5 shows the ROI of 3D palmprint.Figure 5
ROI of 3D palmprint.By the ROI extraction procedure, the 3D palmprints are aligned so that the small translation and rotation introduced in the data acquisition process are corrected. In addition, the data amount used in the following feature extraction and matching process is significantly reduced.


## 3.2. MC and GC Feature Extraction

The authors have found that Gaussian and mean curvature combine these first and second fundamental forms in two different ways to obtain scalar surface features that are invariant to rotations, translations, and changes in parameterization [17]. Considering the rotation, translation, and even some deformation in the 3D palmprint cloud points, we use the mean and Gaussian curvatures to describe the surface of 3D palmprint.Mean (H) and Gaussian (K) curvature images are computed [18] using the partial derivative estimate images:
(4)Hi,j=1+g~v2i,jg~uui,j+1+g~u2i,jg~vvi,jh1+g~v2i,j-2g~ui,jg~vi,jg~uvi,j·21+g~u2i,j+g~v2i,j3-1Ki,j=g~uui,jg~vvi,j-g~uv2i,j1+g~u2i,j+g~v2i,j2,
where
(5)g~ui,j=Du*S*g~i,jg~vi,j=Dv*S*g~i,jg~uui,j=Duu*S*g~i,jg~vvi,j=Dvv*S*g~i,jg~uvi,j=Duv*S*g~i,j,
where
(6)S=s→s→Ts→=1641615201561TDu=d→0d→1TDv=d→1d→0TDuu=d→0d→2TDvv=d→2d→0TDuv=d→1d→1T,
where
(7)d→0=171111111Td→1=128-3-2-10123Td→2=18450-3-4-305T.For more efficient computation, we first normalize the Gaussian curvatureK or mean curvature H into C¯ by the following:
(8)C¯i,j=0.5Ci,j-μ4δ+0.5μ,
where μ is the mean of the curvature. With (8), most of the curvature values will be normalized into the interval [0,1].


## 3.3. MC and GC Feature Matching

We use the method in REF [11] to calculate the matching score of MC and GC feature. Let Bd denote the binarized MC/GC image in the database and denote by Bt the input MC/GC binary image. Suppose the image size is n×m. The matching score between Bd and Bt is defined as
(9)RC=2∑i=1n∑j=1mBd(i,j)⊕Bt(i,j)∑i=1n∑j=1mBd(i,j)+∑i=1n∑j=1mBt(i,j).


## 4. Palmprint Recognition Method

### 4.1. Concept of D-S Evidence Theory

Suppose thatΘ is the set of independent proposition. The 2Θ represents the set of Θ. In this paper, the MC and GC matching results are evidence. The belief degrees of all the propositions are assigned through the basic probability function. The basic probability function is defined as follows.The functionm:→[0,1] must satisfy the following conditions:(1)
the basic probability of impossible event is 0,m(ϕ)=0;(2)
∑A⊂Θm(A)=1; the sum of basic probability of all elements in 2Θ is 1.Suppose that N is the number of palmprint type, M is the number of recognition method, λi is the weighted coefficient of recognition method, and Ci(Oj) is the correlation coefficient of the recognition method i and palmprint type Oj. Considering the effect on recognition of the number of palmprint and environment, the basic probability function can be determined by the following [19]:
(10)miOj=CiOj∑j=1NCiOj+N1-Ri1-λiαiβii=1,2,…,M,j=1,2,…,N,
where
(11)αi=max⁡CiOj∣j=1,2,…,Nξi=Nλi∑j=1NCi(Oj)i=1,2,…,Mβi=ξi-1M-1M≥2,i=1,2,…,MRi=λiαiβi∑j=1Nλiαiβii=1,2,…,M.


### 4.2. Dempster Fusion Theory

Suppose thatm1 and m2 are two basic probability functions which have the elements of A1,A2,…,Ak and B1,B2,…,Br, respectively. The combination basic probability is represented by (13).Suppose that(12)K=∑AI∩BJ=ϕm1(Ai)m2(Bj).
Then,
(13)m(A)=∑AI∩BJ=Am1(Ai)m2(Bj)1-KA≠ϕ0A=ϕ.
The parameter K represents the conflict degree of evidence.In (13), if K≠1, then a basic probability is determined. If K=1, then m1 and m2 contradict. So, Dempster fusion theory has much stronger belief after fusion.


### 4.3. Palmprint Recognition Rules

The evidence is fused by Dempster fusion theory. The problem of making decision is relative to the application. In this paper, the basic probability is adopted and it mainly includes the following.(1)
The target palmprint has the largest believability.(2)
The difference of the believability of the target palmprint and other palmprints must be more than a threshold.(3)
The uncertainty region must be smaller than a threshold.(4)
The believability of the target palmprint must be more than the uncertainty region.Suppose that(14)∃A1,A2⊂Θ,satisfymA1=max⁡mAi,Ai⊂ΘmA2=max⁡mAi,Ai⊂Θ,Ai≠A1.If
(15)m(A1)-m(A2)>ε1m(ϕ)<ε2m(A1)>m(ϕ)
then A1 is final recognition result. In general, ε1=ε2=0.1. m(ϕ) represents the uncertainty probability.


### 4.4. Procedure of Palmprint Recognition Method

The procedure of proposed algorithm is as follows.The belief function of the mean curvature recognitionm1(Ai) and belief function of the Gaussian curvature recognition m2(Bj) were assigned according to (10), respectively.The fusion belief function was determined by (13).Palmprint recognition was accomplished according to the recognition rules in Section4.3.In this paper, we adopt the mean curvature recognition method and Gaussian curvature recognition method [11]. Based on [11], the belief function and recognition results have some prior information by which the parameters of Ci(Oj) and λi can be determined. Specifically, λi and Ci(Oj) can be calculated by the accuracy of corresponding recognition method.Figure6 has shown the flow chart of palmprint recognition based on data fusion.Figure 6
Flow chart of palmprint recognition based on MC and GC data fusion.


## 4.1. Concept of D-S Evidence Theory

Suppose thatΘ is the set of independent proposition. The 2Θ represents the set of Θ. In this paper, the MC and GC matching results are evidence. The belief degrees of all the propositions are assigned through the basic probability function. The basic probability function is defined as follows.The functionm:→[0,1] must satisfy the following conditions:(1)
the basic probability of impossible event is 0,m(ϕ)=0;(2)
∑A⊂Θm(A)=1; the sum of basic probability of all elements in 2Θ is 1.Suppose that N is the number of palmprint type, M is the number of recognition method, λi is the weighted coefficient of recognition method, and Ci(Oj) is the correlation coefficient of the recognition method i and palmprint type Oj. Considering the effect on recognition of the number of palmprint and environment, the basic probability function can be determined by the following [19]:
(10)miOj=CiOj∑j=1NCiOj+N1-Ri1-λiαiβii=1,2,…,M,j=1,2,…,N,
where
(11)αi=max⁡CiOj∣j=1,2,…,Nξi=Nλi∑j=1NCi(Oj)i=1,2,…,Mβi=ξi-1M-1M≥2,i=1,2,…,MRi=λiαiβi∑j=1Nλiαiβii=1,2,…,M.


## 4.2. Dempster Fusion Theory

Suppose thatm1 and m2 are two basic probability functions which have the elements of A1,A2,…,Ak and B1,B2,…,Br, respectively. The combination basic probability is represented by (13).Suppose that(12)K=∑AI∩BJ=ϕm1(Ai)m2(Bj).
Then,
(13)m(A)=∑AI∩BJ=Am1(Ai)m2(Bj)1-KA≠ϕ0A=ϕ.
The parameter K represents the conflict degree of evidence.In (13), if K≠1, then a basic probability is determined. If K=1, then m1 and m2 contradict. So, Dempster fusion theory has much stronger belief after fusion.


## 4.3. Palmprint Recognition Rules

The evidence is fused by Dempster fusion theory. The problem of making decision is relative to the application. In this paper, the basic probability is adopted and it mainly includes the following.(1)
The target palmprint has the largest believability.(2)
The difference of the believability of the target palmprint and other palmprints must be more than a threshold.(3)
The uncertainty region must be smaller than a threshold.(4)
The believability of the target palmprint must be more than the uncertainty region.Suppose that(14)∃A1,A2⊂Θ,satisfymA1=max⁡mAi,Ai⊂ΘmA2=max⁡mAi,Ai⊂Θ,Ai≠A1.If
(15)m(A1)-m(A2)>ε1m(ϕ)<ε2m(A1)>m(ϕ)
then A1 is final recognition result. In general, ε1=ε2=0.1. m(ϕ) represents the uncertainty probability.


## 4.4. Procedure of Palmprint Recognition Method

The procedure of proposed algorithm is as follows.The belief function of the mean curvature recognitionm1(Ai) and belief function of the Gaussian curvature recognition m2(Bj) were assigned according to (10), respectively.The fusion belief function was determined by (13).Palmprint recognition was accomplished according to the recognition rules in Section4.3.In this paper, we adopt the mean curvature recognition method and Gaussian curvature recognition method [11]. Based on [11], the belief function and recognition results have some prior information by which the parameters of Ci(Oj) and λi can be determined. Specifically, λi and Ci(Oj) can be calculated by the accuracy of corresponding recognition method.Figure6 has shown the flow chart of palmprint recognition based on data fusion.Figure 6
Flow chart of palmprint recognition based on MC and GC data fusion.


## 5. Experimental Results and Analysis

To verify the performance of the proposed algorithm, many experiments have been done on the database by 3D palmprint imaging device. The database consists of 1000 samples from 100 volunteers, including 50 males and 50 females. The 3D palmprint samples were collected in two separated sessions, and, in each session, five samples were collected from each subject. The average time interval between the two sessions is two weeks. Each palmprint has 10 images among which there are 2 normal images, 2 under illumination images, 2 over illumination images, 2 dirty images, and 2 sweat images. The original spatial resolution of the data is 1280 × 1024. After ROI extraction, the central part (256 × 256) is used for feature extraction and recognition. Thez value resolution of the data is 16 bits.These 1000 images are divided into four groups by fivefold cross validation; each group has 200 images. Four groups of subblocks (containing 800 images) are used as training samples, and the remaining group (containing 200 images) is used as testing samples. The process repeated five times.During the palmprint recognition, the probability of the mean curvature recognition and Gaussian curvature recognition can be represented by memberships which are coefficients of belief function corresponding to these two methods. Finally, the fused belief function is attained by fusion theory and final recognition is accomplished according to the fusion rules.Table1 shows the basic belief function of MCR and GCR and their fusion belief function and final recognition results of four people based on MCR and GCR data fusion.Table 1
Recognition results based on MCR and GCR data fusion.
Samples
Method
O1
O2
O3
O4
Uncertain
Recognition results
O1
MCR
0.5769
0.0043
0.4135
0.0039
0.0014
Uncertain
GCR
0.6878
0.1308
0.0366
0.1013
0.0435
O1
Fusion
0.8239
0.1008
0.0285
0.0465
0.0003
O1
O2
MCR
0.0308
0.7221
0.0385
0.1633
0.0453
O2
GCR
0.0015
0.5705
0.3810
0.0068
0.0402
Uncertain
Fusion
0.0018
0.8320
0.1538
0.0085
0.0039
O2
O3
MCR
0.0130
0.3685
0.5618
0.0144
0.0423
Uncertain
GCR
0.1750
0.0256
0.7232
0.0372
0.0390
O3
Fusion
0.1025
0.0160
0.8685
0.0128
0.0002
O3
O4
MCR
0.1211
0.1034
0.0019
0.7317
0.0419
O4
GCR
0.0090
0.3706
0.0103
0.5718
0.0383
Uncertain
Fusion
0.0110
0.1180
0.0083
0.8625
0.0002
O4From Table1, for the testing samples O1, the basic believability of MCR and GCR is 0.5769 and 0.6878, respectively. After fusion, the combined basic believability increases to 0.8239. Meanwhile, the basic probability of the uncertainty of MCR and GCR is 0.0014 and 0.0435, respectively. After fusion, the combined basic probability decreases to 0.0003. Thus, the reliability of recognition is improved after fusion.3D palmprint recognition [20–22] is a conventional 3D palmprint recognition algorithm. The authors extract both line and orientation features (LOF) from enhancing the mean curvature image of the 3D palmprint data. Then, line and orientation features are fused at either score level or feature level for the final 3D palmprint recognition. The experimental results show that feature level fusion achieves higher accuracy than score level fusion.To verify the effectiveness of the proposed algorithm, we compare the proposed algorithm with the MCR, GCR, and LOF method in [20] on the five classes images. In LOF [20] experiments, we adopted feature level fusion for 3D palmprint recognition without BLPF [20].Table2 shows the recognition results on the normal images, over illumination images, under illumination images, dirty images, and sweat images, respectively.Table 2
Recognition results by MCR, GCR, LOF [20], and MCR + GCR fusion.
Images
EER (%)
MCR
GCR
LOF [20]
MCR + GCR fusion
Normal images
0.58
0.65
0.21
0.24
Over illumination images
0.78
0.95
0.45
0.46
Under illumination images
0.68
0.86
0.42
0.43
Dirty images
0.72
0.89
0.43
0.45
Sweat images
0.69
0.87
0.42
0.44
Average
0.69
0.84
0.386
0.404From experiment results, we can see that the proposed algorithm achieves much lower EER than MCR and GCR. The experimental results achieve the average equal error rate of 0.404%. However, LOF [20] achieves tiny lower EER than the proposed algorithm. This is mainly because the quality of 3D palmprint data in our experiments is not as good as that of 3D palmprint in LOF [20]. There is much corrupted noise in the data acquisition process and z value accuracy needs further improvement.Also, MCR and GCR D-S fusion palmprint recognition has good robustness to illumination change and condition changes of palmprint. So, the proposed 3D palmprint recognition has great potential.


## 6. Conclusions

In this paper, a novel palmprint recognition algorithm was proposed by combining 3D palmprint features using D-S evidence theory. After the 3D palmprint image is captured, the ROI is extracted to roughly align the palm and remove the unnecessary cloud points.Then, two types of unique features, including mean curvature feature and Gaussian curvature feature, are extracted. A fast feature matching method and Dempster-Shafer (D-S) fusion strategies were used to classify the palmprints. A 3D palmprint database with 1000 samples from 100 individuals was established, on which extensive experiments were performed. The results show that the proposed method 3D palmprint recognition is more robust to illumination variations and condition changes of palmprint. Meanwhile, by fusing mean curvature and Gaussian curvature feature, much higher recognition rate can be achieved. In the future, imaging technique needs further improvement for a better recognition performance.



---

*Source: 252086-2015-02-26.xml*