JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

1

Completely Contactless and Online Finger Knuckle
Identification for Real World Applications
Zhenyu Zhou, Ajay Kumar

Abstract
Completely contactless and at-a-distance personal identification provides enhanced user convenience, andimproved hygiene and is highly sought under the COVID-19 pandemic. This paper proposes an accurate andgeneralizable deep neural network-based framework for the ‘completely’ contactless finger knuckle identification.
We design and introduce a new loss function to enable a fully convolutional network to more effectively learn
knuckle features that are imaged under at-a-distance imaging. A ‘completely’ contactless system also requires
efficient online finger knuckle detection capabilities. This paper, for the first time in our knowledge, develops andintroduces accurate capabilities to efficiently detect and segment finger knuckle patterns from images with complex
backgrounds as widely observed in real-world applications. We introduce angular loss to accurately predict oriented
knuckle patterns and incorporate into our framework. Experimental results presented in this paper on five different
public databases, using challenging protocols and cross-database performance evaluation, illustrate outperforming
results and validate the effectiveness of the proposed framework for completely contactless applications.
Index Terms
Biometrics, Completely Contactless Finger Knuckle Identification, At-a-distance Finger Knuckle Detection

I. I NTRODUCTION
MONG a range of physiological characteristics for biometrics, finger knuckle patterns have the least
privacy concerns, are convenient to be imaged from a distance, are easier to be presented, andmore importantly, they have shown to offer high uniqueness and stability among the adult population[9]. Contactless or at-a-distance biometrics systems have high user acceptance [1], are highly preferredunder the ongoing pandemic, and are effective in preventing cross-contamination [7] during COVID19 pandemic. Finger knuckle is a relatively new biometric and has attracted increasing attention in theforensics and biometrics community.
There have been earlier studies and work on contactless finger knuckle biometrics. However, in thecontext of user convenience and hygiene, it is important to differentiate the contactless and ‘completely’contactless finger knuckle systems. Contactless biometrics systems ensure that there is no physical contact
between the sensor and the imaged biometric. However, this has often included the systems [2], [3], [6],[8], [10], [11] that require the user to present his/her fingers or hands on a surface. On the other hand, a‘completely’ contactless finger knuckle system will ensure that neither the fingers nor hands are presented
in the completely contactless manner [5] in the field of view of the sensor which itself can be mobile.
Such a completely contactless system can allow 6 degrees of freedom during the imaging, and acquire
images with complex background patterns with a high degree of rotational and scale changes. Therefore,
the framework for the completely contactless system should ensure accurate detection (where the position,size, and the number of finger knuckles appearing in each frame can vary) and segmentation of knuckle
patterns under complex backgrounds, and provide higher accuracy under such relatively higher intra-classvariations. This work attempts to address such open challenges and realize a completely contactless andonline system for real-world deployments.

A

Authors are with the Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong.

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

2

A. Related Work
Contactless Finger Knuckle Identification: There have been a several exciting attempts to match
contactless finger knuckle images, e.g. using a range of methods that that attempted to incorporate key
points [12], local spatial features using binary representations [35], subspace methods [13] to encode
local 3D curvature of knuckle creases, and illustrate impressive performance. Reference [4] provides acomprehensive summary of such methods. Even earlier work has considered application scenarios using
smartphones for the contactless finger knuckle recognition [14], but the finger knuckle segmentation
requirement to present the finger in a fixed imaging position i.e. image center, is inconvenient and limited
its practical use.
Conventional methods heavily rely on the selection of parameters when incorporated for matching theimages for the other/different databases or those acquired under different imaging setups. On the contrary,deep neural network-based methods can offer enhanced generalization and robustness and have been
recently attempted in the literature. Reference [8] presents FKNet which is based on the ResNet-50 [15],and can provide robust deep feature representation for the finger knuckle features. Attempts to combine
traditional methods with CNN models appear in [16] which firstly use initialized Gabor filters to extract
features, and then use CNN to learn the most important information, and the Gabor filter parameters canalso update during backpropagation. The models in [8], [16] are complex and lightweight models, like therecently introduced EfficientNetV2 [17] model, can provide a better alternative for real-time deployments.
Contactless Fingers Detection and Segmentation: The accuracy of the detected and segmented regionof interest (ROI) can determine the matching efficiency and accuracy of the online finger knuckle system.
The size of the ROI should be comparable to the size of the actual target object (knuckle) at the pixel
level so that the size obtained after segmentation will not have redundant pixel information, which will
reduce the pixel values to be computed for both the extraction and the subsequent matching sessions.
Conventional contactless finger knuckle methods, their [18], [14] approach is to fix the finger knuckle
position in the image while acquiring the finger knuckle images under plain, instead of the complex
background, and then detect the edges of the finger, and then extract the finger knuckle crease regions.
Prior finger knuckle detection and segmentation algorithm cannot correctly detect/segment the finger
knuckles in the presence of complex backgrounds, multiple finger knuckles in the same field of view,
obscured or finger knuckles regions with high pose and scale changes.
Deep neural network models for object detection have achieved great success, whether it is the sliding
window detection algorithm, the 2-stage series of R-CNN models [19], [20], [21], or the 1-stage YOLO
series [22], [23], [24], [25] and SSD models [26] up to the current position, and even the anchor-freebased object detection algorithm [27] as well. Each of these models has its advantages and limitations.
For the 2-stage model, the object detection accuracy is quite guaranteed, the 1-stage based model is a
speedup based on the positive accuracy, and the anchor-free is a further improvement in the detection
speed. In this paper, we adopt a recent version of the YOLO model, i.e. YOLOv5 [28], for the fast andaccurate detection of finger knuckle regions.
Limitations and Challenges: As outlined in previous sections, conventional methods for finger knuckle
identification require parameter tuning which is specific to the database and even the parameters forthe detection should be tuned for different imaging environments. Deep neural network architectures are
known to offer high generalization capabilities. However, any direct application of such architectures, e.g.
[8], [16], [29] is expected to deliver limited performance. This is due to the fact that new challenges
emerge while incorporating completely contactless finger knuckle images. Earlier work [8], [16], [29]have fail to effectively accommodate inherent rotational and translational changes that are quite common
under completely contactless imaging. Also, the number of output classes is fixed in [8] for the training,
and therefore it requires retraining whenever a new subject is added.
The finger knuckle detection and segmentation approach used in [35] cannot work for completely
contactless imaging under complex background. The knuckle detection and segmentation approach in [8]is based on the Mask R-CNN which is highly complex and not suitable for the real-time identification

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

3

TABLE IC OMPARATIVE S UMMARY OF R ELATED AND R ECENT W ORK ON C ONTACTLESS F INGER KNUCKLE IDENTIFICATION
Ref.

Deployment Scenario
Real-World Knuckle Detection Completely Contactless

Online System

[8]

No

No

No

[11]

No

No

No

[29]

No

No

No

[10]

No

No

No

Ours

Yes

Yes

Yes

Database
A
B
D(1)
A
E
A
D(1)
D(2)
B
C
D(1)
D(2)
E

Performance
EER
GAR @ FAR=10−4
3.90%
86.60%
7.70%
74.50%
2.40%
94.70%
2.72%
N/A
0.66%
N/A
3.97%
30.00%
9.60%
81.50%
10.20%
76.00%
1.33%
96.07%
2.21%
91.48%
1.05%
97.10%
1.60%
96.05%
0.16%
99.80%

Database description: A: PolyU Contactless Finger Knuckle Images Database (Version 1, 501 subjects) [30]; B: PolyU Contactless Hand Dorsal Images
Database (712 subjects) [31]; C: PolyU Contactless Finger Knuckle Images Database (Version 3, 221 subjects) [32]; D: PolyU 3D Finger Knuckle
Images Database (D(1) 3D Images, D(2) 2D Images, 190 subjects) [33]; E: Tsinghua University Finger Vein and Finger Dorsal Texture Database (610subjects) [34].

system. At present, popular deep neural network-based object detection and segmentation methods cannot
simultaneously recover the orientation angle of finger knuckle and segment knuckle region with highprecision. However, accurate estimation of the presented finger knuckle is a key factor for the success of
online finger knuckle identification.
B. Our Works and Contributions
Completely contactless and online finger knuckle identification is a difficult problem, largely due
to the complex capabilities required to match deformable knuckle crease patterns that are often acquired/presented from the varying viewing angle. In order to accurately and robustly match such patterns,
a deep learning-based framework that can recover rotational and translational invariant features should bedesigned. Another open problem in building a completely contactless finger knuckle identification system
is related to accurate detection and segmentation of the ROI under rotational, translation, and scale changes.
Conventional methods in the literature for knuckle detection and segmentation are inadequate and cannot
meet such expectations for completely contactless systems. Table I summarized key advantages of oursystem and the related system.
This paper introduces an accurate and robust contactless finger knuckle feature representation frameworkand develops completely contactless finger knuckle identification capabilities for at-a-distance real-world
applications. It makes significant contributions toward discovering full potential from the deep learning
capabilities and such objectives are yet to be pursued in the biometrics literature.
Key contributions from this paper can be summarized as follows:• We develop a new deep learning-based contactless finger knuckle identification framework withhigh generalization capability for operating on different contactless finger knuckle databases. A new
Rotation and Shift Invariant Loss (RSIL) function has been developed to address the degradation inmatching performance due to high intra-class variations among knuckle crease patterns. The proposed
loss is generalized and also has the potential to be employed in many other contactless matchingtasks. Outperforming experimental results presented in section IV-A to IV-E of this paper using fivedifferent contactless finger knuckle databases, under challenging protocols and using cross-database
performance evaluation, validate the effectiveness of the proposed framework for contactless finger
knuckle identification.
• We also design and develop a completely contactless finger knuckle detector to accurately detect,segment, and predict respective knuckle orientations from the multiple hands in the real-world images.

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

4

4

128

I/

conv2

64

4

128
res4

conv4

1

4

128

I/

128
res3

I/

128

+

4

128
res2

+

I/
4

128

4

128
res1

+

I/

128

I/
4

4

128
conv3

I/

64

I/
4

+

I/

128

I/
4

We use Yolov5 [28] as the backbone with circular smoothing labels (CSL) [36], and introduce angularloss to predict the oriented bounding box to accurately segment the ROI with knuckle patterns. Our
experimental results in section IV-F to IV-G of this paper also validate the effectiveness of angularloss to enhance the performance for the completely contactless finger knuckle detection.
• We design a cross-platform online finger knuckle identification system for completely contactless andreal-world applications. The system can detect and match finger knuckle patterns from finger images
acquired under complex background, using any ordinary smartphone or general laptop camera. It’s
the first completely contactless and online system (please see attached video samples recorded fromits live operation), to the best of our knowledge, and a significant step to avail full potential of
knuckle biometrics for social good.

conv5

I/
2

32

128

I/
4

128

I/
4

conv1

Fig. 1. Architecture of a fully convolutional network employed as the backbone in our framework.

The rest of this paper is organized as follows: In section II we introduce our deep learning framework
for completely contactless finger knuckle matching. This section includes the details on the proposed loss
function and its differentiability requirements for the network training. Section III introduces developed
solution to accurately detect the finger knuckle ROI and their respective orientations, from the completely
contactless hand dorsal images with complex background. Section IV presents all the experimental
results relating to the identification including within-database experiments and cross-database experimental
results, and the comparative performance for the 3D finger knuckle database. This section also includes
performance evaluation from a completely contactless finger knuckle identification developed during thiswork. Finally, the key conclusions from this work are summarized in section V of this paper.
II. M ATCHING C OMPLETELY C ONTACTLESS F INGER K NUCKLE I MAGES
Most relevant information from the knuckle crease patterns can be described using the inter-pixel
relationship among the gray-level arrangements in the segmented images. Such a relationship is expected
to be better preserved in the output templates generated by a fully-convolutional network output, insteadof the fully connected output as in prior work [8]. This is the motivation for choosing a lightweightresidual network [37] with fully convolutional output, which is also referred to as a residual feature
network (RFN), as the backbone for matching contactless knuckle images. Fig. 1 illustrates backbone
architecture for the contactless finger knuckle matching. We use triplet architecture [38] to train the
convolutional kernels of this network. Three such shared networks are interconnected in parallel to enable

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

5

Rotation and Shift I nvar iant L oss
Feedfor war dpixelsPositive

CommonRegionSharedM odel

M RSDAnchorpixelsSharedM odel

RSI L

Feature M ap Shift

M RSD
Negative

angle

SharedM odelCommonRegionFeature M ap Rotation
Backpropagation

Fig. 2. An overview to illustrate the use of rotation and shift invariant loss function (RSIL) to train the network under triplet architecture.
We will use MRSD to compute the similarity between two output templates (feature maps shown on the right). The common knuckle regionis illustrated with red boundary after accommodating for potential rotational and translational changes.

forward and backward propagation of the gradients of network loss during the offline training. Completely
contactless and normalized finger knuckle images often illustrate significant rotational misalignments anddeformations among the curved knuckle creases. Therefore, conventional triplet loss or shifted triplet
loss is not adequate to address such rotational and translational changes. We therefore design a new loss
function which can enable the network to effectively learn the most robust features that are also rotational
invariant due to multiple degrees of freedom during the imaging. This new loss function is detailed in thefollowing section.
A. Rotation and Shift Invariant Loss Function
In this section, we systematically introduce a new loss function RSIL and desire such requirement for
its differentiability. With a differentiable loss, the backpropagation process can proceed smoothly, and thelearnable parameters can be updated to until RSIL is reduced to some minimum value. Our backbone
network is trained using the triplet architecture for which the conventional triplet loss [38] can be statedas in the following:
N

TL =

1 X
[||T (Iia ) − T (Iip )||2 − ||T (Iia ) − T (Iin )||2 + m]+
N i

(1)

where N designates the batch size employed during training iteration; T (I a ), T (I p ), T (I n ) respectively
represent the output feature matrix (template) generated by the network from the input anchor, positive
and negative ROI image I a , I p , and I n , The symbol [◦]+ represents the same function as max(◦, 0) and isused to ensure a non-negative loss. The hard margin parameter m can effectively determine the distance
between different class clusters by pushing them away during the training process.
The triplet loss computed in (1) is however not appropriate for matching completely contactless finger
knuckle images. Normalized contactless finger knuckle images from the same client’s fingers will not only
illustrate the spatial translation of knuckle crease patterns but are often accompanied by rotations which

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

6

can be due to the involuntary local deformations while presenting the fingers for the contactless imaging.
In order to robustly accommodate such intra-class changes in the contactless finger knuckle patterns, we
alter the original triplet loss function and introduce RSIL as follows:N

1 X[L(T (Iia ), T (Iip )) − L(T (Iia ), T (Iin )) + m]+
RSIL =N i(2)

In order to adapt to the intra-class knuckle patterns with varying degrees of rotational and translationalchanges, and balance the performance and speed, the translational and rotational parameters are set as
the hyperparameters. The L(T1 , T2 ) in (2) computes the minimum distance between the two templates
Dsw ,sh ,θ (T1 , T2 ) after accounting for shift and rotation in the range of −SW ≤sw ≤SW , −SH ≤sh ≤SH , −Θ≤θ≤Θ, called minimal rotational and shifted distance (MRSD) in (3). We use SW , SH represent the extent ofhorizontal and vertical pixel translations respectively, and Θ for rotaion angle. The distance Dsw ,sh ,θ (T1 , T2 )computes the pixel-wise MSE value when template T1 is shifted sw pixel along x-axis and sh pixel along
y-axis and rotated by θ angle as also shown in Fig. 2.
L(T1 , T2 ) =min−SW ≤sw ≤SW ,−SH ≤sh ≤SH ,−Θ≤θ≤Θ

1

Dsw ,sh ,θ (T1 , T2 ) =Dsw ,sh ,θ (T1 , T2 )X

|Csw ,sh ,θ |(3)(s ,s ,θ)(T1 w h bn, mc − T2 [x, y])2(4)(x,y)∈Csw ,sh ,θ

where Csw ,sh ,θ represents the common region between two feature maps after one feature map is translated
along the x-axis by sw , translated along y-axis by sh pixels, and rotated by θ degrees, as shown in theFig. 2. In (4), the b◦c is the operator for conventional floor function. When feature maps are rotated andshifted, using affine transformation, the new feature map T (sw ,sh ,θ) bn, mc entries can be directly recoveredfrom the source T [x, y] depending on the sampling method as follows:T(sw ,sh ,θ)bn, mc =

H X
W
X
y

T [x, y]p(n − x)q(m − y)(5)xwhere the height and width of T [x, y] is represented by H and W , while [x, y] represents the original
pixel positions before the affine transformation; p() is the interpolation operation along the x-axis whileq() is the interpolation operation along the y-axis. Since use bilinear interpolation, the above equation canbe rewritten as follows:T(sw ,sh ,θ)bn, mc =

H X
W
X
y

T [x, y]max(0, 1 − |n − x|)max(0, 1 − |m − y|)(6)x

The training of network will require us to compute the gradients, or the partial derivatives, of RSIL
which will be iteratively backpropagated during the training. We can firstly compute the partial derivativeof T (sw ,sh ,θ) bn, mc with respect to T [x, y] as in the following.
H X
WX∂T (sw ,sh ,θ) bn, mc=
max(0, 1 − |n − x|)max(0, 1 − |m − y|)∂T [x, y]yx(7)

As for the (T a , T p ) pair, we can assume when the T a is rotated by angle of θap and shifted with (swap ,shap ) pixels, the minimum distance Dswap ,shap ,θap (T a , T p ) can be achieved. Therefore the resulting loss
L(T a , T p ) = Dswap ,shap ,θap (T a , T p ). Similarly, for the (T a , T n ) pair, with the alignment by swan , shan , θan ,JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

7

minimum distance Dswan ,shan ,θan (T a , T n ) can be computed. We can then compute the partial derivationof RSIL with respect to Tip and Tin , respectively as follows:
if (x, y) ∈/ Cswap ,shap ,θap or RSIL = 0

0,
∂RSIL
a(swap ,shap ,θap )
=
(8)
2(Ti
bn, mc − Tip [x, y])∂Tip, otherwise−
N × |Cswap ,shap ,θap |
if (x, y) ∈/ Cswan ,shan ,θan or RSIL = 0

0,
∂RSIL
a(s
,s
,θ )
=
(9)
2(Ti wan han an bn, mc − Tin [x, y])∂Tin,otherwiseN × |Cswan ,shan ,θan |where Cswap ,shap ,θap represents the common region between the anchor and positive sample, while Cswan ,shan ,θanrepresents respective common region between anchor and negative sample after such rotation and translaa(s,s,θ )tion. Translation and rotation anchor template will generate respective interpolated templates, Ti wap hap ap
a(s
,s
,θ )
and Ti wan han an . The partial derivative of Tia can be computed as follows:0,
if (x, y) ∈/ Capn or RSIL = 0




a(s
,s
,θ
)
wap
ap
p
hap


bn, mc − Ti [x, y]) 2(Ti
∂RSIL
× K+
=
(10)
N × |Cswap ,shap ,θap |
a

∂Ti

a(s
,s
,θ
)
 −2(T wan han an bn, mc − T n [x, y])


i
i

× L, otherwise
N × |Cswan ,shan ,θan |
a(s

,s

,θ

)

where K and L respectively represents the partial derivation of ∂Ti wap hap ap bn, mc/∂Tia [x, y] and
a(s
,s
,θ )
∂Ti wan han an bn, mc/∂Tia [x, y] which can be computed using (7), and Capn represents common region
between tow spatial regions i.e. Cswan ,shan ,θan ∪ Cswap ,shap ,θap as Capn . The RSIL loss computed using theintermediate template. Tia , Tip and Tin will be backpropagated using the chain rule to update the learnable
parameters of neural network during the training.
III. F INGER K NUCKLE D ETECTION IN A W ILD
Real-world deployment of completely contactless finger knuckle identification capabilities will require
accurate detection of the such ROI from a complex background that is often expected. There are several
possibilities to train a deep network based popular object detectors, e.g. YOLO [22], [23], [24], [25],[28], Mask R-CNN [39], to detect and segment knuckle ROI. Such attempts have also been recently usedin the literature, e.g. [8] uses Mask R-CNN [39] to detect and segment Knuckle ROIs. In this context,
we can note that the YOLOv4 [25] object detection model has shown to be superior i.e., the fastest andmost accurate detectors in terms of both speed and accuracy when compared to other state-of-the art
object detection model. However, such methods cannot predict the orientation of knuckle ROI which isindeed required to accurately match completely contactless knuckle images. Therefore, accurate predictionof the orientation of knuckle ROI is desirable to align two feature maps for enhancing the accuracy andefficiency. The YOLO series model can be easily adapted for predicting oriented bounding boxes. We,
therefore, incorporate circular smooth labels to smooth the prediction of knuckle orientation classes using
YOLOv5 [28] model. Our approach for predicting oriented bounding boxes corresponding to ROI undercomplex background images is detailed in the following.
The complexity and detection speed offered by YOLOv5 can meet expectations for inference time andaccuracy. However, a trained YOLOv5 model can just detect horizontal bounding boxes which cannot
offer orientation information and will also segment unwanted background details. Therefore, to address
such challenges for our problem, an oriented bounding box is predicted instead of the conventional or
horizontal bounding box. The loss for the oriented bounding boxes is largely contributed from the angularperiodicity loss, along with exchangeability of edges loss, when bounding boxes are defined using fiveJOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

8

(a)

(b)

(c)

Fig. 3. Defining oriented rectangles and encoding for knuckle detection:(a) Five-parameter definition with 180◦ angular range. (b) CSL [36]coding with periodic Gaussian window function, (c) CSL smoothing for one-hot label during network training.

parameter method (definition in OpenCV library) [36]. Therefore, we use long side definition of boundingboxes i.e. five-parameter method with 180◦ angular range. This is illustrated in figure 3a and can address
problems relating to the exchangeability of edges as detailed in [36].
It can be simpler to train a classification model to predict (knuckle) bounding box orientation angles ascompared to the regression. A periodic coding method called referred to as Circular Smooth Label (CSL)
soft coding with periodicity window function is introduced in [36] and can address two key problems
for the accurate prediction of knuckle orientations: (i) it can distinguish different class relationships asshown figure 3c. When the ground truth orientation angle is 1 deg, the loss between the ground truth and2 deg prediction is smaller than the loss between the ground truth and 177 deg prediction; (ii) it can also
address the periodicity of predicted angles as also shown from illustrations in figure 3b.
Below equation illustrates the encoding of class labels using CSL for the orientation prediction, whereg(x) represents the window function to smooth one-hot label, r is the radius of window function whileθ represents the orientation angle of the respective bounding boxes. The Gaussian window function havethe periodicity of t i.e. g(x) = g(x + t), and we use the t=180 deg representing as the cycle of predicting
orientation labels (figure 3b).
(g(x), θ − r < x < r + θ
CSL(x) =(11)0,otherwise
The original YOLOv5 model uses the loss function which have three components i.e. Loss = Lossbbox +Lossobj +Lossclass . Since the oriented bounding box prediction is based on the YOLOv5, we also introduce
angular orientation loss in this Loss to enable accurate detection of knuckle ROI. Therefore, such additionof Lossangle to YOLOv5 loss function, the total loss function can be stated as follows:
Loss = Lossbbox + Lossobj + Lossclass + Lossangle

(12)

2

S
X
X
Lossangle =Iijobj[Piˆ(a)log(Pi (a)) + (1 − Piˆ(a))log(1 − Pi (a))]
i=0a∈[0,180)(13)

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

9

We use the binary cross-entropy to compute the angular classification loss in (12). In above equation(13), the S 2 represents the output grid size, and the Iijobj represents the computation of the loss when
object fall in the grid cell.
IV. E XPERIMENTS AND R ESULTS
This section presents thorough experimental results, using publicly available databases, to ascertain
various aspects of the performance relating to its possibility for real-world applications i.e. match accuracy,complexity, and also detection accuracy. Our experiments are firstly organized to evaluate within database
performance (WithinDB) which uses some part of the database for the training while using some otherindependent part of this database for the performance evaluation. It is important to note that the crossdatabase performance evaluation (CrossDB) is highly desirable to address the limitations of currently
available methods as none of the finger knuckle recognition these have presented such in CrossDB results
to the best of our knowledge. Therefore, CrossDB performance evaluation results are also in section IV-B.
This section also includes model complexity analysis (section IV-D). Detection performance and online
performance is respectively presented in section IV-F and section IV-G.
Choice of key baseline matching methods: Matching contactless finger knuckle images using a
deep network-based approach has shown to offer state-of-the-art performance. The FKNet [8] has shown
to offer outperforming results over several other popular networks, i.e. ResNet-50 [15], VGG-16 [40],AlexNet [41], DenseNet-121 [42], and SE-ResNet-50 [43], and therefore a judicious choice for the baseline
performance comparisons. Involuntary deformations among the intra-class knuckle patterns are one of the
key challenges in accurately matching completely contactless knuckle images. As compared to the useof convolution layer or dilated convolution [44], the deformable convolution [45] can address such local
knuckle deformations by sampling at different locations and with different weights. Therefore, we also
replace the convolution layer in our backbone (Fig. 1) with the deformable convolution layer, and suchan attempt is referred to as DefConvNet. EfficientNet [17] is another promising but recent backbone foronline knuckle matching. Both DefConvNet and EfficientNet have not been attempted in the literature,and due to their outperforming results over other strong baselines, such results are also presented for thecomparisons.
It is useful to note that the important challenge or focus of this work is on developing a completely
contactless system for real-world applications. Therefore, performance evaluation on publicly available
datasets from [18] is of key importance as it is the only database that provides highly deformed knuckle
images or includes such representative images for completely contactless applications.
A. Within Database Performance Evaluation
1) Completely Contactless Finger Knuckle Images Database (Version 3.0): This publicly available
database [32] is most challenging and relevant to ascertain the effectiveness of our approach for completely
contactless finger knuckle identification. It offers finger knuckle images from 221 different subjects andthe knuckle crease deformations are significant (some even excessive to meet reasonable expectations
during the deployments). It also provides two session images, and a baseline method tertiary contrast
context vector (TCCV), and we use the same protocols as in this reference [18]. Two session protocols
will generate 624 (104 × 6) genuine scores, 64272 (104 × 103 × 6) imposter scores. The network is
trained using independent images from 117 (221 − 104) different subjects who have not provided second
session images. Similarly, the first session experiments will generate 1326 (221 × 6) genuine match scoresand 291720 (221 × 220 × 6) imposter match scores. Comparative experimental results from the firstsession and two session protocols, using the receiver operating characteristics (ROC) and cumulative
match characteristics (CMC) are presented in Fig. 4.
Experimental results in figure 4a and 4b indicate that our approach can achieve outperforming results,
not the just for the verification tasks (ROC) but also for the recognition applications (CMC). It is worth
noting that this is the most challenging database but the most suitable public database for the completely

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

10

Receiver Operating Characteristics

Cummulative Match Characteristics

1

1

0.9

0.95

0.7

Average Recognition Rate

Genuine Acceptance Rate

0.8

0.6
0.5

TCCV [18]; EER: 17.79%
EfficientNetV2-S [17]; EER: 4.06%
FKNet [8]; EER: 6.46%
Ours; EER: 2.21%
DefConvNet [45]; EER: 5.41%

0.4
0.3
0.2
0.1
0

10 -5

10 -4

10 -3

10 -2

10 -1

0.9

TCCV [18]EfficientNetV2-S [17]FKNet [8]Ours
DefConvNet [45]

0.85

0.8

10 0

0.75
1

False Acceptance Rate

2

3

4

(a)

1

5

6

7

8

9

10

Cummulative Rank(b)
Cummulative Match Characteristics

Receiver Operating Characteristics

0.8

0.9

0.7

0.8

Average Recognition Rate

Genuine Acceptance Rate

0.6
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

TCCV [18]; EER: 18.41%
EfficientNetV2-S [17]; EER: 48.72%
FKNet [3]; EER: 49.24%
Ours; EER: 16.65%
DefConvNet [45]; EER: 39.31%
10 -4

10 -3

10 -2

False Acceptance Rate

(c)

10 -1

TCCV [18]EfficientNetV2-S [17]FKNet [8]Ours
DefConvNet [45]

0.5
0.4
0.3
0.2
0.1

10 0

0
1

2

3

4

5

6

7

8

9

10

Commulative Rank(d)Fig. 4. Experimental results on PolyU Contactless Finger Knuckle Images Database (Version 3.0) [32] using one-session database:Comparative ROC plots (a) and CMC plots (b). Respective results from two-session database using comparative ROC plots (c) and CMC
plots (d)contactless finger knuckle identification focussed in this paper. We can also note that the EfficientNetV2S, can offer better performance than FKNet [8] while DefConvNet can offer better performance than
TCCV [18]. This may justify the choice of other baseline networks considered in our work for completely
contactless knuckle matching. Our experiments have also indicated that if we just change the convolution
layer with deformable convolution kernels, it is not adequate to accommodate finger knuckle deformations,and there is a noticeable degradation in the performance. Experimental results using the two-session
database are also shown in Fig. 4. In view of the extensive degree of deformations on the two-session
data, the matching accuracy is much lower than those observed from the one-session protocol. However,the proposed method can still offer the best performance than the other baseline methods considered inthis work.
2) HKPolyU Contactless Hand Dorsal Images Database: This publicly available database [31] offers
knuckle images from the largest number of subjects, as compared to any other public database, and wasJOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 202211

therefore also considered for the performance evaluation. This database is acquired from 712 different
subjects and each subject has provided five different images. We used index finger knuckle images andthe first sample is used to fine-tune the model while the rest of the four images are used as the test
samples. This kind of protocol on this database is exactly same as used in FKNet [8]. Therefore, it will
generate 2848 (712 × 4) genuine and 2024928 (712 × 711 × 4) imposter match scores. This databasehas been acquired while finger poses are largely straight or fixed, and therefore the knuckle creases havemuch fewer deformations than those for the completely contactless database (Version 3.0) [32] used in thelast section. It is also useful to note that it’s contactless, instead of being completely contactless with six
degrees of freedom, database as the fingers rest on a fixed surface while they are imaged using a mobile
camera. It is largely due to such factors that the average performance on this database is much better thanthose in Fig. 4.

1

Culmulative Match Characteristics

Receiver Operating Characteristics

1

0.95

0.8

Recognition Rate

Genuine Acceptance Rate

0.9

0.7
0.6
0.5
0.4
0.3

FKNet [8]; EER: 7.7%
EfficientNetV2-S [17]; EER: 1.82%
Ours; EER: 1.33%
DefConvNet [45]; 3.88%
10 -4

10 -2

False Acceptance Rate(a)FKNet [8]EfficientNetV2-S [17]Ours
DefConvNet [45]

0.9

0.85

0.8

10 0

0.75
2

4

6

8

10

Culmulative Rank(b)Fig. 5. Experimental results on PolyU Contactless Hand Dorsal Image Database [31] using comparative ROC plots (a) and CMC plots in(b).

It can be observed from the comparative performance shown in Fig. 5 that the proposed method can alsoachieve outperforming results on this database. Ours can get the best performance with a 1.33% EER, and96.07% of average rank-one recognition rate, followed respectively by EfficientNetV2-S, DefConvNet,
and FKNet. It is worth noting that the FKNet achieves worst result as compared to the other models.
With classification task, the EfficientNetV2-S model is still better than the FKNet, possibly because
EfficientNetV2-S have larger field of view than the FKNet. Meanwhile, EfficientNetV2-S use the FusedMBConv [46] block while the FKNet is based on the ResNet-50 [15]. The Fused-MBConv block can enable
better network learning than the residual block, and therefore EfficientNetV2-S achieves better performance
than the FKNet. This is the plausible explanation for the superior performance from EfficientNetV2-S
than the FKNet.
3) Contactless 2D Finger Knuckle Images from 3D Finger Knuckle Database: The HKPolyU 3D
Finger Knuckle Images Database [33] is another public database, and it also provides 2D finger knuckle
images (in addition to 3D finger knuckle images) which are from different subjects than those employedin the previous two databases. Therefore, this database from 190 different subjects was also employed
for the performance evaluation. We use the first session 190 subjects images to fine-tune models and thesecond session images are used for the performance evaluation. This protocol is exactly the same as in[8] and generates 1140 (190 × 6) genuine and 215460 (190 × 189 × 6) imposter match scores. It canbe observed from the comparative performance plots in Fig. 6 that the proposed approach can also offer
better performance an EER of 1.60% and with average rank-one accuracy of 96.05%. The performancefrom EfficientNetV2-S is however quite close for the identification task.

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

1

12Receiver Operating Characteristics

Culmulative Match Characteristics
1
0.98

0.9
0.85

Recognition Rate

Genuine Acceptance Rate

0.95

0.8
0.75
0.7
0.65
0.6

EfficientNetV2-S [17]; EER: 1.84%
FKNet [8]; EER: 5.74%
Ours; EER: 1.60%
DefConvNet [45]; EER: 5.01%

0.55
10

-4

10

-2

0.96
0.94

EfficientNetV2-S [17]FKNet [8]Ours
DefConvNet [45]

0.92
0.9

10

0.88

0

2

4

False Acceptance Rate

6

8

10

Culmulative Rank
(b)

(a)

Fig. 6. Experimental results on 2D images from PolyU Contactless 3D Finger Knuckle Images Database [33] using comparative ROC plots(a) and CMC plots in (b).

B. Cross Database Performance Evaluation
This section provides results from cross-database experiments that were performed to ascertain thegeneralization ability of the proposed method. In this set of experiments, we used the same model thatwas trained for the results shown in Fig. 4 i.e. model trained using the contactless finger knuckle images
database (Version 3.0) [32]. This model was used to test the performance from all finger knuckle images
in the Tsinghua Finger Vein and Finger Dorsal Texture Database (THU-FVFDT3) [34] and the HKPolyU
Contactless Hand Dorsal Images Database [31].

1

Culmulative Match Characteristics

Receiver Operating Characteristics

1
0.99
0.98

0.96

Recognition Rate

Genuine Acceptance Rate

0.98

0.94
0.92
0.9
0.88
0.86

EfficientNetV2-S [17]; EER: 0.24%
FKNet [8]; EER: 6.00%
Ours; EER: 0.16%
DefConvNet [45]; EER:0.61%
10 -4

10 -2

False Acceptance Rate(a)

EfficientNetV2-S [17]FKNet [8]Ours
DefConvNet [45]

0.97
0.96
0.95
0.94
0.93

10 0

0.92
2

4

6

8

10

Culmulative Rank(b)Fig. 7. Cross database experimental results on Tsinghua Finger Vein and Finger Dorsal Texture Database [34], using comparative ROC
plots (a) and CMC plots in (b)1) Tsinghua Finger Vein and Finger Dorsal Texture Database: This database [34] has been acquired
from the fingers of 610 subjects, and each subject has provided 4 samples of finger dorsal images. Fromsuch finger dorsal images, we use our knuckle detection model (section III) to automatically detect andsegment ROI of finger knuckle images for this test dataset. During this cross-database experiment, we
generate 2440 (610 × 4) genuine and 1485960 (610 × 609 × 4) imposter match scores. The results from

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

13

this set of experiments are shown in Fig. 7. Intra-class variations among the knuckle images in this dataset
are small and are the reason that all models can achieve relatively high match performance. These crossdatabase results indicate that our approach can achieve outperforming results with EER of 0.16% and an
average rank-one recognition rate of almost 100%. It can also be observed that the EfficientNetV2-S can
also achieve very high performance on this database. This is largely due to the fact that this database [34]has been acquired in the indoor environment with fixed imaging conditions, such as distance, angle, andillumination, and therefore knuckle creases of fingers knuckle have higher quality and little intra-classvariations. This is unlike the hand dorsal image database [31] which has been acquired in a more complex
and outdoor environment with high intra-class variations.

1

Culmulative Match Characteristics

Receiver Operating Characteristics

1

0.9

Recognition Rate

Genuine Acceptance Rate

0.8

0.6

0.4

0.2

0

EfficientNetV2-S [17]; EER: 22.62%
FKNet [8]; EER: 9.04%
Ours; EER: 3.33%
DefConvNet [45]; EER: 10.67%
10 -4

10 -2

False Acceptance Rate
(a)

0.8

0.7

EfficientNetV2-S [17]FKNet [3]Ours
DefConvNet [45]

0.6

0.5

10 0

0.4
2

4

6

8

10

Culmulative Rank(b)Fig. 8. Cross database experimental results on PolyU Contactless Hand Dorsal Images Database [31] using comparative ROC plots in (a)and CMC plots in (b).

2) Index Finger Knuckle of the HKPolyU Contactless Hand Dorsal Images Database: Our crossdatabase experiments on this database [31] use all index finger images from 712 different subjects.
Therefore, such experiments will generate 3560 (712 × 5) genuine and 2531160 (712 × 711 × 5) imposter
match scores. Such comparative experimental results are shown in Fig. 8. These results show that the
proposed method can achieve outperforming results, indicating strong generalization ability from our
method. However, the performance from EfficientNetV2-S has significantly degraded (EER of 22.62%)
under the cross-database scenario, as compared to other models. As compared to the within-database
performance that is shown in Fig. 5, performance from our method has degraded from 1.33% to 3.33% of
EER. However, as for the EfficientNetV2-S model, its performance drop has been too dramatic i.e. fromthe 1.82% to 22.62% of EER. This can be largely due to the fact that EfficientNetV2-S is a classification
network, and trained with the soft-max loss on a database [32] with 104 classes. However in this database
there are 712 different classes or a relatively large number of unseen class numbers which can pose
additional challenges during such cross-database performance evaluation.
C. 3D Finger Knuckle Images DatabaseIn the previous part, our experimental evaluation indicated that the proposed method can achieve
outperforming results on contactless 2D finger knuckle databases, both from the within- and crossdatabase performance evaluation. This has further encouraged us to ascertain the performance for matchingcontactless 3D finger knuckle images. We use a publicly available contactless 3D fingerprint database [33]and use a state-of-art method [8] and other baselines used on this database i.e. SGD [10], DenseNet [42],JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

14

and SE-ResNet [43]. The pre-processing and the match protocols used during our evaluation are exactly
the same as in [8] for this database. Therefore such protocol will generate 1140 (190 × 6) genuine and215, 460 (190 × 189 × 6) imposter match scores. These experimental results are shown in Fig. 9 using
comparative ROC and CMC plots. These results also indicate that our method can achieve outperforming
results, both for the finger knuckle verification and identification. Our method can achieve an EER of
1.05% while the EER of the FKNet is 2.4%. We can note that the backbone used in FKNet is deeper
than in our method while both of these have used the residual block. However, with the introduction of
new RSIL loss, our method can offer better performance than the FKNet.

1

Culmulative Match Characteristics

Receiver Operating Characteristics

1

0.9

0.8

Recognition Rate

Genuine Acceptance Rate

0.9

0.7
0.6

FKNet [8]; EER: 2.4%
SGD [9]; EER: 4.2%
DenseNet [42]; EER: 10.4%
SE-ResNet [43]; EER: 13.3%
Ours; EER: 1.05%

0.5
0.4
0.3
0.2

10 -4

10 -2

False Acceptance Rate
(a)

0.8

FKNet [8]SGD [9]DenseNet [42]SE-ResNet [43]
Ours

0.7

0.6

10 0

0.5
2

4

6

8

10

Culmulative Rank(b)Fig. 9. Comparative experimental results for matching 3D finger knuckle images [33] using comparative ROC plots in (a) and CMC plotsin (b).

D. Model Complexity Analysis
This paper is focused to develop a completely contactless and online finger knuckle identification system,and therefore we must choose a model that can meet the requirements of matching speed for online
identification, and consider the expected trade-off between match accuracy and the speed. Therefore, thissection presents a brief analysis of the complexity. Table II summarizes the computational complexityof various methods, along with the storage requirements and the corresponding feature extraction time.
From the table, it can be observed that the average computational time for extracting the finger knuckle
features is only 0.01957s, and storage requirements or the learnable weights are significantly smaller thanthe EfficientNetV2-S [17] and FKNet [8]. It is, therefore, reasonable to conclude from the experimental
results in this section, that the storage complexity and time complexity of our method are significantly
smaller while it can achieve the best performance from the proposed RSIL loss function.
E. Discussion
This paper has introduced a new loss function, RSIL, and has shown to offer encouraging results
for matching completely contactless finger knuckle images. In order to ascertain the merit of this loss
function over other related loss functions, i.e. triplet loss (TL) [38], and soft-shifted triplet loss (SSTL)[37], we performed additional experiments. These experiments were performed using the same backbone
network (Fig. 1) on completely contactless finger knuckle database (version 3) [32] which has been focusof this work. These results are summarized in Fig. 10, both for the verification and identification, using

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

15

TABLE IIC OMPARISON TIME AND SPACE COMPLEXITY OF DIFFERENT NEURAL NETWORK

Model

Prams
(M)

Template
Size

FLOPs
(B)

DefConvNet
EfficientNetV2-S [17]FKNet [8]
Ours

0.36M
20.18M
7.28M
0.46M

32 × 32
9×9
8 × 1232 × 32

1.29B
5.40B
2.8B
1.39B

FeatureExtraction(s)
0.02880s
0.07501s
0.04492s
0.01957s

Time complexity is the average time on the Ubuntu 22.04 with GeForce RTX
2080 GPU and I7-7800X CPU. For the STTL, the shift size is 4; for the RSIL,
the shift size is 4, and add rotation with 4 angles. The FKNet and EfficientNetV2S serial models obey their same input image size of the original paper [8] and[17], respectively.

different protocols. These comparative results also indicate the effectiveness of the proposed loss function
for accurately matching completely contactless finger knuckle images.
We also present the respective class activation map in Fig. 11, using GRAD-CAM [47] visualization,
to illustrate the salient knuckle regions contributing significantly to the output feature maps. The map
is chosen such that the more red and dark regions illustrate higher importance of corresponding spatial
locations. Our loss (RSIL) can account for frequent rotational and translational changes in such knuckle
texture patterns and enable the model to more effectively learn such rotation and shift invariant local
features. As for texture matching scenario, the key points, e.g. knuckle crease endings or the bifurcations,
are more robust for matching, because these points are relatively stable to the changes in the illuminationor view angle. It can be observed from the images in figure 11b that when the model is trained withthe proposed loss function, then the output feature maps will emphasize (focus) on these key points andimparts higher weights on these key points as compared to the corresponding images in figure 11c or
11d. This can be more clearly visualized from a dark point in the original input knuckle images in figure
11a i.e. only our method can dominantly capture this key point information.
Our experimental results in section IV-A also suggest that EfficientV2-S model can offer better performance than FKNet [8], and even on the Tsinghua finger knuckle database (figure 7). The EfficientNetV2
model has shown to outperform ResNet on the ImageNet [48], i.e. it can extract robust features than
ResNet as EfficientNetV2 replaces the residual block with inverted residual block, and uses MBConv asa block unit. As for MBConv block, it uses depth-wise convolution to decrease the learnable parameters
and uses Squeeze-Excited block for the channel attention. It should be noted that the EfficientNetV2-S
is deeper than the FKNet. On the contrary, the FKNet use the convolution layers of ResNet-50 up to
Conv3 x as the feature extract backbone. EfficientNetV2 uses more light, advanced and efficient module
than ResNet. Due to these factors, the EfficientNetV2-S can perform better than the FKNet for the finger
knuckle identification tasks.
The generalization capability of the RSIL is also quite good as indicated from the cross-databaseexperimental results. As indicated from such evaluation in Fig. 8, the EER achieved from our model on
hand-dorsal database is 3.33% which is although still higher as compared to within-database results in
Fig. 5. However, for rest of the models, such cross-database performance degradation is huge as comparedto ours. For example, on the hand dorsal database [31], the cross database EER value of DefConvNet
drops from 3.88% to 10.67%, and the EfficientNetV2-S drops from 1.82% to 22.62%. In other words,
such performance indicates that our RSIL loss function can alter the back propagation during training
process to learn more effective model weights, which can also be noticed from the class activation maps
illustrated in Fig. 11. In summary, it is reasonable to conclude that the proposed method can offer thebest performance and superior generalization capability as compared to the rest of the baseline models

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

1

16

Culmulative Match Characteristics

Receiver Operating Characteristics

0.99
0.98
0.97

0.9

Recognition Rate

Genuine Acceptance Rate

0.95

0.85
0.8

TL [38]; EER: 2.86%
STTL [37]; EER: 2.57%
Ours; EER: 2.21%

0.75
0.7
10

-4

10

-2

0.96
0.95
0.94

TL [38]STTL [37]
Ours

0.93
0.92

10

0.91

0

2

False Acceptance Rate

4

(a)

1

6

8

10

Culmulative Rank(b)

Culmulative Match Characteristics

Receiver Operating Characteristics

0.8

0.8

0.7

Recognition Rate

Genuine Acceptance Rate

0.75

0.6

0.4

TL [38]; EER: 25.54%
STTL [37]; EER: 18.35%
Ours; EER: 16.65%

0.2

0

10 -4

10 -2

False Acceptance Rate
(c)

0.65
0.6
0.55
0.5

TL [38]STTL [37]
Ours

0.45
0.4

10 0

0.35
2

4

6

8

10

Culmulative Rank(d)Fig. 10. Comparative match performance from our network trained network with different loss functions on PolyU Contactless Finger
Knuckle Images Database (Version 3.0) [32] using one-session database: Comparative ROC plots (a) and CMC plots (b). Respective resultsfrom two-session database using comparative ROC plots (c) and CMC plots (d).

considered in this paper. Our method also offers minimal time complexity to extract the features, as can beobserved from Table II, which is another reason for its suitability in completely contactless and real-world
finger knuckle identification.
We can observe from the result in Fig. 4 that the performance of all models is relatively lower on
the deformable finger knuckle dataset [32], as compared to the rest public finger knuckle datasets. As
for the deformable dataset [32], the volunteers bend their fingers from 0◦ to 90◦ . During such finger
knuckle deformations, the texture formation from the knuckle creases is altered and this can degrade theknuckle detection and the matching performance. Firstly, we want to ascertain the changes in the detection
performance with change in the extent of finger knuckle bending in this dataset (detail on two detectors,
i.e. Our(x) and Our(m), are provided in Section IV-F). We present such detection performance using thefirst session of this dataset with 1326 (221 × 6) test finger knuckle images. Since each subject is provided
with 6 samples, and the 6 samples represent the degree of bending of the finger in turn, roughly in thesteps of 0◦ , 18◦ , 36◦ , 54◦ , 72◦ , and 90◦ . Therefore, we also use this data to test detector precision andrecall in these samples, and these results are summarized in Table III. We set the detector confidenceJOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

17

(a)

(b)

(c)

(d)

Fig. 11. Comparative class activation maps (CAM) on the images from 104th subjects samples, in PolyU Finger Knuckle Images Database(Version 3) [32], using our trained network with different loss functions (a) original input finger knuckle images, (b) results using our loss
function (RSIL), (c) results using SSTL [37] and (d) triplet loss function [38].

threshold as 0.2 to test the degradation in the detection performance but during the segmentation task,
we use a very low threshold to ensure the detection of each of finger knuckle ROI. It can be observed
from the results in the following table that when the finger knuckle is bent by more than 54◦ , there is asignificant degradation in recall and precision.

TABLE III
VARIATIONS IN THE MAJOR FINGER KNUCKLE DETECTION PERFORMANCE WITH THE DEGREE OF FINGER BENDING

Model
SampleFirst(0°)
Second(18°)
Third(36°)
Fourth(54°)
Fifth(72°)
Sixth(90°)

Our(x)
Precision Recall
0.936
0.923
0.943
0.900
0.913
0.896
0.929
0.828
0.890
0.787
0.881
0.756

Our(m)
Precision Recall
0.887
0.887
0.907
0.900
0.904
0.855
0.897
0.828
0.906
0.697
0.879
0.602

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

1

18

Receiver Operating Characteristics

Cummulative Match Characteristics
0.85

0.9

0.8
0.75

Average Recognition Rate

Genuine Acceptance Rate

0.8
0.7
0.6

0 Deg; EER: 16.34%
18 Deg; EER: 16.29%
36 Deg; EER: 19.21%
54 Deg; EER: 21.13%
72 Deg; EER: 22.11%
90 Deg; EER: 23.88%

0.5
0.4
0.3
0.2
0.1
10 -4

0.7
0.65

0 Deg
18 Deg36 Deg54 Deg72 Deg90 Deg

0.6
0.55
0.5
0.45

10 -3

10 -2

10 -1

10 0

1

2

3

4

5

6

False Acceptance Rate

Commulative Rank

(a)

(b)

7

8

9

10

Fig. 12. Influence from finger knuckle flexion deformations for the matching performance using deformable dataset [32]: (a) ROC plots and(b) CMC plots.

Secondly, we also test the performance from our finger knuckle recognition model on the same dataset
using such samples from each of the subjects to ascertain the changes in the match performance with thechange in the finger knuckle flexion. Such result are presented in the following Fig. 12 using the ROC
and CMC plots under two-session match protocols (104 genuine scores and 10712(104 × 103) imposter
scores). These results indicate that the performance from 0◦ -36◦ is quite similar while there is remarkabledegradation in the performance when the extent of finger bend is larger than 54◦ .
F. Completely Contactless Finger Knuckle Detection Performance
We also developed a completely contactless online finger knuckle detector that can robustly detectknuckle ROI from images with complex backgrounds and in a wild. Most publicly available finger knuckle
databases are acquired under a fixed background and are not sufficient to train a detector that is expected
to detect knuckle ROI in the wild. Therefore, we also used a web crawler to acquire finger knuckle
images from public websites [49] that provide free access to download or use [50] such images. We
have downloaded 2347 images, there are 738 images without knuckles, and these images can be usedas background training, and the rest of the 1609 images that contain at least one finger knuckle are thepositive samples for the network model. In addition to 2347 captured images, we also include 233 imagesof finger joints (obtained by us) for composing our dataset, for a total of 2580 (2347 + 233) images.
Among them, we randomly selected 2450 of them as our training set, and the remaining 100 as a testing
set. The pretrained YOLOv5 model is then finetuned using these training images. We also incorporated
data augmentation during the training using the popular methods i.e. flip, rotation, resize, translation andmosaicking, for the training samples. We empirically fixed r, in (11), as 6 for all the experiments in thepaper.
We also performed experiments to ascertain the effectiveness of a specialized finger knuckle detector
developed during this work. The merit of such specialized detector was evaluated using the (i) detection
performance and (ii) average improvement in matching accuracy over key baseline methods in respective
references. In order to evaluate detection performance, we employed YOLOv5x and YOLOv5m models,to predict horizontal bounding boxes, as baseline methods. Although we only used major finger knuckle
during the identification in this work (simply to reduce the complexity), the detection performance wasalso evaluated for the detector of minor finger knuckle. Among these 2580 images, 100 images wererandomly selected as the test set for this part of the evaluation and these results are summarized in Table

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

1

19

Culmulative Match Characteristics

Receiver Operating Characteristics

1
0.98

0.9

0.96

Recognition Rate

Genuine Acceptance Rate

0.95

0.85
0.8
0.75
0.7

Detector [18]; EER: 5.31%
Our Detector; EER: 2.21%

0.65
0.6

10 -4

10 -2

0.94
0.92
0.9

Detector [18]
Our Detector

0.88

10 0

0.86
2

4

False Acceptance Rate
(a)

1

8

10

(b)

Culmulative Match Characteristics

Receiver Operating Characteristics

1

0.9

0.98

0.8

0.96

0.7

Recognition Rate

Genuine Acceptance Rate

6

Culmulative Rank

0.6
0.5
0.4
0.3

Detector [35]; EER: 6.51%
Our Detector; EER: 1.33%

0.2
0.1

10 -4

10 -2

False Acceptance Rate
(c)

0.94
0.92
0.9

Detector [35]
Our Detector

0.88
10 0

0.86
2

4

6

8

10

Culmulative Rank(d)Fig. 13. Comparative match performance from our knuckle detector and using the detection method in respective references: Respective
results using PolyU Contactless Finger Knuckle Images Database (Version 3.0) [32] in (a)-(b) and using PolyU Contactless Hand Dorsal
Image Database [31] in (c)-(d).

Fig. 14. Sample results from our finger knuckle detector using publicly available [49] images, each of which illustrate multiple hands andcomplex background. The number inside the box depicts estimated orientation of respective major or major knuckle.

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

20

IV. In this table, ours(x) represent the specialized knuckle detector using YOLOv5x with CSL and theangular loss that is detailed in section III. Similarly, the ours(m) represents a lightweight version of ourdetector using YOLOv5m. These results indicate that the proposed knuckle detector can achieve highestmAP value of 89.9, especially, the AP of minor finger knuckle can reach to 90.1. On the contrary, without
predicting an oriented bounding box, the YOLOv5x detection accuracy is lower by 1.6 mAP as comparedto the performance from our(x). A video file attached with this paper provides online performance fromthe specialized finger knuckle detector, and none of these fingers or samples were used for training andillustrates its effectiveness for real-world deployments.
TABLE IVC OMPARISON OF THE ACCURACY OF THE YOLOV 5 WITH CSL AND WITHOUT CSL FOR DETECTING FINGER KNUCKLE .

Model
Our(x)
Our(m)
YOLOv5x
YOLOv5m

Total
Time/ms(1024x1024)
40.9ms23.3ms33.3 ms
12.1msNumberof LayersmAP val
0.5

407
263
407
263

89.9
85.7
87.3
84.8

AP of
Major
FK
89.6
88.9
86.5
84.5

AP of
Minor
FK
90.1
80.4
88.0
85.1

The calculated values of mAP were measured at a detection threshold of 0.001 as
well as an IOU threshold of 0.5. All the experiments are test on the GTX 2080 GPU
and I7-7800X CPU, and the total time includes the inference and NMS process.

Another attempt to evaluate the effectiveness of the developed detector (and segmentation) is presented
from the experimental results in Fig. 13. These plots illustrate the relative performance, on respective public
databases when the finger knuckle ROI images from our detector is employed. The baseline performance
represents the detected ROI images in the respective references [35], [18], using the same or our matching
method. It can be observed from these results that the usage of the proposed knuckle detector can alsosignificantly enhance the matching performance.
G. Online Contactless Finger Knuckle Identification PerformanceFig. 15. 14. Sample images from five different subjects in our video finger knuckle dataset for the online performance evaluation in thiswork.

We also attempted to evaluate the feasibility of the developed system for real-world applications.
Therefore, we acquired a database from 52 subjects using a general-purpose smartphone. Each subject
provided 15-20s video of contactless finger dorsal images with significant rotations, distance/scale changes,
and under complex backgrounds (such as traffic flow, pedestrians), which can be reflective of deployment

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

21

expectations. Performance evaluation was done in a cross-database scenario and the model trained in
section IV-A was employed for the performance evaluation. The minimum frame rate of these videos is
30 frames per second. We considered two scenarios for such performance evaluation: (a) under 1 image
per second (Fig. 15) and (b) 6 images per second. Since the shortest video is 15s, for 1 image per second,
we can get a total of 780 (52 × 15) finger knuckle images. Therefore by ensuring the same number ofsamples from each subject, 1789 (52 × 15) genuine and 39780 (52 × 51 × 15) imposter match scores werecomputed. Similarly, for the 6 images per second case, we can get 4680 (52 × 15 × 6) finger knuckle
samples, which resulted in 4680 (52 × 90) genuine and 238680 (52 × 51 × 90) imposter match scores.
The performance plots under both the considered scenarios are showed in Fig. 16. These performanceplots achieve very high accuracy and indicate the suitability of the developed method and the system fora range of real-world applications.

1

Culmulative Match Characteristics

Receiver Operating Characteristics

0.999
0.998
0.997

0.98

Recognition Rate

Genuine Acceptance Rate

0.99

0.97
0.96

1FPS; EER: 1.67%
6FPS; EER: 0.30%

0.95
0.94

10 -4

10 -2

False Acceptance Rate
(a)

0.996
0.995
0.994

1FPS
6FPS

0.993

10 0

0.992
2

4

6

8

10

Culmulative Rank(b)Fig. 16. Comparative performance from our completely contactless and online finger knuckle identification system using (a) ROC and CMC(b).

V. C ONCLUSION AND F URTHER W ORKThis paper has proposed and developed a new framework for completely contactless finger knuckle
identification, to meet the expectations for the real-world deployments. In order to accommodate significant
intra-class variations for the completely contactless system, we proposed a new loss function (RSIL). Our
experimental results presented in section IV using the proposed framework, both for the within- and crossdatabase experiments, indicate consistently shown outperforming results and validate the effectiveness ofthe framework. We also developed a robust finger knuckle detector by incorporating CSL and angularloss with a Yolov5 backbone and experimental results in section IV-F also demonstrated its effectivenessfor the real-world applications. Completely contactless and online knuckle identification system was also
evaluated in section IV-G and is first such system in the best of our knowledge.
Despite the impressive results and ready availability of the developed system for the real-world deployments, there are many challenges that should be addressed in the further extension of this work.
The first challenge is relating to very limited availability of database. For real-world applications, thecurrent amount of data from the finger knuckle images is quite small to allow the model to be adequately
trained i.e. unlike those for the face or fingerprint recognition which have much larger dataset available
for the evaluation or training. Another challenge is relating to completely contactless system is associatedwith the involuntarily deformations in the knuckle patterns during the access. Despite our assumptionthat the client has interest in access or the transaction, and therefore user will present his/her registered

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

22

fingers in the imaging volume with expected pose, some deformations in knuckle patterns are inevitable.
Therefore, more effective algorithms should be designed to address such (unwanted) knuckle deformations
for real-world applications. Another limitation with our system is the use of two-stage approach, wherea network is trained to detect the position of finger knuckle in order to segment while other network isused to extract the finger knuckle features for matching. Development of an-end-to-end approach, e.g.
multitasking, that can also incorporate features extracted for the knuckle detection, should be considered
to further enhance such system. In the further extension of this work, some computationally intensive
tasks can also be performed on the server side, primarily to reduce the amount of computation on mobile
devices, ensuring a good online performance from mobile devices. We can also use pruning techniques
and knowledge distillation for the deep neural networks to further reduce the time and space complexity,
to enable its usage on a range of mobile devices.
R EFERENCES[1] M. Tistarelli, S.Z. Li, and R. Chellappa. Handbook of Remote Biometrics: for Surveillance and Security. Advances in Computer Vision
and Pattern Recognition. Springer London, 2012.
[2] Vivek Kanhangad, Ajay Kumar, and David Zhang. A unified framework for contactless hand verification. IEEE Transactions on
Information Forensics and Security, 6(3):1014–1027, 2011.
[3] Lin Zhang, Lei Zhang, David Zhang, and Hailong Zhu. Online finger-knuckle-print verification for personal authentication. Pattern
Recognition, 43(7):2560–2571, 2010.
[4] Gaurav Jaswal, Amit Kaul, and Ravinder Nath. Knuckle print biometrics and fusion schemes–overview, challenges, and solutions.
ACM Computing Surveys (CSUR), 49(2):1–46, 2016.
[5] MorphoWave Contactless Fingerprint. https://www.idemia.com/contactless-fingerprint, Accessed July 2022.
[6] Shoichiro Aoyama, Koichi Ito, and Takafumi Aoki. A multi-finger knuckle recognition system for door handle. In 2013 IEEE Sixth
International Conference on Biometrics: Theory, Applications and Systems (BTAS), pages 1–7. IEEE, 2013.
[7] Kenneth Okereafor, Iniobong Ekong, Ini Okon Markson, Kingsley Enwere, et al. Fingerprint biometric system hygiene and the risk of
covid-19 transmission. JMIR Biomedical Engineering, 5(1):e19623, 2020.
[8] Kevin HM Cheng and Ajay Kumar. Deep feature collaboration for challenging 3d finger knuckle identification. IEEE Transactions on
Information Forensics and Security, 16:1158–1173, 2020.
[9] Ajay Kumar. Importance of being unique from finger dorsal patterns: Exploring minor finger knuckle patterns in verifying humanidentities. IEEE Transactions on Information Forensics and Security, 9(8):1288–1298, 2014.
[10] Kevin HM Cheng and Ajay Kumar. Contactless biometric identification using 3d finger knuckle patterns. IEEE Transactions on PatternAnalysis and Machine Intelligence, 42(8):1868–1883, 2019.
[11] Wenming Yang, Zhiquan Chen, Chuan Qin, and Qingmin Liao. α-trimmed weber representation and cross section asymmetrical coding
for human identification using finger images. IEEE Transactions on Information Forensics and Security, 14(1):90–101, 2018.
[12] Qian Zheng, Ajay Kumar, and Gang Pan. A 3d feature descriptor recovered from a single 2d palmprint image. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 38(6):1272–1279, 2016.
[13] David Zhang, Xiaoyuan Jing, and Jian Yang. Biometric image discrimination technologies. IGI Global, 2006.
[14] KamYuen Cheng and Ajay Kumar. Contactless finger knuckle identification using smartphones. In 2012 BIOSIG-Proceedings of theInternational Conference of Biometrics Special Interest Group (BIOSIG), pages 1–6. IEEE, 2012.
[15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
[16] Rajiv Kapoor, Dinesh Kumar, Anmol Garg, Anuj Sharma, et al. Completely contactless finger-knuckle recognition using gabor initialized
siamese network. In 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), pages 867–872.
IEEE, 2020.
[17] Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. In International Conference on Machine Learning,
pages 10096–10106. PMLR, 2021.
[18] Ajay Kumar. Toward pose invariant and completely contactless finger knuckle recognition. IEEE Transactions on Biometrics, Behavior,and Identity Science, 1(3):201–209, 2019.
[19] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic
segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 580–587, 2014.
[20] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, pages 1440–1448, 2015.
[21] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal
networks. Advances in Neural Information Processing Systems, 28:91–99, 2015.
[22] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 779–788, 2016.
[23] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pages 7263–7271, 2017.
[24] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018.
[25] Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv
preprint arXiv:2004.10934, 2020.

JOURNAL OF LATEX CLASS FILES, VOL. X, NO. X, JULY 2022

23[26] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single
shot multibox detector. In European Conference on Computer Vision, pages 21–37. Springer, 2016.
[27] Ying Xin, Guanzhong Wang, Mingyuan Mao, Yuan Feng, Qingqing Dang, Yanjun Ma, Errui Ding, and Shumin Han. Pafnet: An
efficient anchor-free object detector guidance. arXiv preprint arXiv:2104.13534, 2021.
[28] Ultralytics. Yolov5. https://github.com/ultralytics/yolov5, 18 May 2020.
[29] Daksh Thapar, Gaurav Jaswal, and Aditya Nigam. Fkimnet: a finger dorsal image matching network comparing component (major,
minor and nail) matching with holistic (finger dorsal) matching. In 2019 International Joint Conference on Neural Networks (IJCNN),
pages 1–8. IEEE, 2019.
[30] The HKPolyU Contactless Finger Knuckle Images Database (V-1.0):. http://www4.comp.polyu.edu.hk/∼csajaykr/fn1.htm.
[31] The HKPolyU Contactless Hand Dorsal Images Database:. http://www4.comp.polyu.edu.hk/∼csajaykr/knuckleV2.htm.
[32] The HKPolyU Contactless Finger Knuckle Images Database (Version 3.0) :. https://www4.comp.polyu.edu.hk/∼csajaykr/fn2.htm.
[33] The HKPolyU 3D Finger Knuckle Images Database:. https://www4.comp.polyu.edu.hk/∼csajaykr/3DKnuckle.htm.
[34] Tsinghua University Finger Vein and Finger Dorsal Texture Database (THU-FVFDT3):. https://www.sigs.tsinghua.edu.cn/labs/vipl/
thu-fvfdt.html.
[35] Ajay Kumar and Zhihuan Xu. Personal identification using minor knuckle patterns from palm dorsal surface. IEEE Transactions on
Information Forensics and Security, 11(10):2338–2348, 2016.
[36] Xue Yang and Junchi Yan. Arbitrary-oriented object detection with circular smooth label. In European Conference on Computer Vision,
pages 677–694. Springer, 2020.
[37] Yang Liu and Ajay Kumar. Contactless palmprint identification using deeply learned residual features. IEEE Transactions on Biometrics,
Behavior, and Identity Science, 2(2):172–181, 2020.
[38] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 815–823, 2015.
[39] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE International Conference
on Computer Vision, pages 2961–2969, 2017.
[40] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint
arXiv:1409.1556, 2014.
[41] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advancesin Neural Information Processing Systems, 25, 2012.
[42] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition, pages 4700–4708, 2017.
[43] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and
Pattern Recognition, pages 7132–7141, 2018.
[44] Fisher Yu and Vladlen Koltun. Multiscale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015.
[45] Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proceedings of theIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9308–9316, 2019.
[46] Suyog Gupta and Mingxing Tan. Efficientnet-edgetpu: Creating accelerator-optimized neural networks with automl. https://ai.googleblog.
com/2019/08/efficientnet-edgetpu-creating.html, 2019.
[47] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visualexplanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer
Vision, pages 618–626, 2017.
[48] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla,
Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252,
2015.
[49] Unsplash. Unsplash. https://unsplash.com/.
[50] Unsplash. Unsplash.com. ”Unsplash License”, Accessed 11 January 2022.