Capturing Micro Deformations from Pooling Layers for
Offline Signature Verification
Yuchen Zheng† , Wataru Ohyama‡ , Brian Kenji Iwana† and Seiichi Uchida† ,† Department of Advanced Information Technology, Kyushu University, Fukuoka, Japan
Email: {yuchen, brian}@human.ait.kyushu-u.ac.jp, uchida@ait.kyushu-u.ac.jp‡ Saitama Institute of Technology, 1690 Fusaiji, Fukaya-shi, Saitama, Japan
Email: ohyama@sit.ac.jp

Abstract—In this paper, we propose a novel Convolutional
Neural Network (CNN) based method that extracts the location
information (displacement features) of the maximums in the
max-pooling operation and fuses it with the pooling features tocapture the micro deformations between the genuine signaturesand skilled forgeries as a feature extraction procedure. Afterthe feature extraction procedure, we apply support vector
machines (SVMs) as writer-dependent classifiers for eachuser to build the signature verification system. The extensive
experimental results on GPDS-150, GPDS-300, GPDS-1000,
GPDS-2000, and GPDS-5000 datasets demonstrate that the
proposed method can discriminate the genuine signatures andtheir corresponding skilled forgeries well and achieve state-ofthe-art results on these datasets.
Keywords-Offline Signature Verification; Micro Deformations; Displacement Features; Max-pooling; Feature Extraction

I. I NTRODUCTION
Signature verification is a very challenging task thataims to verify the genuine signatures and the corresponding
skilled forgeries for a specific user [11]–[13]. Generally,signature verification systems are divided into two categories: online and offline. For online systems, the data iscollected as a sequence which includes the positions of thepen, pressure coordinate sequence, pen elevation coordinate
sequence, etc [8]. For offline systems, the data is collectedfrom digital images. Since the dynamic information is notavailable in offline signature verification systems, the task
becomes very challenging. In addition, the forgeries canbe deliberately imitated by practiced persons, which also
increases the difficulty for the verification systems.
To build a robust offline signature verification system,
extracting the discriminative features between the genuine
signatures and skilled forgeries from original images plays acrucial role. Traditional verification systems apply handcrafted feature extractors to extract the feature representations
from the original images, such as geometrical features [2],Local Binary Pattern (LBP) features [4], Scale Invariant
Feature Transform (SIFT) features [15], etc. However, designing a good handcrafted feature often need to set different
hyperparameters which are just suitable for the specifictasks. Due to the limitation of the handcrafted features, many
deep learning based feature extractors are proposed in recent

years [5], [7], [16], [22]. However, capturing the micro
deformations or distortions between the genuine signaturesand skilled forgeries is still difficult for the existing methods.
In this paper, we propose a novel CNN based method thatextracts the location information (displacement features) of
maximums in max-pooling operations [24], [25] and fuses itwith the pooling features to capture the micro deformations
or distortions between the genuine signatures and their corresponding skilled forgeries as a feature extraction procedure.
The motivation of this paper is to prevent the max-pooling
operation from removing the key micro deformations used
to discriminate genuine signatures and corresponding skilledforgeries. In the feature extraction process, we train a CNN
between the genuine signatures and skilled forgeries ona large scale dataset, named GPDS-10000 [3] to capture
the different behaviors of genuine signatures and skilledforgeries. After CNN training process, we take the trained
CNN as a feature extractor to obtain the discriminative
features from original signature images. Then, we apply
linear SVMs as the writer-dependent classifiers for each user
to build the signature verification system and evaluate thelearned features.
The contributions of this paper are summarized as follows.
We extract the displacement features and fuse it withpooling features to capture micro deformations between
genuine signatures and skilled forgeries for offline
signature verification systems.
• We train a proposed CNN based model on a large scale
dataset.
• We obtain the state-of-the-art results on GPDS-150,
GPDS-300, GPDS-1000, GPDS-2000 and GPDS-5000
datasets.
•

The rest of the paper is organized as follows: Section II
discusses some handcrafted features and deep learning based
features related to our own. We introduce the proposed
method to capture the micro deformations between the
genuine signatures and skilled forgeries in Section III.
Finally, we present the experimental results and discussion
in Section IV, while Section V concludes this paper withremarks and future work.

Original Image

Convolutional
Feature

0

0

0

7

1

8

0

2

4

5

9

4

8

0

0

0

8

7

8

9
Pooling Feature(-1,1)(1,1)(-1,-1)(1,-1)

Displacement
FeatureFigure 1: Extracting the pooling features and the displacement features simultaneously. Here, the pooling size is 2 × 2 withstride 2. The displacement features capture micro-deformation within the pooling window. The displacement vector (-1,1)
means that the vertical displacement from the center to the maximum value is -1 and the horizontal displacement is 1.
II. R ELATED W ORK
A. Handcrafted Features for Offline Signature Verification
Traditional offline signature verification systems often use
different handcrafted features to train the writer-dependent
or writer-independent classifiers, such as geometrical features [2], LBP features [4], SIFT features [15], and Histogram of Oriented Gradients (HOG) features [20]. Morerecently, there are many researches that focus on designingrobust features [9], [14], [26] to build signature verification systems. In [14], Okawa proposed a feature extraction
method based on a Fisher vector (FV) with fused “KAZE”
features from both foreground and background signature images. The “KAZE” features consider the structures between
strokes and stroke contour information more effectively.
In [26], Zois et al. proposed the post-oriented grid features
which encode the geometric structure of the signatures bygrid templates. However, using the handcrafted features are
hard to discriminate the genuine signatures and the corresponding skilled forgeries and often need to set different
parameters for specific tasks, which is hard to apply to otherverification systems and large scale applications.
B. Deep Learning Based Features for Offline Signature
Verification
In recent years, the deep learning based models havewidely applied in many fields, such as image classificationand detection [18], [21], [27], text recognition and detection [17], [19], [23], online and offline signature verification [1], [6], [7], [16]. In the field of offline signature
verification, some deep learning based features are proposed
to capture the behaviors of different signatures [5], [7],[10], [22]. In [22], Zhang et al. proposed an unsupervised
feature for offline signature verification based on Deep
Convolutional Generative Adversarial Networks (DCGANs),
which has a robust generalization ability compared to handcrafted features. In [7] and [5], Hafemann et al. proposed

a CNN based feature extraction approach, named “Signet”
to obtain the discriminative features not only between the
genuine signatures and skilled forgeries but also between
the different users. However, the “Signet” cannot capture
the micro deformations between the genuine signatures andcorresponding skilled forgeries and only trained on 531
different users cannot apply to large scale verification tasks.
Compared to the previous methods, the proposed method
can train with a huge number of users to capture the micro
deformations or distortions between the genuine signaturesand skilled forgeries, which is very useful for signature
verification systems.
III. C APTURING M ICRO D EFORMATIONS BY
D ISPLACEMENT F EATURESIn this section, we introduce the proposed method to fuse
the displacement features and pooling features as discriminative features to capture the micro deformations between
the genuine signatures and skilled forgeries for signature
verification systems. First, we introduce a normal CNN
based architecture trained between the genuine signaturesand forgeries. Then, based on the pre-trained CNN, we
introduce how to extract the displacement features and fuse itwith the pooling features in a combined architecture. Finally,
we introduce how to train the writer-dependent classifiers
based on the fused features.
A. Training a Normal CNN Between the Genuine Signaturesand Forgeries
To distinguish genuine signatures and skilled forgeries,
we design a CNN architecture with 3 convolutional andpooling layers, 2 fully-connected layers and a softmax layer.
In the convolutional layers, the kernel size is 3 × 3 withstride 1, and the number of the filters is 32, 64 and 128,
respectively. The pooling size is 2 × 2 with stride 2. In
the fully-connected layers, the first fully-connected layer

Figure 2: Visualization of the pooling features and the displacement features of some samples on GPDS dataset. The firstrow represents the original image and corresponding pooling features, the second row represents the displacement features.
Each column represents one convolutional filter. The visualization of displacement features is based on an HSV color model
whose color and intensity denote the direction and average length of the displacement features.
Inputs1@128x128

Feature maps
32@128x128

Feature maps
32@64x64

Feature maps
64@64x64.

.

.

Feature maps Feature maps Feature maps Hidden units
128@32x32
128@16x16 2048
64@32x32
.

.

.

.

.

..
..
Max-pooling

Max-pooling Convolution

Convolution

Color model

Max-pooling

Convolution

Feature maps
32@64x64.
.

Feature maps
64@64x64
.

.

.

.

..

..

Fully
connected

Feature maps Feature maps Feature maps Hidden units
128@32x32
128@16x16 2048
64@32x32.
.

Hidden units
1024

..

Fully
connected
Hidden units
1024

Hidden units
1024

..

Outputs
2

Pre Training

..

Fully
connected

Softmax..
..
Max-pooling

Max-pooling Convolution

..

Fully
connected..

..

Verification

FeatureExtraction

Fully
connected

Convolution

SVM

Figure 3: The procedure of feature extraction and verification.
has 2048 nodes and reduces to 1024 in the second fullyconnected layer. Rectified Linear Unit (ReLU) is used as
the activation function for the network. The final softmax
layer contains 2 nodes, which is designed to judge whether
the input is a genuine signature or forgery. The cross-entropy
is used as the loss function to train the network.
B. Extracting and Fusing the Displacement Features with
Pooling Features
To further capture the micro deformations between the
genuine signatures and forgeries, we extract displacement
features [24], [25] from the first pooling layer in previous
pre-trained CNN and fuse it with the pooling features. Fig. 1
shows the procedure of the feature extraction. Here, thepooling size 2×2 with stride 2, the value of the displacement

features both in horizontal and vertical directions belong to[−1, 1]. Fig. 2 presents the pooling features and displacement features of samples from the GPDS-10000 dataset
based on a Hue-Saturation-Value (HSV) color model whosecolor and intensity denote the direction and average lengthof the displacement features. The displacement features describe the location information of maximums in max-pooling
operation, which might capture some micro deformations of2
forgery signatures when some skilled writers imitated thegenuine signatures.
The architecture that we used for fusing the poolingfeatures and displacement features is shown in Fig. 3. We
can see that the architecture for processing the displacement
features is the same as the pre-trained CNN without thefirst convolutional and pooling layers. Here, we divide thedisplacement into the horizontal and vertical directions andapply the same architecture. We then fuse the poolingfeatures and displacement features in the last fully-connected
layer.
C. Training the Writer-dependent Classifiers
After the CNN training procedure, we extract the fusedfeatures of each user from the CNN based feature extractor
and train the writer-dependent classifiers. For each user (notincluded in CNN training procedure), we use the genuine
signatures as the positive samples and genuine signaturesfrom other users as the negative samples to build the training
set. Then, we choose the linear SVM as the writer-dependentclassifier to build the verification system. This procedure isshown in Fig. 3.
To overcome the imbalanced problem that the negative
samples are much more than the positive samples, we use
different weights for the positive and negative class [7]. The
SVM objective function becomes,
min

M

N

yi =+1yi =−1XX
1 T
w w + Cp
ξi + C n
ξi
2
i=1
i=1

(1)

s.t. yi (wT xi + b) ≥ 1 − ξi and ξi ≥ 0
where xi is a training sample with target value yi , ξi is theslack variables, M and N are the numbers of the positive
and negative samples, Cp and Cn are the weights for the
positive and negative class,
N
Cn
(2)
M
For the testing procedure, we used the remaining genuine
signatures of the target user and the skilled forgeries to build
the test set to evaluate the performance.
Cp =IV. E XPERIMENTIn this section, we first introduce the experimental protocol of training CNN and the writer-dependent classifiers.
Then, we present the experimental results in detail.
A. Experimental Protocol
We conducted the experiments on GPDS-10000 dataset 1
to evaluate the proposed method. The GPDS-10000 is alarge scale dataset that contains 24 genuine signatures and
30 skilled forgeries for each user. The number of users
is 10,000, so the GPDS-10000 dataset contains 240,000
genuine signatures and 300,000 skilled forgeries and it
is very suitable for deep learning based methods. In our
experiment, we divided the GPDS-10000 dataset into twoparts, the first 5,000 users, and the final 5,000 users.
For the procedure of training CNN, we used the signaturesfrom the final 5,000 users. To normalize the input images,
1 http://www.gpds.ulpgc.es/

we first resize all images to 128 × 128. Then, we used 90%
data for training and 10% data for validation to train the
network. We use Adam as an optimizer to minimize the loss
function with mini-batch size 32. The model is trained with
40 epochs. The initial learning rate is set to 1e-4 and reduced
by a factor of 0.95 after each epoch. After the CNN training
process, we take the trained model as a feature extractor to
extract the fused features.
To build the signature verification systems, we trained
the linear SVMs as the writer-dependent classifiers for eachuser. Compared to other state-of-the-art methods, we used
5 sub-datasets, GPDS-150, GPDS-300, GPDS-1000, GPDS2000, GPDS-5000 (the first 100, 150, 1,000, 2,000, 5,000
users of GPDS-10000 dataset) for final evaluation. For a
specific user, we randomly selected 5 genuine signatures as
the positive samples and 5 genuine signatures from each of
the final 5,000 users as the negative samples as the training
set. For the training process, the weights Cn are foundby grid search with 5-fold cross-validation, and the Cp is
calculated by Equation 2.
For the evaluation of the test set, the remaining genuine
signatures from the target user are used for calculating theFalse Rejection Rate (FRR). The False Acceptance Ratefor the skilled forgeries (FARskilled ) experiment has beenobtained with forgery samples of the target user. The False
Acceptance Rate for the random impostor (FARrandom )experiment has been obtained with the genuine signaturesfrom all the remaining users. The Equal Error Rate forskilled forgeries experiment (EERskilled ) is calculated by
FARskilled = FRR, and the EER for the random impostor
experiment (EERrandom ) is calculated by FARrandom =FRR.
B. Experimental Results and Discussion
To evaluate the performance of the proposed method,
we test the proposed method on GPDS150, GPDS300,
GPDS1000, GPDS2000, and GPDS5000 datasets and compared it with the state-of-the-art models [2], [4] and the
traditional CNN based features with SVMs. In [2], theauthors applied a Hidden Markov Model (HMM) on thegeometrical features (GF) for verification systems. In [4],the authors extracted the LBP features to train the SVMs
as the writer-dependent classifiers. For the traditional CNN
model, it just extracts the features from the last fullyconnected layer. Then, using the SVMs for each user tobuild the verification system. The experimental results are
the averages of all users with 10 trials.
Table I shows the results on skilled forgeries. This experiment is to verify whether the query samples are genuine
signatures or skilled forgeries. We can see that only using
the SVMs with features extracted from a normal CNN canachieve the desired results. The proposed method obtained
the best results than other state-of-the-art models on all
datasets. When the number of the randomly selected sample

Table I: The skilled forgeries experiment (EERskilled in %). The 5 samples and 10 samples represent the randomly selecting
5 or 10 samples of each user for training the SVMs.
DatasetGPDS-150
GPDS-300
GPDS-1000
GPDS-2000
GPDS-5000

HMM+GF [2](5 samples)
11.48
12.11
11.07
11.34
11.10

SVM+LBP [4](5 samples)
16.45
16.50
17.01
16.63
16.93

Traditional CNN(5 samples)
8.34±0.52
8.41±0.61
8.31±0.47
8.20±0.42
8.08±0.53

Proposed(5 samples)
7.45±0.41
7.48±0.52
7.12±0.45
7.23±0.53
7.15±0.48

Proposed(10 samples)
6.32±0.35
6.25±0.41
6.43±0.38
5.92±0.41
6.11±0.47

Table II: The random impostor experiment (EERrandom in %), where micro-deformations is not important.
DatasetGPDS-150
GPDS-300
GPDS-1000
GPDS-2000
GPDS-5000

HMM+GF [2](5 samples)
4.17
4.32
4.37
4.44
4.53

SVM+LBP [4](5 samples)
1.31
1.45
1.63
1.73
1.63

GenuineSignatures

Traditional CNN(5 samples)
4.89±0.48
4.77±0.52
4.82±0.58
4.94±0.42
4.56±0.43

Proposed(5 samples)
4.64±0.38
4.72±0.54
4.91±0.43
4.95±0.55
4.58±0.41

Proposed(10 samples)
3.08±0.33
3.23±0.42
3.17±0.35
3.22±0.41
2.84±0.43

SkilledForgeries

Figure 4: An example improvement of the displacement features. Here, all signatures are from the same user. The firstcolumn is the original signature images, the second and the third columns are the displacement features extracted from two
filters.
increased to 10, the EERs are smaller than before, whichmeans that using more samples to train the SVMs canimprove the performance of the verification systems. In
addition, as the datasets become larger (the number of users
is increasing), the EER becomes higher when using the
traditional models. It means that the traditional models are
not general for all the users. The proposed method is morestable than the traditional models when the dataset size
becomes huger.
Table II shows the results on random impostors. This
experiment is mainly to discriminate the different users.
Since the purpose of the feature extractor is to capture
different behaviors between the genuine signatures and forgeries, the proposed method did not obtain the best resultson this experiment. However, we achieved a competitive8

performance compared to the state-of-the-art results. Even
if the model in [4] can classify the different users well,
the ability to distinguish the genuine signatures and skilledforgeries of this model is far worse than the proposed
method.
Fig. 4 presents an improved example by using the proposed method. We visualized the displacement features toobserve the micro different behaviors between the genuine
signatures and skilled forgeries. If we use traditional CNN
based features, they all belong to the same class (positiveclass). But using the fused features by the proposed method,they can be classified correctly. From Fig. 4, we can seethat the genuine signatures have similar behaviors on thedisplacement features and the skilled forgeries are different
from the genuine samples in some places. For example, inthe bottom left corner of the first filter, the genuine samples
have some features in yellow and purple directions, but it
is rare in the skilled forgery samples. And in the second
filter, the displacement features can capture some blue andpurple directions in the genuine signatures, but in the skilledforgeries, the corresponding position is green and red. Itmeans that the proposed method can capture some micro
deformations or distortions between the genuine signaturesand skilled forgeries.
V. C ONCLUSION
In this paper, we proposed a novel approach that fuses
the proposed displacement features and pooling features asa new feature to capture the micro deformations between
the genuine signatures and skilled forgeries. Then, we applied linear SVMs as the writer-dependent classifiers tobuild a verification system. Extensive experimental results
demonstrate that the fused features can capture the micro
deformations or distortions between the genuine signaturesand skilled forgeries well, which is helpful for the offline
signature verification systems. For future work, we plan to
apply the proposed method to build an end-to-end offline
signature verification system and introduce the user information to further improve the performance.
ACKNOWLEDGMENT
This work was supported by JSPS KAKENHI (Grant
Number JP18K11373 and JP17H06100) and China Scholarship Council (Grant Number 201706330078).
R EFERENCES[1] K. Ahrabian and B. Babaali, “Usage of Autoencoders andSiamese Networks for Online Handwritten Signature Verification,” Neural Computing and Applications, 2017.
[2] M. A. Ferrer, J. B. Alonso, and C. M. Travieso, “Offline
Geometric Parameters for Automatic Signature Verification
Using Fixed-point Arithmetic,” TPAMI, 2005.
[3] M. A. Ferrer, M. Diaz, C. Carmona-Duarte, and A. Morales,“A Behavioral Handwriting Model for Static and Dynamic
Signature Synthesis,” TPAMI, 2017.
[4] M. A. Ferrer, J. F. Vargas, A. Morales, and A. Ordonez,“Robustness of Offline Signature Verification Based on Gray
Level Features,” TIFS, 2012.
[5] L. G. Hafemann, L. S. Oliveira, and R. Sabourin, “Fixed-sized
Representation Learning from Offline Handwritten Signaturesof Different Sizes,” IJDAR, 2018.
[6] L. G. Hafemann, R. Sabourin, and L. S. Oliveira, “Analyzing
Features Learned for Offline Signature Verification Using
Deep CNNs,” in ICPR, 2016.
[7] L. G. Hafemann, R. Sabourin, and L. S. Oliveira, “Learning
Features for Offline Handwritten Signature Verification Using
Deep Convolutional Neural Networks,” Pattern Recognition,
2017.
[8] L. G. Hafemann, R. Sabourin, and L. S. Oliveira, “Offline
Handwritten Signature Verification-Literature Review,” in IPTA, 2017.

[9] J. Hu and Y. Chen, “Offline Signature Verification Using
Real Adaboost Classifier Combination of Pseudo-dynamic
Features,” in ICDAR, 2013.
[10] S. Lai and L. Jin, “Learning Discriminative Feature Hierarchies for Off-Line Signature Verification,” in ICFHR, 2018.
[11] M. Liwicki, M. I. Malik, C. E. Van Den Heuvel, X. Chen,
C. Berger, R. Stoel, M. Blumenstein, and B. Found, “Signature Verification Competition for Online and Offline Skilled
Forgeries (Sigcomp2011),” in ICDAR, 2011.
[12] M. I. Malik, S. Ahmed, A. Marcelli, U. Pal, M. Blumenstein,
L. Alewijns, and M. Liwicki, “ICDAR2015 Competition on
Signature Verification and Writer Identification for On-and
Off-line Skilled Forgeries (SigWIcomp2015),” in ICDAR,
2015.
[13] M. I. Malik, M. Liwicki, L. Alewijnse, W. Ohyama, M. Blumenstein, and B. Found, “ICDAR 2013 Competitions on
Signature Verification and Writer Identification for On-and
Offline Skilled Forgeries (SigWiComp 2013),” in ICDAR,
2013.
[14] M. Okawa, “Synergy of Foreground-background Images for
Feature Extraction: Offline Signature Verification Using Fisher Vector With Fused KAZE Features,” Pattern Recognition,
2018.
[15] J. Ruiz-del Solar, C. Devia, P. Loncomilla, and F. Concha,“Offline Signature Verification Using Local Interest Points
and Descriptors,” in CIAPR, 2008.
[16] A. Soleimani, B. N. Araabi, and K. Fouladi, “Deep Multitask
Metric Learning for Offline Signature Verification,” Pattern
Recognition Letters, 2016.
[17] F. Wang, L. Zhao, X. Li, X. Wang, and D. Tao, “Geometryaware Scene Text Detection with Instance Transformation
Network,” in CVPR, 2018.
[18] F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang,
X. Wang, and X. Tang, “Residual Attention Network for
Image Classification,” in CVPR, 2017.
[19] Q. Ye and D. Doermann, “Text Detection and Recognition in
Imagery: A Survey,” TPAMI, 2015.
[20] M. B. Yilmaz, B. Yanikoglu, C. Tirkaz, and A. Kholmatov,“Offline Signature Verification Using Classifier Combinationof HOG and LBP Features,” in IJCB, 2011.
[21] X. Zhang, J. Zou, K. He, and J. Sun, “Accelerating Very Deep
Convolutional Networks for Classification and Detection,”
TPAMI, 2016.
[22] Z. Zhang, X. Liu, and Y. Cui, “Multi-phase Offline Signature
Verification System Using Deep Convolutional Generative
Adversarial Networks,” in ISCID, 2016.
[23] Y. Zheng, Y. Cai, G. Zhong, Y. Chherawala, Y. Shi, and
J. Dong, “Stretching Deep Architectures for Text Recognition,” in ICDAR, 2015.
[24] Y. Zheng, B. K. Iwana, and S. Uchida, “Discovering ClassWise Trends of Max-Pooling in Subspace,” in ICFHR, 2018.
[25] Y. Zheng, B. K. Iwana, and S. Uchida, “Mining the Displacement of Max-pooling for Text Recognition,” Pattern
Recognition, 2019.
[26] E. N. Zois, L. Alewijnse, and G. Economou, “Offline Signature Verification and Quality Characterization Using Posetoriented Grid Features,” Pattern Recognition, 2016.
[27] B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le, “Learning
Transferable Architectures for Scalable Image Recognition,”in CVPR, 2018.