Semantic Conformance Testing 

Methodology and intial Results

for Fingeprint Minutia Encoding

Christoph Busch

 Fraunhofer IGD / Hochschule Darmstadt, Germany 

joint work with: 

Dana Hejtmankova, Elham Tabassi, Patrick Grother 

Wolfgang Krodel, Lars Neumann, Thimo Ruhland, 

Michal Dolezel, Ulrike Korte 

Properties of 

Finger Minutiae Detectors 

Christoph Busch Semantic Conformance Testing IBPC-2010 2
Encoding the Minutia Format 

When a ISO 19794-2 compliant 

feature extractor processes 

a biometric fingerprint image 

he generates a 

minutiae template. 

We will find the location 

for the core 

and for the delta 

and for many other minutiae. 

x 

y 

Christoph Busch Semantic Conformance Testing IBPC-2010 3

Deficiencies of the Minutia Encoder 

x 

y 

Unfortunately sometimes a 

feature extractor does not 

detect a landmark 

and thus relevant information 

is missing in the 

minutiae template. 

(sandstorm problem) 

Christoph Busch Semantic Conformance Testing IBPC-2010 4
Deficiencies of the Minutia Encoder 

x 

dy 

dx 

In other cases a 

feature extractor fails to 

properly detect a landmark 

and thus encodes 

the feature at a dis-located 

position in the template. 

(fata morgana problem) 

y 

Christoph Busch Semantic Conformance Testing IBPC-2010 5

Deficiencies of the Minutia Encoder 

x 

y 

Furthermore some 

feature extractor does 

not concentrate to the region 

of interest and thus detect 

spurious minutiae out of area 

or at the border of the imprint 

(globalisation problem) 

Christoph Busch Semantic Conformance Testing IBPC-2010 6
Conformance Testing Methodology of 

Finger Minutiae Detectors 

Christoph Busch Semantic Conformance Testing IBPC-2010 7

Conformance Testing 

Conformance testing is defined in a dedicated standard 

• ISO/IEC IS 29109-1 Information technology — Conformance 

testing methodology for biometric data interchange formats 

defined in ISO/IEC 19794 — Part 1: Generalized 

conformance testing methodology 

• ISO/IEC FDIS 29109-2 Information technology — 

Conformance testing methodology for biometric data 

interchange formats defined in ISO/IEC 19794 - Part 2: 

Finger minutiae data 

• ISO/IEC 29109-1 formulates the relevant test type „A“: 

! attesting that a unit is generating conformant 

biometric data interchange records. 

! in the case of fingerprint data this tests will verify that a unit 

(e.g. a minutia extraction algorithm) 

can create finger minutiae data records (templates) 

from appropriate fingerprint image data. 

Christoph Busch Semantic Conformance Testing IBPC-2010 8
Level of Conformance Testing 

There are various level of conformance tests: 

• Level 1 Basic Data Field Testing: 

! all data fields exist properly (e.g.in the correct encoding.) 

• Level 2 Internal Consistency Testing: 

! all data fields are filled with meaningful values 

and the fields are internally consistent. 

• Level 3 Semantic Testing: 

! a semantic test to verify that a generated biometric data 

interchange record is a faithful representation of 

the initial digital representation. 

Christoph Busch Semantic Conformance Testing IBPC-2010 9

Semantic Conformance Testing 

For fingerprint minutiae data 

• template consists of automatically generated minutiae (agm) 

! agm‘s are encoded by an implementation under test (IUT) 

• semantic test to be covered by ISO/IEC 29109-2 AMD1 

• semantic conformance is assessed by three rates: 

! 1.) Test for the sandstorm and the fata morgana problem: 

- Is the for every ground truth minutia (gtm) in the vicinity 

an automatically generated minutia (agm) in the template? 

! 2.) Test for the out-of-area problem (false minutia): 

- How many automatically generated minutiae (agm) are 

placed outside or at the border of a fingerprint area? 

! 3.) Test for spurious minutiae in the fingerprint area: 

- How many automatically generated minutiae (agm) 

do not have a mate in the gtm-set 

Christoph Busch Semantic Conformance Testing IBPC-2010 10
Proposed Testing Methodology 

• Sandstorm and fata morgana gtm-test: 

! The gt-minutiae assertion test yields a first conformance rate crgtm 

- indicating the proportion of elements 

in the set of gt-minutiae for which a corresponding minutia exists 

in the set of automatically generated minutiae, 

- such that values can be compared for each data field and 

differences can be measured. 

- the assertion requires 

the corresponding minutia 

to be in the vicinity. 

bifurcation detected 

dislocated as ridge ending 

Christoph Busch Semantic Conformance Testing IBPC-2010 11

Proposed Testing Methodology 

• Sandstorm and fata morgana gtm-test: 

! An exclusive gt-minutiae assertion test yields a 

first conformance rate crgtm

Pngtm mcsi crgtm = i=1

ngtm

- where ngtm is the number of elements in the gtm set 

- and mcsi is the minutia conformance score for the i-th gt-minutia 

that indicates the similarity between a gtm and the nearest minutia 

from the automatically generated minutiae set. 

! The mcs is non-zero, if the distance d between the minutiae 

positions is within the tolerance bounds told. 

Christoph Busch Semantic Conformance Testing IBPC-2010 12
Proposed Testing Methodology 

• Minutia conformance score W told = 4

0 if d told

mcs = 1 p otherwise

! where p is a potential punishment p = p

- dissimilarity in angle 

| gtm agm|·0,5

p =

- dissimilarity in minutia type 

0, 25 if tgtm = tagm p t = 0 otherwise

+ p t

Christoph Busch Semantic Conformance Testing IBPC-2010 13

Proposed Testing Methodology 







 









 



• Out-of-area agm-test: 

! identify the number of outside false minutiae (false minutiae) 

! an inside-of-area agm-minutiae assertion test is yielding 

a second conformance rate cr agm that is indicating

the proportion of elements in the set of agm 

that are inside or at the borderline of the fingerprint area. 

Christoph Busch Semantic Conformance Testing IBPC-2010 14
Proposed Testing Methodology 

• Inside-of-area agm-test assertion: 

! second conformance rate cragm 

Pnagm mpsi cragm = i=1

nagm

! where nagm is the number of elements in the agm set and 

mpsi is the minutia position score for the i-th ag-minutia 

that indicates the homogenious distribution of ag-minutia 

with respect to the fingerprint area. 

! metric will reflect a "punishment" for those agm 

that are on the borderline or outside the fingerprint area according 

m

Christoph Busch Semantic Conformance Testing IBPC-2010 15

Proposed Testing Methodology 





0 if agm is outside the fingerprint area

ps = 0, 5 if agm is at the borderline

1 otherwise



• Spurious agm-test: 

! The set of agm minutiae may contain spurious minutiae 

that are located in the fingerprint area 

- scars, bent skin, skin disease, dirt, etc. 

! third conformance rate cramf 

niagm cramf = 1

nagm

- where niagm is the number of focused agm 

inside the fingerprint area, 

which does not correspond to any gtm. 

Christoph Busch Semantic Conformance Testing IBPC-2010 16
Testing Methodology Process Flow 

Christoph Busch Semantic Conformance Testing IBPC-2010 17

Composing 

Ground Truth Fingerprint Minutiae 

Database 

Christoph Busch Semantic Conformance Testing IBPC-2010 18
Semantic Conformance Testing 

Challenge for implementing Semantic Testing: 

• What is the „real minutia coordinate“? 

• need for ground truth database (gtd) with minutiae data 

! need for public available fingerprint image data 

that is not restricted by privacy regulations 

- NIST special databases: 

SD14 rolled data and mostly ink with few live scanned images 

SD29 flat data /plain impression but all ink 

! need for dactyloscopic experts that define the truth! 

- Germany: Federal Criminal Police Office (BKA) 

- Australia: CrimeTrac (Andrew Wall) 

- Czech Republic: Criminalistic institute Prague 

- further volunteers? 

Christoph Busch Semantic Conformance Testing IBPC-2010 19

Graphical User Interface 

Ground Truth Minutia - GUI 
Composing a Ground Truth Database 

Ground Truth Minutia - data records 

Data fields in italics are defined in correspondance to ISO 19794-2:2005 

Composing a Ground Truth Database 

Database Segments 

• GTD-1: NIST SD14/SD29 selection 

! progress at BKA operation 

- Team of 11 dactyloscopic experts working partime on the 

Ground Truth Database 

- Approx. 1500 fingerprint images with 3 opinions 

as of January 15, 2010 

• GTD-2: Latent fingerprints from crime scenes 

! operation to be started in April 2010 

! based on ELFT-EFS latent data 

• GTD-3: Live prints 

! preparation work 

Christoph Busch Semantic Conformance Testing IBPC-2010 22
Benefit of 

a Ground Truth Minutiae 

Database 

Christoph Busch Semantic Conformance Testing IBPC-2010 23

Benefit of a Ground Truth Database 

Database can serve for many purposes 

• providing the ground for development of a 

semantic conformance test methodology 

• providing the ground for 

semantic conformance tests according ISO 29109-2 AMD1 

• providing the ground for development and calibration of 

fingerprint image sample quality metrics 

! NFIQ2-development and training 

• providing the ground for 

dactyloscopic training software 

Christoph Busch Semantic Conformance Testing IBPC-2010 24
Technical Challenges 

Christoph Busch Semantic Conformance Testing IBPC-2010 25

Technical Challenges 

Strong impact on conformance rate by two factors 

• minutiae clustering method 

• fingerprint area definition 

Christoph Busch Semantic Conformance Testing IBPC-2010 26
Technical Challenges 

-

Clustering Minutiae Data 

• Fusion of expert markup is a non-trivial taks 

! the number of clusters (minutiae) is unkown 

! the number of experts - contributing to one clusteris also unkown. 

Christoph Busch Semantic Conformance Testing IBPC-2010 27

Clustering Minutiae Data 

Automated Generated Minutia vs.Ground Truth Minutia 

AGM - NIST mindtct GTM 
Clustering Minutiae Data 

Identify cluster members 

• Minutiae pairs, where: 

! Each minutiae has been placed by different expert. 

! The distance between minutiae is less or equal than W/2 (minutiae 

inside a circle with radius W/4). 

• Triplets from minutiae pairs, where: 

! Pairs have one same (joint) minutiae + prev. cond. 

! Until n-tuples (n is number of experts). 

Christoph Busch Semantic Conformance Testing IBPC-2010 29

Clustering Minutiae Data 

Identify consensus 

Pngtm Pngtm

i=1 xi i=1 yi

• Average position XGTM = ngtm , YGTM = ngtm

• Average type - UNKOWN, if less than 2/3 consensus 

• Average angle 

! Conversion of angles into coordinates. 

! Computation of average coordinates. 

- Less than 1/3 of length means UNKNOWN. 

Christoph Busch Semantic Conformance Testing IBPC-2010 30

0 0

! Conversion of coordinates into angle. 1 

x x -1 1 -1 1 

-1 -1 

y y 
Clustering Minutiae Data 

Identifying Reliable Minutiae Clusters 

• How many experts identified this minutiae and 

how confident are they (quality of minutiae)? 

Pncl

i=1 qi qcl = nexp

! where qi is the minutiauality of the i-th minutia in the cluster, 

ncl is the number of minutia in that cluster and nexp is number of 

experts processing this image. 

gtm from 11 

experts 

All cluster centers Most reliable

cluster centers 

Fingerprint Area Detection 

Currently semi-automatic procedure 

• A) segmentation according area quality of mindtct 

• B) segmentation based on block-based gabor filter response 

! Shen et al. Quality Measures of Fingerprint Images 

! Alonso-Fernandez et al. An Enhanced Gabor Filter-Based 

Segmentation Algorithm for Fingerprint Recognition Systems 

A B 
Fingerprint Area Detection 

Currently semi-automatic procedure 

• segmentation according area quality of mindtct 

Fingerprint Area Detection 

Manual inspection 
Organisation Challenges 

Christoph Busch Semantic Conformance Testing IBPC-2010 35

Challenges for the Ground Truth Database 

Issues 

• Synchronize work with progress of ISO 29109-2 AMD1 

• Ground-Truth Database (gtd) 

! Identify more institutions 

from various dactyloscopic cultures 

contributing to the entire set 

! Keep the ground truth data confidential 

- NDAs with contributing institutions 

- The public fraction should contain approx. 30 percent of the total 

data and will be available in the public domain. 

- The sequester fraction will be provided under strong restriction 

to testing institution only 

! ISO/IEC JTC1 SC37 WG3 need to establish a procedure to 

autorize testing institution to receive a copy of the sequester 

database. 

Christoph Busch Semantic Conformance Testing IBPC-2010 36
Initial Test Results 

Christoph Busch Semantic Conformance Testing IBPC-2010 37

Test Results 

conformance 

rates crgtm cragm cramf 

average 0,353 0,885 0,662 

std. deviation 0,179 0,066 0,178 

Results BIOSIG 2009 

• 17 images, max 11 experts each 

• average ngtm: 59 

• average agm: 100 (for NIST mindtct) 

Christoph Busch Semantic Conformance Testing IBPC-2010 38
Test Results 

Results BioKey 2010 

• Available ground truth data approx. 1000 images 

• Dependency of the database 

! cr = f ( minutiae-extractor, db, ct-methodology) 

• Data set reduced (filtered) in cases 

! when less then 3 experts markups 

were available 

! where images showed archive artifacts 

! with two imprints in the image 

• Data set not reduced 

! where images showed second phalanx 

- second phalanx is considered as 

part of the fingerprint area 

Christoph Busch Semantic Conformance Testing IBPC-2010 39

Test Results 

Results BioKeyS 2010 

• 3 experts opinions each for 975 images (733 used) 

! SD14: 486 images / SD29: 247 images 

• SD14 average ngtm: 76 (min 7 / max 174) 

• SD14 average agm: 201 (min 87 / max 366) (NIST mindtct) 

conformance 

rates crgtm cragm cramf 

average 0,464 0,857 0,645 

std. deviation 0,092 0,063 0,123 

• Generating this result was kindly supported by the German BSI 

under the BioKeyS-Pilot-DB project 

Christoph Busch Semantic Conformance Testing IBPC-2010 40
Open Issues 

Other deficiencies of extractors 

• How to deal with Failure-to-eXtract 

! contribution with a cr=0 

Respecting the minutia type 

! if minutia type is „other“ we should not analyze the orientation 

Christoph Busch Semantic Conformance Testing IBPC-2010 41

Conclusion 

• Conformance testing essential step in system selection 

• Semantic conformance testing requires ground truth data 

• Further datyloscopic experts groups welcome 

• Testing methodology under development 

! fusion of conformance rates 

! thresholds for the conformance rate 

• Further data segments addressed soon 

Christoph Busch Semantic Conformance Testing IBPC-2010 42
Further Information 

on Semantic Conformance Testing 

• On March 5, at 11:30 a session 

on fingerprint feature markup and testing will be held. 

! This workshop will discuss work in this area, 

interoperability, reference datasets, 

and the possibilities for semantic conformance testing. 

• Publications: 

! C. Busch, D. Lodrova, E. Tabassi, W. Krodel: "Semantic Conformance 

Testing for Finger Minutiae Data", in Proceedings of the IEEE International 

Workshop on Security and Communication Networks (IWSCN), Trondheim, 

ISBN 978-82-997105-1-0, pages 17-23, (2009) 

! D. Lodrova, C. Busch, E. Tabassi, W. Krodel, M. Drahansky: „Semantic 

Conformance Testing Methodology for Finger Minutiae Data“, in 

Proceedings BIOSIG 2009, (2009) 

• Website with information on the topic 

! http://www.igd.fraunhofer.de/~busch/gtd 

Christoph Busch Semantic Conformance Testing IBPC-2010 43

Thank you for your attention 

and many thanks 

to the dactyoloscopic experts 

contributing to the 

ground truth database 

Christoph Busch Semantic Conformance Testing IBPC-2010 44
Contact 

Prof. Dr. Christoph Busch Fraunhoferstrasse 5 

64283 Darmstadt, Germany 

Phone: +49-6151-155-536 

Department christoph.busch@igd.fraunhofer.de 

Security Technology www.igd.fraunhofer.de/~busch 

Christoph Busch Semantic Conformance Testing IBPC-2010 45