The Benefit of Ground Truth Data to 

Semantic Conformance Testing of 

Fingeprint Minutia Encoding

Christoph Busch

 Fraunhofer IGD / Hochschule Darmstadt, Germany 

joint work with: 

Dana Hejtmankova, Elham Tabassi, Patrick Grother 

Wolfgang Krodel, Lars Neumann, Timo Ruhland, 

Michal Dolezel, Ulrike Korte 
Introduction 

• What is „Ground Truth Data“ for 

Fingerprint Minutia Encoding? 

• a winter prespective.... 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 2
Introduction 

This is Africa 

and we can consider it as a 

biometric characteristic 

geographic characteristic. 

It is 

a continental part 

of our body globe. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 3
Introduction 

and 

this is the Sahara, 

which is 

a significant part of Africa 

as it is the world‘s largest 

hot desert. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 4
Introduction 

and 

this is a palm tree 

there are many palm trees 

in the Sahara. 

They serve as landmarks 

and we like to see them 

close to some water (oasis). 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 5
Introduction 

x 

y 

and 

this is a representation 

of the geographic characteristic 

that has been generated 

from the geographic sample. 

The representation contains 

features which encode 

the locations of the landmarks. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 6
Introduction 

x 

y 

unfortunately 

the Sahara is the source 

of sandstorms that causes 

regionally a bad visibilty. 

In consequence 

the feature extractor 

will not detect the 

poorly visible landmark 

und Truth Data IBPC-2010 7

 

Christoph Busch Benefit of Gro
Introduction 

dy 

dx 

x 

y 

a second problem 

is a fata morgana 

that causes the 

feature extractor to detect 

the feature (with an oasis?) 

at a dis-located position. 

Thus the encoded feature 

does not represent the landmark 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 8
Introduction 

x 

a third problem 

could be the globalisation. 

If the feature extractor seeks 

for landmarks out of area 

he may detect falsely a feature 

at a position, where 

you can find in reality rather 

other biological entities (elks) 

y 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 9
Properties of 

Finger Minutiae Detectors 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 10
Representation in Minutia Format 

ISO/IEC 19794-2: Biometric data interchange formats -

Part 2: Finger minutiae data 

• Ridges and valleys, core and delta 

• Ridge bifurcation and ridge endings 

‣ finger minutiae 

• Encoded main features 

‣ Minutia point (coordinates x,y) 

‣ Minutia direction (angle θ) 

‣ Minutia type 

• Other features are e.g. 

‣ quality 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 11
Encoding the Minutia Format 

x 

y 

When a ISO 19794-2 compliant 

feature extractor processes 

a biometric fingerprint image 

he generates a 

minutiae template. 

We will find the location 

for the core 

and for the delta 

and for many other minutiae. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 12
Deficiencies of the Minutia Encoder 

x 

y 

Unfortunately sometimes a 

feature extractor does not 

detect a landmark 

and thus relevant information 

is missing in the 

minutiae template. 

(sandstorm problem) 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 13
Deficiencies of the Minutia Encoder 

x 

dy 

dx 

y 

In other cases a 

feature extractor fails to 

properly detect a landmark 

and thus encodes 

the feature at a dis-located 

position in the template. 

(fata morgana problem) 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 14
Deficiencies of the Minutia Encoder 

x 

y 

Furthermore some 

feature extractor does 

not concentrate to the region 

of interest and thus detect 

spurious minutiae out of area 

or at the border of the imprint 

(globalisation problem) 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 15
Conformance Testing Methodology of 

Finger Minutiae Detectors 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 16
Conformance Testing 

Conformance testing is defined in a dedicated standard, 

which is currently under development 

‣ ISO/IEC IS 29109-1 Information technology — Conformance 

testing methodology for biometric data interchange formats defined 

in ISO/IEC 19794 — Part 1: Generalized conformance testing 

methodology 

‣ ISO/IEC FDIS 29109-2 Information technology — Conformance 

testing methodology for biometric data interchange formats defined 

in ISO/IEC 19794 - Part 2: Finger minutiae data 

‣ ISO/IEC 29109-1 formulates the relevant test type „A“: 

- attesting that a unit is generating conformant 

biometric data interchange records. 

- in the case of fingerprint data this tests will verify that a unit 

(e.g. a minutia extraction algorithm) 

can create finger minutiae data records (templates) 

from appropriate fingerprint image data. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 17
Level of Conformance Testing 

 

There are various level of conformance tests: 

• Level 1 Basic Data Field Testing: 

‣ all data fields exist properly (e.g.in the correct encoding.) 

• Level 2 Internal Consistency Testing: 

‣ all data fields are filled with meaningful values 

and the fields are internally consistent. 

• Level 3 Semantic Testing: 

‣ a semantic test to verify that a generated biometric data 

interchange record is a faithful representation of 

the initial digital representation. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 18
Semantic Conformance Testing 

For fingerprint minutiae data 

• template consists of automatically generated minutiae (agm) 

‣ agm‘s are encoded by an implementation under test (IUT) 

• semantic test to be covered by ISO/IEC 29109-2 AMD1 

• semantic conformance is assessed by three rates: 

‣ 1.) Test for the sandstorm and the fata morgana problem: 

- Is the for every ground truth minutia (gtm) in the vicinity 

an automatically generated minutia (agm) in the template? 

‣ 2.) Test for the out-of-area problem (false minutia): 

- How many automatically generated minutiae (agm) are 

placed outside or at the border of a fingerprint area? 

‣ 3.) Test for spurious minutiae in the fingerprint area: 

- How many automatically generated minutiae (agm) 

do not have a mate in the gtm-set 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 19
Proposed Testing Methodology 

• Sandstorm and fata morgana gtm-test: 

‣ The gt-minutiae assertion test yields a first conformance rate crgtm 

- indicating the proportion of elements 

in the set of gt-minutiae for which a corresponding minutia exists 

in the set of automatically generated minutiae, 

- such that values can be compared for each data field and 

differences can be measured. 

- the assertion requires 

the corresponding minutia 

to be in the vicinity. 

bifurcation detected 

dislocated as ridge ending 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 20
Proposed Testing Methodology 

• Out-of-area agm-test assertion: 

‣ asses the number of outside false minutiae (false minutiae) 

‣ an out-of-area agm-minutiae assertion test is yielding 

a second conformance rate cragm that is indicating 

the proportion of elements in the set of agm

that are inside or at the borderline of the fingerprint area. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 21
Proposed Testing Methodology 





0 if agm is outside the fingerprint area

mps = 0, 5 if agm is at the borderline

1 otherwise

• Out-of-area agm-test assertion: 

‣ second conformance rate cragm 

Pnagm mpsi

= i=1 cragm nagm

‣ where nagm is the number of elements in the agm set and 

mpsi is the minutia position score for the i-th ag-minutia 

that indicates the homogenious distribution of ag-minutia 

with respect to the fingerprint area. 

‣ metric will reflect a "punishment" for those agm

that are on the borderline or outside the fingerprint area according 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 22
Proposed Testing Methodology 



• Spurious agm-test assertion: 

‣ The set of agm minutiae may contain spurious minutiae 

that are located in the fingerprint area 

- scars, bent skin, skin disease, dirt, etc. 

‣ third conformance rate cramf 

niagm cramf = 1

nagm

- where niagm is the number of focused agm

inside the fingerprint area, 

which does not correspond to any gtm. 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 23
Composing 

Ground Truth Fingerprint Minutiae Date 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 24
Semantic Conformance Testing 

Challenge for implementing Semantic Testing: 

• What is the „real minutia coordinate“? 

• need for ground truth database (gtd) with minutiae data 

‣ need for public available fingerprint image data 

that is not restricted by privacy regulations 

- NIST special databases: 

SD14 rolled data and mostly ink with few live scanned images 

SD29 flat data /plain impression but all ink 

‣ need for dactyloscopic experts that define the truth! 

- Germany: Federal Criminal Police Office (BKA) 

- Australia: CrimeTrac 

- Czech Republic: Criminalistic institute Prague 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 25
Graphical User Interface 

Ground Truth Minutia - GUI 
Benefit of 

a Ground Truth Minutiae 

Database 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 27
Benefit of a Ground Truth Database 

Database can serve for many purposes 

• providing the ground for development of a 

semantic conformance test methodology 

• providing the ground for 

semantic conformance tests according ISO 29109-2 AMD1 

• providing the ground for development and calibration of 

fingerprint image sample quality metrics 

‣ NFIQ2-development and training 

• providing the ground for 

dactyloscopic training software 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 28
Initial Test Results 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 29
Test Results 

Results BIOSIG 2009 

• 17 images, max 11 experts each 

• average ngtm: 59 

• average agm: 100 (for NIST mindtct) 

conformance 

rates crgtm cragm cramf 

average 0,353 0,885 0,662 

std. deviation 0,179 0,066 0,178 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 30
Test Results 

Results BioKeyS 2010 

• 3 experts opinions each for 975 images (733 used) 

‣ SD14: 486 images / SD29: 247 images 

• SD14 average ngtm: 76 (min 7 / max 174) 

• SD14 average agm: 201 (min 87 / max 366) (NIST mindtct) 

conformance 

rates crgtm cragm cramf 

average 0,464 0,857 0,645 

std. deviation 0,092 0,063 0,123 

• Generating this result was kindly supported by the German BSI 

under the BioKeyS-Pilot-DB project 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 31
Conclusion 

• Conformance testing essential step in system selection 

• Semantic conformance testing requires ground truth data 

• Further datyloscopic experts groups welcome 

• Testing methodology under development 

‣ fusion of conformance rates 

‣ thresholds for the conformance rate 

• Further data segments addressed soon 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 32
Further Information 

on Semantic Conformance Testing 

• On March 5, at 11:30 a session 

on fingerprint feature markup and testing will be held. 

‣ This workshop will discuss work in this area, 

interoperability, reference datasets, 

and the possibilities for semantic conformance testing. 

• Publications: 

‣ C. Busch, D. Lodrova, E. Tabassi, W. Krodel: "Semantic Conformance 

Testing for Finger Minutiae Data", in Proceedings of the IEEE International 

Workshop on Security and Communication Networks (IWSCN), Trondheim, 

ISBN 978-82-997105-1-0, pages 17-23, (2009) 

‣ D. Lodrova, C. Busch, E. Tabassi, W. Krodel, M. Drahansky: „Semantic 

Conformance Testing Methodology for Finger Minutiae Data“, in 

Proceedings BIOSIG 2009, (2009) 

• Website with information on the topic 

‣ http://www.igd.fraunhofer.de/~busch/gtd 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 33
Thank you for your attention 

and many thanks 

to the dactyoloscopic experts 

contributing to the 

ground truth database 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 34
Contact 

Fraunhoferstrasse 5 

64283 Darmstadt, Germany 

Phone: +49-6151-155-536 

christoph.busch@igd.fraunhofer.de 

www.igd.fraunhofer.de/~busch 

Prof. Dr. Christoph Busch 

Department 

Security Technology 

Christoph Busch Benefit of Ground Truth Data IBPC-2010 35