Benchmarking Quality-dep

Cost-sensitive Multimodal 

Fusion Algorithm

Centre for Vision, Speech and Signal Pr

School of Electronics and 

Univers

Norman Poh, Thirimahos Bourlai, Josef K
500 users

Biometric Authenticatio

Access Control Scen
Motivations

How well can a multimodal biometric system coinformation

?

How well can automatically derived quality mea

the fusion system performance?

How well can a multimodal system perform givecomputation and in the presence of hardware/s

Failure to enrol and failure to match

What if the device used during authentication is that used during enrollment? [device mismatchPrincipally interested in performance improvemuse of quaulity measures in fusion 

w.r.t the baseline system w.r.t. a fusion system without given any quality 

Not particularly concerned with state-of-the-art 

Simulate failures by masking the data!
Conventional Fusion Alg

PCA

LDA

DCT GMM

MLP

MSE
Quality-dependent Fusion A

PCA

LDA

DCT GMM

MLP

MSE

Face/image quality 

detectors

3. Quality 

measures

1. Device 2. Reference systems Fusi

t
Score/Quality Meas

Generation Princip

Use baseline systems

standard algorithmsLDA for face, NIST’s fingerprint matcher, Dafor iris

Fully automatic segmentation and matc

If a system cannot process a query, e.g., dusegment or failure to match, output a dumm999’

Automatically computed quality measur

If a quality detector fails, output a dummy ‘

Consequence: Algorithms have to deal wobservations/values
Capturin

g Device
Face Quality Measu

Face

Frontal qualityIlluminationRotationReflectionSpatial resolution (between eyes)Color bit per pixelFocusBrightnessBackground informityGlasses

Glass=89% 

Illum.=100% 

Well 

illuminated

Quality m

Quality mea
Low vs High Quality Fac

webcam

Digital camera

Note: quality (e.g., image resolution), dependents on 

the device and its operational settings (e.g., white 

balance adjustment).

I

C
Examples of Segmente

Images

Face detection may fail, but the 

matching will proceed anyway!

Examp

seg
Experimental Proto

Baseline system

Genuine 

users [51]

Impostors 

[103]

Genui

users [1

Development set Evalua

Genuine 

scores

Impostor 

scores

Genui

score

For fusion algorithm development –tuning 

parameters and decision threshold

Uniquely for 

Sequested d

Same/diff. sessions

Same/diff. devices

Two non-overlapping partition of 
Preliminary Performance 

session 1 test datasession 2 test data

Device mismatch

(7 channels of data)

Fa: fFnf: FwfFo: Ft: fxFa: tempxFt: temp

Same device

(17 channels of data)

Intis coptcomsesDevdeg
Evaluation Resul

Tobias Scheidat (AMSL-BIO, U. of Magdeburg)

Lorene Allano, Institut National des Télécommunications (G

Fernando Alonso, Universidad Autonoma de Madrid, (UPM)

O Fatukasi and N. Poh, U. of Surrey (UniS), UK.

Harald Ganster, Joanneum Research (JR), Austria

Albert Salah and Onkar Ambekar, Centrum voor Wiskunde (CWI), the Netherlands

John Baker, Johns Hopkins University Applied Physics Labo

7 teams, 22 fusion algorithms, 2 evaluation protoco

Examples of algorithms submitted: 

Naive bayes, Bayes classifier with GMM density estimato

factor alayzers, logsitic regression, fixed rule, device-spe

classifier (with error-dependen weights), SVM, baye

Dempster fusion rule
e

Dynamic 

selection

Static 

selection

Baseline performance

Cost-sensitive evalua

Limit of 

system 

performanc

More systemsÆmore costly but also higher robustness to h

1 face

6 fingers

1 iris
Quality-based Evalu

w

Te

Q

1 face

3 fingers
Q&A

Thank you!
Ideally, two thresholds are needed for the m

to remove eye-lids and eyelashes

The threshold for eyelashes are not optimal too (no

Minimally Optimized E

and Eyelashes Segme

Segmented iris

Iris code Iris mask

Wrapped iris imag
Intra-site Diversi

Each row represents data collected at a site
Cross -site Divers
Fingerprint Quality Me

Low quality region (low contrast)

The global quality is the average of local quality 

Optical sensor (impression) Thermal 

image 
Iris Quality Measu

Pupil diameter,

dP

Iris diameter,

dI

Q2 = dI -dP

Q3 = Proportiomasked use for matching

Q1 = Average (similar to fingmap)
Badly Segmented Iris 

Good 

segmentation Bad segmentatio
Devices and Modal
Reference Systems 

Quality Measure
Example of Quality Me

to Distinguish Two D
Ability of Quality Mea

Distinguish Devices (fi

Face quality measures 

used are 5,6,8
Some Observatio

Quality measures can be used estimate the identity of the deQuality measures are device-d
Baseline Performa

Note: If all the data in a channel is used, 

average cost per access is simply 1.