1 

Overview of biometric quality 

Elham Tabassi – NIST 

Biometric Consor7um 2009 

23 September 2009 
outline 

- Why measure quality? 

- What is meant by quality? 

- What are they good for? 

- How to assess quality of quality? 

- What are the challenges in quality computa7on? 

BCC September 23, 2009  elham.tabassi@nist.gov  2 
Why measure quality? 

Push towards zero‐error biometrics 

While recogni7on technologies are good at what they are being 

used, or contemplated for use, their performance drop in difficul

opera7onal scenarios and with imperfectly controlled data. 

Although only a small frac7on of input data are of poor‐quality, 

the bulk of recogni7on errors can be aOributed to poor‐quality 

samples. 

Improving quality either by sensor design, by user interface 

design, or by standards compliance, beOer performance can be 

realized.  

For those aspects of quality that cannot be designed‐in, an ability

to analyze the quality of a live sample is needed.  

t 

BCC September 23, 2009  elham.tabassi@nist.gov  3 
4 

What is meant by quality? 

Predic<ve of performance

Quality problem: “The Last 1%” 

Or maybe “The Last 0.1% or 10%” 

» Frac7on of samples that should not be sent to the matcher 

– finger, iris scanners have been designed specifically for the task, 

face cameras (mostly) have not 

– providing construc7ve feedback only possible if cause of poor 

quality is known 

character, e.g. scar environment, e.g. 

shadows

behavior, e.g 

pose

imaging, e.g. focus

BCC September 23, 2009  elham.tabassi@nist.gov 
What are the uses of quality? 

⚀ Ini7a7ng the reacquisi7on from a user 

- Enrollment 

• for creden7al issuance (visa, passport, access card, PIV) 

• pruning the poorest quality samples (1.65% of dataset) reduced EER from .0047 t0 .0024 

- Verifica7on  

• of the samples just captured which one to send for matching?  

• or acquire s7ll more? 

- Iden7fica7on 

• is the subject offering a poor sample deliberately? 

⚂ Selec7ve invoca7on of different processing methods 

- Preprocessing phase 

• image restora7on algorithms (e.g., contrast adjustment) or a different feature extrac7on  

- Matching phase 

• invoke a slower but more powerful matching algorithm when low‐quality samples are 

compared 

• sending poor quality (NFIQ=4,5) to a more accurate (but perhaps costly)  matcher reduced 

FNMR from 0.0136 to 0.0078 at FMR=0.001 

- Decision phase 

• quality directed fusion, dynamic threshold 

• performing quality based mul7‐algorithm con7ngent likelihood ra7o fusion reduced FNMR 

from 0.0136 to 0.0068 at FMR=0.001 

⚃ Sample replacement/Template update  

• negate template aging 

⚄ Quality monitoring 

• are some biometric field loca7ons giving low quality? 

• only in the evening? 

5 
Predic<ve of performance 

Black Box 

Quality 

Assessment  

93 

26 

A biometric quality assessment method derives a numerical quality 

value from an input biometric sample. The quality value is related to the 

biometric error rates that are likely to be realized when the sample is 

matched.  

BCC September 23, 2009  elham.tabassi@nist.gov  6 
Quality: rank sta<s<c for performance 

NFIQ=3

NFIQ=1

NFIQ=2

NFIQ=4

NFIQ=5

ALL

NFIQ::(fmr,fnmr) at fixed threshold  BCC September 23, 2009  elham.tabassi@nist.gov  7 
NIST fingerprint image quality 

- NIST developed NFIQ in 2004 

∸ Open source 

- Key innova7on: quality as a rank sta7s7c for 

performance 

- NFIQ is a machine learning algorithm  

∸ Exploratory variables: image proper7es (minu7ae, ridge 

density and clarity) 

∸ Response variable: separa7on of genuine and impostor 

comparison 

BCC September 23, 2009  elham.tabassi@nist.gov  8 
quality 

number 

 feature       neural  

extrac<on      network

NFIQ 

- feature extrac<on: computes appropriate signal or image fidelity 

characteris7cs and results in an 11‐dimensional feature vector. 

- neural network:classifies feature vectors into five classes of quality 

based on various quan7les of the normalized match score 

distribu7on. 

- quality number: an integer value between 1(highest) and 5 (poorest).

BCC September 23, 2009  elham.tabassi@nist.gov  9 
Error rates per NFIQ level  

September 17, 2009  elham.tabassi@nist.gov  10 
Quality of quality 

- Biometric quality shall be evaluated 

based on its ability to predict 

performance. 

∸ e.g. error vs. reject curves 

- Comparison of quality algorithms shall 

compare their effec7veness in 

predic7ng performance. 

BCC September 23, 2009  elham.tabassi@nist.gov  11 
Quality challenges 

⚀ Scalar vs vector 

⚁ Matcher dependency 

⚂ How many levels? 

⚃ Pair‐wise quality 

⚄ Calibra7on 

BCC September 23, 2009  elham.tabassi@nist.gov  12 
false non-match rate 

5 4 3 2 1 

NFIQ 

Statistically different level of performance 

How many levels?

BCC September 23, 2009  elham.tabassi@nist.gov  13 
Pair‐wise quality ‐ 1 

fingerprint 

matching algorithm 

similarity 

score 

matching algorithm 

Q1

Q2

Pair-wise quality 

H(Q1,Q2)

when the enrollment sample is of good quality and 

beOer than that of the authen7ca7on sample, the 

authen7ca7on sample’s quality is sufficient to 

predict performance.

BCC September 23, 2009  elham.tabassi@nist.gov  14 
pair‐wise quality 

BCC September 23, 2009  elham.tabassi@nist.gov  15 
Quality  

WHAT AFFECTS QUALITY (IMAGE 

LEVEL) 

⚀ devise metrics for quantifying specific 

aspects of quality to 

∸ distinguish cause of poor quality 

(provide feedback) 

∸ Introduce tolerance (quality-bydesign) 

⚁ design framework for assessing effect 

of quality 

∸ sensitivity analysis to each metric 

∸ statistical method for combining 

effects (neural net, svm, etc) 

⚂ perform analysis on large datasets of 

images 

∸ preferably on subset of images 

with specific defect (focus, pose, 

...) 

WHAT IS THE EFFECT OF QUALITY 

(APPLICATION LEVEL) 

⚀ devise metrics for quantifying the dependence 

of the accuracy of the core algorithms on 

∸ the quality of biometric samples (error vs 

reject), and 

∸ systematic quality variation (quality 

summarization procedure) 

⚁ design framework for assessing effect of qualit

on accuracy and security 

∸ how quality of enrolled samples affects 

probability of false accept? 

∸ How about probability of false reject? 

⚂ perform analysis on large datasets of images 

∸ examine the role of quality in biometric 

zoo 

y 

BCC September 23, 2009  elham.tabassi@nist.gov  16 
BCC September 23, 2009  elham.tabassi@nist.gov  17 
Thank You 

Elham Tabassi 

tabassi@nist.gov 

301 975 5292 

www.itl.nist.gov/iad/894.03/quality 
Dependence on matcher

NFIQ 

FNMR 

Each point corresponds to one algorithm. 

BCC September 23, 2009  elham.tabassi@nist.gov  19 
Quality score interpreta<on 

Black Box 

Quality 

Algorithm 

93 

26 

Another Black 

Box Quality 

Algorithm 

1 

5 

BCC September 23, 2009  elham.tabassi@nist.gov  20 
21 

Calibra<on Curve: Error vs reject : NFIQ 

BCC September 23, 2009  elham.tabassi@nist.gov