IBPC 2010 NIST 

Terry Hartmann 

Vice President, 

Identity Solutions 

4 March 2010 
The Integrator Perspective 

What is an integrator ? 

© 2008 Unisys Corporation. All rights reserved. Page 2 
What I wish I knew the Metrics for ?! 

© 2008 Unisys Corporation. All rights reserved. Page 3 
Opening Address Recap 

PATRICK 

• Integrator = Risk Mitigation 

• Integrator Biometrics Interest 

– Scientific Discovery 

– Technical Discovery 

– R&D 

– Capability of vendor (reliability, avail) 

– Comparative 

– Interoperability 

– Conformance (Reqmts, Stds) 

– Regression (should we update) 

– Calibration (what threshold) 

– Jim (Useability, Vulnerability, ROI) 

Tony 

• Have tests been driven by 

what can be done, rather than 

what should be done ? 

• Are tests missing the point ? 

It’s all about money ! 

© 2008 Unisys Corporation. All rights reserved. Page 4 
NIST FP Vendor Tech 

Eval 2003 

© 2008 Unisys Corporation. All rights reserved. Page 5 
The turn of the century ... 

© 2008 Unisys Corporation. All rights reserved. Page 6 
© 2008 Unisys Corporation. All rights reserved. Page 7 

FRVT2002 

FRVT2006 
FRVT2006 

© 2008 Unisys Corporation. All rights reserved. Page 8 
© 2008 Unisys Corporation. All rights reserved. Page 9

FRVT2002 vs FRVT2006 
© 2008 Unisys Corporation. All rights reserved. Page 10
© 2008 Unisys Corporation. All rights reserved. Page 11
© 2008 Unisys Corporation. All rights reserved. Page 12
© 2008 Unisys Corporation. All rights reserved. Page 13
Customers use NIST tests in Evals 

“If the proposed FR Solution 

product(s) have been 

tested in the National Institute of 

Standards and Technology’s 

(NIST) Facial Recognition Test 

FRVT 2006, the results of the 

test can be used as a 

substantiation of compliance 

with requirements.” 

“Certification that the 

proposed AFIS underwent 

the Fingerprint Vendor 

Technology Evaluation of 

2003 (FPVTE 2003) – 

Large Scale Test Category 

by the National Institute of 

Standards & Technology 

(NIST).” 

© 2008 Unisys Corporation. All rights reserved. Page 14 
??? 

• How many times in the last 2 days have we heard presenters 

only ask questions ? 

• Customers look to industry (integrators and vendors) for 

answers, and end up in frustration doing tests themselves 

disguised as PoC 

• Jim: “Disconnect between lab performance and field results” 

• BSI: “all world is ignoring spoofs” 

© 2008 Unisys Corporation. All rights reserved. Page 15 
Tell me the answers ! 

• terry.hartmann@unisys.com 

• Phone +1 267-475-7618 

• Reston, Virginia, USA 

© 2008 Unisys Corporation. All rights reserved. Page 16 
Have you 

ever bought 

a PC ? 

© 2008 Unisys Corporation. All rights reserved. Page 17 
Maybe you bought this once ? 

© 2008 Unisys Corporation. All rights reserved. Page 18 
www.pc-vendor.com 

© 2008 Unisys Corporation. All rights reserved. Page 19 
Integrator Metric Wish # 1 

That I could buy my 

biometric algorithm 

online 

But my competitors 

couldn’t •

© 2008 Unisys Corporation. All rights reserved. Page 20 
© 2008 Unisys Corporation. All rights reserved. Page 21
What’s under the covers ? 

© 2008 Unisys Corporation. All rights reserved. Page 22 
Metric Wish # 2 

daugman 

© 2008 Unisys Corporation. All rights reserved. Page 23 
Government 

• Conduct Funded Tests 

• Conduct Bakeoffs 

• Speak at conferences about 

– what their program is 

– what they tested for 

– what they did 

– how they did it 

– lessons learned 

• Test labs a bit more open but 

NDA constrained 

© 2008 Unisys Corporation. All rights reserved. Page 24 
But Government 

Don’t speak at conferences about 

• which vendors were in the field 

• how they downselected 

• quantitative measures 

• what the relative results were 

between vendors 

• what’s their “biometric” FRR 

• what’s their operational FAR 

© 2008 Unisys Corporation. All rights reserved. Page 25 
Metric Wish # 3 

That Public Sector 

testors would publish/ 

circulate definitive 

comparative results 

and that the media 

would ignore them ;) 

© 2008 Unisys Corporation. All rights reserved. Page 26 
Face ... the final frontier 

© 2008 Unisys Corporation. All rights reserved. Page 27 
Is this the Same Person ? 

1. Yes 

2. No 

3. It’s a stupid, ill-formed question 

© 2008 Unisys Corporation. All rights reserved. Page 28 
Real People Performance vs Auto FR 

© 2008 Unisys Corporation. All rights reserved. Page 29 
© 2008 Unisys Corporation. All rights reserved. 

ISO 19794-5 

Page 30 
Auto-fixing Positioning 

Centre head 

Eyes

horizontal 

Page 31 
Metric Wish # 4 

Facial Image Quality 

Software Comparative 

Test 

•Do they really meet ISO 19794-5 ? 

•Are they algorithm independent ? 

•Is there value in auto-correction ? 

•Is it an enrolment tool or a matching tool ? 

•Is it ‘safe’ to do so? 

•How do we quantify it’s worthwhile ? 

•QA the eyefinder ? 

© 2008 Unisys Corporation. All rights reserved. Page 32 
Have you 

ever bought 

a digital 

camera ? © 2008 Unisys Corporation. All rights reserved. Page 33 
Now tell me ... 

• Now you have it 

• How many Megapixels do you capture at ? 

• How many Megapixels do you save at ? 

• Email ?, Facebook ? 

© 2008 Unisys Corporation. All rights reserved. Page 34 
Metric Wish # 5 

What resolution to 

capture, save, 

compress ? 

•Low resolution image matching results 

comparisons – study of resolution set images 

ranging from 20 pixels to 120 pixels IED in 10 

pixel increments – plot the curves, ultimately to 

400dpi 

•Can we set a “resolution” quality measure ? 

eg FRVT2006: 6MP, 400 pixels between eyes 

© 2008 Unisys Corporation. All rights reserved. Page 35 
Metric Wish # 6 

Typical customer 

questions 

•How long does it take to enrol 100,000 images ? 

•What age can I reliably match children at ? 

•Effects of aging on FR results – longitudinal study 

over 0-20 years 

•Effects of dramatic weight gain/loss 

© 2008 Unisys Corporation. All rights reserved. Page 36 
Metric Wish # 7 

Sophisticated customer 

questions 

•To what extent can/cannot 2D-3D synthetic image creation of 

the [probe or gallery or watchlist] improve the matching results 

(1:n and 1:1) ? 

•What do the FRVT2002 graphs look like in 2010 ? 

•Where can I buy morphed Facial Image detection software ? 

© 2008 Unisys Corporation. All rights reserved. Page 37 
Face and Iris 

© 2008 Unisys Corporation. All rights reserved. Page 38 
Metric Wish # 8 

What image attributes (eg 

resolution) do you need to 

capture a facial image to 

zoom/crop out an iris 

acceptable for iris 

recognition ? 

© 2008 Unisys Corporation. All rights reserved. Page 39 
© 2008 Unisys Corporation. All rights reserved. Page 40
Metric Wish # 9 

Iris Global 

Implementations 

•What’s the largest ? 

•Have False Matches been detected ? 

•What’s the operational FAR ? 

© 2008 Unisys Corporation. All rights reserved. Page 41 
Metric Wish # 10 

Iris 

(still, motion, distance) 

•Results vs Iris Still 

•Interoperability ? 

•Relative verification performance results ? 

© 2008 Unisys Corporation. All rights reserved. Page 42 
Metric Wish # 11 

FTE: 

Finding the iris 

•Enrolling with/without glasses 

•Effect of glasses 

•Polarised lenses 

•Occlusion 

•Jetlag 

•Hangovers 

© 2008 Unisys Corporation. All rights reserved. Page 43 
Metric Wish # 12 

Legacy and Upgrade 

Conversion Times 

•Face per million 

•Finger per million 

•Iris per Million 

© 2008 Unisys Corporation. All rights reserved. Page 44 
Metric Wish # 13 

Liveness Detection 

•How does it actually work ? 

•Prove it works .. well to what extent ;) 

© 2008 Unisys Corporation. All rights reserved. Page 45 
© 2008 Unisys Corporation. All rights reserved. Page 46 
Finger 

• What is the “maximum” size of a large AFIS system (before 

it ceases to “work”) ? 

• Fingerprint quality and interoperability: is it possible to 

design a standard that is algorithm independent and good 

enough for large databases ? 

• What is the matching performance for ISO standard 

template matching in larger databases? Which vendors 

support this standard template (long/short variant) 

• Difference in FAR/FRR between 250/500/1000 dpi sensors 

for 1:N matches 

• Touch vs no-touch sensors 

© 2008 Unisys Corporation. All rights reserved. Page 47 
In the Vein of Vascular ... 

• Whats the largest global 

implementation ? 

• What veins? (palm, finger, back of 

hand...) 

• Effects of aging for vein recognition eg 

moving to surface 

• At what age do the vein patterns 

become stable enough to use – ie use 

for children/teenagers/adults 

© 2008 Unisys Corporation. All rights reserved. Page 48 
Metric Wish # 14 

Demystifying Vasular 

•Operational Results (FAR, FRR) ? 

•1:1 rates vs other biometrics ? 

•1:1 Results on children .. A key potential market 

•1..N ? 

© 2008 Unisys Corporation. All rights reserved. Page 49 
Voice 

• Comparative testing (FRVTlike) of various products = 

NIST SRE10 ? 

• 1:N ? 

© 2008 Unisys Corporation. All rights reserved. Page 50 
All - Multimodal & Fusion 

• Fusing face/fingerprint: what is the impact on performance 

in terms of speed and accuracy, what is the optimum and 

what to do to prevent weakening the system 

• The same question for face/iris and fingerprint/iris 

• 2 for the price of one ? (face/iris, finger/vein) 

• How fuse ? 

– 1:N for reducing search times in multimodal env 

– 1:1 for increasing confidence in the match 

© 2008 Unisys Corporation. All rights reserved. Page 51 
All - Template sizes 

• What is template size in for this vendor for a given raw 

image ? 

– At rest 

– In cache 

• Can be bigger than the raw image 

• Is growing and growing year-on-year and while storage is 

not an issue, it is an issue for mobile devices, transmission 

speeds etc 

• Multiple pass options (fusing your own matcher) 

© 2008 Unisys Corporation. All rights reserved. Page 52 
All - Spoofing 

• Fingerprint is targeted – eg BSI yesterday 

• What about Face ? 

• What about Iris ? 

• What about Vein (and those claims it can’t be spoofed) ? 

© 2008 Unisys Corporation. All rights reserved. Page 53 
Are tests missing the 

point? or 

Is it just that the 

answers are jealously 

© 2008 Unisys Corporation. All rights reserved. Page 54 guarded ? 
Please send all the answers to: 

terry.hartmann@unisys.com