George W. Quinn

gw@nist.gov

9/23/2010

George W. Quinn

gw@nist.gov

9/23/2010

MBE Still Face Tra

MBE Still Face Tr

Large scale evaluation of 2D F

Recognition Technology
Executing Agency

Director of National Intelligence Intelligence Advanced Research Projects Activity

– Criminal Justice Info– Operational Technol

MBE Sponsors MBE Sponsors

Sponsoring Agencies
Why Face? Why Face?

 Large corpus of imagery available. In many cases it’s the only biometric avail Usability (can be run in ‘passive’ mode).  Fewer privacy concerns Multi-model Method of capture is cheap and widely av

 Large corpus of imagery available. In many cases it’s the only biometric avail Usability (can be run in ‘passive’ mode). 

 Fewer privacy concerns Multi-model Method of capture is cheap and widely av

Advantages

QuickTimeTM and a

 decompressor

are needed to see this picture.
Purpose of MBE S

Purpose of MBE S

WHAT Determine latest state-of-the-art 

of face recognition. The last NIST conducted FR evaluation was the FRVT 2006. 

 To report parameters important 

to procurers. These include 

 Matching accuracy Timing statistics Speed/accuracy trade-off Scalability Template size Computational requirements

WHAT Determine latest state-of-the-art 

of face recognition. The last NIST conducted FR evaluation was the FRVT 2006.

 To report parameters important 

to procurers. These include

 Matching accuracy Timing statistics Speed/accuracy trade-off Scalability Template size Computational requirements

WHY To provide 

governmenexpanding specifically 

 Testing, 

calibratio

 Standard

ANSI/NIS

 Recomm

WHY To provide 

governmenexpanding specifically 

 Testing, 

calibratio

 Standard

ANSI/NIS

 Recomm
Identification

 Duplicate detection (e.g. DMV) Forward criminal search

 Watch-list, covert surveillance Access-control without presentation of 

a credential or PIN.

 Disaster post-mortem Rapid subject identification

Verification

 Access Control (e.g. E-Passport, USVISIT, smart cards)

Applications Applications
Dept. of State (DOS):

5.7 million people.Highly controlled. Low resolution.Legacy (~10 years old)Provides a direct method of tracking the progress 

of FR technology over time.

Law Enforcement (LEO):1.8 million people.Controlled. Some pose / lighting variation.Contemporary.Operational (currently used by FBI).Sample LEO images available as NIST Special Database 32 (a.k.a http://biometrics.nist.gov/

Datasets Overvie
MBE Still Face Tra

Participation 

MBE Still Face Tra

Participation

 Cognitec (DE) L1 Identity (US) Morpho / Sagem (FR) NEC (JP) Pittsburg Pattern Recognition (US) Neurotechnology (LI) Toshiba (JP) University of Surrey (UK) Dalian Tech. Uni. (CN) Tsinghua Uni. (CN)
A rank-based identification systems returns the top K 

search. 

A rank-based identification systems returns the top K 

search.

Rank-based Identific

Rank-based Identific

Large Gallery (1.6 million)

Matcher

Who is this 

person? 

Who is this 

person?
CMC Curve 

(Rank-based Performanc

CMC Curve 

(Rank-based Performanc

0.90.80.70.61.0

1 5 2 10 100

Hit Rate

Rank

5020

 Rank 1 Hit  Rank 5 Hit  Diminishing 

increases.

Gallery: 1.6 mLEO images
A score-based identification systems returns only thos

score above some threshold. The result is a variable s

A score-based identification systems returns only thos

score above some threshold. The result is a variable s

Large Gallery (1.6 million)

Matcher

Who is this 

person? 

Who is this 

person?

Score-based Identific

Score-based Identific
0.8

0.60.41.0

0.0001 0.01 1 10

Reliability

Selectivity

Identification Perform

(score-based) 

Identification Perform

(score-based)

Best algorithm 

desired selec

Gallery: 1.6 mil

Best algorithm desired selec

Gallery: 1.6 mil
Verification Verification

Live Image

(from camera)

Enrolled Image

(from smart card)

v
Algorithm

FNMR

(at FMR=0.001)

Best algorithm

FNMR < 0.00

FRVT 2006 be

FNMR = 0.02

FRVT 2002 be

FNMR = 0.20 

Best algorithm

FNMR < 0.0

FRVT 2006 be

FNMR = 0.0

FRVT 2002 be

FNMR = 0.2

Verification Perform

Verification Perform

0.001 0.01 0.1 1
Sources of Erro

Sources of Erro

Non-frontal

pose

Non-neutral

expression

Bad

lighting
1 3 5 7 1 3 5 7 1 3 

Face Aging Face Aging

Years Between Captures

FNMR
 Revision to Report (NIST IR 7709) will include

 New algorithms Pose effect (class D) Memory Usage

 Follow-up compression study

 Relevant to situations where face images are store

limited devices such as smart cards (e.g. E-Passpocards). 

 Transferring images over limited bandwidth networ Guidelines for compressing face images. Handling compressed video.

 Revision to Report (NIST IR 7709) will include

 New algorithms Pose effect (class D) Memory Usage

 Follow-up compression study

 Relevant to situations where face images are store

limited devices such as smart cards (e.g. E-Passpocards).

 Transferring images over limited bandwidth networ Guidelines for compressing face images. Handling compressed video.

Upcoming in the MBE Still Tra

Upcoming in the MBE Still Tra
http://face.nist.gov/mbe

gw@nist.gov

patrick.grother@nist.gov

jonathon.phillips@nist.gov

Thanks for your Atte

Thanks for your Atte