International Biometric Performance Conference, March 2-4, 2010 

Scott Swann 

Unit Chief 

Services Evaluation and Analysis Unit 

1 
2 

Background: Moving 

from IAFIS to NGI 

• FBI Mission Evolution

– Changed since IAFIS went live in 1999

– Successful criminal, as well as, non criminal justice

missions

• Focus on Technology Advancements

– Improved biometric services

– Expanded support for new counterterrorism mission

– Continually added enhancements to the system

– Continually seeks technological advancements to

improve the system

• Changing Business Needs

– New legislation requires increased background

checks

– Increased noncriminal justice fingerprint

background checks

– Department of Homeland Security support
NGI Drivers for LargeScale Testing 

• Advanced Fingerprint Identification Technology (AFIT)

– Fewer missed identifications

– Faster response times

– Greater throughput

– Determine accuracy of latent fingerprint matching

– Slap fingerprint segmentation

• National Palm Print System (NPPS)

– Establish national palm print repository against which latent prints may

be searched

– Solidify the knowledge of the similarities and differences between

fingerprint and palm print matching

• Enhanced IAFIS Repository (EIR)

– Evaluate state-of-the-art as other modalities are integrated

– Interstate Photo System (IPS)

• Faces, Scars, Marks, and Tatoos

– Iris

– Multi-biometric fusion

• All of the above necessitate large-scale testing efforts 3
What is a Trade 

Study? 

• Trade Studies were intended to provide clarity through an

assessment of market capabilities resulting in high confidence

investment decisions

• Focused on evaluating the ability of biometric algorithms

provided by vendors to meet select NGI performance

requirements

– Accuracy, workload, capacity, response time, availability, and

scalability

• The onus to conduct NGI Trade Studies is on the NGI

integrator

• Trade Studies occur Just in Time (JIT) to support

development activities for a given biometric technology

4 
NGI Implementation 

Schedule 

5 
NIST Evaluation and 

Challenge Events 

Biometric Modality Evaluation / Challenge Year(s) 

Multi-Biometric 

Multiple Biometric 

Evaluation (MBE) 

Multiple Biometric Grand 

Challenge (MBGC) 

2009-201? 

2008-201? 

Evaluation of Latent 

Fingerprint Technology : 

Extended Feature Sets 2009-201? 

(ELFT-EFS) 

Evaluation of Latent 

Fingerprint Fingerprint Technology 

(ELFT) 

2007-2008 

Slap Fingerprint 

Segmentation Evaluation 

II (SlapsegII) 

2008-201? 

Minutiae Exchange II 

(MINEX II) 2007-201? 

Face 

Face Recognition Vendor 

Test (FRVT) 

Face Recognition Grand 

Challenge (FRGC) 

2006-2007 

2004-2006 

Iris Exchange (IREX) 2008-201? 

Iris Iris Challenge Evaluation 2005-2006 (ICE) 6 
Data Preparation 

• The diversity of large-scale testing drivers requires the

development of a variety of datasets which serve as the foundation

of testing

• As part of datasets development the following considerations

needed to be addressed

– Ensure no vendor bias exists in data

– Ensure sufficient quantities of data exist to support system

performance and accuracy projections

– Ensure adequate amount of data can be made publicly available

– Ensure adequate amount of data remains sequestered

– Ensure data is operationally representative

• Led to the recent preparation of multiple test beds

– Rolled and Flat Fingerprint - Platinum Data Repository (PDR)

– Latent Fingerprints and Palm Prints

– Facial Images

7 
Data Preparation for 

NGI Development 

• Rolled and Flat Fingerprints - Platinum Data Repository (PDR)

– FBI GFI to be provided to the NGI Integrator in support of the NGI Trade

Studies.

• Selection of the PDR based on an analysis of IAFIS transaction

metadata over a year long period

– All days between Nov. 11, 2007 and Dec. 31, 2008

– 45.9M samples, an average of 111K per Day

– Idents 10.5M, Nonidents 28.9M

– 21.8GB of ASCII data

– Rolled

• Idents 10.25M

• Nonidents 15.63M

– Flat

• Idents 309K

• Nonidents 13.37M

IAFIS 

Metadata 

Rolled 

Criminal 

High 

Quality 

Low 

Quality 

Civil 

High 

Quality 

Low 

Quality 

Flat 

Criminal 

High 

Quality 

Low 

Quality 

Civil 

High 

Quality 

Low 

Quality 

8 
Data Preparation for 

NGI Development 

• A subset of the PDR was used to support Identification Fingerprint

Search and Slap Segmentation testing

– IdFP search set (IdFP 600K) totaled 598,062 probes (299,810 rolled,

298,252 flat)

• The number of search probes with mates in the repository was 269,323 (149,297

rolled, 120,026 flat)

• Total of 289,880 true matches in the test set (150,843 rolled, 139,037 flat)

– Final IdFP repository size was 3,199,683 subjects.

• Including required mates from search set

• The IdFP repository contained 51.3% Criminal submissions and 48.7% Civil

– RISC search set (RISC 160K) totaled 159,446 probes (151,446 two

finger, 4,560 rolled tenprint and 3,440 flat tenprint).

• The number of search probes with mates in the repository was 83,210 (79,134

two finger, 2,284 rolled tenprint, 1,792 flat tenprint).

• Total of 100,384 true matches in the test set (95,855 two finger, 2311 rolled

tenprint, 2210 flat tenprint).

– Final RISC repository size was 3,199,666 subjects

• 69% rolled and 31% flat; 95% tenprint and 5% 2 print.

– Slap Seg 15,558 (7779 subjects * 2 slaps per subject) 2 inch and total of

32,625 (10,875 subjects * 3 slaps per subject) 3 inch

9 
Role of Human Capital in 

Data Preparation 

• Human capital investments for

data preparation and post-test

analysis

– Manual resources were applied

to substantially improve utility of

datasets

• Ground truthing for slap

segmentation tests

• Assistance in miss analysis

• Analysis of unexpected results

• Latent examiner expertise

• Ensure availability of support tools

10 
Source Name # Latent Images # Distinct Fingers # of Subjects Description 

Casework 1 372 372 272 Operational casework images 

Casework 2 165 165 163 Operational casework images 

WVU 446 446 383 Laboratory collected images 

FLDS 93 93 15 Laboratory collected images 

MLDS 38 38 4 Laboratory collected images 

Laboratory collected images 

Totals 1,114 1,114 837 (small set of publicly releasable 

images for examples in reports) 

Data Preparation for 

Latent Testing 

11 

– Baseline dataset collected from multiple sources

including both operational and laboratory

environments

– Largest latent fingerprint dataset with verified by

examiners (FBI and contractors) including

extended markup data
Data Preparation for 

Face Testing 

• Existing FBI data holdings 

– 7.1 million EBTS submissions 

– 10.8 million photos 

– Assessing JABS extract 

– Include scars, marks, and tattoos 

• Building a Face Test Set 

– NIST Special database of deceased

individuals 

– 2.4 million EBTS submissions 

– 3.5 million face images 

– Sorted by capture angle 

• ≈ 2.6M Frontal 

• ≈ 0.9M Profile 12 
Summary 

– Identification Fingerprint and Slap Segmentation Solution

– Lockheed Martin completed the independent trade study

– Lockheed Martin awarded a subcontract to the Alexandria, Virginia based

MorphoTrak

– Identification Fingerprint Search Capability

– Supports increased accuracy of fingerprint search results and provides

better support for processing flat and less than ten fingerprint

submissions

•Tenprint Performance Comparison •Electronic Tenprint Response Times

Performance Metric Current NGI 

IAFIS Objective 

(Increment 1) 

Tenprint True Match Rate 92% 99% 

Tenprint False Match Rate 1.15% 0.30% 

Processing Capacity (Daily 

Throughput) 

150K 299K 

Storage Capacity (Repository 

Size) 

100M 157M 

IAFIS Criminal Civil 

Response Time 

NGI 

2 hours 

Criminal 

24 hours 

Civil 

High 10 minutes 15 minutes

Routine 30 minutes 2 hours 

Low 24 hours 24 hours 

Non-Urgent 15 days 15 days 

•80% better accuracy at three times the overall

matching capacity at IOC scalability to accommodate 13

greater capacity at FOC
Summary 

• NGI latent fingerprint and palm print

technical evaluation is complete

• NIST ELFT-EFS preliminary report

delivered on 01.26.10

• NIST MBE Still Face Track currently

accepting SDKs

• NGI Face Trady Study approach and

planning underway

14