Test Current 

Systems 

Establish 

Performance & 

Limitations 

Improved 

Latent 

Searches 

Technology 

Improvements 

Improved 

Watchlist 

Searches 

Figure 1 -- Overall Project Goals 

Concept of Operations (CONOPS) for 

Evaluation of Latent Fingerprint Technologies (ELFT) 

(Rev. D, 21 June 2007) 

1. Overv ew 

The National Institute of Standards and Technology (NIST) is conducting a series of 

tests for evaluating the state of the art in automated latent fingerprint matching. The 

intent of the testing is to quantify the core algorithmic capability of contemporary 

matchers. The testing will be conducted using software-only implementations, and 

utilizing NIST hardware. 

The umbrella project for the series of tests has been named Evaluation of Latent 

Fingerprint Technologies (ELFT). The scope and structure of these tests are based 

partly upon lessons learned from the April 2006 NIST Latent Fingerprint Testing 

Workshop, supplemented by technical interchanges with workshop participants and 

vendors. The initial round of tests was initiated in April 2007, and is on-going. 

The principal objective of this report is to provide a “snapshot” of the thinking, analysis, 

and planning that went into ELFT. The report is not intended to provide any test results 

or conclusions. These will be presented in subsequent reports. 

While the immediate goal of ELFT is to assess automated technology, the long-term 

goals go far beyond simply quantifying performance. It is fully expected that 

understanding the performance envelope and limitations of contemporary matchers will 

lead to improvements in technology. These in turn, will lead to enhanced performance 

for searches of ten-prints and plain impressions against unsolved latent 

databases/watchlists. Equally important, technology improvements will provide law 

enforcement the capability to search their unsolved latent fingerprints against ten-print 

files with greatly reduced effort. Figure 1 provides a schematic of the overall ELFT goals. 

1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Publish 

Concept of Operations 

(CONOPS) 

and Software API. 

Create and maintain 

Create/Compile 

Latent Test Sets 

Phases I and II Tests 

--Execute submitted 

Software (SDK) 

using NIST 

Analyze Data. 

Conduct Follow-up 

Workshop. 

Present results. 

Website. Computers and Test Discuss future 

Sets. Work. 

 Figure 2 – Schematic of ELFT07 

EFTS is structured as a multi-year project, and the full impact of this work may not be felt 

for several years. The first part of this project, ELFT07 (07 is the year), consists of two 

tests, run in a “lights-out” environment, followed by a workshop. The two tests have been 

termed Phase I and II. Phase I is a proof of concept test, whose main purpose is to 

demonstrate integrity of the software in a lights-out environment. During Phase I the 

software will demonstrate: a) automated feature extraction from latent images; b) the 

ability to match these features against enrolled 10-print backgrounds; and c) generation 

of candidate lists. Phase II will then employ a larger database to quantify the achievable 

performance (“hit rate”) for automated searches. A schematic diagram for ELFT07 is 

provided below. 

In subsequent years (2008+) we plan to expand the above tests in several ways. First, 

we plan to augment the ten-print databases with a mix of rolled and plain impressions 

(“flats”). These will enhance NIST’s and the latent community’s understanding of the 

challenges of matching latents against flats. Continuing this line of investigation, we will 

then transition to searches of plain impressions against databases of latent images 

(sometimes referred to as reverse searches). Initially these tests will be restricted to 

single-finger searches, and subsequently will be enlarged to multi-finger latent searches. 

Figure 3 provides a diagram of the augmented latent testing plan. 

2 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2007 2008 

Augmented Features/Advanced Matchers 

Testing 

Augmented 

Features 

(CDEFFS and other 

advanced matching) 

2009+ 

Addition Augmented 

Features 

Testing/Advanced 

Matchers 

Phases I and II Latents against Reverse Searches Testing (Latents mixed rolled and Large Background (Plain impressions 

against rolled flats (Phase IV – 2008) searched against ten-prints) (Phase III – 2008) latents) 

Core Testing 

Supplemental Single Latent Enhanced Resolution 

Testing Matched against 

Rolled + Plain 

(dual data) 

(Effect of higher 

resolution scans of 

latent and ten-print) 

Other “Special Test” 

(Multi-finger, types 

of latent prints, etc.) Figure 3 – Core and Augmented Testing Plan (Proposed) 

As shown in the figure, envisioned tests include: a) latents and mates scanned at 

enhanced resolution (1000 and 2000 ppi); b) latents lifted/developed/processed in 

diverse manner (i.e., how was the image actually produced from the latent); c) latents 

matched against latents; and d) searches employing new or non-traditional features 

(e.g., level-3 features). 

NIST is also looking into the development of latent image quality measures (LIQM). The 

principal function of the latent image quality measure is to provide a good indication of 

whether a latent is amenable to automated (“lights-out”) matching. Only latent prints of 

higher quality measure would be submitted for lights-out matching. The development of 

a suitable quality measure might require addition testing, not encompassed in the above 

diagram. 

We have utlined ab ve the full sc pe f this pr ject with a “br ad brush.” 

In the remainder f this d cument we f cus n this year’s p rti n f the pr ject, 

ELFT07. S as n t t break up the fl w, m re detailed discussi n f select t pics have 

been m ved t the end, Secti n 14. 

2. Who Should Part c pate n the Tests? 

Devel pers f latent fingerprint matcher s ftware systems are str ngly enc uraged t 

participate in ELFT07. In additi n, c mpanies, research rganizati ns, r universities 

that have devel ped mature pr t types, r stable research latent fingerprint 

matchers, are invited t participate. The latent fingerprint matching s ftware 

submitted need n t be a “pr ducti n” system, n r be c mmercially available. 

H wever, the system must, at a minimum, be a stable implementati n capable f 

being “wrapped” (f rmatted) in the Applica ion Programming In erface (API) 

specificati n that Na ional Ins i u e of S andards and Technology (NIST) has published1 

f r this evaluati n. An nym us participati n will n t be permitted. The Applicati n 

1 The API is available at http://fingerprint.nist.gov/latent/elft07/elft07_api.pdf 

3 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

f rm f r participati n is f und n the NIST website. The Applicati n f rm g es int 

further details regarding applicati n and qualificati n. 

3. Precedence of Documents 

It is intended that this Concep of Opera ions (CONOPS or ConOps) be the single m st 

c mprehensive d cument c vering La en Tes ing/ELFT c ncepts. It will be 

peri dically revised t reflect updates in NIST’s planning, and t inc rp rate vend r 

c mments and suggesti ns. H wever, this CONOPS is n t guaranteed t be the m st 

accurate f r highly technical data. In the event f c nflict with the Applica ion 

Programming In erface (API) r the Applica ion Form the latter tw will take 

precedence. These tw d cuments may be supplemented by additi nal reference 

d cuments in the future. 

4. Test Object ves – What w ll be Tested dur ng ELFT07? 

As previ usly indicated, the primary purp se f the Phase I and II testing is t quantify 

the c re alg rithmic capability f c ntemp rary matchers, in rder t understand their 

strengths and limitati ns. In the initial tests the emphasis is n matching latents again 

ten-prints. (In subsequent tests this will be expanded t include ther types f 

matching, f r example latents against plain-impressi ns, and latents against latents.) 

The testing will be c nducted using a s ftware- nly implementati n, in a lights- ut 

envir nment, and utilizing NIST hardware and datasets. During Phase I the s ftware 

will dem nstrate: a) aut mated feature extracti n fr m latent images; b) the ability 

t match these features against enr lled 10-print backgr unds; and c) generati n f 

candidate lists. 

While Phase I is primarily intended to be a proof of concept test, it will nevertheless 

provide a certain amount of performance statistics. Only aggregate statistics (combining 

all successful participants) will be published. Individual results will be disclosed only to 

the owner of the respective SDK. 

NIST will c mpute (and rep rt as an aggregate statistic) the number f “hits” in each 

p siti n (first, sec nd, third, up t 50). A specimen f such a chart is sh wn bel w, 

f ll wed by a specimen cumulative chart. 

4 
 

 

 

Number of True Mates ("Hits") as Function of 

Candidate List Position 

0.00% 

10.00% 

20.00% 

30.00% 

40.00% 

50.00% 

1 6 11 16 21 26 31 36 41 46 

List Position 

% of Maximum Possible "Hits" 

Figure 4 – Specimen Chart of “Hits” vs. Position 

 

 

 

Cumulative Number of "Hits" 

At or Above Indicated Position 

0.00% 

20.00% 

40.00% 

60.00% 

80.00% 

1 6 11 16 21 26 31 36 41 46 

List Position 

% of Maximum Possible

"Hits" 

Figure 5 – Specimen Cumulative Distribution 

 

 

 

 

 

NIST will als rep rt tw verall perf rmance metrics: 

1) Metric #1 simply c unt the fracti n f cases in which the c rrect mate appears 

in t p p siti n n the candidate list. This metric ign res all candidates bel w 

t p p siti n. This is basically the same as the first value in the ab ve graph, in 

this case is 0.4. The highest p ssible value is 1.0 and the l west is 0. 

5 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Probability of Matcher Score Exceeding Indicated Value 

for True Mates and Impostors 

(Illustrative data -- not any actual system) 

0 

0.2 

0.4 

0.6 

0.8 

1 

1 10 100 1000 10000 

Matcher Score 

Probability of

Exceeding Score

Impostors 

True Mates 

Figure 6 – Complement of Cumulative Distribution for True mates and Impostors 

 

 

 

 

 

 

2) Metric #2 gives partial credit f r mates appearing in l wer than t p p siti n. It 

assign 1.0 t a mate in t p p siti n, 0.5 t ne in sec nd p siti n; 0.3333 t 

third p siti n, etc. The final sc re is the sum f these sc res divided by the 

number f searches with mates. (F r the validati n data all searches have 

mates.) This sc re will always be at least as high as metric #1, and will 

generally be s mewhat higher. The highest p ssible sc re is 1.0 and the l west 

is 0. 

NIST will als use DET perf rmance metrics as a primary indicat r f ne-t -many 

identificati n search accuracy. This inv lves pl tting False Accep ance Ra e (FAR) and 

True Accep ance Ra e (TAR) f r all values f the thresh ld. (Equivalently, ne can use 

false rejecti n and false acceptance rates.) An example f this pr cedure, empl ying 

hyp thetical data, is sh wn in the f ll wing tw figures. 

Figure 6 provides the complement of the cumulative distribution (1 – Cum_value) for two 

cases: a) when the search is matched against the true mate, and b) when the search is 

matched against an imposter (FAR). (The fact that both curves go to zero reflects the 

fact that very high scores become increasingly rare, for true mates as well as imposters.) 

By eliminating the score between the two curves of Figure 6 one obtains the DET curve, 

as shown in the next figure. 

6 
0.8 

0.6 

0.4 

0.2 

0 

DET Curve 

TAR 

1 

0.00001 0.0001 0.001 0.01 0.1 1 

FAR 

 Figure 7 – Specimen DET Curve 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For Phase II we will employ a larger database so as to improve the statistics. The results 

will be published as aggregate statistics, as well as individual participant statistics (for 

those participants electing to continue to Phase II). 

NIST will als rep rt enr llment and search timing inf rmati n. Speed of execution, 

for both enrollment and latent search, is of secondary importance. However, in order to 

conduct these tests in a reasonable amount of time NIST must impose some limitations. 

These are covered in Section 11. In reporting timing measurements NIST will specify the 

exact hardware that the software was hosted on. NIST will in addition caveat timing 

measurements by noting that operational latent searching algorithms are likely to be 

implemented in more sophisticated hardware. 

NIST recognizes that latent searches pose many special problems. One of these is that 

“strong hits” may have widely different matcher scores (for example, depending upon the 

number of minutiae). This may place challenges on the DET approach. Largely for this 

reason we have recommended the inclusion of a normalized score that attempts to 

compensate for the matching score variations. See Sections 13 and 14 for further 

discussion. 

A comprehensive discussion of performance metrics and their definitions is found Patrick 

Grother, Ross Micheals, and P. Jonathon Phillips, Face Recognition Vendor Test 2002 

Performance Metrics, 31 March 2003. See also 

• http://ois.nist.gov/nistpubs/technipubs/recent/search.cfm?dbibid=13808 

5. Publ cat on of Part c pat on and Results 

NIST understands that this pr ject is entering a relatively unexpl red field, and many 

challenges lie ahead. F r this reas n we have structured ELFT07 t include tw 

phases. We c nsider Phase I t be a Proof of Concep Tes . This means that the 

primary bjective f Phase I will be t dem nstrate that the submitted SDK executes 

 n the Phase I data t c mpleti n, in a lights- ut envir nment, and pr duces a 

7 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

“meaningful” utput. A “meaningful” utput is basically an utput in the c rrect 

f rmat. 

The detailed results f Phase I will be discussed with the Participant n a “ ne- n- ne 

basis,” but will n t be published r submitted t ther g vernment agencies. The 

number, but n t the names, f participants wh attempted and c mpleted Phase I will 

be discl sed. H wever, in the (likely) event there are a significant number f 

participants in Phase I, NIST is c nsidering publishing the aggregate test results (under 

the premise that this may be a fair assessment f the state f the art). By “aggregate 

test results” we mean that results are “lumped,” and that n specific candidate list r 

participant-specific sc res will be menti ned. Participants will have the pti n t 

withdraw an nym usly f ll wing participati n in Phase I. (This means that their 

withdrawal will n t bec me a public ann uncement.) 

Participants wh elect t c ntinue t Phase II may resubmit their SDKs. These need 

n t be identical t th se f Phase I. F ll wing c mpleti n f Phase II testing the 

G vernment will c mbine all results int a Final Rep rt. The Evalua ion of La en 

Fingerprin Technologies Tes , Phase II Final Repor will c ntain, at a minimum, 

descriptive inf rmati n c ncerning ELFT07, descripti ns f each experiment, and 

aggregate test results. Sh uld individual participant’s sc res be published, NIST will 

exercise care that any implied rankings are well supp rted by the underlying statistics. 

(I.e., tw sc res will be c nsidered the same if the difference is n t statistically 

meaningful.) 

Participants will have an pp rtunity t review and c mment n the Final Rep rt. 

Participants’ c mments will be either inc rp rated int the main b dy f the rep rt (if 

it is decided NIST rep rted in err r) r published as an addendum. C mments will be 

attributed t the participant. After the release f the Phase II Final Rep rt, 

Participants may decide t use the results f r their wn purp ses. Such results shall be 

acc mpanied by the f ll wing phrase: “Results sh wn fr m the Evaluati n f Latent 

Fingerprint Techn l gies Test (ELFT07) d n t c nstitute end rsement f any 

particular system by the U. S. G vernment.” Such results shall als be acc mpanied by 

the Internet address (URL) f the ELFT07 Final Rep rt n the ELFT07 website. 

F r Phase III and bey nd NIST intends t publish statements f the perf rmance f all 

implementati ns submitted f r testing. These will include measurements f 

identificati n err r rates and thr ughput. These results will be attributed t 

participants. Acc rdingly, NIST will require an appr priately signed applicati n f rm 

fr m all participants and NIST will n t evaluate any implementati n unless the 

participant c nsents t the discl sure f its perf rmance. The NIST tests use 

sequestered images. These will n t be pr vided t participants. 

6. Protect on of Part c pant’s Software 

NIST rec gnizes the pr prietary nature f the participant’s s ftware and will take all 

reas nable steps t pr tect this. The s ftware submitted will be in an executable 

library (SDK) f rmat, and n alg rithmic details need be supplied. NIST will use the 

Participants SDK s ftware nly f r the agreed-up n testing, and in the event err rs are 

subsequently f und, t rerun pri r tests and res lve th se err rs. NIST agrees n t t 

use the Participants s ftware f r purp ses ther than indicated ab ve, with ut express 

permissi n by the Participant. 

8 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

7. Why “L ghts-out”? 

The term “lights-out,” as used in this document, will indicate that no human assistance 

will be required in conducting the latent searches. In particular, all feature extraction 

steps, both for the enrolled images and for the latent images, must be performed entirely 

by the SDK under test. 

There are good reasons why NIST selected the “lights-out” mode of testing: 

1. It decouples the skill of the human expert from the intrinsic merits of the software. 

2. It protects the privacy of the test data by keeping the data in house, and not 

requiring examination by non-government personnel. This mode of testing allows 

the use of Sensitive but Unclassified test data. 

3. It encourages a forward-looking view of how latent searches might be done in the 

near future. It is anticipated this broader outlook will lead to technical innovations. 

Algorithmically speaking, “lights out” consists of two separate concepts. The first is 

automated feature extraction (and of course matching). The second is candidate list 

reduction. NIST envisages that in the near future automated search capabilities will 

assist latent experts by reducing the size of candidate lists that they need to examine by 

eliminating the more obvious “nuisance” non-matches. We refer to this part of the 

automated matching as candidate list reduction. To achieve effective candidate list 

reduction may require additional computer processing, including the development of new 

algorithms. 

Although this is a “lights-out” test NIST will use some human assistance in the data 

preparation phase. Any such assistance will be provided indirectly by NIST, and might 

include a) cropping and/or re-orienting of selected latent images, and b) specifying a 

region-of-interest in the from of a mask. The mask will be a byte image conformal with 

the size of the latent image. Initially only two values will be used for each pixel, 0 and 

255. A zero value will indicate “do not use this pixel,” while 255 will indicate a “good” 

pixel. In the future these two values may be augmented by other values to indicate finer 

gradations of quality. NIST will also involve latent experts for examining potential 

consolidations and for resolving contested or unclear results. 

8. Test Data 

NIST will select the test datasets fr m its internal s urces. The Test Datasets are 

pr tected under the Privacy Act (5 U.S.C. 552a), and will be treated as Sensiti e but 

Unclassified and/ r Law Enforcement Sensiti e. ELFT07 Participants will have n 

access t ELFT07 Test Data, neither bef re, during, r after the test, with the 

excepti n f the small Validati n Dataset. 

8.1 ELFT07 Datasets 

The Valida ion Da ase is a very small data set, c mprised f ten search latents and 

100 backgr und ten-prints. The purp se f this dataset is t dem nstrate stable 

s ftware, c mpliant with the API. Up n receiving the applicant’s SDK and Validati n 

Dataset results, NIST will rerun the applicant’s s ftware using the Validati n Dataset. 

9 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

F r the applicant t be fficially accepted (and designated a Par icipan ) NIST must be 

able t repr duce the submitted results. 

In the event f disagreement in the tw utputs, r ther difficulties, the Participant 

will be n tified. Participants will be n tified with a detailed descripti n f the 

pr blem(s) and given reas nable pp rtunity t resubmit. 

B th Phase I and Phase II Test Datasets will c nsist f latent images f r searches and 

ten-prints f r the backgr und (“gallery”). F r b th tests, but especially f r Phase I, 

NIST will attempt t make the test set as “benign” as reas nable. This means NIST will 

attempt t av id including kn wn pr blem cases such as: 1) very small latent area; 2) 

extremely busy r therwise difficult backgr und; 3) highly blurred image; 4) multiple 

fingerprint impressi ns; r 5) upside d wn r mirr r images. 

Latent images will be supplied unc mpressed, and will have been scanned at either 

500 ppi r 1000 ppi. The participant sh uld be prepared t handle either res luti n. 

Additi nal image characteristics may be f und in the API. 

Backgr und (“gallery”) data will c nsist f r lled ten-print impressi ns, scanned at 500 

ppi, and presented in a dec mpressed f rm. 

The n minal sizes f the Phase I and Phase II Datasets are as f ll ws: 

ELFT07 Dataset Character st cs 

Dataset Number of Latent Searches Number of Ten-pr nts 

Validati n 10 100 

Phase I 100 1000 

Phase II 1000 10,000 

8.2 S ze of Images 

F r ELFT07 the f ll wing guidelines apply: 

• All mates (backgr und) will be r lled impressi ns. There will be n “flats.” 

• In all cases r lled impressi ns will n t exceed 1000 x 1000 

• In all cases latent images will n t exceed 2000 x 2000, th ugh ften will be 

significantly smaller. 

• Minimum dimensi ns f r latent images under 300 are p ssible, but never 

smaller than 150. 

Over the entire series f planned tests the size f test images may vary c nsiderably. 

F r example, images scanned at 2000 ppi will c ntribute s me very large sizes, 

p tentially as large as 4000 x 4000 pixels. H wever, these very large sizes will nly 

appear in the “d wnstream” tests. 

9. Test ng Platform 

NIST will host the participant’s software (SDK) on a high-end PC (workstation/server 

type). Although these PCs include of a mix of models, a “typical” PC will have the 

equivalent of a Pentium 4, 2.8 GHz processor, or higher; 2 GB of memory; and at least 

10 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

extract_image_data() 

Get the ten rolled 

fingerprint images 

free_image_data() 

De-allocate memory 

for fingerprint images 

NIST-supplied 

Library 

F i 

enroll_background() 

Enrolls the entire set 

of background images 

set_background() 

Selects background 

latents will be 

searched against 

enroll_latent() 

Enrolls the latent 

which is searched 

against background 

image_search() 

Searches background 

for potential mates to 

latent 

Participant’s SDK 

Executable Latent 

Search Module 

Figure 8 – Executable Software is a Combination of Participant-supplied (SDK) and NISTsupplied Software 

 

 

 

 

 

 

 

 

 

 

 

50 GB of disk memory. The participant software must be able to reside and execute on 

this single PC. NIST, at its discretion, must be able to copy the software to several PCs 

to expedite or scale-up the testing. These computers are configured with either a 

Windows 2000 or Linux operating system. 

10. Format of Part c pant Software 

The software undergoing testing will be hosted on NIST-supplied computers. The 

executable modules will be built up from two sources: 1) participant-supplied software 

provided in the form of a Software Development Kit (SDK), and 2) NIST-supplied 

software. The core of the executable module is of course derived from the SDK. The part 

supplied by NIST is mainly concerned with the image retrieval and manipulation. 

The participant may supply their SKD in a Windows 2000 or Linux implementations. The 

Application Programming Interface (API) specification provides further details regarding 

the format in which participants must submit their software. See •

http://fingerprint.nist.gov/latent/elft07.html 

Execution occurs in two passes: 

1. Enrollment of Background Pass: The first pass performs feature extraction of the 

background fingerprints. This process converts images into proprietary “feature” 

representations. The output of this enrollment is at the discretion of the participant, 

except that all extracted data must be written into a single directory. The background will 

consist entirely of rolled impressions of ten fingerprints during Phases I and II, and will 

be augmented by plain impressions (“flats”) during some of the subsequent phases. 

The following diagram summarizes the background feature extraction pass. 

11 
 

 

 

 

 

Background 

Images 

Background 

Features enroll_background() 

free_image_data() 

End 

Start 

Done all 

Background? 

extract_image_data() 

No 

Yes 

Figure 9 – Logic and Data Flow for Executable Enroll_Background 

Module 

 

 

 

 

 

 

 

2. Matching Pass: During this pass each latent search image is matched against the 

enrolled data and a candidate list is generated. The details of the searching process are 

unregulated by the API. In particular, participants may invoke multi-stage algorithms 

within the top level search function call by NIST. 

Prior to the actual search, but part of pass two, is the enrollment of the latent image. The 

details of the latent enrollment process depend whether a latent Region of Interest (ROI) 

Mask is specified or not, as shown by the following figure. 

12 
 

 

 

 

 

 

 

 

End 

Start 

ROI 

specified 

? 

No 

Latent 

Images Latent 

Features 

Extract Features 

Using ROI 

Yes 

ROI Mask 

(Image) 

Extract Features 

Without Assistance 

of ROI 

enroll_latent(*) 

Figure10 – Logic and Data Flow for enroll_latent() Function 

The following diagram summarizes the logic and data flow during the matching pass. 

13 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Start 

End 

Yes 

Latent 

Images 

Latent 

Features enroll_Latent() 

set_background() Background 

Features 

image_search() 

Candidate 

Lists (one 

per search) 

Done all 

Latents? 

No 

NIST-controlled loop 

Figure 11 – Logic and Data Flow for Executable Search Module 

Participants will be supplied a Validation Dataset to test their software before submitting 

it to NIST. Participants must submit candidate lists from the Validation Dataset to NIST. 

Once NIST has received participant implementations it will rerun the test and check its 

candidate lists against those submitted. If a difference is detected, or if other problems 

are encountered, the participant will be required to respond to NIST requests for updated 

implementations. 

NIST may provide a latent image with an accompanying region of interest, in the form of 

a “mask.” This mask will be a byte image conformal with the size of the latent image. 

Initially only two values will be assigned to each mask pixel, 0 and 255. A zero value will 

indicate “do not use this pixel,” while 255 will indicate a “good” pixel. In the future these 

two values may be augmented by additional values to indicate finer gradations of quality. 

The participant’s software should default to “no mask option” when no mask is supplied 

by NIST. The participant may use this information as they see fit. NIST plans to run the 

test with and without masks, but employing the same search and background. It is 

expected that searches with a mask will generally produce superior results. Quantifying 

the amount of improvement is a test objective. 

14 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

NIST will also involve latent experts for examining potential consolidations and for 

resolving contested or unclear results. 

11. Timing Requirements 

For ELFT07 NIST will allow the simultaneous submission of two SDKs, a primary and 

secondary. It is envisioned that the first (primary) SDK would be a “slow but accurate” 

version of the software, while the second would be a “fast and but less accurate.” 

(However, these are suggestions, and need not be followed. The participant may submit 

any two SDKs, subject to the timing restrictions below. The participant need not supply 

any details regarding the differences between the two SDKs which might reveal 

proprietary information. ) 

Execution time is a concern. While we plan to allow “generous” execution time, we 

request that participants remain within the following limits: 

Phases I and & II Execution Time Limits 

Primary SDK 

Background 

Enrollment Total Time (seconds) ≤ 150 * (number of ten-prints in background) 

Latent Enroll 350 seconds per image 

Search Time per Search (seconds) ≤ 0.2* (number of ten-prints in background) 

Secondary SDK 

Background 

Enrollment Total Time (seconds) ≤ 80 * (number of ten-prints in background) 

Latent Enroll 250 seconds per image 

Search Time per Search (seconds) ≤ 0.1* (number of ten-prints in background) 

(Comments and questions should be submitted to the evaluation team at 

latent@nist.gov.) 

12. Test Schedule 

NIST has created a website to serve as the central repository for all information 

regarding NIST's latent fingerprint activity. The URL is • http://fingerprint.nist.gov/latent. 

It links to the ELFT-07 testing • homepage which contains: 

1. Application form for ELFT07 

2. API specifications for participants' implementations 

3. This document (CONOPS) 

4. Summary of NIST Latent Fingerprint Testing Workshop, and 

5. FAQ 

The schedule is tabulated below. The later entries are subject to certain contingencies. 

However, NIST intends to adhere to the schedule as closely as possible. 

15 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

ELFT07 Master Schedule 

Action Actor Date 

Announcement of Latent Evaluation NIST 2006-11-17 

Publication of draft API and Latent Evaluation Concept Paper, and 

solicitation of comments 

NIST 2006-11-17 

Deadline for submission of comments on API and CONOPS to NIST. 

These should be submitted to latent@nist.gov 

Participants 2006-12-22 

Publication of API NIST 2007-01-26 

Publication of Latent Evaluation Application Form NIST 2007-01-26 

Deadline for submission to NIST of Latent Evaluation Application Form Participants 2007-02-23 

Publication of Validation Data NIST 2007-03-02 

Submission of SDK and Validation Dataset results Participants 2007-04-13 

Completion of SDK conformance testing and integration NIST 2007-06-08 

Commencement of Phase I Test (NIST) NIST 2007-06-11 

Completion of Phase I Test NIST 2007-07-13 

Communication of Phase I Results to Participants 2007-07-20 

Announcement of Phase II Test Schedule NIST 2007-07-27 

Submission of Phase II SDK 

(Withdrawal after this time will be reported in NIST publications.) 

Participants T 

Completion of Phase II Test NIST T+6 months 

Publication of formal evaluation report NIST T+6 months 

Workshop to summarize Phase II and to plan subsequent phases NIST T+6 months 

13. Format of Cand date L st 

The utput candidate list sh uld have a fixed length f fifty (50) candidates. We have 

selected this size because it is sh rt en ugh t be c nvenient, yet l ng en ugh t give 

an indicati n f the number f “hits just ut f reach.” (We currently d n’t envisi n 

cases in which the backgr und is less than 50 fingers. Sh uld this situati n arise, the 

candidate list c uld be suitably “padded.”) 

The candidate list c nsists f tw parts, a required and an pti nal part. The required 

part c nsists f: 1) the index f the mating ten-print subject; 2) the matching finger 

number; 3) the abs lute matching sc re; and 4) an estimate f the pr bability f a 

match (0 t 100, see als Secti n 14). The pti nal part c nsists f: 5) the number f 

g d minutiae identified in the latent; 6) the number f latent minutiae which were 

16 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

successfully matched; 7) the quality estimate f the latent (0 t 100, 100 is best); and 

8) the quality estimate f the mate (0 t 100, 100 is best). The API pr vides further 

guidelines regarding the meaning f quality sc res. The candidate list is rdered based 

up n the abs lute sc re, highest sc re in first p siti n. 

Sample Candidate List 

Required Part Optional Part 

Mate 

Index 

Finger 

No. 

Abs. 

Score 

Prob. 

Of 

True 

Match 

No. 

Latent 

Minutiae 

Matched 

Minutiae 

Quality 

Latent 

Quality 

Mate 

1 731 2 3513 93 18 12 70 80 

2 103 1 605 5 18 5 70 60 

3 207 7 513 4 18 5 70 90 

● 18 70 

● 18 70 

● 18 70 

50 107 9 422 3 18 4 70 70 

The parameter Probabili y of True Ma ch in the ab ve table represents an initial step 

t ward candida e lis reduc ion. It is intended t be an estimate f the pr bability 

that the candidate is a true match. Its values range fr m 0 t 100. F r the Phase I test 

candidate list reducti n is n t required, and the participant c uld always enter 100 

int this filed, if s desired. F r Phase II s me type f candidate list reducti n is 

preferred. 

14. Supplemental Notes 

14.1 Supplemental Notes to Section 1.0 -- Concepts of Operation for “Improved 

Watchlist Searches” 

A major goal of this project is to improve searches of watchlist/lookout-lists. It is 

becoming increasingly common to capture live fingerprints of arriving passengers at 

ports of entry and similar venues. The capture fingerprints are then compared to 

fingerprints of the person on file (one-to-one comparison, or validation match); they may 

also be searched against selected watchlists one-to-many search). The diagram below 

shows capture of the two index fingerprints at a port of entry. (This will be supplanted by 

the capture of all ten fingerprints in the near future.) 

F gure 12 –Capture of L vescan F ngerpr nts at a Port of Entry 

17 
 

 

 

 

 

 

 

Match against 

Rolled Tenprint 

Newly 

Captured 

Livescan 

Match against 

Plainimpression 

Match against 

Latent Image 

In watchlist/lookout applications plain-impression fingerprint are acquired onsite (e.g., Port of Entry) 

in real-time, and using livescan equipment. These plain-impressions may then be matched against 

any or all of three types of fingerprints, as shown. Ideally the computer matching should occur in 

real time. 

Figure 13 – Plain-impression may be matched against any of three types of 

fingerprints 

 

 

 

 

 

 

 

 

 

 

 

 

 

The newly acquired fingerprints may then be matched against any r all f three types 

 f finger prints, as sh wn bel w. Verification, or one-to-one searches, are generally 

performed by matching with plain impression taken at a previous time. Watchlists or 

Lookout-lists contain the fingerprints of prior offenders, or persons of interest. They may 

be comprised of any of the types of fingerprints, though latent fingerprints are generally 

kept in separate files, exclusively dedicated to latents. 

The simplest and clearest example of the applicability ELFT is to watchlists comprised of 

latents. However, there are several other ways in which ELFT may contribute to the 

point-of-entry scenario: 

Low-quality livescan images provide many of the same challenges as do latents. 

Improvements in latent matching should therefore transfer to real-time livescan 

matching. (Livescan images they are subject to “retake,” but the number of 

retakes is necessarily very limited, because of the need to expedite the 

processing.) 

18 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

To provide searches of watchlists in near-real-time, substantial algorithmic 

improvements are required. The multi-stage matching approach used by some 

latent matchers may offer a solution. 

For increased search accuracy, additional features (e.g., level 3) might be 

required. A goal of ELFT to examine the performance increases provided by 

selected new features. 

14.2 Supplemental Notes to Section 1.0 -- Concepts of Operation for “Improved 

Criminal Latent Searches” 

A second major goal of ELFT is to provide “an automated latent search capability” to 

latent examiners. By this we mean that latent examiners should have the capability of 

screening their latent images with a minimum of effort. We use the term screening to 

emphasize that such searches are not fully equivalent to traditional searches. Figures 

12a and 12b provide somewhat idealized schematics of how such automated screening 

might work. 

19 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

• -1-Collect “Cold Cases” 

Latent Fingerprints 

• -2-Convert Latent 

Images to Electronic 

Format (as required) 

• -3 -Submit to Automated 

Latent Fingerprint Image 

Quality Evaluation 

(similar to NFIQ) 

• -4-Sort Latent Images 

Based on 

Fingerprint Quality 

• Quality Adequate • Quality Inadequate for Automated Search for Automated Search 

• -5- Collect and “Bundle” 

Adequate Quality Prints 

Figure 12a – Automated Sort (“Triage”) of Latent Fingerprints 

based upon Suitability for Automated Searching 

20 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

• Quality Inadequate 

for automated Search 

--do not use! 

• -9- Output 

Reduced 

(Short) 

Candidate List 

• - 6- Retrieve 

Collected “Bundled” 

Latent Prints of 

Adequate quality 

• -8-Perform 

Automated Search 

• -7- Submit Prints  for Automated Search 

No Candidate 

Found 

Subject = M731 

1 Candidate Found: 

Finger = 2 

Score = 3513 

Figure 12b --Automated Search of Higher Quality Prints 

with Candidate List Reduction 

Following such a screening search the latent examiner may single out specific latents 

images for more traditional (and labor-intensive) searches. 

The examiner can assist the search process by defining a region of interest (ROI) on the 

latent image. The examiner may also single out certain images for a more traditional 

search by improving the features extracted, starting either with the machine extracted 

features, or starting with a “clean slate.” 

14.3 Features for Use in Matching 

Generally speaking, the selection of the features for use in the matching process is left 

to the participant. Matchers need not primarily be a “minutiae matcher.” Matcher 

21 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

architectures in which “advanced matchers” are selectively invoked depending upon 

initial results are allowed. For example the matcher might initially use certain core 

features in comparing the search (probe) with a background (gallery) subject. The result 

of this comparison might produce one of three possible outcomes: 1) the two fingerprints 

are too different, and no further effort should be expended on this candidate; 2) the two 

are so similar that this is definitely a mate; or 3) the two have points of similarity, but the 

match is not conclusive. The third case might then trigger a call to an “advanced 

matcher” for further resolution. 

If using “advanced matchers”, it is up to the participant to decide if the additional features 

(if any) required are to be extracted and stored on disk memory during the enrollment 

phase. Since it may not be possible to keep all gallery images in memory, it might be 

necessary for the software to retrieve the data from disk during searches. This extra 

fetch time will be included in execution time measurements. 

Figure 3 shows that approximately a year-and-a-half downstream NIST intends to test 

the effect of using augmented feature sets. These will be largely based upon the 

CDEFFS feature sets, but are not necessarily limited to these. For a description of the 

proposed CDEFFS features please go to the following website 

• http://fingerprint.nist.gov/standard/cdeffs/index.html 

These augmented feature tests test will be run in a dual mode: a) first without employing 

any new feature, then b) employing designated new features. To allow this mode of 

operation a somewhat different format might be required for the SDK. 

14.4 Supplemental Notes to Sections 7 and 13 -- Candidate List Reduction 

NIST envisages that in the near future automated search capabilities will assist latent 

experts by reducing the size of candidate lists they need to examine through elimination 

of the more obvious “nuisance” non-matches (impostors). For example, assuming that 

one hundred latents are submitted for searches, and that each search produces a 

candidate list of twenty candidates, an examiner needs to look at 2000 candidates. 

Since a typical identification rate for latent searches might be around 4%, this means 

that 2000 candidates need to be examined to find the four true identifications. While it is 

true that skilled examiners can quickly dismiss “nuisance candidates,” nevertheless it 

does take up valuable examiner time. An even larger concern is that too many nuisance 

candidates might result in the true mates being overlooked due to operator fatigue. It is 

therefore desirable to minimize these nuisance candidates. We refer to this part of the 

automated matching process as candidate list reduction. To achieve effective candidate 

list reduction may require additional computer processing, including the development of 

new algorithms. 

Since candidate list reduction poses many challenges, we plan to implement it in stages. 

The initial stage is to introduce a new parameter called Probability of True Match. This is 

intended to give a numerical estimate that the candidate is a true mate of the latent. This 

parameter should be supplied as a number between zero and 100. The number 100 will 

be interpreted as an extremely high confidence “hit.” The intent is to use this parameter 

as a key to candidate list reduction. 

Certainly the raw matcher score by itself provides a strong clue regarding the merit of a 

given candidate. However, by itself it is insufficient. For one thing, there is no agreed22 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

upon standard for the range of matcher scores: Does a value of 5000 indicate a high 

score? A very high score? Secondly, whether a given score belongs to a true mate 

depends upon the size of the background. The larger the background the more likely it is 

that large impostor scores will be created. The Probability of True Match therefore needs 

to take matcher score and background size into account. Additional information might 

also be factored in, such as: a) the score gap to the next candidate; b) the quality of the 

latent; and c) the quality of the mate. 

There does not appear to be any simple way of computing Probability of True Match, 

and participants are encouraged to develop their own procedures. Purely as an 

illustrative example, we offer the following procedure. 

Assume a background size of N. Assume additionally that some candidate has achieved 

a score of S. To fix our ideas assume S = 5000. Assume further that data such as shown 

in Figure 6 is available. Then from this figure we obtain TAR = 0.3 and FAR = .00001. 

This may be interpreted as: the a priori probability of obtaining a score exceeding 5000 

is 0.3 when matching against the true mate. Conversely, the a priori probability of 

exceeding 5000 when matching against an imposter is .00001. Note that this is for a 

single imposter. The probability that one or more imposters exceed 5000 in a 

background of N, assuming independence in match scores, is 1 – (1 – FAR)N . If this is 

taken as the a priori probability of an imposter exceeding 5000, then we can renormalize 

in the Bayesian sense to obtain the probability that the candidate is a true mate. The 

result is shown in the following figure 

Sample Calculation P("hit") 

(Illustrative data) 

0 

0.2 

0.4 

0.6 

0.8 

1 

1.2 

0 20000 40000 60000 80000 100000 120000 

Size Background (fingers) 

P("hit") 

Figure 13 – Illustrative Calculation of P(“hit”) (See text) 

The fact that the pr bability diminished rather rapidly as a functi n f backgr und 

size is a result f the independence assumpti n. It is p ssible that in a real system 

imp st r sc res are n t independent. One f the sec ndary bjectives f ELFT will be 

t btain m re insight int this pr blem. 

23