ELFT‐EFS  

NIST Evaluation of Latent Fingerprint Technologies: Extended Feature Sets

Public Challenge Problem Test Plan

23 April 2009

1 Overview

ELFT‐EFS is an evaluation of automated latent fingerprint matching software. The purpose of

this evaluation is to determine the effectiveness of human latent examiner‐marked fingerprint

features on latent fingerprint search accuracy, specifically with respect to the comparative

accuracy of image‐only searches, image+minutiae searches, and image+extended feature

searches.

The ELFT‐EFS Public Challenge is a practice evaluation: an open‐book test on public data to

validate formats and protocols. The results are not for substantive analysis, and participants will

remain anonymous in their results. The public challenge will be conducted through 15 June 2009.

The ELFT‐EFS Public Challenge instructions and preliminary drafts of all datasets will be made

available in mid‐April and can be used for internal work and testing.  

Note: as of 23 April, additional/changed markup files are continuing to be received, so the 

datasets for the public challenge will be finalized shortly. 

A workshop will be held in late June 2009 to discuss results and lessons learned from the ELFT‐

EFS Public Challenge.  

NIST will conduct the ELFT‐EFS 1st Evaluation (planned to run July‐September 2009), using

participants’ software on NIST hardware at NIST facilities. Datasets will be from multiple

sequestered sources, each broadly representative of casework. The ELFT‐EFS 1st evaluation will

be run specifically to identify any near‐term benefits, NOT to identify long‐term

feasibility/accuracy. The ELFT‐EFS 1st Evaluation timing constraints, subtests, and analysis will

be based in part on the results and lessons learned from the ELFT‐EFS Public Challenge.

ELFT‐EFS Subsequent Evaluations will be conducted to identify long‐term feasibility and

respond to lessons learned.

2 Challenge problem

The challenge problem will be conducted at the participants’ facilities, using the public challenge

data, with self‐reported results.

The challenge problem will involve 1:N searches using latent 1000ppi images provided with

human markup of CDEFFS features. Each latent search will result in a list of candidates, with

scores, across all exemplars in the subtest, including all fingerprint sets for each individual and

all finger positions. Normalized/probability scores shall be provided in addition.

The challenge is composed of the following subtests. Participants are requested to do all 20

combinations (e.g. L1E1 .. L5E4), but may choose to do only some combinations.

 Latent Subtests

o L1 – image only

o L2 – image with EFTS‐LFFS features ( fields 9.014‐9.023)

o L3 – image with EFS features (fields 9.300‐9.373)
o L4 ‐ EFS features alone

o L5 ‐ EFTS‐LFFS features alone

 Exemplar subtests

o E1 ‐ 1000ppi rolled exemplars

o E2 ‐ 500ppi rolled exemplars

o E3 ‐ 1000ppi plain exemplars (unsegmented slaps)

o E4 ‐ 500ppi plain exemplars (unsegmented slaps)

Analysis for the challenge will compare the results for the different subtests, compare inter‐

examiner variation (based on different markups of the same image), and compare the effects of

multiple fingerprint sets in the gallery.

3 Data

The ELFT‐EFS Public Challenge dataset is a dataset of latent images and corresponding

exemplars. This dataset was collected from the same initial source as the Universal Latent

Workstation GroundTruth or NIST SD27 datasets, but is neither a subset nor superset of those.

3.1 Public Challenge Latent Dataset

This dataset contains 255 latent images from 214 subjects (distinct individuals). 173 subjects have

one latent per subject; 41 subjects have two latents per subject.

The latent fingerprints were collected from case work in the mid‐1990s and captured as

photographic images. The physical photographs were rescanned in 2008,1 resulting in these

1000ppi images.

Each latent image is provided with multiple markups to show inter‐examiner variation. The

majority of the images were marked up three times by certified latent examiners:  

 by two examiners, each working alone;

 subsequently by a  ʺjuryʺ  team of two other examiners based on a review of the

individual markups.

Note that the feature markups were based solely on analysis of the latent image, as compared

with the ULW GT/SD27 “Ideal” markup, which used both the latent and exemplar images to

create a best‐case feature markup. These feature markups therefore may be seen as more

representative than the Ideal markup, but are also likely to be less accurate.

Feature markup in each file is saved as Extended Feature Set (EFS) fields, (fields 9.300‐9.373) and

as EFTS‐LFFS features (fields 9.014‐9.023, compliant with FBI EFTS 7.1). The EFTS‐LFFS features

were automatically converted from the EFS features, which is appropriate since EFS is a superset

of EFTS‐LFFS.

The Good/Bad/Ugly quality designation from ULW GT/SD27 is retained in these files and has not

been changed.

                                                     

1 The latents were scanned at 2000ppi, 16‐bpp grayscale and downsampled to 1000ppi, 8‐bpp grayscale.
3.2 Public Challenge Exemplar Dataset

Corresponding (mated) exemplars

202 of the 214 subjects have rolled and plain (slap) exemplars available as 1000ppi images of

inked paper cards. The slap images are not segmented into separate fingers. Each of these

1000ppi exemplar images is also included as a 500ppi image.

111 of the subjects have more than one exemplar set per subject (see the README file for details).

The multiple exemplar sets are only available as 500ppi images, include both rolled and slap

images, and include a mix of inked paper and livescan originals.

Background (unmated) exemplars

This dataset includes an additional 214 subjects for use as background. The same images were

rescanned for the 500ppi and 1000ppi datasets.

3.3 Numbering

Note that the mated exemplars are numbered by subject number, while the latents retain the

same latent image numbers used in ULW GT/SD27. A tab‐delimited crossreference file is

attached (EFS‐PC_Xref.txt) to indicate how the latents correspond to subjects and finger

positions. In those cases in which there is one latent/subject, the subject and latent image numbers

are the same.

3.4 Filename format

Latent files:

Two example filenames are L002G_JURIED_NP.lffs and L300U_M.lffs.

The prefix (L002G or L300U) has three parts: an indicator that the image is a Latent (L); the latent

image number (002 or 300); and a quality classification (Good/Bad/Ugly).

The suffix indicates who performed the markup (jury team of examiners ʺNʺ and ʺPʺ, or examiner

ʺMʺ).

Exemplar Files:

Example filenames are E002.an2, E005A.an2, and B001.an2.

The E indicates the file contains a set of mate exemplars, and the B indicates the file contains a set

of non‐mate (background) exemplars. The number (002 or 005) is the subject number.  In cases in

which there is more than one mate exemplar set per subject, the sets after the first are suffixed

with letters A, B, etc.

4 Format of results

4.1 Candidate Lists

All searches shall return a candidate list. A candidate list has a fixed length of one hundred (100)

candidates. Note that a given search may be associated with zero, one, or more subjects in the

gallery, and the candidate list shall include all of them.

The candidate list consists of two parts, a required and an optional part.  

The required part consists of:  
 the index of the mating exemplar subject   

 the matching finger number   

 the absolute matching score   

 an estimate of the probability of a match (0 to 100)

The optional part consists of:  

 the number of good minutiae identified in the latent   

 the number of latent minutiae which were successfully matched   

 the quality estimate of the latent (0 to 100, 100 is best)   

 the quality estimate of the candidate (0 to 100, 100 is best)

All candidates should be ordered by absolute score.

Sample Candidate List 

Required Part Optional Part 

Rank Mate 

ID 

Finger 

No. 

Abs. 

Score 

Prob. Of 

True Match 

No. Latent 

Minutiae 

Matched 

Minutiae 

Quality 

Latent 

Quality 

Candidate 

1 731 2 3513 93 18 12 70 80 

2 103 2 605 5 18 5 70 60 

3 103 3 513 4 18 5 70 90 

... 

100 107 9 422 1 18 4 70 70 

Each candidate list will be stored in an individual tab‐delimited ASCII text file having the

extension “.CL” and a base filename equal to that of the search latent (e.g. the candidate list for a

search of “L300U_M.lffs” will be named “L300U_M.CL”).  All candidate lists should be returned

in separate directories corresponding to each test performed (e.g. “L1_vs_E1/L300U_M.CL”).   

Within the candidate list file, all required and optional parts for an individual candidate entry

(i.e. row) should be written one per‐line in the order shown above, with each part (i.e. column)

separated by a single tab character.  Note that “Mate ID” shall be written as the base filename of

the exemplar (e.g. “E005A” without quotes).

4.2 Timing Results

Timing information for the following operations must be reported as “wall clock” elapsed time

(not CPU time) measurements, including the time to retrieve, process, and output all test data

and results.

 Exemplar Enrollment

 Latent Search

Enrollment times for each gallery may be reported for individual exemplars, or for the entire set

(i.e. the total time to enroll a complete gallery).    All individual latent search times must be

provided (i.e. one per candidate list).

Participants must provide detailed information on the hardware platform used to produce the

reported timings including processor type & speed and the number of processors/cores utilized.