FINGERPRINT VENDOR TECHNOLOGY EVALUATION 2003 

APPENDIX C 

SYSTEM-SPECIFIC RESULTS 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Table of Contents 

1 INTRODUCTION........................................................................................................................................ 4 

2 123 ID ............................................................................................................................................................. 5 

2.1 123 ID (LST) (Incomplete)................................................................................................................................................ 5 

2.2 123 ID (LST) (Overtime)................................................................................................................................................... 5 

2.3 123 ID M1 (MST) (Error).................................................................................................................................................. 9 

2.4 123 ID M2 (MST)..............................................................................................................................................................10 

3 ANTHEUS ...................................................................................................................................................16 

3.1 Antheus (LST)....................................................................................................................................................................16 

3.2 Antheus (MST)...................................................................................................................................................................17 

4 AVALON ......................................................................................................................................................21 

4.1 Avalon (LST) (Halted)......................................................................................................................................................21 

4.2 Avalon (MST).....................................................................................................................................................................22 

5 BIOLINK..................................................................................................................................................... 26 

5.1 Biolink (LST)......................................................................................................................................................................26 

5.2 Biolink (MST).....................................................................................................................................................................29 

6 BIOSCRYPT................................................................................................................................................ 33 

6.1 Bioscrypt (SST) ..................................................................................................................................................................33 

7 COGENT..................................................................................................................................................... 36 

7.1 Cogent (LST)......................................................................................................................................................................36 

7.2 Cogent (MST).....................................................................................................................................................................38 

7.3 Cogent (SST) ......................................................................................................................................................................42 

8 DERMALOG ............................................................................................................................................... 47 

8.1 Dermalog (LST) (Error)...................................................................................................................................................47 

8.2 Dermalog (LST).................................................................................................................................................................47 

8.3 Dermalog (MST) (Error)..................................................................................................................................................49 

9 GOLDEN FINGER .................................................................................................................................... 50 

9.1 Golden Finger (LST).........................................................................................................................................................50 

9.2 Golden Finger (MST) .......................................................................................................................................................52 

10 GRIAULE .................................................................................................................................................... 56 

10.1 Griaule (LST) .....................................................................................................................................................................56 

11 IDENTIX .................................................................................................................................................... 60 

11.1 Identix (LST)......................................................................................................................................................................60 

11.2 Identix (MST).....................................................................................................................................................................61 

12 MOTOROLA............................................................................................................................................... 65 

12.1 Motorola (LST) ..................................................................................................................................................................65 

12.2 Motorola (MST).................................................................................................................................................................66 

13 NEC ............................................................................................................................................................. 70 

13.1 NEC (LST) .........................................................................................................................................................................70 

13.2 NEC (MST)........................................................................................................................................................................72 

14 NEUROTECHNOLOGIJA ........................................................................................................................ 77 

14.1 Neurotechnologija M1 (MST).........................................................................................................................................77 

14.2 Neurotechnologija M2 (MST) (Error) ...........................................................................................................................79 

15 NIST .............................................................................................................................................................81 

15.1 NIST VTB (LST)...............................................................................................................................................................81 

22 April 2004 Page 2 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

15.2 NIST VTB (MST) .............................................................................................................................................................82 

15.3 NIST VTB (SST)...............................................................................................................................................................85 

16 PHOENIX GROUP .................................................................................................................................... 88 

16.1 Phoenix (MST)...................................................................................................................................................................88 

17 RAYTHEON ............................................................................................................................................... 92 

17.1 Raytheon (LST)..................................................................................................................................................................92 

17.2 Raytheon (MST).................................................................................................................................................................94 

18 SAGEM ........................................................................................................................................................ 98 

18.1 SAGEM L1 (LST).............................................................................................................................................................98 

18.2 SAGEM L2 (LST)...........................................................................................................................................................101 

18.3 SAGEM M1 (MST).........................................................................................................................................................105 

18.4 SAGEM M2 (MST) (Halted).........................................................................................................................................108 

18.5 SAGEM M2 (MST).........................................................................................................................................................108 

19 TECHNOIMAGIA.....................................................................................................................................112 

19.1 Technoimagia (MST) ......................................................................................................................................................112 

20 ULTRASCAN ............................................................................................................................................. 116 

20.1 UltraScan M1 (MST).......................................................................................................................................................116 

20.2 UltraScan M2 (MST).......................................................................................................................................................118 

21 TEST PROCESSING ISSUES ...................................................................................................................122 

22 April 2004 Page 3 of 127 
1 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Introduction 

This appendix presents detailed results for each system that participated in FpVTE. The 

sections are in alphabetical order by company name. The content of each section varies in 

order to highlight any salient characteristics that were observed for each system, but 

generally includes the following: 

• System identification 

• Discussion of any incomplete or unsuccessful runs 

• ROCs for a variety of partitions: 27 combinations of operational sources and image type 

(LST); 8 combinations of sources and image type (MST); or 2 operational sources (SST). 

• Raw score distributions, including histograms of match and non-match score

distributions. 

• Accuracy by image quality (MST and SST only) 

NOTES 

• When comparing charts, please note that the scale of the Y axis varies according to the 

data being shown. 

• For direct comparisons of the various systems, please see Section 5 of the FpVTE

Analysis Report. 

• Some systems encountered major or minor problems during processing. These are 

summarized in Section 21 of this Appendix. 

Company Name ID Website 

123 ID Inc. 123 ID www.123id.us 

Antheus Technology, Inc. Antheus www.antheustechnology.com 

Av@lon Systems, Inc. Avalon www.avalonsysinc.com 

BIO-key International, Inc. BIO-key www.bio-key.com 

BioLink Technologies International BioLink www.biolinkusa.com 

Bioscrypt, Inc. Bioscrypt www.bioscrypt.com 

Cogent Systems, Inc. Cogent www.cogentsystems.com 

Dermalog Identification Systems GmbH Dermalog www.dermalog.de 

Golden Finger System Golden Finger www.etgoldenfinger.com/index_en.htm 

Griaule Tecnologia Griaule www.griaule.com 

Identix , Inc. Identix www.identix.com 

SAGEM MORPHO, Inc. SAGEM www.morpho.com 

NEC NEC www.nec.com 

Neurotechnologija Ltd. Neurotech www.neurotechnologija.com 

The Phoenix Group, Inc. Phoenix www.afix.net 

Motorola Motorola www.motorola.com 

Raytheon Company Raytheon www.raytheon.com 

Technoimagia Co., Ltd. Technoimagia www.technoimagia.co.jp/e_index.htm 

Ultra-Scan Ultra-Scan www.ultra-scan.com 

Table C-1. Companies Participating in FpVTE 

22 April 2004 Page 4 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

2 123 ID 

123 ID entered one system for MST and for LST, with additional hardware for LST. The 

MST system was modified after an overflow error was reported by their Biometric System 

Search cluster system); the test was then successfully rerun. 

System name: Biometric System Search 

2.1 123 ID (LST) (Incomplete) 

The 123 ID LST system successfully completed subtests AxA through IxI (22 out of 31 

subtests, or all subtests except those involving dataset J) when they reached the three week 

time limit for the LST. Their incomplete results were archived as a separate LST run. Their 

complete results, including the results after the time limit, are reported in the next section. 

123ID had two hard drive failures during the LST evaluation. 

2.2 123 ID (LST) (Overtime) 

123 ID (LST) ran over the allotted time for LST, taking 34 days instead of 21 days. The 

participant chose to continue, after being reminded of the section of the Application to 

Participate in FpVTE 2003 that stated, 

3.13. Failure to complete the test during the allotted time will be noted in the 

FpVTE 2003 Final Report. FpVTE personnel will decide, after conclusion of 

the test, if the partial test results will be reported in the FpVTE 2003 Final

Report. 

This section presents the complete results including those produced after the time limit. The

nine LST Subtests JxI, and CxJ through JxJ were all produced after the time limit. 

Since 123 ID (LST) ran over the allotted time, it was not included in the system comparisons

in the FpVTE Analysis Report. If it had been included, its comparative rank would have 

been as shown in Table C-2. 

Distribution of System Rank over LST Partitions where FAR = 10-4 

27 LST Partitions of Operational Data 17 LST Partitions of Controlled (Ohio) Data 

Best Average Median Worst Best Average Median Worst 

123 ID 12 13.8 14 14 12 13.8 14 14 

Table C-2. System rank results for 123ID LST (Overtime) 

Configuration: Thirty-six (36) HP xw6000 search nodes, one HP a330n administrator node,

running the Biometric System Search (BSS), plus Virtual Print Signature Technology. This 

differs somewhat from the configuration described in their System Description Document:

36 CCSI Clusteron UltraCool Athlon nodes; one CCSI Clusteron Athlon Head node. 

Figure C-1 shows separate ROCs for 27 distinct operational partitions of the LST test. 

22 April 2004 Page 5 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-1. 123 ID (LST) ROC for 27 Operational Partitions 

Figure C-2 shows accuracy for the controlled (non-operational) Ohio data, which is much

higher than for the operational data. Note that performance degraded substantially for 1 and 

2-finger partitions including paper sources (xSP or xRP). Note the abrupt drop off in 10-

finger performance (green and purple lines). 

22 April 2004 Page 6 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-2. 123 ID (LST) ROC for 17 Controlled Partitions 

Figure C-3 shows the effect of the number of fingers on a single type of controlled Ohio 

data. 

22 April 2004 Page 7 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-3. 123ID (LST) ROCs by Number of Fingers for Controlled Data 

The figure below shows the score distributions for match and non-match scores on the 

Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the left 

indicates the number of non-match scores (green data). The y-axis on the right indicates the

number of mate scores (red data). Note that the minimum match score and maximum nonmatch score are indicated with red and green tick marks in the chart; this area of overlap is 

described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

22 April 2004 Page 8 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-4. 123ID (LST) Histogram of Match and Non-Match Distributions for BxA Subtest 

In the System Description document, 123 ID (MST) states that this configuration assumes 

that a matching score of 420,000 is the acceptance score for a positive identification, based

on the sample data. 

2.3 123 ID M1 (MST) (Error) 

In the first MST run for 123ID, the participant stated that the results were in error (overflow 

error reported by Biometric System Search cluster system); the test was successfully rerun as 

123 ID M2 with additional hardware and improved filtering software. 

As seen in Figure C-5, it appears that the matcher was not operating properly. 

22 April 2004 Page 9 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-5. 123ID M1 (MST) (Error) ROC 

2.4 123 ID M2 (MST) 

123 ID M2 (MST) completed processing in 1 day. 

Configuration: Sixteen (16) HP xw6000 search nodes, one HP a330n administrator node, 

running the Biometric System Search (BSS), plus Virtual Print Signature Technology. 

Figure C-6 shows separate ROCs for eight partitions of the MST data. Note that 123 ID M2 

(MST) performs markedly better on the controlled Ohio data. 

22 April 2004 Page 10 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-6. 123ID M2 (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

22 April 2004 Page 11 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-7. 123ID M2 (MST) Match and Non-Match Distributions 

The following figures show more clearly the area of overlap between high scoring nonmatches and low scoring matches. Figure C-8 shows performance in the interval from 0 to

100. The leftmost bin includes 241 matches that tie for low score of 0. 

Note the spike in both match and non-match values at 0.0. 

22 April 2004 Page 12 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-8. 123ID M2 (MST) Match and Non-Match Distributions Detail 

Figure C-9 shows performance much further to the right, in the interval from 1000 to 10000. 

This interval shows that the right tail of the non-match distribution remains heavy well into

the mate distribution. 

22 April 2004 Page 13 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-9. 123ID M2 (MST) Match and Non-Match Distributions Detail 

In the System Description document, 123 ID (MST) states that this configuration assumes 

that a matching score of 420,000 is the acceptance score for a positive identification. 

Figure C-10 shows the effect of image quality on accuracy, using the FpVTE aggregated 

image quality metric. 

22 April 2004 Page 14 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-10. 123ID M2 (MST) by Image Quality 

Image quality data was provided by 123 ID M1 and M2, which differed only trivially. The 

123ID M1 IQM was among the seven IQMs used in the FpVTE aggregate image quality 

metric. 

Participant Threshold Min Mean Median Max % FTE 

123ID M1 45.0 

45.0 

0.0 

0.0 

85.2 

85.3 

87.0 

88.0 

99.0 

99.0 

0.27% 

123ID M2 0.25% 

Figure C-11. 123ID (MST) IQM Distribution 

22 April 2004 Page 15 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

3 Antheus 

Antheus entered one system that was used for both MST and LST. 

System name: Agora 

Configuration: 1 Dual Pentium 4, running FpVTE Application Software, running 

UltraMatch AFIS and Windows 2000. 

3.1 Antheus (LST) 

Completion time: 14 days. 

Figure C-12 shows separate ROCs for 27 distinct operational partitions of the LST test. 

Figure C-12. Antheus (LST) ROCs for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

22 April 2004 Page 16 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Figure C-13. Antheus (LST) Match vs. Non-Match Distribution for BxA Subtest 

3.2 Antheus (MST) 

Completion time: 1 day 

Figure C-14 shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 17 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-14. Antheus (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 18 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-15. Antheus (MST) Match and Non-Match Distributions 

Figure C-16 shows the effect of image quality on accuracy, using the FpVTE aggregated 

image quality metric. 

22 April 2004 Page 19 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-16. Antheus (MST) by Image Quality 

Antheus provided image quality data for MST, but every image was assigned an image quality

value above the FTE threshold (specified by Antheus), so this data was not used to develop 

the aggregate metric. 

Participant Threshold Min Mean Median Max % FTE 

Antheus 0.0 12.0 70.8 74.0 100.0 0.00% 

22 April 2004 Page 20 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

4 Avalon 

Avalon entered one system for MST and one for LST. These systems used the same 

software, but different hardware. Avalon (LST) did not complete the test. 

System name: Ultramatch AFIS 

4.1 Avalon (LST) (Halted) 

Avalon (LST) did not complete the test. The test was halted by the participant when it 

became clear that the entire LST would not be completed in time. The results for Subtests 

AxA and BxA are included here. 

Configuration: 4 Celerons + 2 Pentium 4s, running Ultramatch, Windows XP Professional 

Figure C-17 shows separate ROCs for 5 of the 27 distinct operational partitions of the LST 

test. 

Figure C-17. Avalon (LST) ROC for 5 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

22 April 2004 Page 21 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

Figure C-18. Avalon (LST) Match vs. Non-Match Distribution for BxA Subtest 

4.2 Avalon (MST) 

Configuration: 1 Pentium 4, running Ultramatch, Windows XP Professional 

Completion time: 1 day 

Figure C-19 shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 22 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-19. Avalon (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

22 April 2004 Page 23 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-20. Avalon (MST) Match and Non-Match Distributions 

Figure C-21 shows the effect of image quality on accuracy, using the FpVTE aggregated 

image quality metric. 

22 April 2004 Page 24 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-21. Avalon (MST) by Image Quality 

22 April 2004 Page 25 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

5 Biolink 

Biolink entered one system for MST and one for LST. These systems used the same 

software, but different hardware. 

System name: Authenteon-based FpVTE 

5.1 Biolink (LST) 

Configuration: A Control Front End and SQL Server database consisting of 2 Pentium 4s 

running Windows Server 2003; and a Search Engine consisting of 24 RLX 1200i blades

(Pentium 3), running FpVTE Application Software v66 (Authenteon), Windows Server 2003 

Completion time: 6 days 

Figure C-22 shows separate ROCs for 27 distinct operational partitions of the LST test. 

(except note that none of the single-finger or two-finger results are graphed). These unusual 

results are explained in part by Figure C-23. LST_BxA is typical of Biolink (LST) singlefinger results: after rejecting a large percentage of pairs (i.e., assigning a score of 0), it

consistently achieved perfect separation of the remaining match and non-match pairs. On 

the ROC charts, no data can be plotted for this partition because there are only two 

meaningful data points: 1,1 (never plotted), and 0,0.51 (off the chart). 

22 April 2004 Page 26 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-22. Biolink (LST) ROCs for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

22 April 2004 Page 27 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-23. Biolink (LST) Match vs. Non-Match Distribution for BxA Subtest 

Although it is striking that this system managed to assign a zero similarity to every non-mate 

comparison, it did so while assigning zero similarity to more than half of the mate 

comparisons, as well. 

The ROCs in this report do not generally show interpolated values. Figure C-24 shows the 

same results as Figure C-22, but with interpolated values added for the 2-finger results. Bear 

in mind that interpolations over a great range such as this cannot be precise. 

22 April 2004 Page 28 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-24. Biolink (LST) ROCs for 27 Operational Partitions, with interpolated values for 2-finger 

results 

5.2 Biolink (MST) 

Configuration: A Pentium 4 PC, running FpVTE Application Software v66 (Authenteon) 

and Windows Server 2003. 

Completion time: 4 days 

Figure C-25 shows separate ROCs for eight partitions of the MST data. Note the sensitivity 

to DHS2 flat images, which are known to be of especially low quality. 

22 April 2004 Page 29 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-25. Biolink (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 30 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-26. Biolink (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

Biolink provided image quality data, which was used to develop this aggregate metric. 

22 April 2004 Page 31 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-27. Effect of Image Quality 

Image quality data was provided by Biolink (MST), and was among the seven IQMs used in 

the FpVTE aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Biolink 25.0 0.0 75.8 81.0 100.0 3.92% 

Table C-3. Image Quality Distribution 

22 April 2004 Page 32 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

6 Bioscrypt 

Bioscrypt entered one system for SST. 

System name: Bioscrypt Core 

Configuration: 1 Dual Xeon 3.06 GHz, running Bioscrypt Core and Windows XP 

Pofessional. 

6.1 Bioscrypt (SST) 

Completion time: 5 days 

Figure C-28 shows separate ROCs for the BCC and DHS2 partitions of the SST data. 

Standard is simply a combination of BCC and DHS2 data. Note the dramatic difference 

between the BCC and DHS2 results. Note also how horizontal the ROCs are after they leave 

the right axis. 

Figure C-28. Bioscrypt (SST) ROCs for SST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the SST test (single-finger flat and slap images). The y-axis on the 

22 April 2004 Page 33 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

Figure C-29. Bioscrypt (SST) Match vs. Non-Match Distribution 

22 April 2004 Page 34 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

The figure below shows Bioscrypt’s sensitivity to image quality. 

Figure C-30. Effect of Image Quality 

22 April 2004 Page 35 of 127 
 

 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

7 Cogent 

Cogent entered one system in each of SST, MST and LST. 

7.1 Cogent (LST) 

System name: Galaxy V8.60 for LST 

Configuration: 6 IBM xSeries 335 Server, Dual CPU, running Galaxy V8.60 for LST 

(minutae matching) and Windows 2000. 

Completion time: 8 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

Figure C-31. Cogent (LST) ROC for 27 Operational Partitions 

The figure below shows the same results in a Detection Error Tradeoff (DET) chart, in 

which the Y axis is in log scale, and accuracy increases down and to the left. DETs show 

more detail at low error rates. Note that any results with no errors cannot be graphed in a 

DET. 

22 April 2004 Page 36 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-32. Cogent (LST) DET for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 37 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-33. Cogent (LST) Match vs. Non-Match Distribution for BxA Subtest 

7.2 Cogent (MST) 

System name: Galaxy V3.2 for MST 

Configuration: 1 IBM xSeries 335 Server, Dual CPU, running Galaxy V3.2 for MST 

(template matching), Windows 2000 

Completion time: 6 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 38 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-34. Cogent (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for mate and non-mate scores on the 

Standard partition of the MST test (single-finger flat and slap images). The y-axis on the left 

indicates the number of non-match scores (green data). The y-axis on the right indicates the

number of mate scores (red data). Note that the minimum match score and maximum nonmatch score are indicated with red and green tick marks in the chart; this area of overlap is 

described in the caption as the “tail.” 

22 April 2004 Page 39 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-35. Cogent (MST) Match vs. Non-Match Distribution 

22 April 2004 Page 40 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-36. Cogent (MST) Match vs. Non-Match Distribution (detail) 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from the systems that provided image quality data. 

22 April 2004 Page 41 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-37. Effect of Image Quality 

Image quality data was provided by Cogent (MST), and was among the seven IQMs used in 

the FpVTE aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Cogent 6.0 1.0 3.3 3.0 8.0 17.01% 

Table C-4. Image Quality Distribution 

7.3 Cogent (SST) 

System name: Galaxy V3.2T for SST 

Configuration: 1 IBM xSeries 335 Server, Dual CPU, running Galaxy V3.2T for SST (image 

matching). 

Completion time: 7 days 

The figure below shows separate ROCs for the BCC and DHS2 partitions of the SST data.

Standard is simply a combination of BCC and DHS2 data. On the ROC charts, no data is 

plotted for DHS2 because Cogent achieved perfect separation of mate and non-matches 

22 April 2004 Page 42 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

scores on that partition (matches scored 607 or higher; non-matches all scored zero).

Hence, there are only two meaningful data points: 1,1 (never plotted), and 0,0 (off the chart). 

Figure C-38. Cogent (SST) ROCs for SST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the SST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 43 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-39. Cogent (SST) Match vs. Non-Match Distribution 

22 April 2004 Page 44 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-40. Cogent (SST) Match vs. Non-Match Distribution (Detail) 

The lowest mate score on the DHS2 partition was 607, giving perfect separation on that 

partition. 

The figure below shows Cogent’s insensitivity to image quality: images of all qualities 

performed well. The C quality cannot be graphed because of perfect separation of the match 

and non-match values. 

22 April 2004 Page 45 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-41. Effect of Image Quality 

22 April 2004 Page 46 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

8 Dermalog 

Dermalog entered one system for MST and LST. 

System name: DermalogFingerCode3 

Configuration: 6 Pentium 4s, running DermalogFingerCode3 kernel and Windows XP 

Professional. 

As compared to generally similar performance by Motorola (LST), Dermalog (LST) achieved 

higher accuracy on single finger, operational quality images. 

8.1 Dermalog (LST) (Error) 

The first run of the Dermalog LST system deadlocked on processing some subtests. 

Similarity matrices from a few subtests were not successfully saved; those subtests were 

successfully rerun and the resultant similarity matrices were added to the similarity matrices

saved previously and together were called Dermalog (LST), below. 

8.2 Dermalog (LST) 

Dermalog (LST) includes the results from the 1st run for those subtests that were not 

affected by the deadlock problem, plus the results from a 2nd run on those subtests that 

were affected by the deadlock problem. 

Completion time: 11 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

22 April 2004 Page 47 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-42. Dermalog (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 48 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-43. Dermalog (LST) Match vs. Non-Match Distribution for BxA Subtest 

8.3 Dermalog (MST) (Error) 

Dermalog (MST) returned similarity files containing score value 100 on the matrix diagonal 

(self-identifications), and zero values for all other comparisons: a blank matrix. The MD5

signatures were validated, so the blank matrix was in fact what was produced by their system. 

Completion time: less than an hour 

22 April 2004 Page 49 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

9 Golden Finger 

Golden Finger entered one system for MST and LST. 

System name: GAFIS 

Configuration: 4 Dual Xeons, running GAFIS 5.0 Engine Network Edition and Windows 

2003 Enterprise Server 

Golden Finger (MST) and Raytheon (MST) returned equivalent similarity matrices on MST. 

Except for two scores, Golden Finger (MST) scores (integer values) could be computed

from Raython (MST) scores by multiplying the (floating point) Raytheon (MST) scores by 

100 and rounding to the nearest integer. The two exceptions were non-matches: Golden 

Finger (MST) assigned score values of -1, whereas Raytheon (MST) assigned positive score 

values. 

Golden Finger (LST) and Raytheon (LST) returned equivalent results on some subtests, but 

significantly different results on others. The greatest differences were on subtests involving 

datasets I and J. 

9.1 Golden Finger (LST) 

Completion time: 12 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test.

Golden Finger (LST) achieved very low accuracy on the 2-finger SLxRL Ident-Iafis partition 

and on the 1-finger SLxRL Ident-Iafis partition. 

22 April 2004 Page 50 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-44. Golden Finger (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 51 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-45. Golden Finger (LST) Match vs. Non-Match Distribution for BxA Subtest 

9.2 Golden Finger (MST) 

Completion time: 1 day 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 52 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-46. Golden Finger (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 53 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-47. Golden Finger (MST) Match vs. Non-Match Distribution 

Note that the score distribution for Golden Finger (MST) is exactly 100 times the score 

distribution for Raytheon (MST). 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data.

Golden Finger provided image quality data, which was used to develop this aggregate metric. 

Note that the effect of image quality on accuracy is somewhat unusual as compared to many

other systems: much of the degradation occurs quickly as the quality begins to drop 

(between A and B quality), whereas most other systems drop off sharply when the quality is 

especially low. 

22 April 2004 Page 54 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-48. Effect of Image Quality 

Image quality data was provided by Golden Finger (MST), and was among the seven IQMs 

used in the FpVTE aggregate image quality metric. The Golden Finger image quality values

were exactly 100 times the Raytheon image quality values. 

Participant Threshold Min Mean Median Max % FTE 

Golden Finger 35.0 0.0 62.9 70.0 75.0 15.78% 

Table C-5. Image Quality Distribution 

22 April 2004 Page 55 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

10 Griaule 

Griaule participated in LST only. 

System name: Griaule AFIS 

Configuration: 10 Itautec Inforserver PCs (Dual Xeons), running Griaule AFIS. 

10.1 Griaule (LST) 

Completion time: 17 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test.

Note that the ROCs do not include data for large values of FAR. This is due to Griaule’s 

assignment of a zero score to a large number of mated pairs. For further explanation of this

effect, see the discussion under Biolink (LST), which has a similar issue. 

Figure C-49. Griaule (LST) ROC for 27 Operational Partitions 

Griaule (LST) does not, in general, show an increase in accuracy as the number of fingers 

increases beyond 2. Figure C-50 clearly shows this effect on one data source. As can be 

seen throughout the main body of the report, on 1-finger and 2-finger tests Griaule (LST) is 

22 April 2004 Page 56 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

generally about as accurate as Identix (LST) and consistently more accurate than either 

Biolink (LST) or NIST VTB (LST). This relationship changes when more than 2 fingers are 

considered. 

Figure C-50. Griaule (LST) Effect of Number of Fingers 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 220 matches scored 0 on the 

Standard partition of the LST subtest. 

22 April 2004 Page 57 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-51. Griaule (LST) Match vs. Non-Match Distribution for BxA Subtest 

22 April 2004 Page 58 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-52. Griaule (LST) Match vs. Non-Match Distribution for BxA Subtest (Detail) 

22 April 2004 Page 59 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

11 Identix 

Identix entered one system for MST and LST. 

System name: BioEngine Software Developer's Kit (SDK) 

Configuration: IBM X Series 8 Blade System, running BioEngine SDK v4.0 step 1 and 

Windows 2000 Advanced Server. 

11.1 Identix (LST) 

Completion time: 6 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

Figure C-53. Identix (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

22 April 2004 Page 60 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

Figure C-54. Identix (LST) Match vs. Non-Match Distribution for BxA Subtest 

11.2 Identix (MST) 

Completion time: 1 day 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 61 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-55. Identix (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

22 April 2004 Page 62 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-56. Identix (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 63 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-57. Effect of Image Quality 

22 April 2004 Page 64 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

12 Motorola 

Motorola entered one system for both MST and LST 

System name: Motoral Omnitrak 

Configuration: 15 HP DL360G3s (dual Xeon) running Omnitrak matcher; 1 HP DL380G3 

(dual Xeon) master controller. All running Windows 2000 Server. 

12.1 Motorola (LST) 

Completion time: 18 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

As compared to generally similar performance by Dermalog (LST), Motorola (LST) gains a 

slight advantage when the number of fingers is high or the quality of images is high. 

Figure C-58. Motorola (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

22 April 2004 Page 65 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Figure C-59. Motorola (LST) Match vs. Non-Match Distribution for BxA Subtest 

12.2 Motorola (MST) 

Completion time: 1 day 

The figure below shows separate ROCs for eight partitions of the MST data. 

As compared to generally similar performance by Neurotechnologija (MST), Motorola 

(MST) achieved slightly higher accuracy on comparisons of flat images. 

22 April 2004 Page 66 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-60. Motorola (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 67 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-61. Motorola (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 68 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-62. Effect of Image Quality 

22 April 2004 Page 69 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

13 NEC 

NEC entered one system for the MST test and one for the LST test. 

NEC consistently achieved high accuracy on all subtests, often achieving perfect or near

perfect separation of match and non-match scores, and usually ranking in first place relative 

to the competition. 

Larger tests are required to properly measure the capabilities of the NEC Cluster-PC 

Matching Servers. The score distributions presented in this section show how NEC’s

performance on many test partitions exceeded the limits of measurability for FpVTE 2003. 

13.1 NEC (LST) 

System name: NEC Cluster-PC Matching Server for LST 

Configuration: 1 host and 9 backend servers (dual Xeon workstations), running NEC 

Application Software and Windows 2000. 

Completion time: 19 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

22 April 2004 Page 70 of 127 
 

 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-63. NEC (LST) ROC for 27 Operational Partitions 

The figure below shows the same results in a Detection Error Tradeoff (DET) chart, in 

which the Y axis is in log scale, and accuracy increases down and to the left. DETs show 

more detail at low error rates. Note that any results with no errors cannot be graphed in a 

DET. 

Figure C-64. NEC (LST) DET for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

The apparently irregular distribution of scores may be an artifact of the binning intervals 

used to create the histogram. 

22 April 2004 Page 71 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-65. NEC (LST) Match vs. Non-Match Distribution for BxA Subtest 

13.2 NEC (MST) 

System name: NEC Cluster-PC Matching Server for MST 

Configuration: 1 host and 2 backend servers (dual Xeon workstations), running NEC 

Application Software and Windows 2000. 

Completion time: 9 days 

The following chart shows NEC (MST) accuracy on a variety of partitions. Note that there 

is no data for Ohio slaps or Ident-Iafis flats because NEC (MST) achieved perfect separation

of match and non-match scores on these partitions. 

22 April 2004 Page 72 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

22 April 2004 Page 73 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-66. NEC (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

22 April 2004 Page 74 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-67. NEC (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data.

NEC (MST) achieves high accuracy on all but the very worst quality images: 

22 April 2004 Page 75 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-68. Effect of Image Quality 

22 April 2004 Page 76 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

14 Neurotechnologija 

Neurotechnologija participated in MST only. During their first MST run (M1), they 

encountered problems with their WSQ decompression software, and were unable to read 33

images. After completing the M1 run, they modified their software configuration to 

incorporate different WSQ software and ran again; that run was labeled M2. 

System name: VeriFinger 

Configuration: 1 PC (Pentium 4), running VeriFinger 4.2 software test version and 

Windows XP Professional. 

As compared to generally similar performance by Motorola (MST), Neurotechnologija

(MST) achieved higher accuracy on comparisons of segmented slap images, and 

combinations of slap and flat images. 

14.1 Neurotechnologija M1 (MST) 

Completion time: 2 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

Figure C-69. Neurotechnologija M1 (MST) ROCs for MST Partitions 

22 April 2004 Page 77 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Figure C-70. Neurotechnologija M1 (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data.

Neurotechnologija (M1 and M2) provided image quality data, but every image was assigned 

an image quality value above the threshold (specified by Neurotechnologija), so this data was

not used to develop the aggregate metric. 

22 April 2004 Page 78 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-71. Effect of Image Quality 

Image quality data was provided by Neurotechnologija M1 and M2 (MST); the M1 results 

were among the seven IQMs used in the FpVTE aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Neurotech. M1 1.0 

1.0 

0.0 

21.0 

55.4 

55.6 

52.0 

52.0 

134.0 

255.0 

0.31% 

Neurotech. M2 0.00% 

Table C-6. Image Quality Distribution 

14.2 Neurotechnologija M2 (MST) (Error) 

After Neurotechnologija M1 failed to decompress 31 of the images in MST, they 

incorporated a different WSQ encoder into their system and reran MST. Since only the

WSQ decoder was changed, only the 31 problem scores from M1 were expected to be 

different. However, the results of M2 were clearly in error. In general, the scores in the first 

part of the similarity matrix were similar to those from the M1 run, but the later scores 

appeared to be random numbers. This is shown in Figure C-72: the curve that the slap

results follow is the same line that would result if random numbers were plotted. Since the 

bulk of the results in the M2 run were random, they were not included in analysis. 

22 April 2004 Page 79 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-72. Neurotechnologija M2 (MST) ROCs for MST Partitions 

22 April 2004 Page 80 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

15 NIST 

NIST ran all three tests on the VTB as a benchmark. 

System name: NIST VTB 

Configuration: 16 PCs (dual Xeon), running the VTB matcher (Bozorth98) and Red Hat 

Linux 7.2. 

15.1 NIST VTB (LST) 

Completion time: 21 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

Figure C-73. NIST VTB (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

22 April 2004 Page 81 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Figure C-74. NIST VTB (LST) Match vs. Non-Match Distribution for BxA Subtest 

15.2 NIST VTB (MST) 

Completion time: 12 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 82 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-75. NIST VTB (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 83 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-76. NIST VTB (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 84 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-77. Effect of Image Quality 

15.3 NIST VTB (SST) 

Completion time: 1 day 

22 April 2004 Page 85 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-78. NIST VTB (SST) ROCs for SST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the SST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 86 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-79. NIST VTB (SST) Match vs. Non-Match Distribution 

22 April 2004 Page 87 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

16 Phoenix Group 

Phoenix Group participated in MST only. 

16.1 Phoenix (MST) 

System name: AFIX Tracker 

Configuration: 1 PC (Pentium 4), running AFIX Tracker v4.4 and Windows XP 

Professional. 

Completion time: 5 days 

The figure below shows separate ROCs for eight partitions of the MST data. The separation 

in accuracy between the Ohio data (high quality) and the operational data is unusually large 

(as compared to other systems). 

Figure C-80. Phoenix (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

22 April 2004 Page 88 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

Note the spike in both match and non-match values at 0.0. 

Figure C-81. Phoenix (MST) Match vs. Non-Match Distribution 

22 April 2004 Page 89 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-82. Phoenix (MST) Match vs. Non-Match Distribution (Detail) 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 90 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-83. Effect of Image Quality 

Phoenix provided image quality data, but every image was assigned an image quality value 

above the threshold specified by Phoenix, so their image quality results were not used to 

develop the aggregate metric. 

Participant Threshold Min Mean Median Max % FTE 

Phoenix 0.0 1.0 7.7 9.0 9.0 0.00% 

Table C-7. Image Quality Distribution 

22 April 2004 Page 91 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

17 Raytheon 

Raytheon participated in MST and LST. These systems used the same software, but 

different hardware. 

System name: RAYAFIS 

Raytheon (MST) and Golden Finger (MST) returned equivalent similarity matrices on MST.

Except for two scores, Golden Finger (MST) scores (integer values) could be computed 

from Raython (MST) scores by multiplying the (floating point) Raytheon (MST) scores by

100 and rounding to the nearest integer. The two exceptions were non-matches: Golden 

Finger (MST) assigned score values of -1, whereas Raytheon (MST) assigned positive score 

values. 

Raytheon (LST) and Golden Finger (LST) returned equivalent results on some subtests, but

significantly different results on others. The greatest differences were on subtests involving 

datasets I and J. 

17.1 Raytheon (LST) 

Configuration: 4 matchers (2 Pentium 4 and 2 dual Xeon) and 1 PC (1 Pentium) controller,

running RAYAFIS Software and Microsoft Server 2003. 

Completion time: 15 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

Raytheon (LST) achieved very low accuracy on the 2-finger SLxRL Ident-Iafis partition and

on the 1-finger SLxRL Ident-Iafis partition. 

22 April 2004 Page 92 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-84. Raytheon (LST) ROC for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 93 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-85. Raytheon (LST) Match vs. Non-Match Distribution for BxA Subtest 

17.2 Raytheon (MST) 

Configuration: 2 matchers (Pentium 4), running RAYAFIS Software and Microsoft Server 

2003. 

Completion time: 1 day 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 94 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-86. Raytheon (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

Note that the score distribution for Golden Finger (MST) is exactly 100 times the score 

distribution for Raytheon (MST). 

22 April 2004 Page 95 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-87. Raytheon (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data.

Raytheon provided image quality data, but every image was assigned an image quality value 

below the threshold (specified by Raytheon), so this data was not used to develop the 

aggregate metric. Note that this distribution is somewhat unusual as compared to many 

other systems: much of the degradation occurs quickly as the quality begins to drop,

whereas many systems drop off sharply when the quality is especially low. 

22 April 2004 Page 96 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-88. Effect of Image Quality 

Image quality data was provided by Raytheon (MST). Raytheon (MST) set its image quality 

threshold to 1. All images in the standard partition scored below this threshold, meaning 

that all images were of unsatisfactory quality. Presumably the image quality threshold was 

set incorrectly. For this reason, this was not among the seven IQMs used in the FpVTE 

aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Raytheon 1.0 0.0 0.6 0.7 0.8 100.00% 

Table C-8. Image Quality Distribution 

22 April 2004 Page 97 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

18 SAGEM 

SAGEM entered two systems in each of MST and LST. 

System name: SAGEM MetaMorpho 

Configuration: 7 Pentium 4s, running MetaMorpho Software version 3.1.2A.NST.3A, 

Windows 2000 Professional 

18.1 SAGEM L1 (LST) 

Configuration: 1 Workflow Control station (Pentium 4) and 6 Fingerprint Processing

stations (Pentium 4), running MetaMorpho Software and Matcher Engine version 

3.1.2A.NST.3A, and Windows 2000 Professional. 

Completion time: 7 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

Figure C-89. SAGEM L1 (LST) ROC for 27 Operational Partitions 

22 April 2004 Page 98 of 127 
 

 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

The figure below shows the same results in a Detection Error Tradeoff (DET) chart, in 

which the Y axis is in log scale, and accuracy increases down and to the left. DETs show 

more detail at low error rates. Note that any results with no errors cannot be graphed in a 

DET. 

Figure C-90. SAGEM L1 (LST) DET for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 99 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-91. SAGEM L1 (LST) Match vs. Non-Match Distribution for BxA Subtest 

22 April 2004 Page 100 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-92. SAGEM L1 (LST) Match vs. Non-Match Distribution for BxA Subtest (Detail) 

18.2 SAGEM L2 (LST) 

Configuration: 1 Workflow Control station (Pentium 4) and 2 Fingerprint Processing

stations (Pentium 4), running MetaMorpho Software and Matcher Engine version 

3.1.2A.NST.3A, and Windows 2000 Professional. 

Completion time: 6 days 

The figure below shows separate ROCs for 27 distinct operational partitions of the LST test. 

22 April 2004 Page 101 of 127 
 

 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-93. SAGEM L2 (LST) ROC for 27 Operational Partitions 

The figure below shows the same results in a Detection Error Tradeoff (DET) chart, in 

which the Y axis is in log scale, and accuracy increases down and to the left. DETs show 

more detail at low error rates. Note that any results with no errors cannot be graphed in a 

DET. 

22 April 2004 Page 102 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-94. SAGEM L2 (LST) DET for 27 Operational Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the LST_BxA subtest (single-finger flat images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 103 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-95. SAGEM L2 (LST) Match vs. Non-Match Distribution for BxA Subtest 

22 April 2004 Page 104 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-96. SAGEM L2 (LST) Match vs. Non-Match Distribution for BxA Subtest (Detail) 

18.3 SAGEM M1 (MST) 

Configuration: 1 workstation for Workflow Control and Fingerprint Processing (Pentium 

4), running MetaMorpho Software and Matcher Engine version 3.1.2A.NST.2A, and

Windows 2000 Professional. 

Completion time: 4 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 105 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-97. SAGEM M1 (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 106 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-98. SAGEM M1 (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 107 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-99. Effect of Image Quality 

18.4 SAGEM M2 (MST) (Halted) 

SAGEM encountered a variety of problems in their initial 3-workstation configuration for 

MST M2. They reconfigured to run using a single machine, then ran successfully. 

18.5 SAGEM M2 (MST) 

Configuration: 1 workstation for Workflow Control and Fingerprint Processing (Pentium

4), running MetaMorpho Software and Matcher Engine version 3.1.2A.NST.2A, and 

Windows 2000 Professional. 

Completion time: 10 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 108 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-100. SAGEM M2 (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 109 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-101. SAGEM M2 (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric

represents an aggregate of responses from seven systems that provided image quality data. 

22 April 2004 Page 110 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-102. Effect of Image Quality 

22 April 2004 Page 111 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

19 Technoimagia 

System name: FP-Workstation 

Configuration: 1 HP xw6000/CT workstation (dual Xeon), running Fingerprint

Aunthentication System Tool21 (“FAST21”) and Windows XP. 

19.1 Technoimagia (MST) 

Completion time: 5 days 

The figure below shows separate ROCs for eight partitions of the MST data. Technoimagia

(MST) achieved very low scores on DHS2 (low quality flat images). The flat images, in 

general, show a marked drop in accuracy when the operational threshold is set high. 

Figure C-103. Technoimagia (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

22 April 2004 Page 112 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

is described in the caption as the “tail.” Note that the ranges of the match and non-match 

distributions are identical. Note the large number of 0 scores for mates. 

Figure C-104. Technoimagia (MST) Match vs. Non-Match Distribution 

The following detailed histogram of the BCC results shows a significant peak in non-match 

scores at 0.5 and 1.0, which may account for the distinct shape of Technoimagia’s BCC 

ROC. 

22 April 2004 Page 113 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-105. Technoimagia (MST) Match vs. Non-Match Distribution (Detail) 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

Technoimagia provided image quality data, which was used to develop the aggregate metric. 

22 April 2004 Page 114 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-106. Effect of Image Quality 

Image quality data was provided by Technoimagia (MST), and was among the seven IQMs 

used in the FpVTE aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Technoimagia 1.0 0.0 1.6 1.8 2.0 8.55% 

Table C-9. Image Quality Distribution 

22 April 2004 Page 115 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

20 UltraScan 

20.1 UltraScan M1 (MST) 

System name: IDExpress Developer 

Configuration: 1 PC (Pentium 4), running IDExpress Developer version 1.5.3 and Windows 

XP Professional. 

Completion time: 2 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

Figure C-107. UltraScan M1 (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap 

is described in the caption as the “tail.” 

22 April 2004 Page 116 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-108. UltraScan M1 (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data. 

UltraScan (M1 and M2) provided image quality data; the UltraScan M1 image quality data

was used to develop the aggregate metric. 

22 April 2004 Page 117 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-109. Effect of Image Quality 

Image quality data was provided by UltraScan M1 and M2 (MST). Because of the similarity 

of these measures, only UltraScan M1 was among the seven IQMs used in the FpVTE 

aggregate image quality metric. 

Participant Threshold Min Mean Median Max % FTE 

Ultrascan M1 0.7 

0.7 

0.0 

0.0 

0.9 

0.9 

0.9 

0.9 

1.0 

1.0 

1.23% 

Ultrascan M2 1.21% 

Table C-10. Image Quality Distribution 

20.2 UltraScan M2 (MST) 

System name: IV &V 

Configuration: 1 PC (Pentium 4), running IVV version 2.00 and Windows XP Professional. 

Completion time: 2 days 

The figure below shows separate ROCs for eight partitions of the MST data. 

22 April 2004 Page 118 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-110. UltraScan M2 (MST) ROCs for MST Partitions 

The following histogram shows the score distributions for match and non-match scores on 

the Standard partition of the MST test (single-finger flat and slap images). The y-axis on the 

left indicates the number of non-match scores (green data). The y-axis on the right indicates 

the number of mate scores (red data). Note that the minimum match score and maximum 

non-match score are indicated with red and green tick marks in the chart; this area of overlap

is described in the caption as the “tail.” 

22 April 2004 Page 119 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-111. UltraScan M2 (MST) Match vs. Non-Match Distribution 

The following figure shows sensitivity to image quality, where the image quality metric 

represents an aggregate of responses from seven systems that provided image quality data.

UltraScan provided image quality data, which was used to develop this aggregate metric. 

22 April 2004 Page 120 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

Figure C-112. Effect of Image Quality 

22 April 2004 Page 121 of 127 
 

FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

21 Test Processing Issues 

Several of the systems encountered problems while the evaluation was being conducted. A 

summary of the issues is included in Table C-11. 

Several systems encountered WSQ decompression problems. The sample, trivial, and 

evaluation images came from a variety of sources, and therefore were compressed using 

various WSQ implementations. All of the images in FpVTE could be decompressed using 

NIST's WSQ toolkit [LFIS], as well as Aware's WSQ. Some other WSQ implementations

had problems decompressing some of the images in FpVTE. 

Several systems encountered errors writing to USB drives. The MD5 files provided proof 

that the similarity matrices as written by the system were not corrupted, and in several cases 

indicated that similarity matrices needed to be recopied. 

Several systems had scripting problems that were uncovered when the trivial datasets were 

run. These were resolved before the start of the evaluation itself, so they are not reported 

here. 

Two systems submitted results that were subsequently, during analysis, shown to be in error; 

these are noted with asterisks (*). 

System 

123 ID (LST) 

Level of 

Problems 

Major 

Summary of Issues 

Memory problems; node crash; hard drive problems; did not 

complete in time 

123 ID M1 (MST) Major Memory overflow problems; updated code; reran as 123 M2 

123 ID M2 (MST) None 

Antheus (LST) Minor Writing directly to USB drive during test was unsuccessful 

Antheus (MST) None 

Avalon (LST) Major Multiple system crashes; halted test 

Avalon (MST) None 

Biolink (LST) None 

Biolink (MST) None 

Bioscrypt (SST) None 

Cogent (LST) Medium System crash; manual combination of results and load 

balancing 

Cogent (MST) None 

Cogent (SST) None 

Dermalog (LST) Medium Network deadlock required reboot and affected some 

subtests; affected subtests were rerun 

Dermalog (MST) None* (No problems during tests, but the resulting similarity matrix 

contained only zeros) 

Golden Finger (LST) Minor Crash during decompression of WSQ image, restart 

Golden Finger (MST) None 

Griaule (LST) Major System could not recover after failures to enroll; multiple 

reboots; updated code and ran successfully 

Identix (LST) Medium System failed to automatically merge results from different 

nodes 

22 April 2004 Page 122 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

System 

Identix (MST) 

Level of 

Problems 

None 

Summary of Issues 

Motorola (LST) Major Unable to decompress 3 images; changed WSQ software; 

problems with load balancing and throughput; multiple 

restarts; updated code 

Motorola (MST) None 

NEC (LST) Minor Problem with one node 

NEC (MST) None 

Neurotechnologija M1 

(MST) 

Medium Unable to decompress 31 images; ignored those images and 

completed test, but reran with different WSQ decoder as M2 

Neurotechnologija M2 

(MST) 

None* (No problems during tests, but much of the resulting 

similarity matrix contained apparently random results) 

Phoenix (MST) Minor One similarity file had to be recopied 

Raytheon (LST) Minor Problem processing one file; restarted 

Raytheon (MST) None 

SAGEM L1 (LST) None 

SAGEM L2 (LST) None 

SAGEM M1 (MST) None 

SAGEM M2 (MST) Major Problems transferring files between machines; reconfigured 

to run using single machine, restarted successfully 

Technoimagia (MST) None Ran two identical systems simultaneously for redundancy; 

results were identical 

UltraScan M1 (MST) None 

UltraScan M2 (MST) None 

Table C-11. Summary of Processing Issues 

Further detail of issues encountered during processing is included below. 

123 ID LST 

• Memory low, template create failed on some nodes; patched, restarted 

• Found memory leak, new patch, restarted 

• Node crashed—missing 40% of data from that node; installed spare, restarted other

nodes 

• Applied patch to redo data lost on bad node; restarted each node 

• Replaced bad node 

• Another node had hard drive errors—pulled from system 

• Did not complete LST by deadline; completed after time ran out 

• These overtime results are not included in the main body of the report 

123 ID MST 

• Program that collects results from each of the nodes failed 

• Brought in new application to collect results from nodes 

• Discovered memory overflow problems 

• Asked to rerun MST with additional hardware and improved filtering—Test Agents 

agreed 

22 April 2004 Page 123 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

• Reran. Results presented in main body of report are from this second run (M2) 

ANTHEUS LST 

• Planned to read and write to USB on the fly 

• System stopped matching during test; was rebooted and restarted writing to system disk 

rather than USB 

• No other problems 

ANTHEUS MST 

• MST ran with no problems 

AVALON LST 

• System appeared to get stuck on WSQ conversion; shut down and restarted 

• Had to copy templates to 2 nodes because automatic process failed 

• System crashed, restarted 

• System crashed again—major problems 

• Installed program to generate more extensive logging information 

• Continued to have problems processing subtests involving dataset C 

• Installed another program with more extensive logging and restarted 

• Continued to have problems 

• Decided to halt test 

AVALON MST 

• Ran MST with no problems 

BIOLINK LST 

• Did MST and LST simultaneously 

• LST ran with no problems 

BIOLINK MST 

• Did MST and LST simultaneously 

• MST ran with no problems 

BIOSCRYPT SST 

• Ran SST with no problems 

COGENT LST 

• Ran SST, MST, and LST simultaneously 

• Combining of preprocessed files failed to execute on some subtests. Had to manually 

combine and restart matching 

• Crashed at one point, had to be restarted, and files manually combined 

• Moved input files around to make room for results 

• Performed manual load balancing among nodes 

22 April 2004 Page 124 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

• Completed LST with no further problems 

COGENT MST 

• Ran SST, MST, and LST simultaneously 

• MST ran with no problems 

COGENT SST 

• Ran SST, MST, and LST simultaneously 

• SST ran with no problems 

DERMALOG LST 

• Networking was in a deadlock at one point. Rebooted and restarted. 

• Found that results from some subtests were affected by the network problem 

• Saved first results (with a few bad subtests) as Dermalog 1st run 

• Reran the affected subtests 

• Combined reprocessed subtest with good subtests from 1st run to form results analyzed 

in main report 

DERMALOG MST 

• Ran MST in less than an hour 

• MST ran with no apparent problems 

• (No problems during tests, but the resulting similarity matrix contained only zeros) 

GOLDEN FINGER LST 

• Ran LST over AC outage weekend 

• LST system crashed over AC outage weekend while trying to do a WSQ conversion.

Restarted ignoring problem file 

• No other problems 

GOLDEN FINGER MST 

• Ran MST with no problems 

GRIAULE LST 

• Some machines were not responding so had to reboot 9 nodes 

• Same problem again, had to reboot 9 nodes 

• Discovered that not all images were enrolled because the system didn’t recover after an

FTE; revised code 

• Reboot and restart required after problem in watchdog program 

• Finished LST with no further problems 

IDENTIX LST 

• Several nodes stuck on preprocessing; restarted 

• Failed to create templates for several files 

22 April 2004 Page 125 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

• System failed to automatically merge sim files from different nodes. Stopped processing,

restarted; still failed to merge. Wrote script to merge 

• Successfully completed LST 

IDENTIX MST 

• Ran MST with no problems 

MOTOROLA LST 

• Image processing got stuck; stopped and restarted processing 

• Image processing got stuck again; determined that 3 images were causing problems 

• Finally got one of the 3 images to process, still had problems with other two 

• Switched to Aware for WSQ decompression; images processed 

• LST matching progressed at 1/10th expected speed; stopped, restarted process 

• Reduced logging requirements, ran much faster 

• Processes stuck on all nodes; found secondary processing was using original WSQ 

decompression routine; changed to Aware; restarted 

• Still running slowly, applied patch; restarted 

• Applied another patch to fix logging problem; restarted 

• Problems with load balancing; revised configuration, restarted 

• System writing scores to both text log files and sim files; corrected 

• No further problems with LST 

MOTOROLA MST 

• Ran MST with no problems 

NEC LST 

• Found problem with one node; moved files to another node 

• Ran LST with no other problems 

NEC MST 

• Stopped processing over AC outage weekend, then restarted 

• Ran MST with no problems 

NEUROTECHNOLOGIJA MST 

• Software developers in Lithuania could not come due to visa problems, so sent US sales

rep to take the test 

• Had problems with WSQ decompression on some images 

• Modified script to skip images it can’t read; ran MST 

• Skipped 31 images in MST (first run) 

• Wiped disk, then updataed software and reran MST (2nd run) 

• First run results are reported in body of report (with skipped images) because second run

had results in error. 

22 April 2004 Page 126 of 127 
FPVTE 2003 APPENDIX C — SYSTEM-SPECIFIC RESULTS 

PHOENIX MST 

• Ran MST with no problems 

• After MST completed, the last sim file was not written to the CD (error run) 

• Closed message window saying run was complete and last sim file was written 

• The results in the body of the report include the last sim file 

RAYTHEON LST 

• Ran over the AC outage weekend with portable AC units 

• Had problem preprocessing a file; rebooted and restarted 

• Finished LST with no other problems 

RAYTHEON MST 

• Ran MST with no problems 

SAGEM LST 1 

• LST output also had trivial output—had to separate 

• No other problems running LST 

SAGEM LST 2 

• LST output also had trivial output—had to separate 

• No other problems running LST 

SAGEM MST 1 

• Ran MST with no problems 

SAGEM MST 2 

• RAM problems in one machine of 3 machine configuration 

• UPS battery found to be low 

• Later discovered real problem was transfer of files among the 3 machines 

• Saved partial results; reconfigured to use only 1 machine; reran 

• Results reported in body of report are from reconfigured 1 machine run 

TECHNOIMAGIA MST 

• Set up two redundant systems in case one crashed 

• Ran MST on both systems with no problems 

ULTRA-SCAN MST1 

• Set up two separate systems 

• Ran MST on with no problems 

ULTRA-SCAN MST 2 

• Set up two separate systems 

• Ran MST on with no problems 

22 April 2004 Page 127 of 127