FINGERPRINT VENDOR TECHNOLOGY EVALUATION 2003 

APPENDIX D 

DETAILS OF RESULTS 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Table of Contents 

1 Introduction .......................................................................................................................3 

2 LST.....................................................................................................................................4 

2.1 Detail of System Comparisons ..................................................................................................................... 4 

2.2 Source .............................................................................................................................................................10 

2.3 Image Type ....................................................................................................................................................12 

2.4 Number of Fingers .......................................................................................................................................15 

3 MST.................................................................................................................................. 17 

3.1 Detail of System Comparisons ...................................................................................................................17 

3.2 Equal Error Rates .........................................................................................................................................20 

3.3 Image Source .................................................................................................................................................20 

3.4 Image Quality ................................................................................................................................................24 

4 SST ...................................................................................................................................27 

4.1 Detail of System Comparisons ...................................................................................................................27 

4.2 Image Source .................................................................................................................................................28 

4.3 Image Quality ................................................................................................................................................31 

5 Other Results ...................................................................................................................33 

5.1 Relationship of Accuracy to System Size ..................................................................................................33 

5.2 Relationship of Accuracy to Processing Time .........................................................................................33 

5.3 1:1 Matches....................................................................................................................................................34 

22 April 2004 Page 2 of 35 
 

1 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Introduction 

This Appendix shows complete ROCs corresponding to much of the data that was 

summarized in the body of the report (usually at FAR = 10-4). 

• Each chart corresponds to one test partition, showing the ROC for each system on that 

partition. 

• Charts are included for many of the partitions that were summarized in the body of the 

report. In selecting LST partitions for inclusion in this appendix, those having a 

relatively large number of mated pairs were generally favored. 

• Cross references to summary charts in Section 6 of the main report (Key Results) are 

provided. Many of the charts and tables of Section 5 (Comparison of Systems) 

summarize accuracy across multiple partitions, many of which are included in this

appendix. 

• All charts in this appendix are based on partitions having zero or one mate in the gallery 

for each probe. 

• Unless otherwise noted, the range of the y-scale of these charts (TAR) is fixed at [0.5, 

1.0]. 

22 April 2004 Page 3 of 35 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

2 LST 

2.1 Detail of System Comparisons 

The following table defines the partitions used for the comparisons of LST system

comparison charts in Section 4 of the FpVTE Analysis Report. 

LST Partitions used for System Comparison

Probe Gallery Probe set # Fingers Query Set Target Set Partition Partition size Gallery set 

size # Mates 

BxA/BCC 1 B BCC A Inst1 1,584 5,805 1,196 

BxA/DHS2 1 B DHS2 A Inst1 790 5,805 393 

BxA/IdentIafis 1 B IdentIafis A Inst1 303 5,805 282 

BxC/IdentIafis 1 B IdentIafis C Inst1 303 7,983 209 

HxC/IdentIafis 1 H IdentIafis C Inst1 525 7,983 289 

HxD/12k 1 H 12k D Inst1 1,204 3,190 548 

 27 Partitions of Operational DataHxI/IdentIafis 

HxJ/12k 

AxA/BCC 

AxA/DHS2 

GxC/IdentIafis 

GxD/12k 

GxI/12k 

GxI/BEN 

GxI/IdentIafis 

GxJ/12k 

CxD/12k 

CxI/12k 

CxI/BEN 

CxI/IdentIafis 

CxJ/12k 

1 H IdentIafis I Inst1 525 

1 H 12k J Inst1 1,204 

2 A BCC A Inst2 4,184 

2 A DHS2 A Inst2 835 

2 G IdentIafis C Inst1 2,280 

2 G 12k D Inst1 2,226 

2 G 12k I Inst1 2,226 

2 G BEN I Inst1 1,199 

2 G IdentIafis I Inst1 2,280 

2 G 12k J Inst1 2,226 

10 C 12k D Inst1 2,201 

10 C 12k I Inst1 2,201 

10 C BEN I Inst1 2,357 

10 C IdentIafis I Inst1 2,765 

10 C 12k J Inst1 2,201 

7,051 

7,329 

2,119 

2,119 

7,983 

3,190 

7,051 

7,051 

7,051 

7,329 

3,190 

7,051 

7,051 

7,051 

7,329 

393 

808 

1,783 

334 

1,482 

337 

409 

403 

1,255 

1,472 

722 

1,482 

852 

1,638 

537 

DxD/DHS10 10 D DHS10 D Inst2 998 792 498 

DxI/12k 10 D 12k I Inst1 1,486 7,051 810 

DxJ/12k 10 D 12k J Inst1 1,486 7,329 380 

DxJ/DHS10 10 D DHS10 J Inst1 998 7,329 498 

IxI/IdentIafis 10 I IdentIafis I Inst2 2,069 866 762 

JxJ/DHS10 10 J DHS10 J Inst2 3,836 643 359 

HxC/Ohio 1 H Ohio C Inst1 988 7,983 610 

 17 Partitions of Controlled Data HxD/Ohio 

HxI/Ohio 

HxJ/Ohio 

GxC/Ohio 

GxD/Ohio 

GxI/Ohio 

GxJ/Ohio 

CxC/Ohio 

CxD/Ohio 

CxI/Ohio 

CxJ/Ohio 

DxD/Ohio 

DxI/Ohio 

DxJ/Ohio 

1 H Ohio D Inst1 988 

1 H Ohio I Inst1 988 

1 H Ohio J Inst1 988 

2 G Ohio C Inst1 1,000 

2 G Ohio D Inst1 1,000 

2 G Ohio I Inst1 1,000 

2 G Ohio J Inst1 1,000 

10 C Ohio C Inst2 660 

10 C Ohio D Inst1 660 

10 C Ohio I Inst1 660 

10 C Ohio J Inst1 660 

10 D Ohio D Inst2 706 

10 D Ohio I Inst1 706 

10 D Ohio J Inst1 706 

3,190 

7,051 

7,329 

7,983 

3,190 

7,051 

7,329 

336 

3,190 

7,051 

7,329 

792 

7,051 

7,329 

724 

915 

722 

632 

759 

923 

760 

250 

497 

551 

496 

294 

605 

703 

IxJ/Ohio 10 I Ohio J Inst1 799 7,329 605 

JxJ/Ohio 10 J Ohio J Inst2 707 643 283 

Table D-1 

22 April 2004 Page 4 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

The following table shows the detail used to construct the LST system comparison charts in

Section 4 of the FpVTE Analysis Report. These results correspond directly to Figures 5 and 

6 in that document. 

The systems are sorted by the average TAR across the 27 operational partitions. 

NEC 

Comparison of Performance on LST Partitions

Values are True Accept Rates where the False Accept Rate = 10-4 

Sagem L1 Cogent Sagem L2 Dermalog Motorola Identix NIST VTB Biolink Griaule Golden 

Finger Antheus Raytheon 

 27 Partitions of Operational Data

BxA/BCC 

BxA/DHS2 

BxA/IdentIafis 

BxC/IdentIafis 

HxC/IdentIafis 

HxD/12k 

HxI/IdentIafis 

HxJ/12k 

AxA/BCC 

AxA/DHS2 

GxC/IdentIafis 

GxD/12k 

GxI/12k 

GxI/BEN 

GxI/IdentIafis 

GxJ/12k 

CxD/12k 

CxI/12k 

CxI/BEN 

CxI/IdentIafis 

CxJ/12k 

DxD/DHS10 

DxI/12k 

DxJ/12k 

DxJ/DHS10 

IxI/IdentIafis 

JxJ/DHS10 

0.9913 

0.9884 

0.9955 

1.0000 

0.9873 

0.9825 

0.9918 

0.9895 

0.9959 

1.0000 

0.9973 

1.0000 

1.0000 

1.0000 

0.9973 

0.9990 

0.9999 

1.0000 

0.9993 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

0.9996 

1.0000 

1.0000 

0.9786 0.9711 0.9629 0.9436 0.9299 0.8931 0.8253 0.8190 

0.9629 0.9726 0.9395 0.9463 0.9070 0.8006 0.8088 0.6585 

0.9863 0.9851 0.9838 0.9636 0.9508 0.9235 0.8802 0.7888 

1.0000 0.9785 0.9844 0.9858 0.9262 0.8665 0.8745 0.7554 

0.9701 0.9664 0.9565 0.9372 0.9290 0.8914 0.8417 0.8262 

0.9611 0.9587 0.9445 0.9014 0.8690 0.8282 0.7467 0.7891 

0.9504 0.9528 0.9360 0.9051 0.9188 0.7390 0.7732 0.7354 

0.9743 0.9525 0.9639 0.9233 0.8912 0.8135 0.7699 0.7140 

0.9904 0.9867 0.9875 0.9717 0.9692 0.9516 0.9060 0.8969 

1.0000 0.9979 1.0000 0.9987 0.9989 0.9590 0.9785 0.7175 

0.9941 0.9921 0.9933 0.9848 0.9802 0.9680 0.9416 0.9315 

0.9923 0.9918 0.9902 0.9645 0.9519 0.9236 0.8746 0.8430 

0.9991 1.0000 0.9991 0.9990 0.9970 0.9717 0.9785 0.9661 

1.0000 0.9968 1.0000 0.9983 0.9937 0.9768 0.9720 0.9665 

0.9801 0.9824 0.9746 0.9529 0.9664 0.8925 0.8569 0.8461 

0.9987 0.9925 0.9955 0.9788 0.9681 0.9262 0.8657 0.8507 

1.0000 1.0000 1.0000 0.9851 0.9898 0.9851 0.9404 0.9615 

1.0000 1.0000 1.0000 0.9999 0.9993 0.9944 0.9819 0.9967 

0.9997 0.9983 0.9997 0.9949 0.9926 0.9840 0.9732 0.9652 

1.0000 0.9997 0.9998 0.9902 0.9970 0.9870 0.9373 0.9742 

0.9990 1.0000 0.9990 0.9902 0.9959 0.9839 0.9170 0.9721 

0.9993 1.0000 0.9992 0.9952 0.9968 0.9946 0.9444 0.9675 

0.9995 1.0000 1.0000 0.9909 0.9904 0.9629 0.8722 0.9731 

1.0000 0.9969 1.0000 0.9966 0.9955 0.9936 0.9378 0.9775 

0.9994 0.9983 0.9975 0.9883 0.9915 0.9873 0.9070 0.9428 

0.9998 1.0000 0.9998 0.9939 0.9983 0.9913 0.9482 0.9825 

1.0000 1.0000 1.0000 0.9954 0.9993 0.9861 0.9431 0.9682 

0.8826 

0.7070 

0.8790 

0.8964 

0.8399 

0.7779 

0.7900 

0.7927 

0.9297 

0.8752 

0.9460 

0.8980 

0.9872 

0.9846 

0.8909 

0.9028 

0.9043 

0.9825 

0.9666 

0.8856 

0.8970 

0.9362 

0.8786 

0.9319 

0.8832 

0.7190 

0.8121 

0.7940 

0.7222 

0.7654 

0.8092 

0.8490 

0.7412 

0.4415 

0.8043 

0.8338 

0.8698 

0.9357 

0.8278 

0.9765 

0.9548 

0.5775 

0.8739 

0.9725 

0.9940 

0.9761 

0.9289 

0.9715 

0.9386 

0.9525 

0.9652 

0.9338 

0.8744 

0.9703 

0.8174 

0.6148 

0.7405 

0.7486 

0.8052 

0.6565 

0.7503 

0.6150 

0.8997 

0.7919 

0.9143 

0.8027 

0.9214 

0.9308 

0.8625 

0.7022 

0.9562 

0.9806 

0.9597 

0.9758 

0.8469 

0.9672 

0.9416 

0.8378 

0.8262 

0.9678 

0.7449 

0.7940 

0.7222 

0.7654 

0.8092 

0.8490 

0.7412 

0.1164 

0.8043 

0.8338 

0.8698 

0.9357 

0.8278 

0.9797 

0.9609 

0.1838 

0.8739 

0.9725 

0.9935 

0.9748 

0.8099 

0.9715 

0.9386 

0.9484 

0.9651 

0.9338 

0.7646 

0.9703 

Average 

Median 

0.9968 

0.9999 

0.9902 0.9878 0.9854 0.9732 0.9664 0.9324 0.8962 0.8810 

0.9991 0.9968 0.9975 0.9858 0.9898 0.9629 0.9070 0.9315 

0.8806 

0.8909 

0.8613 

0.8744 

0.8362 

0.8378 

0.8263 

0.8698 

 17 Partitions of Controlled Data 

HxC/Ohio 

HxD/Ohio 

HxI/Ohio 

HxJ/Ohio 

GxC/Ohio 

GxD/Ohio 

GxI/Ohio 

GxJ/Ohio 

CxC/Ohio 

CxD/Ohio 

CxI/Ohio 

CxJ/Ohio 

DxD/Ohio 

DxI/Ohio 

DxJ/Ohio 

IxJ/Ohio 

JxJ/Ohio 

0.9996 

0.9896 

0.9978 

0.9983 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

0.9939 0.9926 0.9855 0.9841 0.9575 0.9423 0.9202 0.8929 

0.9637 0.9699 0.9508 0.9347 0.9075 0.8511 0.8034 0.8238 

0.9842 0.9811 0.9795 0.9748 0.9715 0.9253 0.8721 0.8553 

0.9783 0.9801 0.9669 0.9560 0.9452 0.8745 0.7765 0.7230 

1.0000 1.0000 1.0000 0.9983 1.0000 0.9958 0.9979 0.9915 

0.9940 0.9992 0.9927 0.9892 0.9885 0.9620 0.9517 0.9487 

1.0000 1.0000 1.0000 1.0000 1.0000 0.9913 0.9857 0.9696 

1.0000 0.9983 1.0000 0.9948 0.9952 0.9822 0.9606 0.9510 

1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 

1.0000 1.0000 1.0000 0.9971 0.9989 0.9997 0.9822 0.9990 

1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 0.9958 1.0000 

1.0000 1.0000 1.0000 0.9988 1.0000 1.0000 0.9842 1.0000 

1.0000 1.0000 1.0000 0.9918 1.0000 1.0000 0.9886 0.9987 

1.0000 1.0000 1.0000 0.9993 0.9997 0.9954 0.9471 0.9971 

1.0000 1.0000 1.0000 0.9992 0.9995 1.0000 0.9728 0.9926 

1.0000 1.0000 1.0000 1.0000 1.0000 0.9990 0.9916 0.9989 

1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 

0.9387 

0.8576 

0.9285 

0.8685 

0.9985 

0.9541 

0.9901 

0.9690 

0.9966 

0.9638 

0.9932 

0.9672 

0.9672 

0.9580 

0.9599 

0.9111 

0.9592 

0.9155 

0.7885 

0.9134 

0.8382 

0.9905 

0.9284 

0.9848 

0.9731 

1.0000 

0.9908 

0.9985 

1.0000 

0.9850 

0.9885 

0.9915 

0.9931 

1.0000 

0.7863 

0.6940 

0.7665 

0.6220 

0.9772 

0.9015 

0.9580 

0.8270 

1.0000 

0.9841 

0.9962 

0.8807 

0.9922 

0.9805 

0.8781 

0.8799 

0.8494 

0.9155 

0.7885 

0.9158 

0.8368 

0.9905 

0.9284 

0.9862 

0.9718 

1.0000 

0.9908 

0.9986 

0.9985 

0.9850 

0.9853 

0.9901 

0.9906 

1.0000 

Average 

Median 

0.9991 

1.0000 

0.9949 0.9954 0.9927 0.9893 0.9861 0.9717 0.9489 0.9495 

1.0000 1.0000 1.0000 0.9983 0.9997 0.9958 0.9822 0.9926 

0.9518 

0.9599 

0.9576 

0.9885 

0.8808 

0.8807 

0.9572 

0.9862 

Average (all 44) 

Median (all 44) 

0.9977 

1.0000 

0.9920 0.9907 0.9882 0.9794 0.9740 0.9476 0.9165 0.9075 

0.9996 0.9983 0.9995 0.9906 0.9932 0.9795 0.9410 0.9633 

0.9081 

0.9198 

0.8985 

0.9372 

0.8535 

0.8703 

0.8769 

0.9372 

Table D-2 

22 April 2004 Page 5 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Table D-3 shows the detail used to construct the LST system rank comparison tables in 

Section 4 of the FpVTE Analysis Report. These results correspond directly to Tables 14 and 

15 in that document. The systems are sorted in the same order as in Table D-2. 

Comparison of Performance Ranks on LST Partitions 

Values are ranks, based on True Accept Rates where the False Accept Rate = 10-4

NEC Sagem L1 Cogent Sagem L2 Dermalog Motorola Identix NIST VTB Biolink Griaule Golden 

Finger Antheus Raytheon 

 27 Partitions of Operational Data

BxA/BCC 

BxA/DHS2 

BxA/IdentIafis 

BxC/IdentIafis 

HxC/IdentIafis 

HxD/12k 

HxI/IdentIafis 

HxJ/12k 

AxA/BCC 

AxA/DHS2 

GxC/IdentIafis 

GxD/12k 

GxI/12k 

GxI/BEN 

GxI/IdentIafis 

GxJ/12k 

CxD/12k 

CxI/12k 

CxI/BEN 

CxI/IdentIafis 

CxJ/12k 

DxD/DHS10 

DxI/12k 

DxJ/12k 

DxJ/DHS10 

IxI/IdentIafis 

JxJ/DHS10 

1 2 3 4 5 6 7 9 10 8 

1 3 2 5 4 6 8 7 12 11 

1 2 3 4 5 6 7 8 10 9 

1 1 5 4 3 6 9 8 12 7 

1 2 3 4 5 6 7 10 12 11 

1 2 3 4 5 6 7 10 8 9 

1 3 2 4 6 5 10 8 11 7 

1 2 4 3 5 6 7 11 12 10 

1 2 4 3 5 6 7 9 11 8 

1 1 6 1 5 4 8 7 13 9 

1 2 4 3 5 6 7 9 12 8 

1 2 3 4 5 6 7 9 10 8 

1 3 1 4 5 6 11 9 12 7 

1 1 5 1 4 6 8 9 10 7 

1 3 2 4 6 5 7 10 11 8 

1 2 4 3 5 6 7 11 12 8 

4 1 1 1 6 5 7 12 10 13 

1 1 1 1 5 6 8 12 7 11 

3 2 4 1 5 6 7 10 12 11 

1 1 4 3 6 5 7 10 9 12 

1 3 1 4 6 5 7 11 8 12 

1 3 1 4 6 5 7 10 8 13 

1 4 1 1 5 6 8 13 7 12 

1 1 4 1 5 6 7 11 8 12 

1 2 3 4 6 5 7 11 8 12 

1 4 1 3 6 5 7 10 8 13 

1 1 1 1 6 5 7 11 10 12 

12 

9 

11 

10 

8 

11 

12 

8 

12 

10 

10 

11 

10 

12 

12 

9 

8 

9 

8 

11 

9 

11 

9 

9 

9 

11 

8 

11 

13 

13 

13 

13 

13 

9 

13 

10 

12 

13 

13 

13 

13 

9 

13 

11 

13 

13 

8 

13 

9 

11 

13 

13 

9 

13 

12 

9 

11 

10 

8 

11 

13 

8 

12 

10 

10 

11 

8 

11 

13 

9 

8 

10 

9 

13 

10 

11 

10 

10 

10 

12 

8 

Average 

Median 

1.2 2.1 2.8 2.9 5.2 5.6 7.5 9.8 10.1 9.9 

1.0 2.0 3.0 3.0 5.0 6.0 7.0 10.0 10.0 10.0 

10.0 

10.0 

11.9 

13.0 

10.3 

10.0 

 17 Partitions of Controlled Data 

HxC/Ohio 

HxD/Ohio 

HxI/Ohio 

HxJ/Ohio 

GxC/Ohio 

GxD/Ohio 

GxI/Ohio 

GxJ/Ohio 

CxC/Ohio 

CxD/Ohio 

CxI/Ohio 

CxJ/Ohio 

DxD/Ohio 

DxI/Ohio 

DxJ/Ohio 

IxJ/Ohio 

JxJ/Ohio 

1 2 3 4 5 6 7 9 12 8 

1 3 2 4 5 6 8 10 9 7 

1 2 3 4 5 6 8 11 12 7 

1 3 2 4 5 6 7 11 12 8 

1 1 1 1 7 1 9 8 10 6 

1 3 2 4 5 6 7 9 10 8 

1 1 1 1 1 1 7 10 12 8 

1 1 4 1 6 5 7 11 12 10 

1 1 1 1 1 1 1 1 1 13 

1 1 1 1 8 7 5 12 6 13 

1 1 1 1 1 1 1 12 1 13 

1 1 1 1 9 1 1 11 1 12 

1 1 1 1 9 1 1 10 7 13 

1 1 1 1 6 5 8 13 7 12 

1 1 1 1 7 6 5 11 8 12 

1 1 1 1 1 1 7 10 8 12 

1 1 1 1 1 1 1 1 1 12 

10 

11 

10 

9 

11 

11 

11 

8 

1 

9 

10 

1 

11 

9 

9 

9 

1 

13 

13 

13 

13 

13 

13 

13 

13 

1 

11 

11 

13 

8 

11 

13 

13 

13 

10 

11 

9 

10 

11 

11 

9 

9 

1 

9 

9 

10 

11 

10 

10 

11 

1 

Average 

Median 

1.0 1.5 1.6 1.9 4.8 3.6 5.3 9.4 7.6 10.2 

1.0 1.0 1.0 1.0 5.0 5.0 7.0 10.0 8.0 12.0 

8.3 

9.0 

11.6 

13.0 

8.9 

10.0 

Average (all 44) 

Median (all 44) 

1.1 1.8 2.3 2.5 5.0 4.8 6.7 9.7 9.1 10.0 

1.0 2.0 2.0 3.0 5.0 6.0 7.0 10.0 10.0 10.5 

9.3 

10.0 

11.8 

13.0 

9.8 

10.0 

Table D-3 

22 April 2004 Page 6 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

The following charts show the TAR information graphically. In both charts, the systems are

sorted by the average performance across the 27 operational partitions. 

Figure D-1 

Figure D-2 

The comparisons of LST systems in the Analysis Report were made at FAR = 10-4. The 

reasons for the choice of this operating point are explained in Appendix E, Section 2. One 

22 April 2004 Page 7 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

of the reasons for the choice of this operating point was that the comparisons are generally 

valid for lower FARs. The following tables show what the effect would have been if LST 

systems had been compared at FAR = 10-6 instead of 10-4. Table D-4 corresponds directly to

Table D-2, except that the FAR is 10-6 instead of 10-4. 

NEC Sagem L1 

Comparison of Performance on LST Partitions 

Values are True Accept Rates where the False Accept Rate = 10-6 

Cogent Sagem L2 Dermalog Motorola Identix NIST VTB Griaule Biolink Antheus Golden 

Finger Raytheon 

27 Partitions of Operational Data

AxA/BCC 

AxA/DHS2 

BxA/BCC 

BxA/DHS2 

BxA/IdentIafis 

BxC/IdentIafis 

CxD/12k 

CxI/12k 

CxI/BEN 

CxI/IdentIafis 

CxJ/12k 

DxD/DHS10 

DxI/12k 

DxJ/12k 

DxJ/DHS10 

GxC/IdentIafis 

GxD/12k 

GxI/12k 

GxI/BEN 

GxI/IdentIafis 

GxJ/12k 

HxC/IdentIafis 

HxD/12k 

HxI/IdentIafis 

HxJ/12k 

IxI/IdentIafis 

JxJ/DHS10 

0.99175 

1.00000 

0.98096 

0.97726 

0.98528 

1.00000 

0.99939 

1.00000 

0.99889 

0.99991 

1.00000 

1.00000 

0.99850 

1.00000 

0.99825 

0.99567 

0.99976 

1.00000 

1.00000 

0.99339 

0.99593 

0.97311 

0.96380 

0.97943 

0.97753 

1.00000 

1.00000 

0.98676 

1.00000 

0.96581 

0.94440 

0.98277 

0.96875 

1.00000 

1.00000 

0.99933 

1.00000 

0.99847 

0.99875 

0.99923 

1.00000 

0.99883 

0.99331 

0.98904 

0.99847 

1.00000 

0.97790 

0.99516 

0.96550 

0.94865 

0.93716 

0.96624 

0.99937 

1.00000 

0.98220 0.98317 0.94818 0.96123 0.92588 0.84478 0.87341 

0.99669 1.00000 0.98885 0.99097 0.92409 0.93403 0.80096 

0.96922 0.94494 0.89098 0.89744 0.84174 0.73939 0.80908 

0.96339 0.91504 0.87589 0.83190 0.70968 0.69163 0.62684 

0.98395 0.96911 0.94615 0.92563 0.86518 0.76106 0.81008 

0.97115 0.95793 0.90243 0.88928 0.77004 0.70021 0.82081 

1.00000 1.00000 0.98293 0.98781 0.97427 0.92010 0.83798 

1.00000 1.00000 0.99970 0.99889 0.99182 0.97785 0.96256 

0.99766 0.99934 0.99267 0.99025 0.97668 0.96410 0.94677 

0.99935 0.99962 0.98774 0.99448 0.97897 0.92085 0.82098 

0.99846 0.99846 0.98808 0.99468 0.97100 0.89726 0.81546 

1.00000 0.99874 0.99470 0.99575 0.98989 0.93554 0.88730 

0.99977 1.00000 0.98758 0.98801 0.94311 0.83882 0.80281 

0.99604 1.00000 0.99538 0.99321 0.98993 0.92313 0.88676 

0.99683 0.99687 0.98663 0.98907 0.98224 0.89795 0.81254 

0.98533 0.99234 0.97722 0.97826 0.94398 0.87779 0.91326 

0.98271 0.98375 0.93990 0.92803 0.85468 0.75428 0.83791 

1.00000 0.99846 0.99703 0.99544 0.94054 0.93528 0.96571 

0.99612 0.99870 0.99366 0.98769 0.96476 0.95507 0.97387 

0.97964 0.97034 0.92212 0.96040 0.84807 0.76583 0.83645 

0.99051 0.99404 0.94827 0.96241 0.87302 0.78323 0.83032 

0.95998 0.94906 0.89418 0.86672 0.83537 0.73344 0.79259 

0.95403 0.91714 0.83525 0.76530 0.71447 0.59465 0.69868 

0.94908 0.91648 0.81982 0.87273 0.66754 0.59453 0.71251 

0.94882 0.94905 0.86317 0.83505 0.72112 0.63932 0.70276 

1.00000 0.99937 0.99327 0.99794 0.97617 0.93764 0.60415 

1.00000 1.00000 0.99344 0.99849 0.98375 0.93919 0.72760 

0.84282 

0.59422 

0.72854 

0.48770 

0.68322 

0.63317 

0.93725 

0.99354 

0.95167 

0.95168 

0.94234 

0.95763 

0.94425 

0.94709 

0.90798 

0.89566 

0.77539 

0.94918 

0.94980 

0.76815 

0.77609 

0.73933 

0.68370 

0.60305 

0.57096 

0.94992 

0.92623 

0.80305 

0.53871 

0.72359 

0.39669 

0.60486 

0.65985 

0.92476 

0.96548 

0.93717 

0.95232 

0.80776 

0.93568 

0.89041 

0.81550 

0.78372 

0.87198 

0.70443 

0.85387 

0.89582 

0.78532 

0.62182 

0.72261 

0.57335 

0.64941 

0.51700 

0.94775 

0.70739 

0.73899 

0.70500 

0.64607 

0.52433 

0.57994 

0.34554 

0.91281 

0.97069 

0.94784 

0.87225 

0.93681 

0.88612 

0.89132 

0.93415 

0.83427 

0.88008 

0.67505 

0.91098 

0.91573 

0.50763 

0.79826 

0.75452 

0.56332 

0.39756 

0.69285 

0.42506 

0.93226 

0.73899 

0.70500 

0.64607 

0.52433 

0.57994 

0.34554 

0.91281 

0.97822 

0.94943 

0.71012 

0.93681 

0.88612 

0.90905 

0.93415 

0.83427 

0.88008 

0.67505 

0.93093 

0.91799 

0.15301 

0.79826 

0.75452 

0.56332 

0.09745 

0.69285 

0.67466 

0.93226 

 17 Partitions of Controlled Data 

Average 

Median 

CxD/Ohio 

CxI/Ohio 

CxJ/Ohio 

DxI/Ohio 

DxJ/Ohio 

GxC/Ohio 

GxD/Ohio 

GxI/Ohio 

GxJ/Ohio 

HxC/Ohio 

HxD/Ohio 

HxI/Ohio 

HxJ/Ohio 

IxJ/Ohio 

CxC/Ohio 

DxD/Ohio 

0.99292 

0.99889 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

0.99992 

0.99848 

0.98208 

0.99695 

0.99649 

1.00000 

1.00000 

1.00000 

0.98570 0.98522 0.97896 

0.99847 0.99604 0.99687 

1.00000 1.00000 1.00000 

1.00000 1.00000 1.00000 

1.00000 

1.00000 

1.00000 

1.00000 

0.99318 

1.00000 

0.99923 

0.98814 

0.95466 

0.98370 

0.97558 

1.00000 

1.00000 

1.00000 

0.94982 0.94730 0.89474 0.83174 0.81889 

0.98293 0.98769 0.94054 0.87779 0.82081 

0.99668 0.99811 0.99801 0.97825 0.93587 

1.00000 1.00000 0.99840 0.99392 0.98671 

1.00000 1.00000 0.99820 1.00000 0.99863 0.97650 0.93115 

1.00000 1.00000 0.99891 0.99916 0.99317 0.93629 0.90908 

1.00000 1.00000 0.99875 0.99882 0.99895 0.96447 0.90844 

1.00000 1.00000 0.99718 1.00000 0.99051 0.99138 0.99051 

0.99879 0.99151 0.97770 0.98452 0.93967 0.91312 0.91879 

1.00000 1.00000 0.99738 0.99944 0.98244 0.96295 0.98086 

0.99757 0.99912 0.98388 0.99224 0.96381 0.90268 0.93438 

0.98849 0.97977 0.96120 0.92347 0.88835 0.87533 0.89493 

0.95880 0.92951 0.88607 0.84388 0.76819 0.69194 0.78500 

0.97732 0.97527 0.95518 0.93984 0.84319 0.75404 0.86640 

0.97420 0.95326 0.91450 0.89211 0.77810 0.60163 0.76667 

1.00000 1.00000 1.00000 1.00000 0.99740 0.99013 0.85751 

1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 0.98585 

1.00000 1.00000 0.99102 1.00000 0.99885 0.98545 0.93039 

0.81817 

0.89566 

0.99246 

0.99945 

0.99784 

0.99053 

0.98648 

0.98659 

0.92404 

0.95484 

0.92656 

0.83934 

0.73573 

0.78288 

0.58447 

0.99788 

1.00000 

0.99267 

0.76260 

0.78532 

0.96589 

0.99347 

0.86771 

0.95839 

0.87123 

0.96577 

0.84283 

0.92812 

0.76905 

0.73714 

0.58660 

0.72066 

0.51905 

0.87097 

0.99863 

0.96860 

0.74739 

0.79826 

0.96228 

0.99227 

0.98962 

0.97400 

0.97956 

0.97603 

0.87262 

0.95460 

0.92835 

0.79169 

0.63655 

0.80866 

0.66966 

0.98928 

1.00000 

0.97173 

0.72819 

0.75452 

0.96228 

0.99579 

0.98760 

0.97279 

0.97672 

0.97603 

0.87262 

0.97736 

0.92704 

0.79169 

0.63655 

0.83139 

0.66827 

0.98763 

1.00000 

0.97173 

JxJ/Ohio 

Average 

Median 

Min (all 44) 

Average (all 44) 

Median (all 44) 

Max (all 44) 

1.00000 

0.99847 

1.00000 

0.96380 

0.99506 0.98883 

0.99996 0.99903 

1.00000 1.00000 

1.00000 1.00000 1.00000 

0.99379 

1.00000 

0.93716 0.94882 0.91504 

0.98855 0.98319 

0.99762 0.99872 

1.00000 1.00000 

1.00000 1.00000 1.00000 1.00000 0.93875 

0.99383 0.98991 0.97980 0.97480 0.94927 0.91283 0.91302 

1.00000 1.00000 0.99718 0.99916 0.99317 0.96447 0.93039 

0.81982 0.76530 0.66754 0.59453 0.60415 

0.96141 0.95792 0.91581 0.86307 0.85526 

0.98766 0.98966 0.96428 0.91661 0.86195 

1.00000 1.00000 1.00000 1.00000 0.99051 

1.00000 

0.92304 

0.98659 

0.48770 

0.85869 

0.93190 

1.00000 

0.81200 

0.84565 

0.87097 

0.39669 

0.79469 

0.81375 

0.99863 

1.00000 

0.91158 

0.97173 

0.34554 

0.81083 

0.88310 

1.00000 

1.00000 

0.91385 

0.97279 

0.09745 

0.79993 

0.88310 

1.00000 

Table D-4 

22 April 2004 Page 8 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Table D-5 and Table D-6 correspond to Tables 14 and 15 in the Analysis Report. The left

half of each table summarizes the results where FAR = 10-6, and the right half where FAR = 

10-4. Systems for which the average rank would have changed are shown in yellow. These 

tables are sorted by the average rank where FAR = 10-4 (the same order as in Tables 14 and 

15 in the Analysis Report). 

Summary of Rank over LST Partitions where FAR = 10-6 Summary of Rank over LST Partitions where FAR = 10-4 

27 LST Partitions of Operational Data 27 LST Partitions of Operational Data 

Best 5th Best Average Median 5th Worst Worst Best 5th Best Average Median 5th Worst Worst 

NEC 1 1 1.37 1 2 4 1 1 1.19 1 1

Sagem L1 1 1 2.15 2 3 4 1 1 2.07 2 3

Cogent 1 1 2.70 2 4 4 1 1 2.81 3 4 6 

Sagem L2 1 1 2.85 3 4 4 1 1 2.93 3 4

Dermalog 5 5 5.56 6 6 6 3 5 5.19 5 6

Motorola 5 5 5.44 5 6 6 4 5 5.59 6 6

Identix 7 7 7.30 7 8 9 7 7 7.52 7 8 11 

NIST VTB 7 9 9.89 10 11 12 7 8 9.81 10 11 

Griaule 7 8 9.63 8 12 13 7 8 9.93 10 12 13 

Golden Finger 9 9 10.85 11 12 13 8 8 9.96 10 12 

Biolink 7 8 9.48 9 11 12 7 8 10.11 10 12

Raytheon 9 9 10.70 11 12 13 8 8 10.26 10 12 13 

Antheus 8 9 11.48 12 13 13 8 9 11.85 13 13 13 

4

4

5

6

6

13 

12 

13

Table D-5 

Summary of Rank over LST Partitions where FAR = 10-6 

17 LST Partitions of Controlled (Ohio) Data 

Best 3rd Best Average Median 3rd Worst Worst 

Summary of Rank over LST Partitions where FAR = 10-4 

17 LST Partitions of Controlled (Ohio) Data 

Best 3rd Best Average Median 3rd Worst Worst 

NEC 

Sagem L1 

Cogent 

Sagem L2 

Motorola 

Dermalog 

Identix 

Biolink 

1 1 1.00 1 1 1 

1 1 1.53 1 3 3 

1 1 1.59 1 3 4 

1 1 2.00 1 4 4 

1 1 3.65 5 6 6 

1 1 4.88 6 7 8 

1 5 6.35 7 8 8 

1 7 8.06 8 11 12 

1 9 9.18 10 11 12 

1 9 9.12 10 11 11 

1 7 9.06 10 12 12 

7 7 10.41 12 13 13 

10 11 12.41 13 13 13 

1 1 1.00 1 1 1 

1 1 1.47 1 3 3 

1 1 1.59 1 3 4 

1 1 1.88 1 4 4 

1 1 3.59 5 6 7 

1 1 4.82 5 8 9 

1 1 5.29 7 8 9 

1 1 7.59 8 12 12 

1 1 8.29 9 11 11 

1 9 8.94 10 11 11 

1 8 9.41 10 12 13 

6 7 10.24 12 13 13 

1 11 11.65 13 13 13 

Golden Finger 

Raytheon 

NIST VTB 

Griaule 

Antheus 

Table D-6 

22 April 2004 Page 9 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

2.2 Source 

The following three charts correspond to Figure 14 of the main report, showing accuracy on

single-finger, flat LST data. Note that the number of mated pairs is lower on the second two 

charts. Again, the Golden Finger and Raytheon results are identical, and therefore are 

superimposed. 

Figure D-3 

22 April 2004 Page 10 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-4 

Figure D-5 

22 April 2004 Page 11 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

2.3 Image Type 

The next four charts correspond to Figure 22 of the main report, showing accuracy on

various combinations of Slap/Rolled and Live/Paper data, using the non-operational Ohio 

dataset. 

Figure D-6 

22 April 2004 Page 12 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-7 

Figure D-8 

22 April 2004 Page 13 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-9 

22 April 2004 Page 14 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

2.4 Number of Fingers 

The next three charts correspond to Figure 17 of the main report, showing accuracy on 1, 2 

and 10 fingers (FBI 12k, Slap-Live x Slap-Paper). 

Figure D-10 

22 April 2004 Page 15 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-11 

Figure D-12 

22 April 2004 Page 16 of 35 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

3 MST 

3.1 Detail of System Comparisons 

The following table defines the partitions and shows the detail used to construct the MST 

system comparison charts in Section 4 of the FpVTE Analysis Report. These results 

correspond directly to Figure 7 in that document. 

The MST Standard probe partition (instance 1) includes one fingerprint for each of 5800 

subjects; the probe partitions are simply those 5800 fingerprints grouped by source. 

The MST Standard gallery partition (instance 2) includes one fingerprint for each of 3240 of 

the subjects in the Standard probe partition; the flat and slap gallery partitions are simply

those 3240 fingerprints grouped by type (slap or flat). 

The systems are sorted by performance on the MST Standard partition. 

Comparison of Performance on MST Partitions 

Values are True Accept Rates where the False Accept Rate = 10-4 

Probe set 

Gallery set 

BCC DHS2 II Flats II Flats BEN II Slaps Ohio 

Flat2 Flat2 Flat2 Slap2 Slap2 Slap2 Slap2 

Standard 

(Inst1) 

Standard 

(Inst2) 

Probe set size 

Gallery set size 

# Mates 

1514 733 517 517 696 1458 882 

1915 1915 1915 1325 1325 1325 1325 

1062 493 116 216 99 321 689 

5800 

3240 

3240 

NEC 0.996 0.996 1.000 0.985 0.986 0.996 1.000 0.994 

Cogent 0.990 0.999 1.000 0.977 0.972 0.986 1.000 0.991 

SAGEM M2 0.979 0.991 0.996 0.957 0.972 0.981 0.999 0.983 

SAGEM M1 0.970 0.983 0.989 0.954 0.972 0.977 0.994 0.976 

Neurotech. M1 0.938 0.900 0.972 0.915 0.925 0.949 0.980 0.939 

Motorola 0.944 0.939 0.971 0.866 0.892 0.923 0.984 0.934 

Identix 0.907 0.853 0.934 0.830 0.831 0.887 0.959 0.895 

UltraScan M2 0.841 0.818 0.945 0.779 0.881 0.913 0.940 0.863 

UltraScan M1 0.841 0.802 0.945 0.780 0.881 0.913 0.940 0.860 

NIST VTB 0.831 0.834 0.881 0.770 0.808 0.852 0.949 0.850 

Biolink 0.874 0.564 0.822 0.823 0.853 0.883 0.964 0.835 

Antheus 0.814 0.621 0.777 0.717 0.632 0.795 0.883 0.773 

Phoenix 0.710 0.616 0.634 0.646 0.649 0.727 0.951 0.725 

Golden Finger 0.670 0.627 0.700 0.596 0.563 0.758 0.904 0.707 

Raytheon 0.670 0.627 0.700 0.596 0.563 0.758 0.904 0.707 

123 ID M2 0.652 0.496 0.651 0.599 0.653 0.689 0.904 0.673 

Technoimagia 0.794 0.018 0.494 0.673 0.728 0.863 0.943 0.575 

Avalon 0.610 0.275 0.558 0.410 0.506 0.553 0.652 0.529 

Table D-7 

22 April 2004 Page 17 of 35 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

The following table shows the detail used to construct the MST system rank comparison

tables in Section 4 of the FpVTE Analysis Report. These results correspond directly to Table 

16 in that document. The systems are sorted in the same order as the previous table. 

Comparison of Performance Ranks on MST Partitions 

Values are Ranks, based on True Accept Rates where the False Accept Rate = 10-4 

Standard Probe set BCC DHS2 II Flats II Flats BEN II Slaps Ohio (Inst1) Average Median Standard Gallery set Flat2 Flat2 Flat2 Slap2 Slap2 Slap2 Slap2 (Inst2) 

NEC 1 2 1 1 1 1 1 1 1.1 1.0 

Cogent 2 1 1 2 3 2 1 2 1.8 2.0 

SAGEM M2 3 3 3 3 2 3 3 3 2.9 3.0 

SAGEM M1 4 4 4 4 4 4 4 4 4.0 4.0 

Neurotech. M1 6 6 5 5 5 5 6 5 5.4 5.0 

Motorola 5 5 6 6 6 6 5 6 5.6 6.0 

Identix 7 7 9 7 10 9 8 7 8.0 7.5 

UltraScan M2 9 9 7 10 7 7 12 8 8.6 8.5 

UltraScan M1 9 10 7 9 7 7 12 9 8.8 9.0 

NIST VTB 11 8 10 11 11 12 10 10 10.4 10.5 

Biolink 8 15 11 8 9 10 7 11 9.9 9.5 

Antheus 12 13 12 12 15 13 17 12 13.3 12.5 

Phoenix 14 14 16 14 14 16 9 13 13.8 14.0 

Golden Finger 15 11 13 16 16 14 15 14 14.3 14.5 

Raytheon 15 11 13 16 16 14 15 14 14.3 14.5 

123 ID M2 17 16 15 15 13 17 14 16 15.4 15.5 

Technoimagia 13 18 18 13 12 11 11 17 14.1 13.0 

Avalon 18 17 17 18 18 18 18 18 17.8 18.0 

Table D-8 

The following chart shows the TAR information graphically. The systems are sorted by

performance on the MST Standard partition (Complete MST). 

Figure D-13 

22 April 2004 Page 18 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

The comparisons of MST systems in the Analysis Report were made at FAR = 10-4. The 

reasons for the choice of this operating point are explained in Appendix E, Section 2. One 

of the reasons for the choice of this operating point was that the comparisons are generally 

valid for lower FARs. The following tables show what the effect would have been if MST 

systems had been compared at FAR = 10-6 instead of 10-4. Table D-9 corresponds directly to

Table D-7, except that the FAR is 10-6 instead of 10-4. 

Comparison of Performance on MST Partitions 

Values are True Accept Rates where the False Accept Rate = 10-6 

Probe set BCC DHS2 II Flats II Flats BEN II Slaps Ohio Standard 

(Inst1) 

Gallery set Flat2 Flat2 Flat2 Slap2 Slap2 Slap2 Slap2 Standard 

(Inst2) 

NEC 0.988 0.992 1.000 0.969 0.978 0.976 1.000 0.987 

Cogent 0.975 0.986 0.998 0.968 0.958 0.972 0.999 0.977 

SAGEM M2 0.951 0.980 0.983 0.925 0.967 0.962 0.987 0.961 

SAGEM M1 0.938 0.942 0.981 0.900 0.961 0.946 0.984 0.944 

Motorola 0.913 0.902 0.964 0.804 0.823 0.872 0.971 0.892 

Neurotech. M1 0.887 0.827 0.938 0.863 0.867 0.925 0.972 0.880 

Identix 0.861 0.775 0.845 0.758 0.778 0.815 0.920 0.831 

NIST VTB 0.737 0.730 0.834 0.680 0.737 0.739 0.908 0.759 

UltraScan M2 0.693 0.624 0.861 0.676 0.807 0.825 0.895 0.749 

UltraScan M1 0.693 0.612 0.861 0.674 0.807 0.825 0.895 0.746 

Biolink 0.746 0.442 0.772 0.682 0.757 0.800 0.931 0.733 

Antheus 0.735 0.357 0.664 0.610 0.561 0.725 0.816 0.672 

Phoenix 0.605 0.482 0.517 0.555 0.594 0.667 0.926 0.636 

Golden Finger 0.594 0.466 0.534 0.486 0.483 0.660 0.764 0.573 

Raytheon 0.594 0.466 0.534 0.486 0.483 0.660 0.764 0.573 

123 ID M2 0.448 0.318 0.440 0.435 0.524 0.566 0.808 0.516 

Avalon 0.302 0.130 0.233 0.139 0.259 0.289 0.329 0.256 

Technoimagia 0.457 0.011 0.037 0.394 0.672 0.719 0.870 0.063 

Table D-9 

In Table D-9, the results for Technoimagija appear contradictory, but are an artifact due to

the unusual variations in performance based on source. See the ROCs in Section 3.3. 

Table D-10 corresponds to Table 16 in the Analysis Report. The left half of the table 

summarizes the results where FAR = 10-6, and the right half where FAR = 10-4. Systems for 

which the rank would have changed are shown in yellow. These tables are sorted by the 

average rank where FAR = 10-4 (the same order as in Table 16 in the Analysis Report). 

22 April 2004 Page 19 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Distribution of System Rank over 7 MST Partitions where FMR = 10-5 

System Best 2nd Best Average Median 2nd Worst Worst 

Summary of Rank over 7 MST Partitions where FAR = 10-4 

Best 2nd Best Average Median 2nd Worst Worst 

NEC 

Cogent 

SAGEM M2 

SAGEM M1 

Neurotech. M1 

Motorola 

Identix 

1 1 1.0 1 1 

2 2 2.3 2 2 

2 3 2.9 3 3 

3 4 3.9 4 4 

5 5 5.4 5 6 

5 5 5.6 5.5 6 

7 7 8.1 8 9 

7 7 9.1 8.5 11 

7 7 8.9 8 11 

7 8 9.7 9 11 

8 9 9.7 9.5 11 

10 12 12.7 12 14 

1 

4 

3 

4 

6 

6 

9 

11 

11 

14 

11 

15 

1 1 1.1 1 1 

1 1 1.7 2 2 

2 3 2.9 3 3 

4 4 4.0 4 4 

5 5 5.4 5 6 

5 5 5.6 6 6 

7 7 8.1 8 9 

7 7 8.7 9 10 

7 7 8.7 9 10 

7 8 9.7 9 11 

8 10 10.4 11 11 

12 12 13.4 13 15 

2 

3 

3 

4 

6 

6 

10 

12 

12 

15 

12 

17 

UltraScan M1 

UltraScan M2 

Biolink 

NIST VTB 

Antheus 

Technoimagia 12 13 15.3 14.5 18 18 11 11 13.7 13 18 18 

Phoenix 8 11 12.4 13 14 15 9 14 13.9 14 16 16 

Golden Finger 12 13 14.3 14 16 16 11 13 14.3 15 16 16 

Raytheon 12 13 14.3 14 16 

15 15 16.0 16 17 

17 17 17.7 18 18 

16 

17 

18 

11 13 14.3 15 16 

13 14 15.3 15 17 

17 17 17.7 18 18 

16 

17 

18 

123 ID M2 

Avalon 

Table D-10 

3.2 Equal Error Rates 

In some evaluations, systems are compared using the Equal Error Rate, which is the point 

on an ROC where FAR = FRR (FRR = 1 – TAR). This table shows the equal error rates 

(with corresponding TAR and FAR values) for the MST systems, sorted by EER. 

Equal Error Rates for MST systems 

System EER TAR FAR 

NEC 0.002 0.998 2.1E-03 

Cogent 0.003 0.997 3.3E-03 

SAGEM M2 0.006 0.994 6.4E-03 

SAGEM M1 0.009 0.991 8.5E-03 

Neurotech. M1 0.023 0.977 2.3E-02 

Motorola 0.036 0.964 3.6E-02 

UltraScan M2 0.039 0.961 3.9E-02 

UltraScan M1 0.042 0.958 4.2E-02 

Identix 0.044 0.956 4.4E-02 

NIST VTB 0.048 0.952 4.8E-02 

Biolink 0.065 0.935 6.5E-02 

Golden Finger 0.076 0.924 7.6E-02 

Raytheon 0.076 0.924 7.6E-02 

Antheus 0.086 0.914 8.6E-02 

123 ID M2 0.116 0.884 1.2E-01 

Phoenix 0.138 0.862 1.4E-01 

Avalon 0.141 0.859 1.4E-01 

Technoimagia 0.151 0.849 1.5E-01 

Table D-11 

3.3 Image Source 

These six charts correspond to Figure 13 of the main report, showing accuracy on Slap and

Flat images from various sources. The charts are in alphabetic order by source. 

Note that the y-scale of the DHS2 chart is extended to TAR=0 to show the full range of 

performance. 

22 April 2004 Page 20 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-14 

Figure D-15 

22 April 2004 Page 21 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Note that the y-scale of the DHS2 chart is extended to show the full range of performance. 

Figure D-16 

Figure D-17 

22 April 2004 Page 22 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-18 

Figure D-19 

22 April 2004 Page 23 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

3.4 Image Quality 

The next four charts correspond to Figure 12 of the main report, showing accuracy on

images of varying quality. 

The range of the y-scale of these charts is extended to TAR=0. 

Note that the number of mated pairs gets progressively lower over the four charts, 

corresponding to the quality distribution in the data. 

Figure D-20 

22 April 2004 Page 24 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

 Figure D-21 

Figure D-22 

22 April 2004 Page 25 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-23 

22 April 2004 Page 26 of 35 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

4 SST 

4.1 Detail of System Comparisons 

The following table defines the partitions and shows the detail used to construct the SST 

system comparison charts in Section 4 of the FpVTE Analysis Report. These results 

correspond directly to Figure 10 in that document. 

The systems are sorted by performance on the SST Standard partitions. 

Note that these results are at FAR=10-3, not 10-4 as is true for most of the charts in this 

report. This is because of the small size of SST. 

Comparison of Performance on SST Partitions 

Values are True Accept Rates where the False Accept Rate = 10-3 

System Standard BCC DHS2 

Probe set size 

Gallery set size 

# Mates 

473 329 144 

368 368 368 

368 264 104 

NEC (MST-SST) 0.999 0.998 1.000 

Cogent (MST-SST) 0.997 0.996 1.000 

Cogent (SST) 0.997 0.996 1.000 

SAGEM M2 (MST-SST) 0.989 0.985 1.000 

SAGEM M1 (MST-SST) 0.988 0.983 1.000 

Motorola (MST-SST) 0.968 0.973 0.963 

Neurotech. M1 (MST-SST) 0.953 0.966 0.923 

Identix (MST-SST) 0.932 0.942 0.911 

UltraScan M2 (MST-SST) 0.901 0.915 0.874 

UltraScan M1 (MST-SST) 0.896 0.915 0.857 

NIST VTB (MST-SST) 0.892 0.901 0.874 

NIST VTB (SST) 0.892 0.901 0.874 

Bioscrypt (SST) 0.885 0.972 0.668 

Biolink (MST-SST) 0.838 0.923 0.627 

Antheus (MST-SST) 0.828 0.886 0.670 

Golden Finger (MST-SST) 0.752 0.760 0.741 

Raytheon (MST-SST) 0.752 0.760 0.741 

Phoenix (MST-SST) 0.728 0.744 0.694 

123 ID M2 (MST-SST) 0.690 0.742 0.584 

Technoimagia (MST-SST) 0.655 0.862 0.131 

Avalon (MST-SST) 0.638 0.734 0.395 

Table D-12 

22 April 2004 Page 27 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

4.2 Image Source 

Note that in all SST comparisons, the number of mated pairs is low due to the limited size of

the test. On the second chart, no data is shown for Cogent because that system achieved 

perfect separation of mate and non-mate scores (TAR=1, FAR=0). 

Figure D-24 

22 April 2004 Page 28 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

Figure D-25 

The following charts show the same results, but also include the MST systems’ results for 

the corresponding SST partitions. Note that NEC, Cogent, and SAGEM M2 achieve perfect 

separation of the DHS2 match and non-match distributions in this FAR range, and therefore

do not appear on the graph. 

22 April 2004 Page 29 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

 Figure D-26 

Figure D-27 

22 April 2004 Page 30 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

4.3 Image Quality 

These two charts roughly correspond to Figure 12 of the main report, except that these are 

for A and B quality data and SST systems. Note that the number of mated pairs is low on 

both charts (statistically small samples for estimating accuracy on these populations). Charts 

for Quality D and F data are not included here due to the small sizes of those sets. 

Figure D-28 

22 April 2004 Page 31 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

 Figure D-29 

Figure D-30 

22 April 2004 Page 32 of 35 
 

 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

5 Other Results 

5.1 Relationship of Accuracy to System Size 

The amount of hardware used by a system clearly had no relationship to accuracy in any 

FpVTE test. For example, Table D-13 shows the MST systems and hardware, sorted by 

their overall performance in MST. Three of the five most accurate systems were single-CPU 

systems, while the largest systems are at rank positions 6, 10, and 16 in the list. 

System ID Hardware TMR @ FMR=10-4, 

MST Standard 

1 NEC (MST) 

2 Cogent (MST) 

3 SAGEM M2 (MST) 

4 SAGEM M1 (MST) 

5 Neurotechnologija M1 (MST) 

6 Motorola (MST) 

7 Identix (MST) 

8 UltraScan M2 (MST) 

9 UltraScan M1 (MST) 

10 NIST VTB (MST) 

11 Biolink (MST) 

12 Antheus (MST) 

13 Phoenix (MST) 

14 Raytheon (MST) 

14 Golden Finger (MST) 

16 123 ID M2 (MST) 

17 Technoimagia (MST) 

18 Avalon (MST) 

3 dual Xeon 

1 IBM xSeries 335 Server, dual CPU 

1 Pentium4 

1 Pentium4 

1 Pentium4 

16 dual Xeon 

IBM X Series 8 Blade System 

1 Pentium4 

1 Pentium4 

20 dual Xeon 

1 Pentium4 

1 dual Pentium4 

1 Pentium4 

2 Pentium4 

4 dual Xeon 

16 HP xw6000 search nodes + HP a330n 

1 dual Xeon 

1 Pentium4 

99.42% 

99.14% 

98.27% 

97.65% 

93.88% 

93.44% 

89.51% 

86.32% 

86.01% 

85.03% 

83.54% 

77.27% 

72.52% 

70.71% 

70.71% 

67.31% 

57.50% 

52.93% 

Table D-13. Hardware and Accuracy for MST Systems 

5.2 Relationship of Accuracy to Processing Time 

The FpVTE evaluations were not intended to measure system throughput or speed: the only 

throughput requirement was that systems complete the test in the allotted time. 

For MST, there was a limited relationship between each system’s completion time and its 

accuracy, as shown in Table D-14. The three most accurate systems had three of the four 

longest completion times. Some systems allow tuning of algorithms so that increased 

processing time can result in higher accuracy, so it appears that the most accurate systems 

made effective use of the available time. 

22 April 2004 Page 33 of 35 
 

FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

System ID TMR @ FMR=10-4, MST Approx. Completion 

Standard time (days) 

1 NEC (MST) 

2 Cogent (MST) 

3 SAGEM M2 (MST) 

4 SAGEM M1 (MST) 

5 Neurotechnologija M1 (MST) 

6 Motorola (MST) 

7 Identix (MST) 

8 UltraScan M2 (MST) 

9 UltraScan M1 (MST) 

10 NIST VTB (MST) 

11 Biolink (MST) 

12 Antheus (MST) 

13 Phoenix (MST) 

14 Golden Finger (MST) 

14 Raytheon (MST) 

16 123 ID M2 (MST) 

17 Technoimagia (MST) 

18 Avalon (MST) 

99.42% 9 

99.14% 6 

98.27% 10 

97.65% 4 

93.88% 2 

93.44% 1 

89.51% 1 

86.32% 2 

86.01% 2 

85.03% 12 

83.54% 4 

77.27% 1 

72.52% 5 

70.71% 1 

70.71% 1 

67.31% 1 

57.50% 5 

52.93% 1 

Table D-14. Completion Time and Accuracy for MST Systems 

For LST, however, there was no significant relationship between each system’s completion 

time and its accuracy, as shown in Table D-15. 

System ID 

Approx. 

Distribution of TAR over 27 Operational LST Partitions where Completion time 

FAR = 10-4 (days) 

Min Average Max 

NEC (LST) 

SAGEM L1 (LST) 

Cogent (LST) 

SAGEM L2 (LST) 

Dermalog (LST) 

Motorola (LST) 

Identix (LST) 

NIST VTB (LST) 

Biolink (LST) 

Griaule (LST) 

Golden Finger (LST) 

Antheus (LST) 

Raytheon (LST) 

0.983 0.997 1.000 19 

0.950 0.990 1.000 7 

0.952 0.988 1.000 8 

0.936 0.985 1.000 6 

0.901 0.973 1.000 11 

0.869 0.966 0.999 18 

0.739 0.932 0.995 6 

0.747 0.896 0.982 21 

0.658 0.881 0.997 6 

0.707 0.881 0.987 17 

0.441 0.861 0.994 12 

0.615 0.836 0.981 14 

0.116 0.826 0.994 15 

Table D-15. Completion Time and Accuracy for LST Systems 

5.3 1:1 Matches 

In FpVTE 2003, systems may internally have performed 1:1 (verification) matches, or 1:N 

(identification) matches; they were not constrained to perform 1:1 matches. Whether a 

system performed 1:1 or 1:N matches is of some interest during comparisons of 

identification and verification results; see Section 5.4 of the FpVTE Analysis Report for 

context. 

Analysis showed that some systems in FpVTE did in fact perform true 1:1 matches. The 

trivial datasets were subsets of the evaluation datasets. These redundant comparisons 

provided a means to determine whether a system varied individual scores based on the test 

22 April 2004 Page 34 of 35 
FPVTE 2003 APPENDIX D — DETAILS OF RESULTS 

context. The following MST systems always gave the same score for a comparison, 

regardless of context, which is indicative of unnormalized 1:1 matches: 

• Golden Finger 

• Identix 

• NIST VTB 

• Raytheon 

• SAGEM M1 

• SAGEM M2 

• Technoimagia 

• UltraScan M1 

• UltraScan M2 

Note that performing 1:1 matches was not required of systems. Being on this list is of 

interest during analysis, and should not be viewed as either preferable or problematic in any 

way. 

22 April 2004 Page 35 of 35