Beyond Similarity Scores 

Austin Hicklin 

Brad Ulery 

IBPC – March 2010 
Differing perspectives 

• We have been analyzing automated biometric matching for years 

• We have a current focus on assessing human latent examiners 

• Perspective makes a difference 

– terminology 

– semantics 

– subtle or substantially different purposes and functions 

• In our attempts to model human expert examiners as matchers, we have 

found the difference in perspective to be enlightening, and we wanted to 

share some of our observations 

© Noblis 2010 2 
Latent Examiner Studies 

• Black box study 

• Inter-examiner markup variation 

• Extended friction ridge feature set specification (CDEFFS) 

© Noblis 2010 3 
Example 

© Noblis 2010 4 
(Exemplar) 

© Noblis 2010 5 
High similarity 

© Noblis 2010 6 

(Exemplar)
© Noblis 2010 

(Exemplar) 

Moderate similarity 

7 
© Noblis 2010 8 

(Exemplar) 

Low/No similarity: 

Mate with poor quality 
(Exemplar) 

Low/No similarity: 

Nonmate 

© Noblis 2010 9 
(Exemplar) Low/No similarity: 

Mate with no overlap 

© Noblis 2010 10 
High similarity Low/No similarity: 

Mate with no overlap 

(Exemplar) 

© Noblis 2010 11 

Moderate similarity Low/No similarity: 

Mate with poor quality 

Low/No similarity: 

Nonmate 
Automated matching: 

Similarity scores, 

Probability density functions, 

Receiver operator curves 

© Noblis 2010 12 
© Noblis 2010 Hypothetical data 13 
High similarity 

© Noblis 2010 Hypothetical data 14 
Moderate similarity High similarity 

© Noblis 2010 Hypothetical data 15 
Moderate similarity High similarity 

No similarity: 

Nonmate 

© Noblis 2010 Hypothetical data 16 
No similarity: 

Mate with poor quality 

Moderate similarity High similarity 

No similarity: 

Nonmate 

© Noblis 2010 Hypothetical data 17 
No similarity: 

Mate with poor quality 

No similarity: Moderate similarity High similarity

Mate with no overlap 

No similarity: 

Nonmate 

© Noblis 2010 Hypothetical data 18 
© Noblis 2010 Hypothetical data 19 
© Noblis 2010 

No similarity: 

Nonmate 

Hypothetical data 20 
© Noblis 2010 

No similarity: 

Mate with poor quality 

No similarity: 

Nonmate 

Hypothetical data 21 
© Noblis 2010 

No similarity: 

Mate with poor quality 

No similarity: 

Mate with no overlap 

No similarity: 

Nonmate 

Hypothetical data 22 
© Noblis 2010 

Discontinuity: 

failures to match at any 

threshold 

Hypothetical data 23 
© Noblis 2010 

Moderate similarity: 

Allows empirical measurement of error rate tradeoffs 

Hypothetical data 24 
© Noblis 2010 

High similarity: 

No empirical measurement of specificity (false matches) 

Hypothetical data 25 
Human examiner determinations 

© Noblis 2010 26 
Automated matchers & human examiners 

• Automated matchers 

– Single function: similarity 

– Continuous distribution of values 

– 2 determinations: 

• Match 

• Failure to match (Nonmatch/ 

Inconclusive) 

• Human examiners 

– 2 functions: 

• Similarity 

• Difference 

– 3 determinations: 

• Same source / Individualization 

– (highly similar) 

• Different source / Exclusion 

– (highly different) 

• Inconclusive 

© Noblis 2010 27 
Automated matchers & human examiners 

• Decision thresholds 

– Automated matchers 

• Continuous distribution of values 

allows requirements-based 

threshold adjustment 

– Human examiners 

• Responses are Boolean, not 

continuous, and are very 

deliberately cautious 

© Noblis 2010 28 
Feature-level similarity, difference, and 

inconclusive assessments 

© Noblis 2010 29 
© Noblis 2010 30 
© Noblis 2010 31 
© Noblis 2010 32 
•

© Noblis 2010 33 
•

© Noblis 2010 34 
•

•

© Noblis 2010 35 
•

•

• Examiners assess each feature or grouping of features in a comparison 

in terms of similarity, difference, and inconclusive 

© Noblis 2010 36 
•

•

• Feature-level difference measures require 

– Accurate feature detection 

and/or 

– Feature-level quality/confidence algorithms 

© Noblis 2010 37 
Hypothetical matchers 

© Noblis 2010 38 
Hypothetical matcher with both similarity and 

difference functions 

© Noblis 2010 Hypothetical data 39
Hypothetical matcher with both similarity and 

difference functions 

© Noblis 2010 40 
Summary 

• Difference measures are not the same as 1-similarity 

• Automated matchers 

– Return a single continuous monotonic similarity value 

– An adjustable threshold can be applied 

– Decisions are match vs nonmatch (including exclusion and inconclusive) 

• Human examiners 

– Return a tri-state decision 

• Same source (individualization) 

• Different source (exclusion) 

• Inconclusive 

• These decisions are made at three levels 

– Overall 

– Feature groups 

– Individual features 

• Automated matchers could in theory return continuous difference scores in addition to

similarity scores 

– Potential for improved accuracy 

– Would allow additional functionality (exclusion) 

– Requires reliable feature extraction and/or reliable feature-level quality metrics 

• In the absence of accurate difference measures 

– Similarity + Difference + Inconclusive = 1 

– An inconclusive measure could be defined as a measurement of applicability/quality 

© Noblis 2010 41 
Austin Hicklin hicklin @ noblis.org 

Brad Ulery ulery @ noblis.org 

© Noblis 2010 42