A Proposed Method for Scaling of 

Identification False Match Rates

False Positive Identification Rates (FPIR)

Using Extreme Value Theory

Michael Schuckers

St. Lawrence University

schuckers@stlawu.edu

International Biometric Performance Conference

NIST

March 8, 2012

© Michael Schuckers 2012

(c) Michael Schuckers 2012 1
Scaling of Performance

• Known problem

• Identification FMR (FPIR)

• How to predict performance from DB of 

103x103 to DB of 107x107

• How does threshold for FPIR=0.01 change?

(c) Michael Schuckers 2012 2
Previous Bioauthentication Work

• Large-Scale identification System Design

Hervé ]arosz, ]ean-Christophe Fondeur and 

Xavier Dupré

Empirical (Regression line)

Ident as N Verifications

Extreme Value Theory (G. Pareto Dist)

Modeling Distance

(c) Michael Schuckers 2012 3
Notation

• Yij be match score from a comparison of 

sample from individual i compared to 

individual j.

• Xi:m = max j {Yij}, 1 ≤ i≤ n

• Goal is prediction of distribution of Xi:M

(c) Michael Schuckers 2012 4
m

M

n

N

(c) Michael Schuckers 2012 5
Extreme Value Theory

• If there exists sequences of constants {am>0} 

and {bm} such that 

P

Xi:m − bm

am

≤ z → G(z)

As m →∞ then G belongs to one of three 

families of distributions:

Gumbel, Fréchet, Weibull.

(c) Michael Schuckers 2012 6
Limiting Distributions

• Gumbel G(z) = exp(-exp (-(x-b)/a)) 

• Fréchet G(z) = exp(-((x-b)/a)-α)

if z>b, G(z) = 0 o/w

• Weibull G(z) = exp(-(-((x-b)/a)α))

if z<b, G(z) = 1 o/w

(c) Michael Schuckers 2012 7
Comment

• Stuart Coles:

“ The remarkable feature of this result is that 

the three types of extreme value distributions 

are the only possible limits for the distribution 

of [maximums] regardless of the distribution F 

for the population.”

(c) Michael Schuckers 2012 8
Combining Limiting Distributions

Generalized Extreme Value (GEV) Distribution

Cdf given by 

G(z) = exp{-[1+ξ((z-μ)/σ)]-1/ξ]}

if 1+ξ (z-μ)/σ>0

(c) Michael Schuckers 2012 9
Return Level (Target)

Return Level zp is the value that will be exceeded 

with probability p. i.e. P(X1:M>zp)=p

zp=G(1-p)

zp

= μ-σ/ξ(1-(-ln(1-p))-ξ) if ξ≠ 0

μ-σ(1-(-ln(1-p))) if ξ= 0

(c) Michael Schuckers 2012 10
Goals

Take X1:m, ..., Xn:m

Estimate distribution of Xi:M’s and find zp from 

this.

Note: z0.01 is value of scores that gives iFMR of 

0.01.

(c) Michael Schuckers 2012 11
m

M

n

N

12

Extreme Value Theory

(c) Michael Schuckers 2012 
m

M

n

N

13

X1:m

.

.

.

Xn:m

X1:M

.

.

.

Xn:M

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

XN:M

Bioauthentication Problem 

(c) Michael Schuckers 2012 
m

M

n

N

14

Proposed Methodology

(c) Michael Schuckers 2012 
Proposed Methodology

1. Fit a GEV to the n observed Xi:m’s to get MLE’s of μ, σ, ξ. 

2. For each i, 1≤i≤n, generate k = ceil(M/m)-1 values from a GEV with 

parameters μ� + e1sμ�, σ� + e2sσ�, ξ̂ + e3sξ� . 

3. Find X*i=max{Xi:m,X*

i1,X*i2,...,X*ik} 

4. Fit a GEV distribution to the n X*i’s and get MLE’s. Call these 

estimates μ� , σ� , ξ̂ . ∗ ∗ ∗ Use these estimates to get zp* 

5. Repeat steps 2 to 4, saving zp* each time.

6. Use distribution of zp* to make inference for FPIR=p.

(c) Michael Schuckers 2012 15
m

M

n

N

16

zp*

(c) Michael Schuckers 2012 
m

M

n

N

17

zp*

(c) Michael Schuckers 2012 
m

M

n

N

18

zp*

(c) Michael Schuckers 2012 
Testing via Simulation/Synthetic

1. Create N x N database of scores

– Calculate known zp

2. Randomly sample n from {1, ...., N}

3. Run model on selected n x n database

4. Create 95% CI for zp

5. Repeat Steps 2) to 4) 100 times

6. Determine % of times zp inside CI’s.

(c) Michael Schuckers 2012 19
Gaussian (mean =35)

Database Sample Stand. P (FPIR) Coverage

Size Size Deviation

5000 500 10 0.01 0.95

5000 500 10 0.005 0.94

5000 500 5 0.01 0.92

5000 100 10 0.01 0.94

5000 100 5 0.01 0.93

5000 100 10 0.005 0.92*

10000 100 10 0.005 0.90

(c) Michael Schuckers 2012 20
Gamma(mean=500,stddev=225)

Database Sample p(FPIR) Coverage

Size Size

5000 500 0.01 0.96

5000 500 0.001 0.94

5000 100 0.01 0.96

5000 100 0.005 0.96

(c) Michael Schuckers 2012 21
Summary

New method, theoretically grounded

Up to 2(?) orders of magnitude, good 

performance

Need to test on ‘real’ data

Need to look at more orders of magnitude.

(c) Michael Schuckers 2012 22
Limitations

• Issues with correlation (?)

• Needs more testing

– Synthetic: different distributions

– ‘Real’ data

• More orders of magnitude to test

• Changing data collection process

• Address binning, etc.

(c) Michael Schuckers 2012 23
Thank You!

schuckers@stlawu.edu

(c) Michael Schuckers 2012 24