Face Template Recon

from Match Scores

Pranab Mohanty 

Sudeep Sarkar 

Rangachar Kastur

University of South Florida
Outline

IntroductionProposed Scheme Experiment and ResultConclusion
Objective

“To reconstruct the target temp

observed match scores”

Target

Template

Query

Template

Face Recognition

System

?

• The Recognition system is considered as a black 

perform any reverse engineering methods.

• All the target images are completely unknown
Reconstruction of temp

Major Steps

1. Model the recognition syst

an affine transformation of 

2. Embed the target template 

space 

3. Reconstruction of target te
Comparison to Hill-Climbing 

Proposed Method

Grap

Hill Climbing Approach
Modeling Recognition A

i x

j

x

Affine

Transformation

A

Face Recognition

Break-in Set

1. Create an Independent set of images “Break-in 

2. Compute the distance matrix D using the recog

3. Using the same break-in set design an affine tr

0

2

2

ij AxAxd ji ≅−−

Break-in Set
Modeling Recognition A

Affine

Transformation

A = Anr

A

r

A

• a non-rigid • depends orecognition a• approximaalgorithm thrstretching of • derived usi

Ar

- a rigid transformation- independent of the recognition algorithm- derived from the orthonormalsubspace analysis e.g. PCA of images in break-in set
Embedding & Reconst

?

Inverse

Transformation

Unknown

target

x

z

Original image space

 Observe the distance from selected templates 

to unknown target

Embedding

Reconstruction

 Use Invert transformation to reconstruct the u

in original affine space

 Calculate the co-ordinate of the unknown targ

space 

z

y

z

x

znr

T

z

r yAAx Ψ

=
Outline of Proposed Sc

Face 

Recognition 

System

Match 

Scores

Tra

Re

Break-in 

Set

Face 

Recognition 

System

Unknown

Target

Match 

Scores Embeddin

Break-in 

Set

Embedding & Reconstruction

Modeling
Dataset & Recognition A

 Database

 Break-in set : 600 FERET images ( 15 Target Set

 100 FRGC Subjects  100 FERET Subjects (all different from br

 Algorithm

 FRGC baseline algorithm (template based Bayesian intrapersonal/extrapersonal clas Commercial Face Recognition System (fe

All algorithms are set to operate at 1% False Acceptaand 99% True Acceptance Rate* with 100 enrollmen

* Except Bayesian Method on FRGC target set
Results

Modeling Error

2

22

ij

ijij

d

d

Error

−δ

=

dij

δ ij

Original distance computed by the recognition 

Euclidean distance in model space

Baseline Commercial 

Error distribution has zero Mean in a
Results (Contd.)

Reconstructed FERET Target 

Original

Reconstructed 
Results (Contd.)

Reconstructed FRGC Target T

Original

Reconstruction 
Results (Contd.)

 We reconstruct the target sets using fiv

break-in set with 75, 150, 300, 450 and 

Probability of Break-in

“Probability of breaking into a face recognition selecting an enrolled account”
Results (Contd.)

No. of Images i

Original 75 150 300 

Baseline

Bayesian

Commercial
Results (Contd.)

Probability of Break-in on FERET Set

Bayesian & Baseline 100% Comme

Break-In Performance

1 1

0.3

0.6 0.68

0.92

0.35

0.54

0.74

0.93 

0.55

0.94

0

0.20.40.60.8

1

1.2

75 150 300 450 

Number of Attempts

Probability of Break-In

BayesianCommercialBaseline
Results (Contd.)

Probability of Break-in on FRGC Set

Break-In Performance

0.32 0.32

0.64

0.72

0.55

0.98 11

0

0.10.20.30.40.50.60.70.80.9

1

75 150 300 450 

Number of Attempts

Probability of Break-In

CommercialBaseline

p

Baseline 100% Commercial 82
Results (Contd.)

Quantization of Match Scores

Break-In Perfromance on Commercial Algorithm with Quantized Scores with 

450 Attempts

0.24

0.9 0.9 0.88

0

0.20.40.60.8

1

1234

Number of Significant Digits

Probability of Break-In

CommercialAlgorithm

Commercial Algorithm
Conclusion

 The proposed template reconstru

uses an affine transformation to munderlying recognition algorithm 

 It is non-iterative Uses distinct face images to be m

target template

 Requires less number of matchin

to Hill-Climbing Approach

 Robust to Score Quantization
Questions 

Thank Yo