FRGC and ICE Wor

Dr. P. Jonathon Phillips - 

March 22-23, 2006

NRECA Conference Facility

Arlington, Virginia
Workshop Agenda Overv

DAY 1:•

FRGC Overview and Experiment Results

• Individual Organization FRGC Results •

Guest Speaker

–

Alice O’Toole, University o

f Texas a

t Dalla

•

Human verses Machine Face Recognition 

DAY 2:• ICE Overview and v1.0 Experiment Resul• Individual Organization ICE v1.0 Results •

Guest Speaker

–

Philip Wasserman, NIST Consultant

• Iris Imaging Platform Design

•

Next Steps – ICE Phase II
FRGC, FRVT 2006 & ICE Sponsor

Executing Agency

Sponsoring Agencies

Director of National Intelligence Intelligence Technology Innovation Center

–

S

cience & Technology 

–

Transportation Securit
FRGC and ICE Team

• Program Manager for FRGC and ICE

• P. Jonathon Phillips — NIST 

• Evaluation Team

• Todd Scruggs — SAIC• Matt Sharpe — SAIC• William Worek — SIAC• Kevin Bowyer — Univers

ity of Notre D

• Patrick Flynn — University of Notre Da• Ross Beveridge — Colorado State Univ• Alice O’Toole — University of Texas at 

• FRGC and ICE Liaison

• Cathy Schott — Schafer Corp 
Status Update

•

FRVT 2006

• ICE 2006
FRVT 2006 Status Update

• The Face Recognition Vendo

(FRVT) 2006

– Began on 30 January 2006– Currently underway

• Testing executables at this time

– 22 Participants

• 10 countries• 30% of Participants are from Aca
ICE Phase II = ICE 2006

• ICE 2006

– Independent U.S. Governmen– Planned start date is 15 June – ICE 2006 protocol based on th

2006 protocol
ICE Participation

ICE Participation

66

42

9

0

20406080

1

Participation

Number
Face Recognition Grand 

Overview
Outline

• Overview of Face Recognition G

Challenge (FRGC)

• Overview and Results of FRGC v• Overview of the Face Recognitio

Test (FRVT) 2006

• Next Steps 
FRGC and FRVT 2006

• What is the difference betwe

and FRVT 2006?

– FRGC (May 2004 – March 20

– Still and 3D face recognition algo

development project

– FRVT 2006 (30 January 2006

• Independent government evalua

recognition systems

– Measure progress since FRVT 2
FRGC Background

• Renewed interest in developing n

for automatic face recognition 

– Fueled by advances in 

• Computer vision techniques• Computer design• Sensor design• Interest in fielding face recognition sys

• New techniques have potential to 

reduce error rates 
FRGC Participation

FRGC Participation

116

52

22

0

50

100150

Participation

Number
Background

Technology 

Development

Baseline 

July 2002 

May 2004 

–

Mar 2006
FRGC Goal and Objectiv

• The primary goal of the FRGC i

Promote and advance face re

technology to support U.S. Go

face recognition effor

• The primary objective of the FR

Develop still and 3D algorithms 

performance an order of magn

FRVT 2002
Select Point to Measure

• Verification rate at :

– False accept rate = 0.1%

• Current:

– 20% error rate (80% verification 

• Goal: 

– 2% error rate (98% verification 
Measuring Accuracy w/Error 

Rate of 2%

• Non-match scores: 

– Sufficient

• Match scores: 

– Need to design collection for sufficie

• Allows for error ellipses• Minimal demographic analysis

1,000 match scores = ~ 20 err

10,000 match scores = ~ 200 er

50,000 match scores = ~ 1,000 er

50,000 match scores = ~ 1,000 er
FRGC Modes Examined

Multiple Stills

Single Still

Outdoor/

Uncontrolled

3D S

vie
3D Images

3D Sensor

Shape

Texture

3D Image

3D F

Recog

Algor
Data Collection & Distribu

• Total data collected at Notre D

- 4950 subject sessions of 400+ su– 125,000 2D and 15,000 3D images– 125,000 irises (40,000 verified)

Minolta Vivid 900 / 910

Nikon
Grand Challenge Architec

Meta data• eye coordinates• pose• gender

Preprocessing/ReconstructionCompression

Image QualityMeasures

ComparisonAlgorithms/Systems

Modes

Accuracy of:

3D Sensors

3D from stills
FRGC Experiments

Exp 1: Controlled indoor still versus indoor stillExp 2: Multiple still versus multiple stillExp 3: 3d versus 3D

3t - Texture only3s - Shape only

Exp 4: Uncontrolled still versus indoor still
Training and Validation Part

Training 

- Academic Year 2002-03

- Acade

- 12,776 Large Still Training - 16,028 

Set - 8,014 

- 943 3D Subject Sessions - 4,007 3
Three Data Sets

FRGC Challenge Problem

Training

Validation

Target

Query
Demographics

(ver2.0 Validation Partition)

57%

43%

MaleFemale

65%

18%

17%

122

68%

10%
Size of Faces

(ver2 On Validation)

Pixels between center of 

Controlled

Mean

261

M

2

Uncontrolled 144 13D 160 1
Target / Query Sets

(ver2.0 Validation Partition)

466 Subjects; 4,007 Subject sessions; 32,

Subject Sessions 2003-04

56

26

32

36

31 28 27

22 22 25

16 15 16 19 13 1

0

102030405060

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 

Number of Subject Sessions

Validation Partition
Three ROCs

ROC I - Within Semesters

ROC II – Within Year

ROC III – Between Semesters

∩ 
FRGCv2 Exp 1

0.0 0.2 0.4 0.6 0.8 1.0

Algorithm

Verification rate @ FAR = 0.001

Exp 1 Composite Performance

ROC IROC III
FRGCv2 Exp. 1

0.95 0.96 0.97 0.98 0.99 1.00

Algorithm

Verification rate @ FAR = 0.001

Exp 1 Composite Performance

ROC IROC III
FRGCv2 Exp. 2

0.995 0.996 0.997 0.998 0.999 1.000

Algorithm

Verification rate @ FAR = 0.001

Exp 2 Composite Performance ROC 
Exp. 3, 3s, and 3t

0.80 0.85 0.90 0.95 1.00

Verification rate @ FAR = 0.001

Exp 3t (Texture

0.80 0.85 0.90 0.95 1.00

Verification rate @ FAR = 0.001

Exp 3s (Shap

0.80 0.85 0.90 0.95 1.00

Algorithm

Verification rate @ FAR = 0.001

Exp 3 Composite Performance ROC III
Exp 4 and 6

0.0 0.2 0.4 0.6 0.8 1.0

Algorithm

Verification rate @ FAR = 0.001

Exp 4 and 6 Composite Performance R
Scientific Questions

High Resolution 

Computer Vision vs Pattern R

vs
Scientific Questions

High Resolution 

Effects of Training Set 
Training Set Size - Exp 1

0

0.10.20.30.40.50.60.70.8

0 500 1000 1500 2000 

# of Eigens Used

VR @ FAR=0.1%
Training Set Size -

Exp 1

0

0.10.

2

0.

3

0.

4

0.

5

0.

6

0.

7

0.

8

0

500 1000 1500 2000 25

# of Eigens U

se

d

VR @ FAR=0.1%

0.6 0.66 0.7 

0

0.20.40.60.8

1

512 1,024 2,048 

Number of Training I
Training set size -

Exp 4

Courtesy C. Liu 2005
Training set size - Exp 4

1

0.8 0.7 0.64

0.60.40.2

0

1,623 3,194 

Number of Trainin

Courtesy C. Liu 2005
FRVT 2006

• Latest in a series of large scale i

evaluations for face recognition s

– Previous evaluations in the series w

FERET, FRVT2000, and FRVT 200

• Primary goal is to 

– Measure progress of prototype syst

and commercial face recognition syFRVT 2002 

– Conduct comparison across modal– Compare performance with FRGC 
Independent Evaluations 

(Gold Standard)

FAR = 0.1%

Performance Goals and Prog

Starting Point 80%

Measured in

FRVT 2002
Independent Evaluations 

(Gold Standard)

Starting Point 80%

Measured in

FRVT 2002

FAR = 0.1%

Performance Goals and Prog

Goal 98%

To be measured 

by FRVT 2006
Independent Evaluations 

(Gold Standard)

Starting Point 80%

Measured in

FRVT 2002

FAR = 0.1%

Performance Goals and Prog

Goal 98%

To be measured 

by FRVT 2006

Face Recognition 

(Qualified 

99.99% Multi-Still 

(Mar 06)

99% High Resolutio

(Mar 06)

98% Three-Dimensi

(Mar 06)

* First set of results after 4 mo
Independent Evaluations 

(Gold Standard)

Starting Point 80%

Measured in

FRVT 2002

FAR = 0.1%

Performance Goals and Prog

Goal 98%

To be measured 

by FRVT 2006

Face Recognition 

(Qualified 

99.99% Multi-Still 

(Mar 06)

99% High Resolutio

(Mar 06)

98% Three-Dimensi

(Mar 06)

* First set of results after 4 mo
Summary

• Face Recognition Grand Cha

– Order of magnitude increase i

performance

– Systematically investigate still – Formulate series of challenge – Face Recognition Grand Chal

Completion March 2006
Next Steps

• FRGC Final Report

– One more call for similarity file

• Due 30 April 2006

– Provide a list of papers you’ve 

this subject

• FRVT 2006 Final Report

– Completion estimated in Fall 2
Publication of FRGC Resu

• Check with sponsors to d

they want to be cited

• Please include FRGC refe

– P. J. Phillips, P. J. Flynn, T. S

Bowyer, J. Chang, K. Hoffman

J. Min, W. Worek, Overview oRecognition Grand Challenge, Proceedings International Comand Pattern Recognition (CVP